text:
🛑 zdrofit.pl is down

In 595a92c, zdrofit.pl (https://www.zdrofit.pl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: zdrofit.pl is back up in d23b3ff.

source: gharchive/issue

repo: techsupp/uptime

license: MIT

license_type: permissive

license_source: github-api


text:
Repeated failures when fetching the centos:7 image

The  /tests/finish/ansible test recently fails repeatedly when fetching the centos:7 image:
    provision
        how: container
        image: centos:7
    fail: Command 'podman pull -q centos:7' returned 125.
    finish
        summary: 4 tasks completed

Command 'podman pull -q centos:7' returned 125.

stderr (1 lines)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: parsing image configuration: Get "https://cdn03.quay.io/sha256/86/8652b9f0cb4c0599575e5a003f5906876e10c1ceb2ab9fe1786712dac14a50cf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI5LUAQGPZRPNKSJA%2F20230509%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230509T091300Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=c2c2f0aa8a7432dac77824eeb08c5e24a5f7f082e6b89d4c4a09d14f53f35ec7&cf_sign=TwyGtFMDOKOvXJlUIptbQqu96NLFINyFXstV7d2mUJw3Z7s89br7VaojuL%2BirD5Drddxi7P%2BrjwFyWPUAvCa4s4xCiDIEBNJ8mnpIgk3Lkm8k%2FTKXZTa%2F%2FOksO27Tw79Mu8AZ%2Ffw8FGoEavukcSGCNs7XqoWDTL%2FZ%2BCEXsX1Ul2T58H5iPW6GVEjwyVr%2BcP92y8a%2BScf5xcxbNkAwGjayymxCohis%2BaVuV13l5EFpY2aJpBW0kxLAc2Rxd5%2BduG0hcxIyfwsjMXd4Jacs8rxpHzthrF7t%2BDX3FNlKw5N83wdRxebuNrHfTiSUtiEerE8wNfqKkvC6%2FWRltNisXo5Xw%3D%3D&cf_expiry=1683624180&region=us-east-1": dial tcp: lookup cdn03.quay.io: no such host
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Here's a recent failure example. Perhaps, we should implement a retry attempt to fetch the images to cover random network issues like this?

And one more again.

@thrix, here's one more from today on fedora-38.

And one more.

And again, this time on rawhide.

An again on fedora-37.
Seems it would be good to handle this.

The testing farm workaround we use is this:
https://gitlab.com/testing-farm/infrastructure/-/blob/testing-farm/ansible/tasks/dns.yml?ref_type=heads

Ok, ran into another instance today.

Ok, trying to enable the Testing Farm workaround in #2308. Let's see how it works.

Hmmm, so another dial tcp: lookup cdn.registry.fedoraproject.org: no such host failure appeared again:
Error: copying system image from manifest list: parsing image configuration: Get "https://cdn.registry.fedoraproject.org/v2/fedora/blobs/sha256:72c9e456423548988a55fa920bb35c194d568ca1959ffcc7316c02e2f60ea0ff": dial tcp: lookup cdn.registry.fedoraproject.org: no such host

Let's see if there will be more...

For reference, similar error appeared again even with the retry support:

https://artifacts.dev.testing-farm.io/dd982d24-316d-46c7-8282-fc487abc08fc/

The error message says:
stderr (4 lines)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
time="2023-11-01T15:36:22Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://localhost/become-container-test:latest: pinging container registry localhost: Get \"https://localhost/v2/\": dial tcp [::1]:443: connect: connection refused"
time="2023-11-01T15:36:23Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://localhost/become-container-test:latest: pinging container registry localhost: Get \"https://localhost/v2/\": dial tcp [::1]:443: connect: connection refused"
time="2023-11-01T15:36:24Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://localhost/become-container-test:latest: pinging container registry localhost: Get \"https://localhost/v2/\": dial tcp [::1]:443: connect: connection refused"
Error: initializing source docker://localhost/become-container-test:latest: pinging container registry localhost: Get "https://localhost/v2/": dial tcp [::1]:443: connect: connection refused
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The good news is, that retries did work. At least according to the log. The bad news is, well, 5 attempts with 5 seconds between attempts was not enough to make it across the rough seas... It would be nice to set the default big enough to help, e.g. via envvars for plugin keys, but those apply only when the plugin is mentioned on command line, which was not the case here :/

Today failed again, but another good news is that it did not happen under the podman plugin but during the container image preparation:
:: [ 15:36:20 ] :: [  BEGIN   ] :: Running 'podman build -t become-container-test:latest .'
STEP 1/3: FROM quay.io/fedora/fedora:latest
Trying to pull quay.io/fedora/fedora:latest...
Error: creating build container: copying system image from manifest list: parsing image configuration: Get "https://cdn03.quay.io/sha256/a1/a1cd3cbf8adaa422629f2fcdc629fd9297138910a467b11c66e5ddb2c2753dff?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI5LUAQGPZRPNKSJA%2F20231101%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231101T153621Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=c599dc85c3c96419caa2168dcee07fd4d0e545d9630f92bf542416cd743bf4db&cf_sign=QNIVhbRLD14trMM7appn7t4A81uihY2ct7SqqFYI89404Hfa%2BJTNmCpGQvQOSidfm%2FLsGySKGEOqxPiVeovvthcKDeOsRFo6b9nineXOFXpzG2b%2FITW7f3K4hxvEGsLLaQXDT89z%2FLQ1gV7ZnPj6mGPSvHY8BkCTuC%2BeLYhI4Xg8UclA07msz7H1X1L7BuDLRaPC%2B1aVCbT05n4nY4IDzfmSr0rBWX0UgcDi3AL87NmFZnIIrZZuuAEf5Q8lteeAEKfXuuuvhKZ0nVwDQ1FmBB6ZxBGpsVXmkip7eVzWnDAdDOauvfmlYn2isYQ3X%2BoKOROjb%2FnBQTZWzj0pj5ZDHg%3D%3D&cf_expiry=1698853581&region=us-east-1": dial tcp: lookup cdn03.quay.io: no such host
:: [ 15:36:21 ] :: [   FAIL   ] :: Command 'podman build -t become-container-test:latest .' (Expected 0, got 125)

So... perhaps, the retry actually works fine! :) Would be nice to check/grep recent logs to verify...

good news is that it did not happen under the podman plugin but during the container image preparation:

Let's try to repeat the container image creation as well: #2467

On the hacking session we've agreed to disable systemd-resolved for the pull request testing and file a new systemd issue to properly investigate the problem.

@psss - did you ever open an issue against systemd?

authors: dustymabe happz psss thrix

repo: teemtee/tmt


text:
Add documentation on tmt & regular expressions

Fixes https://github.com/teemtee/tmt/issues/2602.
The preview of tmt & regular expression would be at https://tmt--2646.org.readthedocs.build/en/2646/overview.html#regular-expressions
Pull Request Checklist

[ ] write the documentation
[ ] update the specification

@psss @lukaszachy @thrix starting with tmt * ls, what other options would be using regular expressions? I don't mind adding pointers to regex info as we discover places where it's needed, especially while polishing plugin docstrings and options for the "render from docstrings" mini project, but if there are obvious options, maybe not based in plugins, that deserve a link, please, point them out.

/packit test -i full

/packit test -i provision

source: gharchive/pull-request

authors: happz psss


text:
Mac doesn't sleep via Alfred with Karabiner running

Strange issue. If I try to sleep my Mac by typing Sleep in Alfred the screen blacks out and then immediately comes back on. If a text field had focus, then that field is getting plummeted with 's until I hit Esc. Does not happen if quit Karabiner-Elements, also no problem with hitting Sleep in the Apple menu.
The log looks like this, but the log looks identical in the case of hitting Sleep in the Apple menu.
[2016-10-28 12:36:20.242] [info] [grabber] iopm_client::callback kIOMessageSystemWillSleep
[2016-10-28 12:36:20.242] [info] [grabber] device_grabber::suspend
[2016-10-28 12:36:20.243] [info] [grabber] Connected devices are ungrabbed
[2016-10-28 12:36:20.725] [info] [grabber] iopm_client::callback kIOMessageSystemWillPowerOn
[2016-10-28 12:36:20.725] [info] [grabber] device_grabber::resume
[2016-10-28 12:36:20.725] [info] [grabber] Unifying Device is ignored.
[2016-10-28 12:36:20.725] [info] [grabber] USB Receiver is ignored.
[2016-10-28 12:36:20.730] [info] [grabber] Unifying Device is ignored.
[2016-10-28 12:36:20.733] [info] [grabber] iopm_client::callback kIOMessageSystemHasPoweredOn
[2016-10-28 12:36:20.833] [info] [grabber]  is grabbed
[2016-10-28 12:36:20.834] [info] [grabber] Apple Internal Keyboard / Trackpad is grabbed

Repro every time. It looks to me like the mac goes to sleep before Karabiner gets a chance to release the Enter key, as the login form is shaking for being constantly submitted.

I have similar problem. System does not go to sleep via Hammerspoon

Me too and its strange that this happens only with usb keyboard!

It isn't just a USB keyboard. I can replicate with two Bluetooth keyboards connected as well.

Lets hope someone finds a solution to this soon! @tekezo man take a look at this!
An easy alternative to go to sleep without alfred is a hot corner !

I can replicate with the MBP keyboard, so it's with any keyboard.

Looks like its fixed it latest version.

authors: andresbase brandenbyers cormacrelf manojfedx scauer talha131

repo: tekezo/Karabiner-Elements

license: unlicense

license_source: bigquery


text:
Create container image on push to master and on release

See: https://docs.github.com/en/actions/publishing-packages/publishing-docker-images
Publish a container image (with multiple tags that correspond to the release event, whether it's nightly or stable build) to Github Container Registry via Github Actions.

hey @aldy505 I'd like to help with this one!
Are you going to release tags for those events? Can you specify how would you like to trigger them?

repo: teknologi-umum/captcha-lite


text:
Unexpected, irregular reprocessing of resources by tekton-pipelines-controller

Hi,
at our tekton deployment, I noticed that sometimes the work queue (tekton_workqueue_depth) of the tekton pipelines controller spikes. Based on the "size" of the spike, this looks like a full reprocessing of e.g. the PipelineRuns in the cluster. In the log, I can see the following messages when this happens. This happens irregularly.
2021-01-12 14:11:58 Trace[1752598205]: [18.704479331s] [18.704479331s] END
2021-01-12 14:11:58 Trace[1752598205]: ---"Objects listed" 18681ms (13:11:00.652)
2021-01-12 14:11:58 I0112 13:11:58.675267       1 trace.go:201] Trace[1752598205]: "Reflector ListAndWatch" name:runtime/asm_amd64.s:1374 (12-Jan-2021 13:11:00.970) (total time: 18704ms):

I suspect that the connection between the controller and the API server fails however this is just a guess. Is there any way to find out what causes the reprocessing?
Thanks,
Fabian
Additional Info

Kubernetes version: v1.18.12
Tekton Pipeline version: v0.19.0

Sorry for not responding earler.
This should be considered expected behavior. Reconcilers are configured to periodically scan all resources in case they missed a previous update. Normally this has no effect, except a (hopefully) brief spike in the workqueue depth, and not a noticeable additional latency for in-progress PipelineRuns.

Thank you @ImJasonH for picking this up!
Do you know if the periodic reprocessing is deterministic, e.g. every x hours (after the container started?)? Is this basically https://github.com/knative/pkg/blob/main/controller/controller.go#L52 or did I miss where this is overwritten by tekton?
Some background info: we currently have ~15k taskruns and 8k pipelineruns in the cluster. With these numbers, the reprocessing takes ~10 min. This is not ideal but ok. To lower the impact, we also apply sharding into 3 buckets. (really looking forward to the results project in this context ;) )

I believe the resync period is from the time the informer started (which is the time the container started) and not for example since the last time the object was reconciled.
That 10-hour resync period is the one Tekton uses, we don't overwrite it AFAIK.
Results is probably going to be your best bet long-term, if you want to try that out and give feedback that'd probably be helpful to that project. Until then sharding is a reasonable band-aid, I'm sorry I don't have better solutions than that at the moment.

That's perfectly fine :). I´m just not sure if I´m seeing the (expected) periodic reprocessing or something in addition. My guess: lost connection to the API server. I´ll try to monitor this on my end and potentially re-open the issue.

We learned to live with this, but the 10h window is still quite annoying as it is impossible for us to have a timeframe (we use scheduled controller restarts to achieve that as workaround) during a business day where no developer is interrupted (some start their day early, some late).
Imo it would be good to either

extend the full reconciliation loop to something like 12h+
make it configurable
or smoothen it out (add only stuff to the queue if it has little work to do, in small junks for longer)

I see it as defect and the user experience is quite bad, nobody likes to see a stuck build.
I suggest this gets reopened and improved.

1500 Pipelines here and we are heavily impacted.
Devs and ops need to wait 20 minutes sometimes before the pipeline launch if they hit the sync loop issue.

+1 To extend the loop to 12h or 24h to have greater control of when the controller is performing the resync loop
+1 To make it configurable

authors: Fabian-K ImJasonH gerrnot prgss

repo: tektoncd/pipeline


text:
Add optional support for structured logging from Tasks.

Changes
This commit adds a new configmap that contains a setting that can switch the
logging mode of our entrypoint runner to "structured". If this mode is set, the
runners will read lines from stdout/stderr and wrap them in a json object before
logging.
The json object contains a 'message' key, which contains the actual log line,
and then key/value pairs for each tekton.dev/ label is attached to the Taskrun.
See this document for more context:
https://docs.google.com/document/d/19f2PpdxlzOKSKpCDAvZnQg2aCNjtLpFjrFxMiumllcE/edit?ts=5dcf1c5a
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[X] Includes tests (if functionality changed/added)
[X] Includes docs (if user facing)
[X] Commit messages follow commit message best practices

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
There is a new config map (config-entrypoint) to hold configuration for the Tekton entrypoint system. There is currently one setting there (runner: structured|default) that controls which logging output mode is used. If the structred setting is used, Tekton will convert raw stdout lines to json objects.

/lgtm

welcome back @dlorenc :D

/test pull-tekton-pipeline-integration-tests

/test tekton-pipeline-unit-tests

@dlorenc are you planning to get back to this or would it make sense to close it for now?

Closing for now but please reopen to take it over the finish line!

authors: bobcatfish dlorenc sbwsg


text:
🛑 Site web de Tela Botanica is down

In c235bab, Site web de Tela Botanica (https://www.tela-botanica.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Site web de Tela Botanica is back up in 8021a6d.

repo: telabotanica/status


text:
Bot stuck on "invalid forward fields combination" error

For some reason the bot is stuck on this error and is not able to recover
The problem is reproducible in the errors example of the crate
The exact cause is unknown, possibly related to #214 ?
env TELEGRAM_BOT_TOKEN="..."  cargo run --example=errors
Output:
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
[...]
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))
Err(Error(Raw(Error(Json(Error("invalid forward fields combination", line: 2, column: 1242))))))

I have the same error. My bot stopped working today https://github.com/ayrat555/el_monitorro
@gugahoa can you please take a look?

repo: telegram-rs/telegram-bot

license: mit


text:
🛑 Secret Site2 is down

In b9bc0eb, Secret Site2 ($SECRET_SITE_2) was down:

HTTP code: 404
Response time: 100 ms

Resolved: Secret Site2 is back up in 893ee80 after 47 minutes.

repo: templain/mywatcher


text:
🛑 MYSITE2 is down

In 13f8d79, MYSITE2 ($SECRET_SITE_OBOERU) was down:

HTTP code: 0
Response time: 0 ms

Resolved: MYSITE2 is back up in 90d9696 after 16 minutes.


text:
merge from template-arlac77-github

.github/workflows/ci.yml

chore(deps): add 20.15.0 (jobs.test-node.strategy.matrix.node-version)
chore: add 20.15.0 (jobs.release.steps.with.node-version)

.github/workflows/codeql_analysis.yml

chore: add 20.15.0 (jobs.analyze.steps.with.node-version)

.github/workflows/update_package_lock.yml

chore(deps): add 20.15.0 (jobs.update_package_lock.strategy.matrix.node-version)

Pull Request Test Coverage Report for Build 9625509880
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.918%

Totals

Change from base Build 9601618025:
0.0%

Covered Lines:
85

Relevant Lines:
88

💛  - Coveralls

repo: template-tools/template-sync-hook


text:
Make nondeterminism errors more clear

Some errors the SDK returns when it detects non determinism do not make it obvious to users that the root cause is likely non determinism.
Errors like:
lookup failed for scheduledEventID to activityID: scheduleEventID: 5, activityID: 5 [...]

No cached result found for side effectID[...]

more here
All these errors should suggested the error is likely caused by a non deterministic code change so users have a place to start debugging.

Is this a dup of #864?

authors: Quinn-With-Two-Ns josh-berry

repo: temporalio/sdk-go


text:
[Critical bug] fix: default Activity's taskQueue

What was changed

Change the default value of Activity's taskQueue from default to null.
Why?

It allows automatically propagating Workflow's queue to the Activity queue name by default, as described in the documentation
Checklist

Closes

How was this tested:

Tested locally.

Any docs updates needed?

Nope.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Oleg Coroliov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Oleg Coroliov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

It can be breaking change.
@wolfy-j wdyt?

Tests have been fixed and the code has been rebased.
Locally I have a bug(?) with the tests
Error: Class "Grpc\ChannelCredentials" not found

Probably it will be ok in CI
@roxblnfk

It can be breaking change.
It could.

@roxblnfk Some tests failed, but they probably just need to be re-run.

@roxblnfk Some tests failed, but they probably just need to be re-run (I see the same problem in other PRs)

Yea. Don't worry about it

Hi @ruscon. Could you sing CLA again?

@roxblnfk

license/cla — Contributor License Agreement is signed.

Ok. Thank you!

@roxblnfk Could you tag it, please?

@roxblnfk Could you tag it, please?

Certainly!

authors: CLAassistant roxblnfk ruscon

repo: temporalio/sdk-php


text:
[Bug] Investigate "legacy query response along with other commands"

Describe the bug
During test_workflow_dataclass_typed (especially so if you run with the Java test server), the following error occurs:

RuntimeError: Completion failure: Lang SDK sent us a malformed workflow completion for run (77f79520-d7cc-4fbc-beda-fc41aa6450f2): Workflow completion had a legacy query response along with other commands. This is not allowed and constitutes an error in the lang SDK

Originally this was thought to be https://github.com/temporalio/sdk-core/issues/390 but it occurs very rarely even against Temporal(ite). Investigate what is happening here.

It is https://github.com/temporalio/sdk-core/issues/390

repo: temporalio/sdk-python


text:
Release v0.6.0

Tendermint KMS v0.6.0-rc1 is feature-complete and we are presently running it in production at @iqlusioninc.
Here are some items to consider before a final release:
(Re-)Review Security-Critical Changes
It would be good to get another set of eyes on PRs tagged with security. There are two PRs that are particularly noteworthy:

#279: tendermint-rs: Reject low order points is a mitigation for a MitM vulnerability in Secret Connection (#142) in which an attacker can undetectably overwrite ephemeral keys so the shared secret value is 0. There's a separate PR to eliminate handshake malleability altogether by using transcript hashing (#254) which I'd consider the "real" solution to this problem (but isn't ready to ship), however in the meantime the "Prime, Order Please!" academic paper contains a formal security analysis of Secret Connection which detected the vulnerability as well as Tamarin proofs of the security of both solutions.
#300: yubihsm setup: Collect 256-bits entropy from both RNGs is a fix for a potential low-severity issue discovered during a security audit, in which in the event of a silent failure of one of the two RNGs used to derive the randomness for the master YubiHSM2 derivation password, the resulting secret (i.e. the 24-word BIP39 mnemonic phrase) would only have 128-bits entropy instead of the intended 256-bits. Even if this were to happen, 128-bits is still sufficient for security (hence the low severity), but since it addresses an audit finding (or as it were, the only audit finding), I think it deserves a second look.

I also switched to using the hkd32 crate for deriving the key hierarchy from the 24-word mnemonic, which uses the same derivation algorithm (which is a named subset extracted from BIP32) which was validated through the use of test vectors.
README.md updates
Much of the copy in the README.md is out-of-date now, and I think it'd be good to get it updated on the release so it appears on https://crates.io/crates/tmkms as well.
Here are some things I'd suggest updating and/or adding:
Tendermint KMS is currently ALPHA SOFTWARE AND UNAUDITED -- USE AT YOUR OWN RISK
The KMS underwent an audit with one low-severity finding (mentioned above). I think it'd be good to note that, and ideally publish the audit (or the parts of the audit specific to the KMS) and link them from the README.md.
Additionally, I think the KMS is "beta quality" at this point (especially given the large number of usability and feature improvements in this release), but will defer to others to make that call.
Security Issues
All of the security issues presently listed in the README have been addressed and the associated GitHub issues closed (#111, #142). I think this entire section can be removed.
Supported Platforms
This presently lists several 32-bit platforms, which we don't presently test on. Some of these platforms are known to have timing variabilities in core operations which make them unsuitable for cryptographic use (e.g. PPC32 short-circuits multiplication by 0 and 1). I think it'd be good to remove all of the 32-bit platforms (my fault for listing them in the first place), or at the very least remove PPC32.
Add Signing Provider List (w\ Recommendations)
Nowhere in the README.md is there presently a list of the available signing backends, nor is there any guidance anywhere around which ones to use.
I think it'd be good to add a list of the signing backends, in order of their relative maturity (i.e. YubiHSM2, Ledger, Soft Sign), and note that YubiHSM2 or Ledger are "recommended", as well as linking to the respective documentation for YubiHSM2 and Ledger (there is presently no specific documentation for the Soft Sign backend).

LGTM, and I could not be happier to see the update to the readme.

Release PR open: #329

v0.6.0 has been released

authors: jessysaurusrex tarcieri

repo: tendermint/kms


text:
🛑 Beeldoverleg Website is down

In 647a539, Beeldoverleg Website (https://beeldoverleg.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Beeldoverleg Website is back up in 3cea033 after 6 minutes.

repo: tendsgn/upptime


text:
貌似基于vite的vue3子应用目前无法支持？？

根据别人的react子应用的demo能够调通，但是基于vue3的子应用貌似无法正常运行dev

支持的，可以列出你的具体错误看看

错误就是跟直接引用type="module"的js一样，子应用加载之后运行过程中报错（eval方法执行不支持type module 的js）

example 里面有vue3 的例子可以参考下呢

repo: tengmaoqing/vite-plugin-qiankun


text:
Redireciona para o chat existente ao invés de sempre criar um novo chat

https://github.com/tenis-de-mesa/ft_transcendence/assets/26127185/7b4973ca-b1d7-4e3d-a86c-80a50ea6f3f6

Adiciona a rota GET /chats/with/:userId no back, que retorna se o usuário logado tem um chat com outro user
O font pede pro back se tem um chat. Caso tenha, redireciona pro chat, se não, vai pra página de new

Closes #63

Eu testei aqui e o problema só foi resolvido no frontend
function createChat() {
  const url = "http://localhost:3001/chats";
  const body = {
    userIds: [1, 2],
    type: "direct",
    access: "private",
    message: null,
  };
  fetch(url, {
    method: "POST",
    credentials: "include",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  })
    .then((result) => result.json())
    .then(console.log);
}

Se depois de logar essa função for executada ainda é criado um novo direct chat entre duas pessoas

Boa! Dá pro back não deixar criar também (;

Eu testei aqui e o problema só foi resolvido no frontend
function createChat() {
  const url = "http://localhost:3001/chats";
  const body = {
    userIds: [1, 2],
    type: "direct",
    access: "private",
    message: null,
  };
  fetch(url, {
    method: "POST",
    credentials: "include",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  })
    .then((result) => result.json())
    .then(console.log);
}

Se depois de logar essa função for executada ainda é criado um novo direct chat entre duas pessoas

Agora retorna um 409 Conflict quando já tem


text:
Add sanity check for addons

Since #63 has introduced sanity check for addons, can we add it for CI?
cc @av8ramit @karmel @seanpmorgan
command:
make sanity-check
# or
bash tools/ci_build/install/install_ci_dependency.sh --quiet
bash tools/ci_build/ci_sanity.sh --incremental

sanity check should always pass after #64 is merged into master.

This should be possible.

Ah, wait, modulo viewing results. @av8ramit -- for results to be public, does everything have to be run through Bazel currently? That is, we can't run arbitrary scripts like this directly?

I'll try and see if we can just invoke a script and just see the results that way.

Okay, so internally we now just invoke the script. So now you can modify and add whatever you'd like to the script and it will run in presubmit and continuous.

Yeah, it works well, but I prefer to keep them (sanity-check and unit-test) independent. What do you think?

Agree that it's significantly easier to review if we have a separate script ran for linting type errors. @av8ramit do you think it would be possible to run one additional script like addons_style.sh. We can then probably fit most other things we want ran into one of our existing shell scripts.

Do you want a separate job for the linting script? Or do you want that to run as part of CPU tests with Python2/3?

a separate job for the linting script

+1, yes. If possible, run it in the python 3 Linux CPU environment.

Friendly ping @av8ramit

I am working on this now; sorry for the delay. Should have it ready in the next couple of days.

These are now running, though failing: https://source.cloud.google.com/results/invocations/1b2b8286-1722-4c51-a05e-274bad80607b/targets/tensorflow_addons%2Fubuntu%2Fcpu%2Fsanity%2Fpresubmit/log
Looks like a unicode issue; @facaiy / @seanpmorgan can you take a look, and let me know if it's in the test or something we should fix in the test env?

Thanks, Karmel, I'll take a look later. By the way, it seems python 3.4?

@karmel I cannot reproduce it in the tensorflow/tensorflow:nightly-custom-op image:
root@e45e92485b69:/addons# git log -n 1
commit 3bf1b5fde8a2ba8b95f746ba01e8e14206a5bbfc
Author: Kapil Sachdeva <ksachdeva17@gmail.com>
Date:   Tue Mar 19 10:53:21 2019 -0500

    FIX - Use public keras API for custom_object decorator (#97)
root@e45e92485b69:/addons# python3 tools/test/check_futures_test.py
root@e45e92485b69:/addons# echo $?
0
root@e45e92485b69:/addons# python2 tools/test/check_futures_test.py
root@e45e92485b69:/addons# echo $?
0
root@e45e92485b69:/addons# python3 --version
Python 3.4.3
root@e45e92485b69:/addons# python2 --version
Python 2.7.6

Could you print the path before opening it?
https://github.com/tensorflow/addons/blob/3bf1b5fde8a2ba8b95f746ba01e8e14206a5bbfc/tools/test/check_futures_test.py#L51

Gently ping @karmel @av8ramit

Sorry— I’m OOO, but will be able to take a look next week.
On Fri, Mar 22, 2019 at 6:39 AM Yan Facai (颜发才) notifications@github.com
wrote:

Gently ping @karmel https://github.com/karmel @av8ramit
https://github.com/av8ramit
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/addons/issues/68#issuecomment-475624334,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAowocIk0RMEzljMr369ecfMW7Q1R2x2ks5vZN0sgaJpZM4bbvI7
.

That's fine, please update the issue when new information becomes available :-)

Here is what I get. Do you want me to update to Python3.5?
=== Sanity check step 4 of 6:  do_check_futures_test (Check that python files have certain __future__ imports) ===

Traceback (most recent call last):
  File "check_futures_test.py", line 105, in <module>
    main()
  File "check_futures_test.py", line 101, in main
    raise AssertionError('\n'.join(error_msgs))
AssertionError: Error in tf/lib/python3.4/io.py: Missing futures: absolute_import division print_function
Error in tf/lib/python3.4/types.py: Missing futures: absolute_import division print_function

...

Error in tf/bin/activate_this.py: Missing futures: absolute_import division print_function

No, Amit, I don't think py3.5 can fix the problem. We expect that check_futures_test.py) only checks those python files of addons. It looks amazing that tf/bin/activate_this.py is in the addons directory?

Amit, do you know what is the location of the tf/lib directory? Is it in the tensorflow_addons?

@facaiy yeah I think that's it. I moved the virtualenv outside of the addons directory and it works now.

Thanks for the help!

@av8ramit Is there an issue with displaying the logs?
For example see this report:
https://source.cloud.google.com/results/invocations/bb8ac7a3-2ef6-487d-a221-e639f83cf410bmit/log
Coming from this PR:
https://github.com/tensorflow/addons/pull/102

I didn't change anything. Might have just taken a few minutes.

authors: av8ramit facaiy karmel seanpmorgan

repo: tensorflow/addons


text:
Fix local build with GPU support

Description
Allow local build with GPU support against TF 2.10
Partly fixes https://github.com/tensorflow/addons/issues/2712
To build locally it still required to manually replace crosstool_top value in .bazelrc with "@local_config_cuda//crosstool:toolchain"
Type of change

[x] Bug fix
[ ] New Tutorial
[ ] Updated or additional documentation
[ ] Additional Testing
[ ] New Activation and the changes conform to the activation contribution guidelines
[ ] New Callback and the changes conform to the callback contribution guidelines
[ ] New Image addition and the changes conform to the image op contribution guidelines
[ ] New Layer and the changes conform to the layer contribution guidelines
[ ] New Loss and the changes conform to the loss contribution guidelines
[ ] New Metric and the changes conform to the metric contribution guidelines
[ ] New Optimizer and the changes conform to the optimizer contribution guidelines
[ ] New RNN Cell and the changes conform to the rnn contribution guidelines
[ ] New Seq2seq addition and the changes conform to the seq2seq contribution guidelines
[ ] New Text addition and the changes conform to the text op contribution guidelines

Checklist:

[x] I've properly formatted my code according to the guidelines

[ ] By running Black + Flake8
[ ] By running pre-commit hooks

[x] This PR addresses an already submitted issue for TensorFlow Addons
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] This PR contains modifications to C++ custom-ops

How Has This Been Tested?
Local build with GPU support finished successful with these changes

What happen here?

Sorry, didn't know that removing my fork will close this PR.
Here is a full copy of this PR https://github.com/tensorflow/addons/pull/2764

authors: bhack shkarupa-alex


text:
TF 2.9.0 and TFDF 0.2.5 no distribution found messages

Dear Gentlemen,
I tried to install TFDF according to Compatibility table and I an got error:
(metal-venv) jayokocha@Aleksandrs-MacBook-Pro Light_weight_network % pip install tensorflow-decision-forests==0.2.5
ERROR: Could not find a version that satisfies the requirement tensorflow-decision-forests==0.2.5 (from versions: 0.2.3, 0.2.4, 1.0.1, 1.1.0, 1.2.0)
ERROR: No matching distribution found for tensorflow-decision-forests==0.2.5

Hi,
Tensorflow Decision Forests 0.2.5 was not released for MacOS, so please use an earlier version of TF-DF, if possible.
Additional note: We've recently gotten reports that TF-DF is incompatible with tensorflow-metal. If you experience this issue, please let us know.

Hi! Yes, I'm using tensorflow-metal. Earlier version of TF-DF incompatible with TF 2.9.0

Closing this since we have a parallel discussing in https://github.com/tensorflow/decision-forests/issues/152#issuecomment-1436571168

authors: dubrovin-sudo rstz

repo: tensorflow/decision-forests


text:
Change pip install target to "tensorflow>=1.7" from "tensorflow --pre".

PiperOrigin-RevId: 193180815

Ignoring CLA since this is authored by tensorflow hub authors.

authors: andresusanopinto vbardiovskyg

repo: tensorflow/hub


text:
Instead of a separate Keras layer, should we just have a single Keras-like framework API?

This issue continues a discussion that started in #91. Here's a brief excerpt of key points made so far.
@karllessard wrote:

The current Python API is made up of two layers because it is historically the product of a merge between two different projects: the original TF API and the Keras project. I personally think it brings more confusion to the users than benefits and we don't need to follow this schema if we think we can do better in Java since we start from scratch.

I'm slowly leaning now to the idea of having a single API that supports both "beginner" and "advanced" modes, whether we call it Keras or not.

@JimClarke5 wrote:

IMHO, the beauty of Keras is in the simple, straight forward, Model and Layers. Most of the Layers have defaults for constructs like Metrics, Optimizers, Activations, etc. Also, they allow simple strings in their parameters that instruct the underlying layers to construct elements, like new Dense(24, "relu"), so the way these elements are constructed can be hidden from a Keras user.

@Craigacp wrote:

My preference is to have both a low and high level framework, which is how TF python currently is. You don't need to use Keras if you don't want to, but many people do.

One reason to advocate for both frameworks is that it might actually take less development effort. Building out Keras to have full coverage requires a lot of consistent effort, but supporting ops that are added to TF's C API in a lower level API is essentially free for us.

The high level framework is for people who use Keras in TF Python, and want an API that guides them better. I think that we should have stronger typing information than exists in Python, as it's what would be expected from idiomatic Java and it helps IDEs & discoverability.

Here's an argument for just a single framework layer.
Providing two API layers would add code, tests, and other complexity to this repo. We'd have two ongoing API design efforts.
If a Keras layer completely covered lower layers, then we'd have the added cost and complexity of percolating every new bit of lower-level functionality up through the Keras layer. We probably still wouldn't achieve totally airtight, so we'd have our very own leaky abstraction from the outset. If a Keras layer didn't attempt to completely cover lower layers, then we would be asking every user to understand our layering idea from the outset.
Perhaps a better starting point would be to attempt a single framework API that addresses the full spectrum of user needs? We would feel the tension between beginner/implicit/best-practice-defaults and expert/explicit, but that's a normal tension in API design and is addressed (to varying degrees of success) by normal API design techniques.
If we find ourselves forced into some sort of layering, we will understand the motivations much more concretely at that point, so we can make better decisions.

I share the same concerns as @deansher about the complexity of maintaining two distinct APIs. Another thing worth mentioning is that every time someone will try to add a new logic in the Keras layer, chances are that we will ask him to move that logic to tensorflow-framework first, expose it with an API very similar to what Keras is doing and then wrap it up with the original Keras interface in tensorflow-keras (exactly what I did with @JimClarke5 in his optimizer PR). I think we all agree that we want to avoid duplicating the same logic and only what is added to the framework can be shared across libraries.
If we opt for the single API though, we probably don't want to call it "Keras" or we will feel forced to mirror as much as possible the Python Keras library, even if it is not as flexible as we want it to be in some cases. On the other hand, just taking what we like from the Keras API and adding it to our framework kind of dissolve the notorious exposure of having a pure-Keras implementation in Java, which can attract more ML developers to switch to the JVM. Still, that is probably also what would allow us to build the best API in Java to TensorFlow users.

Before we can merge new PRs from @JimClarke5 , I think we really need to reach an agreement on this point.
In addition to my previous comment, I think we can address the question from another angle, by asking ourselves for who do we build the Keras API for Java.
a) For Python users already familiar with it?
b) For any users, presuming that if Keras was that successful on Python, it should then be the right API for Java?
If the answer is a), then we probably want to stay very close to what the Python Keras API offers and the facade pattern as proposed initially, sitting on top of the framework, is probably the right choice. If answer is b), then I feel that we are more free to move away from the original API, bringing only the important pieces and enhancing them with what we think is missing for a more complete solution that can satisfy both beginners and more advanced users. In this case, having a single framework should be enough.

The current PRs, "Optimizer Learning Rate Change" and "Initialization" are focused only on framework, and comprise elements that can be independent of any Keras implementation and can be used on their own. I would think,  concepts like initialization, loss(cost) functions, activation functions, regularization etc. would transcend all ML implementations. To that end, I support adding these stand-alone elements to framework, independently of the decision on Keras. We may revisit some of the method signatures with a view to broader use, but I think the basic functionality will still be needed in many higher level implementations.

I have a hard time picturing Python users of Keras who would be attracted to a Java TensorFlow by a close API kinship to Python Keras, but turned off if the API was more Java-idiomatic with not as close a kinship.
For the foreseeable future, most deep learning will still be in Python. Also, moving from Python to another language like Java is a huge change, no matter how close the API kinship. I expect the users of Java TensorFlow to be in two categories:

Java programmers (or Java shops) who want to incorporate deep learning. These are probably not experts in Python Keras. Also, they probably care a lot about idiomatic Java and about an API that plays to Java's strengths. I expect them to be far more attracted to "it's lovely Java!" than "it's just like using Keras in Python!".

Python TensorFlow programmers (or shops) who decide they should move some work Java for operational reasons. Practically speaking, this is a huge change, so they must have some huge motivation. I can't picturing them saying "yeah, but the Java API is actually rather different from Python Keras, so let's not."

I think there is a third group, which is Java developers that have to port into Java from Python whatever their data science team came up with. There they might appreciate something similar to the Python API as it would be easier to see how to transform the Python source into Java source. In my experience this process of porting something from Python into Java for deployment tends to be pretty common, though that might be because I mainly talk to people who work at massive companies which can afford to have this disconnect between data science and deployment.
I think that we're still quite far away from having any kind of higher level API, and much of the work towards it is building out lower level blocks, so we should proceed on a little ways before trying to make a final decision. Much of the current code is getting out of the C API and wrapping it so that the constructs are usable in Java, as many of the C API pieces seem to be incomplete. We'd need to build these components anyway, and so surfacing a public API on top of them at some level is still further down the line.

Great point from @Craigacp : "Java developers that have to port into Java from Python whatever their data science team came up with. There they might appreciate something similar to the Python API as it would be easier to see how to transform the Python source into Java source." Another example along these lines that will be very common is developers that are studying an existing model in a paper or in open source and reimplementing it in Java.
While also agreeing with @Craigacp that "we should proceed on a little ways before trying to make a final decision", perhaps it would be worth documenting provisional goals in our README? Here's a shot at abstracting the above discussion into provisional goals for our framework API:

If either you know how to implement a model in the Python Keras API, or you are reimplementing an existing Python Keras model in Java, you should be able to cleanly and naturally follow the same high-level structure in the framework API.

Also, given some familiarity with patterns followed throughout the framework API, you should be able to easily translate every detail of a Python Keras implementation into the framework API.

However, the framework API is not intended to literally mimic the Python Keras API. Rather, it should expose the same capabilities in an API that feels natural and idiomatic to a Java programmer who does not know Keras. If we ever find ourselves unable to reconcile this goal with easy translation from Python Keras, we may split out a Keras layer.

Also, the framework API should support fine control over all aspects of modeling, training, and inference. Unlike with Python Keras, we want this to feel like staying in the same API rather than diving into a separate layer. But here again, if we are ever unable to reconcile this goal with easy translation from Python Keras, we may split the framework API into two layers.

Thoughts?

I guess the fundamental difference between Keras and not Keras is the model.compile and model.fit functions. These restrict what can be done with a Keras model in fairly fundamental ways (e.g. they make it hard to do multi-task learning across multiple datasets and losses), but they make it substantially easier to use by having a model object and sensible entry points that show users how to build supervised learning models. If we made the Keras Java implementation more idiomatically Java, then the Model object would own the layers and they wouldn't be mutable outside it, which is in conflict with the Python Keras as it doesn't care (which makes it less safe).
Stepping outside of supervised learning (e.g. to RL, or to multi-task supervised learning) means you have to leave behind bits of the Keras interface (e.g. the Keras RL examples here - https://keras.io/examples/rl/deep_q_network_breakout/) and don't really use compile or fit. My main concern is that we don't force people into something as restrictive as Keras without the appropriate escape hatches, and I think that having those hatches essentially dictates that we have two high-ish level interfaces one Keras, and one non-Keras. But the non-Keras interface is pretty much what we have in frameworks at the moment, which is just a prettied up version of the C API which has the missing bits patched over. Plus we'd need to get the gradient tape, but that's a discussion for another time.

I guess the fundamental difference between Keras and not Keras is the model.compile and model.fit functions.

train_on_batch is the Keras endpoint giving more flexibility to the developers in their training loop, I don't know if that can also apply to the specific use cases you had in mind @Craigacp ?

If we made the Keras Java implementation more idiomatically Java, then the Model object would own the layers and they wouldn't be mutable outside it, which is in conflict with the Python Keras as it doesn't care (which makes it less safe).

Maybe this can be handled by renaming Model to ModelTemplate, which is then concretized as a Model on model.compile, following pretty much the basic builder pattern in Java.
@deansher , I agree with your list of goals. It seems that the general consensus is that we should first build up a complete framework that is both user-friendly and flexible enough to support more complex or advanced tasks, and then reevaluate the need of having a second API that mirrors as close as possible Python Keras.

I guess the fundamental difference between Keras and not Keras is the model.compile and model.fit functions.

train_on_batch is the Keras endpoint giving more flexibility to the developers in their training loop, I don't know if that can also apply to the specific use cases you had in mind @Craigacp ?

train_on_batch works fine if the loss function is the same for each batch, but that's not true for some of the NLP use cases I'm working on (e.g. we train a model on a masked language model loss for some datasets, and similarity losses for others). Though I guess there could be multiple models which share layers, but I don't know how that would work if we did impose ownership of layers.

Add my 5 cents here: I suggest to keep low level API as graph + optimizers + load/saving variables and keep it separated from Keras package. Maybe initializers, losses and metris should be added too. But Activations, training cycle, layers could be developed in Keras package. Of course we should not have two examples of HeNormal initializers for example, only one

Agree with losses and metrics; but initializers should be with activations, no?

One interesting question: If we decide, or definitively determine, that something is implemented wrong in Python, when will we fix it in Java and when will we be carefully bug-for-bug compatible?  Thinking about the goal I proposed above, based on our discussion, "if either you know how to implement a model in the Python Keras API, or you are reimplementing an existing Python Keras model in Java, you should be able to cleanly and naturally follow the same high-level structure in the framework API."

I have 20 years java experience and will never use python, I just want to use a java DL framework that has the strongest capacities but not to refer to python. In China,  java is the top 1 developement language and many developers should not use python because the most business systems are developed in java.

If we find errors or better ways to implement algorithms in Java, I am all for changing TF Java. We have just found a Keras limitation on 1D Softmax inputs, and we removed that restriction in the Java implementation.  We are also leverging Java strong typing which leads to, IMO, cleaner implementations.

Although we don't have complete consensus, perhaps we have sufficient consensus to

drop the empty keras package for now,
change our README accordingly,
document provisional goals for our framework API in our README,
and revisit this issue when someone eventually raises a new issue that proposed a specific, immediate split into a Keras layer?

Just to add that I’m fine with this last proposal from @deansher

We have agreed on a path forward and documented it in READMEs.

authors: Craigacp JimClarke5 KartikChugh SidneyLann deansher karllessard zaleslaw

repo: tensorflow/java


text:
Missing training data

Hello everyone,
i'm having a problem with the missing data:
when i run the following training command i get an error which says the number of steps is 0:
BOARD_SIZE=19 python3 main.py train-dir
gs://$BUCKET_NAME/data/training_chunks
gs://$BUCKET_NAME/models/000001-somename
--model_dir estimator_working_dir
after reading many part of the code i found that with this call
dual_net.train(training_file) #line 92 rl_loop.py
the default value of steps is -1
and in dual_net.py :
if steps == -1:
#some code here then
steps = total_examples // effective_batch_size
going further :
with gsutil ls gs://$BUCKET_NAME, there is no data folder or file:
gs://minigo-pub/v9-19x19/models/
gs://minigo-pub/v9-19x19/sgf/
does any one know how can i get the data

I've started copying over the ~500GB of training chunk data to gs://minigo-pub/v9-19x19/data/golden_chunks. Should be done in a few minutes.

Thank you @brilee, that was hopeful

@brilee
Where can I get your data in China? Such as BaiduNetDisk. Because GS can not be used in China.Thank you.

Ah.. that's unfortunate. I don't know anything about the Chinese tech ecosystem so I'm afraid I can't help you out there.

authors: IsmailMamoru brilee fmscole

repo: tensorflow/minigo


text:
do fine tuning with my own image size

I would like to do transfer learning and fine tuning with some pretrained model in tensorflow.contrib.slim. I searched some examples about this. However, almost all of these examples will resize images to the specified size that the model needs. For example, 224×224 is the size that vgg16 needed. Will it be possible to set my own image size? Like 512×512. In keras, I can do this like following:
`base_model= VGG16(include_top=False, input_shape=(512, 512, 3))`

Up till now, I have not found the solution, can this be down in tensorflow and slim?

That sounds like a question for stackoverflow; please close this issue here. Thanks!

repo: tensorflow/tensorflow


text:
error reported while using TensorForestEstimator. tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'FertileStatsResourceHandleOp' in binary running on...

Have I written custom code - Yes, but provided code in link also has same problem
OS Platform and Distribution - Windows 7
TensorFlow installed from - PIP install
TensorFlow version - 1.4 and 1.5 both on CPU
Bazel version - No bazel version used
CUDA/cuDNN version - N/A
GPU model and memory - N/A
Exact command to reproduce - provided a link to python script below to reproduce this issue
I am trying to use "TensorForestEstimator" defined in "tensorflow.contrib.tensor_forest.client.random_forest". I get following error.
Same error is reported when I run this script
https://www.kaggle.com/biscuitlickz/iris-predictions-using-tensorflow/code
Traceback (most recent call last):
File "titanic_random_forest.py", line 201, in
app.run(main=main)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run
_sys.exit(main(argv))
File "titanic_random_forest.py", line 136, in main
est.fit(input_fn=train_input_fn, steps=TRAIN_STEPS)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\util\deprecation.py", line 316, in new_func
return func(*args, **kwargs)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 480, in fit
loss = self._train_model(input_fn=input_fn, hooks=hooks)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 985, in _train_model
model_fn_ops = self._get_train_ops(features, labels)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 1201, in _get_train_ops
return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.TRAIN)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 1165, in _call_model_fn
model_fn_results = self._model_fn(features, labels, **kwargs)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\tensor_forest\client\random_forest.py", line 168, in _model_fn
device_assigner=dev_assn)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\tensor_forest\python\tensor_forest.py", line 376, in init
tree_variables_class=tree_variables_class)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\tensor_forest\python\tensor_forest.py", line 350, in init
self.variables.append(tree_variables_class(params, i, training))
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\tensor_forest\python\tensor_forest.py", line 318, in init
params, '', self.get_tree_name('stats', tree_num))
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\tensor_forest\python\ops\stats_ops.py", line 102, in fertile_stats_variable
container, shared_name=name, name=name)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\tensor_forest\python\ops\gen_stats_ops.py", line 141, in fertile_stats_resource_handle_op
shared_name=shared_name, name=name)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 3162, in create_op
compute_device=compute_device)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 3208, in _create_op_helper
set_shapes_for_outputs(op)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2427, in set_shapes_for_outputs
return _set_shapes_for_outputs(op)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2400, in _set_shapes_for_outputs
shapes = shape_func(op)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2330, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 686, in _call_cpp_shape_fn_impl
input_tensors_as_shapes, status)
File "F:\programs-all\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'FertileStatsResourceHandleOp' in binary running on ATWOOD_SRAO. Make sure the Op and Kernel are registered in the binary running in this process.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

required details updated above.

The same is happening for me on windows 10, same version installed from pip CPU only

@caisq can you please take a look?

Nagging Awaiting TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Hi, any updates on this issue? I am facing the same issue when trying to import a saved model using Python as well as Java bindings.

@gilberthendry, @thomascolthurst could you also take a look into this?

@nataliaponomareva

any update on this? I am getting same error.
File "load_grph.py", line 20, in load_graph
tf.import_graph_def(graph_def, name="prefix")
File "/home/neva/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 285, in import_graph_def
raise ValueError('No op named %s in defined operations.' % node.op)

I ran into the same issue.

I am getting same error.

I am getting the same issue

Nagging Assignee @caisq: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

+@yupbank

hi... sorry for not the late reply, the  Op type not registered '*' in binary is a know issue in java land, since by default  tf.contrib project .so didn't get package within the java distribution can be addessed by TensorFlow.loadLibrary() https://github.com/tensorflow/tensorflow/issues/10454#issuecomment-344139462
Meanwhile, for the tensorflow in python having this kind of error could be soemthing to do with misconfiguration with bazel? i'll have to spend more time into reproduce the error to achieve a conclusion

Hello @yupbank, I'm also having the same problem, I wrote a minimum script that reproduces the error:
import pandas as pd
import tensorflow as tf
from tensorflow.contrib.tensor_forest.python import tensor_forest
from tensorflow.contrib.tensor_forest.client import random_forest

def main():
    model_dir = './model'

    params = tensor_forest.ForestHParams(
        num_features=1,
        num_classes=2,
        num_trees=100,
        max_nodes=1000,
    ).fill()

    classifier = random_forest.TensorForestEstimator(
        params,
        model_dir=model_dir
    )

    data = pd.DataFrame(data={'x':[2, 3], 'y': [0, 1]})

    x_train, y_train = data, data.pop('y')

    input_fn_train = tf.estimator.inputs.pandas_input_fn(
        x=x_train,
        y=y_train,
        batch_size=1,
        num_epochs=100,
        shuffle=True,
        target_column='y'
    )

    classifier.fit(input_fn=input_fn_train)

if __name__ == '__main__':
    main()

Thank you @lanahra  !! can you have do this
You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Unfortunately I'm running on a Windows machine, so I cannot run the environment capture script.
Here is the second one:
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

C:\Users\%USER%\AppData\Local\Programs\Python\Python36\lib\site-packages\h5py\__init__.py:36:
FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is
deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
b'v1.8.0-0-g93bc2e2072' 1.8.0

sorry guys... according to the latest changes to tensor_forest BUILD file.
https://github.com/tensorflow/tensorflow/commit/38e0139329482d8e44629dea2e87853808eacd0d#diff-894ca5f2ae1e390cc3eba58d9e4052ed
windows support is not being added for now i guess...
tf.contrib projects are not officially supported, and may change or be removed at any time without notice.
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib#tensorflow-contrib

dll missing from the folder

i'll close this issue for now, and will try to make more priority to add windows support for tensor_forest

@yupbank I found the same issue on ubuntu18.04 while trying to read the node info from .pb model file in TF 1.14.0, and this error is caused by compilation with Bazel(I tried to fix it, and I found in tensorflow\contrib\tensor_forest\python\ops\_stats_ops.so the so-called FertileStatsResourceHandleOp is already existed), so I guess that your team has fixed that problem on Ubuntu whereas .dll problem is still in suspensory status and to make the error messages sync among platforms, however, the same error reported on Ubuntu has solutions. Thus, I worked it out with
import tensorflow as tf
from tensorflow.contrib.tensor_forest.python import tensor_forest
from tensorflow.python.ops import resources

and this solution is inspired by https://github.com/aymericdamien/TensorFlow-Examples/issues/212#issuecomment-743026970
In brief words, if anybody has the same problem (in TF 1) on Win10, they can avoid these errors by using Ubuntu and import proper modules in tf.contrib.

authors: Avtarsingh127 Milkve ThomasColthurst bexvan bignamehyp deathmantim gavintgh lanahra nataliaponomareva rainlei2001 rohanag12 sanjayraojjn shivaniag tensorflowbutler yupbank


text:
How to set the truncation time series length parameter for RNN training in this example: tensorflow/tensorflow/contrib/timeseries/examples/lstm.py

In the function of tf.contrib.timeseries.ARRegressor, we can set the input_window_size and output_window_size for building a time series forecasting model.
But in this example, we can only set the parameter of num_units of cell and num_features of input data for _LSTMModel，but we can not set the parameter of the truncation time series length in this model of TFTS.

Please ask questions on StackOverflow instead of opening bugs. For timeseries stuff feel free to send the SO link to allenl@google.com.
But the LSTM example model does not operate on a fixed window; the same model can take any window size, like dynamic_rnn. You can set the truncated backprop length by setting a different window size in the input function. For ARRegressor the model and input function window sizes have to match for training.

authors: MrBigmonster allenlavoie


text:
App crashes when building with gradle

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
NO

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
macOS Sierra 10.12.6

TensorFlow installed from (source or binary):
source

Python version:
python 3.6

Bazel version (if compiling from source):
bazel 0.13.0

GCC/Compiler version (if compiling from source):
GCC 4.2.1

I managed to implement the object detection api in my app with my trained model. Despite that the app works fine I always get an error the the tensorflow_demo.so was not find so the tracking is unavailable which affected the bounding boxes in comparison with the tensorflow object detection demo (when you build it with bazel)
I downloaded the missing .so file and add it to my project by adding this to my gradle :
 lintOptions {
    abortOnError false
}

sourceSets.main {
    jniLibs.srcDirs =  ['libs']
}

and then dropping the files into the created directory (JniLibs)
when I unpack my .apk file I can see clearly that the .so are packed in the .apk :
 20336344  Defl:N  5832450  71% 00-00-1980 00:00 33cca329  lib/arm64-       v8a/libtensorflow_inference.so
  412440  Defl:N   210820  49% 00-00-1980 00:00 f305dd00  lib/armeabi- v7a/libtensorflow_demo.so
  1.3987972  Defl:N  4856867  65% 00-00-1980 00:00 be7adc48  lib/armeabi-v7a/libtensorflow_inference.so
   21178780  Defl:N  6353576  70% 00-00-1980 00:00 addfcb9d  lib/x86/libtensorflow_inference.so
   21811248  Defl:N  6681692  69% 00-00-1980 00:00 4cac1c73  lib/x86_64/libtensorflow_inference.so

the problem now is when I launch the app it crashes with the following error :
     No implementation found for void package_name_tracking.ObjectTracker.initNative(int, int, boolean) (tried Java_package_name_tracking_ObjectTracker_initNative and Java_package_name_tracking_ObjectTracker_initNative__IIZ)

I want to avoid the use of bazel or cmake, is it possible or there is a solution for the error I'm getting !!
Any information would be helpful

Could it be related to the android studio update? After updating, I started getting errors in my old apps

As suggested downloading and including the .so file to your own project was not enough. Since we're including a pre-compiled version of the library the native methods (in the .cpp files) have the package name of the tensorflow demo on their names. That's why this error occur since our own projects packages named differently.

To solve this problem you need to rename your package to org.tensorflow.demo and make sure that ObjectTracker.java is under org.tensorflow.demo.tracking

Second solution would be  changing the naming of the native methods in the .cpp files according to your package name and then rebuild the .so file with bazel (faced many problems there if anyone manage to do it please mention how).

I hope the tensorflow will mention this information when suggesting to download and drop the pre-buit version so the not so experienced developers won't waste unnecessary time on this issue.

Nagging Assignees @petewarden, @angersson: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignees @petewarden, @angersson: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

I haven't had a chance to look at this. Passing to @achowdhery to take a look, since she's been working on this code recently.

@achraf-boussaada Please consider following the instructions in this Tutorial with bazel build. Please suggest if you are still stuck after trying those.

Nagging Assignee @achowdhery: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@achraf-boussaada Are you able to build the app with the latest code release?

Hi, I have the same problem. I managed to implement object detection with my own model but I get the error tensorflow_demo.so is not found and so the tracking support cannot be enabled. I've download the package and replace it to jniLibs folder of my app and I haven't change the default app name. Still the error exists. Could you please help me?
How can I build my own tensorflow_demo.so?

@achowdhery sorry for the late response ! unfortunately i didn't get to test it with the new build since i'm no longer working on the project..  i will keep the issue updated if i did come back to working on it.
@MohammadMoradi did you make sure you supported all CPU architectures? that may be your problem. it should be looking like this :

--jniLibs
----arm64-v8
--------tensorflow_demo.so
----armeabi-v7a
--------tensorflow_demo.so
----x86
--------tensorflow_demo.so
----x86_64
--------tensorflow_demo.so

@achraf-boussaada Thanks for your response. My CPU architecture is armeabi-v7a and I only made the areabi-v7a directory. But maybe I should build all of them. Not sure!

Hello again @achowdhery :)
As you probably remember from here I am still not able to build .tflite file, so I decided to user Tensorflow Mobile with frozen graph instead of ml-kit, as it was in the article
So, I successfully built the demo project with Android Studio as it was described in this tutorial and TF Classify works great, but I need TF Detect. And with enabling TF Detect demo I faced the same problem that guys above T____T
I really hope there is some solution for at least on of these issues...

@kate-kate you are building with Android Studio and cant get the app to build? What is the error message? You already get the TF Lite file correctly?

@kate-kate I am happy to help you. Please answer the following questions

Platform where you are install tensorflow. How did you install Tensorflow? What version?
Are you able to download a mobilenet_v1_ssd coco checkpoint and convert it to frozen graph?
Are you able to convert a frozen graph to tflite file?
Are you able to get the some version of Android Studio SDK and NDK? What is the version?
Are you able to build the app? Note for step 5, you dont need to try 1-3, there is a pre-built coco tflite model already in the app folder if you are trying to build it. What steps did you follow to build app? Is this device on a 32 bit architecture or 64 bit architecture?
Did you try to deploy the app on your Android Device?
In the steps above, please provide detailed instructions where something failed to help us reproduce. Also, it would be useful for you to try this with bazel build if Android Studio is not working (the key challenge is to get all the SDK and NDK versions correct in Android Studio).

@achowdhery Deeply appreciate your help.

I install Tensorflow 1.10 on mac os from source
Yes
I have an issue with "Converting unsupported operation: TFLite_Detection_PostProcess" but after converting the file appears
NDK version 16, SDK - 8.0
I've succesfully built the app (with an Android Studio) and even used TF Classify action, it worked
No, I used only emulator

@kate-kate I believe your TFlite file conversion is successful. "Converting unsupported operation: TFLite_Detection_PostProcess" message is showing up simply because this custom op is only in Tensorflow Lite and not in Tensorflow.
Now the main question that needs debugging is why the Android app does not work. Are you able to print out the logcat messages and copy/paste them here?

@achowdhery Sure
Logcat messages when I use TFlite file and android demo with ml-kit:
08-21 11:22:14.971 16825-16825/com.google.firebase.codelab.mlkit_custommodel W/System.err: com.google.firebase.ml.common.FirebaseMLException: The load task failed
08-21 11:22:14.973 16825-16825/com.google.firebase.codelab.mlkit_custommodel W/System.err:     at com.google.android.gms.internal.firebase_ml.zzgy.zzf(Unknown Source:75)
        at com.google.android.gms.internal.firebase_ml.zzgu.call(Unknown Source:8)
        at com.google.android.gms.internal.firebase_ml.zzgp.zza(Unknown Source:0)
08-21 11:22:14.974 16825-16825/com.google.firebase.codelab.mlkit_custommodel W/System.err:     at com.google.android.gms.internal.firebase_ml.zzgq.run(Unknown Source:4)
08-21 11:22:14.978 16825-16825/com.google.firebase.codelab.mlkit_custommodel W/System.err:     at android.os.Handler.handleCallback(Handler.java:789)
08-21 11:22:14.980 16825-16825/com.google.firebase.codelab.mlkit_custommodel W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:164)
        at android.os.HandlerThread.run(HandlerThread.java:65)
    Caused by: com.google.firebase.ml.common.FirebaseMLException: Internal error has occurred when executing Firebase ML tasks
        at com.google.android.gms.internal.firebase_ml.zzgp.zza(Unknown Source:15)
    	... 5 more
    Caused by: java.lang.IllegalArgumentException: Cannot create interpreter: Didn't find custom op for name 'TFLite_Detection_PostProcess'
    Registration failed.
08-21 11:22:14.981 16825-16825/com.google.firebase.codelab.mlkit_custommodel W/System.err:     at org.tensorflow.lite.NativeInterpreterWrapper.createInterpreter(Native Method)
        at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:63)
        at org.tensorflow.lite.NativeInterpreterWrapper.<init>(NativeInterpreterWrapper.java:51)
        at org.tensorflow.lite.Interpreter.<init>(Interpreter.java:90)
        at com.google.android.gms.internal.firebase_ml.zzhn.zzey(Unknown Source:103)
        at com.google.android.gms.internal.firebase_ml.zzha.zzfb(Unknown Source:75)
        at com.google.android.gms.internal.firebase_ml.zzha.call(Unknown Source:0)
        at com.google.android.gms.internal.firebase_ml.zzgp.zza(Unknown Source:0)
    	... 5 more

When I use Tensorflow Mobile android demo, TF Detect, without changing anything in models
08-21 11:27:01.833 17319-17319/org.tensorflow.demo I/TensorFlowInferenceInterface: Successfully loaded TensorFlow native methods (RunStats error may be ignored)
08-21 11:27:03.391 17319-17319/org.tensorflow.demo I/TensorFlowInferenceInterface: Model load took 1190ms, TensorFlow version: 1.10.0
08-21 11:27:03.392 17319-17319/org.tensorflow.demo I/TensorFlowInferenceInterface: Successfully loaded model from 'file:///android_asset/ssd_mobilenet_v1_android_export.pb'
08-21 11:27:03.394 17319-17319/org.tensorflow.demo I/tensorflow: DetectorActivity: Camera orientation relative to screen canvas: 90
08-21 11:27:03.395 17319-17319/org.tensorflow.demo I/tensorflow: DetectorActivity: Initializing at size 640x480
08-21 11:27:03.400 17319-17319/org.tensorflow.demo E/tensorflow: ObjectTracker: libtensorflow_demo.so not found, tracking unavailable
08-21 11:27:03.400 17319-17319/org.tensorflow.demo I/tensorflow: MultiBoxTracker: Initializing ObjectTracker: 640x480
08-21 11:27:03.400 17319-17319/org.tensorflow.demo E/tensorflow: ObjectTracker: Native object tracking support not found. See tensorflow/examples/android/README.md for details.
08-21 11:27:03.405 17319-17319/org.tensorflow.demo E/tensorflow: MultiBoxTracker: Object tracking support not found. See tensorflow/examples/android/README.md for details.
08-21 11:27:03.405 17319-17319/org.tensorflow.demo I/tensorflow: DetectorActivity: Preparing image 1 for detection in bg thread.

BTW - today the magic happened. I still had the error above while running TF Detect, but it worked! For the first time. For other who faced this problem - this time Android Studio warned be that its going to reinstall APK. After reinstallation it worked
But the error with ml-kit remains :(

@kate-kate Thanks for the update. We pushed some updates to gradle build yesterday.
We will look in to the ML-kit errors next.

@kate-kate ML-Kit has not yet updated to Tensorflow 1.10.
Can you modify your ML Kit to point to Tensorflow Lite 1.10? Here is the link:
https://firebase.google.com/docs/ml-kit/android/use-custom-tflite

Nagging Assignee @achowdhery: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@achowdhery I've tried to follow your advice about ml kit but build didn't complete successfully because of ERROR: /private/var/tmp/_bazel_kate/8be913d8cdea6dd8174f0dac0a48c277/external/flatbuffers/BUILD.bazel:24:1: C++ compilation of rule '@flatbuffers//:flatbuffers' failed (Exit 1)

@achraf-boussaada @tensorflowbutler @umutboz @angersson @petewarden @achowdhery
I get this error:
09-20 18:48:31.364 9726-9726/org.tensorflow.mypackage E/art: No implementation found for long org.tensorflow.contrib.android.RunStats.allocate() (tried Java_org_tensorflow_contrib_android_RunStats_allocate and Java_org_tensorflow_contrib_android_RunStats_allocate__)
09-20 18:48:32.154 9726-9726/org.tensorflow.mypackageE/art: No implementation found for void org.tensorflow.mypackage.tracking.ObjectTracker.initNative(int, int, boolean) (tried Java_org_tensorflow_mypackage_tracking_ObjectTracker_initNative and Java_org_tensorflow_mypackage_tracking_ObjectTracker_initNative__IIZ)
09-20 18:48:32.154 9726-9726/org.tensorflow.mypackageE/AndroidRuntime: FATAL EXCEPTION: main
    Process: org.tensorflow.mypackage, PID: 9726
    java.lang.UnsatisfiedLinkError: No implementation found for void org.tensorflow.mypackage.tracking.ObjectTracker.initNative(int, int, boolean) (tried Java_org_tensorflow_mypackage_tracking_ObjectTracker_initNative and Java_org_tensorflow_mypackage_tracking_ObjectTracker_initNative__IIZ)
        at org.tensorflow.mypackage.tracking.ObjectTracker.initNative(Native Method)
        at org.tensorflow.mypackage.tracking.ObjectTracker.init(ObjectTracker.java:257)
        at org.tensorflow.mypackage.tracking.ObjectTracker.getInstance(ObjectTracker.java:220)
        at org.tensorflow.mypackage.tracking.MultiBoxTracker.onFrame(MultiBoxTracker.java:211)
        at org.tensorflow.mypackage.DetectorActivity.processImage(DetectorActivity.java:254)
        at org.tensorflow.mypackage.CameraActivity.onPreviewFrame(CameraActivity.java:150)
        at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1183)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:158)
        at android.app.ActivityThread.main(ActivityThread.java:7231)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)

Since I have renamed the project org.tensorflow.demo into org.tensorflow.mypackage because I wanted to have more applications running with different models.
The application starts but it crashes because of this error.

Nagging Assignee @achowdhery: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @achowdhery: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @achowdhery: It has been 45 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

did you @achraf-boussaada && @MohammadMoradi  try to right-click on app from Android-Studio -> link Cpp module, choose cmake then point it to tensorflow/examples/android/jni/CMakeLists.txt ?

@NorwayLobster follow this https://github.com/tensorflow/tensorflow/issues/22106#issuecomment-428409506 will solve it

As suggested downloading and including the .so file to your own project was not enough. Since we're including a pre-compiled version of the library the native methods (in the .cpp files) have the package name of the tensorflow demo on their names. That's why this error occur since our own projects packages named differently.

To solve this problem you need to rename your package to org.tensorflow.demo and make sure that ObjectTracker.java is under org.tensorflow.demo.tracking
Second solution would be  changing the naming of the native methods in the .cpp files according to your package name and then rebuild the .so file with bazel (faced many problems there if anyone manage to do it please mention how).

I think I've managed to do that thanks to you. Here are the details:

Changed build to 'cmake'
c files: changed all package #defines to match my custom structure (about 100 cases, use "replace in path")
FROM
#include "tensorflow/examples/android/jni/object_tracking/geom.h"
TO
#include "geom.h"

Now the loading of the ObjectTracker succeeds but running methods are causing exceptions.

Either create and move  ObjectTracker to 'org.tensorflow.demo.tracking' package
Or change the following package under the 'object_tracker_jni.cc'
FROM
Java_org_tensorflow_demo_tracking_ObjectTracker_##METHOD_NAME  // NOLINT
TO
Java_your_package_ObjectTracker_##METHOD_NAME  // NOLINT

@achraf-boussaada thanks for leading me to the right path !

Any updates for 2019 for Tensorflow-lite? The Nightly Build links are now dead.

The nightly build for Tensorflow lite is working appropriately. Please advise what link are you trying.

Nightly Build links are now dead.

Which links in particular? I strongly encourage users to switch to using TensorFlow Lite. The detection sample for that can be found @ https://github.com/tensorflow/examples/blob/master/lite/examples/object_detection/android/README.md.

@achowdhery and @jdduke sorry for the carelessly worded statement, I'm trying to build the Android Demo from source using this link with the hopes of getting Object Tracking in Tensorflow-Lite to work. I'll go through with this process and open a ticket if I encounter any Build Issues.

Nightly Build links are now dead.

Which links in particular? I strongly encourage users to switch to using TensorFlow Lite. The detection sample for that can be found @ https://github.com/tensorflow/examples/blob/master/lite/examples/object_detection/android/README.md.

When I use the above repo I am getting the same error

12142-12142/org.tensorflow.lite.examples.detection I/tensorflow: DetectorActivity: Camera orientation relative to screen canvas: 90
2019-03-27 19:51:02.065 12142-12142/org.tensorflow.lite.examples.detection I/tensorflow: DetectorActivity: Initializing at size 640x480
2019-03-27 19:51:02.072 12142-12142/org.tensorflow.lite.examples.detection E/tensorflow: ObjectTracker: libtensorflow_demo.so not found, tracking unavailable
2019-03-27 19:51:02.073 12142-12142/org.tensorflow.lite.examples.detection I/tensorflow: MultiBoxTracker: Initializing ObjectTracker: 640x480
2019-03-27 19:51:02.073 12142-12142/org.tensorflow.lite.examples.detection E/tensorflow: ObjectTracker: Native object tracking support not found. See tensorflow/examples/android/README.md for details.
2019-03-27 19:51:02.091 12142-12142/org.tensorflow.lite.examples.detection E/tensorflow: MultiBoxTracker: Object tracking support not found. See tensorflow/examples/android/README.md for details.
2019-03-27 19:51:02.092 12142-12142/org.tensorflow.lite.examples.detection I/tensorflow: DetectorActivity: Preparing image 1 for detection in bg thread.
2019-03-27 19:51:02.208 12142-12172/org.tensorflow.lite.examples.detection I/tensorflow: DetectorActivity: Running detection on image 1

And I am not able to find the JNI folder or the libtensorflow_demo.so in any of the folders. If libtensorflow_demo.so is availble can you point me the path? Or is there any procedudre I have to follow to build it?

@achowdhery and @jdduke, I opened a new ticket here

Marking issue as resolved. Feel free to re-open this if it's unresolved or file a new issue

authors: MeghnaNatraj MohammadMoradi SerjSmor Zeit42 achowdhery achraf-boussaada idrissbellil japer21 jdduke kate-kate petewarden sal0w tensorflowbutler umutboz


text:
Is it ok for C++ code to link with _pywrap_tensorflow_internal.so?

I have a Python module written in C++, in which tensorflow serving API is used to load saved model and inference. Unfortunately, some part of my Python application has to use tensorflow python module directly. Because there are some common symbols between libtensorflow_cc.so and _pywrap_tensorflow_internal.so, there will be CHECK failure if the Python module is linked with libtensorflow_cc.so.
The only feasible option seems to link the Python module with _pywrap_tensorflow_internal.so. Is that OK?

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.

@tianyapiaozi did you resolve this issue ? were you able to link to it ?

@tianyapiaozi did you resolve this issue ? were you able to link to it ?

authors: harshini-gadige powderluv tianyapiaozi


text:
tf.contrib.training.HParams domain interval support

Hi, this is my first issue/feature request. I hope it's relevant :)
I am currently working on hyperparameter optimisation and would like to use the HParams API but in my eyes there is a feature missing. Instead of having a discrete list of values for a parameter, I would like to represent a feasible interval, from which I can take values.
Describe the feature and the current behavior/state.
Currently, the HParams class holds a set of hyperparameters as name-value pairs, that is a discrete set of values for each of the hyperparameters. I would like to be able to specify also the feasible interval from which the values are coming. Also the TensorBoard api for reporting experiment summaries provides the possibility to report the domain interval and I think it would be handy to have this information in the HParams class, too.
The TensorBoard API let's you specify the domain_discrete but you could also provide domain_interval.
api_pb2.HParamInfo(name='num_units',
                             display_name='Number of units',
                             type=api_pb2.DATA_TYPE_FLOAT64,
                             domain_discrete=num_units_list_val),

Will this change the current api? How?
Yes, this would require changing the API such that the User can also specify the interval of a parameter, for example:
add_hparam(
    name,
    value,
    start,
    end
)

Furthermore, the data structure would need to be changed to save the interval.
Tensor2Tensor has something similar with RangedHParams.
Who will benefit with this feature?
Anyone working with intervals for hyperparameters compared to a fixed set of discrete values.
System information

TensorFlow version (you are using): r1.12
Are you willing to contribute it (Yes/No): Yes, I am happy to help, but this would be my first contribution and I would need guidance

Hi @moritzmeister !
We see that you are using old version of Tensorflow which is officially considered as end of life, We recommend that you upgrade to 2.6 version and let us know if the issue still persists in newer versions .Please open a new issue in case you face any errors, we will get you the right help .Thanks!

Yes, this is outdated, will close.

authors: mohantym moritzmeister


text:
Restore from SavedModel not compatible with tf.distribute

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Archlinux
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below):  v1.12.0-9492-g2c319fb415 2.0.0-alpha0
Python version: 3.6.0

Describe the current behavior
I'm using Tensorflow Hub to restore a model from a SavedModel, I do expect the restore to work even if it executed inside a distribution strategy scope, but instead it raises an exception.
Describe the expected behavior
It should load the model correctly even it the restore from the savedmodel is performed inside a distribution strategy context.
Code to reproduce the issue
import tensorflow as tf
import tensorflow_hub as hub

strategy = tf.distribute.MirroredStrategy()

with strategy.scope():

    model = tf.keras.Sequential(
        [
            hub.KerasLayer(
                "https://tfhub.dev/google/tf2-preview/inception_v3/feature_vector/2",
                output_shape=[2048],
                trainable=True,
            )
        ]
    )

Other info / logs
The exception:
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow_hub/keras_layer.py", line 98, in __init__
    self._func = module_v2.load(handle)
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow_hub/module_v2.py", line 80, in load
    return tf_v1.saved_model.load_v2(module_handle)
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py", line 324, in load
    export_dir)
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py", line 63, in __init__
    self._setup_functions_captures()
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py", line 100, in _setup_functions_captures
    for node_id in proto.bound_inputs]
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py", line 100, in <listcomp>
    for node_id in proto.bound_inputs]
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow/python/saved_model/load.py", line 117, in _get_tensor_from_node
    return obj.handle
  File "/data/pgaleone/env/lib/python3.6/site-packages/tensorflow/python/distribute/values.py", line 317, in __getattr__
    return getattr(self.get(), name)
AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'handle'

Thanks for the report. @seemuch is looking into SavedModels + distribution strategies.

This works now with the latest 2.0 nightly. Please check out https://www.tensorflow.org/beta/tutorials/distribute/save_and_load for more details.

authors: allenlavoie galeone guptapriya


text:
Keras model to tensorflow trainable pb file and checkpoint

I tried to convert keras h5 model into pb and checkpoint files.
from keras import backend as K
from keras.models import load_model
import tensorflow as tf

model = load_model('model/my_model.h5')

K.set_learning_phase(0) #0 : test, 1 : train

sess = K.get_session()

saver = tf.train.Saver()
saver.save(sess, 'keras/keras.ckpt')

sess.graph.as_default()
graph = sess.graph

with open('keras/keras.pb', 'wb') as f:
    f.write(graph.as_graph_def().SerializeToString())

I succeeded in bringing this pb and checkpoint to tensorflow and running the graph, obtaining an inference.
But the problem is, I have no idea how to train this thing.
Usually, tensorflow models have a training operation, and we just have to sess.run that operation, but I'm assuming keras is working in a different way somehow.
Is it possible to train keras models when they are converted to tensorflow models?

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.

authors: imnotkind ymodak


text:
Inference time by quantized model is longer than that by non-quantized model in Tensorflow

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 1806
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:  ARMv8 AARCH64
TensorFlow installed from (source or binary):  source
TensorFlow version (use command below):  1.12.3
Python version:  2.7.15
Bazel version (if compiling from source):  0.15.0-dist
GCC/Compiler version (if compiling from source):  7.3
CUDA/cuDNN version:  NA
GPU model and memory: NA

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
I want to measure Tensorflow inference time with different models on my 4-core ARMv8 CPU.
I downloaded quantized and float point models from https://www.tensorflow.org/lite/guide/hosted_models. Then run benchmark_model to measure inference time.
Please see attached picture for my benchmarking result.
From the result, you can see the inference time of the quantized models is always a little bit longer than the float point model.
I also did the same benchmark with TFLite, the inference time of the quantized models is much lower as I expect.
Describe the expected behavior
I think the inference time of quantized models should be much lower than that of float point models.
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
The

commands as below:
For quantized models:
echo "inception_v4"
bazel-bin/tensorflow/tools/benchmark/benchmark_model --graph=model/classfication_int8/inception_v4/inception_v4.pb  --input_layer=input --input_layer_type=float --input_layer_shape=1,299,299,3 --output_layer=InceptionV4/Logits/Predictions  --show_run_order=false --num_threads=1 --show_flops
echo "mobilenet_v1_1.0_224"
bazel-bin/tensorflow/tools/benchmark/benchmark_model --graph=model/classfication_int8/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224.pb --show_flops --input_layer=input --input_layer_type=float --input_layer_shape=1,224,224,3 --output_layer=MobilenetV1/Predictions/Reshape_1  --show_run_order=false --num_threads=1

echo "mobilenet_v2_1.0_224"
bazel-bin/tensorflow/tools/benchmark/benchmark_model --graph=model/classfication_int8/mobilenet_v2_1.0_224/mobilenet_v2_1.0_224.pb  --output_layer=output  --show_run_order=false --num_threads=1 --show_flops

For float point models:
echo "inception_v4"
bazel-bin/tensorflow/tools/benchmark/benchmark_model --graph=model/classfication-fp32/inception_v4/inception_v4.pb --show_flops --input_layer=input --input_layer_type=float --input_layer_shape=1,299,299,3 --output_layer=InceptionV4/Logits/Predictions  --show_run_order=false --num_threads=1
echo "mobilenet_v1_1.0_224"
bazel-bin/tensorflow/tools/benchmark/benchmark_model --graph=model/classfication-fp32/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224.pb --show_flops --input_layer=input --input_layer_type=float --input_layer_shape=1,224,224,3 --output_layer=MobilenetV1/Predictions/Reshape_1  --show_run_order=false --num_threads=1

echo "mobilenet_v2_1.0_224"
bazel-bin/tensorflow/tools/benchmark/benchmark_model --graph=model/classfication-fp32/mobilenet_v2_1.0_224/mobilenet_v2_1.0_224.pb --show_flops --input_layer=input --input_layer_type=float  --output_layer=MobilenetV2/Predictions/Reshape_1  --show_run_order=false --num_threads=1

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Hi,
This is because the included models are not fully quantized for TensorFlow, but instead Quantization-aware trained model. This means they are float, but have FakeQuantWithMinMaxVars operations emulating the effect of quantization during training. The intention of these models is to emulate the effects of quantization in TensorFlow graphs so that they can be easily converted to TensorFlow Lite models.
Thanks!

authors: ZhangShuoAlreadyExists suharshs