text:
Any adversarial attack that sustains after resize attack

Hi,
This is Bala. I have a query regarding adversarial attack.
Is there any adversarial attack that sustains/consists of added noise, after resize attack ? (adversarial image -> converting into High / low resolution image -> resize to original adverarial image size)
Thanks,
Bala

Hi Bala, you mean attack or defense? I don't quite follow your question.

Hi Sir,
Sorry for  late reply. I was out of work for last few days.
Thanks for your reply. I need a attack which is robust after resize attack.
I am expecting like below:
target noise added to image of size 28x28 -> resize to 65x75 -> resize to
28x28 and target label should available or sustain.
Tested the process and observed like below:
target noise added to image of size 28x28 -> resize to 65x75 -> resize to
28x28 and observed that target label disturbed (not available).
I checked with PGD and BIM attacks using foolbox. the added noise is
removed or disturbed.
please let me know your suggestions.
On Mon, Dec 16, 2019 at 8:17 PM Zhitao Gong notifications@github.com
wrote:

Hi Bala, you mean attack or defense? I don't quite follow your question.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/gongzhitaao/tensorflow-adversarial/issues/11?email_source=notifications&email_token=ALTLTFXUVAXW4URY4E3N623QY6IG7A5CNFSM4J3IDDF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG66CEI#issuecomment-566092049,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALTLTFRN3IVXVE5TRIOR2F3QY6IG7ANCNFSM4J3IDDFQ
.

So if I understand it correctly, you want an attack that survives the resizing, right?
The resizing attack is a bit confusing, it should be resizing defense?
As far as I know, resizing is not an effective method to defend against the adversarial images.  It will lower the attack success rate, but will not solve the problem.  Basically many of the adversarial examples are still adversarial even after resizing.  Some of the early papers, (e.g., FGSM) on adversarial examples have related results.
Hope this helps.

Hi @gongzhitaao , What do you think about advface[1] or amora[2]. These adversarial attacks changing a few pixels in image. So, I think these methods more vulnerable to resize operation. What do you think about this ?
1: https://arxiv.org/abs/1908.05008
2: https://arxiv.org/abs/1912.03829

Hey @mhmddpkts, I haven't read the papers yet.  Sorry I'm not working on adversarial attack/defense now (it was long time ago), so my opinions might be outdated. :smile:

When I search this problem in google, I found this page. Therefore, I asked you. Anyway, thanks your reply @gongzhitaao

repo: gongzhitaao/tensorflow-adversarial

license: mit

license_source: bigquery


text:
graph/formats/rdf: add a query language

Please take a look.

Codecov Report

Merging #1765 (cc377fa) into master (d4eca1b) will decrease coverage by 0.14%.
The diff coverage is 75.83%.

:exclamation: Current head cc377fa differs from pull request most recent head 47f426e. Consider uploading reports for the commit 47f426e to get more accurate results

@@            Coverage Diff             @@
##           master    #1765      +/-   ##
==========================================
- Coverage   73.37%   73.23%   -0.15%
==========================================
  Files         520      504      -16
  Lines       61361    61028     -333
==========================================
- Hits        45021    44691     -330
- Misses      13806    13807       +1
+ Partials     2534     2530       -4

Impacted Files
Coverage Δ

graph/formats/rdf/rdf.go
79.62% <ø> (+1.23%)
:arrow_up:

graph/formats/rdf/graph.go
70.96% <70.96%> (ø)

graph/formats/rdf/query.go
88.88% <88.88%> (ø)

internal/asm/c64/conj.go
0.00% <0.00%> (-100.00%)
:arrow_down:

graph/coloring/coloring.go
89.72% <0.00%> (-0.61%)
:arrow_down:

internal/asm/f32/stubs_noasm.go

internal/asm/f64/axpy.go

mat/index_no_bound_checks.go

...fourier/internal/fftpack/array_no_bounds_checks.go

graph/iterator/lines_map_safe.go

... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4eca1b...47f426e. Read the comment docs.

source: gharchive/pull-request

authors: codecov-commenter kortschak

repo: gonum/gonum

license_type: permissive


text:
Unable to build LiteRT for Android

bazel build -c opt --cxxopt=--std=c++17 --config=android_arm64
--fat_apk_cpu=x86,x86_64,arm64-v8a,armeabi-v7a
--define=android_dexmerger_tool=d8_dexmerger
--define=android_incremental_dexing_tool=d8_dexbuilder
//tensorflow/lite/java:tensorflow-lite
Starting local Bazel server and connecting to it...
INFO: Reading 'startup' options from /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc: --windows_enable_symlinks
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=123
INFO: Reading rc options for 'build' from /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc:
'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility --config=v2 --config=litert_prefixes
INFO: Found applicable config definition build:short_logs in file /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:litert_prefixes in file /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc: --define=litert_prefixes=external/org_tensorflow/
INFO: Found applicable config definition build:android_arm64 in file /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc: --config=android --cpu=arm64-v8a --fat_apk_cpu=arm64-v8a
INFO: Found applicable config definition build:android in file /Users/wang/IdeProjects/tflite-ai/client/LiteRT/.bazelrc: --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --dynamic_mode=off --define=xnn_enable_avxvnniint8=false --noenable_platform_specific_config --copt=-w --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --define=with_xla_support=false
DEBUG: /private/var/tmp/wang/c286a78069d91d2d2c206403e296a20d/external/local_tsl/third_party/py/python_repo.bzl:98:14:
HERMETIC_PYTHON_VERSION variable was not set correctly, using default version.
Python 3.11 will be used.
To select Python version, either set HERMETIC_PYTHON_VERSION env variable in
your shell:
export HERMETIC_PYTHON_VERSION=3.12
OR pass it as an argument to bazel command directly or inside your .bazelrc
file:
--repo_env=HERMETIC_PYTHON_VERSION=3.12
DEBUG: /private/var/tmp/wang/c286a78069d91d2d2c206403e296a20d/external/local_tsl/third_party/py/python_repo.bzl:109:10: Using hermetic Python 3.11
ERROR: Skipping '//tensorflow/lite/java:tensorflow-lite': no such package 'tensorflow/lite/java': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.

/Users/wang/IdeProjects/tflite-ai/client/LiteRT/tensorflow/lite/java
WARNING: Target pattern parsing failed.
ERROR: no such package 'tensorflow/lite/java': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
/Users/wang/IdeProjects/tflite-ai/client/LiteRT/tensorflow/lite/java
INFO: Elapsed time: 2.220s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)

Is there any tutorial for building LiteRT for Android?

source: gharchive/issue

repo: google-ai-edge/LiteRT

license_source: github-api


text:
When trying to create a Task Bundle using a TFLite file, I'm not allowed to enter the stop token of the model

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
No
OS Platform and Distribution
Linux Ubuntu 16.04
Mobile device if the issue happens on mobile device
No response
Browser and version if the issue happens on browser
No response
Programming Language and version
Python
MediaPipe version
No response
Bazel version
No response
Solution
LLM Inference
Android Studio, NDK, SDK versions (if issue is related to building in Android environment)
No response
Xcode & Tulsi version (if issue is related to building for iOS)
No response
Describe the actual behavior
I created .tflite file using ai-edge-torch for Llama 3.2 1B model and now am trying to deploy it for inference on edge. When trying to create the task bundle, the stop token is asked. When I provide "<|end_of_text|>", it is not able to resolve it. I previously converted the tokenizer to the SentencePiece format through the code given in the ai-edge-torch repository.
Describe the expected behaviour
The task bundle should be created without errors.
Standalone code/steps you may have used to try to get what you need
I tried to manually check the possible tokens the model could identify using its vocab and "<|end_of_text|>" is a token in its vocab.
I also tried changing the stop token and the task bundle was created. However, on using the bundle for deployment, I was getting a Failed to initialize engine : modelError building tflite model. Also, just as a side question, the .task file that's created, can it be used interchangeably with the .bin file that's given in the model path in the repository examples?
Other info / Complete Logs
No response

Hi @Arya-Hari,
Could you please share the complete example you are using from our documentation? Additionally, if you have any error logs, sharing them would help us better understand the issue.
Thank you!!

@kuaashish Hello...is there way to resolve this?

Thanks for all of the detail provided. Two quick items:

re: .task vs. .bin: Yes, you can use the .task wherever you would use a .bin file. The .task extension indicates that the file is a "converted TF Lite model + metadata/tokenizer"
I noticed in your provided script that you have this line:

tflite_model="/content/gemma_2b_quantized.tflite" # @param {type:"string"}

Is this just a copy/paste error? I assumed you would have something like llama3_1_1b_quantized.tflite, not a Gemma model.

Hi @talumbau. To clarify, I used the quantization script provided in the AI Edge Torch repository for quantizing and converting it to the TFLite format. The script used there, by default, saves the output file under the name of gemma_2b_quantized.tflite. I forgot to change it before using it. But, I changed everything else from the script to work for Llama 3.2 1B instead. Sorry for the confusion.

Hi @Arya-Hari,
Could you please confirm if this issue is resolved or any further assistance is needed?
Thank you!!

authors: Arya-Hari kuaashish talumbau

repo: google-ai-edge/mediapipe


text:
Sceneform Objects when resolved from Cloud Anchors are not placed on the surface.

I am hosting multiple nodes using Cloud Anchors and I am able to resolve them too. But the problem is that when we resolve the nodes they are not perfectly placed on the surface. They keep floating in the air and the position of the nodes are slightly shifted up to 2-3 meters.

Are you setting the position of the nodes by setting the world position, or the local position relative to the cloud anchor?

Local position relative to cloud anchor.

Can you share a screenshot of the problem?
To confirm what you are doing:
On the hosting side, you are placing an anchor (is it on a plane or an oriented point or image)?
Capturing the local positions of the children of the anchorNode.
Then on the resolving side you resolve the anchor successfully, retrieve the local positions from the hosting side, and position the child nodes of the anchorNode based on the resolved anchor by calling setLocalPosition().

@claywilkinson Yes this is exactly what i am doing.

I will try to send a proper screenshot in which the problem can be understood easily.

Closing due to inactivity.

@dsternfeld7 I would like to follow up this issue since I'm experiencing the same. These are the steps I follow:
To save in Firebase
`
val frame = mArFragment.arSceneView.arFrame
val anchor = hitResult.createAnchor()
val imageNode = Node()
imageNode.renderable = myRenderable
val anchorNode = AnchorNode(anchor)
imageNode.setParent(anchorNode)
fragment.arSceneView.scene.addChild(anchorNode)
val cloudAnchor = mSession?.hostCloudAnchor(anchor)
mCloudAnchor?.let {
//if cloudAnchor.cloudAnchorState is Anchor.CloudAnchorState.SUCCESS
//Then store cloudId AND the local position
storeInTheCloud(cloudAnchor.cloudAnchorId, anchorNode.localPosition)
}
`
Retrieving  from Firebase
`
val vector3 = // it's stored in Firebase with the given cloudId
val anchorFromCloud = mSession?.resolveCloudAnchor(key)
val imageNode = Node()
imageNode.renderable = myCustomRenderable
val anchorNode = AnchorNode(anchorFromCloud)
imageNode.setParent(anchorNode)
anchorNode.localPosition = vector3
fragment.arSceneView.scene.addChild(anchorNode)
`
If I resolve from the cloud the anchor just holding the phone at the same position as I stored it, it pretty much looks like the original, just a slight hovering over the ground.
If I go somewhere else in the room, the resolved object it's either floating in the room or looking really small and in a different place other than the original.
In either case I apply any scaling or re-sizing etc.
Let me know if you need more info.

Attached here are the images, original and resolved (from another point in the room).

Reopening this to look at it.

In either case I apply any scaling or re-sizing etc.

@luislukas just to clarify, you do or don't apply any scaling or resizing?

Hi @abramyan I do not apply any scaling or resizing. I just resolve the cloud anchor and add the Renderable there.

Just reading the ArCore documentation on cloud anchors, seems that the following recommendations might be the cause of the behaviour I experience:

Hosting
Before hosting an anchor:
Try to look at the anchor from different angles.
Move around the anchor for at least a few seconds.
Make sure you are not too far away from the anchor.
Resolving
When resolving an anchor:
Make sure you are close to where the anchor was hosted.
If you are too far away from a cloud anchor’s position, it might resolve incorrectly.
A Cloud Anchor might resolve incorrectly if your device camera is pointed at a space that is different from but visually identical to where the Cloud Anchors were hosted.

@luislukas Did the recommendation from the documentation solve the issue?

@abramyan following the recommendations does solve the issue, ie. if you create and store a cloud anchor and resolve it in the same position it is displayed well. However, If you move, let's say, 3 or 4 metres away from where you created the anchor including a change in the camera position, it keeps behaving wrongly.
My use case involves creating and storing cloud anchors within a room and then resolving them from people entering the room for example, without knowing exactly where the anchors were created. I would expect that people coming into the room would see the resolved objects in the same position at least, so that they could walk in and explore the area, come closer to the object and see it.

Hi everyone, any update in 2019?

The ArCore team has been paying attention to this and has been incrementally improving it.  Current versions of ArCore resolve cloud anchors much closer now.   This may continue to improve but we are not tracking it as a bug.  Closing this for now since they have marked it resolved internally.

authors: abramyan claywilkinson dsternfeld7 luislukas malik-at-work praveenbbhati rohitagarwal3011

repo: google-ar/sceneform-android-sdk


text:
Android Basics: Lunch Tray app

URL of codelab
In which task and step of the codelab can this issue be found?
Task 4 "Test your app"
Run OrderFunctionalityTests
Describe the problem
The provided code has a test where it clicks on a radio button. The comment says "Select the potato item", but the code below performs a click on "pickles". The price of pickles is different from the price of potatoes, so when it checks the string, it's different throwing an error.
Steps to reproduce?

A completed lunch tray app.
Run the test "OrderFunctionalityTests"
Function radio_buttons_update_accompaniment_menu_subtotal will throw an error.

// Select the potato item
 onView(withId(R.id.pickles)).perform(click())
 onView(withId(R.id.subtotal))
     .check(matches(withText(containsString("Subtotal: $0.50"))))

Get's this error:

androidx.test.espresso.base.DefaultFailureHandler$AssertionFailedWithCauseError: 'with text: a string containing "Subtotal: $0.50"' doesn't match the selected view.
Expected: with text: a string containing "Subtotal: $0.50"
Got: "TextView{id=2131231151, res-name=subtotal, visibility=VISIBLE, width=342, height=67, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=androidx.constraintlayout.widget.ConstraintLayout$LayoutParams@eea6adc, tag=null, root-is-layout-requested=false, has-input-connection=false, x=694.0, y=813.0, text=Subtotal: $0.00, input-type=0, ime-target=false, has-links=false}"
at dalvik.system.VMStack.getThreadStackTrace(Native Method)
at java.lang.Thread.getStackTrace(Thread.java:1724)
at androidx.test.espresso.base.DefaultFailureHandler.getUserFriendlyError(DefaultFailureHandler.java:16)
at androidx.test.espresso.base.DefaultFailureHandler.handle(DefaultFailureHandler.java:36)
at androidx.test.espresso.ViewInteraction.waitForAndHandleInteractionResults(ViewInteraction.java:103)
at androidx.test.espresso.ViewInteraction.check(ViewInteraction.java:31)
at com.example.lunchtray.OrderFunctionalityTests.radio_buttons_update_accompaniment_menu_subtotal(OrderFunctionalityTests.kt:120)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:154)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:395)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2248)
Caused by: junit.framework.AssertionFailedError: 'with text: a string containing "Subtotal: $0.50"' doesn't match the selected view.
Expected: with text: a string containing "Subtotal: $0.50"
Got: "TextView{id=2131231151, res-name=subtotal, visibility=VISIBLE, width=342, height=67, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=androidx.constraintlayout.widget.ConstraintLayout$LayoutParams@eea6adc, tag=null, root-is-layout-requested=false, has-input-connection=false, x=694.0, y=813.0, text=Subtotal: $0.00, input-type=0, ime-target=false, has-links=false}"
at androidx.test.espresso.matcher.ViewMatchers.assertThat(ViewMatchers.java:17)
at androidx.test.espresso.assertion.ViewAssertions$MatchesViewAssertion.check(ViewAssertions.java:15)
at androidx.test.espresso.ViewInteraction$SingleExecutionViewAssertion.check(ViewInteraction.java:10)
at androidx.test.espresso.ViewInteraction$2.call(ViewInteraction.java:11)
at androidx.test.espresso.ViewInteraction$2.call(ViewInteraction.java:2)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loopOnce(Looper.java:201)
at android.os.Looper.loop(Looper.java:288)
at android.app.ActivityThread.main(ActivityThread.java:7839)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003)
Versions
Android Studio version: Android Studio Bumblebee | 2021.1.1 Patch 2
Build #AI-211.7628.21.2111.8193401, built on February 17, 2022
Runtime version: 11.0.11+0-b60-7772763 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 12.3
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 8
Registry: external.system.auto.import.disabled=true
Non-Bundled Plugins: org.jetbrains.kotlin (211-1.6.10-release-923-AS7442.40), org.intellij.plugins.markdown (211.7142.37)
API version of the emulator:
31
Additional information
New to this, sorry If I'm wrong. Or did something wrong.

I experienced the same issue when completing this CodeLab, however I don't think it's due to the price of Potato and Pickles being different. The comment in the test file about Potatoes is likely just an oversight while copying/pasting. You'll notice that the test as written is checking for a value of $0.50, which is the cost of Pickles in DataSource.kt. However the test results you pasted indicates the subtotal is showing $0.00, which is also what happened to me.
Did you try to run the radio_buttons_update_accompaniment_menu_subtotal test by itself? The test passed for me when run individually, but failed when I ran the entire OrderFunctionalityTests file. I also verified manually that the code is working as expected. I don't have enough experience with Android testing yet to troubleshoot why I'm receiving inconsistent results.

authors: ethan-mcc lukegferguson


text:
Write automated tests: Android Basics with Compose

I get an unresolved reference when trying to use composeTestRule in the calculate_20_percent_tip()  function.
package com.example.tiptime

import androidx.compose.ui.test.junit4.createComposeRule
import com.example.tiptime.ui.theme.TipTimeTheme
import org.junit.Rule
import org.junit.Test
import java.text.NumberFormat

class TipUITests {
    @get:Rule
    var composeTestRule = createComposeRule()
}
@Test
fun calculate_20_percent_tip() {
    composeTestRule.setContent {
        TipTimeTheme {
            TipTimeScreen()
        }
    }
    composeTestRule.onNodeWithText("Bill Amount")
        .performTextInput("10")
    composeTestRule.onNodeWithText("Tip (%)").performTextInput("20")
    val expectedTip = NumberFormat.getCurrencyInstance().format(2)
    composeTestRule.onNodeWithText("Tip Amount: $expectedTip").assertExists(
        "No node with this text was found."
    )
}

My mistake, should have been a method in the class.


text:
implement_gsam_jax

Implement GSAM algorithm proposed in Surrogate gap minimization improves sharpness-aware training, ICLR 2022, which is an improvement over SAM (Sharpness-Aware Minimization)
When config.rho_max == config.rho_min and config.alpha=0.0, the  GSAM algorithm reduces to SAM.

Hi,
Thank you for contribution. As stated in the readme, we normally do not accept external contributions, but we are happy to make an exception for open-source implementations of published projects developed in big_vision.
However, according to the codebase principles, project-specific code should not add complexity to the core library parts, such as the main train loop. Thus, standalone projects are expected to fork the main train loop into big_vision/trainers/proj/<project name>/... and apply necessary modifications there. We plan to submit an example of how this works soon (~2 weeks from now). Maybe you wait for the example, and then update this pull request accordingly?

Thanks a lot for the clarification! I will re-format and re-submit later according to the examples.

hey, we now have an example of a project-specific trainer here: https://github.com/google-research/big_vision/tree/main/big_vision/trainers/proj/distill
If you are still interested in submitting gsam (we would like it!), could you sync to head and instead of modifying the core train.py, fork it into trainers/proj/gsam/train.py and do the modifications there?
Sorry for the delay on our side!

Thanks a lot for the example! I have moved all changes to big_vision/trainers/proj/gsam, please let me know if it looks good.

Also, once you're done, could you squash all the commits into just a single one?

authors: akolesnikoff juntang-zhuang lucasb-eyer


text:
Would you like to release the code about how to train a bigbird with other language

@manzilz I want to train a bigbird with other language

Hi Ryan
Can you be more specific in what is challenge you are facing in training bigbird with other languages? For example if you want to train a masked LM, we have provided the pretraining code in bigbird/pretrain/. You should be able to use it to train for any language. You might want to use your vocabulary specific to the language.
Thanks!

Hi Ryan
Can you be more specific in what is challenge you are facing in training bigbird with other languages? For example if you want to train a masked LM, we have provided the pretraining code in bigbird/pretrain/. You should be able to use it to train for any language. You might want to use your vocabulary specific to the language.
Thanks!

@manzilz the pretrain script did not provide a tpu training method?

@manzilz the pretrain script did not provide a tpu training method?

authors: RyanHuangNLP manzilz

repo: google-research/bigbird


text:
Please provide a way to change the evaluation frequency

The evaluation frequency is set to 5000 steps and I can't find a way to change it.

Hi @agaitanis,
Please check the proto (e.g., this one)
Cheers,

This worked:
evaluator_options {
eval_interval: 2000
}
Thanks

authors: agaitanis aquariusjay


text:
Is there anyway to set up a customized scenario without a Goalkeeper?

Thanks for your great work! When I'm working on setting up customized scenes, it seems that each team needs to have a player with the role "e_PlayerRole_GK", otherwise the game won't start.
I'm wondering, is it possible to set up a customized scene without any GK?

Hi, yes, goalkeeper is required, but you can position him outside of the field and mark as lazy:
https://github.com/google-research/football/blob/75e88e49c552f3e096ae4054a1dd1a7ccbfb837b/gfootball/env/scenario_builder.py#L98

authors: Mohan-Zhang-u qstanczyk

repo: google-research/football


text:
About the Synthetic Repetition

Hi @debidatta，i'm trying to figure out how the period length labels are generated when training RepNet, . As it described in the paper,"Then, we sample a clip C of random length P frames from V. This clip C is repeated K times (where K > 1) to simulate videos with repetitions."Is there anything worth noting when choosing the length of clip C and doing the inversion and splicing to ensure good training results? For example, the way of generating random numbers and the choice of the number of repetitions, I hope your answer.

Yes you should ensure that the chosen segment gets repeated more than once. For example if you chose a segment of 32 frames or less it can repeat at least 2 times for the video length of 64. If you choose a longer segment it cannot repeat at least 2 times.

Yes you should ensure that the chosen segment gets repeated more than once. For example if you chose a segment of 32 frames or less it can repeat at least 2 times for the video length of 64. If you choose a longer segment it cannot repeat at least 2 times.

Thank you @debidatta, I got that. I have two other doubts, is it necessary to ensure that the number of duplicate clips of each synthetic video  matches some distribution between 2~32 to prevent overfitting, or  randint() has been sufficient? The second one is whether a large number of training sets are required for training? I want to synthesize the data on a training set of number about 800, roughly how many training samples should be generated， should i use a video for more than one time?

repo: google-research/google-research


text:
kws_streaming.models.utils_test failure

I noticed that utils.model_to_tflite() is throwing errors with the GRU model. kws_streaming.models.utils_test fails as well with the following error. I observed this with tf2.1.0 as well as tf2.2.0-dev20200402.
ERROR: test_model_to_tflite (__main__.UtilsTest)
test_model_to_tflite (__main__.UtilsTest)
TFLite supports stateless graphs.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abhipray/NeoSensory/git-repos/google-research/kws_streaming/models/utils_test.py", line 85, in test_model_to_tflite
    self.assertTrue(utils.model_to_tflite(self.sess, self.model, self.flags))
  File "/Users/abhipray/NeoSensory/git-repos/google-research/kws_streaming/models/utils.py", line 295, in model_to_tflite
    sess, model_stateless_stream.inputs, model_stateless_stream.outputs)
  File "/anaconda3/lib/python3.7/site-packages/tensorflow/lite/python/lite.py", line 792, in from_session
    graph_def = _freeze_graph(sess, input_tensors, output_tensors)
  File "/anaconda3/lib/python3.7/site-packages/tensorflow/lite/python/util.py", line 256, in freeze_graph
    graph_def, input_tensors, output_tensors, config, graph=sess.graph)
  File "/anaconda3/lib/python3.7/site-packages/tensorflow/lite/python/util.py", line 218, in run_graph_optimizations
    return tf_optimizer.OptimizeGraph(config, meta_graph)
  File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/grappler/tf_optimizer.py", line 58, in OptimizeGraph
    graph_id, strip_default_attributes)
ValueError: Failed to import metagraph, check error log for more info.

----------------------------------------------------------------------
Ran 5 tests in 2.938s```

Hi Abhipray,
Please let me know how do you test GRU model and configure GRU testing parameters?
To validate GRU model I run below scripts with env:
Python 3.6.9
tf.version '2.1.0-rc1'
path where kws_streaming lib is located
KWS_PATH=/tmp/kws_streaming
path where data2 are unpacked
DATA_PATH=$KWS_PATH/data2
path where models2 are unpacked
MODELS_PATH=$KWS_PATH/models2
python -m kws_streaming.train.model_train_eval
--data_url ''
--data_dir $DATA_PATH/
--train_dir $MODELS_PATH/gru/
--mel_upper_edge_hertz 7000
--how_many_training_steps 10000,10000,10000
--learning_rate 0.0005,0.0001,0.00002
--window_size_ms 40.0
--window_stride_ms 20.0
--mel_num_bins 40
--dct_num_features 20
--resample 0.15
--alsologtostderr
--train 0
gru
--gru_units 400
--return_sequences 0
--dropout1 0.1
--units1 128,256
--act1 "'linear','relu'"
It should start prinitng something like:
tflite test accuracy, non stream model = 94.53% 200 out of 4890
tflite test accuracy, non stream model = 96.51% 400 out of 4890
tflite test accuracy, non stream model = 95.67% 600 out of 4890
etc..
After you run above script, please let me know if you see an error again?
Before running above script, I would also try
export CUDA_VISIBLE_DEVICES=-1
to validate that issue is related to running TFLite on CPU or GPU
Thanks,
Oleg.

Hi Oleg,
I ran python -m kws_streaming.models.utils_test as it is and it fails with the error message I shared.  The default script tests dnn model. I got the test to pass by disabling eager execution:
if __name__ == '__main__':
    # tf.compat.v1.enable_resource_variables()
    disable_eager_execution()
    tf.test.main()

I am using tf2.1 and python 3.7.1 and have no GPUs.
I ran the same script you shared and see no errors output from running kws_streaming.train.model_train_eval.
My ultimate goal is to build a streaming GRU model with external state from this library and convert it to tflite, however these errors in convert_to_tflite() show up.

I see, in my tests tfv2 was running with disabled eager mode by default, that is why I did not see this error. Goog catch! Tomorrow a patch (with disabled eager execution) will be submitted for all unit tests.
If you are building a key words detector or sounds detector with GRU model then you can try:
python -m kws_streaming.train.model_train_eval
--data_url ''
--data_dir $DATA_PATH/
--train_dir $MODELS_PATH/gru/
--mel_upper_edge_hertz 7000
--how_many_training_steps 400,400,400
--learning_rate 0.0005,0.0001,0.00002
--window_size_ms 40.0
--window_stride_ms 20.0
--mel_num_bins 40
--dct_num_features 20
--resample 0.15
--alsologtostderr
--train 1
gru
--gru_units 400
--return_sequences 0
--dropout1 0.1
--units1 128,256
--act1 "'linear','relu'"
I reduced number of training iterations to 400,400,400 just for test purpose.
This script will:

train GRU model and store it to /tf
Convert TF non streaming model to TFLite and store it to /tflite_non_stream
Convert TF non streaming model to TF streaming one (with external states) then convert it to TFLite and store it to tflite_stream_state_external
4 It can also can do a TFLite model quantization (if you need to reduce model size and make it faster)

The third step is what you are interested.
/Oleg.

Hi Abhipray,
All unit tests are working.
Also I added more tests to demonstrate how gru model is converted to streaming with TFlite at testToStreamInferenceModeTFandTFLite(self, model_name='gru'):. You can try any other models by changing model_name (some models can not be streamed: 'ds_cnn_stride', 'cnn_stride', 'att_rnn', 'att_mh_rnn').
Please confirm and close the issue.
Best,
Oleg.

Hi Oleg,
Thank you for taking a look at the unittests. They all pass for me now. Unfortunately, I still get errors when I work with the functions in kws_streaming outside the context of model_train_eval.py.
Here is a colab showing the error from utils.model_to_tflite(). https://colab.research.google.com/drive/1tIreYkJ5mw1xDbmd7MHMtyRbH26GC4-H
I had some other questions:

Why not use the tf2.0 tf.lite converter from_keras() method? https://www.tensorflow.org/api_docs/python/tf/lite/TFLiteConverter?version=nightly#from_keras_model
Why does your tf lite streaming implementation only support external states? Is there something inherently problematic with internal states and tf lite? I made a simple attempt at converting a keras model with internal state and ran into other converter issues like this one https://github.com/tensorflow/tensorflow/issues/37659.

Please find comments below:
Why not use the tf2.0 tf.lite converter from_keras() method?
It will limit converter to Keras models only. I tried to keep it as generic as possible, so that you can build your model in pure TF or TF.Keras and converter still will work in both cases.
Why does your tf lite streaming implementation only support external states? Is there something inherently problematic with internal states and tf lite?
I implemented kws_streaming in a way that you can convert TF to TF streaming with internal states or to TF streaming with internal states. So both streaming modes are supported in this lib.
But TFlite does not support stateful layers now, that is why I test only streaming TFlite with external states (you can build special custom ops with states for TFlite, but I would avoid this path because you will have to customize every stateful layer - it does not scale). TFlite team is working on supporting internal states as general solution and as soon it is enabled, stateful streaming will work out of of the box with this lib.
I tried your colab and it did not work for me (I do not cotroll all env there so it is hard to say why it does not work - it looks like there are some issues with installed TF version).
Please try attached jupyter notebok - it works ok (here all env set up is controlled by yourself):
cd /tmp/kws_streaming
virtualenv --system-site-packages -p python3 ./venv3
source ./venv3/bin/activate
pip install tensorflow==2.1.0-rc1
jupyter notebook
Untitled.zip

Hi Oleg,
I suspect there might be some other setting on your machine that is different from mine and google colab (https://colab.research.google.com/drive/1tIreYkJ5mw1xDbmd7MHMtyRbH26GC4-H). I followed your steps with virtualenv on my local machine and google colab and get the same error:
WARNING:root:FAILED to convert to mode NON_STREAM_INFERENCE, tflite: Fetch argument <tf.Variable 'gru/cell/kernel:0' shape=(20, 96) dtype=float32> cannot be interpreted as a Tensor. (Tensor Tensor("gru/cell/kernel/Read/ReadVariableOp:0", shape=(20, 96), dtype=float32) is not an element of this graph.)```

Ah I found the issue: the model needs to be instantiated after resetting the graph. Thanks for your help, Oleg!

Good point :)
Just in case here is a working colab

I found another issue, possible memory-related with the tflite conversion. Here is an example GRU model I am working with:

If input shape is [1,1,20] and the model has three GRU layers with 2 units, the keras model.predict() and tf.lite interpreter outputs are the same. If however, I change the input shape to [1,1,40], I get NaN output from tflite interpreter. If I leave the input shape as [1,1,40] but use 2 layers with 2 nodes, I get an incorrect output (sometimes 0, 1 or NaN) with the tflite interpreter.
Here is a colab showing the issue: https://colab.research.google.com/drive/1QC73kZgx6yG7af10-RtWIjrWWLFjOcXc

I am not sure about what are you trying to achieve in your colab.
You do not run keras model in streaming mode, so it is not correct to compare tf non streaming with TFLite streaming models.
If you just need to convert your model to TFLite then you should specify
mode=Modes.NON_STREAM_INFERENCE
as it is done in this colab - the output should be correct on any feature size and model depth.
Also there is no need to use TimeDistributed wrapper - Dense layer is applied on last dimension in time any way, as it is show in dense_test
Please close this issue and create another one, because the last one is not related to kws_streaming.models.utils_test failure and can be confusing for other people (it is a good practice not to mix different issues in one ticket).
In the new issue, I will add another colab which explain the difference between non streaming and streaming models in TF and TFLite with gru example.
Thank you for exploring this lib, I appreciate your feedbacks!

Thanks for the pointers! My problem was that I was not initializing the state inputs to zero before running invoke.

Here is a colab with example of running:
TF in non streaming
TFlite non streaming
TF streaming with internal state
TF streaming with external state
TFLite with external state
Please have a look at graphs of streaming vs non streaming models in above colab.
It demonstrates a difference between them: streaming with external state has additional inputs for states which you have to intialize
inference.zip
and control during inference.
You already figured out your issue :), so hope above colab will be helpful for the next step: running TFLite model in streaming mode.

authors: Abhipray rybakov


text:
[OWL-ViT] Finetuning l14 on own dataset

Hello, is it possible to finetune owl-vit14 on my own dataset? I saw a script for finetuning 32 version, but I really need l14. Also, what format of the dataset should be there to finetune? How do I put my data (images, texts, boxes) in the correct format?

@25icecreamflavors hey :)
Could you please share the 32 version fine-tuning script?

repo: google-research/scenic


text:
[EN] Feedback for: /web/tools/polymer-starter-kit/?hl=en

This link not working:
https://elements.polymer-project.org/
please , check

@gerardoemmanuel what do you mean by "it's not working" ? are you talking about this link https://elements.polymer-project.org/web/tools/polymer-starter-kit/?hl=en ?

Seems to be working now.

authors: gerardoemmanuel manekinekko petele

repo: google/WebFundamentals


text:
Add "Latest contributions from X" page

Before going any further (tests, etc), I'd like to know if you would accept this PR. It basically creates a "Latest contributions from X" page for each contributor accessible at https://developers.google.com/web/resources/contributors/X

Hey @beaufortfrancois - love the idea. Looks like it fails integration tests. Can you take a look? Locally you should be able to run gulp test to get the results. LMK if the results you get are different than what the system finds.

Try it live at https://pr-4973-dot-web-central.appspot.com/web/resources/contributors/beaufortfrancois/

@petele I've fixed tests. WDYT?

Oh - can you also update the gulp clean task in gulpfile.js to clean the new contributors folder.

@petele Here it is!

Thank you!
https://web-central.appspot.com/web/resources/contributors/beaufortfrancois

authors: beaufortfrancois petele


text:
add "manual stage" checklist item to PR template

What's changed, or what was fixed?

adds a checklist item to this PR template in order to prevent mess-ups such as https://github.com/google/WebFundamentals/pull/5486

Target Live Date: 2017-12-19

[ ] This has been reviewed and approved by @petele
[x] I have run gulp test locally and all tests pass.
[x] I have added the appropriate type-something label.

R: @petele

:+1:

authors: WebFundBot kaycebasques


text:
[devtools] add "get started with the console"

What's changed, or what was fixed?

adds a new tutorial

Fixes: N/A
Target Live Date: 2018-03-15

[x] This has been reviewed and approved by @kaycebasques
[x] I have run gulp test locally and all tests pass.
[x] I have added the appropriate type-something label.
[x] I've staged the site and manually verified that my content displays correctly.

CC: @petele

Whoops!
There were 8 warnings that will prevent this PR from being merged. Please take a look, and either fix, or provide a justification for why they can't be fixed.
WARNINGS
src/content/en/tools/chrome-devtools/console/get-started.md#L16 - '

:tada: This has been pushed live to https://developers.google.com/web/


text:
[Navigation Material] Add skipHalfExpanded parameter with default value equal to false.

Hey,
I added skipHalfExpanded boolean to the parameters of rememberBottomSheetNavigator function, because I think it is a popular use case to make it skip half expanded state as a default behaviour, especially when it is treated as separate navigation destination.

Hi, thanks for the PR. This is a change we're consciously not making as we rather want to support a destination-level flag. Due to ongoing work in Material's ModalBottomSheetLayout I don't believe we can make this work yet though. Feel free to prototype a destination-level flag, I might be wrong :)
The concerns are about recreating the state in time when the value changes as the destination changes, but maybe it'd work. We'd want some good test coverage that tests

Navigating from a skipHalfExpanded = false to a skipHalfExpanded = false destination
Navigating from a skipHalfExpanded = false to a skipHalfExpanded = true destination
Navigating from a skipHalfExpanded = true to a skipHalfExpanded = false destination
and ensures that the sheet opens up to the correct state.

Let me know if you'd be interested in contributing this change!

Awesome, thank you!

Hey, there hasn't been any progress on this since 1 year, can I go ahead and create a new pr?

@amanv8060 Yes go on please, I unfortunetley was not able to find time to adress it properly :(

With the release of Compose Material 1.7.0-alpha04, the Material team has added a new artifact: androidx.compose.material:material-navigation, which fully replaces Accompanist Navigation Material.
As such, we are closing all PRs here on Accompanist.

authors: amanv8060 ianhanniballake jossiwolf jstarczewski

repo: google/accompanist


text:
Add CodeQL to GitHub CI Action (fixes #2185)

Fixes #2185 (hopefully, we'll see).
This is an alternative approach to #2204 & #2205 and the earlier #2027.

Abandoning this in favour of #2207, which is the same but on a (much) larger build VM; because this one got "stuck" (23 minutes and counting).

repo: google/android-fhir


text:
Add runtime parameter to limit test in realtime

This change introduces the 'run-time-ms' parameter which dictates how long to run a bigwheels application for before automatically quitting. It also cleans up a little bit around the use of timestamps and MaxFrames.

Ah, sorry, forgot to make the updates a separate CL :/. Hopefully it's small enough to be obvious...

This change introduces the 'run-time-ms' parameter which dictates how long to run a bigwheels application for before automatically quitting. It also cleans up a little bit around the use of timestamps and MaxFrames.

What is the motivation for this PR?
Time is quite unreliable for automated testing, meaning this seems to be for manual usage?
If it's for manual usage is it to replace a ctrl-c? (or a ./vk_01_sample & sleep 1 && pkill vk_01_sample)

Hmm... is time unreliable for automated testing? My desire is to limit the test time to avoid situations where a bad system image causes framerates that are remarkably slower than before. The argument can be used in conjunction with frame count to basically be a worst-case timeout if needed.
To your point, we could have a command line option, but I much prefer a clean shutdown to guarantee (per my other PR) that metrics are written out at exit.

Hmm... is time unreliable for automated testing?

Usually yes, since even the same exact binary will have different results if using time somewhere. (That's why we added the --deterministic flag, which even replaces the time function with a getFrameIndex equivalent.
I thought the motivation was to always have something you have time to look at, even if it's running at 20000fps.

My desire is to limit the test time to avoid situations where a bad system image causes framerates that are remarkably slower than before.

Isn't that the role of the test harness?
Using frame count means you can measure over time how long it took to run 100 frames by ex.
(same can be done the other way around, but frame-count is already available).
Then, the test harness/driver/runner can add an additional timeout, if a frame freezes by ex, to kill a test after N seconds.
Using the time-based timeout like you did means if the frame never completes (for any reason), this timeout won't kick either, since it's handled in the main thread. While a test runner could handle that case.
Am I missing something?

Right, but having a time cutoff for testing is pretty nominal, right?

Sure, wasn't quite convinced of the added value since we already have frame-count which is roughly the same assuming you don't freeze.
It's just that frame-count is more absolute than time.
With frame-count, you can easily separate the warmup frame from the rest by ex.
Time is messier.

this is a method of guaranteeing the application can still e.g. dump data after the completion of the test.

The situation where I expect it to hit the time limit in-app would be performance degradation

Still not convinced of the usefulness TBH. But I won't block the PR on my sole opinion, so if you want to get this merge, we can move forward.
If you use frame-count, and report runtime, you can still track performance over time. Also, I'd expect a performance benchmark to use the metrics framework, and ignore the first frames (or separate it from the actual run).
Then, from the reports, track the variation overtime, and determine if there is a regression either by the delta, or by tracking evolution over time on dashboard (for small, accumulating regressions being under the noise threshold).
The whole logic to link time to frames seems to be the job of the test runner, using the exported metrics etc.
Cutting the application with the implemented time limit means cutting at an arbitrary moment, after an arbitrary number of frames, and seems IMO a bit weird.
Once again, if you are convinced this solves your usecase, we can go ahead. Just want to make sure this is not a "redundant" option vs the frame-count.

Right, but having a time cutoff for testing is pretty nominal, right?

Sure, wasn't quite convinced of the added value since we already have frame-count which is roughly the same assuming you don't freeze. It's just that frame-count is more absolute than time. With frame-count, you can easily separate the warmup frame from the rest by ex. Time is messier.

this is a method of guaranteeing the application can still e.g. dump data after the completion of the test.

The situation where I expect it to hit the time limit in-app would be performance degradation

Still not convinced of the usefulness TBH. But I won't block the PR on my sole opinion, so if you want to get this merge, we can move forward.
If you use frame-count, and report runtime, you can still track performance over time. Also, I'd expect a performance benchmark to use the metrics framework, and ignore the first frames (or separate it from the actual run). Then, from the reports, track the variation overtime, and determine if there is a regression either by the delta, or by tracking evolution over time on dashboard (for small, accumulating regressions being under the noise threshold).
The whole logic to link time to frames seems to be the job of the test runner, using the exported metrics etc. Cutting the application with the implemented time limit means cutting at an arbitrary moment, after an arbitrary number of frames, and seems IMO a bit weird.
Once again, if you are convinced this solves your usecase, we can go ahead. Just want to make sure this is not a "redundant" option vs the frame-count.

I'm confident this won't be redundant, but I'm happy to talk through it sometime if you're still not convinced. I agree with your point about avoiding the first few frames worth of data, though I think there's a longer discussion to be had about the metrics we capture regardless.

authors: Keenuts Slumpwuffle

repo: google/bigwheels


text:
Failed to load monoreil sample plugin

Hi,
whenever i tried to load monoreil sample plugin, i got failure with below console output.
Reason: java.lang.VerifyError: java.lang.VerifyError: (class: org/python/proxies/__builtin__$SkeletonLatticeElement$3, method: super__equals signature: (Lcom/google/security/zynamics/reil/algorithms/mono/interfaces/ILatticeElementMono1;)Z) Illegal use of nonvirtual function call in <script> at line number 53

Could you please let me know how to solve this problem?
Thank you.
J.H. Jeong

Environment

Binnavi version : 6.01
OS : MAC OS X 10.11.2

Hey there,
I'll have to look into this (which may take a few days), but my gut feeling
says it may have something to do
with the obfuscation of the yfiles part ...
Sorry for the likely delay.
Cheers,
Thomas
2016-04-04 7:52 GMT+08:00 Jeong, JiHoon notifications@github.com:

Hi,
whenever i tried to load monoreil sample plugin, i got failure with below
console output.
Reason: java.lang.VerifyError: java.lang.VerifyError: (class: org/python/proxies/builtin$SkeletonLatticeElement$3, method: super__equals signature: (Lcom/google/security/zynamics/reil/algorithms/mono/interfaces/ILatticeElementMono1;)Z) Illegal use of nonvirtual function call in

Thank you for your response.
I have one more thing. In your cansecwest2009 slide(page 49), you mentioned some of monoreil example such as register tracking and negative array indexing.
I could find register tracking example in the script directory but I couldn't find negative array indexing.
Could you release this example so that I can refer to them?
Thank you for your help.
J.H. Jeong

@thomasdullien :
I am also interested in the "negative array indexing" example. Any news on this?
@binoopang:
Does the bug also exist in the latest official release (6.1) ?

Hey there,
regarding the "negative array indexing" example: I think we will safely
have to assume it has been
lost to the ages. Honestly, I think this may be for the better.
It may be a good idea to try to build a proper memory domain into BinNavi
(e.g. something that manages
contents of memory cells) in the near future...
Cheers,
Thomas
2016-05-24 9:26 GMT+02:00 gitttt notifications@github.com:

@thomasdullien https://github.com/thomasdullien :
I am also interested in the "negative array indexing" example. Any news on
this?
@binoopang https://github.com/binoopang:
Does the bug also exist in the latest official release (6.1) ?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/google/binnavi/issues/87#issuecomment-221187599

Hi!
I am having a similar (the same?) error when trying to start the mono reil sample.
I removed as much code as possible to narrow down the error. This code still causes it:
class SkeletonLatticeElement(ILatticeElement):
	def __init__(self):
		self.written_registers = Set()

	def equals(self, rhs):
		pass

If I rename the "equals", e.g. to "equal", then the error is gone.
Here the stack trace (the cmd output) of the error:
D:\jython2.7.0\bin>jython.exe d:\temp\sample.py
Traceback (most recent call last):
  File "d:\temp\sample.py", line 48, in <module>
    class SkeletonLatticeElement(ILatticeElement):
java.lang.VerifyError: (class: org/python/proxies/__main__$SkeletonLatticeElemen
t$0, method: super__equals signature: (Lcom/google/security/zynamics/reil/algori
thms/mono/interfaces/ILatticeElementMono1;)Z) Illegal use of nonvirtual function
 call
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.privateGetPublicMethods(Unknown Source)
        at java.lang.Class.getMethods(Unknown Source)
        at org.python.core.PyJavaType.init(PyJavaType.java:284)
        at org.python.core.PyType.createType(PyType.java:1412)
        at org.python.core.PyType.addFromClass(PyType.java:1348)
        at org.python.core.PyType.fromClass(PyType.java:1439)
        at org.python.core.PyType.setupProxy(PyType.java:685)
        at org.python.core.PyType.newType(PyType.java:212)
        at org.python.core.PyType.type___new__(PyType.java:152)
        at org.python.core.PyType$exposed___new__.new_impl(Unknown Source)
        at org.python.core.PyType.invokeNew(PyType.java:494)
        at org.python.core.PyType.type___call__(PyType.java:1706)
        at org.python.core.PyType.__call__(PyType.java:1696)
        at org.python.core.PyObject.__call__(PyObject.java:496)
        at org.python.core.Py.makeClass(Py.java:1878)
        at org.python.core.Py.makeClass(Py.java:1839)
        at org.python.core.Py.makeClass(Py.java:1831)
        at org.python.pycode._pyx0.f$0(d:\temp\sample.py:107)
        at org.python.pycode._pyx0.call_function(d:\temp\sample.py)
        at org.python.core.PyTableCode.call(PyTableCode.java:167)
        at org.python.core.PyCode.call(PyCode.java:18)
        at org.python.core.Py.runCode(Py.java:1386)
        at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:296
)
        at org.python.util.jython.run(jython.java:362)
        at org.python.util.jython.main(jython.java:142)

java.lang.VerifyError: java.lang.VerifyError: (class: org/python/proxies/__main_
_$SkeletonLatticeElement$0, method: super__equals signature: (Lcom/google/securi
ty/zynamics/reil/algorithms/mono/interfaces/ILatticeElementMono1;)Z) Illegal use
 of nonvirtual function call

Hey there,
I won't be able to look into this until next Friday :-/ sorry for that.
Will check it then!
2016-10-15 20:54 GMT+02:00 gitttt notifications@github.com:

Hi!
I am having a similar (the same?) error when trying to start the mono reil
sample.
I removed as much code as possible to narrow down the error. This code
still causes it:
class SkeletonLatticeElement(ILatticeElement):
def init(self):
self.written_registers = Set()
def equals(self, rhs):
    pass

If I rename the "equals", e.g. to "equal", then the error is gone.
Here the stack trace (the cmd output) of the error:
D:\jython2.7.0\bin>jython.exe d:\temp\sample.py
Traceback (most recent call last):
File "d:\temp\sample.py", line 48, in
class SkeletonLatticeElement(ILatticeElement):
java.lang.VerifyError: (class: org/python/proxies/main$SkeletonLatticeElemen
t$0, method: super__equals signature: (Lcom/google/security/zynamics/reil/algori
thms/mono/interfaces/ILatticeElementMono1;)Z) Illegal use of nonvirtual function
call
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
at java.lang.Class.privateGetPublicMethods(Unknown Source)
at java.lang.Class.getMethods(Unknown Source)
at org.python.core.PyJavaType.init(PyJavaType.java:284)
at org.python.core.PyType.createType(PyType.java:1412)
at org.python.core.PyType.addFromClass(PyType.java:1348)
at org.python.core.PyType.fromClass(PyType.java:1439)
at org.python.core.PyType.setupProxy(PyType.java:685)
at org.python.core.PyType.newType(PyType.java:212)
at org.python.core.PyType.type___new__(PyType.java:152)
at org.python.core.PyType$exposed___new__.new_impl(Unknown Source)
at org.python.core.PyType.invokeNew(PyType.java:494)
at org.python.core.PyType.type___call__(PyType.java:1706)
at org.python.core.PyType.call(PyType.java:1696)
at org.python.core.PyObject.call(PyObject.java:496)
at org.python.core.Py.makeClass(Py.java:1878)
at org.python.core.Py.makeClass(Py.java:1839)
at org.python.core.Py.makeClass(Py.java:1831)
at org.python.pycode._pyx0.f$0(d:\temp\sample.py:107)
at org.python.pycode._pyx0.call_function(d:\temp\sample.py)
at org.python.core.PyTableCode.call(PyTableCode.java:167)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.Py.runCode(Py.java:1386)
at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:296
)
at org.python.util.jython.run(jython.java:362)
at org.python.util.jython.main(jython.java:142)
java.lang.VerifyError: java.lang.VerifyError: (class: org/python/proxies/_main
_$SkeletonLatticeElement$0, method: super__equals signature: (Lcom/google/securi
ty/zynamics/reil/algorithms/mono/interfaces/ILatticeElementMono1;)Z) Illegal use
of nonvirtual function call
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/binnavi/issues/87#issuecomment-254003778, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEYBvG86xXMZ4V4eNACU5wtsrN-duUmwks5q0SFwgaJpZM4H-tJ6
.

The BinNavi project is no longer under active development.

authors: binoopang cblichmann ghost gitttt thomasdullien

repo: google/binnavi


text:
Close trash on toolbox open

Fixes #302

This change is

Rebase this on master without ec31f58a15a121db827f9a402588074b31fd56f5.  Thanks.

repo: google/blockly-android


text:
Adding token and grid

As discussed in #48 and #53
Creates a <Token> that supports animation when changing x and y. It also receives a hidden function from Grid that gets the underlying svg coordinates.

Coverage remained the same at 100.0% when pulling 82e2fab108ab25cb5034c006829bb57b051fd2b6 on Felizardo:token into 022b6d266fcf8015a789c6fd270adc25b050465c on google:master.

Coverage remained the same at 100.0% when pulling 259e82e64112cbc52f620017b3b54bae098c0856 on Felizardo:token into 022b6d266fcf8015a789c6fd270adc25b050465c on google:master.

Coverage remained the same at 100.0% when pulling 259e82e64112cbc52f620017b3b54bae098c0856 on Felizardo:token into 022b6d266fcf8015a789c6fd270adc25b050465c on google:master.

Looks great @Felizardo :)

authors: Felizardo coveralls nicolodavis

license: MIT


text:
Logical Assignment Operators [Feature Request]

Would you please add logical assignment operators as the proposal is in stage 4:
a ||= b;
a &&= b;
a ??= b;

https://github.com/tc39/proposal-logical-assignment

Thanks for the report, this does look like a very useful ES2021 feature.
(Full list is at https://github.com/tc39/proposals/blob/master/finished-proposals.md)

Thanks for the report, this does look like a very useful ES2021 feature.
(Full list is at https://github.com/tc39/proposals/blob/master/finished-proposals.md)

yes, i currently use this syntax for singletons:
class Singleton {
  static instance() {
    return this.instance ?? (this.instance = new this());
  }
}

which can be replaced by:
return this.instance ??= new this();

yes, i currently use this syntax for singletons:
class Singleton {
  static instance() {
    return this.instance ?? (this.instance = new this());
  }
}

which can be replaced by:
return this.instance ??= new this();

authors: blickly rahbari

repo: google/closure-compiler


text:
1.6.0 release does not have Windows version asset

I've tried checking the 1.6.0 release content.
https://github.com/google/filament/releases/tag/v1.6.0
However, I don't think Windows asset is included.

It’s still building :)

Thanks! I confirmed that filament-20200502-windows.tgz was built.

repo: google/filament


text:
Updating primitives names

Fixes https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr/issues/1

Renaming all primitives with its new name except resistors.

I have reviewed all changes and I approve them.


text:
question: can this example be done more cleanly?

In the data structure I wanted to compare, there was a field of type I, where I is an interface. Types A and B both implemented I and nothing else did. I wanted to convert As to Bs to facilitate comparison. (A is a deprecated version of B, and I wanted to treat them equivalently for comparisons.)
My first thought was a transformer from A to B. This failed, because cmp.Equal first checks that its two arguments have identical type before transforming.
I ended up writing a comparer func(x, y I) bool that had to convert each of x and y before comparing them. (Original code).
Is there a better way?

func (I, I) bool seems to be the best approach, since you can't have a transformer occur on just one of the elements and not the other. The rationale for that was to ensure the path to each element was identical for both sides being compared. Otherwise, it would be very confusing how to print differences to the user.

authors: dsnet jba

repo: google/go-cmp


text:
getting "json: unsupported type: func() (io.ReadCloser, error)" on DeleteHook

As the title says, I get json: unsupported type: func() (io.ReadCloser, error) as part of the response (not the error) of the DeleteHook function.
So here is a fragment of my code:
	response, err := git.Repositories.DeleteHook(ctx, m.gitWebhook.Spec.RepositoryOwner, m.gitWebhook.Spec.RepositoryName, *hook.ID)
	log.V(1).Info("DeleteHook", "response", response, "err", err)

the output is:
[manager] 1.6689558493826857e+09	DEBUG	DeleteHook	{"controller": "gitwebhook", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "GitWebhook", "GitWebhook": {"name":"gitwebhook-github","namespace":"test-gitwebhook-github"}, "namespace": "test-gitwebhook-github", "name": "gitwebhook-github", "reconcileID": "a9d29c20-49e0-490c-8dc9-25a17e2947f9", "responseError": "json: unsupported type: func() (io.ReadCloser, error)", "err": null}

the same client works with the other hook related functions. Looking around it seems that this error is due to not setting the content type correctly in the headers. I'm testing with the latest version and against the public instance of github.
Thanks for the help.

Thanks for the report, @raffaelespazzoli !
It might help to use https://github.com/gmlewis/go-httpdebug to debug this.

this is what I got:
[manager]   https://api.github.com/repos/raffaelespazzoli/helloworld-tonic/hooks/389245677 \
[manager]   -H 'Accept: application/vnd.github.v3+json' \
[manager]   -H 'Authorization: <REDACTED>' \
[manager]   -H 'User-Agent: go-github/v48.0.0'
[manager] 1.6689576382338324e+09	DEBUG	DeleteHook	{"controller": "gitwebhook", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "GitWebhook", "GitWebhook": {"name":"gitwebhook-github","namespace":"test-gitwebhook-github"}, "namespace": "test-gitwebhook-github", "name": "gitwebhook-github", "reconcileID": "e5f04e83-79f6-4181-9bd1-256868fdce7f", "responseError": "json: unsupported type: func() (io.ReadCloser, error)", "err": null}

looks good to me and yet the error...

What do you get when you call directly with curl?

is there a way for this library to print the curl command, it looks like so, but the above is all I get...

That is exactly what the other repo is doing.
There was a discussion that it would be better to keep that helper function outside of this repo.

yeah that's what I meant the other repo, anyway it gives me enough to build the curl command.

when  execute with curl, I get the following:
{"message":"Not Found","documentation_url":"https://docs.github.com/rest/webhooks/repos#delete-a-repository-webhook"}

ok, so I had some issue in my code. everything works now. that weird response message remains unexplained, but the delete actually works. closing....

authors: gmlewis raffaelespazzoli

repo: google/go-github


text:
Is there a plan to support the legacy SEV report handling?

Hi there,
In addition to SEV-SNP, the legacy SEV also produces a guest report, even though it's coming from the hypervisor. The report from the legacy SEV can be obtained from ATTESTATION ioctl from the hypervisor, and it contains a signature from PEK. Is there a plan to support the validate and verify for the legacy SEV report in this project?

No, legacy attestation is out of scope.

authors: cowbon deeglaze

repo: google/go-sev-guest


text:
Adsense batch requests not working if use multiple metrics

The batch works great if I insert only one metric, but adding more than one, I always get no rows from the api.
    $this->client->setUseBatch(true);

    $service = new Google_Service_AdSense($this->client);
    $batch = $service->createBatch();

    $metrics= ['EARNINGS','CLICKS'];
    $dimensions = ['DATE'];
    $accountId = 'myaccount';

    $optParams = array(
      'metric' => $metrics,
      'dimension' => $dimensions
    );

    $req1 = $service->accounts_reports->generate( $accountId,
                                                  '2017-04-01',
                                                  '2017-04-20',
                                                  $optParams);

    $req2 = $service->accounts_reports->generate( $accountId,
                                                  '2017-03-01',
                                                  '2017-03-20',
                                                  $optParams);

    $batch->add($req1, "thoreau");
    $batch->add($req2, "shaw");

    $results = $batch->execute();

After some tests came out that laravel have embedded an old version of library (1.1.8). Updating it with composer solved the problem.

repo: google/google-api-php-client-services


text:
Updated to PHPUnit 5.3.*

I've update PHPUnit to 5.3.*, and dropped support to PHP versions 5.4 and 5.5, as PHPUnit 5 requires PHP 5.6.
Why dropped support for PHP versions 5.4 and 5.5?
These versions are no longer supported by PHP.
Why PHPUnit 5.3 and not higher?
Because in 5.4, getMock method was deprecated, and we will need to work on that.

Thank you for your Pull Request! Unfortunately, even though PHP 5.4 and PHP 5.5 are no longer supported, we still have users on those versions. So unless there is a good reason to update it (a better reason than upgrading our PHPUnit version), we are going to continue supporting these versions of PHP.

authors: bshaffer gabriel-caruso


text:
Tolerate null style definitions in project config in local db

We don't have string schema enforcement for remote data, so being more laxed about nullness checking in local db as well.

@scolsen Gentle ping!

repo: google/ground-android


text:
Fixed Tile Providers

[x] Revert problematic MapBox changes
[ ] Ensure all available tiles are rendered on the map.

@gino-m I'm having some trouble with this. Maybe you can steer me in the right direction?
I'm getting tons of SQL corruption errors whenever we attempt to render tiles. Something's going wrong, but I'm not sure what yet. It could be any of these issues:

Tiles are marked as Downloaded too early resulting in incomplete data
MapBoxOfflineTileProviders need to be closed in order not to leak resources--I'm currently not doing this since I'm not sure where exactly we'd do it? On map/fragment destruction?
There's some underlying issue with the source data? (least likely).

Anyway, I'm bashing my head against this one, so let me know if you have any ideas!

Hey Scott!
Did you basically revert back the MapBoxTileProvider to its original state? I'd recommend starting there.
Re destroying TileProviders, not sure you need to do that, I believe they get removed when the Map is destroyed.
Lastly, we should keep in mind that Tile is a misnomer in our model; really their MapBox tile sources; each Tile represents an mbtiles file, which contains many tiles (cmiww).
Please lmk if you're using the default MapBoxTileProvider and we can go from there!
Gino

Also, are you working with an an area we know has completed imagery? Some files may have been corrupt in the current test data source.

Hey Scott!
Did you basically revert back the MapBoxTileProvider to its original state [exactly]?

Yep. Should be equivalent to the original source right now except for the try/catches I added for the corruption errors (the original just crashes outright on a DB corruption).

Re destroying TileProviders, not sure you need to do that, I believe they get removed when the Map is destroyed.

That's correct--what I meant was, according to the MapBoxOfflineTileProvider we need to explicitly close the underlying SQLite DB:
/**
     * Closes the provider, cleaning up any background resources.
     *
     * <p>
     * You must call {@link #close()} when you are finished using an instance of
     * this provider. Failing to do so may leak resources, such as the backing
     * SQLiteDatabase.
     * </p>
     */
    @Override
    public void close() {
        if (this.mDatabase != null) {
            this.mDatabase.close();
            this.mDatabase = null;
        }
    }

Lastly, we should keep in mind that Tile is a misnomer in our model; really their MapBox tile sources; each Tile represents an mbtiles file, which contains many tiles (cmiww).

You're right! I've been trying to use names like mbtiles--maybe we should rename the Tile entity to something like TileSet or some such

Also, are you working with an an area we know has completed imagery? Some files may have been corrupt in the current test data source.

I'm not sure about the integrity of the data source, unfortunately—but I can say with confidence this is the first time that I've encountered these corruption errors with this source, and we'd had this working in the past.

@gino-m It's ready for a look now. I still run into some DB corruptions, and some mbtiles files that are downloaded are only 16kb in size, which is suspicious, but afaict everything should be correct

@gino-m ready for another look whenever

authors: gino-m scolsen


text:
Memory leak for GRPC Binary

I am using Firebase for an iOS app, running on XCode 15.0.1 with Swift 5.9.
I am using the latest version of Firebase (10.17.0), and when I debug memory leaks on my app, I see this:
1 instance of grpc_core::(anonymous namespace)::NativeClientChannelDNSResolver::Request leaked

We have a memory leak in the grpc-core library. It looks similar to this issue. I believe the problem is that this library is downloading grpc version 1.49.1 (as we can see here). I think we should update the dependencies inside Package.swift to be able to fix this issue.

Thanks for the report. It's not feasible for us to support Firestore working with any grpc minor version since adaptation is required for most grpc minor releases because it does not yet honor semantic versioning with Apple package managers.
If you don't need to support iOS 12, using 10.15.0 could be a workaround.  (In 10.16.0, we backported Firebase to 1.49.1 to work around a gRPC crash on iOS 12.)

Thank you!

Closing. See https://github.com/firebase/firebase-ios-sdk/issues/12098#issuecomment-1992732149 for context

repo: google/grpc-binary


text:
Fix the bug of globalCmpFeedback

Fixed a crash when obtaining coverage information due to improper use of globalCmpFeedback, causing honggfuzz to generate invalid test cases

Ok i will resubmit the code later

Ok i will resubmit the code later

repo: google/honggfuzz


text:
Support validating HTML at runtime

Inserting/replacing malformed HTML through Incremental DOM can cause really nasty bugs. Would it be possible to perform some loose validation on the HTML being inserted to detect major errors? Doing this at runtime will be more versatile than what we could do via static analysis or at compilation time.
Probably want it optional since it could be expensive parsing HTML constantly and browsers don't expose any sort of API for doing this natively AFAIK.

We do check that tags are correctly nested in debug mode. Unfortunately, the error can show up in weird places. For example, if you forget to close a tag and everything else is correctly nested, you will only find out at the end of a patch that you did something wrong, so you have to hunt down where you forgot to close a tag.
For the Closure Templates compilation, it would be nice to add a pass that enforces closing tags within any content block ({let}, {if}, {template}), etc, but there are a couple use cases where enforcing this is not possible. I do have the code for it, just need to figure out what is the best way to actually use it.

authors: dbarnett sparhami

repo: google/incremental-dom


text:
This app won't run unless you update Google Play services

When trying to run the app I m prompted to update the google play service , upon clicking on update I don't see any update option. The same thing happen on my Nexus4 which is not he case when I install the Google I/O app directly from play store.
How can this be fixed ?

This type of message is always coming in my Moto G(First Generation) after updating it to android 5.0.2 and after clicking it, it  go to Google Play Services (Google Ink) with Uninstall and Open option. After this I can not understand that how to fix this problem, because there is no other option on the screen to update an application. So that I request you to kindly guide me.

There was a problem when this bug was first posted related to the client library that has been fixed.
The same error message presents itself when the following occurs and isn't typically considered a bug (but we are working to make this experience better for users and devs):
When you flash a device or receive an OTA the embedded Google Play services APK may have been replaced with an older version of the services APK. The client library that is compiled into the app is a newer version that needs to talk to the newer services APK. Google Play services will update itself from Google Play Store in the background and this error will eventually go away. In an OTA or new device setup where you restored your device from an online backup the Google Play services APK can get stuck in the back of a large queue. Rebooting your phone will typically refresh the Google Play Store app update list if you have nothing listed to download.

Can someone please help I have a next book 7.85 tablet and the the same thing happens

Then this happens when I click on it

I have tried everything for about 3-4 months and still nothing my tablet used to be rooted and then this happened when I unroot it. When I try root again with the same method using kingo root it says waiting for device then my tablet turns off. When I turn it on it then it turns back off. I have tried clearing cache but idk how to remove my account from the tablet. When I go to gmail it says google services needs to be updated to work but when I click on it, it says google play services is not compatible with your device and it used to be.

Please someone help I beg I can't take this anymore and I'm only 12 yrs old. I messed with the build.prop but no I can't edit it because I can't root please I beg a lot

I had also faced this type of notification on my Moto G and now I overcome
on this. First you please go to setting then click on apps after this click
on Google Play Services. Funally you should click uninstall updates
of Google Play Services. After doing it, this type of notification will not
appeare in future.
On Mon, Mar 2, 2015 at 8:21 AM, zubairkharoti notifications@github.com
wrote:

Please someone help I beg I can't take this anymore and I'm only 12 yrs
old. I messed with the build.prop but no I can't edit it because I can't
root please I beg a lot
--
Reply to this email directly or view it on GitHub
https://github.com/google/iosched/issues/37#issuecomment-76651385.

--

Yeah I have tried that but didn't work I can't get any app

Hi @zubairkharoti, This issue is for the Google I/O Scheduling app. It does have advice for developers of other apps that run into the same issue, but debugging a rooted device's issue is sort of out of scope. Between rooting, unrooting, not being able to root again, and modifying build.prop I think you might have gotten to a bad enough state to warrant starting back at stock. I suggest you use the support forum for the tool you used to root. Thanks, Paul

I know this is old but i encounted this error due to me turning off androids permissions in security, users, As soon as i allowed android permission it fixed the problem.

The currently public version of the app doesn't target Android M so you would have received a warning when you disabled the permissions indicating the app may not work correctly. The app uses Google Play services and if you do things to your device that result the device not receiving updates to Google Play services this issue will present itself.

yep, finally rooted my tab and couldnt figure out why it wouldnt update, I caused no end of dramas turning that permission off. Ended up factory resetting the thing

The same problem is happen to me from 2 days..please someone help what to do now..I have HTC  e9 plus

This happened to me when I installed a new CM ROM. Just go to Apk download site and get the latest version of Google Play Services to install. If you check I'm sure you'll find the current version you run is probably a few versions older. Mine fixed it self btw but I was able to spot the version difference before it did.

i've the same problem with xiaomi mi4c after i reset it to the reset factory, but i haven't installed the google app.
anybody can fix my problem?

could try https://github.com/opengapps/opengapps/wiki/FAQ

authors: 00001jay DPARJUNAHI DevenCris PaulRashidi VihaanVerma89 mathiastck nisfuf parvezkarim zubairkharoti

repo: google/iosched


text:
LinalgBufferizePass doesn't handle tensor.cast well

After running the pass the flow.dispatch.store is deleted :
This happens in https://github.com/google/iree/pull/5362 if the FoldTensorCastIntoDispatchTensorStore wasn't applied
see:
// *** IR Dump Before mlir::iree_compiler::(anonymous namespace)::LinalgBufferizePass ***
func @large_dot_general_dispatch_0() {
  %c0 = constant 0 : index
  %cst = constant 0.000000e+00 : f32
  %c4 = constant 4 : index
  %c64 = constant 64 : index
  %c1 = constant 1 : index
  %c32 = constant 32 : index
  %0 = hal.interface.binding.subspan @io::@s0b0_ro_external[%c0] : !flow.dispatch.tensor<readonly:4x32x1024xf32>
  %1 = hal.interface.binding.subspan @io::@s0b1_ro_external[%c0] : !flow.dispatch.tensor<readonly:4x1024x64xf32>
  %2 = hal.interface.binding.subspan @io::@s0b2_xw_external[%c0] : !flow.dispatch.tensor<writeonly:4x32x64xf32>
  %workgroup_id_x = hal.interface.workgroup.id[0] : index
  %workgroup_count_x = hal.interface.workgroup.count[0] : index
  %workgroup_id_y = hal.interface.workgroup.id[1] : index
  %workgroup_count_y = hal.interface.workgroup.count[1] : index
  %workgroup_id_z = hal.interface.workgroup.id[2] : index
  %workgroup_count_z = hal.interface.workgroup.count[2] : index
  scf.for %arg0 = %workgroup_id_z to %c4 step %workgroup_count_z {
    %3 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_id_y]
    %4 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_count_y]
    scf.for %arg1 = %3 to %c32 step %4 {
      %5 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_id_x]
      %6 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_count_x]
      scf.for %arg2 = %5 to %c64 step %6 {
        %7 = flow.dispatch.tensor.load %0, offsets = [%arg0, %arg1, 0], sizes = [%c1, %c32, 1024], strides = [1, 1, 1] : !flow.dispatch.tensor<readonly:4x32x1024xf32> -> tensor<?x?x1024xf32>
        %8 = flow.dispatch.tensor.load %1, offsets = [%arg0, 0, %arg2], sizes = [%c1, 1024, %c32], strides = [1, 1, 1] : !flow.dispatch.tensor<readonly:4x1024x64xf32> -> tensor<?x1024x?xf32>
        %9 = linalg.init_tensor [1, 32, 32] : tensor<1x32x32xf32>
        %10 = linalg.fill(%9, %cst) {__internal_linalg_transform__ = "workgroup"} : tensor<1x32x32xf32>, f32 -> tensor<1x32x32xf32>
        %11 = linalg.batch_matmul {__internal_linalg_transform__ = "workgroup", is_root_op} ins(%7, %8 : tensor<?x?x1024xf32>, tensor<?x1024x?xf32>) outs(%10 : tensor<1x32x32xf32>) -> tensor<1x32x32xf32>
        %12 = tensor.cast %11 : tensor<1x32x32xf32> to tensor<?x?x?xf32>
        flow.dispatch.tensor.store %12, %2, offsets = [%arg0, %arg1, %arg2], sizes = [%c1, %c32, %c32], strides = [1, 1, 1] : tensor<?x?x?xf32> -> !flow.dispatch.tensor<writeonly:4x32x64xf32>
      }
    }
  }
  return
}

// *** IR Dump After mlir::iree_compiler::(anonymous namespace)::LinalgBufferizePass ***
func @large_dot_general_dispatch_0() {
  %c0 = constant 0 : index
  %cst = constant 0.000000e+00 : f32
  %c4 = constant 4 : index
  %c64 = constant 64 : index
  %c1 = constant 1 : index
  %c32 = constant 32 : index
  %0 = hal.interface.binding.subspan @io::@s0b0_ro_external[%c0] : memref<4x32x1024xf32>
  %1 = hal.interface.binding.subspan @io::@s0b0_ro_external[%c0] : !flow.dispatch.tensor<readonly:4x32x1024xf32>
  %2 = hal.interface.binding.subspan @io::@s0b1_ro_external[%c0] : memref<4x1024x64xf32>
  %3 = hal.interface.binding.subspan @io::@s0b1_ro_external[%c0] : !flow.dispatch.tensor<readonly:4x1024x64xf32>
  %4 = hal.interface.binding.subspan @io::@s0b2_xw_external[%c0] : memref<4x32x64xf32>
  %5 = hal.interface.binding.subspan @io::@s0b2_xw_external[%c0] : !flow.dispatch.tensor<writeonly:4x32x64xf32>
  %workgroup_id_x = hal.interface.workgroup.id[0] : index
  %workgroup_count_x = hal.interface.workgroup.count[0] : index
  %workgroup_id_y = hal.interface.workgroup.id[1] : index
  %workgroup_count_y = hal.interface.workgroup.count[1] : index
  %workgroup_id_z = hal.interface.workgroup.id[2] : index
  %workgroup_count_z = hal.interface.workgroup.count[2] : index
  scf.for %arg0 = %workgroup_id_z to %c4 step %workgroup_count_z {
    %6 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_id_y]
    %7 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_count_y]
    scf.for %arg1 = %6 to %c32 step %7 {
      %8 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_id_x]
      %9 = affine.apply affine_map<()[s0] -> (s0 * 32)>()[%workgroup_count_x]
      scf.for %arg2 = %8 to %c64 step %9 {
        %10 = memref.subview %4[%arg0, %arg1, %arg2] [%c1, %c32, %c32] [1, 1, 1] : memref<4x32x64xf32> to memref<?x?x?xf32, affine_map<(d0, d1, d2)[s0] -> (d0 * 2048 + s0 + d1 * 64 + d2)>>
        %11 = memref.subview %0[%arg0, %arg1, 0] [%c1, %c32, 1024] [1, 1, 1] : memref<4x32x1024xf32> to memref<?x?x1024xf32, affine_map<(d0, d1, d2)[s0] -> (d0 * 32768 + s0 + d1 * 1024 + d2)>>
        %12 = flow.dispatch.tensor.load %1, offsets = [%arg0, %arg1, 0], sizes = [%c1, %c32, 1024], strides = [1, 1, 1] : !flow.dispatch.tensor<readonly:4x32x1024xf32> -> tensor<?x?x1024xf32>
        %13 = memref.subview %2[%arg0, 0, %arg2] [%c1, 1024, %c32] [1, 1, 1] : memref<4x1024x64xf32> to memref<?x1024x?xf32, affine_map<(d0, d1, d2)[s0] -> (d0 * 65536 + s0 + d1 * 64 + d2)>>
        %14 = flow.dispatch.tensor.load %3, offsets = [%arg0, 0, %arg2], sizes = [%c1, 1024, %c32], strides = [1, 1, 1] : !flow.dispatch.tensor<readonly:4x1024x64xf32> -> tensor<?x1024x?xf32>
        %15 = linalg.init_tensor [1, 32, 32] : tensor<1x32x32xf32>
        %16 = memref.alloca() : memref<1x32x32xf32>
        %17 = memref.alloca() : memref<1x32x32xf32>
        linalg.fill(%17, %cst) {__internal_linalg_transform__ = "workgroup"} : memref<1x32x32xf32>, f32
        %18 = linalg.fill(%15, %cst) {__internal_linalg_transform__ = "workgroup"} : tensor<1x32x32xf32>, f32 -> tensor<1x32x32xf32>
        linalg.batch_matmul {__internal_linalg_transform__ = "workgroup", is_root_op} ins(%11, %13 : memref<?x?x1024xf32, affine_map<(d0, d1, d2)[s0] -> (d0 * 32768 + s0 + d1 * 1024 + d2)>>, memref<?x1024x?xf32, affine_map<(d0, d1, d2)[s0] -> (d0 * 65536 + s0 + d1 * 64 + d2)>>) outs(%17 : memref<1x32x32xf32>)
        %19 = linalg.batch_matmul {__internal_linalg_transform__ = "workgroup", is_root_op} ins(%12, %14 : tensor<?x?x1024xf32>, tensor<?x1024x?xf32>) outs(%18 : tensor<1x32x32xf32>) -> tensor<1x32x32xf32>
        %20 = tensor.cast %19 : tensor<1x32x32xf32> to tensor<?x?x?xf32>
      }
    }
  }
  return
}

iree-opt -iree-codegen-linalg-bufferize

It doesnt handle tensor_cast at all. It needs to handle it during pre-processing to use the same buffer for the source as the result.

I see this is equivalent  but I think better than folding tensor.cast into store which requires having every consumer type to do the folding. I will try that

So that worked (forward tensor.cast input buffer) sending a PR shortly,  but it  introduces another an alloc-copy that isn't getting aliased to the output buffer, see:
https://gist.github.com/asaadaldien/1932380337402c1ca22b37ee6eca670b

You should be using createAliasingBufferOrAllocationForResult https://github.com/google/iree/blob/8fb65cbc1595daa2f5385b60f45c3a4428017eec/iree/compiler/Conversion/Common/LinalgBufferizePass.cpp#L292 . See other uses of it in that pass

authors: MaheshRavishankar asaadaldien

repo: google/iree


text:
Add Mac ARM Python wheels

Please add Mac M1 (ARM) Python wheels to the IREE release!

Let me see about bumping this up in priority when I'm back from vacation.
Peter, since you may have done similar things already: do you have any incantations for doing fat or x86-arm cross builds for MacOS? Or do you all have M1 runners?
The last time I looked at this, there was still a lot of rough edges for targeting M1, but since then, I know that @powderluv has gotten through it with IREE --so hopefully we're down to mechanics.
Are the iree-compiler and iree-runtime wheels sufficient? Or do you need the old xla proto importer? If not, we get to avoid building any part of Tensorflow, which is a lot easier (the TF Mac and windows builds have historically been high enough overhead for us that we stopped doing them).

Looks like https://cmake.org/cmake/help/latest/variable/CMAKE_OSX_ARCHITECTURES.html should do the trick with a new enough cmake and pip.

both the hosted GITHUB Runner and the self-hosted don't seem to support M1. The self-hosted could possibly run the agent under rosetta. I will play around with it and report back.
https://github.com/actions/virtual-environments/issues/2187

It looks to me like IREE's wheels are now universal wheels (i.e., the host code is built for ARM?), but if I attempt to use the wheel on a Macbook M1 I get things like this when running a trivial JAX program via IREE:
JAX_PLATFORMS=iree python t.py
Traceback (most recent call last):
  File "/Users/phawkins/p/jax/t.py", line 2, in <module>
    print(lax.add(1, 2))
  File "/Users/phawkins/p/jax/jax/_src/lax/lax.py", line 402, in add
    return add_p.bind(x, y)
  File "/Users/phawkins/p/jax/jax/core.py", line 323, in bind
    return self.bind_with_trace(find_top_trace(args), args, params)
  File "/Users/phawkins/p/jax/jax/core.py", line 326, in bind_with_trace
    out = trace.process_primitive(self, map(trace.full_raise, args), params)
  File "/Users/phawkins/p/jax/jax/core.py", line 676, in process_primitive
    return primitive.impl(*tracers, **params)
  File "/Users/phawkins/p/jax/jax/_src/dispatch.py", line 101, in apply_primitive
    return compiled_fun(*args)
  File "/Users/phawkins/p/jax/jax/_src/dispatch.py", line 167, in <lambda>
    return lambda *args, **kw: compiled(*args, **kw)[0]
  File "/Users/phawkins/p/jax/jax/_src/dispatch.py", line 700, in _execute_compiled
    out_flat = compiled.execute(in_flat)
  File "/Users/phawkins/p/jax/jax/_src/iree.py", line 102, in execute
    outputs = self.module_object[self.function_name](*inputs)
  File "/Users/phawkins/.pyenv/versions/py310/lib/python3.10/site-packages/iree/runtime/function.py", line 129, in __call__
    self._invoke(arg_list, ret_list)
  File "/Users/phawkins/.pyenv/versions/py310/lib/python3.10/site-packages/iree/runtime/function.py", line 153, in _invoke
    self._vm_context.invoke(self._vm_function, arg_list, ret_list)
RuntimeError: Error invoking function: <vm>:0: UNKNOWN; device not supported in the compiled configuration;
[ 0] bytecode jit_prim_fun_0.main:338 "jit(add)/jit(main)/add"(/Users/phawkins/p/jax/t.py:1:1)

Are we perhaps missing an ARM target configuration somewhere?
(The exact same code works on an x86 Mac.)

IIRC there may be an issue with LLVM auto detecting the host configuration in universal builds. If you specify the target CPU directly it should work (--iree-llvm-target-triple=whatever m1 is).

ok cool I wasn't imagining things: https://github.com/google/iree/issues/9028

extra_args=["--iree-llvm-target-triple=arm64-apple-darwin21.5.0"], did the trick, thanks.
I suspect we can close this issue, given the wheels are now universal and the other issue covers the remaining problems I know about.

authors: benvanik hawkinsp powderluv stellaraccident


text:
Updated validations.py python script

Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!

repo: google/it-cert-automation-practice


text:
Closes: #1

Updated validations.py python script.
Replaced the username first character check to only accept a letter.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!


text:
Updated validations.py python script.

Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py.
Closes: #1
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!


text:
Closes: #1

Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py.
Closes: #1

It's a good idea to open an issue first for discussion.

[x ] Tests pass
[ x] Appropriate changes to README are included in PR

@googlebot I signed it!

@googlebot I fixed it.


text:
jnp.arange type error

Someone ran into a message like this:
  File ".../jax/numpy/lax_numpy.py", line 2295, in arange
    return array(np.arange(start, stop=stop, step=step, dtype=dtype))
TypeError: expected number, got numpy.int64

The calling code looked like:
N = 200
jnp.arange(0, N, dtype=activation_dtype)

This error likely came from #3656.

I'm confused about what I meant with this issue, and I can't figure out how to repro it, so I'm going to close it!

repo: google/jax


text:
Pjit does not handle static_argnums correctly

It seems that pjit does not handle static_argnums correctly
class MyModel:

  def __init__(self, num_layers: int):
    self.num_layers = num_layers

  def compute(self, x):
    for i in range(self.num_layers):
      # Imagine here a real implementation of neural networks.
      x = x + 10
    return x

def run_my_model(x, model):
  print("Runing/Tracing run_my_model")
  return model.compute(x)

input = jnp.arange(4)
model = MyModel(num_layers=3)

with mesh(jax.devices(), ["data"]):
  pjit(run_my_model, in_axis_resources=None,
       out_axis_resources=None, static_argnums=1)(input, model)

This results in error:
TypeError                                 Traceback (most recent call last)
<ipython-input-7-88bb4a2473bc> in <module>()
      4 with mesh(jax.devices(), ["data"]):
      5   pjit(run_my_model, in_axis_resources=None,
----> 6        out_axis_resources=None, static_argnums=1)(input, model)

1 frames
/usr/local/lib/python3.7/dist-packages/jax/experimental/pjit.py in wrapped(*args, **kwargs)
    229   def wrapped(*args, **kwargs):
    230     for arg in tree_leaves(args):
--> 231       _check_arg(arg)
    232     args_flat, params, _, out_tree = infer_params(*args, **kwargs)
    233     out = pjit_p.bind(*args_flat, **params)

/usr/local/lib/python3.7/dist-packages/jax/_src/api.py in _check_arg(arg)
   2685 def _check_arg(arg):
   2686   if not (isinstance(arg, core.Tracer) or _valid_jaxtype(arg)):
-> 2687     raise TypeError(f"Argument '{arg}' of type {type(arg)} is not a valid JAX type.")
   2688
   2689 # TODO(necula): this duplicates code in core.valid_jaxtype

TypeError: Argument '<__main__.MyModel object at 0x7fdff5c5da10>' of type <class '__main__.MyModel'> is not a valid JAX type.

Colab repro - as you can see, the same thing with jit just works.
https://colab.research.google.com/drive/1yiAibGXqfZqhTA7lptG2XCJhnaA95Vh0?usp=sharing

Is this still an issue? Looks like Model should be a pytree.

Closing because of lack of response. Feel free to reopen if you can still repro it.


text:
Proof of concept: add jax.numpy.random wrapping np.random API

One of the more jarring things I found when starting to work in jax was the difference between the API of jax.random and that of np.random. I think in many cases users would benefit from being able to use np.random code directly in jax.
This PR contains an incomplete sketch of what this might look like in practice. On this branch, you can do things like:
import jax.numpy as np
np.random.seed(0)
np.random.randn(10000).sum()

which mirrors familiar numpy syntax; compare to using jax.random directly:
from jax import random
key = random.PRNGKey(0)
random.normal(key, (10000,)).sum()

Would the team be in favor of building-out this API for random array generation?

The reason why we have chosen not to do anything along these lines is because of jit-invariance.
We want functions to act the same whether they are under jit or not. See:
https://github.com/google/jax/blob/master/design_notes/prng.md
A stateful class along these lines does not preserve the jit-invariance property.

Makes sense - thanks!

authors: hawkinsp jakevdp


text:
Partially revert #2548 which breaks tests on macOS when the extractor has different system includes from toolchain [phab:D2175]

Created by shahms at 2018-02-08 18:26:28:
Partially revert #2548 which breaks tests on macOS when the extractor has different system includes from toolchain

salguarnieri wrote at 2018-02-08 18:40:58:
builds clean for me on my mac

salguarnieri accepted this revision at 2018-02-08 18:40:58

shahms wrote at 2018-02-08 20:24:59:

Make adding toolchain builtin directories conditional and only enable it on macOS

shahms wrote at 2018-02-08 20:25:49:
I've updated this so it only adds the toolchain builtins on macOS, can you give it another spin?  Even this is pretty ugly and I'm not sure what the "right" way to go about it is.

zarko accepted this revision at 2018-02-08 21:03:33

shahms wrote at 2018-02-08 21:10:06:

Add missing i in -isystem flag

shahms closed via commit 7b17bf1b916e49734c9ee9712cb5416b4369770d at 2018-02-08 21:15:54

repo: google/kythe


text:
Updated CopyString function

Update CopyString function to return a NULL terminated string. I received garbage strings while using the C interface in Swift.were

I signed it!

CopyString is used for values returned by leveldb_get.  These values can be binary data, not just text and can therefore contain embedded null characters.  Therefore we do not rely on null-termination to indicate the length of the string, but instead return the length (e.g., leveldb_get's vallen argument).
We could in addition also null-terminate for the convenience of people who are storing text, but I would rather not do that because it would encourage libraries to start relying on that and those libraries would then break in a very bad way (silent truncation) if the database ever contained a value with embedded nulls.
I suggest that if you are only dealing with plain-text values, you write a simple wrapper around leveldb_get in your application.

repo: google/leveldb