text:
Data type precision and demote/promote preference config

Please:

[x] Check for duplicate requests.

https://github.com/google/jax/issues/19033
https://github.com/google/jax/issues/16705
https://github.com/google/jax/issues/15580

[x] Describe your goal, and if possible provide a code snippet with a motivating example.

TLDR: to benefit the future of AI innovation, we could add config options to enhance JAX support for mixed and reduced precision numbers
I've hit this error message a billion times in getting rolling with ml_dtypes.

I wish I could set jax.config options to make it easier...
Ideas:

A default_precise_float_dtype value set to f32 (same default dtype as now)
A default_imprecise_float_dtype value set to None or bf16
A float_precision_mismatch_behavior enum with raise, auto_demote, and auto_promote variants which defaults to raise (same behavior as now)
(Optional): A default_float_dtype_precision enum with precise and imprecise variants, defaulting to precise

Crucial point: these config options ought to have sensible defaults which yield no change in behavior unless a user changes them. Thus it would be a purely additive change and not negatively impact backward compatiblity.
(ditto for integers possible)
Explanation:
We keep the default f32 dtype as the default_precise_float type
If the accelerator can support reduced precision like bf16 or f8, then they could specify the desired imprecise dtype as a config option and select auto_demote instead of raise
Then the auto_demote behavior can be: if one of the dtypes is reduced precision and the other is higher precision, jax could know to reduce the precision of the more precise float to the default_imprecise_float_dtype.
Likewise, if the situation is mostly all clearly specified in the codebase as reduced precision, then a savvy jax user could set float_precision_mismatch_behavior to auto_promote and jax could know to convert imprecise floats to the more precise type, for use cases like batch norm or bigger number types (and surely I recognize this is default for some dtype pairings)
Minimal reproduction:
import jax
import ml_dtypes
precise_arr = jax.numpy.ones(3)
imprecise_arr = jax.numpy.ones(3, dtype=ml_dtypes.float8_e5m2)
jax.numpy.array_equal(precise_arr, imprecise_arr)

Error copypasta
(i'm sure you've seen this already if you're working with mixed precision)
args = (Traced<ShapedArray(float32[3])>with<DynamicJaxprTrace(level=1/0)>, Traced<ShapedArray(float8_e5m2[3])>with<DynamicJaxprTrace(level=1/0)>)

    def promote_dtypes(*args: ArrayLike) -> list[Array]:
      """Convenience function to apply Numpy argument dtype promotion."""
      # TODO(dougalm,mattjj): This is a performance bottleneck. Consider memoizing.
      if len(args) < 2:
        return [lax.asarray(arg) for arg in args]
      else:
>       to_dtype, weak_type = dtypes._lattice_result_type(*args)
E       jax._src.dtypes.TypePromotionError: Input dtypes ('float8_e5m2', 'float32') have no available implicit dtype promotion path. To avoid unintended promotion, 8-bit floats do not support implicit promotion. If you'd like your inputs to be promoted to another type, you can do so explicitly using e.g. x.astype('float32')
E       --------------------

Final Disclaimer

I am no expert on jax config options so there might already be something like this (the jax.config.update("jax_enable_x64", True) one I sometimes use comes to mind)
There may exist XLA environment variables to achieve similar aims
There might be better ways to pull this off
There might be better names for these config options
There might be better data structures, imho enums are good, clear
I'm not really familiar enough with this codebase to make a PR for this.
I welcome valuable input from the JAX community, WDYT?

Hi - we've discussed this before, and in general we've found that different users have different opinions when it comes to what the default promotion behavior for float8 should be. Given that, and given that float8 is generally used very deliberately within a very limited range of situations, we've come to the conculsion that it's best for promotion with float8 to be explicit, rather than implicit. What do you think?

f8 will see more use in the future to save memory, so my objective was to prepare to explicitly configure optional sensible defaults to demote to more efficient dtypes, not promote
Sadly, I can't contribute to Google projects knowing the Gemini API and Alphafold Server customer noncompetes are unsafe, so I'm closing, no offense to you Jake, I know you're a 10x dev and I really admire all the jax team members, just can't contribute to JAX in good conscience knowing this clause are up.

my bad for opening this, I was excited because I hadn't noticed that, but that clause makes the set of things one can actually use gemini for without fear of legal risk an empty set, and anything I do to help JAX, just contributes make the "closed output / customer noncompete" problem even worse, so I must cease involvement in the project
if the most critical documents in AI safety are written by corporate lawyers instead of AI engineers, humanity is truly ****ed
here's a better license, not that anyone cares

authors: bionicles jakevdp


text:
Adjust tolerance for TPU on ExamplesTest.testKernelRegressionTrainAndPredict

@mattjj does this look ok? This is a pretty loose tolerance but not sure how else to address it...

Is this a TPU-only failure? If so we could do 0.2 if jtu.device_under_test() == 'tpu' else 1e-3.
But also +1 to Peter's suggestion of trying high-precision matmuls. Maybe this is a good example of why the default precision should be different...

source: gharchive/pull-request

authors: mattjj skye

repo: google/jax

license_type: permissive

license_source: github-api


text:
Revert "Only create epochs if there is something to process or the maximum nu…"

Reverts google/keytransparency#593

Looks like the tests which depend on time are too flakey for Travis. Let's come up with a time-independent testing strategy.

Codecov Report

Merging #599 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #599   +/-   ##
=======================================
  Coverage   59.94%   59.94%
=======================================
  Files          33       33
  Lines        2322     2322
=======================================
  Hits         1392     1392
  Misses        643      643
  Partials      287      287

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dba8a2...cf85c79. Read the comment docs.

authors: cesarghali codecov-io gdbelvin

repo: google/keytransparency

license_source: bigquery


text:
Add a library for rendering MarkedSource to strings. [phab:D1185]

Created by zarko at 2016-12-15 22:20:54:
This is minimal to start. We'll want to add other options
later on to ignore default values, possibly colorize types
vs. identifiers, and so on.

fromberger accepted this revision at 2016-12-15 22:47:32

zarko closed via commit 9e9d14f596b2f79f6d83a5ac6eaa866e15f2924f at 2016-12-15 22:48:35

source: gharchive/issue

repo: google/kythe


text:
about the HandleTable's FindPointer function

hi,
I am reading the source code and the FindPointer's while loop confuses me a little.
First, we get the list by the hash value, but why does the loop check the hash value again?
I think that the '(*ptr)->hash == hash' is always TRUE.  NOT right?
LRUHandle** FindPointer(const Slice& key, uint32_t hash) {
LRUHandle** ptr = &list_[hash & (length_ - 1)];
while (*ptr != NULL &&
((*ptr)->hash != hash || key != (*ptr)->key())) {
ptr = &(*ptr)->next_hash;
}
return ptr;
}

On Mon, Jul 10, 2017 at 7:14 PM, rwen2012 notifications@github.com wrote:

hi,
I am reading the source code and the FindPointer's while loop confuses me
a little.
First, we get the list by the hash value, but why does the loop check the
hash value again?
I think that the '(*ptr)->hash == hash' is always TRUE. NOT right?
LRUHandle** FindPointer(const Slice& key, uint32_t hash) {
LRUHandle** ptr = &list_[hash & (length_ - 1)];

The preceding line only uses the bottom bits of the hash to get the list.
So the list can contain many different hash values, all with the same
bottom bits.

while (*ptr != NULL &&
((*ptr)->hash != hash || key != (*ptr)->key())) {
ptr = &(*ptr)->next_hash;
}
return ptr;
}
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/leveldb/issues/482, or mute the thread
https://github.com/notifications/unsubscribe-auth/AHuM0bWyC1_WzPeNaXsH4irbetJ5x2aDks5sMtp_gaJpZM4OTo7G
.

I see,  thank you very much!

repo: google/leveldb


text:
Test failures

I'm getting the following test failures in the current git master:
[  FAILED  ] 36 tests, listed below:
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/0, where GetParam() = (0, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/1, where GetParam() = (0, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/2, where GetParam() = (0, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/3, where GetParam() = (1, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/4, where GetParam() = (1, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/5, where GetParam() = (1, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/6, where GetParam() = (2, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/7, where GetParam() = (2, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI0AndDecodeTest/8, where GetParam() = (2, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/0, where GetParam() = (0, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/1, where GetParam() = (0, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/2, where GetParam() = (0, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/3, where GetParam() = (1, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/4, where GetParam() = (1, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/5, where GetParam() = (1, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/6, where GetParam() = (2, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/7, where GetParam() = (2, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI1AndDecodeTest/8, where GetParam() = (2, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/0, where GetParam() = (0, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/1, where GetParam() = (0, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/2, where GetParam() = (0, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/3, where GetParam() = (1, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/4, where GetParam() = (1, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/5, where GetParam() = (1, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/6, where GetParam() = (2, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/7, where GetParam() = (2, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI2AndDecodeTest/8, where GetParam() = (2, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/0, where GetParam() = (0, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/1, where GetParam() = (0, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/2, where GetParam() = (0, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/3, where GetParam() = (1, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/4, where GetParam() = (1, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/5, where GetParam() = (1, 2)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/6, where GetParam() = (2, 0)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/7, where GetParam() = (2, 1)
[  FAILED  ] JpegRAPIParameterizedTests/JpegRAPIEncodeAndDecodeTest.EncodeAPI3AndDecodeTest/8, where GetParam() = (2, 2)

36 FAILED TESTS

A git bisect shows the offending commit as: c0c184568d38c8b1cb0ad79a5a86f02a69bc522f.
System Information:

OS: Arch Linux
CMake: 3.30.0
Compiler: gcc 14.1.1
libjpeg (libjpeg-turbo): 3.0.3

these will be fixed once this merged. https://github.com/google/libultrahdr/pull/203
thank you.

@DichenZhang1 this can be closed

authors: dbermond ram-mohan

repo: google/libultrahdr


text:
Search Component Broken In Template

Version: MDL 1.x
Browser: Chrome
OS: OS X 10.11.6
Problem:
The search component in the "Dashboard" template header is broken. It should be laying out neatly aligned to the left of the "More Options" button, but it's breaking and flowing below the header.
See issue here: https://getmdl.io/templates/dashboard/index.html

Duplicate of #4532

authors: Garbee simpsondigital

repo: google/material-design-lite


text:
Aborted MediaPipe Task during initialization on Chrome - Windows 10 - VMWare Workstation

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
No
OS Platform and Distribution
Windows 10 VM
Mobile device if the issue happens on mobile device
No response
Browser and version if the issue happens on browser
Google Chrome Version 117.0.5938.150
Programming Language and version
JavaScript
MediaPipe version
0.10.3
Bazel version
No response
Solution
FaceLandmarker
Android Studio, NDK, SDK versions (if issue is related to building in Android environment)
No response
Xcode & Tulsi version (if issue is related to building for iOS)
No response
Describe the actual behavior
The codepen sample "CodePen Home MediaPipe Face Landmarker Task for web" fails to initialize the MediaPipe Task.
Describe the expected behaviour
Successful initialization
Standalone code/steps you may have used to try to get what you need
To reproduce:

Run Windows 10 on VMWare, run Chrome, and go to https://codepen.io/mediapipe-preview/pen/OJBVQJm.

Click ENABLE WEBCAM and no witness that face landmarking does not occur. Check the console and see:

gl_texture_buffer.cc:205 Check failed: prod_token    vision_wasm_internal.js:9
Aborted()                                            vision_wasm_internal.js:9

Other info / Complete Logs
No response

@delta42,
It seems the issue with VM Camera, VM is not able to find out the camera. Could you please let us know if you have added the camera to VM by using this path "Virtual Machine==>Settings==>Add Device==>Camera"?
If not we recommend you to add it and give a try once. Also, You can find out How to add a camera to a virtual Machine here. Thank you

@delta42,
It appears there may be an issue with the VM Camera as the virtual machine is unable to detect it. Could you kindly confirm whether you have added the camera to the virtual machine using the following path: "Virtual Machine ==> Settings ==> Add Device ==> Camera"?
If not, we recommend adding the camera and testing it. For guidance on adding a camera to a virtual machine, please refer to the instructions available here. Thank you.

@delta42,
Can you confirm that you have followed the recommendation in the previous comment and added the camera to your virtual machine?

repo: google/mediapipe


text:
Running an example in docker image fails with EGL/egl.h not found.

I followed document with installing using docker.
Hello world desktop example works well, but face_mseh_cpu is not working.
Command
root@e655929b92f0:/mediapipe# GLOG_logtostderr=1 bazel run mediapipe/examples/desktop/face_mesh:face_mesh_cpu

Error log
INFO: Build option --define has changed, discarding analysis cache.
INFO: Analyzed target //mediapipe/examples/desktop/face_mesh:face_mesh_cpu (14 packages loaded, 5388 targets configured).
INFO: Found 1 target...
INFO: From ProtoCompile mediapipe/gpu/gl_context_options.pb.h:
bazel-out/k8-fastbuild/bin/external/com_google_protobuf/src: warning: directory does not exist.
ERROR: /mediapipe/mediapipe/gpu/BUILD:410:1: C++ compilation of rule '//mediapipe/gpu:gpu_buffer_multi_pool' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 50 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from ./mediapipe/gpu/gpu_buffer.h:20:0,
                 from ./mediapipe/gpu/gpu_buffer_multi_pool.h:30,
                 from mediapipe/gpu/gpu_buffer_multi_pool.cc:15:
./mediapipe/gpu/gl_base.h:57:10: fatal error: EGL/egl.h: No such file or directory
 #include <EGL/egl.h>
          ^~~~~~~~~~~
compilation terminated.
Target //mediapipe/examples/desktop/face_mesh:face_mesh_cpu failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 10.702s, Critical Path: 7.49s
INFO: 68 processes: 68 processwrapper-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

Got the samee issue when I try to build the hand_tracking_gpu model :
root@6792a9d9e62f:/mediapipe# bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 \ mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu

Error log
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
WARNING: Download from https://mirror.bazel.build/github.com/tensorflow/tensorflow/archive/7c09d15f9fcc14343343c247ebf5b8e0afe3e4aa.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
WARNING: Download from http://mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/cf1e44edb908e9616030cc83d085989b8e6cd6df.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/master.zip) = a07fc8b90c22e42962142504438a84b31567976adcd9ee1a4c9f75d2e3396f8d
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "a07fc8b90c22e42962142504438a84b31567976adcd9ee1a4c9f75d2e3396f8d"
DEBUG: Call stack for the definition of repository 'rules_cc' which is a http_archive (rule definition at /root/.cache/bazel/_bazel_root/4884d566396e9b67b62185751879ad14/external/bazel_tools/tools/build_defs/repo/http.bzl:292:16):
 - /mediapipe/WORKSPACE:33:1
WARNING: Download from https://mirror.bazel.build/github.com/Maratyszcza/FP16/archive/3c54eacb74f6f5e39077300c5564156c424d77ba.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/XNNPACK/archive/5cb16e7ace0fcdcab164af01620a606ba828a3be.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
INFO: Analyzed target //mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu (121 packages loaded, 5339 targets configured).
INFO: Found 1 target...
ERROR: /mediapipe/mediapipe/gpu/BUILD:632:1: C++ compilation of rule '//mediapipe/gpu:gl_simple_shaders' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 23 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from ./mediapipe/gpu/gl_simple_shaders.h:18:0,
                 from mediapipe/gpu/gl_simple_shaders.cc:15:
./mediapipe/gpu/gl_base.h:57:10: fatal error: EGL/egl.h: No such file or directory
 #include <EGL/egl.h>
          ^~~~~~~~~~~
compilation terminated.
Target //mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 223.631s, Critical Path: 116.58s
INFO: 346 processes: 346 processwrapper-sandbox.
FAILED: Build did NOT complete successfully

Did you solve the problem @lackhole ?

https://github.com/google/mediapipe/issues/846

@romainvo
It was because lack of memory issue.
If you're using Docker Desktop, change memory to maximum.

I had this problem under Ubuntu 20.04 (Focal) and OpenCV 4.2 (4.2.0+dfsg-5).
My host has all the possible egl.h installed:
#> dlocate /usr/include/EGL/egl.h /usr/include/epoxy/egl.h /usr/include/GLES/egl.h
libepoxy-dev:amd64: /usr/include/epoxy/egl.h
libgles-dev:amd64: /usr/include/GLES/egl.h
libegl-dev:amd64: /usr/include/EGL/egl.h

I've solved with this:
on the host
cd /tmp ; sudo apt download dlocate libgles-dev libglx0 libglvnd0 libgles1 libgl-dev dctrl-tools libegl-dev libglx-dev libgles2 libgl1 libegl1
sudo docker ps -a
sudo docker start -i <mediapipe_container_id>
for i in *.deb ; do sudo docker cp $i 83fce098962a:/tmp ; done
inside the container
dpkg -i libglvnd0_1.3.1-1ubuntu0.20.04.1_amd64.deb libgles2_1.3.1-1ubuntu0.20.04.1_amd64.deb libgl1_1.3.1-1ubuntu0.20.04.1_amd64.deb libegl1_1.3.1-1ubuntu0.20.04.1_amd64.deb libglx0_1.3.1-1ubuntu0.20.04.1_amd64.deb libgles1_1.3.1-1ubuntu0.20.04.1_amd64.deb
dpkg -i libgles-dev_1.3.1-1ubuntu0.20.04.1_amd64.deb libgl-dev_1.3.1-1ubuntu0.20.04.1_amd64.deb libegl-dev_1.3.1-1ubuntu0.20.04.1_amd64.deb libglx-dev_1.3.1-1ubuntu0.20.04.1_amd64.deb
Then examples compile as expected:
bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu

authors: aviogit lackhole mcclanahoochie romainvo


text:
Support for configuring KHR_lights_punctual

https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Khronos/KHR_lights_punctual
We should support configuration KHR_lights_punctual via 3DOM so that folks can add, remove and/or manipulate punctual lights in a scene.
Corresponding support for this extension would also need to be added to <model-viewer>.

I actually don't want to do this. It opens up a lot of complexity regarding shadows and it's also less efficient than baking all lights into the IBL. I would prefer to make a light baking feature in our editor.

authors: cdata elalish

repo: google/model-viewer


text:
solver.IntVar does not return integer values in the solution

What version of OR-tools and what language are you using?
Version: 8.2 installed via pip (python 3.8)
Language: Python
Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)
GLOP
(except GLOP is there any solver with is under license apache 2 ?)
What operating system (Linux, Windows, ...) and version?
Linux 20.04 LTS in a virtual machine
What did you do?
Steps to reproduce the behavior:
Run the follwing script:

What did you expect to see
Normaly, all elements of Sol1 should be interger and even binary (0 or 1)
Sol 1 =  [
0    0    1     0    0
0    1     0    0    0
1    0     0    1    1
]
Sol 2 = [875 425 187.5 1050 1012.5]
(solutions found with my equivalent Matlab program)
What did you see instead?

As you can see the elements of Sol 1 are not or equal to 0 or 1 but between.  The sum of each column is well equal to 1 as expected. For the third column, instead of [ 1 0 0] i get [0.69 0 0.31].
Similar results with the 5th column ([0.45 0 0.55 instead of [0 0 1])
Theses wrong values (real and not binary) affect of course the  third and fifth values of sol2 which are not correct.
Normally, for my problem i can have several solutions. I saw that with "pywrapcp" (instead of "pywraplp") it is possible.
I tried first with "pywrapcp" but i didn't succed to multiply my integer variable with some constants values, which can be rounded as integer if needed. That's why i moved to "pywraplp". Is there a simple way to get all solutions with pywraplp ?
This problem of mixing a variable which is a matrix of integers values and a variable which is a vector of double value.
Other solver like CP-SAT, Routing Solver, BOP, Gurobi can do the job ? My constraint is to use apache v2 license solvers.
Thanks for looking into this!
Emman
Make sure you include information that can help us debug (full error message, model Proto).
Anything else we should know about your project / environment

Please read the difference between a simplex solver and a MIP solver.
Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53
00
Le mer. 18 nov. 2020 à 11:19, emman91 notifications@github.com a écrit :

What version of OR-tools and what language are you using?
Version: 8.2 installed via pip (python 3.8)
Language: Python
Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP,
Gurobi)
GLOP
What operating system (Linux, Windows, ...) and version?
Linux 20.04 LTS in a virtual machine
What did you do?
Steps to reproduce the behavior:
Run the follwing script:
[image: image]
https://user-images.githubusercontent.com/74408817/99512160-7380a080-2989-11eb-901e-1892dcff4f91.png
[image: image]
https://user-images.githubusercontent.com/74408817/99512349-a460d580-2989-11eb-8163-897d9bdb594c.png
[image: image]
https://user-images.githubusercontent.com/74408817/99512431-bc385980-2989-11eb-9c68-68fd8db16660.png
What did you expect to see
Normaly, all elements of Sol1 should be interger and even binary (0 or 1)
Sol 1 = [
0 0 1 0 0
0 1 0 0 0
1 0 0 1 1
]
Sol 2 = [875 425 187.5 1050 1012.5]
(solutions found with my equivalent Matlab program)
What did you see instead?
[image: image]
https://user-images.githubusercontent.com/74408817/99515602-754c6300-298d-11eb-92fb-d1e4540a424c.png
As you can see the elements of Sol 1 are not or equal to 0 or 1 but
between. The sum of each column is well equal to 1 as expected. For the
third column, instead of [ 1 0 0] i get [0.69 0 0.31].
Similar results with the 5th column ([0.45 0 0.55 instead of [0 0 1])
Theses wrong values (real and not binary) affect of course the third and
fifth values of sol2 which are not correct.
Normally, for my problem i can have several solutions. I saw that with
"pywrapcp" (instead of "pywraplp") it is possible.
I tried first with "pywrapcp" but i didn't succed to multiply my integer
variable with some constants values, which can be rounded as integer if
needed. That's why i moved to "pywraplp". Is there a simple way to get all
solutions with pywraplp ?
Thanks for looking into this!
Emman
Make sure you include information that can help us debug (full error
message, model Proto).
Anything else we should know about your project / environment
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2233, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3LY5Z3Z2QNSUKACEYTSQONRXANCNFSM4TZYQSQQ
.

Thansk for your fast reply. I know what is a MIP solver (Mixed linear problem) and i know that the solver i have used does not solve MIP. As i said, i can convert my problem into a full integer problem if needed. My question was why in the linear solver (GLOP, pywraplp) , a variable declared as integer with solver.IntVar return a real value in the solution.
Emman

Is there any solver for MIP under apache 2.0 ?

You may use the Coin-OR CBC solver which is under a EPL v2 license
ref: https://github.com/coin-or/Cbc/blob/master/LICENSE

Yes but according to my knowledge, with EPL v2  you have to give your code,  which is not the case with apache v2, right ?

And about my issue with GLOP with pywraplp, why a variable declared as integer with solver.IntVar return a real value in the solution ? have you some ideas ?

I would say glop is a pure linear solver to all intVar are converted to floating point value since integer constraint is not supported.

Yes but according to my knowledge, with EPL v2 you have to give your code, which is not the case with apache v2, right ?

Yes and no, you only have to publish the EPL code is you have modified it, it's a weak copyleft so simply using Cbc verbatim should be OK IMHO.

so my initial personnal code is not public until the moment i modify it ?

Only if you modify the code of the cbc solver.
Le mer. 18 nov. 2020 à 12:39, emman91 notifications@github.com a écrit :

so my initial personnal code is not public until the moment i modify it ?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2233#issuecomment-729623712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3IIUX7SJSOO4BHLRBDSQOW7VANCNFSM4TZYQSQQ
.

Forcing the variables to have integer values is exactly the difference
between a simplex and a mip solver.
You can also use SAT_INTEGER_PROGRAMMING be to solve your problem. It is
part of or-tools.
Le mer. 18 nov. 2020 à 12:23, emman91 notifications@github.com a écrit :

And about my issue with GLOP with pywraplp, why a variable declared as
integer with solver.IntVar return a real value in the solution ? have you
some ideas ?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2233#issuecomment-729615474,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3I6TNRWN3LZ5TYPILDSQOVB7ANCNFSM4TZYQSQQ
.

authors: Mizux emman91 lperron

repo: google/or-tools


text:
makefile is broken

Since c6a750b0:
$ make clean
make: *** No rule to make target 'makefiles/Makefile.third_party.unix', needed by 'Makefile.local'.  Stop.

So far as I can see, the fix should just be add the .mk suffix at this line.
Maybe consider setting up CI...?

THis is fixed. Sorry for the delay.

Hi,
I fixed that some time ago, but never reported it.
Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53
00
2017-03-12 8:30 GMT-04:00 David Hotham notifications@github.com:

Since c6a750b
https://github.com/google/or-tools/commit/c6a750b0e08bb8232ad07ad52a9b36b74ce2bed2
:
$ make clean
make: *** No rule to make target 'makefiles/Makefile.third_party.unix', needed by 'Makefile.local'.  Stop.
So far as I can see, the fix should be just to remove the bogus dependency
at this line
https://github.com/google/or-tools/blob/f4d007098d421500c683248626eb0389d195c479/makefiles/Makefile.third_party.unix.mk#L475
.
Maybe consider setting up CI...?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/341, or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17aNMYq6Y0EaFG_6YXAJ6c1KKd6Lxks5rk-V1gaJpZM4Mahl2
.

authors: dimbleby lperron


text:
Infeasible model using presolve

What version of OR-Tools and what language are you using?
9.5.2237
Language: C++/Java/Python/C#
C++
Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)
CP-SAT
What operating system (Linux, Windows, ...) and version?
Ubuntu 22.04.1
What did you do?
With presolve produces 0/2 feasible models and no presolve produces 2/2 feasible models.
Steps to reproduce the behavior:

git clone https://github.com/ctmcdo/Jessica
cd Jessica
git checkout baa02b249d9bcca94005cca3445412951b49bc7c
sed -i 's/model.GetOrCreate()->set_cp_model_presolve(false);/model.GetOrCreate()->set_cp_model_presolve(true);/' filter_pawn/filter_pawn.cc
Create shared library from filter_pawn/filter_pawn.cc, which for me entails (cd ~/ortools/ && make build SOURCE_NAME=/abs/path/to/Jessica/filter_pawn/ SOURCE=/abs/path/to/Jessica/filter_pawn/filter_pawn.cc)
echo -e "11brq11r111p111p1pp1kp1N1pQ1p111111P1111NP11KP111P11P1PPR1B11B1R b - -\nr11q1bnr1p111k1pp111bp1111p1p1p1P1PBP1PP1P111P11n11P1QB11N11K1NR w K -" > myfile
sed -i 's//home/colin/dev/Jessica/weirdsample//abs/path/to/Jessica/myfile/' main.c # replace with abs path to Jessica/myfile
make
./Jessica

What did you expect to see
Sampling from test file.
Potentially reachable positions:
11brq11r/111p111p/1pp1kp1N/1pQ1p111/111P1111/NP11KP11/1P11P1PP/R1B11B1R b - -
r11q1bnr/1p111k1p/p111bp11/11p1p1p1/P1PBP1PP/1P111P11/n11P1QB1/1N11K1NR w K -

Number of potentially reachable positions from sample: 2

What did you see instead?
Sampling from test file.
Potentially reachable positions:

Number of potentially reachable positions from sample: 0

Make sure you include information that can help us debug (full error message, model Proto).
Anything else we should know about your project / environment
My cpu is AMD Ryzen 5 3500U
I know there are a lot of steps to reproduce. It seems like it would be a large investment of time for me to come up with a more minimal example

Why don't you just export the model that fails?

cp_model.ExportToFile("bad_presolve.pb.txt");
Le mar. 27 déc. 2022, 18:10, ctmcdo @.***> a écrit :

And how do I do that?
—
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/3607#issuecomment-1366052134,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3PK7WJC5MFQFP2Z4EDWPMPHTANCNFSM6AAAAAATKPS3MA
.
You are receiving this because you were assigned.Message ID:
@.***>

Sorry
#include "ortools/sat/swig_helper.h"

then
CpModelBuilder cp_model;
// Build model.
operations_research::sat::CpSatHelper::WriteModelToFile(cp_model.Proto(), "bad_presolve.pb.txt");

I will add the API to the CpModelBuilder class.

I ran both models on the main branch with and without presolve. They all
return optimal.
Laurent Perron | Operations Research | @.*** | (33) 1 42 68 53
00
Le mer. 28 déc. 2022 à 10:09, ctmcdo @.***> a écrit :

I removed one term in one constraint and presolved work.
diff bad_presolve.pb.txt good_presolve.pb.txt
4061d4060
<     vars: 205
4067d4065
<     coeffs: -1
4085d4082
<     vars: 206
4089d4085
<     coeffs: -1
good_presolve.pb.txt
https://github.com/google/or-tools/files/10313250/good_presolve.pb.txt
bad_presolve.pb.txt
https://github.com/google/or-tools/files/10313251/bad_presolve.pb.txt
—
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/3607#issuecomment-1366489911,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3PSHMQR6TBXB67EU3LWPP7WDANCNFSM6AAAAAATKPS3MA
.
You are receiving this because you were assigned.Message ID:
@.***>

Thanks for the bug report. I will close it now.

authors: ctmcdo lperron


text:
Agent improvements: Adopt system instructions and allow multiple command executions

Allow passing system instructions to LLM
Allow executing multiple bash commands in one response
Prompt fixes
Minor corrections and bug fixes

In addition to the new features, this also generated buildable fuzz targets for project xs for the first time.

/gcbrun exp -n dg -ag

Report: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2024-11-13-717-dg-comparison/index.html
Seeing many errors like:
File "/usr/local/lib/python3.11/dist-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/grpc/_channel.py", line 1181, in __call__
return _end_unary_response_blocking(state, call, False, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Unable to submit request because the input token count is 35103 but model only supports up to 32768. Reduce the input token count and try again. You can also use the CountTokens API to calculate prompt token count and billable characters. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models"
debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.72.170:443 {grpc_message:"Unable to submit request because the input token count is 35103 but model only supports up to 32768. Reduce the input token count and try again. You can also use the CountTokens API to calculate prompt token count and billable characters. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models", grpc_status:3, created_time:"2024-11-13T04:04:17.961683542+00:00"}"

This is likely due to the new system instructions added.

/gcbrun exp -n dg1 -ag

/gcbrun exp -n dg -ag

/gcbrun exp -n dg -ag

/gcbrun exp -n dg -ag

/gcbrun exp -n dg -ag

Hi @mihaimaruseac, could you please help me check if I did something wrong in this PR that can cause the following error when invoking chat model's send_message()?
I keep getting the following over-long input error (example) in cloud experiments but I cannot reproduce it locally:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/grpc/_channel.py", line 1181, in __call__
return _end_unary_response_blocking(state, call, False, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Unable to submit request because the input token count is 58602 but model only supports up to 32768. Reduce the input token count and try again. You can also use the CountTokens API to calculate prompt token count and billable characters. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models"
debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.188.234:443 {grpc_message:"Unable to submit request because the input token count is 58602 but model only supports up to 32768. Reduce the input token count and try again. You can also use the CountTokens API to calculate prompt token count and billable characters. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models", grpc_status:3, created_time:"2024-11-14T11:52:11.683245084+00:00"}"
>

IIUC, the message says the input has 58602 tokens, exceeding the 32768 token limit.
I guess this relates to using system_instruction because I did not see this error before, but I find it confusing for two reasons:

Re-run the same steps locally won't trigger this error. The number of token is ~57719 (close to 58602, which makes sense), greater than the token limit 32768. But I got a valid response from LLM and did not trigger the error.
Cannot find where the 32768 limit come from. I noticed several places mentioning this number (e.g., context caching) but I reckon since we did not use manually, they are not relevant to us?

Thanks!

Meanwhile, I will attempt using the built-in Tool API.

I think this is because the error log from the compilation is too long?
The model itself has a limit of tokens, afaik, and we might be hitting that?

I think this is because the error log from the compilation is too long?

Hmm... If that's the case, shouldn't we be able to reproduce the error locally?
When running it locally, I did not get any error even when the input has 57719 tokens, which is much large than the 32768 limit mentioned in the error message from cloud experiment.

The model itself has a limit of tokens, afaik, and we might be hitting that?

Please correct me if I am wrong, but I thought gemini-1.5-pro input token limit is much larger:
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-1.5-pro:~:text=Max input tokens%3A 2%2C097%2C152

Oh, I was wrong. I'll think about it, but currently I don't have an idea, sorry

Oh, I was wrong. I'll think about it, but currently I don't have an idea, sorry

Thanks!
If you have some theories later, I am happy to try them out in this PR : )

I believe the token limit issues are because of this:
https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations#ml_processing

Not all regions support longer context for gemini-1.5-pro-002

/gcbrun exp -n dg -ag

Converting this to a draft because Tool is better in generating outputs in required type/struct.
I will manually cherry-pick commits from this PR (#717) and #718 into a new branch.

authors: DonggeLiu mihaimaruseac oliverchang

repo: google/oss-fuzz-gen


text:
test: remove future snapshots

These got included in #938 even though they're for #937 and it seems that go-snaps does not error about this even though it will clean them up if run with UPDATE_SNAPS=clean.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 64.20%. Comparing base (c2bd45e) to head (e8ca7ea).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #960      +/-   ##
==========================================
+ Coverage   63.67%   64.20%   +0.52%
==========================================
  Files         146      146
  Lines       11960     9295    -2665
==========================================
- Hits         7616     5968    -1648
+ Misses       3878     2858    -1020
- Partials      466      469       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: G-Rath codecov-commenter

repo: google/osv-scanner


text:
Attempt to fix renovate bot grouping

Attempt to fix renovate bot grouping, the previous matchPaths key has been removed. It's not clear if matchRepositories has the same functionality.

Yeah, that might be a better match.


text:
A standalone equivalent as catapult trace viewer

Hi, I'm the developer of viztracer and I have been using catapult trace viewer as the UI for a while. I loved the way it shows trace info but I know that project is already deprecated. So, the obvious way is to move forward to perfetto.
However, there are a couple of issues that I have for using perfetto. I was hoping to get some answers from you guys :)
First of all, is there a way, or will there be a way, to open the trace as a standalone file, or to make the user feel like they are opening a standalone file? The main request here is to have the user "open a file" then there's the trace, instead of "open a link then load the trace file", which is at least three times as hard as "opening a file".
What I'm doing now with catapult, is to user their standalone html version, and customize the html file with all the trace data, then write my own javascript code to load the trace, which was not pretty, but it works. The user can open the html file and the data will be there. Can I do something that has the similar result as this?
Second, I was curious if I could somehow customize panel. One thing that I like about viztracer now is that it can show the source code when I click on the function. This is achieved by hacking the html as well. It took me a while to figure out the point where it can be hooked to the whole trace viewer, but it works.
Both features require a local HTML file to work so I could hack on it, which perfetto does not support for now. I'm totally fine for the requirement of "being online" if the js libraries need to be loaded, but is it possible for us to customize our own local version, or to achieve what I expect with some public APIs(for example, using the url parameters to do some trick)?
Thank you for reading this long issue!

First of all, is there a way, or will there be a way, to open the trace as a standalone file, or to make the user feel like they are opening a standalone file? The main request here is to have the user "open a file" then there's the trace, instead of "open a link then load the trace file", which is at least three times as hard as "opening a file".

Unfortunately there isn't a Web API I am aware of to register a browser as a file handler. Workarounds for now:

Drag and drop the file into ui.perfetto.dev
Script around that, in the same way open_trace_in_browser does here: https://github.com/google/perfetto/blob/master/tools/record_android_trace#L218

The user can open the html file and the data will be there

Generating a HTML file that contains the UI was a terrible thing and really doesn't work with modern HTML/JS api:

Some JS features are only availalble on secure (https) schemes. file:// is not considered secure by modern browsers. The vulcainzed HTML thing is unmaintanable.
Our UI requires a web worker (a dedicated thread) and WebAssembly to work. Neither of them are availlable from a single standalone HTML file.
That causes the UI to be pinned at the version that was used to capture that trace, which makes bugfixes and new features undiscoverable.
That requires that traces are captured via some proxy host script, which wraps them in a vulcanized thing, but that is hard to enforce in the various ways traces can be captured from the various tools (studio, gpuinspector.dev, adb shell, ...)

Second, I was curious if I could somehow customize panel.

Not right now but making the UI more customizable is on our radar (but will involve writing code outside, not hacking the trace file)

Both features require a local HTML file to work so I could hack on it

Because of the above, generating HTML trace files will never happen, it's just impossible to maintain (and even make work in the first place).

Okay that's understandable.
So it seems like perfetto is able to load a trace file through http server with url parameters? That is helpful I think. I can wrap some scripts to make the user feel like they just need to open an url in Chrome, which is still much better than dragging or selecting files. BTW is there a documentation about various parameters that I can send to perfetto(for example, this url) to make it work differently?
Also, I will follow on the customize panel. When you say writing code outside, do you mean customize perfetto source? That's not how I expect to work. If there's a way to write code without having to clone the perfetto project and modify, that would be interesting. I'm totally fine about writing custom code to make it work.
For now, I have extra information in the trace file(the source files of the program) to visualize it in panel, not sure if that's how it will be in the future, but I look forward to it!
Thanks.

BTW is there a documentation about various parameters that I can send to perfetto(for example, this url) to make it work differently?

Not really unfortunately. I agree we should add it.

do you mean customize perfetto source?

Yes

If there's a way to write code without having to clone the perfetto project and modify, that would be interesting. I'm totally fine about writing custom code to make it work.

I am not following what you are trying to do then. How do you customize a tool without checking out its code?

I am not following what you are trying to do then. How do you customize a tool without checking out its code?

Yeah I know it sounds weird. But for example, it would be a common usage for third party users like me to add our custom panel, with our own data. The output format will always be HTML eventually. Is it possible to feed the panel with only data, not the logic? For example, perfetto can load trace file from http servers and display it, maybe it can load other files, like a md file or an HTML file in an iframe to feed the panel? I'm just brainstorming here. I know there must be other concerns like safety or maintainability on your side. I'm just saying that, if we need to do our own customization on perfetto, then we have to build our own server for the tracer UI right? It would be nice if we can still use ui.perfetto.dev and be able to display some customized information in a panel.

Would be nice if you could spec a bit more concretely your proposal, making very clear:

What's the problem you are trying to solve
How you propose to solve it / mocks / design proposals.

I find hard to brainstorm as-is as I see mainly generic and handwavy requests

It would be nice if we can still use ui.perfetto.dev and be able to display some customized information in a panel.

Which kind of information? cotextual (i.e. based on the selection?)? Static?

You are right, I should be more specific. I'll try to explain my needs first:
I want the user to be able to click on a function entry(which is represented as a rectangle on canvas) and see customized panel, instead of the current default panel(current selection). For now, perfetto and catapult shows name, category, duration, etc. It would be nice if that panel is customizable.
It is definitely doable if I change the source code and run my own customized perfetto server, but it would be nice if I did not have to do that.
One way that I can think of to solve this issue is adding an optional template in trace file. Of course since I'm only using chrome trace format json file, that's what I'm thinking about. I know there's already metadata in chrome trace format, like process name and thread name. If we can add an optional html template in the trace file, then maybe perfetto can render based on the template, instead of the default panel?
Using pure HTML template will give the third party a maximum freedom to customize their own panel, but I'm not the safety expert so I'm not sure if there will be safety issues(XSS or something else). If that's an issue, maybe other templates could be used.
For example, in the json file, we could have
{
  ...
  "templates" : {
    "FEE": "'name': {name}\n 'code': {code}"
  }
}

For each entry with the category "FEE", we extract name and code from the entry and fill the template.
This will likely increase the trace file size, but we could use reference as stack frame is using now.
In this way, anyone can customize their own panel without having to edit the source code of perfetto, and perfetto can keep all the default behaviors as it is for users that do not require this feature.
I would be also interested in know if it would be considered as a security vulnerability if the user can load customized javascript code with trace file. That would give a even more flexibility for developers.

This could be done, but only if limited to text-only. HTML templates is never going to fly on ui.perfetto.dev because this would effectively be an official way to do XSS attacks :). Taking data from untrusted input (a trace) and rendering as html is too problematic security-wise.
Note that there is already support for annotating traces (https://perfetto.dev/docs/analysis/trace-processor#annotations) but again they require to edit the source code.

Okay I can understand the HTML part. What about md? That is like a place between pure text and html, and is safer I think? I would imagine that a common usage was to display the source code. The reason I asked for HTML or even JS was that I wanted to show the full source of a file, but scrolling to the point of this specific function. This is a more specific request, but maybe not out of the way for a generic implementation?
Anyway, it would be nice to have something like this as a data-driven feature. However, I would definitely look into changing source code and build my own custom perfetto server for my usage as well.
Thanks for the explanation!

@gaogaotiantian @primiano
If I understood it right, do we have to download the whole perfetto ui source and embed in my html to show the perfetto UI in my page? I mean, I wish there was something like this with perfetto:
// js
import {Perfetto} from 'perfetto';
---
const traceView = new Perfetto({data: myTraceData, options: someOptions});

traveView.init({bind: someElement(in my html)})

Can I achieve something like above rather than having all the perfetto source code and include them in my project code base?

@binDongKim this request is unfeasible.
There are too many reasons why what you ask is too hard to do:

The UI makes uses workers, workers imply a strict deployment model (to do new Worker('file.js')) where some code is built and bundled in a different piece of JS with a different global scope (worker vs window). That has implications all the way down to the build system
The use of WebAssembly which has other implications on the build system
Most of the dom elements assumes the UI is full screen, the CSS scope is global and so on. and embedding in a component would require a non-trivial rewrite + would be doomed to be constantly broken because nobody would test that workflow.
The presence of a security policy that would be violated by your surrounding app

In general it is NOT a goal of the project to provide an embeddable component that you can drop in a DIV. there is too much complexity and, while doing that is not strictly impossible, has a very high cost and, from my view, an extremely little benefit.
The only suported way to "embed" in some form perfetto is deep linking, as described in
https://perfetto.dev/docs/visualization/deep-linking-to-perfetto-ui

First, thanks for your detailed answer.
In my thought, I don't see much difference between passing the trace data to perfetto and have them create the UI in an embeddable component structure and the deep-linking way you mentioned in a "getting data and build UI" way no matter what you use for that UI(worker, assembly etc). The only difference is that with the deep-linking way, I need to OPEN the perfetto UI page specifically and pass the data through message. The possible design I think of is some sort of a initialize function which I could invoke which is like the main function executed when the page is open and some properties I could pass.
Anyway, I took a look at the deep-linking and it seems just opening a new tab derived from one's page. I thought it was more like a iframe-ish thing at least if it was a embedding. Did I get it right?

It's not an iframe, it's a window.open. iframes are subjected to further restrictions and are generally too dangeous cross-origin from  a security perspective:  you could overlay invisible layers on top of that and trick users into believing they are interacting with ui.perfetto.dev while instead they are interacting with something malicious.

I know it's a window.open and I'm not suggesting it should work as an iframe. What I was trying to say is it should've been like an iframe if someone calls it "embedding".

authors: binDongKim gaogaotiantian primiano

repo: google/perfetto


text:
Error coming from Recaptcha script on Instagram/Facebook browser

Issue description
Starting from June 16th we've started seeing many errors coming from the Recpatcha script when a page was opened from Facebook/Instagram browsers.
This is the error:
null is not an object (evaluating 'J.style')
This is the script version (Recaptcha v2):
https://www.gstatic.com/recaptcha/releases/FDTCuNjXhn1sV0lk31aK53uB/recaptcha__en.js
Couldn't find anything online about this so posting here to generate visibility for this issue.
Thanks!

Up, I'm getting the same problem

up, similar error here, in IOS devices only, receiving only this: {"success": false, "error-codes": ["browser-error"]}

up, same here!

Any news? solutions/workarounds?

Same here. It seems to have started sometime in July and affects the Instagram browser on iOS. Facebook/Messenger browser seems to work fine.

Hey there!
Same issue here. I'm trying to access a page (via instagram profile link) that has Recaptcha V3, but there is no success. It shows browser-error.
IPHONE XS

up, same here

we are losing real clients because of this weird thing....no one is not taking responsibility to fix this! since 3 years.
We are now migrating to cloud flare's Captcha (Turnstile) because of this!!!
Good job Google keep in this way!

authors: AndersonHqds FSDelite alessondejesus amenon btrazzini davit555 pb-devsherrard uriklar

repo: google/recaptcha


text:
Current toolset requirements for building native code?

What is the current minimum toolset requirement for building native code?  From the "VS2005 build environment" forum thread, my understanding is: JDK (README at the top of the tree says minimum 1.5 ), DirectX SDK, Windows SDK v6.1 and Visual Studio 2005.  Can someone confirm or correct this?  I ask because there are recent commits that seem to refer to system #includes that aren't satisfied by the environment described above.  Example: "include <inttypes.h>".   It might be helpful for us to be in sync on the toolsets & versions.  I understand that the goal is to move to a newer version of VS (presumably VS2015 Community) in the near future.  But what are people building native code with now?

Was hoping that the community would chime in.  With no response, I'm going to propose the following:

DirectX SDK (June 2010), MS's "final" release of DXSDK
Windows SDK v6.1, (last release that included qedit.h)
Visual Studio 2015 (Community Edition)

Based on the above, I'm very close to a commit that will update the build environment for windows native code.

This all sounds fine to me as long as VS 2015 (Community Edition) is
something that can be downloaded freely. IIRC there is a way to do that;
although it's not on Microsoft's site.
On Sun, Oct 11, 2015 at 11:56 AM, Keith Fischer notifications@github.com
wrote:

Was hoping that the community would chime in. With no response, I'm going
to propose the following:

DirectX SDK (June 2010), MS's "final" release of DXSDK
Windows SDK v6.1, (last release that included qedit.h)
Visual Studio 2015 (Community Edition)

Based on the above, I'm very close to a commit that will update the build
environment for windows native code.
—
Reply to this email directly or view it on GitHub
https://github.com/google/sagetv/issues/38#issuecomment-147235178.

--
Jeffrey Kardatzke
jkardatzke@google.com
Google, Inc.

VS2015 Community Edition is a free download.  It includes MS's refactored libraries, full toolset and integration with Git.  I've successfully migrated the Solution, got everything to build in VS2015 and am running the resultant binaries (exe, dll and ax).  I've also brought the Directshow BaseClasses into third_party\Microsoft so that it automatically builds within the Solution.  Doing so avoids the old way of having to manually copy from SDK 6.1 & manually build them "someplace else",  It insures that the toolset settings are consistent across the entire Solution.  As part of the commit, there is a HOWTO that details exactly which environment variables, SDKs, etc. are needed and where to download and set them up.   I plan to generate a pull request as soon as I understand exactly how to use Git's branching model.

Awesome work so far. :)
On Mon, Oct 12, 2015 at 6:38 PM, Keith Fischer notifications@github.com
wrote:

VS2015 Community Edition is a free download. It includes MS's refactored
libraries, full toolset and integration with Git. I've successfully
migrated the Solution, got everything to build in VS2015 and am running the
resultant binaries (exe, dll and ax). I've also brought the Directshow
BaseClasses into third_party\Microsoft so that it automatically builds
within the Solution. Doing so avoids the old way of having to manually copy
from SDK 6.1 & manually build them "someplace else", It insures that the
toolset settings are consistent across the entire Solution. As part of the
commit, there is a HOWTO that details exactly which environment variables,
SDKs, etc. are needed and where to download and set them up. I plan to
generate a pull request as soon as I understand exactly how to use Git's
branching model.
—
Reply to this email directly or view it on GitHub
https://github.com/google/sagetv/issues/38#issuecomment-147569990.

--
Jeffrey Kardatzke
jkardatzke@google.com
Google, Inc.

Posted the info about completed migration to VS2015 to http://forums.sagetv.com/forums/showthread.php?t=62588

repo: google/sagetv


text:
could not dlopen DSO: libcupti.so.9.0

Issues:
2018-10-07 12:40:45.846093: I tensorflow/stream_executor/dso_loader.cc:141] Couldn't open CUDA library libcupti.so.9.0. LD_LIBRARY_PATH: /usr/local/cuda-9.0/lib64
2018-10-07 12:40:45.846370: F tensorflow/stream_executor/lib/statusor.cc:34] Attempting to fetch value instead of handling error Failed precondition: could not dlopen DSO: libcupti.so.9.0; dlerror: libcupti.so.9.0: cannot open shared object file: No such file or directory
Solution:
export LD_LIBRARY_PATH="/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-9.0/lib64:/usr/local/cuda-9.0/extras/CUPTI/lib64"

That worked ! thanks.

Issues:
2018-10-07 12:40:45.846093: I tensorflow/stream_executor/dso_loader.cc:141] Couldn't open CUDA library libcupti.so.9.0. LD_LIBRARY_PATH: /usr/local/cuda-9.0/lib64 2018-10-07 12:40:45.846370: F tensorflow/stream_executor/lib/statusor.cc:34] Attempting to fetch value instead of handling error Failed precondition: could not dlopen DSO: libcupti.so.9.0; dlerror: libcupti.so.9.0: cannot open shared object file: No such file or directory
Solution:
export LD_LIBRARY_PATH="/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-9.0/lib64:/usr/local/cuda-9.0/extras/CUPTI/lib64"

thanks a ton, marked!

authors: HareshMiriyala Zoomfly nlp4whp


text:
How to build a debug version of the library without having the variable and function names changed by closure compiler?

Have you read the Tutorials?
Yes
Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
2.x.x and 3.x.x
Please ask your question
How to build a debug version of the library without having the variable and function names changed by closure compiler?
More info
I would like to create a debug build which makes debugging easier, in order to investigate and fix bugs in the Player.
In my effort to do so, I found out that the configuration I need to change is located in the file build/build.py. More specifically, I have changed the compilation type from SIMPLE to BUNDLE in debug_closure_opts.

By doing so, the compiler leaves the original variable and function names as I would like to, BUT it doesn't handle the following function calls.

Any help would be appreciated 🙂
Thanks!

You can use ./build/build.py --debug to create a build with the original names.  You can also use shaka-player.compiled.debug.js or shaka-player.ui.debug.js  in the pre-built releases.

You can use ./build/build.py --debug to create a build with the original names.  You can also use shaka-player.compiled.debug.js or shaka-player.ui.debug.js  in the pre-built releases.

If the renaming of local variables in the debug build is an issue, check out the debugging tutorial, which has a section called "Advanced: Loading the uncompiled library".  Loading the full, uncompiled library is what the Shaka team does when we are debugging Shaka Player.

If the renaming of local variables in the debug build is an issue, check out the debugging tutorial, which has a section called "Advanced: Loading the uncompiled library".  Loading the full, uncompiled library is what the Shaka team does when we are debugging Shaka Player.

You can use ./build/build.py --debug to create a build with the original names. You can also use shaka-player.compiled.debug.js or shaka-player.ui.debug.js in the pre-built releases.

Thanks a lot for the suggestion @TheModMaker !
The player.compiled.debug.js is the one that I usually use. But all variables are "uglyfied" (not use about 2.x.x versions but for 3.x.x for sure), so this is not an option.
Using ./build/build.py --debug makes things a bit better but the variable names get replaced.

If the renaming of local variables in the debug build is an issue, check out the debugging tutorial, which has a section called "Advanced: Loading the uncompiled library". Loading the full, uncompiled library is what the Shaka team does when we are debugging Shaka Player.

@joeyparrish thanks for pointing out the Advanced topic, being able to load the uncompiled library will be great!
Closing the issue to keep the issue tidy, will re-open if needed.

You can use ./build/build.py --debug to create a build with the original names. You can also use shaka-player.compiled.debug.js or shaka-player.ui.debug.js in the pre-built releases.

Thanks a lot for the suggestion @TheModMaker !
The player.compiled.debug.js is the one that I usually use. But all variables are "uglyfied" (not use about 2.x.x versions but for 3.x.x for sure), so this is not an option.
Using ./build/build.py --debug makes things a bit better but the variable names get replaced.

If the renaming of local variables in the debug build is an issue, check out the debugging tutorial, which has a section called "Advanced: Loading the uncompiled library". Loading the full, uncompiled library is what the Shaka team does when we are debugging Shaka Player.

@joeyparrish thanks for pointing out the Advanced topic, being able to load the uncompiled library will be great!
Closing the issue to keep the issue tidy, will re-open if needed.

authors: TheModMaker joeyparrish spyros-uk

repo: google/shaka-player


text:
Page Templates: Hide Save new template button for blank page

Feature Description
Currently, the button for saving a new page template is shown for a blank page even though a blank page can't be saved as a button. Let's hide the button for a blank page.
We can also remove the snackbar logic currently used for displaying information when trying to add a blank page since this will not be even possible anynore.

Alternatives Considered

Additional Context

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria

Implementation Brief

Verified in QA

authors: csossi miina

repo: google/web-stories-wp


text:
Delete Element By Dragging Off Page

Feature Description
As an editor I want items that I drag off the page and onto the workspace to be deleted
UX

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria

When elements are dragged halfway off teh page ,the elements are all clipped in the workspace (see UX link)
Once elements are dragged off the work space and the user "let's go" the elements are deleted.

QA Instructions

FI: There's a duplicate: https://github.com/google/web-stories-wp/issues/445

authors: jauyong miina


text:
Dashboard: Add List View UI to Dashboard

Add list view to dashboard

#770

@BrittanyIRL Fixed the sizing issues and updated the code to use the index imports and theme!

@carlos-kelly As mentioned on Slack, this PR caused an issue with the storybook, specifically assets/src/dashboard/components/table/stories/table.js did. Deleting that file resolves the issue, if that helps as a hint.
Note: the error only happens with npm run build:storybook, not npm run storybook.

authors: carlos-kelly swissspidy


text:
Page Layouts: Only run isDefaultPage if currentPage is not null

Summary
This should fix flaky e2e tests as seen in #5976
When the story is still being loaded, currentPage will be null, whereas isDefaultPage expects an object.
Relevant Technical Choices

To-do

User-facing changes

Testing Instructions

Fixes #

That's not a scenario I expected!

That's not a scenario I expected!

authors: swissspidy zachhale


text:
Add FromBytes::read_from_io and IntoBytes::write_to_io

Makes progress on #158.

Codecov Report
Attention: Patch coverage is 0% with 18 lines in your changes missing coverage. Please review.

Project coverage is 87.32%. Comparing base (1fb6365) to head (29fb787).

Files with missing lines
Patch %
Lines

src/lib.rs
0.00%
18 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           v0.8.x    #2016      +/-   ##
==========================================
- Coverage   87.59%   87.32%   -0.27%
==========================================
  Files          16       16
  Lines        5988     6006      +18
==========================================
  Hits         5245     5245
- Misses        743      761      +18

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Backporting in https://github.com/google/zerocopy/pull/2046

authors: codecov-commenter joshlf jswrenn

repo: google/zerocopy


text:
NoSuchMethodError after upgrading to v 14.0.0

Which version of the client library are you using?
14.0.0
Which version of Java are you using?
11
Which operating system (Linux, Windows, ...) and version?
Mac Catalina
Actions taken
I have upgraded all referenced to the API to use version 8 and when I run
googleAdsClient.getLatestVersion().createAdGroupCriterionServiceClient()
I get
java.lang.NoSuchMethodError: 'com.google.api.gax.rpc.StubSettings$Builder com.google.ads.googleads.v8.services.stub.AdGroupCriterionServiceStubSettings$Builder.setMtlsEndpoint(java.lang.String)'
at com.google.ads.googleads.v8.services.stub.AdGroupCriterionServiceStubSettings$Builder.createDefault(AdGroupCriterionServiceStubSettings.java:258) ~[google-ads-14.0.0.jar:na]
at com.google.ads.googleads.v8.services.stub.AdGroupCriterionServiceStubSettings$Builder.access$000(AdGroupCriterionServiceStubSettings.java:183) ~[google-ads-14.0.0.jar:na]
at com.google.ads.googleads.v8.services.stub.AdGroupCriterionServiceStubSettings.newBuilder(AdGroupCriterionServiceStubSettings.java:162) ~[google-ads-14.0.0.jar:na]
at com.google.ads.googleads.v8.services.AdGroupCriterionServiceSettings$Builder.createDefault(AdGroupCriterionServiceSettings.java:165) ~[google-ads-14.0.0.jar:na]
at com.google.ads.googleads.v8.services.AdGroupCriterionServiceSettings$Builder.access$000(AdGroupCriterionServiceSettings.java:145) ~[google-ads-14.0.0.jar:na]
at com.google.ads.googleads.v8.services.AdGroupCriterionServiceSettings.newBuilder(AdGroupCriterionServiceSettings.java:127) ~[google-ads-14.0.0.jar:na]
at com.google.ads.googleads.lib.catalog.GeneratedCatalog$V8Client.createAdGroupCriterionServiceClient(GeneratedCatalog.java:4100) ~[google-ads-14.0.0.jar:na]
at com.wayfair.adtech.adverts.services.google.upload.keywords.KeywordGoogleUploadService.createKeywords(KeywordGoogleUploadService.java:68) ~[classes/:na]
Expected result
The code should return AdGroupCriterionServiceClient googleClient

It's likely that you have a conflicting version of some of our dependencies.
Here is the list of the expected dependencies for v14.0.0. Can you check your dependencies against this?
The specific depedency that you have an issue with is gax, so adding a direct dependency on com.google.api:gax:1.65.1 should do the trick.

Thanks, upgrading gax and gax-grpc to 1.66.0 resolved the issue

Glad to hear it, thanks for confirming.

authors: nwbirnie visl

repo: googleads/google-ads-java


text:
Ads are Skipped on iPad 2 & iPad Air

We are testing ads on videoJS and due to some reason ads are skipped on iPad devices. You can see the issue below URL.
http://dmscg.com/tech/video/jsplayer_v1_11_0/examples/simple/
iPad air browsers
Chrome: Version 92.0.4515.159
Firefox: Version 89.0
iPad: Air 2, iOS 14.6
iPad 2, iOS 14.6
Safari browser
Anyone else faced the same problem?
Thanks

Hello @abi3 ,
Have you tried calling disableCustomPlaybackForIOS10Plus(true)? More information is available in the plugin's readme section: additional settings.
Thank you,
Jackson
IMA SDK. team

@Kiro705 Thank you for the response. I tried the same but still ad is skipped by the player.

Any update on this issue?

Hello @abi3 ,
Sorry for the delay on this issue. Are there any error messages you are seeing related to the ad not initiating?
My initial test of the shared page on an iPad Air was able to successfully play ads.
Jackson
IMA SDK team

repo: googleads/videojs-ima


text:
Videojs IMA Chromecast

Hello everybody:
I have a web page and my player and my ads is working but when I try send to the chromecast with the plugin silrvermine-videojs-chromecast the ads play in my browser and the video play in my chromecast and I want that all play it at the chromecast
https://sportsvr.tv
Thanks for all
Best regards

Hi,
This is beyond the scope of this issue tracker, we don't support the issues related to the videojs-chromecast  plugin. However, the IMA SDK which is the underlying SDK of our plugin also provides the IMA SDK for Cast that you can use to integrate ads display feature in your Cast app.

Hi,
I’m closing this issue because it has been inactive for more than a month. Please reopen if you still encounter this issue with the latest stable version
Thank you!

authors: arnaudcasame jalowin


text:
Enable multiple config patterns for batch generation

This PR enables Artman to batch generate clients from multiple paths
Artman config PR: https://github.com/googleapis/googleapis/pull/349

Codecov Report

Merging #194 into master will increase coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #194      +/-   ##
==========================================
+ Coverage   69.68%   69.72%   +0.04%
==========================================
  Files          48       48
  Lines        1979     1982       +3
  Branches      232      233       +1
==========================================
+ Hits         1379     1382       +3
  Misses        567      567
  Partials       33       33

Impacted Files
Coverage Δ

artman/pipelines/batch_generation.py
96.61% <100%> (+0.18%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 203c16e...c0ca8b6. Read the comment docs.

What is the API syntax for this?

@lukesneeringer This PR enables batch generation to include core/IAM packages (You can look at the artman config PR for more details). Right now for Java staging process, we still need to run extra command to generate those packages.

authors: codecov-io lukesneeringer shinfan

repo: googleapis/artman


text:
chore: release 2.1.0

:robot: I have created a release *beep* *boop*
2.1.0 (2019-08-07)
Bug Fixes

deps: use the latest extend (#523) (f665938)
deps: use the latest extend (#525) (1b24522)

Features

breaking!: support API endpoint override (#509) (9bd110e)

This PR was generated with Release Please.

Closing this PR.
We're not releasing right now, and this version includes a breaking change.

repo: googleapis/cloud-profiler-nodejs


text:
Regenerate RuntimeConfig client

This PR was generated using Autosynth. :rainbow:
Here's the log from Synthtool:
synthtool > Executing /tmpfs/src/git/autosynth/working_repo/synth.py.
synthtool > Cloning https://github.com/googleapis/elixir-google-api.git.
synthtool > Running: docker run --rm -v/home/kbuilder/.cache/synthtool/elixir-google-api:/workspace -v/var/run/docker.sock:/var/run/docker.sock -e USER_GROUP=1000:1000 -w /workspace gcr.io/cloud-devrel-public-resources/elixir16 scripts/generate_client.sh RuntimeConfig
synthtool > Wrote metadata to clients/runtime_config/synth.metadata.

Closing so this will regenerate with the otp_app fix.

authors: dazuma yoshi-automation

repo: googleapis/elixir-google-api


text:
ECP subprocess improvements for Windows OS

Currently, we check for pid of parent to determine when to terminate the ecp subprocess, but this may not work correctly for Windows. Need to verify and use windows specific api if needed for cleaning up ecp subprocess.
https://github.com/googleapis/enterprise-certificate-proxy/blob/main/internal/signer/windows/signer.go

Fixed in https://github.com/googleapis/enterprise-certificate-proxy/pull/74

repo: googleapis/enterprise-certificate-proxy


text:
Add gapic-common gem

Replaces google-gax 2.0 in order to avoid dependency conflicts.

@dazuma Should we remove the coverage stuff before merging? Or would you like to have this on all PRs moving forward?

repo: googleapis/gapic-generator-ruby


text:
Support new resource annotations

@andreamlin PTAL. Still some work to do:

Fix ResourceNameMessageConfigsTest
Documentation of new ResourceDescriptorConfig class
Release common protos package

@andreamlin PTAL

We can use proto-google-common-protos v1.16.0 now! :)

Maybe the version for proto-google-common-protos should be changed everywhere to the newest? Including in the circle ci config for the showcase stuff

Oh no I bet that the newest google common protos will break showcase, and the showcase generation won't work until https://github.com/googleapis/gapic-generator/pull/2646 is updated in the same manner it had been.
I would be ok with disabling the showcase generation for now, with an issue/TODO filed to restore it. We've whitelisted tests before.

LGTM after showcase issue comment is resolved.

I'm a little sad about the loss of functionality, but it sounds like we can re-add most of it via the gapic config v2 later. This was as good as could be done given the design constraints, thanks!

authors: andreamlin michaelbausor

repo: googleapis/gapic-generator


text:
Investigate Path template performance

Recent investigation by @blowmage found that GAPIC utility methods that rely on the GAX path template code are very slow, to the point that they are the bottleneck for some google-cloud-ruby acceptance tests. This is likely due to the use of rly to implement template parsing.
Investigate these performance concerns. If rly is the issue, addressable may be a good substitute, although be cautious of differences between Google's path templates and RFC 6570. Another approach might be to drop parsing and use basic string interpolation, which @pongad has already done in the Go GAPICs.

Python also has just string interpolation (with a touch of validation). It's likely fairly trivial to port the Python implementation to Ruby: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/api_core/google/api_core/path_template.py

Python also dropped the use of a full-blown parser and its path template module is extremely small, simple, and well tested: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/api_core/google/api_core/path_template.py
I don't think we should deprecated them, I think we should provide them as helpers to construct strings in the correct format.

@jonparrott I'd bet you'd prefer that all things follow Python's approach though 😄
I think we'll have a complication either way here. These are currently implemented as class methods (IIRC this was why @frankyn didn't like them), which I do think looks weird and often verbose. If we made them available as instance methods it would look exactly like Python, but we'd probably have to keep both around for awhile and it does introduce the (remote) possibility for things like name collisions.
I just have a hard time believing that folks really needs these things (i.e. in the pub sub docs only 3/7 languages use them). @blowmage do you think we'll get complaints if we don't have these, or have we gotten issues/complaints related to this in the past?
Let's also check with our fellow Python friend @lukesneeringer - he had a long conversation about this the other day and I'd be curious what he'd suggest we do.

@jonparrott I'd bet you'd prefer that all things follow Python's approach though 😄

Bias aside, the Python implementation was well-thought out and intentionally intended to provide a basis for other languages and was reviewed by several others (including @lukesneeringer). This actually goes for all of api core.

I just have a hard time believing that folks really needs these things (i.e. in the pub sub docs only 3/7 languages use them)

I agree. I don't think anyone needs them, but they are a bit useful to have around.

These are currently implemented as class methods (IIRC this was why @frankyn didn't like them), which I do think looks weird and often verbose. If we made them available as instance methods it would look exactly like Python, but we'd probably have to keep both around for awhile and it does introduce the (remote) possibility for things like name collisions.

I have no strong feelings one way or the other here.

do you think we'll get complaints if we don't have these, or have we gotten issues/complaints related to this in the past?

It depends on what you mean by "these". Do you mean the _path methods on the GAPIC client classes? Or the Google::Gax::PathTemplate class?

Both.
I think we have to do something about PathTemplate if it's so slow. Given that, I'm suggesting we deprecate and stop using the _path helpers as others have done rather than re-implement them.
It seems like something that's better solved by documentation rather than code, which has to be documented and explained in addition. Just to use a concrete example, what do you think of this?
https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/master/google-cloud-logging/lib/google/cloud/logging/v2/logging_service_v2_client.rb#L280
To me, dropping the helpers here would be an improvement and a useful simplification. However, I don't want to drop them if you're really sensing that folks need the helpers. Or, maybe there are more complex cases where they are genuinely useful?

If we have any GA clients using Path templates then we can not deprecate or remove them.
I'm certainly in favor of just changing the implemention. The Python one took less than a day.

I'm sure we have GA clients that use them.
The time to change is not an issue. To recap, there are two potential problems here:

poor performance
currently implemented as class methods

I'm still hung up on the second item and I've pinged @frankyn to weigh in. This issue is causing us to intentionally not use the helper methods in favor of strings in sample code. If we only consider the first issue then it's pretty simple and we could just change the internal implementation, but I'd like to find a way to solve both problems if we can. Duplicating the helpers methods is an obvious solution, but I'm not sure it's a good one.

FWIW, my thinking on helper methods in Ruby is that they are there as a convenience to the programmer. If the method is not in the most convenient place, then add it. Convenience is more important than duplication.
DATABASE_PATH_TEMPLATE = Google::Gax::PathTemplate.new(
  "projects/{project}/instances/{instance}/databases/{database}"
)

private_constant :DATABASE_PATH_TEMPLATE

# Returns a fully-qualified database resource name string.
# @param project [String]
# @param instance [String]
# @param database [String]
# @return [String]
def self.database_path project, instance, database
  DATABASE_PATH_TEMPLATE.render(
    :"project" => project,
    :"instance" => instance,
    :"database" => database
  )
end

# Returns a fully-qualified database resource name string.
# @param project [String]
# @param instance [String]
# @param database [String]
# @return [String]
def database_path project, instance, database
  self.class.database_path project, instance, database
end

Thanks @blowmage that's great input.
Let's wait to confirm w/ Frank that this will unblock the sample code issue. If that works out and you're okay with the duplication then I'm happy to push this through as outlined above.

@jbolinger As you said, there are really two issues. I believe they are orthogonal and can be solved independently. Therefore I propose we move the helper method convenience discussion to #125.

This afternoon @quartzmo and I paired on a spike of implementing the python code, to see if it was a viable option. Sadly, the approach in the python code cannot 100% replace the functionality provided by Google::Gax::PathTemplate. With this in mind, we have come to the conclusion that there are 3 major options to take:

Replace Google::Gax::PathTemplate with a new class - this should deliver a boost in performance, but will also include introducing API incompatibility. Specifically, Google::Gax::PathTemplate#match doesn't seem to be easily provided by the regexp approach.
Leave Google::Gax::PathTemplate alone, and simply stop using it for the simple templates used by most of the path helper methods. All of the path templates we have checked so far in the generated GAPIC code have used named single segments. It should be possible to use the simple ruby string interpolation proposed in an earlier comment in these cases. Any future helper method that used a template with multi segment patterns could use the slower Google::Gax::PathTemplate.
Leave Google::Gax::PathTemplate alone, introduce a separate class based on the python regexp code. This would maintain backwards compatibility of the GA's google-gax library, while allowing helper methods to switch to using a different class with increased performance.

Responding to second item in #125.
As for performance concerns, removing the underlying implementation for the Python version sounds acceptable, but we may still want to verify performance.

The python code implements a simple validation of a path string to a template. The Google::Gax::PathTemplate#match method validates a path string to the template, and returns the parameters taken from the path that look like they could be sent to Google::Gax::PathTemplate#render to generate the path again. I'm not sure the regexp approach will enable this behavior.
I say "look like they could be sent to", since we also noticed some unexpected behavior in #match, which is detailed in #126.
I find the behavior of Google::Gax::PathTemplate#match to be surprising in other ways as well. If the path doesn't match the method raises an error. I expected it would behave similar to Regexp#match, which will return nil when it doesn't match instead of raising.

I might be thinking about this too naively, but if you have a/{b}/c/{d}, isn't it just regexp a/[^/]*/c/[^/]*? You could use capture groups to take out the params right? I also resonded to #126.

I suspect that one of the reasons this gem has a dependency on the rly gem is because it needs a full lexer to do lookahead. I mean, I don't know that for sure, but that is what I suspect.
Can you use capture groups for a template like v1/*/{name=parent/**}?

We found in python that while there was some utility in validating and
expanding path template, literally no one was parsing them.
I would offer up that we could drop the parsing feature.
On Fri, May 4, 2018, 4:50 PM Mike Moore notifications@github.com wrote:

I suspect that one of the reasons this gem has a dependency on the rly
gem is because it needs a full lexer to do lookahead. I mean, I don't know
that for sure, but that is what I suspect.
Can you use capture groups for a template like v1/*/{name=parent/**}?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/googleapis/gax-ruby/issues/119#issuecomment-386762030,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc4Q_BUBerZQuopA2radF-SHOMH7gks5tvOk9gaJpZM4SUGCd
.

I would be cool dropping features, ala option #1, but the gem already GA, so API compatibility would be a consideration IMO.

I can't comment on how disruptive things would be if we drop the feature, but the capture group for that template should be like v1/([^/]*)/(parent/.*) right?
I don't think any of these template actually needs full-featured LALR parsers. I think lex/yacc was used to parse the path template (v1/*/{name=parent/**}, etc) themselves, not the strings that our generated functions are supposed to match. I think it's possible to turn the path template into a regexp while generating the client, though I haven't thought much about it.

My understanding is this issue is still in progress.
(Replying to bring the issue back into compliance.)

@jbolinger Any updates?

I'm going to close this. The plan is to rewrite path template handling so there is no need to examine the performance any more than we already have.

authors: blowmage eoogbe frankyn geigerj jbolinger jonparrott lukesneeringer pongad

repo: googleapis/gax-ruby


text:
add path for client_secrets.json in sample_tools.py

I add path for client_secrets.json in sample_tools.py, because we have some accounts then we need to prepare some client_secrets.json.
To use the same script with several json files, I made each directory and set env files in the directories.

This PR is stale, and needs attention.  @t-Asai, do you want to follow up on comments, or should we close this?

Closing this as the original author @t-Asai has not responded. Please feel free to re-open.

authors: busunkim96 sduskis t-Asai

repo: googleapis/google-api-python-client


text:
Example code not working on google compute engine, returns Forbidden message

Environment details

OS: Debian 9
Node.js version: 10150
npm version: 6.6.0
google-auth-library version: 3.0.1

Steps to reproduce

Create a compute engine in one of your gcloud projects with the above configuration.
Put in the example code from README.md:

const {auth} = require('google-auth-library');

/**
 * Instead of specifying the type of client you'd like to use (JWT, OAuth2, etc)
 * this library will automatically choose the right client based on the environment.
 */
async function main() {
    const client = await auth.getClient({
        scopes: 'https://www.googleapis.com/auth/cloud-platform'
    });
    const projectId = await auth.getProjectId();
    const url = `https://www.googleapis.com/dns/v1/projects/${projectId}`;
    const res = await client.request({ url });
    console.log(res.data);
}

main().catch(console.error);

Run the code node <filename>
Instead of the API working the code throws an error:

{ Error: A Forbidden error was returned while attempting to retrieve an access token for the Compute Engine built-in service account. This may be because the Compute Engine instance does not have the correct permission scopes specified. Request had insufficient authentication scopes.
    at Gaxios.<anonymous> (/home/victor/test-iam/node_modules/gaxios/build/src/gaxios.js:64:27)
    at Generator.next (<anonymous>)
    at fulfilled (/home/victor/test-iam/node_modules/gaxios/build/src/gaxios.js:16:58)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Great to hear things are working for you.  Assigning the right permissions to the GCE service account for the VM sounds like the right thing to do.  There's not a lot we can do with this lib to improve that experience.  We just read the service accounts the metadata service gives us :)
If there's something still broken here - let us know!

authors: JustinBeckwith victorbadila

repo: googleapis/google-auth-library-nodejs


text:
feat: Skeleton for spanner::Client

Fixes #29

This change is

The doxygen stuff isn't perfect but let's fix that later so we can unblock people.
I'm not sure why the comment only shows up for one overload, maybe I read the manual incorrectly.

On Thu, Aug 1, 2019 at 10:00 PM Todd Derr notifications@github.com wrote:

@mr-salty commented on this pull request.
Reviewed 1 of 4 files at r1, 3 of 5 files at r2, 1 of 1 files at r3.
Reviewable
https://reviewable.io/reviews/googleapis/google-cloud-cpp-spanner/276
status: 5 of 6 files reviewed, 10 unresolved discussions (waiting on
@coryan https://github.com/coryan, @devbww https://github.com/devbww,
@devjgm https://github.com/devjgm, @mr-salty
https://github.com/mr-salty, @remyabel https://github.com/remyabel,
@return https://github.com/return, @returns https://github.com/returns,
and @scotthart https://github.com/scotthart)

google/cloud/spanner/client.cc, line 97 at r1
https://reviewable.io/reviews/googleapis/google-cloud-cpp-spanner/276#-LlAhsW8swFkh_8Ti5Q9-r1-97:-LlF1uOKCBAdR-hjlWMS:bc37w3h
(raw file
https://github.com/googleapis/google-cloud-cpp-spanner/blob/4f8ee1352e4cb903f5a946ba42f89ebc2a4ccf39/google/cloud/spanner/client.cc#L97):
Previously, devjgm (Greg Miller) wrote…
I vote to remove them, because I don't think we should spend time working
on them until the higher-priority alpha stuff is done. These are empty
function bodies, so we wouldn't be removing much.
If you'd rather keep them, please put a comment here reminding future us
to ignore these until alpha is done.
I added a warning comment... personally I'd rather have an indication that
something is intentionally missing than having to wonder if that's the case
or if it was an oversight. As one example from this afternoon, Brad and I
were discussing Session management and weren't sure why there were no
labels in ClientOptions - intentional or not? Then it derailed us for a
while trying to figure that out.

OK. Please add the comment to the .cc file too. The comment is for us,
not customers.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/googleapis/google-cloud-cpp-spanner/pull/276?email_source=notifications&email_token=ACICRW75TQYXUNIUYSPXQV3QCOIL3A5CNFSM4IIM62C2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCALS7PA#pullrequestreview-269955004,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACICRW7OO62RULKSXG7JUSLQCOIL3ANCNFSM4IIM62CQ
.

authors: devjgm mr-salty

repo: googleapis/google-cloud-cpp-spanner


text:
impl(bigquery): Add JobStatistics to Job and ListFormatJob

This PR adds the all the new statistical classes from previous PRs to the Job and ListFormatJob resource.
I'll do all the DebugLogging changes in separate PRs as they have tendency to bloat the PR size.

This change is

Thank you for all the reviews thus far. Much appreciated!

repo: googleapis/google-cloud-cpp


text:
ci: run integration tests with Windows+Bazel

Fixes #3540

This change is

Ping, this may have been missed in the shuffle.


text:
spanner: add benchwrapper.

This is a benchwrapper for spanner, which is similar to the existing storage-benchwrapper.

This is mainly used for benchmarking tests.
It exposes a gRPC service for Cloud Spanner client libraries, which contains three basic operations, i.e., read, insert, and update.
The uniform gRPC service makes the caller easy to benchmark Cloud Spanner client libraries in different languages.

Hi @elharo, I fixed the issue but I wonder why codecov reports a decrease of coverage. I did not modify those files in my commit. How can I rerun the codecov check again?

some more nits.
You should also open a bug and get commitment from the owners that this is something they want to add to the repo. They might not.

I am from the Cloud Spanner client library team and we are running benchmarking tests regularly across all spanner client libraries. The go version has been done in https://github.com/googleapis/google-cloud-go/tree/master/spanner/internal/benchwrapper. So we are working on expanding it to other client libraries.

repo: googleapis/google-cloud-java


text:
[securitycenter] v2 client's apiEndpoint property is wrong when a regional endpoint is specified

Please make sure you have searched for information in the following guides.

[X] Search the issues already opened: https://github.com/GoogleCloudPlatform/google-cloud-node/issues
[X] Search StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js
[X] Check our Troubleshooting guide: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/troubleshooting
[X] Check our FAQ: https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/faq
[X] Check our libraries HOW-TO: https://github.com/googleapis/gax-nodejs/blob/main/client-libraries.md
[X] Check out our authentication guide: https://github.com/googleapis/google-auth-library-nodejs
[X] Check out handwritten samples for many of our APIs: https://github.com/GoogleCloudPlatform/nodejs-docs-samples

A screenshot that you have tested with "Try this API".
n/a
Link to the code that reproduces this issue. A link to a public Github Repository or gist with a minimal reproduction.
https://gist.github.com/hegemonic/cf807026d8b83c14aa0b225b16967599
A step-by-step description of how to reproduce the issue, based on the linked reproduction.
Run the code. It creates a v2 SecurityCenterClient; sets the apiEndpoint to securitycenter.me-central2.rep.googleapis.com; and prints the expected and actual apiEndpoint for the client.
A clear and concise description of what the bug is, and what you expected to happen.
The code prints the following:
expected apiEndpoint: securitycenter.me-central2.rep.googleapis.com
actual apiEndpoint: securitycenter.googleapis.com
client._opts.apiEndpoint: securitycenter.me-central2.rep.googleapis.com

I expected it to print the following:
expected apiEndpoint: securitycenter.me-central2.rep.googleapis.com
actual apiEndpoint: securitycenter.me-central2.rep.googleapis.com
client._opts.apiEndpoint: securitycenter.me-central2.rep.googleapis.com

A clear and concise description WHY you expect this behavior, i.e., was it a recent change, there is documentation that points to this behavior, etc. **

If I specify an API endpoint, the client object shouldn't say that it's using the default endpoint.
You shouldn't need to look at private properties to figure out which endpoint the client is using.

@sofisl Why was this bug closed? I provided a valid link to a publicly visible gist.

repo: googleapis/google-cloud-node