text:
Verilator compile in Conda env can't find the compiler

To reproduce: go to this colab: https://colab.research.google.com/drive/1_GlX-pO4rune8GMIK4q_IuhxpIhnPTni?resourcekey=0-jMbF8wzg0csZu0fxpOy78A&usp=sharing (currently only readable within Google)
Then select "Runtime" --> "Run all"
Output:
make -j -C /content/CFU-Playground/soc/build/sim.proj_template_v/gateware/obj_dir -f Vsim.mk Vsim
make[3]: Entering directory '/content/CFU-Playground/soc/build/sim.proj_template_v/gateware'
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
Vsim.mk:13: warning: NUL character seen; rest of line ignored
ccache x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /content/CFU-Playground/env/conda/envs/cfu-common/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /content/CFU-Playground/env/conda/envs/cfu-common/include -I.  -MMD -I/content/CFU-Playground/env/conda/envs/cfu-common/share/verilator/include -I/content/CFU-Playground/env/conda/envs/cfu-common/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=1 -DVM_TRACE_FST=1 -faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /content/CFU-Playground/env/conda/envs/cfu-common/include -ggdb -Wall -O0   -DTRACE_FST -I/content/CFU-Playground/third_party/python/litex/litex/build/sim/core  -std=gnu++14 -Os -c -o veril.o /content/CFU-Playground/third_party/python/litex/litex/build/sim/core/veril.cpp
ccache: error: Could not find compiler "x86_64-conda-linux-gnu-c++" in PATH
Vsim.mk:71: recipe for target 'veril.o' failed
make[3]: *** [veril.o] Error 1
make[3]: Leaving directory '/content/CFU-Playground/soc/build/sim.proj_template_v/gateware/obj_dir'
/content/CFU-Playground/third_party/python/litex/litex/build/sim/core/Makefile:38: recipe for target 'sim' failed
make[2]: *** [sim] Error 2
make[2]: *** Waiting for unfinished jobs....
make[4]: Leaving directory '/content/CFU-Playground/soc/build/sim.proj_template_v/gateware/modules/gmii_ethernet'
cp gmii_ethernet/gmii_ethernet.so gmii_ethernet.so
make[3]: Leaving directory '/content/CFU-Playground/soc/build/sim.proj_template_v/gateware/modules'
make[2]: Leaving directory '/content/CFU-Playground/soc/build/sim.proj_template_v/gateware'
/content/CFU-Playground/soc/sim.mk:56: recipe for target 'run' failed
make[1]: *** [run] Error 1
make[1]: Leaving directory '/content/CFU-Playground/soc'
../proj.mk:354: recipe for target 'load' failed
make: *** [load] Error 2

To highlight the error:
ccache: error: Could not find compiler "x86_64-conda-linux-gnu-c++" in PATH

Looking under env/conda/envs/cfu-common/bin/, I see:
/content/CFU-Playground/env/conda/bin/x86_64-conda-linux-gnu-ld
/content/CFU-Playground/env/conda/bin/x86_64-conda_cos6-linux-gnu-ld

and nothing else with an x86_64 prefix.

Looking at the earlier warning
Vsim.mk:13: warning: NUL character seen; rest of line ignored

...that line in /content/CFU-Playground/soc/build/sim.proj_template_v/gateware/obj_dir/Vsim.mk is suspicious:
VERILATOR_ROOT = /content/CFU-Playground/env/conda/envs/cfu-common/share/verilator��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������

There is a similar issue when running locally directly (not in colab) in the Conda env.   The error is still
ccache: error: Could not find compiler "x86_64-conda-linux-gnu-c++" in PATH

However, looking in $CFU_ROOT/env/conda/envs/cfu-common/bin, there are now many executables, but with a slightly different prefix including "cos6":
x86_64-conda_cos6-linux-gnu-addr2line     x86_64-conda_cos6-linux-gnu-g++         x86_64-conda_cos6-linux-gnu-ld.bfd
x86_64-conda_cos6-linux-gnu-ar            x86_64-conda_cos6-linux-gnu-gcc         x86_64-conda_cos6-linux-gnu-ld.gold
x86_64-conda_cos6-linux-gnu-as            x86_64-conda_cos6-linux-gnu-gcc-ar      x86_64-conda_cos6-linux-gnu-nm
x86_64-conda_cos6-linux-gnu-c++           x86_64-conda_cos6-linux-gnu-gcc-nm      x86_64-conda_cos6-linux-gnu-objcopy
x86_64-conda_cos6-linux-gnu-cc            x86_64-conda_cos6-linux-gnu-gcc-ranlib  x86_64-conda_cos6-linux-gnu-objdump
x86_64-conda_cos6-linux-gnu-c++filt       x86_64-conda_cos6-linux-gnu-gcov        x86_64-conda_cos6-linux-gnu-ranlib
x86_64-conda_cos6-linux-gnu-cpp           x86_64-conda_cos6-linux-gnu-gcov-dump   x86_64-conda_cos6-linux-gnu-readelf
x86_64-conda_cos6-linux-gnu-ct-ng.config  x86_64-conda_cos6-linux-gnu-gcov-tool   x86_64-conda_cos6-linux-gnu-size
x86_64-conda_cos6-linux-gnu-dwp           x86_64-conda_cos6-linux-gnu-gprof       x86_64-conda_cos6-linux-gnu-strings
x86_64-conda_cos6-linux-gnu-elfedit       x86_64-conda_cos6-linux-gnu-ld          x86_64-conda_cos6-linux-gnu-strip

To reproduce locally, set up CFU Playground according to instructions, then:
make env
make enter
cd proj/proj_template_v
make PLATFORM=sim load

This looks similar: https://stackoverflow.com/a/46457813 -- although that is more than four years old -- you'd think they'd have fixed the issue by now.

@PiotrZierhoffer - Any idea what is going on here?

@PiotrZierhoffer I gave you edit permission on the colab.
I added symbolic links in the colab to provide the executables where the build expects them to be.  I also figured out that the x86_64 compiler is gcc 7.3, which is quite old and may be related to the new error after adding the links:
x86_64-conda-linux-gnu-c++: error: unrecognized command line option '-fcf-protection=none'; did you mean '-flto-partition=none'?
Vsim.mk:71: recipe for target 'veril.o' failed
make[3]: *** [veril.o] Error 1

To see the old behavior, comment out the ln -s ... lines in the last code cell.
P.S. now in the colab, I see the same behavior as "local" described above (many x86_64 executables, all with cos6 in their name).   I don't know what changed!

I may have found a mistake on my part: I was using source env/conda/bin/activate && .... when I should have been doing source env/conda/bin/activate cfu-common && .....    I'm rerunning now; we'll see if it makes a difference.

This is a candidate workaround for this issue (for when running locally, not in colab):

Install Verilator in your system via sudo apt get verilator.
don't be in the conda shell (if you have typed make enter, then type exit).
delete the current Conda env: from $CFU_ROOT, rm -rf env/
edit ./conf/environment.yml to remove or comment out (#) the "verilator" line
rebuild the env: make env
enter the new env: make enter

Now if you cd to a project and "make PLATFORM=sim load", it should work; at least it works for me.

I'm seeing this same issue. When I use Tim's workaround mentioned in the last comment, I get this error instead:
OSError: Unable to find any of the cross compilation toolchains:
- riscv64-unknown-elf
- riscv64-unknown-linux-gnu
- riscv64-elf
- riscv64-linux
- riscv64-linux-gnu
- riscv-sifive-elf
- riscv64-none-elf
- riscv32-unknown-elf
- riscv32-unknown-linux-gnu
- riscv32-elf
- riscv-none-embed
- riscv-none-elf

@aman26kbm , if you did re-enter the environment after rebuilding (after commenting out Verilator), then you should get a usable cross compiler.   Can you try typing "riscv" and then auto-complete (usually tab) to see what is in your path with that prefix?

Thanks for your message, @tcal-x .
For some reason, make enter doesn't seem to work for me. I have to activate the environment manually. When I do that, I don't get the missing riscv toolchain error, but I get another verilator related error.
I am moving to another machine for multiple reasons. If I still see issues, I'll report back.

The verilator error was related to the version of verilator that got installed when I did "sudo apt-get install verilator" (it was 3.something). I ended up cloning verilator, pulling tag v4.038 (to match the version you had mentioned earlier in your message) and then building and installing it. After that make PLATFORM=sim load worked.

I added https://github.com/tcal-x/verilator-make-env-test as a minimal testcase showing the issue.

Hi @tcal-x! Thanks for the minimal testcase, it was super useful for debugging this problem.
I've researched the problem in a clean debian:bullseye container and it seems by default Conda installs an old compiler package (v7.3.0) which doesn't contain the x86_64-conda-...-prefixed executables, just the x86_64-conda_cos6-... ones. The g++ 8+ Conda compiler packages contain both (the _cosX ones are links).
There's no problem to install the latest compiler package right after using make env && make enter which is the simplest workaround:
root@01fd5943cb29:~/verilator-make-env-test# conda install 'gxx_linux-64>11'

(...)

The following packages will be UPDATED:

  binutils_impl_lin~                      2.33.1-he6710b0_7 --> 2.38-h2a08ee3_1
  binutils_linux-64                      2.33.1-h9595d00_15 --> 2.38.0-hc2dff05_0
  gcc_impl_linux-64                        7.3.0-habb00fd_1 --> 11.2.0-h1234567_1
  gcc_linux-64                            7.3.0-h553295d_15 --> 11.2.0-h5c386dc_0
  gxx_impl_linux-64                        7.3.0-hdf63c60_1 --> 11.2.0-h1234567_1
  gxx_linux-64                            7.3.0-h553295d_15 --> 11.2.0-hc2dff05_0
  ld_impl_linux-64                        2.33.1-h53a641e_7 --> 2.38-h1181459_1

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

root@01fd5943cb29:~/verilator-make-env-test# cd test/

root@01fd5943cb29:~/verilator-make-env-test/test# make
make[4]: Entering directory '/root/verilator-make-env-test/test'
verilator -cc add.v
make -C obj_dir -f Vadd.mk
make[5]: Entering directory '/root/verilator-make-env-test/test/obj_dir'
Vadd.mk:13: warning: NUL character seen; rest of line ignored
perl /root/verilator-make-env-test/env/conda/envs/verilator-make-env/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include Vadd.cpp Vadd___024root__DepSet_h53499b2f__0.cpp Vadd___024root__Slow.cpp Vadd___024root__DepSet_h53499b2f__0__Slow.cpp Vadd__Syms.cpp > Vadd__ALL.cpp
x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /root/verilator-make-env-test/env/conda/envs/verilator-make-env/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /root/verilator-make-env-test/env/conda/envs/verilator-make-env/include -I.  -MMD -I/root/verilator-make-env-test/env/conda/envs/verilator-make-env/share/verilator/include -I/root/verilator-make-env-test/env/conda/envs/verilator-make-env/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow      -std=gnu++14 -Os -c -o Vadd__ALL.o Vadd__ALL.cpp
echo "" > Vadd__ALL.verilator_deplist.tmp
Archive x86_64-conda-linux-gnu-ar -rcs Vadd__ALL.a Vadd__ALL.o
rm Vadd__ALL.verilator_deplist.tmp
make[5]: Leaving directory '/root/verilator-make-env-test/test/obj_dir'
make[4]: Leaving directory '/root/verilator-make-env-test/test'

(please notice the Verilator isn't downgraded in the process so it doesn't conflict with the latest compiler packages)
I'm going to make the new Verilator packages require the compiler to be v8+ on Linux because obviously it can't cooperate with the older ones and, if there won't be any additional problems, the fixed Verilator package might be available even today.
You might want to add gxx_linux-64 >11 package to the environment.yml files to fix the problem. It should help until the fixed Verilator v2.224 package gets uploaded.
Btw:

The latest compiler package contains _cos7 links since the compiler is based on CentOS 7: https://anaconda.org/anaconda/gxx_impl_linux-64/11.2.0/download/linux-64/gxx_impl_linux-64-11.2.0-h1234567_1.tar.bz2
so making the Verilator use _cosX-prefixed compiler wouldn't be a good option.
From the download counts, it seems the v7.3.0 package is the Conda's favourite for some mysterious reasons: https://anaconda.org/anaconda/gxx_impl_linux-64/files
For "normal" packages, always the one with the "highest" version string is used unless it conflicts with other packages so it might be somehow forced to be the default one.

Current status -- still broken:
x86_64-conda-linux-gnu-c++ -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib -Wl,-rpath-link,/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib -L/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib -lpthread -Wl,--no-as-needed -ljson-c -lz -lm -lstdc++ -Wl,--no-as-needed -ldl -levent    veril.o sim_init.o verilated.o verilated_dpi.o verilated_vcd_c.o Vsim__ALL.a   modules.o pads.o sim.o libdylib.o parse.o -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib -Wl,-rpath-link,/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib -L/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib -lpthread -Wl,--no-as-needed -ljson-c -lz -lm -lstdc++ -Wl,--no-as-needed -ldl -levent    -o Vsim
/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib/libz.so: undefined reference to `memcpy@GLIBC_2.14'
/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib/libstdc++.so: undefined reference to `aligned_alloc@GLIBC_2.16'
/media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /media/tim/GIT/tcal-x/CFU-Playground/env/conda/envs/cfu-common/lib/libstdc++.so: undefined reference to `clock_gettime@GLIBC_2.17'
collect2: error: ld returned 1 exit status
make[4]: *** [Vsim.mk:76: Vsim] Error 1
rm Vsim__ALL.verilator_deplist.tmp
make[4]: Leaving directory '/media/tim/GIT/tcal-x/CFU-Playground/soc/build/sim.proj_template_v/gateware/obj_dir'
make[3]: *** [/media/tim/GIT/tcal-x/CFU-Playground/third_party/python/litex/litex/build/sim/core/Makefile:39: sim] Error 2
make[3]: Leaving directory '/media/tim/GIT/tcal-x/CFU-Playground/soc/build/sim.proj_template_v/gateware'
make[2]: *** [/media/tim/GIT/tcal-x/CFU-Playground/soc/sim.mk:56: run] Error 1
make[2]: Leaving directory '/media/tim/GIT/tcal-x/CFU-Playground/soc'
make[1]: *** [../proj.mk:367: load] Error 2
make[1]: Leaving directory '/media/tim/GIT/tcal-x/CFU-Playground/proj/proj_template_v'

That is without any modification / pinning in the environment.yml file.

Note: it does work correctly on my work laptop (gLinux).   So yes, confirming that it is dependent on the host system environment.
** I did need to add libevent and json-c to conf/environment.yml **   I'll open a PR to make that change.

@ajelinski ping -- hopefully you have access to an Ubuntu 20.04 system to take a look.
I'm fine with pinning the compiler/library versions if you can find a combination that works in CFU Playground (I wasn't able to).

@ajelinski @PiotrZierhoffer important update -- I am playing with a conda-verilator job in CI (running on GHA's default Ubuntu 20.04), and I DON'T see the issue there.   So there may be something messed up with my laptop's Ubuntu installation.
It would of course be preferable that there was no dependence on the host system at all.   But, for now, we can treat this as lower priority.

It's really weird glibc 2.14/2.16/2.17 errors occurred on Ubuntu 20.04 since it seems to have been released with glibc 2.31 from the very beginning... I have really no idea what could cause that.
I tested CFU with a fresh Ubuntu 20.04 docker container and found a single irrelevant minor issue running commands from the workflow: https://github.com/tcal-x/CFU-Playground/commit/73e2256af43004ce2f1563fa91fd40fae26bf36c -- I had to remove sudo for the automatic installation of the missing packages to work here: https://github.com/google/CFU-Playground/blob/main/scripts/setup#L71
Apart from that, I could run the commands without any surprises and both make PLATFORM=sim RUN_MENU_ITEMS="5" run and make renode worked (though without GUI it's better to use make renode-headless which also worked).
As for the dependence on the host system it's just the Conda's secret requirement that it requires glibc >= 2.17 and even if we modify Verilator then there will be problems with many other packages. The error you had seems to be about ld.
Btw. I think this issue can be closed because the compiler troubles caused by Verilator have been solved.

Findings
Not all Ubuntu 20.04.4 LTS  systems are the same. My laptop has kernel 5.4.0-122-generic, while Github Actions runners and GCP (both 20.04.4) have kernels 5.15.0-1014-azure and 5.15.0-1013-gcp respectively.
Github and GCP hosts with the 5.15 kernel do not see this issue; Verilator compiles and runs fine in Conda.
It's only on my laptop with the 5.4 kernel.
I diffed the Conda package downloads and found this:
122,123c122,125
< sysroot_linux-64-2.12-he073ed8_15
< sysroot_linux-64-2.12-he073ed8_15.tar.bz2
---
> sysroot_linux-64-2.17-h57e8cba_10
> sysroot_linux-64-2.17-h57e8cba_10.tar.bz2
> _sysroot_linux-64_curr_repodata_hack-3-haa98f57_10
> _sysroot_linux-64_curr_repodata_hack-3-haa98f57_10.tar.bz2

and also found this difference:
my laptop:  env/conda/envs/cfu-common/x86_64-conda-linux-gnu/sysroot/lib/libc-2.12.so
Github/GCP: env/conda/envs/cfu-common/x86_64-conda-linux-gnu/sysroot/lib/libc-2.17.so
So, it's not using a host libc.so; Conda is downloading different libc versions for some reason.
I tried the simple fix, adding this to environment.yml on my laptop to try to force 2.17:
- sysroot_linux-64=2.17

...and it works!  It also fixes #629 (similar issue running renode).
We need to make sure this change works correctly on Ubuntu 18 (used by colab).

Hi @ajelinski , I thought I had submitted the above comment last night, but I hadn't.   Anyway, I think I got to the bottom of it; at least the fix #631 is working for me on my laptop, and it doesn't break things anywhere else that I've tested it.

Oh, that's an interesting finding. So it seems Conda makes a mess by using its own libc if it's available.
I call it a mess because the sysroot_linux package you mentioned is mostly installed with the compiler packages so it's used during linking instead of possibly-the-very-latest host libc. It isn't required by other packages. For example, if you check the packages in the base environment after installing Conda you won't find sysroot_linux there even though there are libstdcxx-ng, Python etc. packages which all won't work without libc.
Nevertheless, this won't be an issue in any new installation because the old package has been removed and isn't available anymore: https://anaconda.org/main/sysroot_linux-64/files

Btw. I've just created a PR for the sudo problem: https://github.com/google/CFU-Playground/pull/632

The remaining issue seems to be specific to my laptop (it has not been reproduced on other Ubuntu 20.04 systems), so I'll close this as not planned.

authors: ajelinski aman26kbm mithro tcal-x

repo: google/CFU-Playground


text:
No one knows about our social channels (Twitter mainly) - add a link in header

I would love to get a link to ChromiumDev account in the header or some other prominent place across all pages.

I was thinking we could do something like this, except with logos instead of text, thoughts?

Yah, I think that looks good. Potential concern - what about on pages where there's an author/date, think that people might think those are the authors social network links?
Maybe not in that order though.

source: gharchive/issue

authors: PaulKinlan gauntface petele

repo: google/WebFundamentals

license_type: permissive

license_source: bigquery


text:
App crashing when dynamically setting the range for a date item

Describe the bug
The use case is that when entering a patient's test date, the date range should be after the patient's birth date and also not a future date. When the former is applied and the date picker is tapped, the app crashes.
Component
SDC library
To Reproduce
N/A
Expected behavior
I expect the date picker to open with the range applied accordingly.
Screenshots
N/A
Smartphone (please complete the following information):

Device: Samsung A12
Android version: 11
Build number: RP1A.200720.012.A127FXXU4AUK1
Version: 30

Additional context
Here is the questionnaire I'm using to test this. Specifically, this line is what is causing the crash, "expression": "@Patient.birthDate". I've tried a number of ways to do this but can't seem to get it to work.
Here is the log:
E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.google.android.fhir.demo, PID: 23037
    org.hl7.fhir.r4.utils.FHIRLexer$FHIRLexerException: Error in ?? at 1, 1: Premature ExpressionNode termination at unexpected token "Patient"
        at org.hl7.fhir.r4.utils.FHIRLexer.error(FHIRLexer.java:145)
        at org.hl7.fhir.r4.utils.FHIRLexer.error(FHIRLexer.java:141)
        at org.hl7.fhir.r4.utils.FHIRPathEngine.parse(FHIRPathEngine.java:340)
        at org.hl7.fhir.r4.utils.FHIRPathEngine.parse(FHIRPathEngine.java:331)
        at org.hl7.fhir.r4.utils.FHIRPathEngine.evaluate(FHIRPathEngine.java:520)
        at com.google.android.fhir.datacapture.validation.ValidationUtilKt.valueOrCalculateValue(ValidationUtil.kt:32)
        at com.google.android.fhir.datacapture.validation.MinValueConstraintValidator.getMinValue$datacapture_debug(MinValueConstraintValidator.kt:50)
        at com.google.android.fhir.datacapture.views.QuestionnaireItemDatePickerViewHolderFactory$getQuestionnaireItemViewHolderDelegate$1.getCalenderConstraint(QuestionnaireItemDatePickerViewHolderFactory.kt:156)
        at com.google.android.fhir.datacapture.views.QuestionnaireItemDatePickerViewHolderFactory$getQuestionnaireItemViewHolderDelegate$1.createMaterialDatePicker(QuestionnaireItemDatePickerViewHolderFactory.kt:150)
        at com.google.android.fhir.datacapture.views.QuestionnaireItemDatePickerViewHolderFactory$getQuestionnaireItemViewHolderDelegate$1.init$lambda-3(QuestionnaireItemDatePickerViewHolderFactory.kt:78)
        at com.google.android.fhir.datacapture.views.QuestionnaireItemDatePickerViewHolderFactory$getQuestionnaireItemViewHolderDelegate$1.$r8$lambda$5UhY9wSdxD4KX8dzfdN4hNLY5vA(Unknown Source:0)
        at com.google.android.fhir.datacapture.views.QuestionnaireItemDatePickerViewHolderFactory$getQuestionnaireItemViewHolderDelegate$1$$ExternalSyntheticLambda0.onClick(Unknown Source:4)
        at android.view.View.performClick(View.java:7792)
        at android.view.View.performClickInternal(View.java:7769)
        at android.view.View.access$3800(View.java:910)
        at android.view.View$PerformClick.run(View.java:30218)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loopOnce(Looper.java:226)
        at android.os.Looper.loop(Looper.java:313)
        at android.app.ActivityThread.main(ActivityThread.java:8669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:571)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1135)

Would you like to work on the issue?
This can be a follow up to https://github.com/google/android-fhir/pull/1442

Thanks for raising this @KhumboLihonga.
Unfortunately i don't think the sdc library supports this right now - we do try to evaluate FHIRPath expressions in the calculatedValue extension (with canonical url http://hl7.org/fhir/StructureDefinition/cqf-calculatedValue) but i think we might need to fix that (as that extension is clearly pointing to cql libs)
@brynrhodes can you help here please? what should the questionnaire be if we want to reference a patient's birthdate? should we use launch context?
fyi @maimoonak

also @costateixeira might be able to help

@jingtang10  have been trying to simulate this issue and here is my approach. I have made an update to this questionnaire and here is my output from the demo app.

I think this is now possible with launch context, variable expression and constraits.

This is obsolete and confirmed by Ager.

authors: Itskiprotich KhumboLihonga jingtang10

repo: google/android-fhir


text:
close navigation drawer on sync menu selection

IMPORTANT: All PRs must be linked to an issue (except for extremely trivial and straightforward changes).
Fixes #1745
Description
Closed navigation drawer before returning from onNavigationItemSelected() mehod
Alternative(s) considered
Have you considered any alternatives? And if so, why have you chosen the approach in this PR?
Type
Choose one: Bug fix
Screenshots (if applicable)
Checklist

[x] I have read and acknowledged the Code of conduct.
[x] I have read the Contributing page.
[x] I have signed the Google Individual CLA, or I am covered by my company's Corporate CLA.
[x] I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
[x] I have run ./gradlew spotlessApply and ./gradlew spotlessCheck to check my code follows the style guide of this project.
[x] I have run ./gradlew check and ./gradlew connectedCheck to test my changes locally.
[x] I have built and run the demo app(s) to verify my change fixes the issue and/or does not break the demo app(s).

Hi @PallaviGanorkar I have also added a commit to make sure "sync" isn't selected by returning false instead of true, because it shouldn't be aselected item, it's just a one-time option.

source: gharchive/pull-request


text:
Create codeql.yml

See if CodeQL will work on default settings.
IMPORTANT: All PRs must be linked to an issue (except for extremely trivial and straightforward changes).
Fixes #[issue number]
Description
Clear and concise code change description.
Alternative(s) considered
Have you considered any alternatives? And if so, why have you chosen the approach in this PR?
Type
Choose one: (Bug fix | Feature | Documentation | Testing | Code health | Builds | Releases | Other)
Screenshots (if applicable)
Checklist

[x] I have read and acknowledged the Code of conduct.
[x] I have read the Contributing page.
[x] I have signed the Google Individual CLA, or I am covered by my company's Corporate CLA.
[ ] I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
[ ] I have run ./gradlew spotlessApply and ./gradlew spotlessCheck to check my code follows the style guide of this project.
[ ] I have run ./gradlew check and ./gradlew connectedCheck to test my changes locally.
[ ] I have built and run the demo app(s) to verify my change fixes the issue and/or does not break the demo app(s).

@williamito is this feature still being worked on?

The defaults weren't cooperating, I'll take a few more stabs at it once I'm available full time in August but may ask for help or shelve for now.

closing for now but leaving issue open


text:
fit "zoom to basicblock" to reading symbols

now "zoom to basicblock" fit to block inside the diff window
its not comfortable, there are need to many press "zoom in" to allow reading symbols
more usefull will be fit blocks to reading symbols by "zoom to basicblock"
look at the pics
this is zoom fit for now

this is zoom fit which i need

block can be too long width or too long height
i want default zoom basic block to allow read chars inside the basic block

repo: google/bindiff

license_source: github-api


text:
binexport1164 install failure and IDA 7.x crashes (macOS 10.15.7)

Steps to reproduce the problem:

Install IDA 7.O (macOS 10.15.7 Catalina)
2.Install Bindiff 6 (installs correctly)
Install binexport1164 (fail installation) and load Mac binary

What is the expected behavior? binexport11 fails installation and Bindiff 6 fails to find binexport11
What went wrong? binexport11 install failure and IDA 7.x crashes.
What version of the product are you using? On what operating system?
Install IDA 7.O (macOS 10.15.7 Catalina),  Bindiff 6,  binexport1164
Any other comments? I need help fixing this.
ERRORS:  dlopen(/Applications/IDA Pro 7.0/ida64.app/Contents/MacOS/plugins/binexport1164.dylib): dlopen(/Applications/IDA Pro 7.0/ida64.app/Contents/MacOS/plugins/binexport1164.dylib, 2): Symbol not found: _get_enum_name2
Referenced from: /Applications/IDA Pro 7.0/ida64.app/Contents/MacOS/plugins/binexport1164.dylib
Expected in: flat namespace
in /Applications/IDA Pro 7.0/ida64.app/Contents/MacOS/plugins/binexport1164.dylib
/Applications/IDA Pro 7.0/ida64.app/Contents/MacOS/plugins/binexport1164.dylib: can't load file

Please note that BinDiff 6 is only supported on IDA Pro 7.4 or higher (see here: https://www.zynamics.com/bindiff/manual/index.html#nyyyy6)
The version of BinExport that ships with it requires API functions introduced in IDA 7.2. Specifically, as the error message shows, it needs get_enum_name2() to be available, among others. If you have a copy of IDA 7.2, you can try to rebuild the  BinExport binary with the matching IDA SDK 7.2.

authors: cblichmann krisgnyc

repo: google/binexport


text:
Solitaire game with others watching over websockets?

Hi, I'm designing a board game for single-player.
I was wondering if it's possible with boardgame.io to create a proper game with number of players to equal 1? (right now I'm automatically skipping a turn of other players)
I'm also interested if I can use boardgame.io 's multiplayer mechanism for other players to join watching the solitaire game in real-time?

Yes, just set numPlayers to 1 and it should work. To have others spectate, use multiplayer: true when creating the client, set playedID = "0" on the client that is actually playing, and have other clients connect without a playerID.
There is an example to do just this in the repository. Have a look!

license: MIT


text:
Add multiple YUV filesets

There needs to be parameters for switching between filesets.
Suggested design: Fileset names = directory names under "video".
Bitrates are taken from a fixed table based on width x height.
Name "local" should be reserved for non-global results (not uploaded or made consistent).

New request: Add subsets.

repo: google/compare-codecs


text:
Detailed profiling data for terrain processing

For debugging purposes it would be useful to have detailed profiling information from the gecombineterrain executable. This will help us understand the performance of the executable and how we can optimize it.

Closed via PRs #683 and #695.

repo: google/earthenterprise


text:
Remove old scripts for creating Portable Globe Server installers.

This code is replaced by https://github.com/google/earthenterprise/blob/master/earth_enterprise/src/portableserver/build.py and is no longer functional after unbundling of python
Resolves #143

looks good.

Have assigned to @tst-ppenev for reviewing/approval, since he's worked on the Portable installer.

We could possibly also remove the <earthenterprise/earth_enterprise/legacy_installer/Scripts/portable/windows-installer> directory, as well. I don't think anyone has tested the new Python script on Mac, so we can leave that part around.

@bradh: I can't find your comment, but, yes, I vote for removing the Windows and the Linux directories. I don't think they're doing much good, and may cause confusion. I'd even approve removing the Mac directory, since it's also obsolete, and I don't think anyone would look at it when testing and fixing the Mac installer.
Since no one has objected, I'd say, let's just remove them.
I welcome any objections.

@tst-ppenev @bradh I have no issue with just removing them all. If we need to refer to them, we can always look back at the history.

:cool: We can rename the ticket to something like 'Remove old Portable Server installer creation scripts', or whatever sounds more human.

OK. Will catch up with that tonight.

Thanks, @bradh. I'll update the title with something temporary.

Should be completely removed now. Suggest squash on merge.

@tst-ccamp, @avnishb, @asmitaw, @andreisel: Do you know if the icons under locations, such as under <earthenterprise/earth_enterprise/legacy_installer/Scripts/portable/windows-installer/server_extra/> can be used in GEE Open Source?

@bradh: This looks fine to me. I'm just wondering if we want to keep the icons for possible use in future installers.

@tst-ppenev They can always be pulled out of the history as of the previous commit:
https://github.com/google/earthenterprise/tree/d91a2b8daf899e31e301041db2dc182c7c9b46ee/earth_enterprise/legacy_installer/Scripts/portable/windows-installer/server_extra/icons

@bradh: This issue seems to require agreements among people who we may not hear from immediately. My vote is to move the <server_extra> directory under <earth_enterprise/artwork> for now, and remove the rest of the portable legacy installer generators.

@tst-ppenev: OK. Abandoning this attempt. Can you update the original ticket (#143) with what you really want to happen?

authors: andreisel asmitaw bradh tst-ccamp tst-ppenev


text:
I play the game with my own trained model. But it can't identify objects.

Hi, Emoji Scavenger Hunt is really cool game.
I play the game with my own trained model. But it can't identify objects. It takes a picture of any object and tells me I found the item.
I didn't have any error, so I think maybe the problem is from the image data.
I prepare 400 images about OK(gesture) for training.
Should I need to prepare more images or select pure background images?

Hi Bella,
Thank you for playing the game. I believe 400 images is large enough.
Please make sure to update src/js/scavenger_classes.ts according to
output_labels.txt, which is generated by the retraining program. It
describes the new class names for the retained model.
Thanks,
Shuhei
2018年5月16日(水) 17:05 Bella notifications@github.com:

Hi, Emoji Scavenger Hunt is really cool game.
I play the game with my own trained model. But it can't identify objects.
It takes a picture of any object and tells me I found the item.
I didn't have any error, so I think maybe the problem is from the image
data.
I prepare 400 images about OK(gesture) for training.
Should I need to prepare more images or select pure background images?
https://camo.githubusercontent.com/40ee3a2904e63f81604773bd8f0545a233fa7b55/68747470733a2f2f692e696d6775722e636f6d2f706a444f3372702e706e67
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/emoji-scavenger-hunt/issues/24, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAs2ud9r_NS9b0mfb7ZhSKqbsOxRijoLks5ty93ZgaJpZM4UA1E8
.

Thank you for your reply. I'm not sure my code is correct or not.
Here is the content of my files:
output_labels.txt
travelfrog
ok
scissors

scavenger_classes.ts
export const SCAVENGER_CLASSES: {[key: number]: string} = {
    0: 'travelfrog',
    1: 'ok',
    2: 'scissors'
};

game_levels.ts
export interface EmojiItem {
  [index: string]: string;
}

export const EMOJIS_LVL_1: Array<EmojiItem> = [
  {
    'name': 'scissors',
    'emoji': '',
    'path': '/img/emojis/game/scissors.svg'
  },
  {
    'name': 'travelfrog',
    'emoji': '',
    'path': '/img/emojis/game/travelfrog.svg'
  },
  {
    'name': 'ok',
    'emoji': '',
    'path': '/img/emojis/game/ok.svg'
  }
];

The change you made looks good to me. Please let me know if you see any trouble after compiling them. Also, you can see how the image detection is working by passing ?debug=true parameter as instructed at https://github.com/google/emoji-scavenger-hunt/blob/master/src/js/game.ts#L381

Thank you for your help.
I found the problem is from the logic of game.
When I modified the function of  checkEmojiMatch , the game worked very well.
if (this.currentEmoji.name === emojiNameTop1.label && emojiNameTop1.value > 0.8)  {
      this.emojiFound();
};

Same here.
4 categories, 200/300 each category. Any idea?

authors: tinchoforever tushuhei yea759

repo: google/emoji-scavenger-hunt


text:
How to l2-normalize the embedding feature in flax.linen.Embed

Hi,
I am quite new to Jax/Flax, and I am looking forward to an example about flax.linen.Embed

I would like to know how to normalize the embedding features with unit-norms when init the module?
how to renormalize after each update during training?

Thanks a lot in advance!

Hi @longbowzhang,
Here is one way of achieving this:
Suppose you have the following simple network:
class Net(nn.Module):

    @nn.compact
    def __call__(self, c):
        x = nn.Embed(num_embeddings=10, features=5)(c)
        x = nn.Dense(features=5)(x)
        return x

Then you could apply the following normalization after each training step (either in your training loop or directly in your update function):
params = state.params.unfreeze()
params['params']['Embed_0']['embedding'] /= (jnp.linalg.norm(params['params']['Embed_0']['embedding'], axis=-1, keepdims=True) + 1e-10)
state = state.replace(params=frozen_dict.freeze(params))

Of course you can apply the same normalization after initializing your parameters.
Let me know if you need more details, I could create a small Colab for you.
Hope that helps!

authors: longbowzhang matthias-wright

repo: google/flax


text:
added rmsnorm to api docs

added rmsnorm to api docs

Codecov Report

Merging #3406 (35326d4) into main (6635436) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #3406   +/-   ##
=======================================
  Coverage   83.60%   83.60%
=======================================
  Files          56       56
  Lines        6747     6747
=======================================
  Hits         5641     5641
  Misses       1106     1106

authors: chiamp codecov-commenter


text:
[iOS]Crash on app upgrade

I am facing a strange issue with Google cloud messaging for iOS. The app is crashing few seconds after calling
[[GGLContext sharedInstance] configureWithError:&configureError];

Crash is happening when a older version of the app without GCM is installed from app store. Then a new version of the app with GCM is installed via Xcode.
More details on StackOverflow

This is not a GCM error although still somewhat related. Looking at your crash report here is where it crashes
<GMR/INFO> App measurement v.1100000 started
<GMR/ERROR> Exception on worker queue: data parameter is nil

It crashes while it's trying to configure some things before GCM will be started. We're looking into why this is happening for you, will let you know once we have more info.

Hi guys,
After a continuous research I finally got this crash. It's not connected with GCM, however there are definitely problems with Google Framework.
Environment:

iOS7
production build

[[GGLContext sharedInstance] configureWithError:&configureError];

Each method call leads to crash:

Thread : Fatal Exception: NSInvalidArgumentException
0  CoreFoundation                 790605699 __exceptionPreprocess
1  libobjc.A.dylib                966442191 objc_exception_throw
2  CoreFoundation                 790605509 -[NSException initWithCoder:]
3  Foundation                     800316859 +[NSPropertyListSerialization propertyListWithData:options:format:error:]
4  Tabs Evo                       2319667 +[GMRIdentity iTunesMetadata]
5  Tabs Evo                       2317421 -[GMRIdentity initWithPersistedConfig:configuration:]
6  Tabs Evo                       2323851 -[GMRMeasurement startMeasurementWithDatabaseName:configuration:]
7  Tabs Evo                       2348895 __38-[GMRScheduler scheduleOnWorkerQueue:]_block_invoke
8  libdispatch.dylib              971585587 _dispatch_call_block_and_release
9  libdispatch.dylib              971587053 _dispatch_queue_drain$VARIANT$up
10 libdispatch.dylib              971588247 _dispatch_queue_invoke$VARIANT$up
11 libdispatch.dylib              971663501 _dispatch_root_queue_drain
12 libdispatch.dylib              971664161 _dispatch_worker_thread2
13 libsystem_pthread.dylib        972905427 _pthread_wqthread
14 libsystem_pthread.dylib        972905112 start_wqthread

@ollirus Your crash seems a bit different. @ntsh crash should have already been fixed.
Do you see the crash for all of your clients or just specific ones?

@mohitsharma0690  All iOS 7 users have the crash at app launch.

This should be fixed in the latest pod. @ntsh @ollirus can you guys confirm with the latest pods?

Thanks @mohitsharma0690
I upgraded to the latest pod.
When I try to test it again on iOS 7, the app didn't crash. However I was not able to get the push notification token.
Following are the relevant logs:
2015-12-03 20:08:55.685 App[1772:501] <GMR/INFO> App measurement v.1201000 started
2015-12-03 20:08:55.831: GCM | Invalid key in checkin plist: GMSInstanceIDDeviceAuthIdKey
2015-12-03 20:08:55.832: GCM | Invalid key in checkin plist: GMSInstanceIDSecretTokenKey
2015-12-03 20:08:58.600 App[1772:6003] ERROR: unable to get the receiver data from the DB!

Is this expected? Do I need to add GMSInstanceIDDeviceAuthIdKey  or   GMSInstanceIDSecretTokenKey somewhere?

That should be fine what does the server response say?

There were no more logs after that. I can test the scenario once more
tomorrow.
Is it now safe to use it for iOS 7 as well?
On Thu, Dec 3, 2015 at 10:59 PM, Mohit Sharma notifications@github.com
wrote:

That should be fine what does the server response say?
—
Reply to this email directly or view it on GitHub
https://github.com/google/gcm/issues/113#issuecomment-161816332.

I'm not talking about the client logs. What is the response you get from the server? Also do you successfully get a GCM registration token?

Closing for lack of activity.

I am getting a crash at
[[GGLContext sharedInstance] configureWithError:&configureError];
For ios version 6.1.3.
below is the crash error I am getting.
dyld: lazy symbol binding failed: Symbol not found: _asl_add_output_file
Referenced from: /var/mobile/Applications/C9326A4E-514A-47A8-A35F-172F065C87A7/Test.app/Test
Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: _asl_add_output_file
Referenced from: /var/mobile/Applications/C9326A4E-514A-47A8-A35F-172F065C87A7/Test.app/Test
Expected in: /usr/lib/libSystem.B.dylib
Does the GCM library for ios defines a minimum sdk version ?

Yeah it's 7.0, here https://github.com/CocoaPods/Specs/blob/master/Specs/GoogleCloudMessaging/1.0.3/GoogleCloudMessaging.podspec.json#L13

authors: anuragbhowmick mohitsharma0690 ntsh ollirus

repo: google/gcm


text:
Add support within Actions for creating a workflow dispatch event

I noticed that the github client hadn't added support for this functionality yet.
According to https://docs.github.com/en/rest/reference/actions#create-a-workflow-dispatch-event, this endpoint is utilized for manually triggering a Github Actions workflow run.
I purposely left out support for the inputs key, since due to its dynamic nature, might be a larger  and more complicated change than we want.

Thank you, @wesleimp !
Merging.

authors: AntonNguyen gmlewis

repo: google/go-github


text:
Implement the GitHub Authorizations API per issue #119

Implement the GitHub Authorizations API. See issue #119.

Unit and integration tests are provided.
Integration tests require that certain environment variables are specified, see the updated tests/README.md
For example usage, see the integration tests in tests/integration/authorizations_test.go
Edited top-level README.md to provide a link to the tests/README.md and removed content that was duplicated in the top-level file.

So this largely duplicates the work in #241 that I just hadn't gotten around to merging in yet.  You do have a lot of additional niceties here like the integration tests, so I'll see about applying that and some other things on top of #241 once I get it merged.

I'll be merging this in just a moment.  I broke it into two commits, one that builds on top of #241 to include things like the Scope type and values, as well as some updates to the function docs that you had.  Then the second commit has the integration tests.  There were some small changes to that, mostly relating to variable naming.  Thanks for contributing this!

Well dang @willnorris, remind me in future to check the other PRs! Glad to be of assistance.

authors: neilotoole willnorris


text:
Mention googleapi subpackage more prominently in docs

google.golang.org/api/googleapi has a number of useful functions that help with other APIs in this repo.  Now that the list of APIs is so long, we need to call it out in the docs as part of basic usage.

Before you do so, somebody should do a pass over googleapi and make it not embarassing.
Because it also was the "gensupport" package for a long time, before "gensupport" was created, it still is a little gross.
To pick one example: https://godoc.org/google.golang.org/api/googleapi#CombineFields

// CombineFields combines fields into a single string.

How? Why?
And is https://godoc.org/google.golang.org/api/googleapi#ContentTyper deprecated? I thought so.
Actually, the package isn't as bad as I remember. The move to gensupport cleaned it up a fair bit.

@bradfitz 's sentiment is captured in issue #113 , which I filed a short time after I introduced the gensupport package (then called "internal"). I don't think that we ever did much of the cleanup migration -- the introduction of gensupport just gave us a home to put new non-user-facing stuff, so the bloat of implementation details in googleapi stopped getting worse.

This repo is in maintenance mode. We don't plan on making cosmetic changes like this.

authors: bradfitz jba mcgreevy zombiezen

repo: google/google-api-go-client


text:
The library is not compatible with webpack

The library is not compatible with webpack

It's not meant to be, as it's meant to be used on the server only. It doesn't need to be passed through any Webpack build.

@jmdobry I believe Webpack supports some Node packing - however, I agree with your statement.
@ddimitrioglo As @jmdobry mentioned, this (and Node.js) is designed for use on the server (or backend), whereas webpack is designed for use with client-side (or frontend) code.
What exactly are you trying to do?

In our project we are working with AWS.Lambda and we used to zip all the node_modules together with the handler.js (lambda code).
To reduce lamda.zip size we pass all the dependencies through the webpack and that is why we faced the issue with googleapis incompatibility.
P.S. As you can see it's still backend execution.

Can you provide more details beyond "The library is not compatible with webpack"? Exact reproduction steps will allow us to help you, thanks.

FWIW - it's not that uncommon to use webpack with Node.js for electron applications.  I actually do this in github.com/JustinBeckwith/trebuchet.

Lambda code (in general):
import google from 'googleapis';

let OAuth2 = google.auth.OAuth2;
let sheets = google.sheets('v4');

export function handler(event, context) {
  get auth() {
    // returns OAuth2 client;
  }

  getRows(spreadsheetId, worksheetName) {
    return new Promise((resolve, reject) => {
      sheets.spreadsheets.values.get(
        {auth: this.auth, spreadsheetId: spreadsheetId, range: `${worksheetName}!A:Z`},
        (err, response) => {
          if (err) {
            return reject(`Spreadsheet read error: ${err}`);
          }

          resolve(response.values);
        });
    });
  }
}

and when I'm trying to "pack" it with webpack it throws me an error the request of a dependency is an expression

Is it an error or a warning?

An error, because webpack can not resolve something like this:
require(someVar == true ? 'getSomething' : 'getOther')

There are options for this. Webpack will simply include all possible values of the expression.
http://webpack.github.io/docs/configuration.html#automatically-created-contexts-defaults-module-xxxcontextxxx

Same issue, apparently the problem is here and this can not be resolved

Anyone find a solution for this?

@yoyeung I needed spreadsheet API and I just used another library npm install google-spreadsheet

@ddimitrioglo Good idea...But in my case, i need to use GSuite api....lol

I have same trouble like @yoyeung,
Use externals to avoid to pack.
// webpack.config.js
externals: {
    googleapis: "commonjs googleapis"
}

// mycode.js
const google = require("googleapis");

See why "commonjs googleapis" works:
https://webpack.js.org/configuration/externals/#function
(I honestly hope google-api-nodejs-client working under webpack!)

I use google analytics reporting and it gets stuck for me with error module initialization error:  ENOENT: no such file or directory, scandir '/apis'
see: googleapis.js:58
// Dynamically discover available APIs
fs.readdirSync(path.join(__dirname, '../apis')).forEach(function (file) {
    apis[file] = requireAPI('../apis/' + file);
});

do you have any idea or could you point me somewhere?

We bumped into the same issue. I think this should be fixed.

@jmdobry @ofrobots interested in your thoughts here.  This could be fixed by generating an apis.ts at generation time, and having that explicitly require each generated API.  This would get rid of the weird fs.readdirsync that happens on module load by making it a compile time operation.
Only negative is that you lose some flexibility in terms of running with a dynamic set of generated APIs in the apis/ folder.  For example, today we support generating a single API at a time.  Are there places where generating the require list at build time falls down?

I confirm that it's a pain to package a lambda with google apis for node.
My process was this:

package with all node_modules, look at the package size and realise it's a bad idea
enumerate all package and sub-packages files that should be included, realise it's also a bad idea
try to use webpack until you come to the "readdirsync" and come across this issue

Finally, i'll enumerate all package and sub-packages files but I feel like a cavemen.

Sucks this is still an issue 1.5 years after I first encountered it.  Here is how I resolved for lambda use...
I have a top level dir in my project named lambda_node_modules. Install googleapis there.
For local dev, a simple symlink points node_modules/googleapis to lambda_node_modules/node_modules/googleapis
In Webpack Config (like I said 18 mths ago so this is Webpack 1):
plugins: [
// Moves files
new TransferWebpackPlugin([
{from: './lambda_node_modules/node_modules/', to: './dist/node_modules/'},
], __dirname),
],
externals: {
'aws-sdk': 'commonjs aws-sdk',
'googleapis': 'commonjs googleapis',
},
That's all it took for me.  Webpack now thinks googleapis is externally available so it won't try to pack it, though Lambda needs it so I ship it to lambda in the deployment zip.

Guys, this still an issue.
I'm using this library to implement OAuth 2 and Gmail API in Electron application, and I can't use it from a renderer process.
Versions:
Googleapis: 33.0.0
Webpack: 4.17.1
Electron: 3.0.0-beta.6

Oh hai!  We're tracking this in #1286.   We are super close :)

This is still broken.
By the way, is this is meant to be used server side, why is it labeled as client side? I just want to cloud-save some app data in a users G-Drive :/

For the folks who track this issue - can you give it a try now? We don't publish the webpacked minified files anywhere yet but you can webpack individual APIs and it should just work. Let us know if it does not (in a separate issue please :) )

Hi, I'm still having issues:
ERROR in ./node_modules/@google-cloud/storage/node_modules/mime/index.js
Module not found: Error: Can't resolve './types/other' in '/home/xxx/xxx/gcp-typescript-kata/node_modules/@google-cloud/storage/node_modules/mime'

ERROR in ./node_modules/gtoken/node_modules/mime/index.js
Module not found: Error: Can't resolve './types/other' in '/home/xxx/xxx/gcp-typescript-kata/node_modules/gtoken/node_modules/mime'

ERROR in ./node_modules/@google-cloud/storage/node_modules/mime/index.js
Module not found: Error: Can't resolve './types/standard' in '/home/xxx/xxx/gcp-typescript-kata/node_modules/@google-cloud/storage/node_modules/mime'

ERROR in ./node_modules/gtoken/node_modules/mime/index.js
Module not found: Error: Can't resolve './types/standard' in '/home/xxx/xxx/gcp-typescript-kata/node_modules/gtoken/node_modules/mime'

I found my error. I had to include .json in the resolve block of my webpack.config.js. E.g.
module.exports = {
[…]
  resolve: {
    extensions: ['.tsx', '.ts', '.js', '.json'],
  },
[…]
};

authors: Enase JustinBeckwith MrEmelianenko STRML ace-n alexander-fenster aries1980 ddimitrioglo jmdobry jonmanzo milonti mtgto oktapodia portenez prevostc yoyeung

repo: google/google-api-nodejs-client


text:
List search not paging when using both publishedBefore and publishedAfter params

Steps to replicate:

Go here: https://developers.google.com/youtube/v3/docs/search/list
Set the following:

part: id,snippet
publishedAfter: 2015-05-01T00:00:00+01:00
publishedBefore: 2015-05-01T01:00:00+01:00
query: Path of exile
type: video

Execute, take note of the first videoid which is FoRmdifCh20. It indicates the total results are also 17147
Copy the nextPageToken into the corresponding box above, to fetch the next page.
Execute, see the same list of videos, the first videoid is again FoRmdifCh20

Seems like it's not an issue within the library

authors: elliotthill tmatsuo

repo: google/google-api-php-client


text:
Blog status

Hi everyone,
I'm talking about Blogger Service.
Does the function getStatus and setStatus work for you? It doesn't return / modify anything when using it.

$status = $service->blogs->get($blogId)->getStatus();
echo $status;

Can you help me to fix that?
By the way, what are the differents status values ? LIVE, DELETED and ?
Thanks !

I think those methods are still in the client library even though they have been removed from the latest version of the API itself: https://developers.google.com/blogger/docs/3.0/reference/
Were you able to find a different solution?

This issue will be addressed by a fix to google-api-php-client-services#44

authors: JuloxRox mattwhisenhunt


text:
libpam/demo: fix overflow in code input

Correct an incorrect allocation resulting in an overflow of the response
buffer:
$ ./demo
Verification code: 0123456789012345678901234567890
*** Error in `./demo': malloc(): memory corruption: 0x00000000025300e0 ***

I signed it!

repo: google/google-authenticator


text:
[Offline sync] Consolidate and simplify notifications

We should have notifications for these background operations:

[ ] New LOIs added to a job by organizer
[ ] New jobs added to a survey by organizer
[ ] Submitted data uploaded (notify about photos separately?)

Offline basemap tiles download can happen in the background but the status should be reported in the UI rather than notification.
The notifications are currently transient, but with #1125 they should be persistent.

We've removed all notifications for now, closing as obsolete.

repo: google/ground-android


text:
Fix issues with recursive loading

Crash with nice Guice errors rather than Guava crashes or deadlocks.

I’m reviewing old PRs and I abandoned this one before I pushed it over the finish line. That’s my fault.
As an alternative, please consider reviewing and merging #1389. That PR demonstrates the bug without a specified fix. Y’all can figure out what fix you like best!

repo: google/guice


text:
[IO 2015] SessionDetailFragment initViews() practice

It seems that the way to retrieve component views of this Fragment (so sorry I'm reading this class only) doesn't seem to be a good practice. I personally consider it a bad way.
Specific: calling Activity#findViewById(int) from inside a Fragment instead of View#findViewById(int) in SessionDetailFragment#initViews(). I guess an initViews(View root) should be the proper way to go, and of course, this method will be called inside Fragment#onViewCreated(View, Bundle). Well, let us make the source code better!

how about if the view is not inside the fragment layout? but in the activity layout? i don't see any bad practice about it tho, as the fragment is within the activity so it should be okay. if the view is inside the fragment then i 100% support your opinion.

getView() does seem to be the better approach here at first glance. @freewheelnat, it looks like you updated this code a bit last, could you advise?

@PaulRashidi, calling getView() for a view that present in the layout inflated in the fragment should cause no problem as long as you call it after onViewCreate/d. as i mentioned if the view isn't within the fragment layout then calling getActivity().findViewById should be no problem. correct me if im wrong.

@k0shk0sh I think you're saying you want a Fragment to access views in other fragments or the parent activity. While this is possible, it's generally not the best approach as it makes the code a little bit brittle. In very complex hierarchies or ones with duplicates IDs (think duplicate fragments) it can introduce performance or logic problems as well.
https://developer.android.com/training/basics/fragments/communicating.html

@k0shk0sh Generally a Fragment can be attached to any Activities, so you could not guarantee that your Fragment knows about its host Activity. Furthermore, latest support library suggest that your android.support.v4.app.Fragment could be attached to any Context holder host (so the Host might not be an Activity. A good practice for this is not out there, but yeah they have reason for deprecating the current familiar Fragment#onAttached(Activity)), so I think you should take a good care of your practices. I agree that getView() instead of getActivity() looks good at first, since we know that this Fragment has a proper View (not null).

thanks guys for the explanation, however you didn't get my point, as i pointed out in my first comment, i did say if the view is presenting within the fragment then calling getActivity().findViewbyId is bad practice, but in some cases calling a view that is presenting in the activity view shouldn't be bad as long as you know what are you doing. i can't see any bad reason about as long as you know what are you doing :D . btw for the fragment Context i haven't tried it yet but it doesn't seem to play very well.

@k0shk0sh as @PaulRashidi mentions, it could work on simple implementation though, since you are able to read and maintain the whole class. But when you turn to a bigger concept, you may got headache instead :D. And in my opinion, the concept of Fragment would suggest you to always use its getView() instead of getActivity(): a Fragment is a piece of UI (well, modern Fragment can have no View) which is supposed to be attached and detached from Windows. So for more flexibility, and correctness, I would consider to used the Fragment's view to do the rest. But yeah I agree that which is working, and stable, we keep it ;).

Agreed with comments here as in this instance, the Fragment has a View and we are calling views within that View so we should use getView() instead of getActivity().

Obsolete

authors: JoseAlcerreca PaulRashidi eneim freewheelnat k0shk0sh

repo: google/iosched


text:
C API headers should be exported through a runtime.h

As I'm writing docs for the C API (https://github.com/google/iree/issues/5463), I'm writing sections like this:

Component header file
Overview

iree/base/api.h
Core API, type definitions, ownership policies, utilities

iree/vm/api.h
VM APIs: loading modules, I/O, calling functions

iree/hal/api.h
HAL APIs: device management, synchronization, accessing hardware features

Include headers:
#include "iree/base/api.h"
#include "iree/hal/api.h"
#include "iree/vm/api.h"

// The VM and HAL native modules will typically be included, along
// with those for the specific HAL drivers your application uses.
// Functionality extensions can be used via custom modules.
#include "iree/modules/hal/hal_module.h"
#include "iree/hal/dylib/registration/driver_module.h"
#include "iree/vm/bytecode_module.h"

We could group the runtime APIs into a single runtime.h header. Maybe HAL drivers could be included as well (using #ifdef sections?).

Fixed by https://github.com/google/iree/pull/5707
The new iree/runtime/ higher level API only requires a single include: https://github.com/google/iree/blob/b5f3dda010b4b3ed4982f11c3e3b72277ad84451/iree/runtime/demo/hello_world_terse.c#L15-L16
This isn't yet documented in https://google.github.io/iree/bindings/c-api/ (source) though.

repo: google/iree


text:
Merge google -> main

8db0183c Merge main -> google
8f98e0ae Synchronize submodules with LLVM at llvm/llvm-project@f3f3c9c2549a

Should fix the merge commit from Copybara before merging this ("files changed" should be nearly empty)

Should fix the merge commit from Copybara before merging this ("files changed" should be nearly empty)

authors: NatashaKnk ScottTodd


text:
Updated Course3/Lab4/validations.py script.

Closes: #1
In line 16 added a regex field for the first character to verify that its a letter.
Fixes #1
added tests
validate_user("blue.kale", 3) -> True
validate_user(".blue.kale", 3) -> False
validate_user("red_quinoa", 4) -> True
validate_user("_red_quinoa", 4) -> False

@googlebot I signed it!

@googlebot I fixed it.

repo: google/it-cert-automation-practice


text:
Update validation.py python script.

Update validation.py python script.
Fixed the behavior of validated_user function in validation.py.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

This is not the branch I intended to merge


text:
Faster versions of isin, in1d, unique for integer arrays

Recently, faster versions of isin, in1d, and unique were added or proposed to numpy for 1D integer arrays, using lookup tables:

https://github.com/numpy/numpy/pull/12065/
https://github.com/numpy/numpy/pull/21843/

I am wondering about the feasibility of porting these to JAX. Currently, the JAX version of in1d is as follows:
https://github.com/google/jax/blob/7e09e63faaa54320e3ddf318a08d38b0bb54afe5/jax/_src/numpy/setops.py#L55-L58
This is $O(n^2)$ in both memory usage and compute for arrays of size $n$.
This proposed method, similar to a counting sort, is only $O(n+k)$ in compute and $O(k)$ in memory usage, for $k$ the range of input data. It's similarly reliant on vectorized array operations, rather than sorts, so it should be even faster in XLA than the current method.
Now, since JAX recompiles a function when the types change (?), so the control flow should not be an issue if this new method is only used for integer arrays. However, the array allocation will be equal to the range of input data (similar to a counting sort).
Therefore I am wondering if it is feasible to add this to JAX. The following implementations are possible:

Add a kind parameter to isin, in1d, and unique, where kind="table" triggers this new faster method regardless of required allocation. Turn this method off by default.
Using lax.cond, check the range of the input data, and only if the range is not too big, switch to this new method by default.

The other possibility which I strongly recommend against is to use this new method by default if the input arrays are integers. The reason I recommend against this is because the memory allocation may be massive even for small input arrays. (e.g., unique([1, 1_000_000]) would allocate a one million-length array).
What do you think? I am happy to submit a PR if this is desirable.
CC'ing @jakevdp who looks to have worked on the set operations.
Cheers,
Miles

For jit compatibility, one could have a parameter to isin specifying the maxsize of the lookup table (e.g., 10,000), then do dynamic slicing of that.
If desired, I think the same thing could be done for unique, with the extra unused portions being set to nan or something.

FWIW, lax.cond only works without vmap, its batching rule is lax.select, which is not lazy.
I think user can implement such optimization by themselves.

Hi Jake,
Great questions. The cool thing about this method is it is not technically hash-based. The closest analogy to this comparison would be counting sorts vs radix sorts. A radix sort (https://en.wikipedia.org/wiki/Radix_sort) uses a hash table, whereas a counting sort (https://en.wikipedia.org/wiki/Counting_sort) pre-allocates an array of length equal to the range of integer data. A radix sort has scaling O(n*k), for k the key size, and a counting sort has scaling O(n+k), for k the range of your data. However, a counting sort might have a better constant in front of the scaling, for fixed k, since it avoids having to create the hash table, and gets to use super efficient array operations. This method is analogous to a counting sort, but a radix sort-like method is described on that blog (and would be useful for generic types).
(copied parts of my answer on the numpy mailing lists to a similar question: https://mail.python.org/archives/list/numpy-discussion@python.org/thread/3SJTDDF7D4VLZIUPQKHJMSYU2RVY3MHH/)
Here is basically how the method works, for jnp.unique:

calloc a "lookup table" with each element corresponding to an integer: x = jnp.zeros(MAX_RANGE)
Fill that table with 1 where a number exists in the array: x[ar - ar_min] = 1
Return the indices of elements with 1: return jnp.nonzero(x) + ar_min

So, the only "hash" here is hash(integer) = integer.
How fast are non-contiguous indexing operations in XLA, like segment_sum? If they are quick, then I think this method will be optimized in XLA. As long as the user chooses kind="table", and pre-specifies a max range, then it should be optimized in XLA.
Thanks!
Miles

Looking at this again: for JAX, I think this is much less useful than for numpy due to the static array size criterion. We could not set kind="table" by default, because this would make these functions incompatible with JAX transforms like jit, vmap, and others unless the user provides additional static information about the size of the required table. This means it would be a breaking change.
On the other hand, we could use the default of kind="sort", which would be backward compatible but would probably be ignored by the vast majority of users of these functions.
That said, I'm still in support of adding this argument, because numpy is adding it and we strive for feature parity with numpy.

@MilesCranmer are you interested in preparing a PR for this?

Sure, I can get around to this.

Closing since there's not been much action. Feel free to re-open this if you're interested in working on it!

Hey, sorry for never getting around to it @jakevdp... I sadly find I'm just too busy these days maintaining things 😪

authors: MilesCranmer YouJiacheng jakevdp

repo: google/jax


text:
Optix documentation

The optix optimization library has docstrings for most of its classes already, but it is not included in the docs, is that by design?
Edit: @tomhennigan @gnecula

I think optix is in a slightly odd space where it is in jax.experimental to make it easy for folks to try but it is also an orthogonal project to JAX (to me it is clearly a library that sits on top of JAX).
@gnecula @mtthss should we consider putting optix in its own repo? It is a very simple project of course, but it's not clear to me that it should live in JAX core and be documented alongside it.

I agree that optix being in jax.experimental is not ideal. It is indeed a library on top of JAX. Putting it into its own repo feels costly though (will need its own Travis, Readthedocs, copybara, ... settings).
(How about optimizers.py, should be part of the same discussion?)
@jekbradbury WDYT?

Meanwhile, I am adding the documentation

Optix is a great little library! But my weak opinion is that it should be in its own repo, for these weak reasons:

no JAX core devs are very familiar with the code and we have no vision for it, and so it's not something we could easily maintain or develop (i.e. we're relying on @mtthss) which makes it different from the other code in the JAX repo;
fixes and improvements by @mtthss, @tomhennigan, and others have to be merged into this repo by JAX core devs, which slows progress when we're slow to respond (@mtthss is relying on us);
issues like this one incur a small but nonzero time cost for the JAX core team.

Those are only weak reasons because the code is so concise and currently low-velocity. If the code were big, or being developed rapidly, we'd have already felt the pain and decided to move it out (or never merged it at all). But perhaps regardless of size the same principles could be applied.
If it were deemed too small to be in its own repo (though it's not clear to me there is such a minimum size!), it's not clear to me whether it's better to be bundled with JAX or with, say, Haiku.
Overall, though, I want to do whatever is best for @tomhennigan and @mtthss because this isn't costing us much. Even though my weak opinion is that it should be taken out of JAX, I defer to your judgement. (Though if you're ambivalent, we'll take it out!)
(By the way, #1620 is where it got merged, but there doesn't seem to be a relevant discussion in that PR thread.)

IIUC this specific issue was solved by #2312, so I'm going to close it. I'll start an email thread about taking optix out of jax.

authors: gnecula joaogui1 mattjj tomhennigan


text:
Automatically doing TensorFloat32 Math on Ampere GPUs

NVIDIA's new Ampere cards support much faster fp32 computation by doing certain operations on TensorCores at a lower precision and accumulating them in fp32. This lower precision format for math is called TensorFloat32 (TF32). TensorFlow 2.4 and PyTorch 1.7 both enable TF32 math by default, and provide flags to disable it.
Can this be supported in JAX as well? Thanks!

Thanks for the question! So it sounds like this is a specialized dtype somewhat similar to bfloat16, which JAX does support, so there is definitely precedent for this sort of thing. That said, for JAX to support TF32, I think XLA would have to support it first, and I'm not certain whether it does. Maybe @jekbradbury might know?

So the TF32 case is simpler than bfloat16 because it's not actually a storage format (so, for instance, we would not have to include TF32 in the JAX type system or make it possible to create a TF32-typed array). Instead, it's a configuration option for matrix multiplies and convolutions with float32 inputs running on an NVIDIA A100 chip. We have a precedent for this: matmuls and convolutions with float32 inputs running on TPU chips can use lax.Precision.DEFAULT (rounding to bfloat16), lax.Precision.HIGH (essentially the same as TF32), and lax.Precision.HIGHEST (full float32 precision). We might want to change the names or otherwise modify the precision API, but that's where TF32 support would go.
As a prerequisite, XLA:GPU needs to support setting this configuration option on cuBLAS and cuDNN calls. I think that's WIP, and we'll update when it's ready.

I'm actually not 100% sure that this isn't already enabled in XLA: the relevant changes have been merged into the TF tree and I believe shared with XLA. But I don't have any Ampere GPUs to test with. Can someone with such a GPU try it out?

@hawkinsp I tried on an A100 and it looks like it's not yet enabled in TF-XLA. I posted an issue in TensorFlow with the details of how I tested this: https://github.com/tensorflow/tensorflow/issues/44887.

@n2cholas @hawkinsp  I got JAX to work with TF32 and Flax implements mixed-precision training as well. Although JAX didn't explicitly implement TF32 noobs such as call cublasSetMathMode when creating cublas handles.
NVIDIA NGC container provides an environment variable NVIDIA_TF32_OVERRIDE=0, see more here. that enable TF32 regardless how the framework call cuda libraries.
Simply install JAX into https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow and you got TF32 enabled.

Setting NVIDIA_TF32_OVERRIDE=1 does not fix the issue for me. Ther performance is identical with or without. Is there any update on this issue?

According to the official documentation, NVIDIA_TF_OVERRIDE=0 can only disable tensor cores, it cannot enable them.

Update: with the newest version TF32 is enabled by default:
(pn) gaoni@mdsi-gpu02:~$ ipython
Python 3.8.12 (default, Oct 12 2021, 13:49:34)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.0.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import jax.numpy as jnp

In [2]: import numpy as np

In [3]: A = jnp.array(np.random.normal(size=(4096, 4096)))

In [4]: %timeit (A@A).block_until_ready()
1.25 ms ± 149 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

(pn) gaoni@mdsi-gpu02:~$ NVIDIA_TF32_OVERRIDE=0 ipython
Python 3.8.12 (default, Oct 12 2021, 13:49:34)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.0.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import numpy as np

In [2]: import jax.numpy as jnp

In [3]: A = jnp.array(np.random.normal(size=(4096, 4096)))

In [4]: %timeit (A@A).block_until_ready()
7.56 ms ± 138 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

Thanks @n-gao! I get similar results as well with an RTX 3090, CUDA 11.6, CUDNN 8.2:
(test-env) nicv@nicv-desktop ~> ipython
Python 3.10.2 (main, Jan 15 2022, 18:02:07) [GCC 9.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import jax.numpy as jnp; import numpy as np

In [2]: A = jnp.array(np.random.normal(size=(4096, 4096)))

In [3]: %timeit (A@A).block_until_ready()
3.98 ms ± 21.2 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [4]: %timeit (A@A).block_until_ready()
4 ms ± 11.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [5]:
Do you really want to exit ([y]/n)? y
(test-env) nicv@nicv-desktop ~> NVIDIA_TF32_OVERRIDE=0 ipython
Python 3.10.2 (main, Jan 15 2022, 18:02:07) [GCC 9.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import jax.numpy as jnp; import numpy as np

In [2]: A = jnp.array(np.random.normal(size=(4096, 4096)))

In [3]: %timeit (A@A).block_until_ready()
5.64 ms ± 123 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [4]: %timeit (A@A).block_until_ready()
5.94 ms ± 16.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

authors: hawkinsp jakevdp jekbradbury mengdong n-gao n2cholas


text:
Use lax.iota also for jnp.arange(0, stop)

This PR updates jnp.arange to also use lax.iota when calling it with jnp.arange(0, stop). jnp.arange(0, stop) and jnp.arange(stop) will now return the same Jaxpr.
This was inspired by https://github.com/google/jax/pull/10176#discussion_r846166229 as jnp.arange is often called with an explicit start argument that equals zero.

Want to add a test using jax.make_jaxpr? I can advise more if that's unclear.

Want to add a test using jax.make_jaxpr?

Good idea! I added a small test that verifies the Jaxpr and a test that checks for the correct floating behaviour.

@mattjj or @jakevdp looks like this PR slipped through the cracks, do you mind taking a quick look at it?

authors: lgeiger mattjj


text:
Add tree_labels, tree_print.

This PR follows from https://github.com/tensorflow/tensorflow/pull/56202, where the pytree walk function has been extended to pass node_data to walkers.  For example, this allows implementation of custom printers (see https://github.com/google/jax/issues/5047), which can e.g. print a tree with path labels like:
obj/a1 = leaf-a1
obj/a2/b1/0 = 111
obj/a2/b1/1 = 2
obj/a2/b1/2 = 3
obj/a2/b2 = [[11. 12. 13.]; [21. 22. 23.]]
obj/a2/b3/c1 = 33
obj/a2/b3/c2/0 = 44
obj/a2/b3/c2/1 = 55
obj/a3/x1/x11 = a
obj/a3/x1/x12/y1/y11 = ya
obj/a3/x1/x12/y1/y12 = yb
obj/a3/x1/x13 = b
obj/a3/x2/x21 = 2a
obj/a3/x2/x22/y1/y11 = ya
obj/a3/x2/x22/y1/y12 = yb
obj/a4 = fred

@hawkinsp as you are looking at the corresponding TF PR, this is the JAX one.

@hawkinsp I have rebased this on top of main - hopefully it's OK to approve given the TF change went in.

@hawkinsp I have rebased this on top of main - hopefully it's OK to approve given the TF change went in.


text:
Possible bug in tree_where when tree has a single leaf

I'm not entirely sure, but I suspect that tree_where is assuming that a pytree with a single leaf is an np array, which can lead to issues of "TypeError: Field elements must be 2- or 3-tuples,". In my case, I had a list of a single numpy array. I couldn't directly test my hypothesis because I was using tree_where through OSQP, but when I hacked my code for the primal variable in OSQP to have two leaves, the code worked.

CC @Algue-Rythme

Hi Ferran
Could you give me more details on the failure mode of this function ?
I am not sure to understand which input format would cause those errors.

Fixed by #164

authors: Algue-Rythme FerranAlet mblondel

repo: google/jaxopt


text:
How to prevent jitting in jaxopt solver?

Dear jaxopt team,
I am writing to inquire about an issue I encountered when using jaxopt's LBFGS solver with a custom objective function that is not jittable but has defined custom VJPs. Despite specifying jit=False to disable JIT compilation, the solver still throws a TracerArrayConversionError as if it was attempting to JIT-compile the objective function. I have prepared a minimal example to reproduce the issue:
import jax
import numpy as np
from jax import custom_vjp
from jaxopt import LBFGS

@custom_vjp
def f(x):
    # The function deliberately uses numpy functions which are not jittable
    return -(x[0] + np.sin(x[0])) * np.exp(-x[0]**2.0)

def f_fwd(x):
    return f(x), (x, )

def f_bwd(x, g):
    return -(1.0 + np.cos(x[0])) * np.exp(-x[0]**2.0) - 2.0 * x[0] * (
        x[0] + np.sin(x[0])) * np.exp(-x[0]**2.0),

f.defvjp(f_fwd, f_bwd)

print_errors = False  # Flag for printing the errors

# Check if the custom function is jittable
try:
    f_jitted = jax.jit(f)
    f_jitted(1.0)
    print("Function is jittable")
except Exception as e:
    print("Function is not jittable")
    if print_errors:
        print("Error: ", e)

# Check if the custom function is differentiable
try:
    f_grad = jax.grad(f)
    print("Function is differentiable")
except Exception as e:
    print("Function is not differentiable")
    if print_errors:
        print("Error: ", e)

# Run the solver (optimal input is around 0.679579)
solver = LBFGS(fun=f, jit=False, maxiter=10)
res = solver.run(np.array([1.0]))

If my understanding is correct, setting jit=False should prevent JIT compilation of the objective function. However, this does not seem to be the case. Could you please let me know if I am missing something here or is this perhaps a bug?

Update: I noticed that my implementation of the backward derivative f_bwd was incorrect. Here is an updated example:
import jax
import numpy as np
#import jax.numpy as np
from jax import custom_vjp
import jaxopt

@custom_vjp
def f(x):
    # The function deliberately uses numpy functions which are not jittable
    return -(x[0] + np.sin(x[0])) * np.exp(-x[0]**2.0)

def f_fwd(x):
    return f(x), (x, )

def f_bwd(res, g):
    x = res[0]
    grad_x = np.exp(-x[0]**2) * (2 * x[0]**2 + 2 * x[0] * np.sin(x[0]) - np.cos(x[0]) - 1)
    return (grad_x * g,)

f.defvjp(f_fwd, f_bwd)

print_errors = False  # Flag for printing the errors

# Check if the custom function is jittable
try:
    f_jitted = jax.jit(f)
    f_jitted(np.array([1.0]))
    print("Function is jittable")
except Exception as e:
    print("Function is not jittable")
    if print_errors:
        print("Error: ", e)

# Check if the custom function is differentiable
try:
    f_grad = jax.grad(f)
    f_grad(np.array([1.0]))
    print("Function is differentiable")
except Exception as e:
    print("Function is not differentiable")
    if print_errors:
        print("Error: ", e)

# Run the solver (optimal input is around 0.679579)
solver = jaxopt.LBFGS(fun=f, jit=False, maxiter=10000)
res = solver.run(np.array([0.0]))

solution = res.params
ground_truth = np.array([0.679579])

print("Solution: ", solution)
print("Ground truth: ", ground_truth)
print("Error: ", np.linalg.norm(solution - ground_truth))
print("Ground truth value ", f(ground_truth))
print("Ground truth gradient ", jax.grad(f)(ground_truth))
print("Solution value: ", f(solution))
print("Solution gradient: ", jax.grad(f)(solution))

However, now I got a different error, which might be related to this particular example:
Traceback (most recent call last):
  File "/home/igork/IRP/NeurOpt/jax-am-forked/applications/fem/top_opt/jaxopt_jitting.py", line 49, in <module>
    res = solver.run(np.array([0.0]))
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/base.py", line 354, in run
    return run(init_params, *args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/implicit_diff.py", line 251, in wrapped_solver_fun
    return make_custom_vjp_solver_fun(solver_fun, keys)(*args, *vals)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/traceback_util.py", line 166, in reraise_with_filtered_traceback
    return fun(*args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/custom_derivatives.py", line 614, in __call__
    out_flat = custom_vjp_call_p.bind(flat_fun, flat_fwd, flat_bwd,
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/custom_derivatives.py", line 763, in bind
    outs = top_trace.process_custom_vjp_call(self, fun, fwd, bwd_, tracers,
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/core.py", line 809, in process_custom_vjp_call
    return fun.call_wrapped(*tracers)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/linear_util.py", line 188, in call_wrapped
    ans = self.f(*args, **dict(self.params, **kwargs))
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/implicit_diff.py", line 207, in solver_fun_flat
    return solver_fun(*args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/base.py", line 316, in _run
    opt_step = self.update(init_params, state, *args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/lbfgs.py", line 313, in update
    product = inv_hessian_product(pytree=grad, s_history=state.s_history,
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/lbfgs.py", line 109, in inv_hessian_product
    return tree_map(fun, pytree, s_history, y_history)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/tree_util.py", line 210, in tree_map
    return treedef.unflatten(f(*xs) for xs in zip(*all_leaves))
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/tree_util.py", line 210, in <genexpr>
    return treedef.unflatten(f(*xs) for xs in zip(*all_leaves))
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/lbfgs.py", line 60, in inv_hessian_product_leaf
    r, alpha = jax.lax.scan(body_right, v, indices, reverse=True)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/traceback_util.py", line 166, in reraise_with_filtered_traceback
    return fun(*args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/lax/control_flow/loops.py", line 257, in scan
    _check_scan_carry_type(f, init, out_tree_children[0], carry_avals_out)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jax/_src/lax/control_flow/loops.py", line 314, in _check_scan_carry_type
    raise TypeError(
jax._src.traceback_util.UnfilteredStackTrace: TypeError: Scanned function carry input and carry output must have equal types (e.g. shapes and dtypes of arrays), but they differ:
  * the input carry r has type float32[] but the corresponding output carry component has type float32[1], so the shapes do not match

Revise the scanned function so that all output types (e.g. shapes and dtypes) match the corresponding input types.

The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.

--------------------

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/igork/IRP/NeurOpt/jax-am-forked/applications/fem/top_opt/jaxopt_jitting.py", line 49, in <module>
    res = solver.run(np.array([0.0]))
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/base.py", line 354, in run
    return run(init_params, *args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/implicit_diff.py", line 251, in wrapped_solver_fun
    return make_custom_vjp_solver_fun(solver_fun, keys)(*args, *vals)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/implicit_diff.py", line 207, in solver_fun_flat
    return solver_fun(*args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/base.py", line 316, in _run
    opt_step = self.update(init_params, state, *args, **kwargs)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/lbfgs.py", line 313, in update
    product = inv_hessian_product(pytree=grad, s_history=state.s_history,
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/lbfgs.py", line 109, in inv_hessian_product
    return tree_map(fun, pytree, s_history, y_history)
  File "/home/igork/miniconda3/envs/jax-nrto/lib/python3.10/site-packages/jaxopt/_src/lbfgs.py", line 60, in inv_hessian_product_leaf
    r, alpha = jax.lax.scan(body_right, v, indices, reverse=True)
TypeError: Scanned function carry input and carry output must have equal types (e.g. shapes and dtypes of arrays), but they differ:
  * the input carry r has type float32[] but the corresponding output carry component has type float32[1], so the shapes do not match

Revise the scanned function so that all output types (e.g. shapes and dtypes) match the corresponding input types.

I also tried  two different solvers - GradientDescent worked as expected while BFGS recreated the TracerArrayConversionError I mentioned at the beginning.

Hi! Thanks for the bug report. There were two problems here.
One on your side: the VJP should return an array of size 1, not a scalar (the output of the VJP should always have the same shape as the input of the function)
    return (np.array([grad_x * g]),)

One on our side: zoom line search and Hager-Zhang line search don't work with non-jittable functions for the moment (we'll fix this). As a temporary workaround, you can use the backtracking linesearch
solver = jaxopt.LBFGS(fun=f, jit=False, maxiter=10, linesearch="backtracking")

Hi @mblondel. Thank you so much for your response! I really appreciate the fact that the team is already working on addressing this!


text:
xref.proto: remove deprecated fields [phab:D1646]

Created by schroederc at 2017-05-17 16:55:41:

fromberger accepted this revision at 2017-05-17 17:05:03

schroederc closed via commit 831e6bdfb5a578ba7cd96cf5b36b054ff6cb4ca9 at 2017-05-17 17:30:15

repo: google/kythe


text:
Revert "Added logic to extract both baseName and qualifiedName from marked source extractor." [phab:D2088]

Created by craigbarber at 2017-12-08 00:50:32:
This reverts commit 6a782c9fbc99d2c1e23cf3a982d92cdfe92b6d43.
After an offline discussion with zarko we came to the realization that the baseName logic here is not correct. Will re-submit this change with the corrected logic.

zarko accepted this revision at 2017-12-08 00:55:30

craigbarber closed via commit 460d81ddb9530ece9a5a28c19598d64995506073 at 2017-12-08 00:58:20


text:
zephyr: Port liblc3 to Zephyr as external module

Add Zephyr module definition so that the project can be used as Zephyr
external module.
Fixes: https://github.com/zephyrproject-rtos/liblc3codec/issues/10
Signed-off-by: Mariusz Skamra mariusz.skamra@codecoup.pl

Related Zephyr PR: https://github.com/zephyrproject-rtos/zephyr/pull/46970


text:
gulp failed to load external module babel

hi everyone i'm always get this
`RoliesDonalds-MacBook-Pro:berita roliesdonald$ gulp
[15:32:41] Failed to load external module babel-core/register
[15:32:41] Failed to load external module babel/register
/Users/roliesdonald/Documents/job/newsweb/berita/gulpfile.babel.js:25
import fs from 'fs';
^^^^^^
SyntaxError: Unexpected reserved word
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:413:25)
at Object.Module._extensions..js (module.js:448:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Liftoff.handleArguments (/Users/roliesdonald/.nvm/versions/io.js/v3.3.1/lib/node_modules/gulp/bin/gulp.js:116:3)
at Liftoff. (/Users/roliesdonald/.nvm/versions/io.js/v3.3.1/lib/node_modules/gulp/node_modules/liftoff/index.js:198:16)
at module.exports (/Users/roliesdonald/.nvm/versions/io.js/v3.3.1/lib/node_modules/gulp/node_modules/liftoff/node_modules/flagged-respawn/index.js:17:3)`
my gulp version is
[RoliesDonalds-MacBook-Pro:berita roliesdonald$ gulp -v [15:40:05] Failed to load external module babel-core/register [15:40:05] Failed to load external module babel/register [15:40:05] CLI version 3.9.0 [15:40:05] Local version 3.9.0
anybody can help me please....

@donrolies I assume you're building from the current state of mdl-1.x? There's several possible causes for your issue, so I'd recommend:

git pull, to make sure you have the latest changes, including all the recent fixes.
Double-checking you have the latest package.json, which includes the correct Babel configuration for the latest Babel releases.
Running npm i to make sure you have the latest version of every package installed.

Looking at your output, it also looks like you're running io.js. That's a pretty old runtime, so I'd recommend upgrading to a stable (v7.x) or LTS (v6.9.x) version of node.js.
I've run a clean build from scratch to confirm everything is right in mdl-1.x, and it looks that way, so I'm closing this issue.

hi @sgomes this is my
https://github.com/RoliesDonald/mdl-sass
stil had problem
RoliesDonalds-MacBook-Pro:mdl-sass roliesdonald$ gulp [13:19:39] Requiring external module babel-register [13:19:40] Using gulpfile ~/Documents/localgithub/mdl-sass/gulpfile.babel.js [13:19:40] Starting 'clean'... [13:19:40] Finished 'clean' after 14 ms [13:19:40] Starting 'default'... [13:19:40] Starting 'styles'... [13:19:40] Starting 'styles-grid'... [13:19:41] styles-grid all files 18.82 kB [13:19:41] Finished 'styles-grid' after 995 ms [13:19:43] styles all files 812.97 kB [13:19:43] Finished 'styles' after 3.07 s [13:19:43] Starting 'lint'... [13:19:44] 'lint' errored after 740 ms [13:19:44] TypeError: Cannot convert undefined or null to object     at Function.keys (<anonymous>)     at copyConfiguration (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/config/configuration.js:920:12)     at NodeConfiguration.Configuration._processConfig (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/config/configuration.js:459:5)     at NodeConfiguration.Configuration.load (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/config/configuration.js:211:10)     at Checker.configure (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/string-checker.js:62:29)     at Checker.configure (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/checker.js:27:39)     at Object.module.exports (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/gulp-jscs/index.js:35:10)     at Gulp.<anonymous> (/Users/roliesdonald/Documents/localgithub/mdl-sass/gulpfile.babel.js:107:13)     at module.exports (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/orchestrator/lib/runTask.js:34:7)     at Gulp.Orchestrator._runTask (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/orchestrator/index.js:273:3) [13:19:44] 'default' errored after 3.83 s [13:19:44] TypeError in plugin 'run-sequence(lint)' Message:     Cannot convert undefined or null to object Stack: TypeError: Cannot convert undefined or null to object     at Function.keys (<anonymous>)     at copyConfiguration (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/config/configuration.js:920:12)     at NodeConfiguration.Configuration._processConfig (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/config/configuration.js:459:5)     at NodeConfiguration.Configuration.load (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/config/configuration.js:211:10)     at Checker.configure (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/string-checker.js:62:29)     at Checker.configure (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/jscs/lib/checker.js:27:39)     at Object.module.exports (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/gulp-jscs/index.js:35:10)     at Gulp.<anonymous> (/Users/roliesdonald/Documents/localgithub/mdl-sass/gulpfile.babel.js:107:13)     at module.exports (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/orchestrator/lib/runTask.js:34:7)     at Gulp.Orchestrator._runTask (/Users/roliesdonald/Documents/localgithub/mdl-sass/node_modules/orchestrator/index.js:273:3)

authors: RoliesDonald sgomes

repo: google/material-design-lite