text:
Support direct quantization for FP8 matmul

Historically, FP8 matmul quantization followed the pattern of fake quantization, which involved a sequence of operations: quantization -> dequantization -> dot. Here, (de)quantization refers to type casting and the application of scaling factors. The XLA GemmWriter pass was designed to transform this pattern into a custom cublasLt call.
This PR proposes a departure from the historical approach by adopting direct quantization, which is quantization -> dot -> dequantization. This adjustment aligns better with mainstream quantization implementations for other data types. However, the success of this PR hinges on another PR in JAX (PR-21211) because of the mixed fp8 type matmul.
cc @lukaszlew @kaixih

I think the new design is much clearer of the idea of direct quantization. By the way, do you think we should create a new Fp8DotGeneral op for it and keep the existing fake quant Op untouched? And then we gradually change downstream uses to migrate to the new op?

I think the new design is much clearer of the idea of direct quantization. By the way, do you think we should create a new Fp8DotGeneral op for it and keep the existing fake quant Op untouched? And then we gradually change downstream uses to migrate to the new op?

Praxis doesn't use Fp8DotGeneralOp directly but dot_general_with_precision. Given that most PAXML models access fp8 from praxis. So instead, I think it makes more sense to keep both fake and direct quant Op there.

A gentle reminder to @lukaszlew

Sorry, I don't have cycles to review this PR. I'm focusing on AQT.

Could we get @levskaya to help review since Lukasz is busy with something else these days?

@levskaya could you take a look?

Sorry we block if trailing spaces are left in the file, there's some after the line class Fp8Test(parameterized.TestCase): in the tests (you can see the failed precommit) - could you fix and we can run the tests.   I think things look ok.

@levskaya I think all the issues have been resolved by @wenscarl . Can you help review and merge?

@levskaya Just resolved some formatting issues. I think all the tests should pass now. Can you help review and merge?

@wenscarl @kaixih - hey also, I invited you as collaborators to this repo, apologies that I hadn't done that earlier, it should immediately trigger presubmit tests once you join.

@wenscarl - I'm seeing a failed test?
E       UserWarning: The function dot_general_with_precision will set the precision/preferred_element_type and disregard any provided values.

@levskaya Thanks for reviewing. All checks are passed.

authors: kaixih levskaya lukaszlew wenscarl zhangqiaorjc


text:
glog error log was incorrrectly written to unexpected memory

Hi,
I know glog ERROR log message will be written to stderr by default. Most of the time, it work as expected, but occasionally I found it will not be written into stderr and was written into some unexpected memory.
I have checked my source code and then I found there's something that I may did incorrectly:
class NullLogger : public google::base::Logger
{
	virtual void Write(bool, time_t, const char*, int) {

	}
	virtual void Flush() {

	}
	virtual google::uint32 LogSize() {
		return 0;
	}
};

void init()
{
      ...
      google::SetLogDestination(google::GLOG_INFO, fullpath.c_str());

      // do not generate separate files for warning and error.
	static NullLogger nullLogger;
	google::base::SetLogger(google::GLOG_WARNING,&nullLogger);
	google::base::SetLogger(google::GLOG_ERROR,&nullLogger);
       google::InitGoogleLogging("");
}

note I called google::base::SetLogger() before google::InitGoogleLogging().
If I called google::InitGoogleLogging() first, then call google::base::SetLogger(), just like following code, everything seems work fine, but I can't explain why.
void init()
{
      ...
      google::SetLogDestination(google::GLOG_INFO, fullpath.c_str());
      google::InitGoogleLogging("");
      // do not generate separate files for warning and error.
	static NullLogger nullLogger;
	google::base::SetLogger(google::GLOG_WARNING,&nullLogger);
	google::base::SetLogger(google::GLOG_ERROR,&nullLogger);

}

I know for glog some operations should be done before google::InitGoogleLogging(), such as "setting value of FLAGS_log_dir".
I have checked google::InitGoogleLogging() source code, it looks it only defines the g_main_thread_id and marked if it has been initialized.  my question is:  if I call google::base::SetLogger() before google::InitGoogleLogging(), will there be some issue that you can imagine?

even SetLogger() after init(), still same issue, I'll go to check by myself

source: gharchive/issue

repo: google/glog

license_type: permissive

license_source: github-api


text:
Pass urls map (Endpoint) to acme Client

That way we could skip url string param in each call to the client. Maybe provide Endpoint as additional parameter to acme client, maybe additional wrapper.

I see what you mean. Some of the url args of the client refer to an account URI, not Endpoint.
Do you think it would make sense to merge Account and Client at this point? Otherwise, it may feel inconsistent - some method will require url, others wont'

For instance, Register indeed takes an Endpoint url, while GetReg requires Account.URI.

Fixing this in https://go-review.googlesource.com/23972.

Fixed by https://github.com/golang/crypto/commit/c2f4947f41766b144bb09066e919466da5eddeae.
(already synched this repo with that change)

repo: google/goacme


text:
Wrongly Deleting Sites

On the Goolge Authenticator iPhone app, I tried to delete my login for one site (let's say Facebook) and it initially didn't disappear. So I tried to delete it again, after which not only had that login disappeared but also another login for another site (let's say Amazon) was gone too. This got me locked out of Amazon.
I know there's no way to prove this but I promise I didn't accidentally delete the Amazon login the second time. The fact that it then disappeared makes me think that somehow the id's of the different logins are getting reassigned after a login is deleted, which (in a case like this where for some reason there was a delay in the first login being removed from my screen) made it possible for me to delete a login named "Facebook" and cause my Amazon login to disappear instead.
It goes without saying but given the nature of GA this seems like a critical bug -- it could (and did) get me completely locked out of a service I was using (Amazon) because I was suddenly left without the 2FA code I needed to log in.

This is for the current app store version of Authenticator, right? Do you know of any way to reliably reproduce this bug? Unfortunately, there's not much anybody can do to investigate or fix this unless the bug is reproducible.

Does this work?
Steps to reproduce:

Use GA on iPhone
Have multiple logins listed (let's say Google, Amazon, Facebook)
Delete the last login (Facebook)
Before the app has fully refreshed/updated and removed the Facebook login from your screen, delete the same login again
After app has processed this, instead of just deleting the Facebook login (and throwing an error the second time around), it will have deleted both the Facebook and Amazon logins (i.e. the last and second-last login in your list of logins).

I believe @cbjartli had the same issue, perhaps he has more details?

authors: MrPickles marbiru

repo: google/google-authenticator

license_source: bigquery


text:
Face ID

This repository is for the iPhone and Blackberry app.
Please note that the code in this repository is an open source fork of the app in the App Store, so there is no guarantee that both apps will be in sync. Please check which version of the app your bug applies to below.

[ ] This issue is present in the app downloaded from the [App Store][appstore].
[ ] This issue is with the [Github version][opensource] of the app.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Better security
Describe the solution you'd like
Google auth app using Face ID technology to approve of logins instead of generated codes
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
n/a
Additional context
Add any other context or screenshots about the feature request here.
n/a
[appstore]: https://apps.apple.com/gb/app/google-authenticator/id388497605
[opensource]: https://github.com/google/google-authenticator

What? Please clarify.
And please fill in the template.

I'm pretty sure @spindriftur means that they want FaceID/Touch ID for displaying the generated codes, rather than showing them plainly when you open the app.

Seems to lean that way. But since they have not bothered to fill in the rest of the template or clarify for two weeks, I'm going to assume they're not actually that interested.

authors: ThomasHabets mygizli04 spindriftur


text:
Extract extended DNS Rcode from OPT record

Add decoding of the extended DNS ResponseCode from OPT records as specified in RFC 6891 section 6.1.3.
Additionally also adds the BADCOOKIE DNS ResponseCode (from RFC 7873) to the list of known values.

Thanks

source: gharchive/pull-request

authors: notti x-way

repo: google/gopacket


text:
Create a new data class for representing locally stored TileSource

Currently we use the same class in different contexts. We should create a new one for better readability.

We now have OfflineAreaEntity to represent this.

repo: google/ground-android


text:
Wiki is out of date about disabling the bridge ClassLoader

https://github.com/google/guice/wiki/ClassLoading#disabling-guices-bridge-classloader says to pass -Dguice.custom.loader=false to disable the bridge ClassLoader.  But since 0c3a880d89bda0fe769f9fb422eb212b72934e66, the correct option is -Dguice_custom_class_loading=OFF.

Fixed, thanks!

authors: sameb tavianator

repo: google/guice


text:
CGGI / TFHE scheme: Common LUT optimizations?

Some LUTs generated by Yosys for a circuit seem more common that others - namely, 128, 120, 1, and others.
It may be an interesting optimization to target these LUTs to some optimized modules, or to use some linear rewrites for these LUTs and avoid the bootstrap operation in the PBS.

These LUTs correspond to the following operations, where the input bits a, b, c are combined into a single value via concatenating their bits, or equivalently input = 4*c + 2*b + a. The "operands" below are referring to a, b, c.

1: output 1 if all operands are 0, otherwise 0
128: output 1 if all operands are 1, otherwise 0
120: output 1 if input >= 3, i.e., if c=1 or (b=1 and a=1)

For fun, I tried asking GPT4 to interpret the third LUT, and it was unable to.

It will be a tradeoff with increased noise to use multiplications for the 1 and 128, but it's possible that passes that find better bootstrapping placement mitigates that!

Combing the tfhe-rs codebase, I didn't find anything existing that handles these without a bootstrap.

repo: google/heir


text:
Modernize and relocate iree/runtime Python package to runtime/bindings/python.

Non-functional changes:

Moves bindings/python/iree/runtime -> iree/runtime/python/iree/runtime.
Fixes dash vs undescore inconsistency in compile install path. Now both use underscores (python_packages/iree_compiler and python_packages/iree_runtime).
Moves build directory for iree/compiler/python to iree/compiler/python (was outputting to bindings/python). Updates locations that were hard-coded.

Functional changes:

Removes the old build-dir only setup.py in favor of an iree/runtime/setup.py that works from either the source or build dir.
Reworks the releases to use the new setup.py as-is vs scripting the build manually.
iree.runtime.version is now generated in the same way as iree.compiler.version.
Users can now run iree/runtime/setup.py with pip themselves to generate a wheel (i.e. pip wheel iree/runtime).

It is now possible to integrate python package testing into the presubmit and have build jobs that generated Python installable binaries for subsequent steps.
The only file left in bindings/python is build_requirements.txt. It is referred to by some docs and CI jobs so leaving as-is for the moment (will find it a new home in a followup).

Confession: I put this in a suboptimal place so that Ben and I could have a conversation about repo layout... which has now happened :)
Here's the new plan: Ultimately, we want top-level compiler and runtime directories that are self contained. It will take a few steps to get there. Treating this as a first step, I will move reorganize this commit as:

runtime/setup.py
runtime/pyproject.toml
runtime/bindings/python/...

A future PR will move the runtime C code into runtime/src. The iree/compiler directory is already self contained and could also have a fairly trivial directory transpose done on it to become the top-level compiler/ directory. That is also separable.
Generally, we would like language bindings under a bindings/<language>  directory under each top-level. For language-specific project descriptors like setup.py, we allow those to go in the top-level, since that is typical for projects and often the most discoverable for users (i.e. the project descriptor files in most languages can be at a more rootward directory than the actual source code, and it aids in discovery/docs/etc).
Moving to this structure for the Python side first has the advantage of getting this sticky thing moved once (i.e. a number of build/CI things need to be updated) and will make the subsequent C code move more NFC. I'll update this PR accordingly.

Agree on push and iterate.

authors: powderluv stellaraccident

repo: google/iree


text:
Closes: #1 Updated validations.py script

Updated validations.py python script
Fixed the behavior of validate_user function in validations.py
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

Updated validations.py script

repo: google/it-cert-automation-practice


text:
Closes: #1Updated validations.py python script.

Fixed the behavior of validate_user function in validations.py.
Closes: #1
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[x] Tests pass
[x] Appropriate changes to README are included in PR

made the valid changes so the python script works for all testcases


text:
NPE when compiling simple enum class with lambda expressions

Describe the bug
I have a simple enum java class that define an inner Functional interface, the interface is used a constructor argument for the enum class, enum entires are defined passing a lambada expression of that functional interface.
compiling with j2cl throws a NullPointerException.
To Reproduce
we can reproduce this issue in using the hello world sample:

add the following class next to the HelloWorld.java

package com.google.j2cl.samples.helloworldlib;

public enum Unit{
    px(value -> value + "px"),
    em(value -> value + "em");

    private final UnitFormatter unitFormatter;

    Unit(UnitFormatter unitFormatter) {
        this.unitFormatter = unitFormatter;
    }

    public String of(Number value) {
        return unitFormatter.format(value);
    }

    @FunctionalInterface
    public interface UnitFormatter{
        String format(Number value);
    }
}

try to transpile the Hello world sample

this will produce an exception with the following stack trace
ures.wrapAndThrowUnchecked(Futures.java:1546)
        at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1532)
        at com.google.j2cl.transpiler.J2clTranspiler.transpile(J2clTranspiler.java:111)
        at com.google.j2cl.transpiler.BazelJ2clBuilder.run(BazelJ2clBuilder.java:74)
        at com.google.j2cl.bazel.BazelWorker.processRequest(BazelWorker.java:62)
        at com.google.j2cl.bazel.BazelWorker.execute(BazelWorker.java:116)
        at com.google.j2cl.bazel.BazelWorker.runPersistentWorker(BazelWorker.java:98)
        at com.google.j2cl.bazel.BazelWorker.start(BazelWorker.java:71)
        at com.google.j2cl.transpiler.BazelJ2clBuilder.main(BazelJ2clBuilder.java:158)
Caused by: java.lang.NullPointerException
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:1617)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:647)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:1384)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:627)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convertLambdaBody(CompilationUnitBuilder.java:1037)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:1023)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:631)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.base/java.util.Iterator.forEachRemaining(Unknown Source)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convertArguments(CompilationUnitBuilder.java:1501)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:299)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:208)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:190)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.convert(CompilationUnitBuilder.java:177)
        at com.google.j2cl.frontend.CompilationUnitBuilder$ASTConverter.access$100(CompilationUnitBuilder.java:133)
        at com.google.j2cl.frontend.CompilationUnitBuilder.buildCompilationUnit(CompilationUnitBuilder.java:1914)
        at com.google.j2cl.frontend.CompilationUnitBuilder.lambda$build$0(CompilationUnitBuilder.java:1936)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
        at com.google.j2cl.frontend.CompilationUnitBuilder.build(CompilationUnitBuilder.java:1938)
        at com.google.j2cl.transpiler.J2clTranspiler.convertUnits(J2clTranspiler.java:141)
        at com.google.j2cl.transpiler.J2clTranspiler.transpileImpl(J2clTranspiler.java:125)
        at com.google.j2cl.transpiler.J2clTranspiler.lambda$transpile$0(J2clTranspiler.java:110)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
---8<---8<--- End of log snippet, 791 chars omitted ---8<---8<---
Target //src/main/java/com/google/j2cl/samples/helloworld:helloworld failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.388s, Critical Path: 1.15s, Remote (0.00% of the time): [queue: 0.00%, setup: 0.00%, process: 0.00%]
INFO: 2 processes: 1 local, 1 worker.
FAILED: Build did NOT complete successfully

for some reason the CurrentType is being Null in the CompilationUnitBuilder during the process of converting the lambda expression body.

expanding the lamda to anonymous classes works.

Thanks for the clear repro.

Fix confirmed, it is working now. :+1:

authors: rluble vegegoku


text:
Why and how to use check_grads?

I'm curious what is the purpose of check_grads  and how should it be used - it seems to return no values and there are no comments associated with the code.

check_grads is an internal test utility for verifying that gradients are calculated correctly by JAX's automatic differentiation rules. It compares to finite differences in a randomly chosen direction, so the runtime is still O(1) (independent of the dimensionality of the input/output space).

Hi Stephen,
Thank you for the info - will check_grads throw an exception if its comparison between algorithmic diff and finite differences fails?
Dominic

From: Stephan Hoyer notifications@github.com
Sent: Thursday, April 9, 2020 4:01 AM
To: google/jax jax@noreply.github.com
Cc: Barraclough, Dominic (ext. 414) djb@qvii.com; Author author@noreply.github.com
Subject: [EXTERNAL] Re: [google/jax] Why and how to use check_grads? (#2648)
CAUTION:
This email originated from outside of QVI. Do not click links or open attachments unless you recognize the sender and are expecting to receive this content from them. If you suspect the email is not legitimate, please forward it as an attachment to SPAM@qvii.com and delete it from your Inbox.
check_grads is an internal test utility for verifying that gradients are calculated correctly by JAX's automatic differentiation rules. It compares to finite differences in a randomly chosen direction, so the runtime is still O(1) (independent of the dimensionality of the input/output space).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/google/jax/issues/2648#issuecomment-611390133, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAIRK6ZKT33YOW53BA7AOG3RLV6F5ANCNFSM4MEFQ7SQ.

Yes, see #2656 for a docstring.

Thank you!

repo: google/jax


text:
Error caused by shutil.rmtree

Traceback (most recent call last):
  File "\\?\C:\Users\cloud\AppData\Local\Temp\Bazel.runfiles_vfpgffuf\runfiles\__main__\build\install_xla_in_source_tree.py", line 83, in <module>
    shutil.rmtree(jaxlib_dir)
  File "C:\Users\cloud\miniconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\cloud\miniconda3\lib\shutil.py", line 400, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\cloud\miniconda3\lib\shutil.py", line 398, in _rmtree_unsafe
    os.unlink(fullname)
WindowsError: [Error 5] Access is denied.: 'D:\\jax\\build\\jaxlib\\cublas_kernels.pyd'

This only happens on rebuild.
The reason is shutil.rmtree will not delete readonly file on Windows.

@cloudhan can you try with https://github.com/google/jax/pull/4982 patched in?
This reworks the build a bit more so it builds a wheel file rather than writing into the source tree. Since it uses a temporary directory it doesn't need this particular line of code.

I just updated that PR to fix some more windows-specific problems and verified I was able to build a windows wheel.

#4982 has now been submitted.

Python issue tracker discussion

authors: cloudhan hawkinsp


text:
Converting Concrete Tracer values to regular float values doesnt work

Hello,
I am trying to convert concrete tracer values to regular float values using astype(float) but it still seems to return a concrete tracer value. However when I do astype(int) it seems to correctly convert the concrete tracer into an int.
Here is an example:
import jax
from jax import numpy as jnp

def func(mu):

  print((mu.astype(float)))
  print(mu)
  print(mu.astype(int))

return mu

f = jax.grad(func)

print(f(jnp.array(5.0)))

This returns the following:
Traced<ConcreteArray(5.0)>with<JVPTrace(level=2/0)>
  with primal = Traced<ConcreteArray(5.0):JaxprTrace(level=1/0)>
       tangent = Traced<ShapedArray(float32[]):JaxprTrace(level=1/0)>
Traced<ConcreteArray(5.0)>with<JVPTrace(level=2/0)>
  with primal = Traced<ConcreteArray(5.0):JaxprTrace(level=1/0)>
       tangent = Traced<ShapedArray(float32[]):JaxprTrace(level=1/0)>
5
1.0

Clearly print(mu) and print(mu.astype(float)) seem to return the exact same thing while print(mu.astype(int)) returns the correct int value. Is this expected? This is a simple example but in my original function I would like to convert one of the input parameters into a regular float value.
Thanks in advance!

Thanks for the question – if anything, I think the behavior of mu.astype(int) returning an array rather than a tracer looks to me like a bug.
Can you say more about why you're interested in extracting values from concrete tracers? Questions like this often indicate that someone is trying to work around JAX's computational model in ways that can lead to unintended results.

Thanks for the quick response!
So I want to use JAX to do auto-differentiation of the negative log-likelihood function which takes as input an array with the first entry of the array being the parameter mu and the second entry being the parameter alpha. Take for instance the following simple example:
def NLL(mu1):

    mu = mu1[0]                                                                                # the first parameter mu

    alpha=mu1[1]                                                                                # the second parameter alpha

    alpha_lumi_kFact = t.obj("alpha_ATLAS_LUMI")                     # initializing a ROOT object. Essential for the NLL computation

    alpha_lumi_kFact.setVal(alpha)                                                 # passing alpha value to the ROOT object

    result = mu+alpha_lumi_kFact.getVal()                                    # simplified result for illustration purpose

    return result

Without going into too much technical detail, the setVal method of alpha_lumi_kFact object in the above function takes as input only regular float values. It may not seem apparent based on the simplified example, but it is essential for my original function to set the value of alpha_lumi_kFact to be equal to mu1[1] and then do differentiation. As it is, the following gradient computation:
f = jax.grad(NLL)
print(f(jnp.array([1.0,0.5])))

gives an error:
---------------------------------------------------------------------------
UnfilteredStackTrace                      Traceback (most recent call last)
<ipython-input-92-7f31c8f92bd3> in <module>
      1 f = jax.grad(NLL)
----> 2 print(f(jnp.array([1.0,0.0])))

    [... skipping hidden 1 frame]

~/.local/lib/python3.6/site-packages/jax/_src/api.py in grad_f(*args, **kwargs)
    828   def grad_f(*args, **kwargs):
--> 829     _, g = value_and_grad_f(*args, **kwargs)
    830     return g

    [... skipping hidden 1 frame]

~/.local/lib/python3.6/site-packages/jax/_src/api.py in value_and_grad_f(*args, **kwargs)
    900     if not has_aux:
--> 901       ans, vjp_py = _vjp(f_partial, *dyn_args, reduce_axes=reduce_axes)
    902     else:

~/.local/lib/python3.6/site-packages/jax/_src/api.py in _vjp(fun, has_aux, reduce_axes, *primals)
   1996     out_primal, out_vjp = ad.vjp(
-> 1997         flat_fun, primals_flat, reduce_axes=reduce_axes)
   1998     out_tree = out_tree()

~/.local/lib/python3.6/site-packages/jax/interpreters/ad.py in vjp(traceable, primals, has_aux, reduce_axes)
    114   if not has_aux:
--> 115     out_primals, pvals, jaxpr, consts = linearize(traceable, *primals)
    116   else:

~/.local/lib/python3.6/site-packages/jax/interpreters/ad.py in linearize(traceable, *primals, **kwargs)
    101   jvpfun_flat, out_tree = flatten_fun(jvpfun, in_tree)
--> 102   jaxpr, out_pvals, consts = pe.trace_to_jaxpr(jvpfun_flat, in_pvals)
    103   out_primals_pvals, out_tangents_pvals = tree_unflatten(out_tree(), out_pvals)

~/.local/lib/python3.6/site-packages/jax/interpreters/partial_eval.py in trace_to_jaxpr(fun, pvals, instantiate)
    504     fun = trace_to_subjaxpr(fun, main, instantiate)
--> 505     jaxpr, (out_pvals, consts, env) = fun.call_wrapped(pvals)
    506     assert not env

~/.local/lib/python3.6/site-packages/jax/_src/errors.py in __init__(self, tracer, context)
    148     super().__init__(
--> 149         "Abstract tracer value encountered where concrete value is expected: "
    150         f"{tracer}\n{context}{tracer._origin_msg()}\n")

UnfilteredStackTrace: AttributeError: 'str' object has no attribute '_origin_msg'

The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.

--------------------

The above exception was the direct cause of the following exception:

AttributeError                            Traceback (most recent call last)
<ipython-input-92-7f31c8f92bd3> in <module>
      1 f = jax.grad(NLL)
----> 2 print(f(jnp.array([1.0,0.0])))

~/.local/lib/python3.6/site-packages/jax/_src/errors.py in __init__(self, tracer, context)
    147   def __init__(self, tracer: "core.Tracer", context: str = ""):
    148     super().__init__(
--> 149         "Abstract tracer value encountered where concrete value is expected: "
    150         f"{tracer}\n{context}{tracer._origin_msg()}\n")
    151

AttributeError: 'str' object has no attribute '_origin_msg'

The error is a bit cryptic, but I assume it is trying to say "Abstract tracer value encountered where concrete value is expected".
Note that this error goes away if I pass an explicit float value to the alpha_lumi_kFact object:
def NLL(mu1):

    mu = mu1[0]                                                                    # the first parameter mu

    alpha=mu1[1]                                                                  # the second parameter alpha

    alpha_lumi_kFact = t.obj("alpha_ATLAS_LUMI")

    alpha_lumi_kFact.setVal(0.0)                                    # setting the parameter to regular float 0.0

    result = mu+alpha_lumi_kFact.getVal()

    return result

which then gives the correct result for the following code:
f = jax.grad(NLL)
print(f(jnp.array([1.0,0.0])))

[1. 1.]

In summary, the setVal method takes as input only regular float values and does not seem to work with tracer values. Any suggestions?

It sounds like you're trying to use a JAX transform on a non-JAX function: this will not work. The way JAX traces the effect of inputs on outputs for the purposes of autodiff is via these Concrete tracer values: if you extract the underlying floating point values from these tracers and compute results with them, grad has no way of computing the correct result.
I'd suggest either finding (or creating) a JAX implementation of the algorithm you want to use, or otherwise finding a different method of computing gradients for your code.

Okay, this is clear to me now. Thank you for the insight!

authors: JaySandesara jakevdp


text:
Allow negative padding in pad gradient, fixes #511

Paired with @j-towns, fixes #511

Thanks for the PR!
It looks like the Travis build failed — looks like a problem with mismatched types?
Otherwise this looks good.

authors: JuliusKunze hawkinsp


text:
Add context.Context parameter to the model interfaces

Having the context.Context parameter in the JSONAPILinks,
JSONAPIMeta, JSONAPIRelationshipLinks and JSONAPIRelationshipMeta
allows for accessing contextual information such as the request URI,
which can be used to generate absolute URLs for example.
Signed-off-by: Xavier Coulon xcoulon@redhat.com

That being said, I understand that this PR breaks the current API, since all consumers of the library would need to update their code by adding this context.Context argument in the JSONAPIxxx() methods of their model structs in order to keep the marshalling/unmarshalling working.

We will not merge this as it breaks the current api.  I have https://github.com/google/jsonapi/blob/master/runtime.go to allow for state.  I will legitimize this--refactor, add tests, and document--so people can use state in their processing.

thanks for your feedback, @shwoodard !

authors: shwoodard xcoulon

repo: google/jsonapi

license: mit


text:
add --spot flag to kctf-cluster to use spot VMs

big savings during dev time with spot VMs, so it would be nice to have this flag

Thanks Arxenix! The change looks good to me. Please check out the message from the google-cla bot and then I can merge it.

I'm not really sure why it's failing, I signed the CLA

Thanks, I made release 1.7.0 with this change.

authors: arxenix sroettger

repo: google/kctf


text:
Adapting sequence numbers to timestamps

Original issue 248 created by mvonmaszewski on 2014-08-13T21:38:21.000Z:
I am considering a custom modification where sequence numbers are actually timestamps.  I would use this to automatically prune older records during compactions.  New sequence numbers would be generated like this:
cur_time=env->NowMicros();
if (cur_time<=version->LastSequence)
++version->LastSequence;
else
version->LastSequence=cur_time;
return(version->LastSequence);
No, this is not the exact code … just the concept.  The "then" branch handles case where multiple writes happen within the resolution of the clock.  And I might create a NowNanos() just to reduce that too.
I believe snapshots and such would work as is.  Then I only have to add logic to the block iterators to skip old stuff.
Thoughts??

Compaction already ignore deleted records - that's one of the primary functions performed by a compaction. Timing is very problematic, as you mention, due to clock resolution, time changes, etc.

repo: google/leveldb


text:
Model v1

Hi,
Is there any possibility to add support for running v1 version of the model?
Maybe in a Cargo.toml feature.
Thanks!

Hello,
what would the use case for this? Ensure backward compatibility? Or you noticed specific issues with v2? Additional context would help!

As far as I seen, v2 is slower than v1 as it is a larger model, so we prefer using v1 for working with data in scale :)

thanks for the feedback, this is very useful! /cc @ia0 @invernizzi
We do have a smaller version of our v2 models (https://github.com/google/magika/tree/main/assets/models/fast_v2_1), it can be easily used by the python module (by pointing model_dir to it), but it's not integrated yet with the rust codebase (and it's currently not so trivial to do so).
We'll discuss internally on how to approach this. In the meantime, please let us know if you have additional context to share. For example: it seems you are integrating the magika rust cli within an existing pipeline... in which language is this pipeline written? If, for example, the pipeline is written in python, the python module would be the wait to go: most of rust's performance improvements are about avoiding the initial one-off starting time, and after things are loaded in python, the inference time rust vs python should be roughly the same.

repo: google/magika


text:
Get landmarks and iris points in the Iris tracking project in javascript

Hello ,
I want to attach a new Listener to get landmarks and iris points (in https://viz.mediapipe.dev/demo/iris_tracking). I've added this line in the "default_demo_bin.js" in "module$contents$google3$third_party$mediapipe$web$default_demo_VideoController.prototype.populateDemoSelector" function, but it doesn't work.
this.api.attachListener("irisanddepthrenderergpu__right_eye_contour_landmarks", { onNumber: function (c) { console.log(c); } });
this.api.attachListener("irisanddepthrenderergpu__iris_landmarks", { onString: function (s) { console.log(s); } });
could you please help me how can I do this?

This is a closed source project, and we don't support any sort of editing.  But, as a fellow hacker:
populateDemoSelector fills out a drop down with different mp4 names... I don't even think this code is reachable in the public demo.  The function you are probably looking for is module$contents$google3$third_party$mediapipe$web$default_demo_SingleFrameController.prototype.attachListeners
I haven't checked to see if irisanddepthrenderergpu__right_eye_contour_landmarks or irisanddepthrenderergpu__iris_landmarks are legal stream names.  But if they are actually present in the graph, this should probably work.

repo: google/mediapipe


text:
Feasibility of Objectron support on iOS

Hello, at present, my Objectron Android project has basically completed the required functions, but we need to use this function on the iOS side, so is there any way to provide the use scheme of Objectron on the iOS side, or to port the completed Android Studio project to the iOS side? Because I found that the most important part of Android project is the use of AAR package, but the iOS terminal does not have such function, so the iOS terminal cannot use this function. I would like to ask if you have any way to solve this problem, thank you.

Is there any planned roadmap for IOS support ?

Hello, at present, my Objectron Android project has basically completed the required functions, but we need to use this function on the iOS side, so is there any way to provide the use scheme of Objectron on the iOS side, or to port the completed Android Studio project to the iOS side? Because I found that the most important part of Android project is the use of AAR package, but the iOS terminal does not have such function, so the iOS terminal cannot use this function. I would like to ask if you have any way to solve this problem, thank you.

I'm working on same. but I don't have any idea regarding this. if you have any ref regarding this, pls share.

Hi @z7212819 ,
This is issue can be resolved in the latest version of MediaPipe , could you please check and close this issue.


text:
Layer 2: MetalLB shouldn't listen on interfaces which have master

Is this a bug report or a feature request?:
BUG
What happened:
My nodes have a lot of interfaces, eg. 4 hardware interfaces, 2 bonding, bridge, vlans, and veth interfaces.
When I use arping, I have 8 replies from the metallb speaker. From each interface which passing the traffic.
Eg if you have the next scheme:
   |       |
[eth0]   [eth1]
    \     /
    [bond0]
       |
     [br0]

I will get answer from all of them. That's totally wrong because connectivity will not work if ip address will be associated with interface which have master (eth0, eth1 or bond0)
What you expected to happen:
I expected answer only from br0 interface.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

MetalLB version:
Kubernetes version:
BGP router type/version:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):

Probably we can also exclude interfaces with NOARP flag set

Thanks for the report! This is definitely a bug, in exactly the way you describe.
Unfortunately, getting information about which interfaces are sub-interfaces of others is hard from Go right now, so it's not a 100% trivial fix. There's also a related bug #316 which would allow selecting specific interfaces to listen on, although I still want the default to be automagic.
Thinking...

Hi @danderson, thanks for quick answer.
I've checked possibilities right now, I think we can use the check for the next directory:
/sys/class/net/<interface>/master

If catalog exists - then ignore interface

authors: danderson kvaps

repo: google/metallb


text:
Move nodeHasHealthyEndpoints to internal/k8s

Make it public and call it from {bgp,arp}-speaker.
Fixes #29

Needs rebase. Github got confused by the concurrent PRs, and my attempt to fix it in the online editor have just made things worse. Can you rebase+push to your branch from your client?

authors: danderson miekg


text:
Placement of objects in AR mode

Hi,
This is more of a query than a bug and I understand it relates to both quick look and scene-viewer rather than model-viewer itself but am posting here in the hope that someone can enlighten me.
I'm trying to understand how models are placed in space when in AR mode. It seems to me from testing that once the plane is detected, the model is placed directly on the plane at a point as though a straight line is drawn directly from the camera view to the plane.
Assuming that's correct, that makes sense. What is a little bit more baffling is that IOS and android seem to handle this slightly differently. IOS seems to place the model so that the origin point of the model matches the anchor point of the viewer. Android on the other hand, seems to match the front edge of the model's bounding box to the anchor point. Is that what others experience?
To clarify what I think is going on to myself (and to help get some advice) I've made a diagram.

The difference is not ideal but as long as its consistent that's fine. My final query is one that I can't find any logic to. Sometimes when models contain animation, they are elevated above the ground level even though they exist in the original file, at same height as a static version. This seems to happen more in IOS than android but as I say there is no consistency to it. You can see an example of what I mean by that here https://ninth-worried-mice.glitch.me/ you can see the cube is elevated into the air even though it is on the groun when in model-viewer (and in the original file).
If anyone could clarify any of the above for me it would be much appreciated.
Great work on the project
Testing on
Safari, iPhone XR, IOS 13.3
Chrome, firefox, Pixel 3, Android 9
Versions
Latest version of model viewer

Hi @elbobo thanks for the thoughtful investigation and the inquiry.
This project is not the authoritative source for Quicklook or Scene Viewer information. That said, I can try to offer some commentary about your investigation and its findings:
Scene Viewer and Quicklook are different native "apps" made by different teams at different companies. To the extent that they share anything in common at all, it is a coincidence due to the fact that their designers arrived at the same common sense conclusion about what makes a good user experience when placing objects in AR via a handheld device.
There is a ton of room for subjectivities here. One app may decide that the best way to place an object is based on the object's bounding box. Another app may decide to place objects based on their relative position in the model's root scene. Ultimately there is no canonical "spec" for how AR placement should work (perhaps we need one? 😅), so every app should be assumed to have come to its own conclusions about what makes for good UX.
As far as animations are concerned, I would drop this into the same bucket as above. There are probably articulable reasons why any given app does something peculiar when animations are playing, but those reasons will be specific to that app. I'll CC @tpsiaki who may be able to comment on any peculiarities related to Scene Viewer.
If you happen to have specific questions about <model-viewer>, I'm happy to answer those as well. We are about to embark on UX refinement for browser-based AR via WebXR, so this topic is top-of-mind for use right now. CC @elalish who will be spending time in this area soon.

Hi @elbobo - you are correct regarding how the positioning of the models works in Scene Viewer.  Most of the initial use cases for Scene Viewer involved placing an object in front of the user, starting with the animals in Google Search (some of which are quite large).
This behavior was what we decided on to avoid the experience where the user starts with the model placed in a way that will intersect with their phone.
I understand that for more immersive experiences this behavior might be undesirable, so if there are particular use cases where a different behavior would be helpful we'd love to hear about it.
We're happy to consider exposing alternative options if needed.

This is also a bit different in WebXR mode. Here the model can be turned before it is placed (if you move your phone a bit), so instead of anchoring on the middle of the forward edge of the bounding box, it uses the point where the view vector to the bounding box center crosses over the floor projection of the bounding box, but it amounts to the same thing if the object is facing the camera. In addition, the hit test ray I use to place the object is not the one from the center of the screen, but one pointed 20 degrees downward, which helps both with hitting a detected plane quicker and with placing the object so that it is better centered onscreen.

Sometimes when models contain animation, they are elevated above the ground level even though they exist in the original file, at same height as a static version. This seems to happen more in iOS than android but as I say there is no consistency to it. You can see an example of what I mean by that here https://ninth-worried-mice.glitch.me/ you can see the cube is elevated into the air even though it is on the groun when in model-viewer (and in the original file).

I have experienced the same issue. Are you using a .glb file? I have experinced that this issue  might be related to 2 things: the viewer used and the file format used. So if you use .glb on scene viewer you might experience this "elevation" with animations. I tried using .gltf and the "elevation" disapeared. Let me know if this works for you.

authors: akemora cdata elalish elbobo tpsiaki

repo: google/model-viewer


text:
How to disable blending with scene viewer?

By default, object blending is enabled in scene-viewer. Is there a way we can have it disabled as default?

@tpsiaki Is there an intent parameter for this?

Thanks @elalish for investigating, I am also interested in a possible solution.

Blending is a cool feature but so often yields undesirable results that I agree it should be off by default.

Fully agree on that. In most cases "off" setting is better.

I would also like to know how to disable this. Is there any parameter that we can use?

@elalish Yes there's an intent parameter for this.  You can pass disable_occlusion=true on the intent and that will turn off blending.

authors: Saicopate antekj creativehims ctrlventure elalish ronpadz tpsiaki


text:
Make it possible to set custom Scene Viewer link and title

Description
I found https://github.com/GoogleWebComponents/model-viewer/pull/855#pullrequestreview-307394678 which is great - it basically allows to pass parameters such as sound= and link= directly into SceneViewer since they get preserved when just appending them to the original source.
However, that pull request mentions that it's not exactly a good idea to append "link" (I guess the parameter will be double then). Is there a "clean" and documented way of adjusting what parameters get passed into SceneViewer?
(link= and title= work right now, I'm just afraid that might change at any point if not documented)
Live Demo
https://glitch.com/edit/#!/pfc-modelviewer-audio-tests

Done.

Hello, is there any reference in the  documentation how to use this feature?

Yes: https://developers.google.com/ar/develop/java/scene-viewer

Hi, thanks for your response I know this one. But, how can one configure „link“ and „title“ with a GLB imported in ModelViewer?

Just add those query parameters to your src URL.

Thx. Should the parameters be added url-encoded or plain?

I believe you can do them plain. Feel free to file an issue if anything about their use is not very ergonomic. We haven't had a lot of feedback on it.

I got it now, I always used webxr as ar-mode, therefore none of my scene-viewer arguments were recognized. But thanks for your input :)

Ah yes indeed. With WebXR you can do it all with child HTML elements.

Ah nice. If I would set it to default WebXR I wouldn't have to mess around with scene-viewer.
If I set it to default are there many devices that can't handle webXR or do all scen-viewer devices also support webXR? ar-modes="webxr" (only)

I believe any phone that supports scene-viewer also supports WebXR (assuming you're in a Chrome browser), since it's all based on ARCore availability. WebXR actually does better, since many people do not have scene viewer installed.

authors: elalish hybridherbst sprengerst


text:
fcitx-mozc: please review and merge fcitx module to trunk

There were users requesting mozc support of Fcitx some time ago, and Fcitx's
maintainer has created a patch for mozc to add the support (since mozc support
can only be built together with mozc itself). Please review and merge it to
trunk.

http://code.google.com/p/fcitx/downloads/detail?name=fcitx-mozc-1.5.1053.102-1.p
atch

Original issue reported on code.google.com by happyaro...@gmail.com on 30 Apr 2012 at 6:16

https://download.fcitx-im.org/fcitx-mozc/fcitx-mozc-2.23.2815.102.1.patch
Please review and migrate it. Thanks.

repo: google/mozc


text:
add timestamps to out and web

This is useful for tracking performance.

/gcbrun exp -n dk-test-infra-99x2 -m vertex_ai_gemini-1-5 -b minor-for-ci -i

These are the visuals added:

https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2024-07-16-485-dk-test-infra-99x2-minor-for-ci/index.html

/gcbrun exp -n dk-test-infra-99x23 -m vertex_ai_gemini-1-5 -b minor-for-ci -i

Experiment is looking good

repo: google/oss-fuzz-gen


text:
chore: capture coverage and upload it to codecov

This is a real basic starter for 10 - I considered doing something more fancy with run_tests.sh but realised it would just be better to discuss the general approach to scripts for the scanner as its own thing (i.e. we could use make, we could move things into scripts, we could just have multiple scripts for things...) so for now have just made coverage collection always enabled and documented how to generate the HTML report.
I have also included uploading the results to codecov but ultimately that's for Google folks to decide what they want to do - if folks are happy to use codecov I think this can be merged as is and it'll just automatically create the project etc and you should be able to access that via logging in with GitHub.
Also this is the coverage currently:
❯ ./run_tests.sh
?       github.com/google/osv-scanner/cmd/osv-reporter  [no test files]
ok      github.com/google/osv-scanner/cmd/osv-scanner   9.997s  coverage: 73.9% of statements
?       github.com/google/osv-scanner/internal/cachedregexp     [no test files]
ok      github.com/google/osv-scanner/internal/ci       0.006s  coverage: 81.5% of statements
ok      github.com/google/osv-scanner/internal/local    0.014s  coverage: 46.2% of statements
ok      github.com/google/osv-scanner/internal/output   0.003s  coverage: 9.0% of statements
ok      github.com/google/osv-scanner/internal/sbom     0.003s  coverage: 35.0% of statements
ok      github.com/google/osv-scanner/internal/semantic 0.172s  coverage: 96.7% of statements
ok      github.com/google/osv-scanner/internal/sourceanalysis   1.240s  coverage: 46.1% of statements
?       github.com/google/osv-scanner/internal/sourceanalysis/govulncheck       [no test files]
?       github.com/google/osv-scanner/internal/testutility      [no test files]
?       github.com/google/osv-scanner/internal/thirdparty/ar    [no test files]
ok      github.com/google/osv-scanner/internal/utility/vulns    0.004s  coverage: 88.6% of statements
ok      github.com/google/osv-scanner/pkg/config        0.004s  coverage: 35.4% of statements
ok      github.com/google/osv-scanner/pkg/grouper       0.002s  coverage: 83.3% of statements
ok      github.com/google/osv-scanner/pkg/lockfile      0.017s  coverage: 94.0% of statements
ok      github.com/google/osv-scanner/pkg/models        0.003s  coverage: 71.6% of statements
?       github.com/google/osv-scanner/pkg/osv   [no test files]
ok      github.com/google/osv-scanner/pkg/osvscanner    0.016s  coverage: 11.9% of statements
ok      github.com/google/osv-scanner/pkg/reporter      0.002s  coverage: 26.8% of statements

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@99b9bfc). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #512   +/-   ##
=======================================
  Coverage        ?   68.18%
=======================================
  Files           ?       70
  Lines           ?     4667
  Branches        ?        0
=======================================
  Hits            ?     3182
  Misses          ?     1403
  Partials        ?       82

You can see the report for this branch here: https://app.codecov.io/gh/google/osv-scanner/tree/capture-coverage-in-tests

@another-rex if you're happy to land this now, I've got a couple of extra tests for coverage that I can PR and we'll be set for a good dive in our upcoming catch up

authors: G-Rath codecov-commenter

repo: google/osv-scanner


text:
Swift 4.1 error: Ambiguous use of operator '=='

Hi,
I got an error when working with Swift 4.1 and Xcode 9.3 beta 3 (9Q117m), when installing the framework using CocoaPods v1.4.0:

Not critical now, stable Xcode 9.2 and Swift 4 work correctly.
Removing this piece of code makes it compile:
    static func == (lhs: When<Value?>, rhs: When<Value?>) -> Bool {
      switch (lhs, rhs) {
      case (.value(let lhs), .value(let rhs)):
        switch (lhs, rhs) {
        case (nil, nil):
          return true
        case (nil, _?), (_?, nil):
          return false
        case let (lhs?, rhs?):
          return lhs == rhs
        }
      case (.error(let lhs), .error(let rhs)):
        return (lhs as NSError).isEqual(rhs as NSError)
      case (.value, .error), (.error, .value):
        return false
      }
    }

Introduction of the conditional conformance in Swift 4.1 might be the root cause.

Thanks, Richard!
The issue should be resolved in 1.0.6.

Thanks for the quick release!

authors: richardtop shoumikhin

repo: google/promises


text:
Add entry_points

Adding entry_points to setup.py to get console scripts
diff --git a/setup.py b/setup.py
index 7fb1172..ec02932 100644
--- a/setup.py
+++ b/setup.py
@@ -57,5 +57,12 @@ Android project's ADB.
         'Programming Language :: Python :: 2',
         'Programming Language :: Python :: 3',
         'Topic :: Software Development :: Testing'
-        ]
+        ],
+    entry_points={
+        "console_scripts": [
+            "adb = adb.adb_debug:main",
+            "fastboot = adb.fastboot_debug:main",
+        ],
+    }
+
 )

Can you send this as a pull request?

Done #84

authors: delijati fahhem

repo: google/python-adb


text:
How to Use it Stand Alone?

I want to use it with PHP not composer how to install it in my PHP script?

With the addition of #34 you should now be able to follow the instructions here: https://github.com/google/recaptcha#manual-installation

okay, thanks

authors: rajlaksh rowan-m

repo: google/recaptcha


text:
Hoq to Decodr text

您的邮件已收到，，，我会尽快回复的，，，


text:
Video doesn't render if ShakaPlayerView is initialised with .zero frame

I've been struggling with this for ages until I found the culprit. I wasn't getting any video rendered and I was literally pulling my hair out. I'm creating this issue in case anybody else comes upon it.
After doing lots of tests I finally found out that if you create the ShakaPlayerView this way:
let playerView = ShakaPlayerView(frame: .zero)

instead of this way
let playerView = ShakaPlayerView()

no video gets rendered in the view.
This view was integrated inside a playback view, and the code was properly setting auto layout constraints:
[playerView, ...].forEach {
    $0.translatesAutoresizingMaskIntoConstraints = false
}

NSLayoutConstraint.activate([
    playerView.leadingAnchor.constraint(equalTo: leadingAnchor),
    playerView.topAnchor.constraint(equalTo: topAnchor),
    playerView.trailingAnchor.constraint(equalTo: trailingAnchor),
    playerView.bottomAnchor.constraint(equalTo: bottomAnchor)
])

I even tried to debug the app with the Xcode's view hierarchy debugger, thinking that I wouldn't have properly setup the views, and the view was being properly laid out, as you can see in the attached screenshot:

If the view is created without any parameter, later changes in layout seem to work properly (for example, rotating the device causes the video to rotate and adapt to the screen), so I guess somehow auto layout is handled properly except for this case, where the view is explicitly assigned a .zero frame in the beginning.

Oh, no, I found the reason: ShakaPlayerView defines init() and init(player:), but it doesn't override the initialisers inherited from UIView 😭

repo: google/shaka-player-embedded


text:
DRM Dash Live Stream doesn't work

Have you read the FAQ and checked for duplicate issues: yes
What version of Shaka Player are you using: 2.4.0
Can you reproduce the issue with our latest release version: yes
Can you reproduce the issue with the latest code from master: yes
Are you using the demo app or your own custom app: demo app and custom app
If custom app, can you reproduce the issue using our demo app: yes
What browser and OS are you using: Win 10 Chrome
What are the manifest and license server URIs:
(you can send the URIs to shaka-player-issues@google.com instead, but please use GitHub and the template for the rest)
sent to shaka-player-issues@google.com
What did you do?
Load live stream and press play
What did you expect to happen?
Stream started
What actually happened?
Infinite loading indicator.
In console I see:
Updating manifest...
dash_parser.js:1251 actual update period 5.864
streaming_engine.js:1387 (audio:11) looking up segment: presentationTime=4195502.9263 currentPeriod.startTime=0
streaming_engine.js:1387 (video:2) looking up segment: presentationTime=4195502.9263 currentPeriod.startTime=0

Hi @desenhista, I'm getting a 403 error for your manifest. Are you restricting access by region or something like that?

Hi @ismena
has restriction by region. sorry for that.
I will remove the restriction and as soon as I do, I'll let you know
thank you so much

Hi @ismena
Can you give me an ip or range of ip to put on the white list of akamai?
Thanks

Here are our IP blocks:
IPv6 block: 2620:0:1008::/48
IPv4 block: 104.132.51/24

@michellezhuogg @joeyparrish
Could one of you take this over is this/next week's bug cop?

@desenhista, I tried to reproduce the issue today, but I'm getting HTTP 403 errors.

Hi @joeyparrish
Please, can you try again?
thank you very much

I can see the issue now, but I've run out of time to work on it today.  I'll look again tomorrow and dig into why it is failing.

Here's my analysis of the content:
Current unix timestamp when I fetched the manifest (now): 1528995655
Stream start according to the manifest's availabilityStartTime attribute (ast): 1527269598
Time in seconds since stream start (live_edge = now - ast): 1726057
Most recent video segment start time (latest_start): 501643046354
Most recent video segment duration (latest_duration): 900900
Most recent video segment end time (latest_start + latest_duration): 501643947254
Video timescale (timescale): 90000
Most recent video segment end time in seconds (latest_end_seconds = (latest_start + latest_duration) / timescale): 5573821.636
Offset between live edge and segment (latest_end_seconds - live_edge): 3847764.636
The most recent segment ends 3847764.636 seconds (44.5 days) in the future.  Since the content timeline is inconsistent, this is bad content that we don't currently support.
In the future, after #999 is closed, we will be able to play content like this.  In the mean time, I suggest you file a bug with the encoder vendor to fix this.

Thank you very much @joeyparrish

authors: desenhista ismena joeyparrish

repo: google/shaka-player


text:
Event for RECOVERABLE networking errors and subsequent retries

We would like to know when a networking error occurs which results in a retry. Our application tracks this information for performance metrics purposes.
To implement such a change, it seems NetworkingEngine would need to accept a PlayerInterface, but I don't know how that would work with alternative use cases as in #1297. Perhaps the PlayerInterface could be optional, and NetworkingEngine could do:
// NetworkingEngine#send_
if (error && error.severity == shaka.util.Error.Severity.RECOVERABLE) {
  if (this.playerInterface) {
    this.playerInterface.onNetworkingRetry_(error); // new method
  }
  // Move to the next URI.
  index = (index + 1) % request.uris.length;
  let shakaError = /** @type {shaka.util.Error} */(error);
  return this.send_(type, request, backoff, index, shakaError);
}

Is this feasible?

@theodab I don't mind sending in a PR if we agree on a design.

@chrisfillmore No, I should probably do it. @vaage is currently working on #1297, so it'd probably be best to do this in-house, to avoid collisions.

I had a discussion with the team, and we decided that it would probably be best to avoid anything that changes the constructor of networking engine, since networking engine is also being modified as part of #1051. Instead, I will add a method on the networking engine that adds a listener function. So you'd use it like:
player.getNetworkingEngine().addNetworkingRetryListener((error) => {
  // Your listener code here.
});

Sounds good, thank you!

Okay, after another round of code review, this should be out soon.
There was some further discussion, and we decided that it should work as an event listener, using our FakeEventListener code. So you'd use it like:
player.getNetworkingEngine().addEventListener('retry', (event) => {
  let error = event.error;
  // Your listener code here.
});

Sorry we changed our minds so often on this!

Thanks, @theodab!

authors: chrisfillmore theodab


text:
Refactor isSupportedSyzkall

Related to https://github.com/google/syzkaller/pull/2001#issuecomment-668390600, @dvyukov What do you think about this?

CC: @balsini

Related to #2001 (comment), @dvyukov What do you think about this?

Looks good. Thanks for the refactoring this.

authors: StefanoDuo dvyukov

repo: google/syzkaller


text:
pkg/build: update fuchsia tools target name

Fuchsia's //bundles:tools target was renamed to
//bundles/tools.

Before sending a pull request, please review Contribution Guidelines:
https://github.com/google/syzkaller/blob/master/docs/contributing.md

This together with https://fuchsia-review.googlesource.com/c/fuchsia/+/750369 should fix the build.


text:
all: move spanner writes to dashboard/app

dashboard/app knows about subsystems much more

I don't know how to test it locally.
The proposal is to deploy this dashapi extension, test it manually and if Ok - switch to its usage.
@dvyukov

Sounds good.

authors: dvyukov tarasmadan


text:
weather_dl: 'Fetch data' counters reset after error

I'm running a mars download job with 3 licenses, 3 workers, 6 CPUs, running 2 downloads per license. The 'Fetch data' counters kept increasing as expected as downloads progressed. However, at some point all 6 downloads timed out at almost the same time with
 File "/usr/local/lib/python3.8/site-packages/download_pipeline/fetcher.py", line 93, in process
    self.fetch_data(partition, worker_name=worker_name)
  File "/usr/local/lib/python3.8/site-packages/download_pipeline/fetcher.py", line 76, in fetch_data
    client.retrieve(dataset, selection, temp.name)
  File "/usr/local/lib/python3.8/site-packages/download_pipeline/clients.py", line 172, in retrieve
    self.c.execute(req=selection, target=output)
  File "/usr/local/lib/python3.8/site-packages/ecmwfapi/api.py", line 642, in execute
    c.execute(req, target)
  File "/usr/local/lib/python3.8/site-packages/ecmwfapi/api.py", line 549, in execute
    size = self._transfer(
  File "/usr/local/lib/python3.8/site-packages/ecmwfapi/api.py", line 183, in wrapped
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/ecmwfapi/api.py", line 504, in _transfer
    chunk = http.read(1048576)  # 1MB chunks
  File "/usr/local/lib/python3.8/http/client.py", line 459, in read
    n = self.readinto(b)
  File "/usr/local/lib/python3.8/http/client.py", line 503, in readinto
    n = self.fp.readinto(b)
  File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out [while running 'Fetch data']

It looks like the downloads restarted after that, but the 'Fetch data' counters are now back to 2 for each license, when before they were at ~12 (last I checked). The 'Prepare partitions' counters are still the same they were before.

I had another look at the logs, and it's actually not just the counters resetting. After the errors, the workers start again with
"[key1.1] Starting requests..."
"[key3.0] Starting requests..."

Since they don't run the partitioning again, already finished downloads are not skipped, it restarts the exact download list it ran the first time around.

This requires a 2-part fix:

[ ] Add skipping to the fetch_data step, to skip downloads before they are attempted again (short-term fix)
[ ] Add retry logic to each part that fails (e.g. the request vs the download from EC vs the upload). This depends on #139

authors: alxmrs uhager

repo: google/weather-tools


text:
Font Picker: Search and Popular fonts

Feature description
This is a follow up to ampproject/amp-wp#3759

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance criteria

User is able to search for fonts
When the user clicks anywhere inside the text picker, display a list of popular and suggested fonts (see design for list)
Recently used fonts are displayed above the popular font list
After the user enters chars, search results are displayed based on the input
If there are no matches for a search, indicate that no matches are found
After the user clicks on font picker, the focused is placed on search field
The dropdown should default to the user’s system font

Implementation brief

QA testing instructions

Demo

Changelog entry

Do we have any requirements for popular fonts and suggested fonts?

^ @jauyong @pbakaus @o-fernandez

^ @samitron7

I'd go with what's in the Figma spec for now.

I don‘t think Comic Sans is popular :-)
The design on Figma looks like the old font list from the legacy editor. We need to cone up with a better definition of popular fonts.

@samitron7 @pbakaus anything you had in mind when it comes to popular fonts? If not, I'd say we skip this for now. Seems like a plausible enhancement to do later if not readily spec'd.

@pbakaus @o-fernandez @samitron7 Do we have any updates here for popular fonts, suggested fonts?

Personally I'd say, when opening the fonts dropdown, we should just show the N first fonts from the list. Popular fonts can be looked at later.

Agree, per my prior comment I suggest separating the "Popular fonts" into its own ticket. I'll update this ticket to reflect that.

+1. I'd do popular fonts after launch.

If this issue addresses search only then it looks like it already has a PR and is in review by https://github.com/google/web-stories-wp/pull/1300 which in addition to preview also implements search. cc @ndev1991 to confirm.

If this issue addresses search only then it looks like it already has a PR and is in review by #1300 which in addition to preview also implements search. cc @ndev1991 to confirm.

Yes, that basic features already implemented on #1300 .

Input search box focus on open
List default fonts lists
Search has two separate lists that starts with and includes
Recent used fonts feature added (Just temp state for used fonts. Refresh will clean that state)

I just realized that we could get the most popular fonts from Google Fonts. That would not include system fonts like Arial though, but I don't think that'd be a huge issue.

I just realized that we could get the most popular fonts from Google Fonts. That would not include system fonts like Arial though, but I don't think that'd be a huge issue.

That sounds great! Can you post some more details?

@ndev1991 well, the Google Fonts API supports sorting fonts by popularity, so basically we could create something like the existing fonts.json file, but just with the top X popular fonts.

Something we can pursue after launch. Unless we want to determine popularity ourselves.

I think this is great as first step (popularity from Google fonts).

@ndev1991 well, the Google Fonts API supports sorting fonts by popularity, so basically we could create something like the existing fonts.json file, but just with the top X popular fonts.

Something we can pursue after launch. Unless we want to determine popularity ourselves.

@swissspidy Do we have any plan to add that fields to current fonts.json file? Maybe fonts.json will be generated by some cron job from php?

Do we have any plan to add that fields to current fonts.json file?

No plans yet. We'll get there once we need to implement popularity. Could be either be a field in fonts.json or a new popular-fonts.json file - we will see.

Maybe fonts.json will be generated by some cron job from php?

The fonts.json file is already generated by a cron job in the repository. This cannot be done in PHP within the plugin.

Agree with @swissspidy - to clarify, no popularity to be implemented as part of this ticket. We can explore this in the future. :)

authors: jauyong miina ndev1991 o-fernandez pbakaus swissspidy

repo: google/web-stories-wp


text:
Update Moveable to latest 0.25 release

Feature Description
There have been a few new releases of react-moveable. As per #5952, it would be wise to address the failing tests the latest update causes, and otherwise adjust the code where needed.

It looks like the update breaks the rotation indeed, will need some adjustments probably. On the changelog -- checking quickly it looks like there are some things that would clean up the code a little bit but nothing urgent. Perhaps we can create an enhancement issue for it. It might be better anyway to keep it up to date in case we encounter an update that we require in the future.

Seems like it might be the last release before a planned 1.0.0 version of Moveable.
Changelog: https://github.com/daybrush/moveable/blob/master/CHANGELOG.md?rgh-link-date=2021-01-25T09%3A18%3A25Z
Alternatives Considered

Additional Context

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria

Implementation Brief

Ping @barklund FYI and for prioritizing.

Ping @barklund FYI and for prioritizing.

Verified in QA

authors: csossi miina swissspidy


text:
Returning to the initial scene with the root GAv4 object creates duplicate instances

The usage of DontDestroyOnLoad() in the script GoogleAnalyticsV4.cs seems incorrect and leads to duplicate GAv4 objects when the scene is loaded again.
To Reproduce:

Add the GAv4 object to the root of hierarchy of the initial scene.
Run the initial scene and then request the same scene to be loaded again.
Notice there are now two instances of GAv4 in the root of the hierarchy.

Each time you reload the scene that contained the GAv4 object, a duplicate copy will be created. To fix this, the awake method needs to deal with the call to DontDestoryOnLoad():
void Awake()
{
    if (instance == null)
    {
        instance = this;
        DontDestroyOnLoad(gameObject);
    }
    else if (instance != this)
    {
        // someone else is the master instance, so kill itself
        Destroy(gameObject);
        return;
    }

    InitializeTracker ();
...

In the code above the duplicate objects are automatically deleted before they call InitializeTracker ().
I don't see this v4 source code in the repo otherwise I would do a pull request. I used the V4 unity package to get the source code into Unity.

I'm having exactly the same issue. My game consists of only one scene. The scene is reloaded after each "level" completion. After each level I'm getting another instance of GAv4 in the hierarchy.

Me too (Registered to follow issue...)

Yepp, having the same issue.

authors: 11Sheep christianstrang jamesvecore tmarciszewski


text:
chore: Add HttpJson Showcase test for Location Mixins

Thank you for opening a Pull Request! For general contributing guidelines, please refer to contributing guide
Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[x] Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
[x] Ensure the tests and linter pass
[x] Code coverage does not decrease (if any source code was changed)
[x] Appropriate docs were updated (if necessary)

Part of #1439 ☕
Update to showcase v0.27.0 is to incorporate this fix: https://github.com/googleapis/gapic-showcase/issues/1299

LGTM. Looking for approval from @blakeli0

repo: googleapis/gapic-generator-java


text:
Release Google.Api.Gax version 3.7.0

No changes since 3.7.0-beta01
Changes since 3.6.0:

Addition of RoutingHeaderExtractor and corresponding ApiCall methods to use it

Closing for now as we need to investigate streaming first.

Thanks - will release on Monday.

Created release for Google.Api.Gax-3.7.0

authors: jskeet yoshi-automation

repo: googleapis/gax-dotnet


text:
How to validate idToken?

So I know this library could be sued but adding 18MB just to validate idToken is is bad idea for me, so how can i pull the selected files from this library to achieve the same

you can install just the google/auth to validate ID tokens:
composer install google/auth

$auth = new Google\Auth\AccessToken();
$auth->verify($idToken);

Or you could configure composer to skip installation of the google/apiclient-services repository entirely (which is the repository taking up all the 18MB):
{
    "require": {
        "google/apiclient": "^2.12"
    },
    "replace": {
        "google/apiclient-service": "*"
    }
}

authors: ashiqdey bshaffer

repo: googleapis/google-api-php-client


text:
The sharedWithMe request parameter is being processed incorrectly

I tried to get files from Google Drive that owned by user, not shared with him.
Environment details

OS type and version: Ubuntu 20.04.1 LTS
Python version: Python 3.8.5
pip version:  19.0.3
google-api-python-client version:  2.0.2

Code example
import google.oauth2.credentials
from googleapiclient.discovery import build

cred = ...  # credentials google.oauth2.credentials.Credentials
service = build('drive', 'v3', credentials=cred)

# This works well
service.files().list(
    pageSize=999,
    fields='nextPageToken, files(id, name, mimeType, parents, modifiedTime, size, ownedByMe)',
    q='sharedWithMe'
).execute()

# But I need to get files Not shared with the user and use operations =, != from here https://developers.google.com/drive/api/v3/ref-search-terms I got an error
service.files().list(
    pageSize=999,
    fields='nextPageToken, files(id, name, mimeType, parents, modifiedTime, size, ownedByMe)',
    q='sharedWithMe=false'
).execute()

Stack trace
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/serhio/Projects/google_drive/venv/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/serhio/Projects/google_drive/venv/lib/python3.8/site-packages/googleapiclient/http.py", line 915, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/drive/v3/files?pageSize=999&fields=nextPageToken%2C+files%28id%2C+name%2C+mimeType%2C+parents%2C+modifiedTime%2C+size%2C+ownedByMe%29&q=sharedWithMe+%21%3D+true&alt=json returned "Invalid Value". Details: "Invalid Value">

# Response
{'vary': 'Origin, X-Origin', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 18 Mar 2021 07:10:46 GMT', 'expires': 'Thu, 18 Mar 2021 07:10:46 GMT', 'cache-control': 'private, max-age=0', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'content-security-policy': "frame-ancestors 'self'", 'x-xss-protection': '1; mode=block', 'server': 'GSE', 'alt-svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '226', '-content-encoding': 'gzip'}

I've tried to change q expression to:
q='sharedWithMe!=true'
q='sharedWithMe=false'
q='not sharedWithMe'
q='!sharedWithMe'

But it didn't help.

Hi @mef1stofel ,
This appears to be a bug with a drive API given that this feature is mentioned in the documentation. Please comment on the existing issue here to get a status update. I've also updated the internal ticket to increase visibility of this issue. Googlers see b/173109050. I'm going to close off the issue since it is an issue with the API rather than the python client library in this Github repo.

repo: googleapis/google-api-python-client


text:
Cleanup Examples for Bigtable

[x] Refactor command-line parsing and auto-running to a common library.
[x] Refactor bigtable_grpc_credentials.cc
[x] Refactor data_snippets.cc
[x] Refactor data_async_snippets.cc
[x] Refactor bigtable_hello_app_profile.cc
[x] Refactor bigtable_hello_instance_admin.cc
[x] Refactor bigtable_hello_table_admin.cc
[x] Refactor bigtable_hello_world.cc
[x] Refactor bigtable_instance_admin_snippets.cc
[x] Refactor bigtable_quickstart.cc
[x] Refactor instance_admin_async_snippets.cc
[x] Refactor table_admin_async_snippets.cc
[x] Refactor table_admin_snippets.cc

Broke this down into smaller issues, closing.

authors: coryan devjgm

repo: googleapis/google-cloud-cpp


text:
Update Google.Apis.Storage.v1 dependency

In particular, this supports the "CustomTime" property on Storage
objects, as well as the corresponding object-lifecycle management
conditions.
Fixes #5115.

Self-merging to unblock creating a release.

repo: googleapis/google-cloud-dotnet


text:
Release Google.Cloud.CertificateManager.V1 version 1.0.0-beta01

Changes in this release:
Initial release.

Created release for Google.Cloud.CertificateManager.V1-1.0.0-beta01


text:
docs: add "support_documentation" to libraries.json

RE: https://github.com/googleapis/synthtool/pull/582
This will add a best-effort link to each product's "Getting Support" page. If it can't be found, it will re-use the "product_documentation" link.

I like this idea, let's try it.

authors: bcoe stephenplusplus

repo: googleapis/google-cloud-node


text:
Synthesis failed for datacatalog

Hello! Autosynth couldn't regenerate datacatalog. :broken_heart:
Here's the output from running synth.py:
Cloning into 'working_repo'...
Switched to a new branch 'autosynth-datacatalog'
Cloning into '/tmp/tmpp6dqig9h/googleapis'...
Note: switching to '35c6057ac2f4db2a69c932e3b7301a0600e3e150'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 35c6057a chore: sample context aware commits for a broader base of services (#2932)
Note: switching to '3f5f8a2258c6a41f9fbf7b80acbca631dda0a952'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 3f5f8a22 feat: added support for project id in scope feat: added support for order_by in SearchAllResources rpc feat: added support for search on location, labels and networkTags fields docs: updated according to above features
Switched to a new branch 'autosynth-datacatalog-12'
2020-04-28 17:52:44 [INFO] Running synthtool
2020-04-28 17:52:44 [INFO] ['/usr/local/google/home/rennie/env3.6/bin/python', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
/usr/local/google/home/rennie/env3.6/bin/python: No module named synthtool
2020-04-28 17:52:44 [ERROR] Synthesis failed
HEAD is now at 35c6057a chore: sample context aware commits for a broader base of services (#2932)
Switched to branch 'autosynth-datacatalog'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synth.py", line 578, in <module>
    main()
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synth.py", line 459, in main
    return _inner_main(temp_dir)
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synth.py", line 568, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synth.py", line 352, in synthesize_loop
    synthesize_inner_loop(toolbox, synthesizer)
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synth.py", line 362, in synthesize_inner_loop
    synthesizer, len(toolbox.versions) - 1
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synth.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(self.environ)
  File "/usr/local/google/home/rennie/gitrepos/synthtool/autosynth/synthesizer.py", line 115, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/usr/local/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/usr/local/google/home/rennie/env3.6/bin/python', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Autosynth passed, closing! :green_heart:

repo: googleapis/google-cloud-php


text:
[Text-to-Speech] Synthesis fails with grpc extension if result is larger than 4 MB: "Received message larger than max (... vs. 4194304)

Environment details

OS: Ubuntu 18.04
PHP version: 7.3.17
Package name and version:

google/auth                           v1.9.0   Google Auth Library for PHP
google/cloud-core                     v1.37.0  Google Cloud PHP shared dependency, providing functionality useful to all components.
google/cloud-speech                   v1.1.1   Cloud Speech Client for PHP
google/cloud-storage                  v1.21.0  Cloud Storage Client for PHP
google/cloud-text-to-speech           v1.0.2   Cloud Text-to-Speech Client for PHP
google/cloud-translate                v1.7.3   Cloud Translation Client for PHP
google/common-protos                  1.2      Google API Common Protos for PHP
google/crc32                          v0.1.0   Various CRC32 implementations
google/gax                            1.3.0    Google API Core for PHP
google/grpc-gcp                       0.1.5    gRPC GCP library for channel management
google/protobuf                       v3.12.2  proto library for PHP
grpc/grpc                             1.27.0   gRPC library for PHP

Steps to reproduce

Have grpc extension installed and enabled.
Use Google\Cloud\TextToSpeech\V1\TextToSpeechClient to synthesize.
Synthesis fails if the result is larger than 4 MB:

Google\ApiCore\ApiException: {
    "message": "Received message larger than max (7010673 vs. 4194304)",
    "code": 8,
    "status": "RESOURCE_EXHAUSTED",
    "details": [
        {
            "@type": "grpc-status",
            "data": "8"
        },
        {
            "@type": "grpc-message",
            "data": "Received message larger than max (7010673 vs. 4194304)"
        }
    ]
}

Code example
<?php
use Google\Cloud\TextToSpeech\V1\AudioConfig;
use Google\Cloud\TextToSpeech\V1\AudioEncoding;
use Google\Cloud\TextToSpeech\V1\SynthesisInput;
use Google\Cloud\TextToSpeech\V1\TextToSpeechClient;
use Google\Cloud\TextToSpeech\V1\VoiceSelectionParams;

require_once('vendor\autoload.php');

$text = str_repeat('Test. ', 150);

$input = new SynthesisInput();
$input->setText($text);

$voice = new VoiceSelectionParams();
$voice->setLanguageCode('de-DE');

$audioConfig = new AudioConfig();
$audioConfig->setAudioEncoding(AudioEncoding::LINEAR16);
$audioConfig->setSpeakingRate(0.5);

$client = new TextToSpeechClient();
$res = $client->synthesizeSpeech($input, $voice, $audioConfig);

echo strlen($res->getAudioContent());

Workaround
A workaround is to set grpc.max_receive_message_length and grpc.max_send_message_length (as suggested in #1690 for the BigtableClient):
-$client = new TextToSpeechClient();
+$client = new TextToSpeechClient([
                 'transportConfig' => [
                     'grpc' => [
                         'stubOpts' => [
                             'grpc.max_receive_message_length' => -1,
                             'grpc.max_send_message_length' => -1
                         ]
                     ]
                 ]
             ]);

Solution
Allow to to specify grpc.max_receive_message_length => -1 and grpc.max_send_message_length => -1 globally, or make them defaults in google/gax's GapicClientTrait.

After thinking on this, I'd like to trust the settings gRPC has defaulted to. On a case by case basis where we have a wrapper client (like Bigtable) I am open to defaulting to sane standards that help solve the majority of use cases, but I'm more apprehensive to set it across all clients. Happy to have more discussion on this if anyone disagrees or would like to see differently, please feel free to re-open or reference this issue.

@dwsupplee Wouldn't the TextToSpeechClient also qualify for setting a better default? I guess the gRPC defaults just don't account for transmitting binary data, as is the case with the TTS api.

Yes, but unlike something like our storage or bigtable clients,  the text to speech client is a generated codebase and I'd prefer we avoid manual edits when possible. I'll bring this up with the team this week and see what other language teams feel comfortable doing.

authors: caugner dwsupplee


text:
chore: secure hermetic_library_generation workflow

Thanks to @diogoteles08 for the inspection on our repos. This PR inlines environment variables to avoid overriding script injections.

Could you create a forked repo and test whether the workflow will be skipped?

Could you create a forked repo and test whether the workflow will be skipped?

@JoeWang1127 we tried this in https://github.com/googleapis/java-bigtable/pull/2317#discussion_r1731458621. I think we still confirmed it works the other way around: non-forked repos actually get triggered.

repo: googleapis/java-bigtable