text:
[GSK-2015,GSK-2012,GSK-1998] LLM tests refactoring

Clean up duplicate LLM tests
Add clear documentation to each test
Standardize the LLM test names
Improve naming of the generated datasets

Depends on #1513

you're missing the check of the debug bool, It should be False by default, and checked inside the test. If it's true, the ouput_ds should be filled

you're missing the check of the debug bool, It should be False by default, and checked inside the test. If it's true, the output_ds should be filled

Since I have the dataset, I just return it regardless of whether debug is set or not. I don’t see the point in not setting it since I already have it. Also since the LLM-based tests are not deterministic, it’s best to have the debug dataset from the specific run (without having to run again with debug=True, which will give different results).

I'm not sure this is well defined. The standard procedure today is that once you click debug in the hub, you re-run the test with debug=True, you save the output_ds.
Not sure if the dataset is persisted only when the debug is clicked, or whether by always providing the dataset, the dataset gets saved twice.
But in any case, even with no check on debug, the debug click will produce a new dataset either way.

In fact the mechanism of debug is also going to change with this https://github.com/Giskard-AI/giskard-hub/pull/172/files, where only the list of indices are now transfer to the backend, and the dataset gets retrieved and only filtered, not created...

In fact the mechanism of debug is also going to change with this https://github.com/Giskard-AI/giskard-hub/pull/172/files, where only the list of indices are now transfer to the backend, and the dataset gets retrieved and only filtered, not created...

Nice! That’s a significant improvement, which also avoids polluting the project artifacts with tens of temporary datasets.

Not sure if the dataset is persisted only when the debug is clicked, or whether by always providing the dataset, the dataset gets saved twice.

I will double check. It doesn’t seems it’s saved in the tests I’ve done, but I may have missed that.

@rabah-khalek I fixed a problem with the single value test and added a couple of unit tests. The debug output_df is not a problem (does not create duplicates).

authors: mattbit rabah-khalek

repo: Giskard-AI/giskard


text:
Feature 192 move filter icons to side panel

Added right side panel and moved all the filters with their visible labels and switches.

Looks good!
But I'm not sure if we should blur/disable the map while the filters panel is open?
At least I would like to be able to click on various points that appear/disappear on the map while selecting the filter? This makes me have to go back and forth between the map and opening/closing the filter panel, to check the new data that appeared on the map.

This has now been resolved. For mobile users the panel will hide when selecting search results, but on desktop the map and info panels are accessible when the panels are open.

source: gharchive/pull-request

repo: GispoCoding/tarmo

license: MIT

license_type: permissive

license_source: github-api


text:
pre-commit warning

[WARNING] repo https://github.com/gitguardian/ggshield uses deprecated stage names (commit, push) which will be removed in a future version.

as far as I can tell, this is just a doc bug. The full warning is
[WARNING] hook id `ggshield` uses deprecated stage names (commit) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.

If I run the suggested command pre-commit migrate-config, it makes the following changes:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -14,10 +14,10 @@ repos:
     hooks:
     -   id: ggshield
         language_version: python3
-        stages: [commit]
+        stages: [pre-commit]
 -   repo: https://github.com/gitguardian/ggshield
     rev: v1.32.0
     hooks:
     -   id: ggshield-push
         language_version: python3
-        stages: [push]
+        stages: [pre-push]

So, the stages just need to be modified in these 2 doc pages:

https://docs.gitguardian.com/ggshield-docs/integrations/git-hooks/pre-commit
https://docs.gitguardian.com/ggshield-docs/integrations/git-hooks/pre-push

Hi, thanks for the suggestion, this change should appear in the public doc soon.

The changes have been deployed.

I believe this should be reopened.
I just received the same warning on a repository using ggshield's pre-commit hook.
The full warning message is:
[WARNING] repo `https://github.com/gitguardian/ggshield` uses deprecated stage names (commit, push) which will be removed in a future version.  Hint: often `pre-commit autoupdate --repo https://github.com/gitguardian/ggshield` will fix this.  if it does not -- consider reporting an issue to that repo.

Running the suggested command doesn't fix the issue.
The stages should be modified in the hooks file so that a very minimal local config doesn't report a warning:
  - repo: https://github.com/gitguardian/ggshield
    rev: v1.34.0
    hooks:
      - id: ggshield

source: gharchive/issue

authors: Crocmagnon agateau-gg dotysan tapple-cisco

repo: GitGuardian/ggshield


text:
Switch to pyright, complete type-hinting

Description
This PR improves type-safety by doing two things:
Switch to pyright
switches to pyright, which catches more errors. As a result I had to introduce 2 mixins to make Marshmallow code more type-safe: FromDictMixin provides a from_dict() method, ToDictMixin provides the matching to_dict() method.
Base inherits from ToDictMixin, but not from FromDictMixin because unfortunately some of our classes do not deserialize to objects and changing that would break API compatibility. This is the case for Document and HealthCheckResponse.

Codecov Report

Merging #61 (fa3e51e) into master (af54d23) will increase coverage by 0.12%.
The diff coverage is 95.74%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##           master      #61      +/-   ##
==========================================
+ Coverage   94.60%   94.72%   +0.12%
==========================================
  Files           5        5
  Lines         463      474      +11
==========================================
+ Hits          438      449      +11
  Misses         25       25

Flag
Coverage Δ

unittests
94.72% <95.74%> (+0.12%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pygitguardian/client.py
86.06% <83.33%> (ø)

pygitguardian/iac_models.py
100.00% <100.00%> (ø)

pygitguardian/models.py
99.24% <100.00%> (+0.02%)
:arrow_up:

LGTM.

authors: Walz agateau-gg codecov-commenter

repo: GitGuardian/py-gitguardian


text:
Actually add the manifest file to the list of file assets produced

So when you compress your extension with https://www.npmjs.com/package/zip-webpack-plugin the manifest is part of the zip file.

Coverage increased (+0.9%) to 92.593% when pulling ae66b2376240756a917b7710e55730ad8a6b2be6 on revmischa:master into 9fdf59adbc80d2d78f8b345632cc8db0d18b452e on GitScrum:master.

Coverage increased (+0.9%) to 92.593% when pulling ae66b2376240756a917b7710e55730ad8a6b2be6 on revmischa:master into 9fdf59adbc80d2d78f8b345632cc8db0d18b452e on GitScrum:master.

@revmischa thank you very much ✌

Fo sho. Release soon?

@revmischa done.

authors: GitScrum coveralls revmischa


text:
No css file in /css

Can the CSS file be implemented? Would be amazing!

How can I work on it.

authors: AnkitSingh-zm perryraskin

repo: GitbookIO/markdown-css

license_source: bigquery


text:
🛑 gadgethouse is down

In 05edf42, gadgethouse (gadgethouse.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: gadgethouse is back up in b63c19a after 22 minutes.


text:
🛑 gadgethouse is down

In 0bd0810, gadgethouse (gadgethouse.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: gadgethouse is back up in 51661b9 after 18 minutes.


text:
🛑 youporn is down

In 328f125, youporn (youporn.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: youporn is back up in 4e1c843.


text:
🛑 komets is down

In 281e03b, komets (komets.com) was down:

HTTP code: 503
Response time: 3481 ms

Resolved: komets is back up in 97c5bb0.


text:
🛑 orkut is down

In c486e8b, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 0ed9434.


text:
🛑 orkut is down

In 5248a07, orkut (orkut.com.br) was down:

HTTP code: 403
Response time: 317 ms

Resolved: orkut is back up in aee70e4.


text:
🛑 ojolink is down

In ae5a870, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in c7a6828.


text:
🛑 ojolink is down

In bc7ca26, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 724fcbd.


text:
🛑 orkut is down

In 364f512, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 3513e49.


text:
🛑 penispedia is down

In 4352c8e, penispedia (penispedia.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penispedia is back up in 461ef0d.


text:
🛑 orkut is down

In 9f3b8c1, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in b31117e.


text:
🛑 ojolink is down

In 6e143e6, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 403 ms

Resolved: Ojolink is back up in 1e7bb3e.


text:
🛑 eatbigbuns is down

In 0fa07cc, eatbigbuns (eatbigbuns.com) was down:

HTTP code: 502
Response time: 6537 ms

Resolved: eatbigbuns is back up in bb806d7.


text:
🛑 ojolink is down

In 40c64ac, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 428 ms

Resolved: Ojolink is back up in 329b11a.


text:
🛑 penispedia is down

In e52165c, penispedia (penispedia.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penispedia is back up in fa8bf5b.


text:
🛑 feest-start is down

In a6b4025, feest-start (feest-start.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: feest-start is back up in fac33ac.


text:
🛑 orkut is down

In cf07f6c, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 762 ms

Resolved: orkut is back up in 8b2b9b3.


text:
🛑 orkut is down

In 17dac5e, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 872 ms

Resolved: orkut is back up in 53792f3.


text:
🛑 ojolink is down

In 5ee237f, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 404 ms

Resolved: Ojolink is back up in a000733.


text:
🛑 inkcityusa is down

In 3f1429e, inkcityusa (inkcityusa.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: inkcityusa is back up in 7c90099.


text:
🛑 orkut is down

In 379ac06, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 977 ms

Resolved: orkut is back up in 801ca16.


text:
🛑 ojolink is down

In 4c15e57, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 434 ms

Resolved: Ojolink is back up in e44b3b0.


text:
🛑 ojolink is down

In 83a3101, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 381 ms

Resolved: Ojolink is back up in 9eb7580.


text:
🛑 orkut is down

In e9e773e, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 817 ms

Resolved: orkut is back up in 690f6e8.


text:
🛑 ojolink is down

In d821c14, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 498 ms

Resolved: Ojolink is back up in c06c87c.


text:
🛑 ojolink is down

In 5cf79d9, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 357 ms

Resolved: Ojolink is back up in 04b6fb1.


text:
🛑 urbanlime is down

In 061c533, urbanlime (urbanlime.com) was down:

HTTP code: 500
Response time: 433 ms

Resolved: urbanlime is back up in f9a5880.


text:
🛑 orkut is down

In a625ace, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 1230 ms

Resolved: orkut is back up in 03bbf78.


text:
🛑 benjohnson is down

In dadb066, benjohnson (benjohnson.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: benjohnson is back up in 0e6e9bc.


text:
🛑 ojolink is down

In e41fc18, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 827e597.


text:
🛑 ojolink is down

In f93c9ed, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 8cdccdc.


text:
🛑 baidu is down

In 1737ba7, baidu (baidu.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: baidu is back up in 4dc9ab0.


text:
🛑 ojolink is down

In eb7c419, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 16e9932.


text:
🛑 ojolink is down

In 37f2e66, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 8bdfcb1.


text:
🛑 carlinha is down

In 7f324b1, carlinha (carlinha.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: carlinha is back up in e82a2be.


text:
🛑 ojolink is down

In 529497d, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in b9be6ae after 836 days, 56 minutes.


text:
🛑 ojolink is down

In 01b7d69, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 5fd7368 after 8 minutes.


text:
🛑 ojolink is down

In d1fe785, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 9041ddf after 30 minutes.


text:
🛑 youjizz is down

In 2d57d06, youjizz (youjizz.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: youjizz is back up in 26ada10 after 39 minutes.


text:
🛑 feest-start is down

In 3e239b4, feest-start (feest-start.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: feest-start is back up in ab4dca0 after 17 minutes.


text:
🛑 youjizz is down

In 1fa0534, youjizz (youjizz.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: youjizz is back up in 2d64098 after 46 minutes.


text:
🛑 carlinha is down

In 1d4e2ab, carlinha (carlinha.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: carlinha is back up in 1b51afa after 1 hour, 21 minutes.


text:
🛑 twoc is down

In 3c1f785, twoc (twoc.co.uk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: twoc is back up in 5e07b05 after 24 minutes.


text:
🛑 inkcityusa is down

In fc20934, inkcityusa (inkcityusa.com) was down:

HTTP code: 403
Response time: 301 ms

Resolved: inkcityusa is back up in 1b2313d after 1 day, 21 hours, 43 minutes.


text:
🛑 ojolink is down

In 2aef381, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in eb738a3 after 20 minutes.


text:
🛑 ojolink is down

In a7a8133, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 59e8e76 after 8 minutes.


text:
🛑 ojolink is down

In cff265b, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 16c0810 after 10 minutes.


text:
🛑 penisdicke is down

In 6f69697, penisdicke (penisdicke.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penisdicke is back up in f39bb0a.


text:
🛑 ojolink is down

In afc5a49, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 011fcf8 after 9 minutes.


text:
🛑 ojolink is down

In 21376a8, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in b827578 after 12 minutes.


text:
🛑 ojolink is down

In d8542ff, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in b7fdd46 after 16 minutes.


text:
🛑 metroflog is down

In 127facb, metroflog (metroflog.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: metroflog is back up in 797041e after 41 minutes.


text:
🛑 urbanlime is down

In 1f8ac4f, urbanlime (urbanlime.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: urbanlime is back up in b9bfc3f after 1 hour, 1 minute.


text:
🛑 ojolink is down

In 6d51f82, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 2a5732d after 18 minutes.


text:
🛑 ojolink is down

In 046014b, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 4cb78ce after 8 minutes.


text:
🛑 ojolink is down

In 58f4888, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 0916117 after 16 minutes.


text:
🛑 ojolink is down

In 34511f1, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 97caac6 after 9 minutes.


text:
🛑 orkut is down

In b309fe4, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 60d867f after 11 minutes.


text:
🛑 orkut is down

In 42012f0, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 1032768 after 58 minutes.


text:
🛑 orkut is down

In b95e758, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 8423f0a after 2 hours, 23 minutes.


text:
🛑 twoc is down

In e2b12ae, twoc (twoc.co.uk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: twoc is back up in 61e9e75 after 17 minutes.


text:
Add JSON Object to Project Created Activity

When we call the project-created activity we currently get the following information:

projectTitle
email
projectLink
firstName
lastName

and we update our person's information in Ortto with:

email
firstName
lastName

Question
Is it possible to append JSON objects to a profile in Ortto? - this might allow us to keep track of all the projects owned by a single user in the platform - but it would require us to send that info every time a user creates a project.
I don't know how heavy of an operation that is but it is a nice to have.
Say the object might include just 3 fields for every project:

project title
project slug
verified?
I don't know how heavy of an operation that is but it is a nice to have.

Say the object might include just 3 fields for every project:

project title
project slug
verified?

For now..
I'm having a meeting with some of the comms team tomorrow to understand better their product needs - will check back to see how necessary or useful this will be.

We're going to close this one since the value it provides is unclear - we're waiting on a feature release from Ortto that would allow people to be affiliated with multiple organizations - this would solve our problem

repo: Giveth/impact-graph


text:
Chore: Combine logic for scalar_function and table_function.

Description
https://github.com/GlareDB/glaredb/pull/2288#discussion_r1435207088 :

previously the only reason we needed different for loops for table_funcs() and scalar_funcs() was for the handling of runtime_preference. Since that is removed, we should follow up with cleaning up that logic.

Just to clarify, is this just for how we build the builtin catalog, and not something more involved with refactoring the trait(s)?

Just to clarify, is this just for how we build the builtin catalog, and not something more involved with refactoring the trait(s)?

I think yeah! @universalmind303 can comment better on it though

authors: scsmithr vrongmeal

repo: GlareDB/glaredb


text:
Update sbt-sonatype to 3.11.0

About this PR
📦 Updates org.xerial.sbt:sbt-sonatype from 3.9.5 to 3.11.0
📜 GitHub Release Notes - Release Notes
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.xerial.sbt", artifactId = "sbt-sonatype" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.xerial.sbt", artifactId = "sbt-sonatype" }
}]

labels: sbt-plugin-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #261.


text:
Set up Travis CI

Set up Travis CI to run cargo fmt and cargo clippy for each commit.

Let me try this in my forked repository first.


text:
Dock interface

i forgot to switch branches to dev 👀

repo: GodotModding/godot-mod-tools


text:
[BUG] Can't play in 1.27.0 anymore.

MultiplayerCore Version and Download Source
1.1.2
Your Platform
PC, Beat Saber 1.27.0 (downgraded on purpose)
Describe the bug
When trying to play in multiplayer, a message that says to upgrade to 1.2.0 displays.
If I upgrade to 1.2.0, a message to upgrade beat saber to 1.28.0 appears.
Additional context
Used to work just fine about a few weeks ago. This bug happens to both me and a friend.

The minimum supported game version depends on the server you have selected. The minimum supported version for BeatTogether is 1.28.0.

So is there no way to play in 1.27 anymore ? I don't really mind upgrading I just generally tend to stick to older versions so I don't have any problems with mods not being up to date

Not on BeatTogether, no.

All good then, I was under the impression that the mod used peer to peer so I though it was just broken but good to know that there are dedicated servers.

Beat Saber used to use P2P, but switched to dedicated in 1.16.3. Most of the servers set up by the community can be found at https://bssb.app/master-servers

authors: Tarroook rcelyte

repo: Goobwabber/MultiplayerCore


text:
Speech Recognition js

Hello Dears,
Is Speech Recognition Supported on current Chrome ios app?
Thanks,
Ameer

Robert M Garcia

repo: GoogleChrome/chrome-app-samples


text:
chrome.scripting.executeScript: func with args throws error

I tried to execute
await chrome.scripting.executeScript({
  target: {tabId: tab.id},
  func: (text) => {
    (document.activeElement! as HTMLInputElement).value += (text || '');
  },
  args: [text],
});

and TypeScript tells me
TS2769: No overload matches this call.
   Overload 1 of 2, '(injection: ScriptInjection): Promise<InjectionResult[]>', gave the following error.
     Type '(text: any) => void' is not assignable to type '() => void'.
       Target signature provides too few arguments. Expected 1 or more, but got 0.
   Overload 2 of 2, '(injection: ScriptInjection, callback?: ((results: InjectionResult[]) => void) | undefined): void', gave the following error.
     Type '(text: any) => void' is not assignable to type '() => void'.

as well as
TS7006: Parameter 'text' implicitly has an 'any' type.

the type definition looks a bit too simple compared to @types/chrome
    export interface ScriptInjection {
      func?: () => void;
      args?: any[];
      ...
    }

    export type ScriptInjection<Args extends any[], Result> = {
        target: InjectionTarget;
        world?: ExecutionWorld;
        injectImmediately?: boolean;
    } & ({
        files: string[];
    } | ({
        func: () => Result;
    } | {
        func: (...args: Args) => Result;
        args: Args;
    }))

Following might also be a separate issue, but maybe add to the docs that one has to add /// <reference types="chrome-types" /> to a .d.tssince at least for me, it didn't take it automatically.

I meet the same problem.

repo: GoogleChrome/chrome-types


text:
Cursor pointer for reset icon

A detail I forgot to add last time in #24

🎉

authors: oldergod surma

repo: GoogleChrome/css-triggers


text:
chrome launch error on centos7

Steps to reproduce
Tell us about your environment:

Puppeteer version: v1.5.0
Platform / OS version: centOS7.2 x64
URLs (if applicable):
Node.js version:  V10.4.0

What steps will reproduce the problem?

run the example.js, get those dependency warnings
install dependencies
run again,  no more dependency warning, but got some trace stack. Refer to the search engine results, It seems something wrong  with the chromium installed itself. The following is the guess verification

cd node_modules/puppeteer/.local-storage/linux-xxxx/chrome-linux
[root@sz1m40487 chrome-linux]# ./chrome
[8350:8350:0717/202030.494929:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
[root@sz1m40487 chrome-linux]# su sora
[sora@sz1m40487 chrome-linux]$ ./chrome --no-sandbox

(chrome:8372): Gtk-WARNING **: cannot open display:
[sora@sz1m40487 chrome-linux]$ [0717/202044.797879:ERROR:nacl_helper_linux.cc(310)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
^C
[sora@sz1m40487 chrome-linux]$ ./chrome --no-sandbox --headless
Received signal 11 SEGV_MAPERR 000000000000
#0 0x55b7b9ffcbac base::debug::StackTrace::StackTrace()
#1 0x55b7b9ffc711 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7fb43fb92100 <unknown>
#3 0x55b7b9ff3930 <unknown>
#4 0x55b7b9cbbdf2 service_manager::MergeManifestWithOverlay()
#5 0x55b7b8f211f1 content::(anonymous namespace)::BuiltinManifestProvider::AddServiceManifest()
#6 0x55b7b8f1e7a0 content::ServiceManagerContext::ServiceManagerContext()
#7 0x55b7b8c46730 content::BrowserMainLoop::InitializeMojo()
#8 0x55b7b8c442e1 content::BrowserMainLoop::BrowserThreadsStarted()
#9 0x55b7b8fabd87 content::StartupTaskRunner::RunAllTasksNow()
#10 0x55b7b8c43ca2 content::BrowserMainLoop::CreateStartupTasks()
#11 0x55b7b8c479e5 content::BrowserMainRunnerImpl::Initialize()
#12 0x55b7b8c47957 content::BrowserMainRunnerImpl::Initialize()
#13 0x55b7bdf7b3a9 headless::HeadlessContentMainDelegate::RunProcess()
#14 0x55b7b9cb3c9b content::ContentMainRunnerImpl::Run()
#15 0x55b7b9cbcc32 service_manager::Main()
#16 0x55b7b9cb1cc4 content::ContentMain()
#17 0x55b7bdf7a389 headless::(anonymous namespace)::RunContentMain()
#18 0x55b7bdf7a412 headless::HeadlessBrowserMain()
#19 0x55b7b9cb9a55 headless::HeadlessShellMain()
#20 0x55b7b82471ac ChromeMain
#21 0x7fb439e6cb15 __libc_start_main
#22 0x55b7b824702a _start
  r8: 0000178fb2ec6740  r9: 0000000000000002 r10: 0000000000000000 r11: 00007fb439fcb4f0
 r12: 000000000000000f r13: 000055b7b6072800 r14: 0000178fb2ebee00 r15: 0000178fb2ed7880
  di: 0000000000000000  si: 00007ffdd5b7e8f8  bp: 00007ffdd5b7e910  bx: 0000178fb2ed78c0
  dx: 0000000000000004  ax: 00007ffdd5b7e978  cx: 000055b7b6162034  sp: 00007ffdd5b7e8e8
  ip: 000055b7b9ff3930 efl: 0000000000010206 cgf: 0000000000000033 erf: 0000000000000004
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

those stack traces are familar with the prompts when run the official example in the previous Step 3
chrome launched well without  the --headless option which however may needed by the puppeteer,
so……how to fix this or just avoid it

@aslushnikov please, it's not a duplicate question. I had already study that issue before. This is different. It's not about the dependency or sandbox issues. The chrome just can not launch in --headless mode  and I had offered the trace stack above.  hope you can take a look.

The same question, wired

authors: JX-Wang YxxY

repo: GoogleChrome/puppeteer


text:
Sending 'Enter' key does not work with 'await'

Steps to reproduce
Tell us about your environment:

Puppeteer version: ^1.13.0
Platform / OS version: macOS Mojave 10.14.3 (18D109)
URLs (if applicable): proprietary :/
Node.js version: 6.9.0

Please include code that reproduces the issue.

describe("Test search_customer", function () {
  describe("autocompletes the input", function() {
    it("allowing users to quickly find the desired customer", async function() {

            this.retries(3);
            this.timeout(60000);

            const browser = await puppeteer.launch(puppeteer_opts);
            const page = await browser.newPage();
            await page.goto(`${get_test_url()}index.php`);
            await page.addScriptTag({path: require.resolve('jquery')});

            await page.type("#js_search_customers", "ACS");

            let autocompleted_value = await page.evaluate(() => {
                return $("#cust_id :selected").text();
            });

            expect(autocompleted_value).to.be.equal("ACS - Xerox Government Systems");

            await page.keyboard.press('Enter');

            let input_value = await page.evaluate(() => {
                return $("#js_search_customers").val();
            });

            expect(input_value).to.be.equal("ACS - Xerox Government Systems");

            await page.type("#js_search_customers", ":");

            ({ autocompleted_value, input_value } = await page.evaluate(() => {
                return {
                    autocompleted_value: $("#cust_id :selected").text(),
                    input_value: $("#js_search_customers").val()
                }
            }));

            expect(autocompleted_value).to.be.equal("ACS - Xerox Government Systems:FH Cloud Portal");
            expect(input_value).to.be.equal("ACS - Xerox Government Systems:");

            await page.type("#js_search_customers", "W");
            await page.keyboard.press('Enter');

            ({ autocompleted_value, input_value } = await page.evaluate(() => {
                return {
                    autocompleted_value: $("#cust_id :selected").text(),
                    input_value: $("#js_search_customers").val()
                }
            }));

            browser.close();

            expect(autocompleted_value).to.be.equal("ACS - Xerox Government Systems:Web development");
            expect(input_value).to.be.equal("ACS - Xerox Government Systems:Web development");
        });

What steps will reproduce the problem?

Run the test with default puppeteer options, it will fail
Run the test with slowMo: 200, it will pass
It also passes in headless: false - and you can see it work just fine.  This is clearly a timing issue.

What is the expected result?
The test hits enter to autocomplete the text, and it autocompletes and then checks against that value.
What happens instead?
The test finishes before the input can be made to the page, despite the 'await' statements

@kevinforrestconnors this looks like a race condition to me.

The test finishes before the input can be made to the page

It looks like page takes some time to load before it is interactive. E.g. it might attach event listeners after certain timeout, e.g. when a deferred script gets to load.
I'd try to use page.goto(url, {waitUntil: 'networkidle0'}) to make sure everything loads. If it doesn't help, you'd need to look into some markers on the page to figure if it's loaded or not.
Let me know if we can somehow help more here.

Thanks, I will try that.  My solution was to use page.waitFor(200), but
your solution seems better.
On Thu, Apr 4, 2019, 4:47 PM Andrey Lushnikov notifications@github.com
wrote:

@kevinforrestconnors https://github.com/kevinforrestconnors this looks
like a race condition to me.
The test finishes before the input can be made to the page
It looks like page takes some time to load before it is interactive. E.g.
it might attach event listeners after certain timeout, e.g. when a deferred
script gets to load.
I'd try to use page.goto(url, {waitUntil: 'networkidle0'}) to make sure
everything loads. If it doesn't help, you'd need to look into some markers
on the page to figure if it's loaded or not.
Let me know if we can somehow help more here.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleChrome/puppeteer/issues/4231#issuecomment-480075506,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEwlq_8IiYRxZrYZvZi-2PtJDoA9Y3htks5vdnLUgaJpZM4cYDKk
.

authors: aslushnikov kevinforrestconnors


text:
GitHub pages support and better error handling.

Running this sample currently fails on GitHub pages because the tweets.json file is loaded from an absolute path, when it should be relative. I have changed this.
I also noticed that this wasn't handled very well, and so although this shouldn't really be a problem, I have added a quick 200 status check to prevent the transformer parsing non JSON pages.
Finally, I removed a rogue i hiding in the code for when the request has completed.
Keep up the awesome work! Hoping to see more Supercharged videos soon.

Yes! Good stuff. Thanks @oliverdunk!

200 is no the only valid response to check for. We should also consider checking for statuses like 304 Not Modified (when served from browser cache).

@ramlmn That's a good point. What do you say to only failing if it isn't a 2xx or 304 status code? Or are there other factors which I've missed?

I also feel really stupid. Looks like I didn't add anything to the if statement... :man_facepalming:

@oliverdunk Something like /^0|([123]\d\d)|(40[14567])|410$/ from sw-toolbox. 😊

That's what Response.ok is for. But no need to productionize this really. Anything but 200 means something is wrong in this case :)

GitHub pages still working, as at the end, bytesCounted > bytesTotal - no idea why, but am investigating :man_shrugging:

authors: oliverdunk ramlmn surma

repo: GoogleChrome/ui-element-samples


text:
Feature Request: Add removeEventListener/unobserve equivalent

Currently every invocation creates a new PerformanceObserver that never unobserves so there's no way to remove a handler. It'd be nice to be able to remove old listeners.
EDIT: This affects the web vitals extension which leaks listeners on every activation of tab. I'm fixing over there, but this seems generally useful too 😃

I didn't add an unobserve/disconnect method for a few reasons:

We don't want to encourage folks tracking these metrics by slightly different definitions (e.g. CLS prior to load, etc.)
When bfcache ships in Chrome we're going to need to re-invoke the user callbacks on cache loads, and since (in general) developers are less familiar with nuances of bfcache, I didn't want them to remove listeners prematurely—thinking the event have already happened so they're no longer needed.

We could probably disconnect the LCP and FID listener because it's looking less and less likely that the API is going to change to re-emit those entries (a different name will probably be used), but I suspect the one causing the extension problems in CLS, and I don't want to change the API to allow that.
Out of curiosity, why is the extension creating new listeners with the activation of each tab?

We don't want to encourage folks tracking these metrics by slightly different definitions

Interesting I would've expected that to be accomplished with documentation rather than an inability to ever remove it. In my experience developers tend to forget to remove listeners anyway (like the bug in web vitals extension ;), so I would've been surprised to folks using it on their own if it's discouraged in examples and just a silent .unobserve method sitting on the API.

since (in general) developers are less familiar with nuances of bfcache

You can definitely say that again 😆  I personally have no idea how bfcache works. If you don't mind me asking how does this impact things? Is the concern that page removes the listener onbeforeunload/pagehide? And does bfcache preserve the v8 isolate and resume it once the user clicks back without reexecuting anything? 😮

Out of curiosity, why is the extension creating new listeners with the activation of each tab?

I think it was just an oversight. I found it when looking into why it was pegging my CPU usage at 100%. I don't fully understand the architecture of the extension to answer for them but I'm sure the original authors had good reasons :)

@brendankenny linked me to the bfcache design doc, and I think I have more questions now then when I didn't know how it worked 😆
From the page lifecycle it seems like bfcache will just be identical to page freeze in which case why would anyone want to count it as a separate navigation and have their same listeners reinvoked? It seems like it would actually be very desirable to be able to unobserve once a page goes into freeze to avoid having uninformative instant 0ms LCP timings reported for what is basically someone hitting the back button and the page coming out of hiding instead of an actual navigation. Am I missing something here?

Sorry for the slow reply!

From the page lifecycle it seems like bfcache will just be identical to page freeze in which case why would anyone want to count it as a separate navigation and have their same listeners reinvoked?

I agree it seems a bit counter-intuitive, but here's the basic argument.

bfcache improves the page load experience for users because those pages appear to be loaded instantly, and bfcache does this by restoring those pages from memory rather than reloading them over the network.
However, this now means that most sites will have fewer page loads in their datasets, and the removed page loads would likely have been relatively fast (assuming some amount of caching is used), which will skew the distribution.
The result is that enabling bfcache in Chrome will likely result in an increase in LCP and FID at p75—just because there are now fewer cache loads in the dataset.

To address this concern (as a temporary solution until proper bfcache-measuring APIs are developed), CrUX will report a zero value for both LCP and FID for all bfcache loads. While obviously not perfect, the thinking here is that the likelihood of a bfcache load actually being above the 75th percentile is so small, it's an acceptable tradeoff.
The goal is to try to measure—as accurately as possible given the APIs we have—what the 75th percentile of these metrics is for real usage.
Does that make sense?

That certainly makes sense from the CrUX stakeholder "don't move the goalposts" perspective 👍
As a site owner though, I'm not sure I share this motivation in my own measurements. I'm not interested in my site's performance as a measure of how frequently my users hit the back button :) That's certainly a separately useful insight, but being forced to count this as a separate navigation via the canonical web-vitals API still feels odd. Knowing this, the first thing I'm going to try to do when analyzing my site's performance is going to be excluding these 0s skewing the distribution and anyone that isn't aware of this distinction is probably going to get a falsely rosy picture of their performance because of how counterintuitive it is.
The rabbit hole of "constructing a site that encourages back button interaction to goose CrUX numbers" is probably something not worth exploring here but does add another arrow in my "RUM is too complex to boil down to aggregate thresholds" quiver 😆

As a site owner though, I'm not sure I share this motivation in my own measurements. I'm not interested in my site's performance as a measure of how frequently my users hit the back button :)

It's possible that pages can do things that make them ineligible for bfcache, and if they do, users will experience what feels like slower loads (from their perspective). If none of the metrics we're promoting account for this, then developers may not be aware of the real experiences users are having.

That's certainly a separately useful insight, but being forced to count this as a separate navigation via the canonical web-vitals API still feels odd.

The goal of the web-vitals library is to enable RUM for sites in a way that matches CrUX as closely as possible (in fact, the original name for the library in the design doc was CrUX.js). Since Chrome and CrUX have chosen to measure bfcache loads this way, I want to continue to align with that. (If you still feel this measurement is the wrong approach, you could raise the issue on blink-dev)

Knowing this, the first thing I'm going to try to do when analyzing my site's performance is going to be excluding these 0s skewing the distribution

I think this is totally fine, and I'll likely do the same. Also long as there's some metadata in the callback that makes it clear a particular metric instance came from a bfcache load, you'll be able to annotate the data you send to your analytics back end—allowing you to report on load performance with and without these entries.
I don't think is that much different from a site that adds offline-first service worker caching to make their pages load instantly on repeat visits, which (AFAICT) most people still track as "real" page load in their analytics, since it's what the user is actually experiencing.

anyone that isn't aware of this distinction is probably going to get a falsely rosy picture of their performance because of how counterintuitive it is.

I want to push back here a bit and suggest that this can only be interpreted as a rosy picture if you're focusing on traditional page load performance, but the teams working on Web Vitals at Google are moving more and more away from "page load" and more and more toward a concept of "page visit" or "page session"—we've just not yet identified the right normalization and delineation strategies (e.g. should we delineate any time an SPA updates the URL? How about when a user backgrounds the tab?)
bfcache is one of the more clear-cut delineation candidates, so it was an easier decision to go ahead with it, but we expect to do more of this kind of thing in the future, we just need to make sure it's not easily game-able (as you also point out).

If none of the metrics we're promoting account for this, then developers may not be aware of the real experiences users are having.

Very good point, I hadn't really offered an alternative for how to still incentivize these improvements. I'll get on that :)

Also long as there's some metadata in the callback that makes it clear a particular metric instance came from a bfcache load, you'll be able to annotate the data you send to your analytics back end—allowing you to report on load performance with and without these entries.

True! I must've missed it but how are these annotated right now such that it's clear and obvious to the listener it's a bfcache navigation? Is CrUX going to separate these out into a separate bucket so page loads can continue analyzed without these mixed in? Maybe that's all I'm really looking for.

I don't think is that much different from a site that adds offline-first service worker caching to make their pages load instantly on repeat visits

I think we have different assumptions about the user intent behind hitting the back button then 😆 Perhaps my usage pattern is atypical (if there's any data that was analyzed for user habits with the back button I'd love to learn from them! :D), but hitting back for me is almost always because of an accident where I didn't mean to go somewhere else, e.g. more of a continuation of my last visit and not a new, separate visit. An intentional repeat visit from my perspective is then in a totally different class and deserves to be counted as a separate navigation (along with all the rewards of delivering a high-performing warm load experience).

the teams working on Web Vitals at Google are moving more and more away from "page load" and more and more toward a concept of "page visit" or "page session"

As one should, that's all great. I guess my core beef at the end of the day is it feels like they keep being mixed into the same bucket with "page loads" without any distinguishing characteristics in CrUX. This makes "page loads" more and more difficult to analyze when the bucket is mixture of very different user journeys.

Sorry again for the slow reply! (Believe it or not, I've been out of the country...)

I must've missed it but how are these annotated right now such that it's clear and obvious to the listener it's a bfcache navigation?

Well, the bfcache changes aren't currently in the library (and I'm not 100% sure how I'm going to expose this info), but what I am definitely planning to do is update the Metric.id value, so these metric instances can be separated from the metrics reported prior to the page entering the bfcache.
At a minimum, developers could self-annotate by adding their own pageshow listener:
let isBFCacheLoad = false;

addEventListener('pageshow', (event) => {
  if (event.persisted) {
    isBFCacheLoad = true;
  }
});

I could potentially add something like isBFCacheLoad to the Metric object, but I want to be careful not to do something too specific to bfcache, since it's entirely possible we do something similar for SPA loads or restores from long-backgrounded pages as well.

Is CrUX going to separate these out into a separate bucket so page loads can continue analyzed without these mixed in? Maybe that's all I'm really looking for.

Hmmm, I don't think so?

I guess my core beef at the end of the day is it feels like they keep being mixed into the same bucket with "page loads" without any distinguishing characteristics in CrUX. This makes "page loads" more and more difficult to analyze when the bucket is mixture of very different user journeys.

I can understand that, but I think the intention behind CrUX was to be a reflection of the user experience and not to be a reflection of page load performance (cc: @rviscomi in case he has any thoughts).
For page load performance, specifically, RUM tools should ideally be granular enough to allow segmentation of all Web Vitals metrics by "visit type" (or whatever it ends up being called).

Believe it or not, I've been out of the country...

Woah! I have more questions and fascination to hear these tales of adventure than anything else here 😄 But probably best saved for another medium. Back to it...

At a minimum, developers could self-annotate by adding their own pageshow listener:

I feel like this thread has drifted a little from the original intent, so I'll try to refocus here :) I'm concerned that developers can be trusted enough to know they have to do this in order to fully understand the output of this library but they can't be trusted enough to know how to use a removeEventListener function with documentation 😕
Even if it were to simplify everything with bfcache I'm still incredulous that some form of removeEventListener/unobserve is bad 😆  There are just many cases where a developer needs the ability to juggle and/or remove event listeners they have installed to avoid memory leaks. I definitely see someone wrapping this in a react useEffect hook and getting lots of accidental copies beaconed unnecessarily. At the very least if this is a hard line in the sand and the official position is that removing the installed listeners should be impossible with this library then IMO the docs should come with a big warning to not call these functions more than once.

Hmmm, I don't think so?
CrUX was to be a reflection of the user experience and not to be a reflection of page load performance
RUM tools should ideally be granular enough to allow segmentation of all Web Vitals metrics by "visit type"

Is it accurate to say then from your perspective that CrUX is not a RUM tool? Or at least not trying to be a good RUM tool even if it can somewhat serve as one? I'm not looking for a particular answer here, it's just very helpful to know that there is not alignment on this position of what CrUX is across CWV efforts so we can discuss from a place of shared understanding.

I feel like this thread has drifted a little from the original intent, so I'll try to refocus here :)

Yes, let's refocus :)

I'm concerned that developers can be trusted enough to know they have to do this in order to fully understand the output of this library but they can't be trusted enough to know how to use a removeEventListener function with documentation 😕

I wouldn't say my concern is that they can't be trusted to use a removeEventListener function with documentation, my concern is that I'm not convinced there's a strong enough need to have such a function.
For FCP, FID, and LCP, the library already disconnects the PerformanceObserver once the final value is known, but it doesn't let users unobserve the callback since we'll likely need to invoke it again after a bfcache load. And for CLS we don't actually want people to disconnect the observer, since if they do they'll be measuring CLS incorrectly.
(EDIT: hmm, looks like only FID disconnects the observer, but that's a bug, I'll update the others to do so too)

I definitely see someone wrapping this in a react useEffect hook and getting lots of accidental copies beaconed unnecessarily. At the very least if this is a hard line in the sand and the official position is that removing the installed listeners should be impossible with this library then IMO the docs should come with a big warning to not call these functions more than once.

Sure, I can see how this could be misused, but presumably such a situation would be a mistake, and adding an unobserve function wouldn't necessarily prevent mistakes like that. Also, I don't think there's a valid use case for calling these functions and then disconnecting all of them in a useEffect hook.
All this being said, I'd be happy to add clarifying language indicating that these functions should not be called more than once per page load. (And we'll definitely need clarifying language once the bfcache functionality is added.)
In the end, I want this library to just "do the right thing" as much as possible since we've learned over and over again from lots of experience that people don't thoroughly read the documentation, and there are lots of edge cases that are easy to overlook. For example, with LCP it's clearly documented in the code sample that people measuring LCP should ignore cases where the tab was loaded in the background, yet 100% of the RUM libraries I've audited since we announced Web Vitals have missed this. And these are not random developers, these are folks whose business is building RUM products. So, as a results, I've chosen to keep the API surface for the library as small as possible.

I'll update the others to do so too

Awesome 👍

I'd be happy to add clarifying language indicating that these functions should not be called more than once per page load.

I want this library to just "do the right thing" as much as possible

Great 👍 In this case I would also encourage either renaming away from getX patterns or tracking internally to only allow a single global installation if the intent is that it should never be called more than once. I have a hunch that the author in the CWV extensions situation believed that it was simply getting the value and not installing new listeners that would grow forever, which a name like getLCP certainly suggests to me without any other knowledge about how the library works.

authors: patrickhulce philipwalton

repo: GoogleChrome/web-vitals


text:
Add Web Share Target API support

Relevant issue: #500

Works for me! excited to see this land.

authors: PaulKinlan fstanis

repo: GoogleChromeLabs/gulliver


text:
[Question] Where are the quantization tables for MozJPEG?

Are the quantization tables for MozJPEG generated per image, when selecting the quantization mode in advanced settings, or are they pregenerated tables, I've been looking through the code a bit for the "Flat" one, because I've seen it gets extremely good performance for cartoon/anime image compression and I would like to research it a bit more, but maybe I just don't know what I'm looking for, anyone has any idea where the tables might be, or how are they generated?

Just noticed it's a built in MozJPEG function and not part of Squoosh, sorry!

repo: GoogleChromeLabs/squoosh


text:
Reposition the TwoUp handle on resize

Fixes #105
Repositions the handle for TwoUp when the element resizes.
I have 2 questions:

Do we have constraints on Browser compatibility? Should I include a polyfill for ResizeObserver?
Making TS happy with ResizeObserver was a bit elaborate and I'm a newbie in TS - please advise if there's a better approach!

Do we have constraints on Browser compatibility? Should I include a polyfill for ResizeObserver?

Nah, just fall back to window.onresize.

PTAL

🎉

authors: devnook jakearchibald


text:
add Intel benchmark set and edit README.md

Please test this out and merge if it works. Thanks.

ok to test

test this please

Thanks for making the change and fixing comments.  Once the tests pass I'll merge the CL.
For the next change can you please do it in the Dev branch off master in your fork.  This should allow the pull to go into Dev vs master.  To set the branch on your repo just run "git checkout dev" (http://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging)
Going into master requires more extensive tests and an update on the version tag.  Dev merges are simpler and then we pick everything up in an ~weekly release and do the bigger tests.

Looked into the errors and its a resource limit on CPUs that prevented some tests from running.  The integration tests did all pass on 2 platforms so thats a pass.

LGTM

Hi,
I created a fork off of the dev branch. I just checked the network and it
also showed intel-branch was forked off from the dev branch, not from the
master: https://github.com/hungvelo/PerfKitBenchmarker/network
I followed the instruction on your slides. Did I miss something?
1.Start on GitHub and create a *fork *off of the project.  This will remain
private unit the top level project is public.
2.From your new repository clone the code using the steps on the prior slide
3.Start from dev in your fork (git checkout dev).
4.Create a branch for your changes (git checkout -b ).
5.Make your modifications to the code in one or more commits (guidelines
for useful git commit messages
http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
6.Push your changes to GitHub (git push origin)
7.Create a pull-request to integrate your change into dev. This can be done
on github.com, or by installing and using the command line tool gh
https://github.com/jingweno/gh (gh pull-request -h  -b
dev).
Please let me know if these instructions are correct.
Thank you.
On Tue, Feb 24, 2015 at 5:15 AM, Anthony Voellm aka Tony the P3rfguy <
notifications@github.com> wrote:

Thanks for making the change and fixing comments. Once the tests pass I'll
merge the CL.
For the next change can you please do it in the Dev branch off master in
your fork. This should allow the pull to go into Dev vs master. To set the
branch on your repo just run "git checkout dev" (
http://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging)
Going into master requires more extensive tests and an update on the
version tag. Dev merges are simpler and then we pick everything up in an
~weekly release and do the bigger tests.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/128#issuecomment-75753911
.

Hi Hung,
Thanks for this update! We could have been more clear here - when creating the pull request, the base branch in the comparison should be dev (it defaults to master).  I'll update the guidelines.
Connor

Ditto to Connors comment.  Sorry for not being more clear.  We'll be
updating a Contributing.md file with instructions.
The good new is the commit is complete and master is now labeled at v0.10.1
On Tue, Feb 24, 2015 at 8:05 AM, Connor McCoy notifications@github.com
wrote:

Hi Hung,
Thanks for this update! We could have been more clear here - when creating
the pull request, the base branch in the comparison should be dev (it
defaults to master). I'll update the guidelines.
Connor
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/128#issuecomment-75784948
.

--
Anthony F. Voellm (aka Tony)
Google Voice:  (650) 516-7382
https://www.google.com/voice/b/0?pli=1#phones
Blog: http://perfguy.blogspot.com/

authors: cmccoy hungvelo voellm

repo: GoogleCloudPlatform/PerfKitBenchmarker


text:
Enable Cloud (Stackdriver) Debugger on Java flex by default

The Cloud Debugger team confirmed that the Debugger issue that forced us to disable it temporarily on the flex env (including Phython runtimes) has been resolved. The team asked me to take an action to enable it. Note that the Debugger is already enabled by default on the standard env (including jetty9-compat), and it should be automatically enabled for Java flex as documented here.
I took the liberty to make a change to setup-env.bash at my discretion, but feel free to jump in.
The document here should also be revised to correct the explanation for the DBG_ENALBE env variable.

Internal bug number: 28192592

LGTM

@gregw. Thanks for the update. Actually, @ludoch is concerned about security and other issues about having it on by default. I guess we will have some internal discussions for a while.

Okay! It turns out @ludoch mistook Stackdriver Debugger for a local debugger. We're good to go.

Yes, Greg, if you have fine as well, integrate.
On Thu, Apr 14, 2016 at 7:27 PM, Chanseok Oh notifications@github.com
wrote:

Okay! It turns out @ludoch https://github.com/ludoch mistook
Stackdriver Debugger for a local debugger. We're good to go.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/appengine-java-vm-runtime/pull/219#issuecomment-210256464

Also changed Cloud Debugger to Stackdriver Debugger. It's official.

authors: chanseokoh gregw ludoch


text:
globStatus call runs very slowly from Hadoop to GCS compared with gsutil

Run a glob listing on a big tree of around 890K entries.

Using Hadoop:
hadoop fs -ls gs://bucketabc/1???/1/1/small-1.txt

takes  3+ minutes;

Using Spark
spark.read.format("com.databricks.spark.csv")
    .option("header", "false").option("delimiter", "\t").option("inferSchema", "true").option("mode", "DROPMALFORMED")
    .load("gs://bucketabc/1???/1/1/small-1.txt")

takes 7+ minutes

While gsutil:
gsutil ls gs://bucketabc/1???/1/1/small-1.txt

only takes 8 seconds.

It turns out that the full glob pattern has not been pushed down to GCS by the GCS connector.
This problem could have significant impact on queries on data sets with huge number of partitions for either query performance consideration and/or as result of huge number of distributed processing tasks like MR and Spark.
A simple prototype fix has been observed to be able to close the significant perf gap.

We need to address globbing performance in 2 phases:

run default glob and flat glob algorithms concurrently and return result as soon as one finishes.
parallelize flat glob algorithm implementation in the same way as gsutil does it.

repo: GoogleCloudPlatform/bigdata-interop


text:
com.google.api.client not shaded in GCS Connector fat jar

GCS connector has a dependency on google-api-client, but does not shade it. When an application brings its own version of google-api-client, this results in 404 errors like the following:
Job setup failed : com.google.api.client.http.HttpResponseException: 404 Not Found
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1070)
	at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:241)
	at com.google.cloud.hadoop.gcsio.BatchHelper.flushIfPossible(BatchHelper.java:118)
	at com.google.cloud.hadoop.gcsio.BatchHelper.flush(BatchHelper.java:132)
	at com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.getItemInfos(GoogleCloudStorageImpl.java:1493)
	at com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage.getItemInfos(ForwardingGoogleCloudStorage.java:221)
	at com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem.getFileInfos(GoogleCloudStorageFileSystem.java:1159)
	at com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem.mkdirs(GoogleCloudStorageFileSystem.java:530)
	at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.mkdirs(GoogleHadoopFileSystemBase.java:1381)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1881)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:313)
	at org.apache.hadoop.mapred.FileOutputCommitter.setupJob(FileOutputCommitter.java:131)
	at org.apache.hadoop.mapred.OutputCommitter.setupJob(OutputCommitter.java:265)
	at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobSetup(CommitterEventHandler.java:254)
	at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:234)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

This seems to be caused by not shading com.google.api.client dependencies in the maven-shade-plugin config.

The same issue was reported to Dataflow Java SDK: https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/607
Workaround is to use google-api-client 1.22.0 instead of 1.23.0.
Maven:
<dependency>
  <groupId>com.google.api-client</groupId>
  <artifactId>google-api-client</artifactId>
  <version>[1.22.0]</version>
</dependency>

Gradle:
compile ('com.google.api-client:google-api-client:1.22.0') {
  force = true
}

authors: jmikula medb


text:
Allow attaching PSA for multiple services

This allows defining PSA connectivity for multiple providers in one module call.

Checklist

I applicable, I acknowledge that I have:

[x] Read the contributing guide
[x] Ran terraform fmt on all modified files
[x] Regenerated the relevant README.md files using tools/tfdoc.py
[ ] Made sure all relevant tests pass

Superseded by #2218


text:
feat: add test org organization IAM cleaner

This PR aims to

define current org IAM policy and remove previous org IAM bindings
add cloud build yaml to apply this config
add cloud scheduler config to trigger cloud build every weekend to reconcile IAM

TODO:

[x] confirm  IAM policy
[x] confirm scheduler API enabled in ci-project

All changes have been applied.


text:
Test cleanup: Delete exported disk image if export is successful.

Background
The image export tests create terabyte-sized files, which are costly for storage, and are not required when the test was successful. This change deletes the image files if the test passed.
Testing

Ran daisy_integration_tests/image_export_vmdk_given_size_10gb.wf.json and verified that the two image files were removed. (created-by-export-test-given-size.vmdk and the image in outs)

/retest

repo: GoogleCloudPlatform/compute-image-tools


text:
Update README.md

Added a note to limitations about interactions with the Soft Delete feature

@bernardhan33 Is anything pending my action to merge? Thx!

@dutchiechris Sorry for the delay. The PR was pending a Kokoro test run and I just added it. It's all good now and I'm going to merge. Thanks!

repo: GoogleCloudPlatform/dataflux-pytorch


text:
[publishing] [notebook] MySQLToSpanner notebook publish blogpost

Publish a blogpost with screenshot explaining deployment and execution of MySQLToSpanner notebook

Draft version created, currently in review

Published : https://medium.com/google-cloud/mysql-to-cloud-spanner-migrate-mysql-database-to-cloud-spanner-using-vertex-ai-notebooks-and-gcp-ad7d2ed8a317

repo: GoogleCloudPlatform/dataproc-templates


text:
Using .flac format Syncrecognize gives null as results and Etag

I am using .flac format file for google speech api SyncRecognize method but it always returns results and Etag as null. I am using Naudio to record my .wav file and i am converting it to .flac ,and after sending this .flac file to syncRecognize it gives null result always.Its Urgent.
Please Help to resolve this.

I just added a sample that calls SyncRecognize on a .flac file.  What happens when you run:
PS C:\...\dotnet-docs-samples\speech\api> .\Recognize\bin\Debug\Recognize.exe rec -e Flac .\resources\
\audio.flac
how old is the Brooklyn Bridge

Closing due to inactivity.

authors: SurferJeffAtGoogle khopadesnehal

repo: GoogleCloudPlatform/dotnet-docs-samples


text:
Example not working on Windows 😁

This is what I did, also tried running build_runner command but no luck.
I use Flutter so does that make some problem for the environment?
PS C:\Users\yazee\Desktop\hello> dart run .\bin\server.dart
Unhandled exception:
SignalException: Failed to listen for SIGTERM, osError: OS Error: The request is not supported.
, errno = 50

Dart Version
PS C:\Users\yazee> dart --version
Dart SDK version: 2.12.0-141.0.dev (dev) (Thu Dec 10 06:04:48 2020 -0800) on "windows_x64"

Hmm, Windows doesn't support SIGTERM ... therefore, it looks like the mere act of trying to register a signal handler fails. This is a devex issue for Windows since it will work on Linux in the container that gets deployed. We can fix by detecting if we're on Windows before attempting to register the handler.

@YazeedAlKhalaf Did you run pub get first?
@kevmoo Are there any issues you're aware of using build_runner on Windows with a Flutter-only install?

@subfuzion  Yes, I did run pub get. It works fine when using docker but otherwise, it crashed with the exception above.
I believe it will be fixed once your pull request is merged?

@kevmoo Need a Windows expert to weigh in on this. While the exception would seem to indicate that SIGTERM isn't supported, I found this old Microsoft doc that seems to indicate that SIGTERM is supported. 🤷🏻♂

@YazeedAlKhalaf Yeah, it works with Docker since the container actually runs in a Linux VM. As to my PR, it's just a proposed fix -- but I'm not entirely certain about it and want someone with Windows expertise to review. According to this doc, Windows includes SIGTERM for ANSI compatibility.

Really weird, anyway. I can confirm the fix works and I edited the pub cache with your edits and it now runs on my windows machine! 🚀
Thanks a lot 🚀 @subfuzion

authors: YazeedAlKhalaf subfuzion


text:
cleanup

Remove TODO that's already completed
Improve error messages and exit codes for failures

Sorry, approved prematurely. "Correctly return 404 for robots.txt and favicon.ico " really ought to be its own PR ("Closes #2"). Thanks!

I will land these all separately – no worries!


text:
Datastore transactions do not work without context manager

The docs state that you can use transactions without a context manager, but this is not exactly true because you need to resort to some private methods on Client:

this file uses the context manager and produces correct output
this file uses transactions as described by the aforementioned docs and fails the assertion
this file uses transactions along with calls to client._push_batch and client._pop_batch and produces correct output

The issue is evident in Client.get_multi and Client.put_multi where transaction and current are None unless a Batch or Transaction has been pushed to the _batch_stack and the only time that happens is when you __enter__ a Batch.

@Bogdanp thanks for finding this!
In the case where the assertions fail, is it both values or just one of them that isn't getting set when you t.commit()?

Both values are updated. However, due to the fact that there is no transaction id "in scope" for those get and put requests, the serializability of the operations is not enforced.
Here's some sample output:
(tempenv-3357153745070) ~ [1]> env DATASTORE_DATASET="test" DATASTORE_PROJECT_ID="test" DATASTORE_HOST="http://127.0.0.1:9999" python failing_test.py
60
40
Traceback (most recent call last):
  File "failing_test.py", line 54, in <module>
    assert a1["balance"] == 100 - n * 10
AssertionError

I just added a couple of print statements before the assertions in the failing example:
    print a1["balance"]
    print a2["balance"]

    assert a1["balance"] == 100 - n * 10
    assert a2["balance"] == n * 10

Client.put_multi is never called in your failing example:  the changes are all accumulated inside the transaction's _commit_request protobuf.  I think the error is that the transaction ID (fetched during Transaction.begin) is not being stored into that protobuf.

I just tried to write a system test replicating your failing report, and found that it passes:
    def test_transaction_via_imperative_begin_commit(self):
        BEFORE_1 = 100
        BEFORE_2 = 0
        TRANSFER_AMOUNT = 40
        key1 = Config.CLIENT.key('account', '123')
        account1 = datastore.Entity(key=key1)
        account1['balance'] = BEFORE_1
        key2 = Config.CLIENT.key('account', '234')
        account2 = datastore.Entity(key=key2)
        account2['balance'] = BEFORE_2
        Config.CLIENT.put_multi([account1, account2])

        def transfer_funds(client, from_key, to_key, amount):
            xact = client.transaction()
            xact.begin()
            from_account = client.get(from_key)
            to_account = client.get(to_key)
            from_account['balance'] -= amount
            to_account['balance'] += amount

            xact.put(from_account)
            xact.put(to_account)
            xact.commit()

        transfer_funds(Config.CLIENT, key1, key2, TRANSFER_AMOUNT)

        after1 = Config.CLIENT.get(key1)
        after2 = Config.CLIENT.get(key2)
        self.assertEqual(after1['balance'], BEFORE_1 - TRANSFER_AMOUNT)
        self.assertEqual(after2['balance'], BEFORE_2 + TRANSFER_AMOUNT)

One issue might be thatClient.get_multi is not guaranteed to return the items in the order of the requested keys:  can you update your failing example to compensate (e.g., either use Client.get or else check the keys of the returned entities)?

(Of course, my example is not threaded with multiple writers, so it is not really a good enough replica).

Here's a failing example w/o any get_multi calls.
(tempenv-3357153745070) ~ [1]> env DATASTORE_DATASET="test" DATASTORE_PROJECT_ID="test" DATASTORE_HOST="http://127.0.0.1:9999" python failing_test.py
60
40
Traceback (most recent call last):
  File "failing_test.py", line 56, in <module>
    assert a1["balance"] == 100 - n * 10
AssertionError

BTW, I picked this example specifically because it's what's being used in the official docs.

Are you sure this isn't the culprit? transaction_id ends up being None there unless _push_batch is called.

I was still investigating:  you are correct that get/get_multi do not provide any support for doing the reads in the context of an explicit transaction.  I have a PR in progress which will add that support.

@Bogdanp Thanks for the detailed bug report!  I was able to get a version of your failing script passing on the branch for #1861 (after failing on master in exactly the way you describe here).

authors: Bogdanp daspecster tseaver

repo: GoogleCloudPlatform/gcloud-python


text:
feat(trace): small updates to OTel sample so it works in Cloud Shell

Description

A few updates to the docker-compose.yaml so it works in Cloud Shell with minimal friction
For local dev, added docker-compose.adc.yaml which mounts GOOGLE_APPLICATIONS_CREDENTIALS inside the docker container. This is only needed if there is no metadata server (e.g. cloudtop).

Testing
I manually tested this is working in Cloud Shell with these steps:
git clone --branch=cloud-shell-fixes --single-branch https://github.com/aabmass/golang-samples
cd golang-samples/opentelemetry/instrumentation
docker compose up --abort-on-container-exit

Checklist

[x] I have followed Contributing Guidelines from CONTRIBUTING.MD
[x] Tests pass:   go test -v ./.. (see Testing)
[x] Code formatted:   gofmt (see Formatting)
[x] Vetting pass:   go vet (see Formatting)
[ ] These samples need a new API enabled in testing projects to pass (let us know which ones)
[ ] These samples need a new/updated env vars in testing projects set to pass (let us know which ones)
[ ] This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample
[ ] This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample
[ ] Please merge this PR for me once it is approved

@telpirion can you help review this as well?  It is a follow-up to #3312

authors: aabmass dashpole

repo: GoogleCloudPlatform/golang-samples


text:
Support 2.5.x version of ESP8266 Library

Currently, newer versions of the ESP8266 library (those past 2.4.2) are not working properly with Cloud IoT Core. There's a resolution to disable fingerprint verification but this is not desirable.

I am having some issues with the library and I think it is because of this. Can you share how to disable fingerprint verification?I just want to setup a simple DIY thermostat at home that I can query from Google Home so I am not really concerned with security.

If you use 2.4.2, it should not have the issue with certificate validation.

This should be resolved with release 1.1.7. Closing :)

authors: gguuss raresserban


text:
Fix rare failure modes of monitoring test

Address 3 intermittent issues with integration test 3:

timeout on startup-script completion (change 5 minutes to 10 minutes)
failure to create an Ansible tmp directory under /home when /home is mounted over NFS during startup-script
rare occasions when ghpc binary is over-written during execution because of multiple invocations of create_deployment.sh in a single build

Submission Checklist

[X] Have you installed and run this change against pre-commit? (pre-commit install)
[X] Are all tests passing? (make tests)
[X] Have you written unit tests to cover this change?
[X] Is unit test coverage still above 80%?
[X] Have you updated all applicable documentation?
[X] Have you followed the guidelines in our Contributing document?

For posterity: the Group 1 check failed for a reason unrelated to this commit (Slurm appears not to have started up before sinfo command is invoked to list partitions)

repo: GoogleCloudPlatform/hpc-toolkit


text:
Test flake in kubelet TestSyncPodsDoesNothing

https://s3.amazonaws.com/archive.travis-ci.org/jobs/35977365/log.txt

Still an issue?

I have not seen this in forever.

authors: roberthbailey smarterclayton

repo: GoogleCloudPlatform/kubernetes


text:
Problems specifying memory limits in Pods

I'm trying to create a Pod using a very simple example. In fact is the pod.yaml example only adding a memory limit:
apiVersion: v1beta3
kind: Pod
metadata:
  labels:
    name: nginx
  name: nginx
  namespace: default
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    ports:
    - containerPort: 80
      protocol: TCP
    resources:
       limits:
         memory: 536870912
  restartPolicy: Always

The Pod is created also the corresponding docker container.
But the Pod remains in state "Pending" and it shows this message:
Query container info failed with error (json: cannot unmarshal number 5.36870912e+08 into Go value of type int64)

It seems that it has some problems getting the amount of memory of the container provided by docker.
If I inspect the created container, the amount of memory provider by docker is
"Memory": 5.36870912e+08

If fact I try to delete the Pod and the associated container is not deleted.
I'm using the last kubernetes version downloaded from the Github repo using the v1beta3 API version.
I launch kubernetes using the local-up-cluster.sh script.
I'm using Go version 1.4.2 on an Ubuntu 14.04.2.

What Docker version are you using?
cc @vishh

Which docker version and kubernetes version are you using? There is a compatibility issue between docker (remote_api_v1.18) and docker_client library kubernetes are using before 0.18. The detail is discussed in #8358

My config for issue #8780
root@pmd-122:~# kubectl version
Client Version: version.Info{Major:"0", Minor:"17+", GitVersion:"v0.17.0-1-gbe15e84e83f123-dirty", GitCommit:"be15e84e83f12300c48081860d36624d1695fb02", GitTreeState:"dirty"}
Server Version: version.Info{Major:"0", Minor:"17+", GitVersion:"v0.17.0-1-gbe15e84e83f123-dirty", GitCommit:"be15e84e83f12300c48081860d36624d1695fb02", GitTreeState:"dirty"}
root@pmd-122:~# docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64

I'm using the docker version that comes with the Ubuntu 14.04 distribution.
I see that is an old version ...
root@ubuntu:~# docker version
Client version: 1.0.1
Client API version: 1.12
Go version (client): go1.2.1
Git commit (client): 990021a
Server version: 1.0.1
Server API version: 1.12
Go version (server): go1.2.1
Git commit (server): 990021a

This is the kubernetes version:
root@ubuntu:~# kubectl.sh version
Client Version: version.Info{Major:"0", Minor:"17.1+", GitVersion:"v0.17.1-dev", GitCommit:"", GitTreeState:"not a git tree"}
Server Version: version.Info{Major:"0", Minor:"17.1+", GitVersion:"v0.17.1-dev", GitCommit:"", GitTreeState:"not a git tree"}

I have updated docker to version:
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64

And now I'm getting another error:
API error (500): Cannot start container 759d8d2b736fd94c07cb0cb95bb2a9b569578e2003d283fc2ed04a57c64e48f6: [8] System error: open /sys/fs/cgroup/memory/docker/759d8d2b736fd94c07cb0cb95bb2a9b569578e2003d283fc2ed04a57c64e48f6/memory.memsw.limit_in_bytes: permission denied

I have changed the docker version to:
Client version: 1.3.0
Client API version: 1.15
Go version (client): go1.3.3
Git commit (client): c78088f
OS/Arch (client): linux/amd64
Server version: 1.3.0
Server API version: 1.15
Go version (server): go1.3.3
Git commit (server): c78088f

And, with this version, it works correctly.
If I inspect the created container, the amount of memory provider by docker is
"Memory": 536870912,

So it does not have problems "unmarshaling" the amount of memory.

It turns out there is an issue in docker.  Docker remote API never check if memory swap is enabled by kernel or not. If memory limit is passing to docker, by default it will set swap limit = memory_limit *2, thus return API error (500) if swap is disabled. GKE image is disabling swap.
PR #9358 explicitly passing -1 to disable swap.

cc/ @brendandburns We might want to patch this workaround to our v0.18 release. I will file a separate issue to docker.

@smarterclayton - need to cherry pick this.  Will add to list.
Sent from my iPhone

On Jun 5, 2015, at 9:22 PM, Dawn Chen notifications@github.com wrote:
cc/ @brendandburns We might want to patch this workaround to our v0.18 release. I will file a separate issue to docker.
—
Reply to this email directly or view it on GitHub.

See https://github.com/GoogleCloudPlatform/kubernetes/issues/7294
Sent from my iPhone

On Jun 5, 2015, at 9:22 PM, Dawn Chen notifications@github.com wrote:
cc/ @brendandburns We might want to patch this workaround to our v0.18 release. I will file a separate issue to docker.
—
Reply to this email directly or view it on GitHub.

@krousey Thanks.
@zmerlynn @roberthbailey @brendandburns Should we patch v0.18 release or wait for v0.19? All GKE users cannot specify memory limit to their workload due to this regression.

@dchen1107 Do you know if this functionality is covered in end-to-end tests?  If not, it should be.  We should have caught this earlier, (per @brendandburns).

@ihmccreery Yes, that is the plan. We do have docker container e2e tests, but it doens't verify the resource limits. We are going to improve that.

authors: bgrant0607 dchen1107 derekwaynecarr ihmccreery micafer njuicsgz


text:
Rkt pull

@vmarmol @dchen1107 @jonboulle
Still depends #7465

LGTM, only minor nits

LGTM

authors: vmarmol yifan-gu


text:
Copyright year change

cc/ @rosbo

The magician is wiping the directory before generating it again.
The reason to keep the .last_run.json was to have an history of test coverage but I don't think we are updating it every time. In fact, the last time the file was updated was in October '17.
We could add travis

The reason the file is missing at the end is because the tests are not being run. It is the rspec that updates the last_run.json file. Please run the tests and do not delete this file from the source tree.

We know why the last_run.json file is not there. That is not the question.
The question is: is this file useful anymore? Last time the file was updated is in Oct'17. It clearly has no use.
If we want to include the last_run.json file, we will need to change the concourse code to enable the test target to update the generated branches. I am not a big fan of that.
The way it works is the magician first runs the mm-generate task to generate the code for downstream repos.
Then it runs puppet-test which run rspec. However, this step doesn't update the branch created in mm-generate to update the last_run.json file.
If we decide to get rid of it, we should add an entry to .gitignore.

The fact is not updated does not take its usefulness away ;) The reason for it is for us to track over time the code coverage. We used that to make sure we do not degrade the code. Given all the things we've been doing it got stale, but I'd rather fix that instead of getting rid of it.
Here you can see an example on how this is useful: https://presubmit.graphite.cloudnativeapp.com/c/dd43cd4b23c0d723fe15752769d3ef6181e924a8. Look at the code coverage table.
If the reason of this question is complexity to fix this we can find a way. Let me know if you need help devising a scheme to update it appropriately.

In my book, when you stop doing a process (update the last_run.json file) to update a metric (code coverage) for 7 months and nobody notice, I would argue that the value that metric brings is questionable, potentially close to zero.
I will talk with @ndmckinley to see what is involved in fixing this. Also, no CI system that I know keep track of the code coverage results in the checked-in code, they all track it using as metadata of their build in their database. This avenue is probably better than keeping with the current system (file in checked-in code).

I strongly agree that checking in this file is an unusual way to keep track of code coverage and I don't really like the idea of The Magician doing it.  How strongly are you insisting on that?  I can do it, but, as you know, every time you introduce additional complexity you introduce more chance for error.

I added back the last_run.json file manually for now. We can discuss this at our next MM meeting.

authors: modular-magician ndmckinley nelsonjr rosbo

repo: GoogleCloudPlatform/puppet-google-sql


text:
issues/7 If trailing slash is missing, then force redirect.

See #7

To test locally, I used a small Go server: serve_at.go

repo: GoogleCloudPlatform/region-picker


text:
FailedMount Configmap not found warning on spark driver pod

I'm running spark operator on kubeadm. and got the error:
 Warning  FailedMount  7m26s  kubelet            MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-0251af7c7dfbe657-conf-map" not found

Hello,
I'm having the same warning, but I still haven't identified what it is!!!
MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-5d2ffa7c88dec4ff-conf-map" not found

However, the process ends successfully.

I believe this is expected if you're not also running the mutating admission webhook. Check out the README for what it's used for. Can you confirm whether or not you're running that?

I'm having the same warning _with_the webhook running. it looks like the config map is created 2s after it has failed to load on the driver.

I built spark operator docker image from cloned repo and pushed to my local artifactory with tag 1.3.1-3.1.1. after deployed operator, could see deployment results like here
deployment.apps/sparkoperator created clusterrole.rbac.authorization.k8s.io/spark-mutating-webhook created clusterrolebinding.rbac.authorization.k8s.io/spark created role.rbac.authorization.k8s.io/spark-role created rolebinding.rbac.authorization.k8s.io/spark created serviceaccount/spark created job.batch/sparkoperator-init created service/spark-operator-webhook created
then spark operator log shows below:
`root:x:0:0:root:/root:/bin/bash

echo root:x:0:0:root:/root:/bin/bash
[[ -z root:x:0:0:root:/root:/bin/bash ]]
exec /usr/bin/tini -s -- /usr/bin/spark-operator -v=2 -logtostderr -namespace=spark-operator -enable-ui-service=true -ingress-url-format= -controller-threads=10 -resync-interval=30 -enable-batch-scheduler=false -label-selector-filter= -enable-metrics=true -metrics-labels=app_type -metrics-port=10254 -metrics-endpoint=/metrics -metrics-prefix= -enable-webhook=true -webhook-svc-namespace=spark-operator -webhook-port=8080 -webhook-timeout=30 -webhook-svc-name=sparkoperator-webhook -webhook-config-name=sparkoperator-webhook-config -webhook-namespace-selector= -enable-resource-quota-enforcement=true
I0112 12:50:57.252870      10 main.go:145] Starting the Spark Operator
I0112 12:50:57.253588      10 main.go:178] Enabling metrics collecting and exporting to Prometheus
I0112 12:50:57.253848      10 metrics.go:142] Started Metrics server at localhost:10254/metrics
I0112 12:50:57.255734      10 watcher.go:43] Creating new resource usage watcher
I0112 12:50:57.356252      10 webhook.go:219] Starting the Spark admission webhook server
I0112 12:50:57.365166      10 webhook.go:424] Creating a MutatingWebhookConfiguration for the Spark pod admission webhook
I0112 12:50:57.394247      10 webhook.go:452] Creating a ValidatingWebhookConfiguration for the SparkApplication resource quota enforcement webhook
I0112 12:50:57.411905      10 main.go:220] Starting application controller goroutines
I0112 12:50:57.412003      10 controller.go:165] Starting the workers of the SparkApplication controller
I0112 12:50:57.412361      10 controller.go:97] Starting the ScheduledSparkApplication controller
I0112 12:50:57.412398      10 controller.go:103] Starting the workers of the ScheduledSparkApplication controller`

after I deploy my spark-sream app, driver not mounting configmap, but configmap actually exists.

how to resolve this warning ?

This issue resolved in below spark operator image. Start using this image.
ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.4-3.1.1

From: yangqi766 @.>
Sent: Monday, May 30, 2022 10:31 AM
To: GoogleCloudPlatform/spark-on-k8s-operator @.>
Cc: 7amarivu @.>; Comment @.>
Subject: Re: [GoogleCloudPlatform/spark-on-k8s-operator] FailedMount Configmap not found warning on spark driver pod (#1368)
how to resolve this warning ?
—
Reply to this email directly, view it on GitHubhttps://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/1368#issuecomment-1140699015, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AXIVRPLPP235NRQVVVFCS6TVMRDQ3ANCNFSM5F7DYISA.
You are receiving this because you commented.Message ID: @.***>

I use the version is  v1beta2-1.3.6-3.1.1 !   And I also test the version v1beta2-1.3.4-3.1.1 , but the warning still exists .

Is custom confgmap mounted in both versions?
if config volume mounted to driver and executor pod, can ignore this warning.

From: yangqi766 @.>
Sent: Monday, May 30, 2022 10:56 AM
To: GoogleCloudPlatform/spark-on-k8s-operator @.>
Cc: 7amarivu @.>; Comment @.>
Subject: Re: [GoogleCloudPlatform/spark-on-k8s-operator] FailedMount Configmap not found warning on spark driver pod (#1368)
I use the version is v1beta2-1.3.6-3.1.1 ! And I also test the version v1beta2-1.3.4-3.1.1 , but the warning still exists .
—
Reply to this email directly, view it on GitHubhttps://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/1368#issuecomment-1140713804, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AXIVRPKTZ75WQYXF642MYHTVMRGQ7ANCNFSM5F7DYISA.
You are receiving this because you commented.Message ID: @.***>


text:
Set default container limits and concurrency value

Set default variable values to fix #30

I updated the files. Checks seem to be failing on terraform fmt checks, but I can't tell what is wrong with the format.

I added comment on the #30. Please review.