text:
Notebook coastalerosion

Proposed changes
Updated the coastal erosion notebook to use the new template, and moved over all necessary functions required for the notebook.
Checklist (replace [ ] with [x] to check off)

[X] Notebook created using the DEA-notebooks template
[X] Remove any unused Python packages from Load packages
[X] Remove any unused/empty code cells
[X] Remove any guidance cells (e.g. General advice)
[X] Ensure that all code cells follow the PEP8 standard for code. The jupyterlab_code_formatter tool can be used to format code cells to a consistent style: select each code cell, then click Edit and then one of the Apply X Formatter options (YAPF or Black are recommended).
[X] Include relevant tags in the final notebook cell (refer to the DEA Tags Index, and re-use tags if possible)
[X] Clear all outputs, run notebook from start to finish, and save the notebook in the state where all cells have been sequentially evaluated
[ ] Test notebook on both the NCI and DEA Sandbox (flag if not working as part of PR and ask for help to solve if needed)
[ ] If applicable, update the Notebook currently compatible with the NCI|DEA Sandbox environment only line below the notebook title to reflect the environments the notebook is compatible with

Hey @robbibt, this is looking great. Having a quick read through, I noticed that under the "Compute Modified Normalised Difference Water Index" section, you have a hyperlink to the research paper. What do you think about changing this to be a citation that's hyperlinked, similar to the paper reference in the NDCI notebook?

@caitlinadams  No problem, will update. I've just tried testing the notebook on the NCI and the Collection 3 database seems to be out of action, which means that right now, all the other "NCI compatible" notebooks will cause errors too...

Really nice case study, @robbibt. One quick comment, is it possible to get a color ramp on the shoreline migration plot to make it obvious that the red colours indicate more recent shorelines?  Maybe this is already implemented on the interactive plot (I just read the notebook, I haven't run it), in which case ignore this.

@cbur24 I'd really like a key on that figure too, although custom legends are always a hassle in matplotlib! I'll see if there's an easy option

@caitlinadams @cbur24 Have made both your requested changes (Geopandas made adding a colourbar easy!). If you're happy I think it should be OK to have this merged in even though we can't test it on the NCI - we can come back and test it at the end before we make develop the new master

authors: caitlinadams cbur24 robbibt

repo: GeoscienceAustralia/dea-notebooks


text:
Update ColorPicker version

Fix ColorPicker bug.

Has been updated separately in v27.1.0.0

source: gharchive/pull-request

authors: Gericop tcqq

license_type: permissive

license_source: bigquery


text:
🛑 RSS-Bridge is down

In 307a07f, RSS-Bridge ($RSSBRIDGE) was down:

HTTP code: 0
Response time: 0 ms

Resolved: RSS-Bridge is back up in 3134244.

source: gharchive/issue

repo: GetRSS/upptime

license: MIT

license_source: github-api


text:
Links for desktop installer broken.

Prerequisites

[X] Put an X between the brackets on this line if you have done all of the following:

Checked that your issue isn't already filed: https://github.com/GetStream/Winds/issues

Description
Opening https://s3.amazonaws.com/winds-2.0-releases/latest.html, none of the links for Windows, Mac or Linux are working. Error message follows. See below.
Steps to Reproduce

Open https://s3.amazonaws.com/winds-2.0-releases/latest.html
Click the links

Expected behavior: [What you expect to happen]
Expect to download the installer
Actual behavior: [What actually happens]
On clicking on installers
Error Message:
This XML file does not appear to have any style information associated with it. The document tree is shown below.

AccessDenied
Access Denied
AHDR1K4X8P2MBJBY
OU8w4EP74taHSgpL7SXbv0qN3L4ezdtPqcKPw4ESuBLfkD0D9gF1fFAclzTDNCnJWoWP8OpiyH8=

Reproduces how often: [What percentage of the time does it reproduce?]
100% of time

The newest one I could find was 3.0.0. Here is a working download link.
https://s3.amazonaws.com/winds-2.0-releases/releases/Winds Setup 3.0.0.exe

Same thing happens for the Mac link, install using Homebrew works however:
brew cask install winds

Why aren't the AppImages simply hosted in releases?

I tried to find a download link and it doesn't seem to work: the link in the readme navigates to the website, and there clicking on download gets you back to this repo.
Is there a link where official AppImages can be downloaded?

authors: balthasars evdcush medic-code ngetchell serpro69

repo: GetStream/Winds


text:
"vg ensure -- -update" does not play well with persisted local packages

Below github.com/qubit-sh/go-crypto is a persisted package (symlinked from /Users/mmuthanna/.virtualgo/qubit/src/github.com/qubit-sh/go-crypto.)
$ vg ensure -- -v -update
Running "dep ensure -v -update"
Root project is "github.com/qubit-sh/xpoller"
 1 transitively valid internal packages
 2 external packages imported from 2 projects
(0)   ✓ select (root)
(1)     ? attempt github.com/qubit-sh/go-crypto with 1 pkgs; 1 versions to try
(1)         try github.com/qubit-sh/go-crypto@master

<SNIP>

Uninstalling "github.com/qubit-sh/go-crypto" from workspace
  Removing sources at "/Users/mmuthanna/.virtualgo/qubit/src/github.com/qubit-sh/go-crypto"
Error: Couldn't move the vendor directory to the active workspace: rename vendor /Users/mmuthanna/.virtualgo/qubit/src: no such file or directory
Usage:
  vg ensure [-- [arguments to dep ensure]] [flags]

Flags:
  -h, --help   help for ensure

Couldn't move the vendor directory to the active workspace: rename vendor /Users/mmuthanna/.virtualgo/qubit/src: no such file or directory

The workaround is to call vg ensure immediately after to relink the persistent package.

Looking at the output it seems related to #13. It seems like for some reason the vendor directory is not created after dep ensure -v -update (maybe caused by an error in the snipped part). This should probably be checked by vg, before unlinking and a better error message should be suplied. However, I cannot reproduce this issue myself. Could you supply some steps to reproduce?
Furthermore, on Monday I will probably be releasing an update to vg with some big changes and improvements to local install. Keep in mind that the default for import mode for new workspaces will be global fallback again, because that causes the least issues. You can try it out by checking out the bindfs branch, maybe it contains a fix to your issue by accident. If you do try it out, be sure to install bindfs for the best experience. And you should probably read the import mode section in the readme: https://github.com/GetStream/vg/tree/bindfs#workspace-import-modes

Thanks for digging in. There were no errors in the snipped part, however I'm not sure what happened because when I just ran it again, it worked!
Good to hear about improvements to local install -- it's the most important part of my workflow :-)
Looking at the new doc (from the bindfs branch), must I now always add the --full-isolation flag during vg activate? Does it not remember the setting during vg init?

Weird that it doesn't happen again. However, one of my colleagues reported a similar random issue a couple of days ago. I just released a new bug release version with a better error message when vendor cannot be found after dep ensure. Hopefully that allows for better investigation if the issue comes up again.
Furthermore, thanks for the feedback on the updated doc. No need to specify the --full-isolation flag for every vg activate, because it does indeed remember it. However it's a per workspace setting so each time you do vg init for a new project you have to specify it. In a future release there might be a global vg config file where this can be set. I will try to make this clearer before releasing the changes. Also, I'll write a summary of the changes for existing users with links to updated sections of the README.

Reverted the commit build was failing

I fixed the commit with better output and released it yesterday. I also just released the changes from the bindfs branch : https://github.com/GetStream/vg/releases/tag/v0.7.0
I'll be closing this until it happens again and more info or steps to reproduce it are available.

Thank you!

repo: GetStream/vg

license: mit


text:
can't connect to geyser connect

so i was about to use geyser connect to see a few things and i update the jar from the jenkins repository, when i try to join trough windows 10 edition, nothing, timed out and unable to connect to world, try lan on my phone, the same error, even tho the thing take a while to say "timed out" thething is that geyser sntandalone, plugins, bedrock servers and the mojang recommended servers work. also the public geyser connect work

This is likely due to the 2 port requirement, please have a go with this PR as it removes that requirement.
https://github.com/GeyserMC/GeyserConnect/pull/30

Closing due to inactivity

repo: GeyserMC/GeyserConnect


text:
Sentry SSO を何とかする

現状のSentry の Helm Chart は、SSO に利用する GitHub App の情報を Secret リソースから取得させることができない ( https://github.com/getsentry/self-hosted/issues/1501 )。
Sentry の Pod 側でこの問題を解決するためには、Secret リソースから値を読んで最終的にコンテナに食わせる設定ファイルを生成する initContainer (+その他いろいろな設定)を頑張って pod spec に仕込む必要がある。
通常の Helm install ではそんなことはできないので、Helm を直接(ArgoCDのApplication CRで)使う代わりに、 Kustomize + Helm generator を用いてマニフェストを生成し、それにパッチを当てる必要がある。

少なくともシークレットの挿入までやる


text:
basic CLI

Create a CLI using Argparse, showing the status and allowing control of individual and groups of webhooks.

[ ] Show number of pending webhooks
[ ] Show details of specific webhook
[ ] Cancel a webhook
[ ] Submit a new webhook

A help menu must be provided, and commands should follow a two-tiered structure. Examples of the desired command structure:
> webhook add message url --retryat=140000000 --attempts=0
> webhook cancel id
> webhook status
> webhook status id

Git Guild Bounty Program

Total
Dev
Sales
Reporter
Tester(s)
Maintainer
Documenter

N/A
assignee(s)
@isysd
@isysd
@isysd, @d3brouille
@isysd
@d3brouille

$833.00
$500.00
$41.65
$41.65
$166.60
$41.65
$41.65

@agnerio I started on this one already, which should make implementation a bit easier. It is in the experimental branch, and also is intertwined with a non-working daemonization attempt. Please check out the experimental branch, learn from my mistakes, and implement a better CLI based on the master.

Thank you for the briefin, @isysd. Starting to work on this issue.

authors: agnerio isysd

repo: GitGuild/webhook-sender


text:
🛑 detonics is down

In 1dca544, detonics (detonics.com) was down:

HTTP code: 403
Response time: 425 ms

Resolved: detonics is back up in 544891b after 19 minutes.


text:
🛑 xxlhd is down

In ae15982, xxlhd (xxlhd.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: xxlhd is back up in 3b8a10d after 7 hours, 42 minutes.


text:
🛑 orkut is down

In d8ad6dd, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in dade780.


text:
🛑 orkut is down

In d466d7e, orkut (orkut.com.br) was down:

HTTP code: 404
Response time: 121 ms

Resolved: orkut is back up in 5b48685.


text:
🛑 orkut is down

In 7551816, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 6e02989.


text:
🛑 orkut is down

In 0c98818, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 828fff0.


text:
🛑 orkut is down

In 06a318b, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 60d7703.


text:
🛑 ojolink is down

In 5c8a9d4, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in fd4bff1.


text:
🛑 ojolink is down

In 3d82612, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in b27cba2.


text:
🛑 ojolink is down

In 86de30e, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 14fdd18.


text:
🛑 penisdicke is down

In ca214d8, penisdicke (penisdicke.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penisdicke is back up in 60dce92.


text:
🛑 youku is down

In 0186db4, youku (youku.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: youku is back up in a287549.


text:
🛑 penisdicke is down

In 308cee2, penisdicke (penisdicke.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penisdicke is back up in e2ee42f.


text:
🛑 ojolink is down

In d5756d1, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 515 ms

Resolved: Ojolink is back up in 4e31bbb.


text:
🛑 ojolink is down

In 059d46a, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 509 ms

Resolved: Ojolink is back up in 1370745.


text:
🛑 orkut is down

In dc3b7a2, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 735 ms

Resolved: orkut is back up in 095443f.


text:
🛑 ojolink is down

In e0efb17, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 370 ms

Resolved: Ojolink is back up in eb979cb.


text:
🛑 orkut is down

In 37f36f8, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 1161 ms

Resolved: orkut is back up in a9b1bfe.


text:
🛑 ojolink is down

In cdcd57e, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 446 ms

Resolved: Ojolink is back up in 156eecb.


text:
🛑 orkut is down

In 533b1e2, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 1097 ms

Resolved: orkut is back up in 71311fe.


text:
🛑 ojolink is down

In a0cfd8d, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 473 ms

Resolved: Ojolink is back up in 01a0e0d.


text:
🛑 orkut is down

In a572429, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 682 ms

Resolved: orkut is back up in 44abfd9.


text:
🛑 ojolink is down

In 9f33d38, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 3210d22.


text:
🛑 ojolink is down

In 1b53555, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 25024ad.


text:
🛑 ojolink is down

In 492c84f, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 7bc798d.


text:
🛑 ojolink is down

In 7363ae7, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 1312e23.


text:
🛑 penisdicke is down

In 42981cc, penisdicke (penisdicke.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penisdicke is back up in ccec7c8 after 831 days, 4 hours, 48 minutes.


text:
🛑 ojolink is down

In 0a397e9, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 0d15aa8 after 8 minutes.


text:
🛑 orkut is down

In 282b742, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 3d2770d.


text:
🛑 ojolink is down

In c8c038d, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 8d2bb0c after 8 minutes.


text:
🛑 ojolink is down

In 9d11741, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 56c519f after 9 minutes.


text:
🛑 orkut is down

In 8a21575, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 76ff3ff.


text:
🛑 carlinha is down

In 1f243dd, carlinha (carlinha.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: carlinha is back up in 0ebbed2 after 18 minutes.


text:
🛑 uzboys is down

In c124f23, uzboys (uzboys.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: uzboys is back up in 13313b7 after 16 minutes.


text:
🛑 ojolink is down

In 7786d82, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 631ea2f after 12 minutes.


text:
🛑 ojolink is down

In 57f6fbf, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in b38f2a9 after 8 minutes.


text:
🛑 gadgethouse is down

In 2645a11, gadgethouse (gadgethouse.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: gadgethouse is back up in 08ecd05 after 6 hours, 10 minutes.


text:
🛑 ojolink is down

In b66e4e6, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in f2bb772 after 8 minutes.


text:
🛑 ojolink is down

In da368d9, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in e139d84 after 23 minutes.


text:
🛑 orkut is down

In 7faee6d, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 407167d after 8 minutes.


text:
🛑 orkut is down

In fc6c5c8, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 81ab41e.


text:
🛑 uzboys is down

In 9d8607f, uzboys (uzboys.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: uzboys is back up in 89a565f after 33 minutes.


text:
🛑 orkut is down

In bffcd30, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 633804e after 10 minutes.


text:
🛑 orkut is down

In 5952fe6, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 7c0f28b after 5 minutes.


text:
refactor: move catalog into a datafusion extension

So this ended up being a bit trickier than the session vars.
The context is used within async/await a lot more than the vars were & caused the parking_lot::RwLock to not work. I had to swap it out with the tokio::sync::RwLock which unfortunately causes all read/write actions to be async. This caused qquite a ripple effect as all dependent function calls in the stack also had to be converted to async/await.
I also added a small example exec CreateTableExec, it's not fully wired up yet, but it does show that we can mutably access the catalog & session vars from the exec now!
you can test it out by swapping out the planner in sqlexec/src/session.rs:create_physical_plan for the extension aware planner.
There's a few small todo's & unwraps left in here, but I think the bulk of the work is ready for review.

I made a change that splits out the metastore client from the session catalog and puts it in a struct called CatalogMutator. We can stick that on the datafusion context.

closed in favor of #1607

repo: GlareDB/glaredb


text:
[Bug]: Horizontal panning not working

Summary
With ResumableZoom and example code from the module docs I can pinch zoom and pan vertically, but horizontal panning is not working: it either does not pan at all (example code) or with my own images and testing different props I could get some horizontal panning, but not really working properly and also I got sometimes weird effects where Image started to flip between centered image and horizontally panned image (have videos of those if needed).
Happy to help in debugging to identify the root cause of the problem.
Environment
Expo SDK 51, expo router v3
react-native-gesture-handler 2.16.2.
react-native-reanimated 3.10.1
react-native-zoom-toolkit 3.1.0
Expo dev build, dev-mode, tested on iPhone 14 with iOS 17.6.1
Steps to Reproduce

Copy-paste ResumableZoom example from module documentation as-is, put it into a file which is defined as a Stack.Screen for Expo Router v3
Navigate to said screen and try to pinch zoom and pan the image

Your issue sounds quite strange to me, please share me the videos you've taken.

@MaRaSu Nice catch, I just fucked up the example in the docs real bad. I forgot if you have the {flex: 1, justifyContent: "center", alignItem: "center"} in the parent component it becomes unmesurable... So you just need to remove justifyContent and alignItems from the View enclosing ResumableZoom.

@Glazzes thanks for quick replies! Removing those style props was an instant fix, now zooming works very nicely indeed.

authors: Glazzes MaRaSu

repo: Glazzes/react-native-zoom-toolkit


text:
Sneak attack damage from the dagger is unreliable.

While troubleshooting an issue with a health bar + damage indicator mod I did some data gathering.  After drinking a potion of invisibility I whacked a 220HP skeleton with a wooden dagger until it died.  The negative numbers in the following list are damage reports.  I believe the -5 numbers are successful sneak attacks which properly did 4.5 damage (the damage indicator erroneously rounded up).  The -3 numbers seem to be sneak attack failures that actually dealt 3 damage.

220
(-5) 216
(-5) 211
(-5) 207
(-3) 204
(-5) 199
(-3) 196
(-3) 193
(-5) 189
(-3) 186
(-5) 181
(-5) 177
(-3) 174
(-5) 169
(-3) 166
(-3) 163
(-5) 159
(-5) 154
(-3) 151
(-3) 148
(-5) 144
(-3) 141
(-3) 138
(-3) 135
(-3) 132
(-5) 127
(-5) 123
(-3) 120
(-3) 117
(-5) 112
(-3) 109
(-5) 105
(-3) 102
(-5) 97
(-5) 93
(-3) 90
(-5) 85
(-5) 81
(-3) 78
(-3) 75
(-3) 72
(-3) 69
(-5) 64
(-5) 60
(-3) 57
(-5) 52
(-3) 49
(-3) 46
(-5) 42
(-5) 37
(-5) 33
(-3) 30
(-5) 25
(-5) 21
(-5) 16
(-3) 13
(-5) 9
(-5) 4
(-4) Dead
If you sum the damage reports you get 235, which exceeds the skeleton's HP by 15.  There were exactly 30 reports of (-5), so the accumulated rounding error accounts for this excess.  That's why I believe the (-3) damage attacks were failures to sneak attack.  There were 27 of those.  The killing blow appears to be a sneak attack, so it's roughly a 53% success rate in this trial.  I did not sneak at all; I only used an invis potion.

I'm not sure if this is intended or not, but I thought I should let you know since I had all the data anyway =P

Thanks for the report, already fixed this bug a few days ago ;) next update it should work fine

Fixed it in the newest update


text:
Godot 4 Beta 14 - XR Tools Demo - Latest Plugin

Hello,
I wanted to test Godot 4 with VR on the Quest 2. I tried to export the APK and got following issue, as soon i enable Godot Open XR Meta:

Task :preBuild UP-TO-DATE
Task :preDebugBuild UP-TO-DATE
Task :mergeDebugNativeDebugMetadata NO-SOURCE
Task :compileDebugAidl NO-SOURCE
Task :compileDebugRenderscript NO-SOURCE
Task :generateDebugBuildConfig UP-TO-DATE
Task :checkDebugAarMetadata UP-TO-DATE
Task :generateDebugResValues UP-TO-DATE
Task :generateDebugResources UP-TO-DATE
Task :mergeDebugResources UP-TO-DATE
Task :mapDebugSourceSetPaths
Task :createDebugCompatibleScreenManifests UP-TO-DATE
Task :extractDeepLinksDebug UP-TO-DATE

Task :processDebugMainManifest FAILED
C:\Users\stefa\Downloads\godot-xr-tools-demo-4.0\godot-xr-tools\android\build\src\debug\AndroidManifest.xml:55:13-32 Error:
Attribute meta-data#com.oculus.supportedDevices@value value=(all) from AndroidManifest.xml:55:13-32
is also present at [godotopenxrmeta-debug.aar] AndroidManifest.xml:20:13-50 value=(quest|quest2|questpro).
Suggestion: add 'tools:replace="android:value"' to  element at AndroidManifest.xml:53:9-55:35 to override.

See http://g.co/androidstudio/manifest-merger for more information about the manifest merger.
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':processDebugMainManifest'.

Manifest merger failed : Attribute meta-data#com.oculus.supportedDevices@value value=(all) from AndroidManifest.xml:55:13-32
is also present at [godotopenxrmeta-debug.aar] AndroidManifest.xml:20:13-50 value=(quest|quest2|questpro).
Suggestion: add 'tools:replace="android:value"' to  element at AndroidManifest.xml:53:9-55:35 to override.

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 1s
8 actionable tasks: 2 executed, 6 up-to-date
Exit Code: 1
Any Idea to fix that?

There's a bug for this in the godot_openxr_loaders. For now I would jump back to the previous build of the quest loader.
https://github.com/GodotVR/godot_openxr_loaders/issues/19

Thank you for the quick reply! It solved my problem!

Should all be resolved now with the latest RC and loaders plugin. There were a bunch of manifest changes we had to apply.

authors: BastiaanOlij Malcolmnixon sprobst76

repo: GodotVR/godot-xr-tools


text:
Port to godot 4.0 - first steps

Taking our first steps to porting this plugin to Godot 4 based on the work done in the godot-xr-reference implementation and the godot_openvr plugin.
Currently relies on:

https://github.com/godotengine/godot/pull/51179

To try out the work so far:

build Godot with the above mentioned PR
run godot ----dump-extension-api
copy the generated extension_api.json into thirdparty/godot-cpp/godot-headers
build thirdparty/godot-cpp using release=debug
build the plugin using release=debug

@BastiaanOlij Looking at some of the changes, it looks like we're gonna need a separate demo for Godot 4.x since that won't be compatible with Godot 3.x.
Also similar for godot-cpp; we're going to need a Godot 3.x godot-cpp dependency, and a Godot 4.x godot-cpp dependency.

We'll manage it with branches. The source is going to drastically change between Godot 3 and Godot 4, there is no way we can keep them in one branch.
So plugin versions 1.x.y is for Godot 3, and plugin versions 2.x.y for Godot 4.
Right now master is still 1.x.y and we have the 4.0-dev branch for the new implementation, but eventually we'll maintain 1.x-dev branch for Godot 3 and master becomes Godot 4.

While this is not working and far from finished, thinking about merging the work so far into the 4.0-dev branch so @ChristophHaag and I can work in parallel and issue PRs.
There willl be days where we'll need to rebase 4.0-dev on master which will be a PITA but it already is a PITA as it stands right now so I think the benefits outweigh the negatives.
@m4gr3d what do you think?

@BastiaanOlij I agree with merging into the dev branch. It should improve on the current state of development and enable additional eyes on the code.

I've got a few more things to push in later today and than I will merge this. Its far from working but it means people can start fixing things.
I'm focusing on rewriting the action map stuff atm so thats all half done but mostly disabled. I'm running in a bunch of roadblocks because extensions is just not finished enough, but groud and bruvzg are helping out now so we should be able to push this forward.


text:
Download specific episode

Wondering if there is a way to download a specific episode using the episode URL. Please close if not possible.

No as far as I know it's not possible, but I will not close it for a simple reason, that an option that should be there!

So I found this project that encountered the same issue with HSL, but before that issue, it accepted a Crunchyroll URL and would download that episode. Might be worth taking a look at: https://github.com/einstein95/crunchy-xml-decoder

Thanks for that project, I wil take a look, I think there are a few interesting part, especially for the subtitles which is one other big issue for me with current version of crunchy

On the Deathspike version there were some comments about fixing the episode filter by changing line 96 of the series.js file from:
episode: parseInt(episode[0], 10),
to:
episode: parseInt(episode[1], 10),
Then you would be able to actually select a particular episode by adding -e <some number> to the command like it was intended. I've noticed that it's off by one. So if you want to download episode 21 you need to say -e 20.
I would definitely push for the URL option, though. Using the URL of the video directly would be a lot cleaner in my mind. Some Anime have specials/movies so how are they numbered (see One Piece for example)? There are also a number of episodes on Crunchyroll that have episode numbers like 7.5 or 13.5. How would those be numbered and would they offset the next episodes? For example would Episodes 7, 7.5, and 8 be indexed like 7, 8, 9?
I definitely like the batch downloading functionality, but the URL option would be a lot cleaner.

That's a good idea to clearly add support for full episode URL.
For 7.5, 10.5 etc. My (future) change take care of them. It does not support all of the extra episodes, but most of them
In fact I'm thinking about a major change in how episodes are listed and saved for batch. Will see lots of things are still in progress.

authors: Godzil Jfigs101 PrestonCook


text:
Update documentation now that user study is over

Hi Roberto,
could you please update readme.md now that user study is over? The user study mode / spaceship ranker references are now obsolete.

Update readme and documentation in fcb8727dc2edc549034bca23fb9af3ef20c42c88

Thanks Roberto. I noticed the readme does not show how to enter "developer mode" - is it OK for you to include this information there?

From: gallorob @.***>
Sent: Thursday, December 15, 2022 10:57:55 AM
To: GoodAI/space-engineers-ai-spaceship-generator
Cc: Martin Poliak; Author
Subject: Re: [GoodAI/space-engineers-ai-spaceship-generator] Update documentation now that user study is over (Issue #20)
Update readme and documentation in fcb8727https://github.com/GoodAI/space-engineers-ai-spaceship-generator/commit/fcb8727dc2edc549034bca23fb9af3ef20c42c88
—
Reply to this email directly, view it on GitHubhttps://github.com/GoodAI/space-engineers-ai-spaceship-generator/issues/20#issuecomment-1352816586, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADMBABKGFBBJOPBFHPNWIYLWNLTSHANCNFSM6AAAAAAS6V72BI.
You are receiving this because you authored the thread.Message ID: @.***>

Good point, I've updated the readme

authors: gallorob martinpoliak-goodai


text:
Low definition lines in WebGL

Hi, I'm using pixi to draw shapes with graphics.
If I use CanvasRenderer this look good. But if I try to use WebGLRenderer, the lines are pixelated.
Any ideas?
http://jsfiddle.net/pbt67sv8/3/

Having the same issue, Using graphics in webgl looks much worse than canvas.

The stencil buffer has no AA support. The best we can do is FXAA afterwards, but it will not look as good as canvas which is AA.
If your shape doesn't change often you can generate a texture from it, and drawing that instead may look better.

Closing this since it seems answered.

authors: doskallemaskin englercj fbrussa


text:
Improved performance of particle container for small amounts

The particle container always upload a full batch size (15000 particles). When the number of particles is smaller it becomes a huge performance drawback in terms of CPU.
In my use case: maximum of roughly 500 sprites, bufferSubData accounts for 25% of my whole program cpu usage. After this optimisation it accounts only for 5%.
This is far from negligible when developing for smartphones where the maximum number of particles will max at 1000.

Ohhh so that's why is was slower than Container. Good job !

This looks awesome, but according to #1926 it can cause GC stalls. We need to figure out of subarray is the cause of this before merging.

We could create an array that 'grows' based on how much is required (it its too small we double the size)

The problem is that, if the buffer does not get downsized, someone who empties his particle container might wonder why he still gets poor performance. Would it be acceptable in terms of performance to make the vertex buffer shrink as well whenever the number of particles gets too small?

IMHO the sensible way to use a particle system is to create a fixed amount of particles that you then switch on/off as you need them. In other words, you allocate how ever many you need for your buffers, and take the hit to resize them if on the off chance you need to add/remove some permanently in that frame.
So what I reckon is the buffer should always be exactly the size needed to render the current amount of sprites/particles.

Well I would like this container to be as flexible as possible. As a user I do not want to think about the number of particles I might need nor worry about the performance drawback if I do not use the full buffer.
In my use case I have a particle container to display from 50 to 500 particles (could be more, there is no actual upper bound). But in average it is around 100. If I have to decide beforehand the container size, I would put something like 700 to be safe, but then I get performance 7 times slower than what I need in average.

What if we let the user specify an initial size (optionally), we grow the array dynamically (doubling when full), and we shrink dynamically (halfing when < 1/2 used).
We have a sensible default for the common case, but the initial size specification allows you to tweak it to be just right for your application.

That seems good to me.

I like the initial size specification.
For our use case, as long as the subarray isn't done every frame no matter what the conditions are, we're happy ;)

I created a new pull request that solves the problem differently: https://github.com/GoodBoyDigital/pixi.js/pull/1990/files
If the new solution is fine I suggest we close this PR.

Closing this in favor of the other PRs we have improving ParticleContainer.

authors: GoodBoyDigital JiDW bchevalier davedx englercj


text:
QR code scanner does not work.

Description
QR code scanner does not work.
Preconditions

The user is logged in
The “Receive G$” page is opened

Steps to reproduce

Click the “Scan QR code” button on the top of the page
Try to use valid QR code
Pay attention to the page

Actual result: The QR code scanner does not display the image from the camera on the "SCAN QR CODE" tab after clicking on the "SCAN QR CODE" button.
Expected result: You are redirected to the web wallet page on the "SCAN QR CODE" tab after clicking on the "SCAN QR CODE" button.
Supporting data
Env https://goodqa.netlify.com
Chrome/ Samsung Galaxy S8
https://www.screencast.com/t/7qnj7wjCtIe

@qatests1001 needs more information.
not clear exactly what is the issue. is the issue that the video stream is not shown? to me it is shown.

explain better the problem
show a video
include the console output in the video

Tested
The bug is not reproduced. The QR code scanner is working okay - https://www.screencast.com/t/vJyswFz4

repo: GoodDollar/GoodDAPP


text:
send keypress directly to tf2 application

send keypress directly to tf2 application so you can alt+tab without pausing the bot kicker

Initially I thought this would not be possible while keeping compatibility across platforms, however I am investigating a way to do this using RCon which would remove the need to send commands via keypress at all. This may be in a future release.

Implemented, will be in next release.


text:
Adding readiness_checks

Added readiness checks which allow for much better control of the status of the different containers (load balancer will not send new requests to busy instances)
https://cloud.google.com/appengine/docs/flexible/java/configuring-your-app-with-app-yaml#updated_health_checks

Good PR, thank you @cedricbellet, I will run few tests before approving it.

Apparently the path argument does not work as described in the documentation. I deployed to my own projects and the paths for health checks was still the default liveness_check and readiness_check endpoints. I had to manually override the paths. That being said, moving to updated health checks is still the right thing to do.

Disregard previous comment - although the path shown in the UI does not match the YAML file, it seems to be working. Manually overwriting the paths messed everything up :) I think the commit works well as is (doing some more experiments)

authors: cedricbellet juliantoledo

repo: GoogleChrome/gulliver


text:
Support new headless in nodeJS runner

Currently, we NodeJS mapper runner supports only support old headless.

--HEADLESS=[false, new, old]

authors: sadym-chromium thiagowfx

repo: GoogleChromeLabs/chromium-bidi


text:
Fix Web Bundles's url

Following url is file not found. This change fixes the url.
https://wicg.github.io/webpackage/draft-yasskin-wpack-bundled-exchanges.html

Good find, thank you! 💯

repo: GoogleChromeLabs/webbundle-plugins


text:
Trouble when trying to correctly build and run a basic Svelte javascript project

Describe the bug
I'm trying to use the GCP buildpacks to build a basic, getting started Svelt app, and I"m running into an issue on the build step
Additional context
How are you using GCP buildpacks?

[x] pack and the gcr.io/buildpacks/builder

Did this used to work?
Nope, probably never worked
What language is your project primarily written in?
Nodejs, Svelte framework
Steps To Reproduce

Run the commands to start a basic project here https://kit.svelte.dev/docs/creating-a-project
run pack build my-app --builder gcr.io/buildpacks/builder:google-22
run docker run my-app

Actual Behavior
The app correctly builds into an OCI image locally. However, when I try to get the image up and running I get the following error:
node:internal/modules/cjs/loader:1147
  throw err;
  ^

Error: Cannot find module '/workspace/index.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
    at Module._load (node:internal/modules/cjs/loader:985:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
    at node:internal/main/run_main_module:28:49 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Actual behavior
It should just work

I don't think it should "just work". I reprod and looking at the package.json, it is missing a start command.
"scripts": {
		"dev": "vite dev",
		"build": "vite build",
		"preview": "vite preview",
		"check": "svelte-kit sync && svelte-check --tsconfig ./jsconfig.json",
		"check:watch": "svelte-kit sync && svelte-check --tsconfig ./jsconfig.json --watch"
	},

Since it is missing a start command, our buildpacks configures the entrypoint because npm run dev is not suitable for a production environment.

@lukasberger we're out of our depth here on Svelte-kit, what's a good way to stand up a server to run a Svelte app? Just doing some digging, it seems like Svelte uses adaptors https://kit.svelte.dev/docs/adapter-node

I agree that we shouldn't use "dev". I'm not familiar enough with Sveltekit to know, but I think "preview" might be the recommended way?
Can we hold off on closing this issue for the moment while we look into it?

Yeah, for sure we'll leave this issue open until we find a reasonable way to get Sveltkit up and running

After more exploration, I now agree with @kennethye1. Working with SvelteKit's node-adapter docs, this was the shortest path I could find to deploying a SvelteKit application out of the box.

npm create svelte@latest sveltekit-app
cd sveltekit-app
npm i -D @sveltejs/adapter-node
npm uninstall -D @sveltejs/adapter-auto
npm pkg set scripts.start="node build"
Change auto to node in svelte.config.js
gcloud run deploy

I think we can likely close this for now and consider reopening if this ever gets easier.

TL;DR
The SvelteKit team seems open to a change that would allow us to deploy SvelteKit applications out of the box if Cloud Buildpacks can:

Detect a SvelteKit application
Add a unique environment variable to the Cloud Build environment before the standard npm run build step

The much longer story
The SvelteKit team has shown an openness to accepting an adapter that would make Cloud Run work if we give them the name of an environment variable that they can detect: https://github.com/sveltejs/kit/issues/11951
SvelteKit has the concept of using different adapters that generate different outputs for different target environments. You can select your adapter (like adapter-node for Cloud Run), but SvelteKit also has the concept of a zero-config adapter-auto which is the default for new projects. The automatic adapter selects which adapter to use based on the environment variable present. So for default projects, the steps are:

adapter-auto runs through this code to check for the existence of environment variables like VERCEL or a logical check like process.env.GITHUB_ACTION_REPOSITORY === 'Azure/static-web-apps-deploy'
If it finds one that matches, it will pull down that npm package (like @sveltejs/adapter-vercel for Vercel). This is where our deployment currently fails. Our deployment succeeds if you configure the adapter for node servers (adapter-node).
Builds the output as described by the adapter

The adapter package is a dev dependency and is only required during the build (not during the runtime). So we could essentially add a "Step 0" to this list that detects if this is a SvelteKit application: if it contains svelte.config.js. Additional confirmation to use the node adapter could be if @sveltejs/adapter-auto is a dev dependency in package.json or if the string @sveltejs/adapter-auto exists in svelte.config.js. Then add the environment variable during the build. The variable could be something like USE_SVELTEKIT_NODE_ADAPTER.
If we got this working, that would take the number of steps required to deploy a SvelteKit application from seven down to four (or even three)!
Current steps:

npm create svelte@latest sveltekit-app
cd sveltekit-app
npm i -D @sveltejs/adapter-node
npm uninstall -D @sveltejs/adapter-auto
Change auto to node in svelte.config.js
npm pkg set scripts.start="node build"
gcloud run deploy --allow-unauthenticated

Possible future steps:

npm create svelte@latest sveltekit-app
cd sveltekit-app
npm pkg set scripts.start="node build" (might not be necessary if we can add this to the start script)
gcloud run deploy --allow-unauthenticated

@jinglundong - The SvelteKit maintainers would prefer that our environment variable stay specific to Cloud Run (Link to conversation).

Detect a SvelteKit Application

Contains svelte.config.js file
Check package.json for confirmation:

@sveltejs/adapter-auto is a dependency
npm start is not defined

Add an environment variable unique to Google Cloud Buildpacks

I suggest GOOGLE_CLOUD_BUILDPACKS, but it could be anything the team prefers
I can make the pull request to have this bring in the @sveltejs/adapter-node once we confirm the environment variable name

Run npm run build (Shouldn't require any changes since this is standard for Node.js projects)
Set the start command to node build

We prefer using 'adapter-auto' so that the same build gets pushed to Cloudflare, Vercel and now to CloudRun (CloudRun is the only LLM compatible solution as Vercel timeouts after 60 seconds, and Cloudflare doesn't let and reasonable amount of CPU)
To make adapter-auto pick adapter-node we just set environment variable:
ENV GCP_BUILDPACKS="dummy-value"
so that this code picks node correctly.
Full Dockerfile:
FROM node:20

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm install

COPY . .

ENV GCP_BUILDPACKS="dummy-value"
RUN npm run prepare
RUN npm run build

EXPOSE 3000

CMD ["node", "build"]

We also made secret management work by storing a entire ".env" file as a single secret.

@ljoukov - That's a very interesting use case and a clever solution. We intentionally scoped this variable very narrowly to avoid impacting anyone who didn't want this behavior. It's interesting that it seems like this would be a "desired" side effect for GCP_BUILDPACKS to be present everywhere. Thanks for sharing.

repo: GoogleCloudPlatform/buildpacks


text:
logging.Verbosef statements are dropped when structuring logging is enabled

When structured logging is enabled, logging.Verbosef is set to Debugf. However, the zap production config disables anything below Info.
The fix needs to allow users to opt-out of verbose logging (see here.

Likely blocked by whatever fix is made for https://github.com/GoogleCloudPlatform/cloudsql-proxy/issues/725.

repo: GoogleCloudPlatform/cloudsql-proxy


text:
Add sqlserver workflows

The workflows for the Guest OS pipeline migration.
Internal: 205740807, 205740122

/assign @bkatyl

/unassign @bkatyl
/cc @bkatyl

Closing PR for now. Will send a new one with just one image's workflow.

repo: GoogleCloudPlatform/compute-image-tools


text:
Add support for chunked transfer encoding

Write a big file to GCP storage raise exception.
/usr/local/lib/python2.7/dist-packages/gcp/storage/_item.pyc in write_to(self, content, content_type)
    183       self._api.object_upload(self._bucket, self._key, content, content_type)
    184     except Exception as e:
--> 185       raise e
    186
    187

OverflowError: string longer than 2147483647 bytes

Any workaround for storage.write_to() of large file (more than 2GB)?

Thanks for reporting this. It looks like the issue comes from the requests library, and the fix is to use chunked encoding. I'll look into what we need to change the code appropriately.

Thanks for the answer and let's know when it is fixed a lot of our data are bigger than 2 GB.

It looks like httplib2 doesn't support chunked encoding; we may have to switch to requests, which could be an interesting challenge with oauth2client. It would be a good change though so definitely worth some investigation.

Potentially the thing to do is using streaming uploads... https://cloud.google.com/storage/docs/streaming

Same thing, really; streaming uploads use chunked encoding. But yes, using gsutil directly in a %bash cell is a good workaround until we do it directly.

This is non-trivial; oauth2client does not support requests library, only httplib2, and httplib2 does not support chunked transfer encoding.

Do you have a similar issue with large files (> 2GB) on other operations (gs.read_from(), bq.results(), etc.)?

Read/pull operations provide paginated results with a continuation token so
shouldn't have this issue.
On Tue, May 10, 2016 at 6:01 PM, Velizar VESSELINOV <
notifications@github.com> wrote:

Do you have a similar issue with large files (> 2GB) on other operations
(gs.read_from(), bq.results(), etc.)?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/datalab/issues/784#issuecomment-218335403

authors: VelizarVESSELINOV gramster nikhilk

repo: GoogleCloudPlatform/datalab


text:
FireStore - Collection group queries Support

Collection group queries support is missing as it is available in other libraries
https://firebase.google.com/docs/firestore/query-data/queries#collection-group-query

https://github.com/googleapis/google-cloud-dotnet/issues/4907
this says it is already available for .net

As you can see from that issue, the feature is already supported - use FirestoreDb.CollectionGroup. Yes, the sample is still missing, but there's an internal issue for that already - we don't need an issue in this repository.
(I've been writing Functions Framework samples recently, so I should be able to write this Firestore sample next week.)

Reopening as it is an issue for this repo - that's the downside of trying to handle issues on a mobile over the weekend.
It turns out that when I added samples for the previous issue, I missed the last one. Hoping to get it done today.

The sample is now merged, which is why this issue is fixed - but it won't show up in the public docs until we've made an internal change as well. So @mitulpanch please refer to the linked PR for now.

The documentation is now published: https://firebase.google.com/docs/firestore/query-data/queries#c_12

authors: jskeet mitulpanch

repo: GoogleCloudPlatform/dotnet-docs-samples


text:
Update gsuite-group-collection.md

Thanks for opening a Pull Request!
Here's a handy checklist to ensure your PR goes smoothly.

[x] I signed Google's Contributor License Agreement
[x] My code conforms to Google's python style.
[x] My PR at a minimum doesn't decrease unit-test coverage (if applicable).
[x] All of the unit-tests still pass
[x] Running pylint --rcfile=pylintrc passes.

These guidelines and more can be found in our contributing guidelines.

Thanks. I'll hold on including a screenshot until we can decide on screenshots in general with @Tabrarian

Adding cla: yes label, as the PR author is a googler at the time of this PR was created.

authors: blueandgold mirons-google

repo: GoogleCloudPlatform/forseti-security


text:
time to restore service badge shows null

When instrumenting a new repository, the Time to Restore service badge shows null:

We've launched a new dashboard in Grafana! Check it out! https://github.com/GoogleCloudPlatform/fourkeys/tree/main/dashboard

authors: dinagraves wpannell

repo: GoogleCloudPlatform/fourkeys


text:
Project and namespace rename to use Google.Storage.V1

This will be consistent with other API code moving forward. More changes
to be as consistent as possible will be coming.

LGTM

authors: chrisdunelm jskeet

repo: GoogleCloudPlatform/gcloud-dotnet


text:
Create sink/metric with simple name, rather than its full path

Fixes https://github.com/GoogleCloudPlatform/gcloud-python/issues/1890

@bjwatson Thanks for the detective work!

@tseaver No problem.

authors: bjwatson tseaver

repo: GoogleCloudPlatform/gcloud-python


text:
feat: Adding RAG_FAQ.MD to retrieval-augmented-generation

Description
Thank you for opening a Pull Request!
Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[x] Follow the CONTRIBUTING Guide.
[ ] You are listed as the author in your notebook or README file.

[ ] Your account is listed in CODEOWNERS for the file(s).

[ ] Make your Pull Request title in the https://www.conventionalcommits.org/ specification.
[ ] Ensure the tests and linter pass (Run nox -s format from the repository root to format).
[ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

I added the spell checker, fix the identified errors

Closing this PR due to inactivity. Please re-open if changes are made.

authors: holtskinner sermolin

repo: GoogleCloudPlatform/generative-ai


text:
Increase minimum Go version requirement

The Go project provides security support for the latest two major release versions, so increment the workspace version and build toolchain to match.

Thanks for reviewing & merging @rosmo!


text:
Google.Cloud.Dialogflow.V2 The WebhookRequest from Dialogflow has added a new field alternativeQueryResults

As specified in the official Actions on Google documentation, they have added a new field called alternativeQueryResults.
https://cloud.google.com/dialogflow-enterprise/docs/reference/rest/v2beta1/DetectIntentResponse
If you have active the "Enable beta features and APIs" section on the Dialogflow panel, the request will contain this field and the WebhookRequest.Parser.ParseJson will fail giving the following error:
2018-08-10T18:32:17.808 [Info,Binding] Function started (Id=cc525b02-5bd6-4201-a35d-8ab4fe3542e5)
2018-08-10T18:32:17.808 [Info,Binding] Executing 'Binding' (Reason='This function was programmatically called via the host APIs.', Id=cc525b02-5bd6-4201-a35d-8ab4fe3542e5)
2018-08-10T18:32:17.808 [Error] A ScriptHost error has occurred
2018-08-10T18:32:17.808 [Error] Unknown field: alternativeQueryResults
2018-08-10T18:32:17.841 [Error,Binding] Exception while executing function: Binding 2018-08-10T18:32:17.841 [Error,Binding] Unknown field: alternativeQueryResults
2018-08-10T18:32:17.841 [Error,Binding] Function completed (Failure, Id=cc525b02-5bd6-4201-a35d-8ab4fe3542e5, Duration=30ms)

You can - and probably should - use a parser which ignores unknown fields. I'm not in a position to provide an example right now, but I'll do so when I get a chance.

(But you won't be able to retrieve that value until we build a v2beta1 library.)

I tried:
WebhookRequest webhookRequest = new JsonParser(new JsonParser.Settings(100)).Parse(json);
var webhookRequest = WebhookRequest.Parser.ParseJson(json);
And both fail to parse the request.

Yes, you'd need to create a settings object which explicitly permits unknown fields. Your first attempt is nearly there - I think there's an "WithAllowUnknownFields" or similar method on the JsonParser.Settings. Again, will check when I'm in a suitable position.

Yeah that's it.
WebhookRequest webhookRequest = new JsonParser(new JsonParser.Settings(100).WithIgnoreUnknownFields(true)).Parse<WebhookRequest>(json);

repo: GoogleCloudPlatform/google-cloud-dotnet


text:
Integration Tests with Emulators

[Testcontainers project[(https://github.com/testcontainers/testcontainers-java) allows to use services running in containers to be used during testing. It will be great to add such tests for available emulators using google/cloud-sdk container.
I have a proof of concept here that shows how to run such tests in Travis.
If this idea look good to everyone I can try to add some simple tests after https://github.com/GoogleCloudPlatform/google-cloud-java/issues/1973 is fixed.

I don't think google-cloud-java contains any source code of gcloud emulators.

Right, @shinfan. But wouldn't it be great to verify google-cloud-java works properly with emulators to catch problems like #1973?

Oh I sorry I misunderstood. I think it makes sense to make sure gcloud work with emulators.

It's not clear what this request is for currently. Our current stance is for users of google-cloud-java to start emulators themselves, and then customize the API clients in google-cloud-java to use the running emulator instead of the real service. What else is needed?

The problem I've seen before that google-cloud-java stopped working with emulators. It will be nice to make sure changes to google-cloud-java are not breaking it ability to talk to emulators.

It might be fine to not have them since google-cloud-java is getting more and more stable. 🤷♂

We intend on continuing to support the special functionality necessary for emulators (plain text, no credentials). With that, I don't think emulators would generally break. It's a challenge to verify emulators in google-cloud-java because that requires extra dependencies that are problematic (either the python-based gcloud utility, or downloading the emulator from the internet). I think I'll close this out - feel free to open a new issue if emulators break in the future.

authors: fkorotkov garrettjonesgoogle shinfan

repo: GoogleCloudPlatform/google-cloud-java


text:
storage: tests: test for list instead of update permission

Fixes #2224

Changes Unknown when pulling b734c002e4b48a01923ca05e212c0d3221c9ac15 on stephenplusplus:spp--2224 into ** on GoogleCloudPlatform:master**.

authors: coveralls stephenplusplus

repo: GoogleCloudPlatform/google-cloud-node


text:
Vision 1.1 API Client

Final merge of the Vision 1.1 API client.
Needs CI to complete (and possibly the 0.15.2 GAPIC to propagate).

/cc @jonparrott for samples.

Fixing the lint issue.

I'm not sure what's going on with Travis-pr. I cleared the caches for it I think. Once the Travis-push turns green I'll delete the other caches and try Travis-pr again.

Deleted caches and restarted travis.

@lukesneeringer I think you force pushed and removed my commits before.

Yeah, I did. Sorry, I thought I was plowing over one of mine.

I am waiting on the last Travis run to finally pass (even though it is running twice sequentially due to the way I pushed).

Finally done. (And docs even build!)

@gguuss are you taking on samples for this?

@jonparrott correct, the samples are ready.

Am I correct to assume this means docs are updated as well?

authors: daspecster gguuss jonparrott lukesneeringer

repo: GoogleCloudPlatform/google-cloud-python


text:
kubernetes on vagrant boot up error message

==> master:     True
==> master: kubernetes-master:
==> master:     Data failed to compile:
==> master: ----------
==> master:     The function "state.highstate" is running as PID 1455 and was started at 2015, Mar 20 05:47:51.213713 with jid 20150320054751213713

I also occur it. @netroby  wuhan china

Can you give list of steps you followed?
$ make release
$ export KUBERNETES_PROVIDER=vagrant
$ cluster/kube-up.sh
??
Can you vagrant ash master and look in journalctl -ru salt-master and see if there are any errors?

@derekwaynecarr yes, absolutely as the getting started guide told me.

@netroby can you let us know if you fixed this, if yes, how ?

Not fix it . just gave up trying. so close it .
maybe after a while ,somebody may fixes it .

If you still have the issue, then leave it open.
Can you ssh into the master and append any error output you see when doing journalctl -ru salt-master.

I see the same message, but it does not prevent provisioning of the cluster.  Will look into why salt highstate is happening twice.

I just tried to reproduce from HEAD this afternoon and was no longer able to reproduce.
Closing issue, but feel free to re-open if you encounter it again.

I'm getting the same error at when calling ./cluster/kube-up.sh:
....
==> master: ‘/srv/salt-new/reactor’ -> ‘/srv/reactor’
==> master: /home/vagrant
==> master: Executing configuration
==> master: kubernetes-master:
==> master:     True
==> master: kubernetes-master:
==> master:     Data failed to compile:
==> master: ----------
==> master:     The function "state.highstate" is running as PID 2629 and was started at 2015, Aug 14 01:05:35.850225 with jid 20150814010535850225

The master is up, but the minion-1 is not. There's no log in master:
$ journalctl -ru salt-master
Hint: You are currently not seeing messages from other users and the system.
      Users in the groups 'adm', 'systemd-journal', 'wheel' can see all messages.
      Pass -q to turn off this notice.
-- Logs begin at Tue 2015-02-24 15:28:11 UTC, end at Fri 2015-08-14 01:12:11 UTC. --

@netroby @ntquyen
I met the same issue today, I deleted old vms in VirtualBox and used "vagrant destroy" to clean up the cluster, and then "./cluster/kube-up.sh" worked well.
Hope that helps.

I am getting the same and I don't know how to fix it, every time I need to "vagrant destroy" to restart...

I'm getting the exactly same issue. I found a self-answered question from stackoverflow. http://stackoverflow.com/questions/24357058/vagrant-provision-with-salt-the-function-state-highstate-is-running

I figured out the problem.

The minion configuration file had the startup_states: key set to highstate so the highstate command was run when the minion was booted. To fix the issue, I commented out the line in the configuration and now everything works!"

But I couldn't catch what exactly he did. Someone helps me what to try..?

authors: Bretto Jamlee derekwaynecarr netroby ntquyen qwang1 sangwonl uruddarraju

repo: GoogleCloudPlatform/kubernetes


text:
Using the installsuffix flag causes a permission problem on my system

@ArtfulCoder Can you explain why changing the suffix is necessary? I think it's just the CGO_ENABLED=0 that causes it to be statically linked. ...also turning off CGO and then installing under the suffix cgo seems misleading, if anything it should be the suffix nocgo...

@lavalamp sadly this is the recommended way to statically compile with Go 1.4. Just using CGO_ENABLED=0 no longer works :(

I see. So I should indeed sudo build ... once and then it should be ok... thanks.

Yeah sorry it's not easier. @ArtfulCoder is working on a PR to have the scripts suggest that when you hit this error.

@vmarmol is there a way to trigger a go std library rebuild? sudo does not play nicely with our build script and my setup.

@lavalamp
sudo CGO_ENABLED=0 go install -a -installsuffix cgo std

@ArtfulCoder Thanks! my go installation isn't available from my root account for whatever reason, so I have to do sudo CGO_ENABLED=0 $(which go) install -a -installsuffix cgo std

authors: ArtfulCoder lavalamp vmarmol


text:
Challenged by 'hello world'

I'm unable to get the Cloud Monitoring exporter working:
https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/master/exporter/metric/README.md
There are appear to be a couple of issues in addition to my likely incompetence; I'm reasonable familiar with Cloud Monitoring and OpenCensus and this is my first dib into OpenTelemetry. Code at end based upon the README example. The code works with the stdout exporter.
I've exported:

GOOGLE_APPLICATION_CREDENTIALS to a known working Service Account with roles/monitoring.admin
PROJECT to a project with Stackdriver enabled

The module appears to have a bug using InstallNewPipeline(opts, nil):
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x972274]

goroutine 1 [running]:
go.opentelemetry.io/otel/sdk/metric/controller/push.New(0xbfa380, 0xc0001f08c0, 0xbebb60, 0xc000010740, 0xc00000fba0, 0x2, 0x2, 0x1)
	/home/dazwilkin/go/pkg/mod/go.opentelemetry.io/otel/sdk@v0.11.0/metric/controller/push/push.go:56 +0x84
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric.NewExportPipeline(0xc0002dfc50, 0x1, 0x1, 0xc0002dfc88, 0x1, 0x1, 0x4d1c01, 0xc00000f800, 0xc0002dfb98)
	/home/dazwilkin/go/pkg/mod/github.com/!google!cloud!platform/opentelemetry-operations-go/exporter/metric@v0.11.0/metric.go:166 +0x1cf
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric.InstallNewPipeline(0xc0002dfc50, 0x1, 0x1, 0xc0002dfc88, 0x1, 0x1, 0x7, 0x6, 0xc00a20)
	/home/dazwilkin/go/pkg/mod/github.com/!google!cloud!platform/opentelemetry-operations-go/exporter/metric@v0.11.0/metric.go:147 +0x6d
main.main()
	/home/dazwilkin/Projects/go-open-telemetry/main.go:25 +0xe1
exit status 2

This occurs in metric.go line 166 because the append results in len(opts)==2 with the second value being void:
pusher := push.New(
	checkpointer,
	exporter,
	append([]push.Option{
		push.WithPeriod(period),
	}, popts...)...,
)

Which then breaks push.go line 56 on the second apply:
for _, opt := range opts {
	opt.Apply(c)
}

I can fudge this by creating an non-nil popts:
opts := []cloudmonitoring.Option{
	cloudmonitoring.WithProjectID(projectID),
}
popts := []push.Option{}
pusher, err := cloudmonitoring.InstallNewPipeline(opts, popts...)

But this then results in a second failure (which bemuses me);
2020/09/09 14:49:49 [main] ProjectID: [[REDACTED]]
2020/09/09 14:49:49 Error during exporting TimeSeries: currently the aggregator is not supported: &{{0 0} 1 [1]}
2020/09/09 14:49:49 rpc error: code = InvalidArgument desc = Request was missing field timeSeries.

Code
package main

import (
	"context"
	"log"
	"math/rand"
	"os"
	"time"

	"go.opentelemetry.io/otel/api/metric"
	"go.opentelemetry.io/otel/label"
	"go.opentelemetry.io/otel/sdk/metric/controller/push"

	cloudmonitoring "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric"
)

func main() {
	projectID := os.Getenv("PROJECT")
	if projectID == "" {
		log.Fatal("Environment variable `PROJECT` is required and was unset")
	}

	log.Printf("[main] ProjectID: %s", projectID)
	opts := []cloudmonitoring.Option{
		cloudmonitoring.WithProjectID(projectID),
	}
	popts := []push.Option{}
	pusher, err := cloudmonitoring.InstallNewPipeline(opts, popts...)
	if err != nil {
		log.Fatal(err)
	}
	defer pusher.Stop()

	meter := global.Meter("cloudmonitoring/example")

	counter := metric.Must(meter).NewInt64Counter("counter.foo")

	lemonsKey := label.Key("ex.com/lemons")
	labels := []label.KeyValue{
		lemonsKey.Int(10),
		label.Key("key").String("value"),
		label.String("dog", "Freddie"),
	}

	ctx := context.Background()

	s := rand.NewSource(time.Now().UnixNano())
	r := rand.New(s)

	counter.Add(ctx, r.Int63n(100), labels...)
}

Error during exporting TimeSeries: currently the aggregator is not supported: &{{0 0} 1 [1]}

This error is returned in this block I believe: https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/71af267c4659cd8b4f71bcf2238fe6da203bd60c/exporter/metric/metric.go#L505-L508
and it will be fixed once #92 is merged AFAICT.
The issue is that we are trying to cast to sum.Aggregator, but should be using array.Aggregator instead.
/cc @james-bebbington

authors: DazWilkin nilebox


text:
Update OTel dependency

Update each OTel dependency and format go.mod.
Also, remove the deprecated (removed) go.opentelemetry.io/otel/metric/unit package and unified semconv version with v1.18.0.

go.opentelemetry.io/otel@v1.15.0-rc.1
go.opentelemetry.io/otel/metric@v1.15.0-rc.1
go.opentelemetry.io/otel/metric/instrument@v1.15.0-rc.1
go.opentelemetry.io/otel/sdk@v1.15.0-rc.1
go.opentelemetry.io/otel/sdk/export/metric@v0.38.0-rc.1
go.opentelemetry.io/otel/sdk/metric@v0.38.0-rc.1
go.opentelemetry.io/otel/trace@v1.15.0-rc.1
go.opentelemetry.io/otel/exporters/prometheus@v0.38.0-rc.1
go.opentelemetry.io/contrib/detectors/gcp@v1.16.0-rc.1
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.41.0-rc.1
go.opentelemetry.io/collector@v0.73.0
go.opentelemetry.io/collector/featuregate@v0.73.0
go.opentelemetry.io/collector/semconv@v0.73.0
go.opentelemetry.io/collector/pdata@v1.0.0-rc7
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/translator/prometheus@v0.73.0

Drop Go 1.18 because otelcol changed sync/atomic instead of go.uber.org/atomic.

https://github.com/open-telemetry/opentelemetry-collector/issues/7160
https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/actions/runs/4381458226/jobs/7669549957

If you accept this, I'll squash below commit:

Update DEFAULT_GO_VERSION to 1.20 and drop 1.18 on compatibility-test

Failed lint job because doesn't support current golangci-lint to go1.20. So update to v1.51.2.
Will squash Update golangci/golangci-lint to v1.51.2 for go1.20 as well.

/gcbrun

@damemi PTAL

/gcbrun

@damemi I'll fix conflicts

@damemi I'll fix conflicts

sorry @zchee, I caused those conflicts because I wanted to get at least 1 release published on go 1.19 before moving forward with this (instead of jumping from 1.18->1.20 in one release).
I'm still not sure about the rc tags, and would rather not publish a release based on those if possible. Are these updates urgent?

@damemi

sorry @zchee, I caused those conflicts because I wanted to get at least 1 release published on go 1.19 before moving forward with this (instead of jumping from 1.18->1.20 in one release).

np, that I also thought makes sense.

I'm still not sure about the rc tags, and would rather not publish a release based on those if possible. Are these updates urgent?

Okay, I agree.
So how about updating to the previous version of opentelemetry-go RC version (v1.14.0/v0.37.0/v0.0.4) and waiting for it to become GA? As far as the release notes of v1.14.0/v0.37.0/v0.0.4 are concerned, it seems reasonable as an upgrade while waiting for the GA release.
Like #616.

authors: damemi zchee


text:
Feature request: support the ?stats=1 query parameter

Prometheus supports stats=1 as part of the /api/v1/query and api/v1/query_range endpoints. It returns an additional json field called "stats" with some execution timings. For /query:
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": ["..."],
    "stats": {
      "timings": {
        "evalTotalTime": 0.0000875,
        "resultSortTime": 0,
        "queryPreparationTime": 0.000051542,
        "innerEvalTime": 0.000029166,
        "execQueueTime": 0.00004725,
        "execTotalTime": 0.000141459
      },
      "samples": {
        "totalQueryableSamples": 1,
        "peakSamples": 1
      }
    }
  }
}

Right now this parameter causes gmp-frontend to return an error:
{
  "error": {
    "code": 400,
    "message": "Invalid JSON payload received. Unknown name \"stats\": Cannot bind query parameter. Field 'stats' could not be found in request message.",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "description": "Invalid JSON payload received. Unknown name \"stats\": Cannot bind query parameter. Field 'stats' could not be found in request message."
          }
        ]
      }
    ]
  }
}

This means we can't make use of some excellent Prometheus tooling out there like pint to lint for expensive queries, because it relies on the stats=1 parameter (and not just for that one check, but all that call Prometheus APIs). Is it possible to increase compatibility with Prometheus and start supporting this param?

Thanks! Great request. 💪🏽
The main issue is to map internal workings of Google Cloud Monitoring PromQL engine to those stats which assumes Prometheus' TSDB PromQL engine.
We could make it work, at the minimum level, so tools like [pint](https://cloudflare.github.io/pint/checks/query/cost.html could work OR we could contribute to pint to make sure it's more generic. Anyway, the acceptance criteria here:
Acceptance Criteria

Make GMP work with pint

authors: bwplotka faevourite

repo: GoogleCloudPlatform/prometheus-engine


text:
GMP rule evaluation will not update

I've been editing a GlobalRule resource to update the alert expression and alert message. The original alert was too noisy, so I was changing the range vector duration to make it less spammy. I've simultaneously updated the alert message. However, I still get alerts with the old message, which makes me think the alert expression is not updated either. I've tried deleting the GlobalRule resource and recreating the GlobalRule. I've tried renaming the GlobalRule too. It doesn't seem to work. Currently, I'm trying to change the alertname to see if that fixes it.

Hey Will, this is a known issue and will be fixed in our upcoming 0.4.0 release, likely ready this upcoming week.

authors: lyanco willchan


text:
Mounting local scratch dir conflicts with default directory

Running a volume named spark-local-dir-1 creates a duplicate volume exception
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure
executing: POST at: https://10.100.0.1/api/v1/namespaces/datainfra/pods. Message: Pod "job-
driver" is invalid: spec.volumes[3].name: Duplicate value: "spark-local-dir-1". Received status:
Status(apiVersion=v1, code=422, details=StatusDetails(causes=
[StatusCause(field=spec.volumes[3].name, message=Duplicate value: "spark-local-dir-1",
reason=FieldValueDuplicate, additionalProperties={})], group=null, kind=Pod, name=cue-news-
job-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Pod
"job-driver" is invalid: spec.volumes[3].name: Duplicate value: "spark-local-dir-1",
metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}),
reason=Invalid, status=Failure, additionalProperties={}).

  driver:
    cores: 1
    memory: 1024m
    serviceAccount: spark
    volumeMounts:
      - name: spark-local-dir-1
        mountPath: /mnt/spark/work
  executor:
    cores: 1
    instances: 8
    memory: 4096m
    volumeMounts:
      - name: spark-local-dir-1
        mountPath: /mnt/spark/work
  volumes:
    - name: spark-local-dir-1
      emptyDir: {}

Which version of Spark is your app based upon?

I'm using 2.4.4

OK, looks like you ran into https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/pull/762.

Ah makes sense. I guess I'll have to wait for the next release.

Yeah, we will make the next release as soon as Spark 2.4.5 is out.

Thank you for the help.

A new release based on Spark 2.4.5 has been cut and published.

@liyinan926 I am still encountering the duplicate volume exception using spark-local-dir-1 in the latest release for both HostPath and PersistentVolumeClaim.
As a workaround, you can start from spark-local-dir-2 and it seems working.

@liyinan926 I am still encountering the duplicate volume exception using spark-local-dir-1 in the latest release for both HostPath and PersistentVolumeClaim.
As a workaround, you can start from spark-local-dir-2 and it seems working.

There are two Spark local directory if we use spark-local-dir-2 .  Who creates volume spark-local-dir-1, Spark-on-k8s-operator or Spark?


text:
Please support istio with auto inject

When istio is set to auto inject the executors fail to talk to the driver and jobs never finish.  Please add first-class support for istio.

@AceHack It's a little different in Istio. Istio don't support headless kubernetes svc by default. @liyinan926 Should we expose how to create the driver svc ?

Istio does support headless services as of 1.4
https://github.com/istio/istio/issues/7495
Istio does not work with job like things in Kubernetes
https://github.com/istio/istio/issues/6324
The problem is spark would need to use something like envoy pre-flight to properly wait for envoy to come up and then shut down pre-flight when the spark application ends.
https://github.com/monzo/envoy-preflight
We are using envoy-preflight for all other job in Kubernetes.

One work around that we tested is configuring the application to run outside the istio service mesh.

Running outside the service mesh is not really a valid work around we want all the observability a service mesh brings with our spark jobs.

Working with the upstream community, we have added a configuration workaround for Zookeeper along with Casssandra, Elasticsearch, Redis and Apache NiFi.  I am sure there are additional applications that are not compatible with sidecars.  Please inform the community if you know of any.
https://www.cncf.io/blog/2020/10/26/service-mesh-is-still-hard/
We have yet to really dig into enhancements in Spark 3, but we may need to accept that we are apart of the "applications that are not compatible with sidecars."

hi, what is exactly incompatible? we added an istio waiter to the image and executor starts. But then fails with:
20/11/19 07:55:58 INFO TransportClientFactory: Successfully created connection to spark.spark.svc/10.X.X.X:7078 after 99 ms (0 ms spent in bootstraps) 20/11/19 07:55:58 ERROR TransportResponseHandler: Still have 1 requests outstanding when connection from  spark.spark.svc/10.X.X.X:7078 is closed Exception in thread "main" java.lang.reflect.UndeclaredThrowableException 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1713) 	at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:64) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:281) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala) Caused by: org.apache.spark.SparkException: Exception thrown in awaitResult: 	at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:226) 	at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75) 	at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:101) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:201) 	at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:65) 	at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:64) 	at java.security.AccessController.doPrivileged(Native Method) 	at javax.security.auth.Subject.doAs(Subject.java:422) 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) 	... 4 more Caused by: java.io.IOException: Connection from  spark.spark.svc/10.X.X.X:7078 closed 	at org.apache.spark.network.client.TransportResponseHandler.channelInactive(TransportResponseHandler.java:146) 	at org.apache.spark.network.server.TransportChannelHandler.channelInactive(TransportChannelHandler.java:108) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) 	at io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) 	at org.apache.spark.network.util.TransportFrameDecoder.channelInactive(TransportFrameDecoder.java:182) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917) 	at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822) 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) 	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) 	at java.lang.Thread.run(Thread.java:748)
What can it be?

Its discussed here: https://github.com/istio/istio/issues/27900. So may be Istio has a fix in upstream but may have not released it yet.

Its discussed here: istio/istio#27900. So may be Istio has a fix in upstream but may have not released it yet.

There's nothing for Istio to fix there - running Spark in vanilla standalone cluster mode thru Istio requires a small amount of Spark master configuration is all - I've tested Spark and Istio in that fashion and it works fine, workers talk to masters thru Istio via headless services.
If on top of that spark-on-k8s-operator has some additional incompatibility with Istio, more information is needed on what the actual error people are experiencing is.

Its discussed here: istio/istio#27900. So may be Istio has a fix in upstream but may have not released it yet.

There's nothing for Istio to fix there - running Spark in vanilla standalone cluster mode thru Istio requires a small amount of Spark master configuration is all - I've tested Spark and Istio in that fashion and it works fine, workers talk to masters thru Istio via headless services.
If on top of that spark-on-k8s-operator has some additional incompatibility with Istio, more information is needed on what the actual error people are experiencing is.

#2 is the problem I'm seeing, also what is the master configuration you speak of?

#2 is the problem I'm seeing, also what is the master configuration you speak of?

As @bleggett said, 2 has a number of workarounds. @AceHack already mentioned envoy-preflight.
For 1, In Spark apps created by Spark operator and In general for Spark on k8s, Driver listens on the POD_IP. Istio has been dependent on the primary container to be listening on 127.0.0.1. This part is hardcoded in the Spark code.
https://github.com/apache/spark/blob/3a299aa6480ac22501512cd0310d31a441d7dfdc/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverServiceFeatureStep.scala#L34

istio/istio#27900 is a fix for that, Istio should be able to work when the driver pod listens on POD_IP.
Spark Master and workers, they are a different way of deployment. In a way they are independent of being deployed on Kubernetes or Mesos as they are not using the platform to create workers/executors.

As @bleggett said, 2 has a number of workarounds. @AceHack already mentioned envoy-preflight.
For 1, In Spark apps created by Spark operator and In general for Spark on k8s, Driver listens on the POD_IP. Istio has been dependent on the primary container to be listening on 127.0.0.1. This part is hardcoded in the Spark code.
https://github.com/apache/spark/blob/3a299aa6480ac22501512cd0310d31a441d7dfdc/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverServiceFeatureStep.scala#L34

istio/istio#27900 is a fix for that, Istio should be able to work when the driver pod listens on POD_IP.
Spark Master and workers, they are a different way of deployment. In a way they are independent of being deployed on Kubernetes or Mesos as they are not using the platform to create workers/executors.

@AceHack @puneetloya For the bitnami Spark Helm chart, all you have to do is set the SPARK_MASTER_HOST environment variable to 127.0.0.1 - no Istio changes are required.
Example values.yaml I pass to the Bitnami Spark chart:
master:
#   ## Annotations to add to the statefulset
#   ##
#   ##
#   podAnnotations:
#     proxy.istio.io/config: '{ "holdApplicationUntilProxyStarts": true }'
#     sidecar.istio.io/rewriteAppHTTPProbers: "false"
    extraEnvVars:
     - name: SPARK_MASTER_HOST
       value: "127.0.0.1"

Works fine.

@AceHack @puneetloya For the bitnami Spark Helm chart, all you have to do is set the SPARK_MASTER_HOST environment variable to 127.0.0.1 - no Istio changes are required.
Example values.yaml I pass to the Bitnami Spark chart:
master:
#   ## Annotations to add to the statefulset
#   ##
#   ##
#   podAnnotations:
#     proxy.istio.io/config: '{ "holdApplicationUntilProxyStarts": true }'
#     sidecar.istio.io/rewriteAppHTTPProbers: "false"
    extraEnvVars:
     - name: SPARK_MASTER_HOST
       value: "127.0.0.1"

Works fine.

@bleggett I think this advice is outdated with v1.10 that changes the default route envoy forward inbound packets to, to be eth0 rather than lo.
Also on this topic, can we make this operator name the ports according to their protocols?
$ istioctl analyze -n flows
Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name blockmanager (port: 7079, targetPort: 7079) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name driver-rpc-port (port: 7078, targetPort: 7078) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name spark-ui (port: 4040, targetPort: 4040) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service sparkpi-py-ui-svc.flows) Port name spark-driver-ui-port (port: 4040, targetPort: 4040) doesn't follow the naming convention of Istio port.

@haf Does #1239 address this?

Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name driver-rpc-port (port: 7078, targetPort: 7078) doesn't follow the naming convention of Istio port.

AFAIK, the naming of ports is not controlled by the spark operator. Its a code change required in the Apache Spark distribution(this was valid atleast till Spark 2.4, I am not sure if this was changed in Spark 3). The headless service is created by the driver once it is up, to make itself discoverable to executors.

@jkleckner Yes, but it would probably make sense to change the default name, too.

Asking here, since it's seems to be the thread. Currently trying to make the Spark operator work with the following stack:

Kubernetes v1.20.8-gke.1500
Istio 1.9.6 (Anthos Service Mesh more specifically)
Spark operator 1.1.6

I have enabled the istio autoinjection on the namespace where I create my SparkApplication (I'm using the spark-pi example)
I have customised the docker image I'm using to include scuttle
I have tried to set the mTLS policy of istio to DISABLE and PERMISSIVE via a PeerAuthentication resource

Whatever I do, the executor fails with the following error:
21/08/24 16:43:47 INFO TransportClientFactory: Successfully created connection to spark-pi-bb55307b78fb84d6-driver-svc.spark.svc/192.168.5.156:7078 after 151 ms (0 ms spent in bootstraps)
21/08/24 16:43:47 ERROR TransportClient: Failed to send RPC RPC 7365963296669017718 to spark-pi-bb55307b78fb84d6-driver-svc.spark.svc/192.168.5.156:7078: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.newClosedChannelException(AbstractChannel.java:957)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:865)
...

I'm not sure what I'm doing wrong: from various comments on this thread and others, this should work, no? The issue with the istio master listening on the pod ip and not 127.0.0.1 has been fixed in the versions I'm using.
If anyone has an idea, I'd be very thankful!

Hi @MrTrustor, did you resolve your issue? I am running into the same thing, though with slightly older istio so I am setting SPARK_MASTER_HOST

Failed to send RPC RPC xx to xx: java.nio.channels.ClosedChannelException
FWIW, I was able to get executor->driver communication to work by overriding the env var SPARK_DRIVER_BIND_ADDRESS to 127.0.0.1 on the driver. The default is hardcoded to PodIP and was not working with our version of istio.

We were facing the same error as well org.apache.spark.shuffle.FetchFailedException: Failed to send RPC RPC 5700455101843038265 to /xxx: java.nio.channels.ClosedChannelException, and we were able to get communication between driver & executor pod up & running by the following :

Set up the blockManager port by adding the SparkConf spark.blockManager.port: "7080"
Create the following headless service :

apiVersion: v1
kind: Service
metadata:
  name: spark-executor
spec:
  clusterIP: None
  ports:
  - name: blockmanager
    protocol: TCP
    port: 7080
  selector:
    spark-role: executor

Note that it doesn't have to be 7080 port, instead it can be any port number. By default it's assigning a random port.
Credits to @dayyeung

authors: AceHack MrTrustor Owen-CH-Leung afilipchik bleggett dmoore62 haf jkleckner meggers puneetloya ringtail


text:
Retrying connect to server: 0.0.0.0/0.0.0.0:8032.

I'm running a PySPark SparkApplication with Spark version 3 and the application fails when trying to connect to the ReourceManager.
What would you suggest for the next troubleshooting steps?
Here are my app logs:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
20/07/30 17:47:40 INFO SparkContext: Running Spark version 3.0.0
20/07/30 17:47:40 INFO ResourceUtils: ==============================================================
20/07/30 17:47:40 INFO ResourceUtils: Resources for spark.driver:

20/07/30 17:47:40 INFO ResourceUtils: ==============================================================
20/07/30 17:47:40 INFO SparkContext: Submitted application: transloc
20/07/30 17:47:40 INFO SecurityManager: Changing view acls to: root
20/07/30 17:47:40 INFO SecurityManager: Changing modify acls to: root
20/07/30 17:47:40 INFO SecurityManager: Changing view acls groups to:
20/07/30 17:47:40 INFO SecurityManager: Changing modify acls groups to:
20/07/30 17:47:40 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
20/07/30 17:47:40 INFO Utils: Successfully started service 'sparkDriver' on port 7078.
20/07/30 17:47:40 INFO SparkEnv: Registering MapOutputTracker
20/07/30 17:47:40 INFO SparkEnv: Registering BlockManagerMaster
20/07/30 17:47:40 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
20/07/30 17:47:40 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
20/07/30 17:47:40 INFO SparkEnv: Registering BlockManagerMasterHeartbeat
20/07/30 17:47:40 INFO DiskBlockManager: Created local directory at /var/data/spark-4977dbb1-1910-41f2-99e1-90be8e3fedd4/blockmgr-c9b31541-797e-4622-a0c9-1740960f556f
20/07/30 17:47:40 INFO MemoryStore: MemoryStore started with capacity 366.3 MiB
20/07/30 17:47:40 INFO SparkEnv: Registering OutputCommitCoordinator
20/07/30 17:47:41 INFO Utils: Successfully started service 'SparkUI' on port 4040.
20/07/30 17:47:41 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://transloc-cb04b873a0d59ef1-driver-svc.default.svc:4040
20/07/30 17:47:41 INFO RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
20/07/30 17:47:42 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
20/07/30 17:47:43 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

Fixed: I still had the master set to yarn.
change
spark = SparkSession \
            .builder \
            .master(master) \
            .appName(app) \
            .getOrCreate()

to
spark = SparkSession \
            .builder \
            .appName(app) \
            .getOrCreate()

Thanks this is helping me


text:
feat: Test 2 nodes instead of 3

Please see this Google-internal Buganizer comment for the rationale behind this change.
I have already set the default and minimum Node count inside Cloud Console to 2 (via Google-internal cl/610724610).

Thanks, Drew, for the quick review/approval!
Merging is technically still blocked since we're not CODEOWNERS.
But we can override that as repo admins — I will do that. (My guess = @tpryan is okay with that. 😄 )
We should look at the CODEOWNERS files inside all JSS GitHub repos and correct them. I've opened Google-internal bug, b/327473818.


text:
Prepare initial MEAN stack reference app to use for testing

https://github.com/mongodb-developer/mean-stack-example
Link to public container images:
Server: gcr.io/abis-345004/mean-stack-example
Client: gcr.io/abis-345004/mean-stack-example_client

Consensus on using https://github.com/mongodb-developer/mean-stack-example


text:
Add support for Sumo Logic Terraform Provider

Add support for Sumo Logic Terraform Provider
Add user resource

Sorry, this is not ready yet.

repo: GoogleCloudPlatform/terraformer


text:
JibDockerBuild fails on download image (manifest) which is only locally available

Description of the issue:
Using a from-clause with an base-image only available in the local Docker daemon cache, jib still tries to download the image from dockerhub. This, of course, fails
Expected behavior:
jibDockerBuild is able to use the image from the local cache and does not fail on missing metadata from dockerhub.
Steps to reproduce:

create local base image
use base-image in jib-from

Environment:
OpenSuse Leap 15.1
Gradle 5.4.1
jib-gradle-plugin Configuration: Kotlin-DSL
jib {
	from.image = "open-liberty:webProfile8-java12" // not yet available on dockerhub. locally forked
	to.image = "test/mystuff:openliberty"
	container {
		appRoot = "/opt/ol/wlp/usr/servers/defaultServer/apps/some.war"
		ports = mutableListOf("8080")
	}
	extraDirectories {
		setPaths(mutableListOf(Paths.get(libertyDockerFolder)))
	}
}

Log output:
Execution failed for task ':jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Tried to pull image manifest for registry-1.docker.io/library/open-liberty:webProfile8-java12 but failed because: manifest unknown | If this is a bug, please file an issue at https://github.com/GoogleContainerTools/jib/issues/new

This is not supported in jib right now, but we expect to support it eventually.

Closing as a dup of #1468.
@lostiniceland you can use the offline mode (--offline) with JibDockerBuild, which will not attempt downloading the manifest of the base image. The offline mode will work provided you ran jibDockerBuild online at least once. However, note that this still won't make use of the Docker cache, so this may not cover your case if the image is only available in your local Docker cache.

authors: chanseokoh loosebazooka lostiniceland

repo: GoogleContainerTools/jib


text:
Add retries to mitigate the flakiness

The flakiness of TestLargeCRD was identified while working on https://github.com/GoogleContainerTools/kpt-config-sync/pull/215:
https://oss.gprow.dev/view/gs/oss-prow-build-kpt-config-sync/pr-logs/pull/GoogleContainerTools_kpt-config-sync/215/kpt-config-sync-presubmit-e2e-mono-repo-test-group2/1582107528844546048

/retest

/retest

authors: haiyanmeng sdowell

repo: GoogleContainerTools/kpt-config-sync


text:
Remove Zip-inflation

Based on superbrothers/fix-extract-zip 1ed5e47
This introduces a breaking change. Single
dir zip files won't be inflated anymore.
File globs after this point should honor
the real file paths.
See: #9
Closes: #10

/cc @superbrothers

I'm guessing this requires some update to the design or developer guide as well (if we mentioned this zip directory structure).

authors: ahmetb lbb

repo: GoogleContainerTools/krew


text:
make ReplAsyncExecutor send sent messages back to the repl generator

Allows snippets such as:
m = yield '*smash* that mf like button'
await m.add_reaction('👍')

With this, a message would be sent, and a reaction added to that same message.
This required the addition of a dependency: async_generator. This library has a function called
yield_from_ which allows delegating to another async generator without a nasty while True loop, akin
to what yield from does in regular, synchronous generators. Generators created with this library are 100%
compatible with regular generators.
Doing so also made the async for loop in the jishaku_py command uglier: it was made into a while True loop
with two except StopAsyncIteration: break occurrences inside it. While this is uglier, this is the standard way
to commmunicate two-ways with an async generator, and is not that much of a hardship considering how little
ReplAsyncExecutor.traverse had to change.

This implementation is being rejected in favor of the tidier, simpler implementation I wrote in 691287143fae09aa87af4f073626c7a40b4203f6.

Thanks! I'm not sure I agree with simpler and tidier, though 🤨

authors: Gorialis bmintz

repo: Gorialis/jishaku


text:
fixed task naming conflict

fixed gradle 3.0 standard lifecycle plugins task naming conflict.

Thanks for the PR! I'll merge it as soon as I've got some time to verify it.

in the meantime, feel free to use this:
https://bintray.com/cabecao1/generic/gradle-fx-1.3.3-patched-for-gradle-3.x/

Merged. This fixed is currently available on our snapshot repository (https://oss.sonatype.org/content/repositories/snapshots) as version 1.4.0-SNAPSHOT

Do you have rough idea when this change would be released?

I've just released GradleFx 1.4.0 which includes this fix. Sorry for the delay, last few months have been very busy.

Thank you, this is very much appreciated!

repo: GradleFx/GradleFx


text:
Sprint 2/step 2

Hello, everything are ready. I am waiting for your review.

Упс я случайно замержил на мастер.

Отлично, практически все критичные комментарии исправлены. В процессе исправления появилось одно небольшое замечание:
https://github.com/GrafDev/react-burger/pull/6#discussion_r1189627398
Пожалуйста, учтите данное замечание.
При повторной проверке, если замечание будет исправлено, работа будет принята.
Удачного рефакторинга кода!

Отлично, все критичные комментарии исправлены. Работу принимаю. Удачи в дальнейшем обучении!

authors: GrafDev vadmas

repo: GrafDev/react-burger


text:
About the ''edge_weight'' in ''PPRDiffusion'' augment

Thanks for opensourcing.
When I read examples "MVGRL_graph.py", I found  only augmented 'edge_index2' and 'x2' were used as the input of 'gcn2'. Why did not input the  augmented  ''edge_weight2'' to 'gcn2' ?
It would be appreciated if anyone could help tell me the reason.

Thanks for mentioning this problem. We will take a closer look.

Fixed. Please checkout the newest develop branch. Thanks for your kind suggestion!

repo: GraphCL/PyGCL


text:
Add a "warming" service to ensure faster usage via shortcut

Google Camera is doing this on the stock OS. This may be decided to be unnecessary.
See: config_keep_warming_services in frameworks/base

I'd rather not do this.

authors: flawedworld thestinger

repo: GrapheneOS/Camera


text:
Use Case: Immutable objects

Demonstrate populating an immutable object and collection of objects.

[x] Test cases
[x] ADO
[x] Chain
[x] Dapper
[x] EF Core
[x] NHibernate
[ ] RepoDB
[ ] ServiceStack
[x] Documentation

Tickets opened for remaining examples.

repo: Grauenwolf/DotNet-ORM-Cookbook

license: unlicense


text:
Logical and Objects matchers implementation

We need to implement several main matchers, according to description in README.md file: .equal-to, .or, .and

and implement: .all-of, .any-of, .not

#11

We don't need .or and .and objects . We can use .all-of or .any-of instead

repo: Graur/eo-hamcrest


text:
Insufficient permissions for maniaexchange

As I said in the tittle, I get the error "Insufficient permissions for "maniaexchange"", I have no idea why or how this is happening, but it is the only error that says something about this plugin, I have tried reinstalling it and installing optional plugins (better chat, itemexchange), and it did not help, it's my first time using openplanet and maniaexchange.

You need to run Standard edition of the game since it allows to play custom maps

You need to run at least Standard edition of the game since it allows to play custom maps

ohhhh, okay thanks!

authors: GreepTheSheep fily-gif


text:
Comparing ImmutableArray results in StackOverflowException

Comparing System.Collections.Immutable.ImmutableArray results in StackOverflowException.
Reproducible as simple console app:
namespace ConsoleApp1
{
    // .NET runtime 4.7.1
    using System.Collections.Immutable; // 1.3.1
    using KellermanSoftware.CompareNetObjects; // 4.4.0

    internal class Program
    {
        internal static void Main()
        {
            var a1 = new[] { 1, 2 }.ToImmutableArray();
            var a2 = new[] { 1, 2 }.ToImmutableArray();

            var comparer = new CompareLogic();

            comparer.Compare(a1, a2); // -> StackOverflowException
        }
    }
}

@jdvor Immutable arrays are not supported.  Please implement.

authors: GregFinzer jdvor

repo: GregFinzer/Compare-Net-Objects

license: MS-PL