text:
"Google Api Error: Unauthorized - Request is missing required authentication credential" running create_app_on_managed_play_store/upload_to_play_store lanes

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description

When we execute the "create_app_on_managed_play_store" lane we're always getting a "Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential." error since a month ago. The App in the Managed Google Play Console is not created.
The json we use was generated on a project in Google Cloud Platform and has privileges to execute Google Play APIs (Google Play Android Developer API and Google Play Custom App Publishing API) and also has admin privileges on Managed Google Play Console. Prior to a month ago, it always worked fine.
It seems that Google requires a different authentication method now? It is possible that fastlane has not adapted?
Command executed
fastlane run create_app_on_managed_play_store
Complete output when running fastlane, including the stack trace and command used

 vmontana@K9XLDQKJR9 fastlane % fastlane run create_app_on_managed_play_store language:"es_ES"
[✔] 🚀
[08:38:47]: fastlane detected a Gemfile in the current directory
[08:38:47]: However, it seems like you didn't use `bundle exec`
[08:38:47]: To launch fastlane faster, please use
[08:38:47]:
[08:38:47]: $ bundle exec fastlane run create_app_on_managed_play_store language:es_ES
[08:38:47]:
[08:38:47]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
[08:38:48]: ----------------------------------------------
[08:38:48]: --- Step: create_app_on_managed_play_store ---
[08:38:48]: ----------------------------------------------
[08:38:48]: To not be asked about this value, you can specify it using 'json_key'
[08:38:48]: The service account json file used to authenticate with Google: ./mdona-cloud-dev-cicdmovd.json
+----------+------------------------------------------------------------+
|             Summary for create_app_on_managed_play_store              |
+----------+------------------------------------------------------------+
| language | es_ES                                                      |
| json_key | /Users/vmontana/Documents/fastlane/mdona-cloud-dev-cicdmo  |
|          | vd.json                                                    |
| apk      | oficinas_and-1.2.61(23022711).apk                          |
| timeout  | 300                                                        |
+----------+------------------------------------------------------------+
[08:39:04]: To not be asked about this value, you can specify it using 'app_title'
[08:39:04]: App Title: oficinas_and
[08:39:45]: To not be asked about this value, you can specify it using 'developer_account_id'
[08:39:45]: The ID of your Google Play Console account. Can be obtained from the URL when you log in
(https://play.google.com/apps/publish/?account=... or when you 'Obtain private app publishing rights'
(https://developers.google.com/android/work/play/custom-app-api/get-started#retrieve_the_developer_account_id): <HIDDEN_VALUE>
[!] Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

Environment

 ✅ fastlane environment ✅
Stack

Key
Value

OS
12.6.3

Ruby
3.2.1

Bundler?
false

Git
git version 2.37.1 (Apple Git-137.1)

Installation Source
/opt/homebrew/lib/ruby/gems/3.2.0/bin/fastlane

Host
macOS 12.6.3 (21G419)

Ruby Lib Dir
/opt/homebrew/Cellar/ruby/3.2.1/lib

OpenSSL Version
OpenSSL 1.1.1t  7 Feb 2023

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode-14.2.app/Contents/Developer/

Xcode Version
14.2

Swift Version
5.7.2

System Locale

Variable
Value

LANG
es_ES.UTF-8
✅

LC_ALL

LANGUAGE

fastlane files:
No Fastfile found
No Appfile found
fastlane gems

Gem
Version
Update-Status

fastlane
2.212.1
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

error_highlight
0.5.1

did_you_mean
1.6.3

syntax_suggest
1.0.2

atomos
0.1.3

rexml
3.2.5

CFPropertyList
3.0.6

claide
1.1.0

colored2
3.1.2

nanaimo
0.3.0

xcodeproj
1.22.0

rouge
2.0.7

xcpretty
0.3.0

terminal-notifier
2.0.0

unicode-display_width
1.8.0

terminal-table
1.8.0

plist
3.7.0

public_suffix
5.0.1

addressable
2.8.1

multipart-post
2.0.0

word_wrap
1.0.0

optparse
0.1.1

tty-screen
0.8.1

tty-cursor
0.7.1

tty-spinner
0.9.3

artifactory
3.0.15

babosa
1.0.4

colored
1.2

highline
2.0.3

commander
4.6.0

excon
0.99.0

faraday-em_http
1.0.0

faraday-em_synchrony
1.0.0

faraday-excon
1.1.0

faraday-httpclient
1.0.1

faraday-multipart
1.0.4

faraday-net_http
1.0.1

faraday-net_http_persistent
1.2.0

faraday-patron
1.0.0

faraday-rack
1.0.0

faraday-retry
1.0.3

ruby2_keywords
0.0.5

faraday
1.10.3

unf_ext
0.0.8.2

unf
0.1.4

domain_name
0.5.20190701

http-cookie
1.0.5

faraday-cookie_jar
0.0.7

faraday_middleware
1.2.0

fastimage
2.2.6

gh_inspector
1.1.3

json
2.6.3

mini_magick
4.12.0

naturally
2.2.1

rubyzip
2.3.2

security
0.1.3

xcpretty-travis-formatter
1.0.1

dotenv
2.8.1

bundler
2.4.6

simctl
1.6.10

jwt
2.7.0

uber
0.1.0

declarative
0.0.20

trailblazer-option
0.1.2

representable
3.2.0

retriable
3.1.2

mini_mime
1.1.2

memoist
0.16.2

multi_json
1.15.0

os
1.1.4

signet
0.17.0

googleauth
1.3.0

httpclient
2.8.3

webrick
1.8.1

google-apis-core
0.11.0

google-apis-playcustomapp_v1
0.12.0

google-apis-androidpublisher_v3
0.34.0

google-cloud-env
1.6.0

google-cloud-errors
1.3.0

google-cloud-core
1.6.0

google-apis-iamcredentials_v1
0.17.0

google-apis-storage_v1
0.19.0

rake
13.0.6

digest-crc
0.6.4

google-cloud-storage
1.44.0

emoji_regex
3.2.3

jmespath
1.6.2

aws-partitions
1.716.0

aws-eventstream
1.2.0

aws-sigv4
1.5.2

aws-sdk-core
3.170.0

aws-sdk-kms
1.62.0

aws-sdk-s3
1.119.1

set
1.0.3

forwardable
1.3.3

logger
1.5.3

pathname
0.2.1

shellwords
0.1.0

cgi
0.3.6

date
3.3.3

timeout
0.3.1

securerandom
0.2.2

uri
0.12.0

openssl
3.1.0

digest
3.1.1

ipaddr
1.2.5

resolv
0.2.2

time
0.2.1

stringio
3.0.4

open-uri
0.3.0

mutex_m
0.1.2

net-http
0.3.2

net-protocol
0.2.1

ostruct
0.5.5

english
0.7.2

erb
4.0.2

abbrev
0.1.1

tempfile
0.1.3

delegate
0.3.0

fileutils
1.7.0

tmpdir
0.1.3

base64
0.1.1

singleton
0.1.1

open3
0.1.2

nkf
0.1.2

prettyprint
0.1.1

pp
0.4.0

find
0.1.1

yaml
0.2.1

psych
5.0.1

generated on: 2023-02-28

@vmontana try this guide
https://medium.com/@muhammedqazi/mastering-fastlane-the-ultimate-guide-to-effortless-android-and-ios-deployments-with-github-e0878c3f9529

Hi. Does the "create_app_on_managed_play_store" lane work for anyone?
To check if it is a general issue or not.
Thanks.

@vmontana try this guide https://medium.com/@muhammedqazi/mastering-fastlane-the-ultimate-guide-to-effortless-android-and-ios-deployments-with-github-e0878c3f9529

Your tutorial guided me to use json_key with upload_to_play_store and solved authentication issue.
upload_to_play_store(track: 'internal', json_key: json_key_file)

@vmontana @mvrty What happens when you put json_key property to your create_app_on_managed_play_store?
create_app_on_managed_play_store(
  json_key: 'path/to/you/json/key/file',
  ...
)

Actually re-ran the job after a while and it went through without a problem. Some issues at play store today maybe?

@marcosrocha85  & @mvrty : You have commented the behavior of the "upload_to_play_store" lane. That is not the lane for which the case has been opened. Does the exact lane "create_app_on_managed_play_store" work for you?
@marcosrocha85 : We were already putting the json_key parameter as you indicate and it fails with the same error.
create_app_on_managed_play_store(
  json_key: 'path/to/you/json/key/file',
  ...
)

Same here, started failing more frequently today

same here, it is failing since today at 11:30 CET
previous build working perfectly, nothing changed in our configuration

same here

any news

Hello,
I'm still facing this issue only with "create_app_on_managed_play_store" lane.
Does it work for anyone?

It seems that few people use this lane.
Has anyone used it recently?
Thanks.

The problem was there we should set this Access token location: access_token url paramater

Hi @fxcjahid ,
It's an issue that the client side can solve? Or is a new release of fastlane needed?
Thanks!

I don't know, But this is server-side

Hello,
Could any fastlane developer please confirm if this is an issue that could be resolved with a new version of fastlane?
Thanks!

hello, is there any update on this one? i am also experiencing same issue.

same issue too

Having the same problem intermittently while using supple

Same Issue, are there any updates?

Any workaround here?
Build stopped working this week.

I'm facing the same issue, any updates here?

We have the same problem, JSON keys are ok, permissions are there. If anyone has a solution, please tell us! :)

This is happening on our project as well, started facing it 2 days ago.

facing this same issue from today

Started getting this yesterday as well, not on every run, but about 75% of the time.

I'm also seeing this on the upload_to_play_storecommand. Last ran successfully on 7th September. No runs over the weekend but since Monday it's failed consistently. No config changes on our end

The same issue if we are trying to upload more than one localization at a time.

We're facing the same issue. Please help. We've a production release today.

We're facing the same issue. Please help. We've a production release today.

Started getting this from yesterday. It is being triggered when fastlane is trying to sync image metadata. Please help!!

We are having the same issue

Same issue in Bitrise workflow

Same issue in Bitrise workflow

rebuild fix the problem. for now.

https://github.com/fastlane/fastlane/issues/21507
the same scenario for us too, the error gets triggered during screenshots upload

I ended up building and uploading our build manually for now.

Same on my side for upload_to_play_store since today - worked all the time. Checked JSON_KEY_FILE but all good. No code changes at all.

same issue, i ended up deploying it manually

I opened this case for the "create_app_on_managed_play_store" lane (Android enterprise private apps). I am still facing this problem since February without a solution yet.
Now, I'm getting the same error in the 'public' lane "upload_to_play_store" like you guys. It seems like the same problem is affecting both lanes.
Please, Fastlane developers, could you check what's going on?

We just started getting this error yesterday for the upload_to_play_store lane. It was working fine as of last Friday (9/8) 😑 🤦

I think Google has made some changes to the API. I'm encountering this issue because I've integrated the Play Store API into my project, and I've been receiving the same error since yesterday..

We have the same issue with Bitrise, and they have similar reports. It's an issue on Google's side.

@mirkos93 If you refresh your Json key does it solve the issue?
Anyone knows if there is Changelog for the service deployed by Google? That would be a start

Google Cloud status is green: https://status.cloud.google.com/
Previous incident last night, not sure it's related: https://status.cloud.google.com/incidents/71zrZxw3PRkiNYhM1JsZ

Started facing this issue again today while using fastlane on Azure pipelines...last months release worked fine

+1

We have the same issue with Bitrise, and they have similar reports. It's an issue on Google's side.

@mirkos93 If you refresh your Json key does it solve the issue?
Anyone knows if there is Changelog for the service deployed by Google? That would be a start

Nothing to do: I refreshed the JSON key and created a new service account, but the result is the same.

I do believe something changed/is happening with Google.  My store updates usually took about 1 - 2 hours for approval.  Then just before this issue surfaced, the last one took almost 24 hours, then all subsequent uploads/updates have failed.

It randomly works again here for us (using Bitrise but same error). We haven't changed anything, so it's very intermittent.

Here we are having the same issue as @fabiendem

We also just had a random successful upload. No change in our config.

We're seeing this and not seeing a pattern, no config changes.

any updates? I also encountered this issue

Exactly same issue here! No config changes since last sync.

Also I think I found a duplicated issue here #21507.

Same issue here. It was working fine until 3 hours ago.

This is probably a temporary Google Issue. Found one report on their issue tracker, and one member said they're looking into it.
https://issuetracker.google.com/issues/300236995#comment2

For me this issue started between Sept 7th and Sept 9th.
The google issue tracker noted in the comment above notes the issue started Sept 7th for them.
This thread got resurrected Sept 8th and started picking up steam a few days later.

Facing the same issue ...

Start to face the same issue here. It's stopped to work some hours ago. (14/09)

same issue, hope it's temporary on Google's end

I confirm, by setting these params, the upload goes through successfully:
      skip_upload_metadata: true,
      skip_upload_images: true,
      skip_upload_screenshots: true,

this workaround worked for me

Try setting the root_url:
upload_to_play_store(
root_url: "https://androidpublisher.googleapis.com/",
...
)

this has recently started for me as well.
skip_upload_images: true,
skip_upload_screenshots: true,

also worked for me

same here. It stopped working today

I run supply from a bash script and use environment variables.  I did:
export SUPPLY_ROOT_URL="https://androidpublisher.googleapis.com/"
and it still failed.

SUPPLY_ROOT_URL=https://androidpublisher.googleapis.com/
SUPPLY_METADATA_PATH=play-listing
SUPPLY_VERSION_CODE=60
SUPPLY_JSON_KEY=/secrets/playstore_api.json
SUPPLY_APK=/artifacts/Ladakhi_Bible-3.7.6.apk
SUPPLY_TRACK=production
SUPPLY_PACKAGE_NAME=org.wa.lbjnt.ladakhi
+ fastlane supply --verbose
[14:44:20]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
+------------------------------------+--------------------------------------+
|                        Summary for supply 2.214.0                         |
+------------------------------------+--------------------------------------+
| package_name                       | org.wa.lbjnt.ladakhi                 |
| version_code                       | 60                                   |
| release_status                     | completed                            |
| track                              | production                           |
| metadata_path                      | play-listing                         |
| json_key                           | /secrets/playstore_api.json          |
| apk                                | /artifacts/Ladakhi_Bible-3.7.6.apk   |
| skip_upload_apk                    | false                                |
| skip_upload_aab                    | false                                |
| skip_upload_metadata               | false                                |
| skip_upload_changelogs             | false                                |
| skip_upload_images                 | false                                |
| skip_upload_screenshots            | false                                |
| track_promote_release_status       | completed                            |
| validate_only                      | false                                |
| root_url                           | https://androidpublisher.googleapis  |
|                                    | .com/                                |
| check_superseded_tracks            | false                                |
| timeout                            | 300                                  |
| deactivate_on_promote              | true                                 |
| changes_not_sent_for_review        | false                                |
| rescue_changes_not_sent_for_review | true                                 |
| ack_bundle_installation_warning    | false                                |
+------------------------------------+--------------------------------------+
DEBUG [2023-09-15 14:44:23.58]: Fetching a new access token from Google...
INFO [2023-09-15 14:44:24.21]: Preparing apk at path '/artifacts/Ladakhi_Bible-3.7.6.apk' for upload...
INFO [2023-09-15 14:44:28.43]: 	Version Code: 60
INFO [2023-09-15 14:44:28.43]: Updating track 'production'...
INFO [2023-09-15 14:44:29.16]: Preparing to upload for language 'en-AU'...
INFO [2023-09-15 14:44:29.68]: Uploading image file play-listing/en-AU/images/featureGraphic.png...
INFO [2023-09-15 14:44:31.29]: Uploading image file play-listing/en-AU/images/icon.png...
INFO [2023-09-15 14:44:32.94]: Uploading screenshot play-listing/en-AU/images/phoneScreenshots/Oppo Audio playback.png...
/root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/fastlane_core/lib/fastlane_core/ui/interface.rb:141:in user_error!': \e[31m[!] Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.\e[0m (FastlaneCore::Interface::FastlaneError) 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/fastlane_core/lib/fastlane_core/ui/ui.rb:17:in method_missing'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb:95:in rescue in call_google_api' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb:80:in call_google_api'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb:573:in upload_image' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:290:in block (2 levels) in upload_screenshots'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:288:in each' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:288:in block in upload_screenshots'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:281:in each' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:281:in upload_screenshots'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:100:in block (2 levels) in perform_upload_meta' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:92:in each'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:92:in block in perform_upload_meta' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:84:in each'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:84:in perform_upload_meta' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:35:in perform_upload'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/commands_generator.rb:40:in block (2 levels) in run' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:187:in call'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:157:in run' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/runner.rb:444:in run_active_command'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:124:in run!' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/delegates.rb:18:in run!'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/commands_generator.rb:61:in run' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/commands_generator.rb:13:in start'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/fastlane/lib/fastlane/cli_tools_distributor.rb:115:in take_off' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/bin/fastlane:23:in <top (required)>'
from /root/.rbenv/versions/2.7.1/bin/fastlane:23:in load' 	from /root/.rbenv/versions/2.7.1/bin/fastlane:23:in '
/root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/http_command.rb:230:in check_status': Unauthorized (Google::Apis::AuthorizationError) 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/api_command.rb:137:in check_status'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/http_command.rb:202:in process_response' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/upload.rb:174:in process_response'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/upload.rb:259:in execute_once' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/http_command.rb:126:in block (2 levels) in do_retry'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/retriable-3.1.2/lib/retriable.rb:61:in block in retriable' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/retriable-3.1.2/lib/retriable.rb:56:in times'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/retriable-3.1.2/lib/retriable.rb:56:in retriable' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/http_command.rb:123:in block in do_retry'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/retriable-3.1.2/lib/retriable.rb:61:in block in retriable' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/retriable-3.1.2/lib/retriable.rb:56:in times'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/retriable-3.1.2/lib/retriable.rb:56:in retriable' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/http_command.rb:113:in do_retry'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/http_command.rb:104:in execute' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-core-0.11.1/lib/google/apis/core/base_service.rb:418:in execute_or_queue_command'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/google-apis-androidpublisher_v3-0.48.0/lib/google/apis/androidpublisher_v3/service.rb:1078:in upload_edit_image' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb:574:in block in upload_image'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb:81:in call_google_api' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb:573:in upload_image'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:290:in block (2 levels) in upload_screenshots' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:288:in each'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:288:in block in upload_screenshots' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:281:in each'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:281:in upload_screenshots' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:100:in block (2 levels) in perform_upload_meta'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:92:in each' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:92:in block in perform_upload_meta'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:84:in each' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:84:in perform_upload_meta'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/uploader.rb:35:in perform_upload' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/commands_generator.rb:40:in block (2 levels) in run'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:187:in call' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/command.rb:157:in run'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/runner.rb:444:in run_active_command' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:124:in run!'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/commander-4.6.0/lib/commander/delegates.rb:18:in run!' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/commands_generator.rb:61:in run'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/commands_generator.rb:13:in start' 	from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/fastlane/lib/fastlane/cli_tools_distributor.rb:115:in take_off'
from /root/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/fastlane-2.214.0/bin/fastlane:23:in <top (required)>' 	from /root/.rbenv/versions/2.7.1/bin/fastlane:23:in load'
from /root/.rbenv/versions/2.7.1/bin/fastlane:23:in `'

See https://github.com/fastlane/fastlane/issues/21507#issuecomment-1721426491 . It's a problem on Google's end. Simply repeatedly retrying the deployment eventually works.

It's a problem on Google's end; I provided a quick-and-dirty patch in the other issue to get you out of the bind in the meantime until Google fixes it.

Is there an issue reported on Google's End? Do they know this is an issue?
I saw the one that looks like an issue with uploading to YouTube. I am not convinced they are the same issue.

Is there an issue reported on Google's End? Do they know this is an issue? Is there someone with Fastlane that has a contact with Google API team that can communicate with them?
I saw the one that looks like an issue with uploading to YouTube. I am not convinced they are the same issue.

the one with youtube seems to be the same to me because it is also related to missing authorization, there is no other recent report at the same channel, and the user states they're using the same code/token for years, just like we are. And also because someone at Google said they're looking internally.
Let's hope it's the same issue. I'll try to link this issue on their issue tracker so they can see it's happening to more people

@JoaoBGusmao I was not able to make a comment on that issue to try to tie them together. I hope you can.

I am having the same issue. Any work around? apart from retrying?

I am having the same issue. Any work around? apart from retrying?

Yes, the one I linked above

I have it too on upload_to_play_store and it's not going away even retry multiple times ...

For us it's kinda random. We have tried setting the timeout to 900 inside the upload_to_play_store as stated in the docs, which makes the fail rate lower but it's still not stable.
Exchanging the JSON Key file for a newer one also does not help...

We shipped fastlane 2.215.1 this weekend, which includes my patch to workaround this Google API issue.
Once you update to fastlane 2.215.1, set the SUPPLY_UPLOAD_MAX_RETRIES  env var to an integer to auto-retry each randomly failed Google API error.
You can do this either in your fastalane/.env file (if you use one), or inside your Fastfile using ENV['SUPPLY_UPLOAD_MAX_RETRIES']=5 for example.

@AliSoftware I just tried it with v2.215.1 but it still fails:
[10:33:30]: ----------------------------------
[10:33:30]: --- Step: upload_to_play_store ---
[10:33:30]: ----------------------------------

+------------------------------------------------------------------------------------------+
|                                Summary for supply 2.215.1                                |
+------------------------------------+-----------------------------------------------------+
| package_name                       | com.finstreet.xxxx                                |
| track                              | internal                                            |
| aab                                | ../build/app/outputs/bundle/release/app-release.aab |
| skip_upload_changelogs             | true                                                |
| timeout                            | 900                                                 |
| release_status                     | completed                                           |
| metadata_path                      | ./fastlane/metadata/android                         |
| json_key                           | ./fastlane/service_account_credentials.json         |
| skip_upload_apk                    | false                                               |
| skip_upload_aab                    | false                                               |
| skip_upload_metadata               | false                                               |
| skip_upload_images                 | false                                               |
| skip_upload_screenshots            | false                                               |
| sync_image_upload                  | false                                               |
| track_promote_release_status       | completed                                           |
| validate_only                      | false                                               |
| check_superseded_tracks            | false                                               |
| deactivate_on_promote              | true                                                |
| changes_not_sent_for_review        | false                                               |
| rescue_changes_not_sent_for_review | true                                                |
| ack_bundle_installation_warning    | false                                               |
+------------------------------------+-----------------------------------------------------+

[10:33:31]: Preparing aab at path '../build/app/outputs/bundle/release/app-release.aab' for upload...
[10:33:59]: Updating track 'internal'...
[10:34:00]: Preparing to upload for language 'de-DE'...
[10:34:02]: ⬆  Uploading screenshot ./fastlane/metadata/android/de-DE/images/phoneScreenshots/01_Cover_Android.jpg...
+----------------------------------------+
|              Lane Context              |
+------------------+---------------------+
| DEFAULT_PLATFORM | android             |
| PLATFORM_NAME    | android             |
| LANE_NAME        | android development |
+------------------+---------------------+
[10:34:03]: Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

Legend. Setting the ENV works for me!

I had ran into this issue. Adding the skip Images and screenshots step fixed it for me but i'll also give it a whirl with the SUPPLY_UPLOAD_MAX_RETRIES env and report back

Alright, addding the MAX_RETRIES env var also fixes it so both fixes work! i also added the root_url for good measure

For me MAX_RETRIES was not enough. I also needed to set skip_upload_screenshots to false. Trying with fastlane 2.216.0.

Hi community
I'm having the same issue with the latest fastlane version
[!] Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
I already placed the SUPPLY_UPLOAD_MAX_RETRIES=5 in my .env file and it's not working.
It's working like the 30% of the times for me.

Initially SUPPLY_UPLOAD_MAX_RETRIES=5 was working. Now it failing with 5 retries.  So we are upping the value to 12. I hope Google finds a solution soon. I followed the previously posted issue in Google's bug tracker (https://issuetracker.google.com/issues/300236995#comment7).  They said yesterday:

Thanks for sharing all the details! The engineering team has fixed this issue and you should be no longer seeing auth errors during reference upload. Let me know if you still see this issue persist.

We are still having the problem today. So I think their work was limited to YouTube content upload.  Does anyone know of a Google issue related to Google Play Publishing?

In fact on some cases it even many retries don't fix the issue if upload is large, we need a better solution or get in touch with Google?

Initially SUPPLY_UPLOAD_MAX_RETRIES=5 was working. Now it failing with 5 retries. So we are upping the value to 12. I hope Google finds a solution soon. I followed the previously posted issue in Google's bug tracker (https://issuetracker.google.com/issues/300236995#comment7).

There was another attempt today to publish the same app with retries=12. Good news is there were no failures that needed to be retried.  So maybe they have fixed the underlying problem?

Good news is there were no failures that needed to be retried. So maybe they have fixed the underlying problem?

5 hours ago, I received an email from someone on the Android DevRel team at Google saying that they some of their partners reported the issue and that they thus started to monitor https://github.com/fastlane/fastlane/issues/21507. In the email they mention:

I discussed with the Play team yesterday, and they had identified an issue which may have contributed to this issue. They believe they have now fixed this issue.
It isn't confirmed that the issue that they identified and fixed was indeed the cause of the issue Fastlane has been experiencing, but we'd be interested to know whether users of Fastlane are now seeing an improvement, with no more failures, and no more need for retries?

This looks very promising and might explains why @chrisvire 's attempt today passed without any retry needed 🙏

1 hour ago i tried to release my app and still needed 1 retry... but at least it was only one

I believe that Google has fixed its issue.  Personally, I am leaving the retry logic in place for a while.

Hi
I'm encoutering this issue while using create_app_on_managed_play_store, I did try with five retries without success.
Do you have any updates ?
Here is my lane config
desc "Create the private app on the Google Play store"
lane :create_private_app do
  gradle(
    task: 'assemble',
    build_type: 'Release'
  )

  CredentialsManager::AppfileConfig
  sa = CredentialsManager::AppfileConfig.try_fetch_value(:json_key_file)
  id = CredentialsManager::AppfileConfig.try_fetch_value(:team_id)

  create_app_on_managed_play_store(
    json_key: sa,
    developer_account_id: id,
    app_title: 'MyApp',
    language: 'en_US',
    apk: lane_context[SharedValues::GRADLE_APK_OUTPUT_PATH]
  )
end

Hi, do you have any updates ?
I retried today with the same errror. I used my service account to update an existing private app and it worked. It really seems to only affect create_app_on_managed_play_store.

Any Updates on this? This is still an ongoing issue.

Hello,
I'm still facing this issue only with "create_app_on_managed_play_store" lane.
Any update?

Hi, still having this issue today
Any updates ?

Is there any success for create_app_on_managed_play_store

Has it worked for anyone?

I have posted the problem to various community pages
https://www.googlecloudcommunity.com/gc/Developer-Tools/Google-Play-Custom-App-Publishing-API-401-Error/m-p/763409#M2474
https://stackoverflow.com/questions/78592337/cannot-upload-private-app-with-google-play-custom-app-publishing-api-401-unaut
Also asked to Google Play Developer support, but I could not get any answers. If you are interested in this issue please support my entries on stackoverflow and googlecloudcommunity

We had this issue and for us it helped to set the environment variable SUPPLY_UPLOAD_MAX_RETRIES to something between 5-10.

authors: AliSoftware AlkanV FranciscoPDNeto JaredMKatz JoaoBGusmao MisterJack49 NabeelZanabeet Pipding SaiBhargav30393 YaserH25 adamplenti ademirqueiroga agap androideveloper antonengelhardt banane barisinanc bimusiek brianjd canturkay capyvara christian-hess-94 chrisvire denissb dhimanrascon dlorddd dsyrstad ecorreyu fabiendem fotiDim fxcjahid giona69 hakunatomata hemachandsai humanzai imtoori jesusamaro25 joao-albach joarkosberg johnbizokk jonatasteixeira lostdev marcosrocha85 matthewparavati mirkos93 muhammadqazi musabagab mvrty nicholaszuccarelli paladin952 pfay-fc phil-hudson rmdvector scott-hill-alkami shekit soulduse sregg tijojose27 vmontana wxxedu

repo: fastlane/fastlane


text:
Docs stopped getting updates on releases

New Regression Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Regression Information

Breaking version: 2.221.0+
Last working version: 2.220.0

Regression Description
Since moving version bumps to GH Releases automation, documentation updates are not being pushed (their PR opened) against fastlane/docs [list]

"Is it somehow possible that these new bump lanes now avoid the release lane, which is the only place to run :update_docs and it's not being pushed now?"

— Originally posted by @janbrasna in https://github.com/fastlane/fastlane/pull/22090#discussion_r1660308917

"I'm trying to trace the Fastfile lanes to when they're being called (:release to :update_docs) but with not much luck, it only seems as the new bump logic somehow avoids these calls?

basically here https://github.com/fastlane/fastlane/pull/22080#issuecomment-2159671381 is mentioned the moving to GHA, but I can't seem to locate some bigger epic tracking what's being done, what's done, what's known to be missing for now etc., so can't tell if it's a known issue, or is supposed to be run manually somehow etc.?
but it feels that in the new automation this ought to be called upon cutting a Release on GH, so that'd be perhaps during :create_github_release?"

— Originally posted by @janbrasna in https://github.com/fastlane/fastlane/pull/22090#discussion_r1676834000
Environment
GHA/CI

THis is cool!!! What you did?

source: gharchive/issue

license: MIT

license_type: permissive

license_source: github-api


text:
Supporting activesupport  5

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I have read the Contribution Guidelines

Issue Description
I develop application incorpolated with fastlane, and that app needs activesupport ver 5.
But now that gem in fastlane is locked version < 5 (ref #5294 , #5298 ).
I need conditioning by RUBY_VERSION.

Perhaps @neonichu can provide a workaround, per https://github.com/fastlane/fastlane/pull/5298

Unfortunately, I don't think this can really be worked around effectively. Using RUBY_VERSION does not work, see this comment for more information.
IIRC, @segiddins was working on handling this better in rubygems, but even that would not help as long as the goal is to support macOS system ruby, because that also ships with an old version of rubygems.

I've only been working on fixing it in Bundler.

@segiddins that's actually better, because Bundler doesn't ship with system Ruby, does it?

Not yet

Thanks! @segiddins Keep us updated 👍

Is there any progress on this? Or which thread in Bundler can track this?

This is implemented on bundler master, will go out in bundler 1.14

@segiddins 👍

@segiddins Awesome! 👍

bundler v1.14.0.pre.1 was released, can we test it?

I can test it, but don't know how to test.

authors: TKBurner chrhsmt cpunion neonichu segiddins vpolouchkine


text:
[gym] Detect true build configuration for archive action

Checklist

[x] I've run bundle exec rspec from the root directory.
[x] I've followed the fastlane code style and run bundle exec rubocop -a.
[x] I've read the Contribution Guidelines.
[x] I've updated the documentation if necessary.

Motivation and Context

If no build configuration is specified and -scheme is not passed then [whatever is set as default configuration for command-line builds] is used.
— Technical Note TN2339: Building from the Command Line with Xcode FAQ, Listing 5

Currently, Gym doesn't work like that when detecting the provisioning profile mapping. It actually goes straight to the default configuration for command-line builds even if a scheme is given.
This is unexpected behavior for projects which specify different bundle identifiers and code signing settings per scheme – as Gym, unlike Xcode, fails to find the proper provisioning profile mapping.
This pull request fixes that issue and makes Gym behave in the same way as Xcode.
Description
In short, Gym now checks what configuration is set on the scheme used to archive the app, instead of just falling back directly to the "default for command-line builds" or, what's worse, to the first one it finds.
It does so by loading the scheme file using Xcodeproj::XCScheme and extracting the values from it. If no scheme was given or if there's a problem loading it, it falls back to existing behavior.
A new test case has been added to make sure the new behavior works as expected.

@akashivskyy Heyyyy 👋  We got that new release from xcodeproj for my fix in https://github.com/CocoaPods/Xcodeproj/pull/656 🚀  Do you mind reviewing / testing this out to make sure this works like you intended? 😊 Then we can ship this!

@joshdholtz I confirm that everything works as expected. :shipit:

source: gharchive/pull-request

authors: akashivskyy joshdholtz


text:
[action] setup_ci – always setup keychain when running on a CI

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
Currently setup_ci must be passed a valid :provider (either travis or circleci) that controls which setup is performed for each of them. However, the only difference between the providers is that passing circleci also configures the output paths, while the temporary keychain is created for both of them.
As pointed out in #15445, running setup_ci in GitHub Actions without passing one of the valid providers currently results in a noop, even though the environment is recognized as being a CI. However, one would expect that the action also works on other CI providers and e.g. creates a temporary keychain.
Description
Based on the @tlvenn's proposal put forward in #15445 I changed setup_ci to always create a temporary keychain, but only after performing any extra tasks necessary for a specific CI provider.
Furthermore the CI provider is now detected automatically, if no :provider is passed. I left the validation of a passed :provider in place to ensure backwards compatibility, even though passing travis does  not change the behavior of the action compared to not passing any :provider.
To not having to duplicate parts of the CI detection I added a new method Helper.is_circle_ci, that is used both in Helper.ci? and setup_ci.

@googlebot I signed it!

@googlebot I signed it!

That didn't work it seems. Usually this is because your GitHub account uses a different email address or the commits were made using a different email address.

That didn't work it seems. Usually this is because your GitHub account uses a different email address or the commits were made using a different email address.

I think this is due to the company CLA we signed last week. The verification process has not yet completed.

@googlebot I signed it!

authors: janpio svenmuennich


text:
[spaceship] Initial support for updated review submissions flow

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
Apple is introducing a new submission workflow on App Store Connect. Once an account is updated to follow the new flow, Fastlane's current submission methods will not work. This PR contains the initial work done for adding support for new concepts like ReviewSubmission-s and ReviewSubmissionItem-s.
While these endpoints have not been made public from Apple's side, the work done in this PR follows the observed HTTP calls that are made when creating a submission on App Store Connect.
Description
This PR introduces the concepts of ReviewSubmission and ReviewSubmissionItem.
Support is added for:

creating a new review submission
fetching an existing review submission
adding items to an existing review submission (only the App Store version, for now)
submitting a review submission
cancelling a review submission

The App Store Connect API supports additional concepts, states and operations which have not been included in this PR. In other words, this PR does not aim to provide an exhaustive coverage of the functionalities connected to the new submission flow; rather, its goal is to enable the development of a version of Deliver that is compliant with the new submission flow and that can be used when Apple eventually pulls the trigger and enforces it on all developer accounts.
Note that these new classes have not been integrated in Deliver yet. This was a conscious choice, as the new submission flow allows users to batch contributions of different kinds (an app version update, a new custom product page, in-app events, etc.); assuming that the submission happens instantaneously as it does now would therefore have been a very opinionated choice, which I think should be taken by the Fastlane community at large.
Rubocop is currently failing due to the increased length of the App class caused by the addition of the new submission methods. While I would tentatively argue that such issue could be resolved by re-issuing the rubocop file, I figured it would have been best to hear the community's opinion on the matter.
Testing Steps

The PR includes unit tests that verify the correctness of the methods and that are executed with the usual bundle exec rspec command. In addition, the flow has been tested locally against my company's App Store Connect account using a slightly twisted (and opinionated) version of Deliver.

Thanks for the review, @joshdholtz! I added the two comment lines to the App class, the linter should be happy now. 👍

@joshdholtz Just a heads up that Apple plans to start enforcing the new submission flow from January 25th, so Deliver will most likely start to fail (at least for some people) unless we integrate this new logic. This is what they say on App Store Connect:
Last year, we introduced an updated way to submit apps, in-app events, custom product pages, and product page optimization tests for review. Starting January 25, 2022, the submission experience will be automatically updated for all App Store Connect accounts.

I don't know what the best approach would be to support this within Deliver, especially if we consider that the new submission flow makes it possible to have "asynchronous" or "deferred" submissions (you can create the submission, add items to it, and submit the items in bulk) – which may be against the way Deliver has traditionally operated.
I hope that by letting you know in advance (if you're not already aware of it) we may be able to smoothen the transition when January 25th comes 🙂

@valerio-castelli I have this started in https://github.com/fastlane/fastlane/pull/19838 😊 I'm going to be finishing it up tomorrow and then validate it whenever the App Store Connect API gets the official support for this 💪

authors: joshdholtz valerio-castelli


text:
[WIP] Stub out env vars that are being set by CircleCI

CircleCI is now setting some environment variables for match to handle issues in macOS Sierra. This PR stubs out those values to make sure the ones we need are getting returned.

          1 Warning

:warning:
Pull Request is Work in Progress

  Generated by :no_entry_sign: Danger

authors: fastlane-bot-helper ohayon


text:
fsthttp, internal: add get_addr_dest_ip/port hostcalls

Fixes #121

Not sure I like how these hostcalls are exposed, but the hostcalls themselves work.

Can you add a RemoteAddr() method to return a net.Addr instead of exposing host and port as fields on Response?

Agree with other comments that this should be a method to return a friendly standard Go type.

Consider exposing this as a net/netip.AddrPort instead of or in addition to a net.Addr

API fixed. PTAL.

authors: cee-dub dgryski joeshaw ydnar

repo: fastly/compute-sdk-go


text:
WAF efficacy score discussion

The WAF efficacy score seems to work not as good as expected.
Imaging having a WAF which does not filter anything. All XSS payloads are not detected and all true negatives are passed, too.
This would result in an efficacy score of 50% as seen below:
-------------XSS-------------
True Positives 0
False Negatives 66
True Negatives 2
False Positives 0
Efficacy 50.0%

According to the efficacy score one could say "the WAF is 50% effective in providing protection against XSS attacks." when in reality, the WAF does not provide any protection against XSS attacks.
Same applies to a WAF which just blocks everything:
-------------XSS-------------
True Positives 12
False Negatives 0
True Negatives 0
False Positives 34
Efficacy 50.0%

Again 50% efficacy, while in reality this would be completely useless.

Balanced efficacy is a better-than-nothing overall number.
We can print out a breakdown of positive and negative efficacy so people can drill down past the overall number, but really the better answer may be to include lots more test vectors so we're nowhere near zero on any particular number.

I totally understand that you have to use a lot of test data to achieve good results, but in these two (admittedly rather unrealistic) special cases where the WAF does not filter anything or just blocks everything, the dataset won't matter. The efficacy will always be 50%.
What I am trying to say is, that in such cases there should either be a warning - like: "No WAF in use" or "WAF seems to block everything" or something similar - or the efficacy should be 0%, because for obvious reasons.

I agree with your concern about providing some type of waring. In general 50% is a failing score and would likely indicate that your WAF is misconfigured or you need to take a closer look at the test data. The problem with providing a general statement of "No WAF in use" is that you can still receive a 50% without non-zero values.
-------------XSS-------------
True Positives 4
False Negatives 26
True Negatives 26
False Positives 4
Efficacy 50.0%
------------- WAF Efficacy -------------
50.0%

Indeed a warning would be good in this case.

authors: dkegel-fastly skhalsa-sigsci whoot

repo: fastly/wafefficacy


text:
Improve "reserve button" texts

See mockups page 9.

Closed in #399.

repo: fastmonkeys/respa-ui

license: mit

license_source: bigquery


text:
What is the ASCII-thing in the README.md supposed to depict?

Given that googling "web app shelf" or whatever arrangement of similar words does not help understanding what a "collapsible left shelf" is, would it be possible to see a couple of screenshots of a coffin in the README?
I believe nesting a pic costs 100X less time than weaving that incomprehensible ASCII-artwork. Thank you for your attention.

it's a coffin, +1 for ascii art. You can also click the link on top of project http://fat.github.com/coffin/ it's demonstrated there

authors: marco-faustinelli wires

repo: fat/coffin


text:
Vragen en best practices voor deze sprint

Beschrijving
Ik heb deze issue aangemaakt met vragen over toegankelijkheid, omdat dit een nieuw concept voor mij is en ik hierover verschillende meningen hoor. Ik heb geprobeerd om zelf wat informatie op te zoeken en heb al wat basiskennis, maar ik wil er zeker van zijn dat ik de beste methoden begrijp en toepas. Daarom stel ik deze vragen om verdere verduidelijking te krijgen, zodat ik deze sprint de juiste aanpak kan gebruiken voor toegankelijkheid.
vragen
Ik begrijp dat het aanbevolen is om aria-labels zoveel mogelijk te vermijden en te focussen op semantische HTML. In sommige situaties is het echter moeilijk om volledig semantisch te werken, bijvoorbeeld wanneer elementen zoals <div> en <span> nodig zijn. Mag ik in deze gevallen aria-labels gebruiken om de toegankelijkheid te verbeteren, of is het gebruik van aria-labels zelfs dan nog steeds afgeraden? En zijn er specifieke richtlijnen over wanneer aria-labels wel of niet gepast zijn?
Ik begrijp dat je zowel aria-labels als visually-hidden kunt gebruiken om extra informatie toe te voegen voor screen readers. Met visually-hidden kun je beschrijvingen zichtbaar houden in de code, tenzij ze worden verborgen met CSS. Ik ben nog niet helemaal zeker waarom het handig kan zijn om beschrijvingen op deze manier in de code te houden. Hoe weet ik wanneer ik beter aria-labels kan gebruiken en wanneer visually-hidden beter werkt? Zijn er situaties waarin de ene optie beter is voor toegankelijkheid dan de andere?
Ik heb begrepen dat een skiplink hyperlink wordt gebruikt om gebruikers sneller door een webpagina te laten navigeren, bijvoorbeeld door direct naar de hoofdinhoud te springen. Is het in de praktijk echt nuttig om een skiplink toe te voegen? En is het aan te raden om dit te gebruiken, ook al is het niet verplicht volgens de WCAG-richtlijnen?
Ik begrijp dat het gebruik van tabindex="1" en andere specifieke waarden voor tabindex wordt afgeraden, omdat dit de navigatie verwarrend kan maken. Ik weet dat je bij semantische HTML meestal geen tabindex nodig hebt, maar hoe zit het met niet-semantische elementen? Kan ik daar tabindex="0" gebruiken, of wordt dat ook afgeraden? Zijn er alternatieven die beter werken voor toegankelijkheid?
Ik heb een vraag over het gebruik van een breadcrumb trail. Is het aanbevolen om een breadcrumb trail te gebruiken voor de navigatie op een website, en zo ja, in welke situaties is dit het meest nuttig voor de gebruiker?
We gaan in deze sprint voor de opdrachtgever een contactformulier maken. Is het gebruik van aria-describedby en het for-attribuut in formulier-elementen verplicht of wordt het vooral aangeraden? Wanneer en hoe kunnen we deze attributen het beste gebruiken om de toegankelijkheid van het formulier te verbeteren?
Ik zou graag wat bronnen willen ontvangen om meer informatie te vinden. Dit zijn mijn basisvragen over toegankelijkheid, maar ik verwacht in de toekomst meer vragen te hebben.

@joostf, Hoi Joost, zou u graag naar deze issue willen kijken? Ik heb hier een aantal vragen over toegankelijkheid geplaatst en zou graag uw feedback ontvangen. Alvast dank!

@fatimahilali wat een goede vragen!
Ik zou je adviseren om per vraag een issue te maken, vervolgens de vragen eerst zelf onderzoeken (en dus zelf bronnen te zoeken). Je kan de  bevindingen per vraag/issue in de comments van de betreffende issue zetten. Dan kan ik of een van de andere docenten je bronnen en bevindingen valideren!
Goed idee?

@joostf Hoi Joost, bedankt voor uw feedback! Goed plan. Ik zal de vragen apart als issues aanmaken en zelf online onderzoeken. Per issue deel ik mijn bevindingen, zodat u of iemand anders ze kan beoordelen.ik ga ermee aan de slag!

@joostf Welke docenten kan ik het beste assignen voor deze issues? Koop, Krijn of Dion? Of kan ik het beter eerst klassikaal bespreken en daarna een issue aanmaken en de docent daaraan assignen?

authors: fatimahilali joostf

repo: fatimahilali/the-client-website


text:
Allow creating and adding pygments.styles

Instructions on how to create styles in python are given here: https://pygments.org/docs/styledevelopment/#creating-own-styles
Loading styles into pygments requires tinkering with setuptools, see https://pygments.org/docs/plugins/
I managed to get it to work in python.
It remains to figure out how to load it into PyCall, since the new styles have to be installed as a plugin using setuptools.
Open questions:

Updating a style file requires to rerun the installation of the plugin. Can we do this from within the same julia session using PyCall or do we have to restart the REPL everytime we change the style?
Can we provide an environment variable and/or default directory in which setuptools can look for style files?

There is a discussion going on about alternative ways to register custom lexers or styles in pygments: https://github.com/pygments/pygments/issues/1096


text:
能否用 免费的Gemini Flash 代替 Gemini Pro 做图片等的总结？

发现sum4all的代码(如下)中是使用gemini-pro-vision做图片识别的，能否用免费的Gemini Flash 1.5 代替 pro 呢？
        elif self.image_sum_service == "gemini":             api_key = self.gemini_key             api_base = "https://gemini.sum4all.site/v1/models/gemini-pro-vision:generateContent"

Gemini 1.5 Flash不是免费的吧？

我更新为Gemini 1.5 Flash了

我更新为Gemini 1.5 Flash了

个人使用完全是够的吧

对

authors: CatByteDance bingjiw fatwang2


text:
load_pretrained - BASE_URL 404 Not Found

I want to see models by using load model weights for a known configuration (BASE_URL), as commented.
But this Page is Not Found https://github.com/faustomorales/vit-keras/releases/download/dl.
is there any issue?

I was able to download all four models today (b16, b32, l16 and l32).

I have similar issue.
I try to load Model ViT-B_16-224.npz from google cloud by using the URL
[https://github.com/faustomorales/vit-keras/releases/download/dl/ViT-B_16-224_imagenet21k+imagenet2012.npz](https://github.com/faustomorales/vit-keras/releases/download/dl/ViT-B_16-224_imagenet21k+imagenet2012.npz
but it doesn't work.
it would be nice if there were also a support for a tiny model  ViT-T_...
regards

Closing because the original issue appeared to be transient, as confirmed by a later comment showing the download worked.

authors: StephanElter apardogi faustomorales nattafahhm

repo: faustomorales/vit-keras


text:
load from local file

As I understand, epubjs allow to get book from bytesArray

There is a fork here with an interesting option to load local files:
https://github.com/polymorph001/epub_viewer
but is not completed yet because I can't read the chapters from that fork and the callbacks are no working well

There is a fork here with an interesting option to load local files:
https://github.com/polymorph001/epub_viewer
but is not completed yet because I can't read the chapters from that fork and the callbacks are no working well

This seems to be a good approch

authors: fayis672 pandora-house rodrigocanepa


text:
ignore certain types of errors

Adds ability to ignore errors you do not want to log

And what about tests for this feature ?

What about test for this feature ?

Sorry, no idea how to test plug. any advices/examples?

Thanks, implemented. Testing plug is still the issue, naive approach didn't work for me

@fazibear any updates here?

Can we get this merged?

authors: fazibear joshuaswilcox thousandsofthem

repo: fazibear/airbrakex


text:
accuracy stop improving in simple convolution network

I've been using keras to build convolution neural networks for binary classification. Using the same input data, I've tried to vary the model structure (i.e. filter size, number of filters, number of hidden layer neurons) for better performance. I noticed that for certain models, the training accuracy remains unchanged at a low value through all 50 training epochs. One of the model structure is as follows:

  n_filters=32
  n_conv_row=n_conv_col=400
  weight_init='he_normal'
  img_rows=img_cols=400
  nn_dense=25
  n_classes=2
  loss_func='categorical_crossentropy'
  model = Sequential()
  model.add(Convolution2D(n_filters, n_conv_row, n_conv_col,
                          border_mode='valid',init=weight_init,
                          input_shape=(1, img_rows, img_cols)))
  model.add(Activation('relu'))
  model.add(Flatten())
  model.add(Dense(nn_dense, init=weight_init))
  model.add(Activation('relu'))
  model.add(Dense(n_classes, init=weight_init))
  model.add(Activation('softmax'))
  model.compile(loss=loss_func, optimizer='sgd')

I used the whole training set as validation so I can get the training accuracy at the end of each epoch. The output after each epoch looks like:
259s - loss: 10.4525 - acc: 0.4903 - val_loss: 11.4543 - val_acc: 0.4473
269s - loss: 11.4543 - acc: 0.4473 - val_loss: 11.4543 - val_acc: 0.4473
251s - loss: 11.4543 - acc: 0.4473 - val_loss: 11.4543 - val_acc: 0.4473
252s - loss: 11.4543 - acc: 0.4473 - val_loss: 11.4543 - val_acc: 0.4473
...

I plotted the filter weights of the convolution layer at each epoch and they stayed the same for most of the epochs and had minor changes rarely.
at the beginning of epoch 1:

at the beginning of epoch 3:

at the beginning of epoch 5:

I also tried rmsprop as optimizer, and the accuracy at each epoch was exactly the same 0.4473 . However the filter weights formed different pattern at the end of epoch 1 but remained unchanged after that.
What may I have missed? I tried this model structure directly with Theano and the problem persisted, so I guess this problem was not from keras.
To identify the cause of this problem, I think it would be helpful to track the activation and gradient values of each mini-batch. How do I do these with Keras or Theano? Or what would be the right direction to look into?
Thanks a lot.
Cheng

are you using default learning rates?

@EderSantana In the above code I used the default learning rate for SGD which is 0.01. I've also suspected this might cause the problem, so I tried 0.1, 0.3, 0.9 and got slight different patterns in the filters but the same unchanged validation accuracy. As I mentioned I also tried rmsprop with default parameters, only to get the same accuracy.

I think your learning rates are too large, actually... and try some dropout. Maybe you are getting stuck in high eigenvalue spread regions (narrow valleys where you move a lot without changing much in the overall cost function). How large and correlated is your input data?

wait, are you using a filters of size 400?

@EderSantana Thanks a lot for your advice. I tried using learning rate 0.0001 and 0.0003 and the validation accuracy improved over 30 epochs. Now I have validation accuracy (actually training accuracy on the whole data set) at around 0.6 . It seems that the problem was really caused by a large learning rate.
It's possible that the model got into a valley within a few mini-batches due to large learning rate. Is there a way that I can confirm and/or avoid this?
I've not checked my input data for correlation yet. Can I do this by unrolling the input images into vectors and test for correlation between these vectors? What should I do about correlated inputs? Maybe cluster them by correlation and pick one from each cluster?
The filter size is equal to my input image size, i.e. 400 by 400. My input 'images' are actually not images but grid of values with each position indicating certain feature. So I hope to fix the filter position without striding, and let them learn patterns from my input. I understand this may not be a legitimate way to use convolution networks. I'm trying to run with filter size at 50 by 50, but still trying to figure out a way for them to work with my limited computation resources.

using a filter of the size of the image is what we mean by fully connected layer. It is much faster. Also, if you want convolutions, try 3x3 and stuff like that... check lenet5 architecture
If you have images and the neighboring pixels are correlated like in regular images you will probably have correlations issues, that was why batch normalization was invented. Give Keras's BatchNormalization layer a try. It seems to be working correctly now.

@EderSantana Thanks for the comments. I understand the fully connected layer part, a filter with fixed position is indeed a hidden layer neuron. I'll try some different filter sizes.
I tried adding batch normalization layer after each layer in my model, only after the convolution layer, and only after the fully connected layer. It seems only the model with batch normalization after each layer allows gradually improving of accuracy over 50 epochs. However the speed of improvement seems slower than a model with correct learning rate (e.g. 0.0003) and no batch normalization. Is it supposed to perform like this? Thanks.

nah, its supposed to learn faster with batch normalization, maybe with it you can try a learning rates that are a little bit larger

@EderSantana Yes, I'm able to use a learning rate of 0.001 with batch normalization for faster training. Thanks for all the help.

awesome! good to hear that
don't forget to close the issue when you are done.

I trying to run a CNN for image classification in CIFAR, but accuracy stops improving after 10% This is my model.
model=Sequential()
model.add(Convolution2D(32,3,3,border_mode='same',input_shape=(3,32,32)))
model.add(Activation('relu'))
model.add(Convolution2D(32,3,3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(.25))
model.add(Flatten())
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(.5))
model.add(Dense(10))
model.add(Activation('softmax'))
sgd = SGD(lr=0.003, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',optimizer=sgd)
model.fit(train,y_train,nb_epoch=15,show_accuracy=True,verbose=1,validation_data=(test,y_test))
I have tried varying the learning rate also. Nothing seems to work.
Train - > (40000,3,32,32)
Test - > (10000,3,32,32)
y_train -> (40000,10) # the classes have been binarized
y_test -> (10000,10) # the classes have been binarized

did you try adam instead of sgd with decay? That model should not be too bad, I think...

I'm trying to use the vgg19 architectures for my model but the validation accuracy I got is constant value around 56% and I used too many values of the learning rate such as 0.1, 0.01, 0.001,0.0001 and all of them gave the same result

How did you plot the filter weights of convolutional layer.

I have a trained model file with three convolutional layer and another model file with 2 convolution and 1 lstm layer. I want to plot the filter weights of each layer. How to do this.

I am not that much familiar with python, so i am using MATLAB for CNN training. I have a data set of 27,000 images and angles corresponding to that images.
My sample code is :
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
layers = [imageInputLayer([32 32 1])
convolution2dLayer(5,20)
reluLayer()
maxPooling2dLayer(2,'Stride',2)
fullyConnectedLayer(size(categories(trainAngle)))
softmaxLayer
classificationLayer];
options = trainingOptions('sgdm', 'MaxEpochs', 20, ...
'InitialLearnRate', 0.0003);
convnet = trainNetwork(trainZ, trainAngle, layers,options);
resultant_Train = classify(convnet,trainZ); %Training data
resultant_Valid = classify(convnet,validZ); %Validation data
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
My training accuracy is 70%
but test accuracy is only 2%;
I am completely blank what to do next. Do you have any suggestion? How can I improve my test accuracy?

This is a perfect example of overfitting - try using dropout layers, data augmentation, adding noise, or make the model simpler (it is memorizing your own training data).

I had a same issue. When I was training the CNN, the val_acc never updated. And there was minor updates in acc. but it seemed to vibrate. At last, I found the solution.
The issue was in my training dataset. The size was about 4000. And the size of validation dataset was 500
I thought that this issue came from the network weight initialization in flatten layer.
So I changed the dataset and tested. The result was tremendous.
This was my first training part.
model.fit_generator(
        train_data,
        steps_per_epoch=4091 // 32,
        epochs=100,
        validation_data=test_data,
        validation_steps=474 // 32,
        verbose=1,
        workers=4)

I changed it as follows:
model.fit_generator(
        test_data,
        steps_per_epoch=474 // 32,
        epochs=25,
        verbose=1,
        workers=4)

And for the next training, I saved the model weights and reused for next training.
model.save_weights('weight_model.h5')
And initialized network weights with this:
# Load model weights
model.load_weights('weight_model.h5')

# Compiling the ANN
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

This worked perfectly. I hope this will be helpful. Thanks.

@Paulraita  Hey! I'm having the same problem that you described. But I don't fully understand your solution. Did you run both codes one after the other? or directly trained with your test_data?
I mean, is this what you run?:
model.fit_generator(
train_data,
steps_per_epoch=4091 // 32,
epochs=100,
validation_data=test_data,
validation_steps=474 // 32,
verbose=1,
workers=4)
model.fit_generator(
test_data,
steps_per_epoch=474 // 32,
epochs=25,
verbose=1,
workers=4)

Or did you only fit the test_data at first? I mean, just run:
model.fit_generator(
test_data,
steps_per_epoch=474 // 32,
epochs=25,
verbose=1,
workers=4)
Thank you very much!

Hello @cmosquer
first, I started with small dataset. and saved the model.
I increased the dataset size gradually. In this way, I could overcome this issue.
I hope this will be helpful for you.

@gauravgola96
I checked its validation accuracy with separate dataset.
For some solutions, it could be helpful to start from small size of dataset.
And we can't solve all the kind of issues like this with the method I used before.
There are many solutions to overcome this. For example, in my case, batch normalization was one of the solutions. And for your last question, I never changed it. Nowadays I often use callback to monitor the training process. Did you try to change the learning rate?

@Paulraita
Yes, I have changed the learning rate and also used ReduceLROnPlateau callback as well.
When trained my inceptionv3 with first 90 layers freezed and trained it on small dataset of 350 images from 12 classes.I used the weights (trained on small dataset) for training the large dataset afterwards on same network and with same number of freezed layers. The training accuracy increased very much but the validation accuracy still able to cross 60%.
Train data - 7200 images from 12 classes (balanced)
Test data - 350 images from 12 classes

authors: EderSantana Paulraita cmosquer fatemaaa gauravgola96 nhulkund ravishk1 vinayakumarr vsuriya93 zhangchengcsb

repo: fchollet/keras


text:
Shouldn't the dimensionality of output metric  kb-divergence be 1?

Isn't the kullback_leibler_divergence method in metrics supposed to return a scalar value? Right now it doesn't:
def kullback_leibler_divergence(y_true, y_pred):
    '''Calculates the Kullback-Leibler (KL) divergence between prediction
    and target values.
    '''
    y_true = K.clip(y_true, K.epsilon(), 1)
    y_pred = K.clip(y_pred, K.epsilon(), 1)
    return K.sum(y_true * K.log(y_true / y_pred), axis=-1)

Maybe the last line should be
return K.mean(K.sum(y_true * K.log(y_true / y_pred), axis=-1))

(or with axis=-1 as extra argument in objectives.py)


text:
A way to exclude certain files?

A switch that allows you to skip files, either for Add, List or Extract.
I used "-not" but it keeps adding the files.

Please use " (in particular on linux) and * as wildcard
Multiple -not can be used

Before I had used:
-not *.7z
and it kept adding the 7z file, but using the " worked perfectly, I think you should document it in the help, including clarifying the use of quotes.

*NIX AND DOUBLE QUOTES
Please, on non-Windows systems, DO NOT FORGET THE DOUBLE QUOTES,
especially with multipart files.
"test_????.zpaq" is good
test_????.zpaq   is BAD
I'll try to write even more clearly 😄

What do you think about this disclaimer?
Any suggestions?
Too long?
A key note is the use of double quotes, especially on *nix systems,
and even more so when using wildcard characters such as * and ?

Translation

When command-line strings are entered, there are differences between
Windows and *nix.

On the latter, the "explosion" of wildcards normally takes place
BEFORE programs are started.

This happens because the relevant shells (bash, etc.) process parameters
and pass them to the program (zpaqfranz in this case) already expanded.
Enclosing characters in double quotes preserves the literal value of all
characters within the quotes, with the exception of $, `, \

Actually, the situation is much more complex, but I don't want to go
into too much detail.

In Windows, however, the parameters are passed to the program as they are.

Therefore, sometimes, you write on *nix as you would on Windows,
getting different results or even wrong one.

A few examples I hope will better clarify the fundamental importance of
appropriately using the " on *nix

"test_????.zpaq" is good
test_????.zpaq   is BAD

"*francia*"      is good
*francia*        is BAD

This is OK on Windows, but not on Linux
zpaqfranz a z:\test_??? * -only *.cpp

You have to write something like that
zpaqfranz a "/tmp/test_???" /whatever -only "*.cpp"

This is good
zfsadd /tmp/kong.zpaq "tank/d@2021" "--60d"

There is a bit of heuristics inside zpaqfranz to make some commands
run on *nix with the same logic as on Windows but It is partial,
precisely because the   parameters arrive "pre-digested" on *nix.

Short version: Luke, in addition to using the Force, uses double quotes!

Now, after each help, there is a disclaimer
root@aserver:/tmp/kazzo # zpaqfranz h f
zpaqfranz v58.10i-JIT-L,HW SHA1/2,(2023-09-14)
CMD   f (fill, or wipe)
+ :               Fill (wipe) 99% of free disk space in 500MB chunks
                  Check if disk-controller-system-RAM-cache-cables are working fine
+ : -verbose      Show write speed (useful to check speed consistency)
+ : -force        Do NOT delete (after run) the temporary filename. By default free
+ : -zero         Zero-fill instead of random. Use to prepare a thin VMDK shrink
+ : -verify       For -zero: do a verify.
    Examples:
Fill (wipe) almost all free space:   f z:\
Fill (wipe) keep temp files:         f z:\ -force -verbose
Zero free space (VM shrink):         f z:\ -zero
Zero free space (WITH verify):       f z:\ -zero -verify
--------------------------------------------------------------------------------------
************ REMEMBER TO USE DOUBLE QUOTES! ************
*** -not *.cpp    is bad,    -not "*.cpp"    is good ***
*** test_???.zpaq is bad,    "test_???.zpaq" is good ***

*NIX AND DOUBLE QUOTES Please, on non-Windows systems, DO NOT FORGET THE DOUBLE QUOTES, especially with multipart files. "test_????.zpaq" is good test_????.zpaq is BAD
I'll try to write even more clearly 😄

I use Windows, I also needed to use "

A long explanation would be necessary, so I made it short: " for all
😭

Can you please take a look here?
https://github.com/fcorbelli/zpaqfranz/wiki
Do you have any suggestions?
Thank you

repo: fcorbelli/zpaqfranz


text:
Added PAX Exam to testing framework

See: https://jira.duraspace.org/browse/FCREPO-1897

Moved features.xml provisioning into its own module
Added PAX-Exam to testing framework
Updated documentation

Does this PR render the existing https://github.com/fcrepo4-labs/fcrepo-karaf project obsolete?

Not at all. fcrepo-karaf is for provisioning the Fedora artifacts, which are independent of the camel toolbox

The naming conflict introduces confusion. How do you feel about renaming this current module?

I am not convinced that there will be confusion, as these use different groupIds, but I have renamed the fcrepo-karaf module to toolbox-features, following another convention for feature repositories that I'm familiar with.

Resolved with: https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/commit/8523ac995c6e0ab9e7ba40582b11b397b1ef2b0e

authors: acoburn awoods


text:
added better IT tests for binary content

This partially addresses https://jira.duraspace.org/browse/FCREPO-1269
This commit added better IT tests for binaries and containers
It also adds much imporved integration tests with an external triplestore and an external Solr instance
renamed Namespaces class so as not to conflict with the o.a.c.builder.xml.Namespaces class
What remains for the JIRA issue is some OSGi testing.

Resolved with: https://github.com/fcrepo4-labs/fcrepo-camel/commit/a361ca163f4ee6a910f1fde9f72f879f0d4e123f


text:
remove fedora:uuid

add created = fedora:created :: xsd:dateTime;
lastModified = fedora:lastModified :: xsd:dateTime;
hasParent = fedora:hasParent :: xsd:string;
Resolves: https://jira.duraspace.org/browse/FCREPO-1552

Merged via 3908141a1879df03ef1fa5de4a0672644f447569

authors: acoburn yinlinchen


text:
server bug

i make 2 fd project,a typescript,and a createjs.
i run ts is the typescript.html.
and i run createjs proj,also the ts project.
i must clear the chrome history?

What version of FD? How do you run the project? Using Webserver Test Movie option?

version 92beb1f3eb
i run the project use f5. the option just the same new typescript project.
and the ts project ,i change the  the main.ts,it is work.but i change the project,it can not work.
and i use http://xxxxxxx/index.html?v=1.
add a diff verion "v=1".it is work.

The webserver has a fixed IP so the first server started wins. FD will keep the server running until you close the project.

can not use ,i have restart my os.
i think it is a browse Cache iussee

authors: Meychi elsassph matrix3d

repo: fdorg/flashdevelop


text:
Fix patch method with complex query

Hi,
This change allow patch method with a query like { $and : [ {x: 3 } ] }] and data like { x: 4 }
Without this change, a "NotFound" error is thrown.
Corresponding test also added.

Thanks @Thomas-git , released in v7.3.0

authors: Thomas-git dekelev

repo: feathersjs-ecosystem/feathers-objection


text:
add params to get requests

Summary
(If you have not already please refer to the contributing guideline as described
here)

[X] Tell us about the problem your pull request is solving.
[ ] Are there any open issues that are related to this?
[ ] Is this PR dependent on PRs in other repos?

Other Information
Stripe has the option to expand fields i.e on a customer
expand: ['invoice_settings.default_payment_method']

but currently the get requests don't accept params so this isn't possible. This PR will resolve this.

hello @dottodot this PR is still open so your adds are not implemented for the moment, am i right ?
I passed a couple of hours trying to make the expand property work, until i found this PR .... Any other solutions have been found ?
Thanks

I have made some updates in PR: https://github.com/feathersjs-ecosystem/feathers-stripe/pull/107
The updates made in that PR may likely clash with these. But, I am happy to update this PR (or open another) to accomplish this. It's definitely a worthwhile addition to the library.

authors: DaddyWarbucks Datebayo350 dottodot

repo: feathersjs-ecosystem/feathers-stripe


text:
Backwards-compatibility doesn't work on Mac

          When I run this locally on a Mac I get:

  = note: ld: framework not found SystemConfiguration
          clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

error: could not compile `fedimint-ldk-node` (lib) due to previous error

Originally posted by @justinmoon in https://github.com/fedimint/fedimint/issues/4215#issuecomment-1922397378

@justinmoon When I've previously bumped into linker errors, it's been a red herring for OOM. Any chance you saw a related OOM?

This was because I was using an old nix shell which didn't include SystemConfiguration. Operator error!

authors: bradleystachurski justinmoon

repo: fedimint/fedimint


text:
Consensus and Api versioning - APIs

Currently rebased on top of #1575 to avoid merge conflict. See only the last commit.
Implement the basic types and module-related primitves to support
versioning.
These are currently not used for anything, and here only for
moving forward with the discussion from https://github.com/fedimint/fedimint/issues/1548.

@elsirion
I haven't really thought through all the details, and I hope exactly that looking at the code will make the vision and discussion more precise.
I agree that in many cases importing multiple versions of module codebase is way better idea than cramming it together glued with bunch of if statements. But then - won't majority of cases be trivial? E.g. no changes?
One thing that is not clarified (in my head and in the code) is if module will have its own counter for consensus versions and then there be a global one for core Fedimint protocol. I guess it has to be both.

I don't think the module consensus version has to be a function of the FM consensus version. Either the module needs to be adapted to work with a newer FM version (likely) then it can change its own version or not, then there is no sense in changing the module version.

Either the module needs to be adapted to work with a newer FM version (likely) then it can change its own version or not, then there is no sense in changing the module version.

That means it is a function of fm consensus version, no?
We at least should aim at not letting users make accidental mistakes.
So
fn version(core: CoreConsensusVersions) -> [ModuleConsensusVersion]

If FM changes consensus from 2 to 3, and module doesn't care then:
module_init.versions(2) == module_version(3)

If it does, then it will have it's own version change and
module_init.versions(2) + 1 == module_version(3)

By having this function we can check for the user that all the modules are compatible with FM consensus changes. Initially just bunch of checks and crashing with an error message explaining it, but in the future we might even support automated scheduled consensus changes where guardians use UI to agree on the consensus change that gets scheduled and "switched into" automatically at the designed epoch.

Maybe I'm wrong here, but in my mind a new consensus version of Fedimint is a new major code version, so the old module wouldn't compile with the new Fedimint anyway. So the
module_init.versions(2).last() + 1 == module_init.module_version(3).first()

case doesn't really exist.
I also don't get what first/last of the returned slice would be. What's the semantic of the returned array? Supported versions? But how would a version be even selected when e.g. processing an output? We don't have version fields in all our consensus relevant structs rn, are we planning on changing that?
So far I assumed we just treat a new consensus version of a module as a completely new module with its own instance id and a related kind and let clients figure out migration. I would like to have a version field in the module kind type for that.

But how would a version be even selected when e.g. processing an output?

Versions are on ModuleGen so what it means is: this module codebase can instantiate module instance for the following module consensus versions. This could be used e.g. in the Admin UI to tell user that their codbase is/isn't compatible with module consensus upgrade guardians are trying to schedule, and prompt them to upgrade their binary.
In the simplest case, a module returns just one version meaning - this code base supports only this module consensus version.

Maybe I'm wrong here, but in my mind a new consensus version of Fedimint is a new major code version, so the old module wouldn't compile with the new Fedimint anyway.

I don't think this is necessary. Initially we probably will tie them together for simplicity, but our module code API surface layer is somewhat narrow, and nicely typed.
We might at some point change some consensus rules which most (all) of the module won't care about, and they won't have to change a single line of code, other than adding the new consensus to their list of supported FM consensus version, purely as a safeguard.
On the other hand, we might want to change some module trait APIs, without actually changing consensus rules.

So far I assumed we just treat a new consensus version of a module as a completely new module with its own instance id and a related kind and let clients figure out migration. I would like to have a version field in the module kind type for that.

I guess there's an overlap here. You might be right. I need to think about about it a bit longer and run thought more scenarios for theoretical modules, and how practical it is. For a module like mint such a static migration seems very nice, but I wonder if it will always be sufficient.

Currently we have a CODE_VERSION which is stored in the ServerConsensusConfig so that if they don't match, servers will fail to join consensus. How would these new versions interact with the existing one?

Replace. We should compare consensus versions (global FM one, and each module's instance). If the code says it's compatible, we believe it.

@elsirion

I guess there's an overlap here. You might be right. I need to think about about it a bit longer and run thought more scenarios for theoretical modules, and how practical it is. For a module like mint such a static migration seems very nice, but I wonder if it will always be sufficient.

OK, so I think it's not hard to come up with examples where rolling over to new instance is not  really desireable / possible.
Let's say we have a smart contract module, Ethereum style. Let's ignore for a second if that's a bad idea to do it. Since the contracts in the existing instance interact with other contracts, this creates a strong network effect.
Migrating to another instance with new rules in slow motion is close to impossible. The module becomes it's own side-chain, not unlike any alt-coin. What the module authors want is a scheduled hard-fork to change the rules (e.g fee structure), and not a slow migration.
Similarly if the module is storing a lot of long term backup (like a threshold encrypted-backup), a module consensus hard-fork is preferred over bothering users with migration.
So, yeah - I don't think rolling over is going to work in every case. Thought I do agree that when possible it's very elegant. The module consensus upgrade is orthogonal to it. A mint-v2 KIND becomes a completely different module kind than mint-v1 (if we do it through just different KIND string or an extra "module kind version" is implementation detail. It will be up to module developers to decide if rolling over to a new kind&instance is preferable over "module consensus hard-fork", and we will guide the user through this difference in the UIs.

Currently we have a CODE_VERSION which is stored in the ServerConsensusConfig so that if they don't match, servers will fail to join consensus. How would these new versions interact with the existing one?

Replace. We should compare consensus versions (global FM one, and each module's instance). If the code says it's compatible, we believe it.

So we could run with different binary versions potentially?  Seems like that could be error-prone.
If this is replacing CODE_VERSION can we also remove it and put these versions into the consensus config then?

So we could run with different binary versions potentially? Seems like that could be error-prone.

Eventually we have to discard the training wheels and graduate to the big league of distributed software.
Right now the stakes are low, so it is a good opportunity to learn and gain some experience.
Whenever we touch anything that smells like breaking consensus, we just increment the corresponding number and it comes to the same thing.

If this is replacing CODE_VERSION can we also remove it and put these versions into the consensus config then?

I am not doing it in this PR, but yes, that would exactly be the next step. We start comparing consensus versions between peers, and module code compatibility against the federation consensus and we error out if anything is wrong.

Sounds reasonable, will look at it again with fresh eyes tomorrow, not too opposed to land in this form anymore. I still think one code version only supporting one consensus version would be more elegant, but this flexibility is likely enough to be useful in the not too distant future to make sense.

Dev call: add test to CI to assert that versions didn't change if not declared in some way

I amended with changes corresponding to some conclusions from the discussion here.

Let's try this! Next steps are inclusion in cfg/client side checks?

I think so, yes. And we have to remember to try to increment these numbers when we break api/consensus rules. Not a big deal if we forget, but we should try.

authors: dpc elsirion jkitman justinmoon


text:
Clientd

Did clean up stuff by writing it again but use decent commits this time. I'm not quite sure if my get_active_issuances in mint/mod makes sense because theres already a list_active_issuances but I needed  CoinFinalizationData rather than OutPoint (if anybody sees how to sum up pending transactions and get the total 'value' with OutPoint I can remove this. Also I still used StructOpt because I thought it would be better refactoring to clap in one PR for all files istead of file by file (if this is even wanted, I heard that it is advised to switch)

just fyi : I just noticed that the API design is extremely bad and want to refactor it all using this as a reference :
https://www.jsonrpc.org/specification
https://www.youtube.com/watch?v=yobsLtsMPL8&list=WL&index=1
takes longer than expected tho because the rust I'll need for that is kind of a leap for my current skill

just fyi : I just noticed that the API design is extremely bad and want to refactor it all using this as a reference :

Ah, had missed that comment

I just noticed it would be a bad idea to work on the jsonrpc-client in this branch/PR so I'll remove (https://github.com/fedimint/minimint/pull/55/commits/730e77699cdb736afeaf65b380f328c5f46d5adc) in a later rebase. I'd like to get feedback on how I want to approach things from now:
In the end the mint-client crate should look like this:
---src
   ---bin
      ---clientd
            ---main.rs
            ---lib.rs
      ---cli
            ---main.rs
            ---lib.rs
   ---jsonrpc
            ---clients.rs  <-- the client which makes https requests to the server doMethod()
            ---json.rs  <-- holds data structures for interacting with the API, Request/Response and also API Results (not API Errors)
            ---error.rs  <-- organizes errors (RpcError, JsonError, RPCClientError ..?)
   --- (...)

The now existing rpc crate would be removed and the 'router' would move to clientd/lib, the Request/Response structs would move in jsonrpc/json and the StandartError/RpcError would move in jsonrpc/error. Also the data structures for the API results (and some others) would also move in jsonrpc/json (right now in clients/user.rs).
I'll start a new jsonrpc branch for this and later also an jsonrpc-cli branch. I'd love to hear feedback on my organization approach since I'm still very confused on where to put things.
Also just fyi: I'll try to use https://github.com/apoelstra/rust-jsonrpc as the main 'role model' for the jsonrpc

Opening another clientd PR soon

authors: NicolaLS elsirion

repo: fedimint/minimint


text:
Review annost'd ActivityPub spec and add annotations for missing tests

Documents are here: https://github.com/fediverse-devnet/feditest-tests-fediverse/tree/develop/tests/activitypub/ANNOST

Done


text:
Jenkins console parser: display mentioned method definitions in the log

Our log sometimes includes messages like DeprecationPerformedNotification: Automatic deprecation code rewrite: The method BrList>>#display: called from BrSimpleListExamples>>#singleLineWithTinyIcon has been deprecated. Please use items: method instead.
We would like to embed Coders in such situations.

By clicking on the method class, the coder is displayed next to it with the method selected:

This is really cool!

Similarly for NewUndeclaredWarning:

authors: JurajKubelka girba

repo: feenkcom/gtoolkit


text:
Universes view does not work.

And in native windows, it looks like this:

authors: chisandrei georgeganea girba


text:
how to implement 1-pixel threshold error on Scene Flow dataset?

Thanks for your generous sharing.
I have one problem with the evaluation mentioned in your paper. You said "GA-Nets are compared with other state-of-the-art DNN models by evaluating with the average end point errors (EPE) and 1-pixel threshold error rates on the test set. "
However, I only find the code for EPE evaluation. Could you please provide how to compute 1-pixel threshold error rates?
Many thanks!

Evaluation scripts added.

authors: feihuzhang zhiwenfan

repo: feihuzhang/GANet