text:
change gremlin-server version for vagrant to 3.0.2, added composer.lock

@chrismichaels84 can you just double check the sanity of the vagrant changes? I can't really test vagrant in my env yet (will try to get it up and running)

Or @smolinari if you get a chance to test vagrant on this.

@smolinari does this work in the end? can I merge this branch or do changes need to be made?

@PommeVerte - yes. It is working fine. I think the issue I had yesterday afternoon was caused by a flaky internet connection and somehow screwed up the Gremlin install. This morning all worked as it should on the Gremlin/Neo4j side.
Scott

ok awesome. adding this then :)

Theoretically, if Chris accepts my changes (plus some needed cleanup work), then we'd also have ODB up to 2.1.6 too.
Scott

authors: PommeVerte smolinari

repo: spider/spider


text:
[feature] add spidernode CRD which preallocted ips for each node

1 Code requirement
2 observe opensource license
3 sign-off your commit
4 Is your feature request related to a problem? Please describe.
5 Describe the solution you'd like
Now in aws ECS environment could eni-cni, and design like eni in ciliumnodes resource
maybe spidernet could add another CRD, like spidernode which will preallocte ips for each node, because ip is from vpc, but only support ip not subnat, so spidernode record ip list which is preallocted.
below is some likely steps:

daemon(spider agent)

Use spidernode as the source for IP allocation
Synchronize spidernode and spiderpool resources
Can do some extra operations on nodes, such as adding network interfaces, setting IP/MAC etc.

controller(spider controller)

Create and update spidernode from spiderpool
Synchronize spec, status information with daemon decision

6 Describe alternatives you've considered
7 Additional context

@yylt Hi, it looks like a public cloud vpc support right?
At present, in the public cloud with spiderpool usage you could just define every node IPs for different SpiderIPPool resource with IPPool.Spec.NodeName or IPPool.Spec.NodeAffinity to restrict this SpiderIPPool resource only serves for the Node you specified.
And you could check alibaba cloud and aws cloud blogs.
For the new CRD support, our team will talk or add some public cloud support later. And you could also commit a PR to solve this proposal if you would like.
Thansk.

some CNIs record the ip resource in Node object, spiderpool record it in SpiderIPPool, currently, the affinity settings of SpiderIPPool support to bind some IP resource to a specific interface of a specific vm. That is convenient for creating an ipvlan interface from a specific master interface
I think the SpiderIPPool way also meet the same expectation

Another reason， as https://spidernet-io.github.io/spiderpool/v0.7/reference/crd-spiderippool description.
The spec.ips is iprange format which can not compatible with eni

close by #2536

source: gharchive/issue

authors: Icarus9913 weizhoublue yylt

repo: spidernet-io/spiderpool

license_type: permissive

license_source: github-api


text:
failed to cherry pick PR 3873 from cyclinder, to branch release-v1.0

commits 22a33847f9755f972c03aa121b41f19cc3b62a22 of cyclinder conflict when merging to branch release-v1.0, please manually cherry pick it by yourself.
PR https://github.com/spidernet-io/spiderpool/pull/3873 ,  action https://github.com/spidernet-io/spiderpool/actions/runs/10557667197
Auto-merging test/e2e/reclaim/reclaim_test.go
Auto-merging test/scripts/debugEnv.sh
CONFLICT (content): Merge conflict in test/scripts/debugEnv.sh
error: could not apply 22a33847... Merge pull request #3873 from cyclinder/coordinator/tune_pod_route
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config advice.mergeConflict false"

https://github.com/spidernet-io/spiderpool/pull/3971

authors: cyclinder weizhoublue


text:
feat: Add make a example database

Feature Description
Make a test sample database
Add sqlite database option is not exists
Motivations
Be able to test the self-hosted system quickly and easily
Very thanks.

Could you let me know more about your plans with self-hosting? I just want to gauge if termic would indeed the best fit for you or if there's a better way of achieving what you want to do. Do you already have data to show on the frontend?
I don't believe it reasonable to think about termic as handling anything related to database management per se (one should instead rather see it as a pre-configured web app for browsing and searching terminology data). I expect people who self-host to already have their database available. But I can always add an example database to show what it can look like, yes.

My idea is to analyze the option of having a place where you can upload translations and be able to consult them, in my case I am a translator and I do a lot of queries to see how certain terms have been translated in the past.
I would like to be able to upload .po files (gettext)
I think it would be great to have a docker compose that installs a blank database and if termic had a manager that allowed, once installed, to upload translations of tmx, po, csv files...

I see. I guess that's a reasonable use of termic and something that could be supported.
I originally made some tests with a SQLite database when first developing termic, but I eventually moved to PostgreSQL (which is a much better fit for deployment). If you plan on using termic locally though, this could totally be done.
I can't give an estimate on how much time this will take to implement as I have quite a lot on my plate right now — but I hope I can have a look at it in January.

Hello: 🙂
OK! Wonderful if work with sqlite.
Being able to use Termic inside a docker container with a preloaded database would be great.
Including with postgresql in separated container same stack.
Something interesting would be to be able to remove translations from languages that you don't need.
I think an export option that allows you to send translations to Termic in their own format would be great. This would make it easier to contribute to the Termic project.
None of this is urgent.
Thank you very much anyway.

authors: damufo spidersouris

repo: spidersouris/termic

license: MIT


text:
M1 Build for macOS

An Apple Silicon release would be nice!

Yep thanks, meant to but forgot, will do that in a day or two. The x86_64 version should work through Rosetta however. If you try and find that it doesn't please let me know.

I replaced the x86_64 version with a universal binary.

repo: spieglt/FlyingCarpet

license_source: bigquery


text:
Processing killed on compute canada

with the following SLURM config:
#SBATCH --account=def-jcohen
#SBATCH --time=0-03:00        # time (DD-HH:MM)
#SBATCH --ntasks=128          # number of MPI processes
#SBATCH --mem-per-cpu=4096M   # memory; default unit is megabytes

and this YML config for sct_run_batch:
path_data: /scratch/jcohen/data-single-subject-master
path_output: /scratch/jcohen/results
task: /home/jcohen/code/spine-generic/spine-generic/processing/process_data.sh
jobs: 128

i get this job killed:
/home/jcohen/code/spine-generic/spine-generic/processing/process_data.sh: line 76:  4663 Killed                  sct_deepseg_sc -i ${file}.nii.gz -c $contrast -qc ${PATH_QC} -qc-subject ${SUBJECT}

and this other job got the following error:
OSError: [Errno 12] Cannot allocate memory

This issue was solved by raising the memory per core, with this SLURM config:
#SBATCH --account=def-jcohen
#SBATCH --time=0-01:00        # time (DD-HH:MM)
#SBATCH --ntasks=128          # number of MPI processes
#SBATCH --mem-per-cpu=16384   # memory; default unit is megabytes
#SBATCH --mail-user=***
#SBATCH --mail-type=ALL

repo: spine-generic/spine-generic


text:
[BUG] Unable to install using composer

I tried installing the project using the composer method as outlined here: https://roadrunner.dev/docs/intro-install, in a fresh empty composer project, however, I get the following error:
PHP Fatal error:  Uncaught Error: Class 'Spiral\RoadRunner\Version' in ./vendor/spiral/roadrunner-cli/bin/rr:67

I tried this code:
composer require spiral/roadrunner
./vendor/bin/rr get

The version of RR used: ^2.0 (v2.0.1)

@jwillp Feel free to close the issue if the problem was resolved.

repo: spiral/roadrunner


text:
PAPP-22704: Update release note

Please ensure your pull request (PR) adheres to the following guidelines:

Please refer to our contributing documentation for any questions on submitting a pull request, link: Contribution Guide

Pull Request Checklist
Please check if your PR fulfills the following requirements:

[ ] Testing of all the changes has been performed (for bug fixes / features)
[ ] The readme.html has been reviewed and added / updated if needed (for bug fixes / features)
[ ] Use the following format for the PR description: <App Name>: <PR Type> - <PR Description>
[ ] Provide release notes as part of the PR submission which describe high level points about the changes for the upcoming GA release.
[ ] Verify all checks are passing.
[ ] Do NOT use the next branch of the forked repo. Create separate feature branch for raising the PR.
[ ] Do NOT submit updates to dependencies unless it fixes an issue.

Pull Request Type
Please check the type of change your PR introduces:

[ ] New App
[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation
[ ] Other (please describe):

Release Notes (REQUIRED)

Provide release notes as part of the PR submission which describe high level points about the changes for the upcoming GA release.

What is the current behavior? (OPTIONAL)

Describe the current behavior that you are modifying.

What is the new behavior? (OPTIONAL)

Describe the behavior or changes that are being added by this PR.

Other information (OPTIONAL)

Any other information that is important to this PR such as screenshots of how the component looks before and after the change.

Pay close attention to (OPTIONAL)

Any specific code change or test case points which must be addressed/reviewed at the time of GA release.

Screenshots (if relevant)

Thanks for contributing!

Note: github integration test pipeline is failed because of time out, run in gitlab.
Backend run: https://cd.splunkdev.com/phantom-apps/app-tests/-/jobs/24903653
UI run: https://cd.splunkdev.com/phantom-apps/app-tests/-/jobs/24903654

source: gharchive/pull-request


text:
ADDON-34291: Added Validation utility

ADDON-34291

Created Predefined Validators for entity type - string, regex, number, URL, email, ipv4 and date.
doValidation method of class Validator checks for validation based on entity type and returns dictionary containing error and errormsg if any.

:tada: This PR is included in version 5.0.0-develop.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 5.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

authors: dkhatri-crest rfaircloth-splunk

repo: splunk/addonfactory-ucc-generator


text:
Update Node.js version

The action is currently using an old version of nodejs. Ensure that this is updated to a more recent release.

Won't fix, breaking change released

authors: dfederschmidt mbruzda-splunk

repo: splunk/appinspect-api-action


text:
Feature Request

I'd like to be able to whitelist/blacklist when using token..replacement. For example, if I''m referencing a file like "allhosts.csv" I want to be able to blacklist a set of hosts with a regex. Possibly an additional parameter like below:
replace server name
token.2.token = (SERVERNAME)
token.2.replacementType = file
token.2.replacement = $SPLUNK_HOME/etc/apps/oidemo/master_eventgen_replace/webhosts.sample:2
token.2.replacementWhitelist.0 = *
token.2.replacementBlacklist.0 = webserver-[0-9]

Totally valid request, but why wouldn't you just modify the source file or create a different copy of it for that token?

I have a few scenarios going at the same time. For example, I need webserver-02 to spike in CPU at 30min past the hour. To do this I create eventgen stanzas and sample files:

An "issue" sample file which spikes the CPU data at 30min past the hour for webserver-02
A noise sample for the first 30min of webserver-02
A problem sample file(s) which runs for the whole hour that generates cpu data for all hosts replacing SERVERNAME from allhosts.sample which has all hosts except webserver-02.

Next I want to do something similar (say a spike in queries and cpu) for dbserver-02 but now I have to create another file with all hosts except dbserver-02 and non-db servers.
As I continue to add scenarios I have to make a lot of files. A white/black list allows me to substantially reduce the number of replace files required and makes it easier to update/maintain.
Make sense?

this is a good idea.  I just don't want to touch that code :).  I'm leaving it open for the time being.

Since this issue has been open for a while and we have released new versions 6.x.x of Eventgen, please recreate the issue if you still see fit.

authors: arctan5x coccyx dataPhysicist

repo: splunk/eventgen


text:
[BUG] O365 Mailbox Inbox Folder Shared with All Users. Field "object" doesn't exist.

Correlation search, O365 Mailbox Inbox Folder Shared with All Users, is currently using a field called "object", as object=Inbox. But I do not see this field being sent as part of O365 exchange data. Instead, I see a field called Item.ParentFolder.Name with values such as Inbox, Calender, Contacts etc.
Should "object=Inbox" be replaced with "Item.ParentFolder.Name=Inbox" for this correlation search?
App Version:

ESCU: 4.18.0

@atgithub11 this might be due to how the data for o365 is being collected in your environment. I believe for this detection we expect the user to be leveraging https://splunkbase.splunk.com/app/4055 let me know if this is the case?

Hey @atgithub11 thanks for opening this issue. Here is some clarification that might help you understand this.
You can see in the detection, the raw log does contain the field "Item.ParentFolder.Name"

And this is what you are probably ingesting.
But as @josehelps said, the analytics expects ingestion of O365 via the splunk app https://splunkbase.splunk.com/app/4055 as it is stated by the how to implement section
how_to_implement: You must install the Splunk Microsoft Office 365 Add-on and ingest Office 365 management activity events.

Behind the scene, this app is creating the field object and assigning its value based on some condition:

In this case if the Operation field is "ModifyFolderPermissions" or "AddFolderPermissions", the value of the Object field will be set to Item.ParentFolder.Name.
Hence the detection should be correct, and this is just an issue of ingestion.
Hope this helps.
I'll be closing this as complete, feel free to re-open the issue in case you have further questions

authors: atgithub11 josehelps nasbench


text:
Source typing container logs

Right now, source types are derived from the container or pod name. Are there any plans to provide the option for users to set the source type? This allows users to create the same containers with a common source type rather different ones. I've seen over 1000 sourcetypes created on a single cluster.

Hey Sayeed!
Technically you can set the sourcetype by customizing the jq_transformer filter in the logging configmap:
https://github.com/splunk/splunk-connect-for-kubernetes/blob/ec00d8fcf6b4030cca0ea434d8b54a606add85d0/manifests/splunk-kubernetes-logging/configMap.yaml#L177
Ideally, I'd like to see us support setting the sourcetype, among other options in annotations.

Hey Matt
Ideally, I'd like the users to set the sourcetype from Openshift (maybe via labels or annotations) since how containers are named is controlled by them. It can be anything, which makes setting source types via jq difficult.

Currently a user can set the sourcetype using https://github.com/splunk/splunk-connect-for-kubernetes/blob/develop/helm-chart/splunk-kubernetes-logging/values.yaml#L108
I know its static, for setting sourcetypes dynamically through labels and annotations will require a significant overhaul of the current implementation.

resolved as part of #294

authors: chaitanyaphalak matthewmodestino rockb1017 sayeedc

repo: splunk/splunk-connect-for-kubernetes


text:
📝 Add contribution guidlines

TODO :

Development convention
Contribution rules (Issue, PR, commit message, ...)

:heavy_check_mark:

repo: spontoreau/mediateur


text:
AttributeError: 'NoneType' object has no attribute 'lower' with some tracks when synced is enabled as a lyrics provider

System OS
Windows
Python Version
3.11 (CPython)
Install Source
pip / PyPi
Install version / commit hash
v4.1.7
Expected Behavior vs Actual Behavior
No response
Steps to reproduce - Ensure to include actual links!
spotdl download https://open.spotify.com/track/3i5bc53F2glMZC7GFXZQ7T
Traceback
[09:40:16] DEBUG    MainThread - Downloader settings: {'audio_providers': ['youtube-music'],           downloader.py:115
                    'lyrics_providers': ['synced', 'genius', 'azlyrics', 'musixmatch'],
                    'playlist_numbering': False, 'scan_for_songs': True, 'm3u': None, 'output':
                    'D:\\Music\\{album-artist}\\{album} ({year})\\{disc-number} - {track-number} -
                    {title}.{output-ext}', 'overwrite': 'metadata', 'search_query': None, 'ffmpeg':
                    'ffmpeg', 'bitrate': None, 'ffmpeg_args': None, 'format': 'mp3', 'save_file':
                    None, 'filter_results': True, 'threads': 4, 'cookie_file': None, 'restrict':
                    False, 'print_errors': True, 'sponsor_block': False, 'preload': True, 'archive':
                    None, 'load_config': True, 'log_level': 'DEBUG', 'simple_tui': False,
                    'fetch_albums': True, 'id3_separator': '/', 'ytm_data': False, 'add_unavailable':
                    False, 'generate_lrc': True, 'force_update_metadata': True,
                    'only_verified_results': True, 'sync_without_deleting': True,
                    'max_filename_length': 100}
[09:40:16] DEBUG    MainThread - FFmpeg path: ffmpeg                                                   downloader.py:133
[09:40:16] INFO     MainThread - Scanning for known songs, this might take a while...                  downloader.py:152
[09:40:25] DEBUG    MainThread - Found 2018 known songs                                                downloader.py:158
[09:40:28] DEBUG    MainThread - Archive: 0 urls                                                       downloader.py:192
[09:40:28] DEBUG    MainThread - Downloader initialized                                                downloader.py:194
[09:40:28] INFO     MainThread - Processing query: https://open.spotify.com/track/3i5bc53F2glMZC7GFXZQ7T   search.py:123
[09:40:29] DEBUG    MainThread - Found 1 songs in 0 lists                                                  search.py:249
[09:40:29] INFO     MainThread - Fetching 1 album                                                      downloader.py:228
[09:40:30] ERROR    asyncio_0 - Traceback (most recent call last):                               progress_handler.py:358
                      File "C:\spotDL\venv\Lib\site-packages\spotdl\download\downloader.py",
                    line 495, in search_and_download
                        lyrics = self.search_lyrics(song)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\spotdl\download\downloader.py",
                    line 350, in search_lyrics
                        lyrics = lyrics_provider.get_lyrics(song.name, song.artists)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\spotdl\providers\lyrics\synced.py",
                    line 62, in get_lyrics
                        lyrics = syncedlyrics.search(f"{name} - {artists[0]}",
                    allow_plain_format=True)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    ^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\syncedlyrics\__init__.py", line 40,
                    in search
                        lrc = provider.get_lrc(search_term)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File
                    "C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py", line
                    30, in get_lrc
                        a_tag = soup.find_all("a", string=lambda t: text_match(t) > 80, limit=4)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\bs4\element.py", line 2030, in
                    find_all
                        return self._find_all(name, attrs, string, limit, generator,
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\bs4\element.py", line 841, in
                    _find_all
                        found = strainer.search(i)
                                ^^^^^^^^^^^^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\bs4\element.py", line 2320, in
                    search
                        found = self.search_tag(markup)
                                ^^^^^^^^^^^^^^^^^^^^^^^
                      File "C:\spotDL\venv\Lib\site-packages\bs4\element.py", line 2291, in
                    search_tag
                        if found and self.string and not self._matches(found.string,
                    self.string):
                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    ^
                      File "C:\spotDL\venv\Lib\site-packages\bs4\element.py", line 2352, in
                    _matches
                        return match_against(markup)
                               ^^^^^^^^^^^^^^^^^^^^^
                      File
                    "C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py", line
                    30, in <lambda>
                        a_tag = soup.find_all("a", string=lambda t: text_match(t) > 80, limit=4)
                                                                    ^^^^^^^^^^^^^
                      File
                    "C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py", line
                    26, in <lambda>
                        text_match = lambda t: rapidfuzz.fuzz.token_sort_ratio(_t(search_term),
                    _t(t))

                    ^^^^^
                      File
                    "C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py", line
                    25, in <lambda>
                        _t = lambda s: s.lower().replace("-", "")
                                       ^^^^^^^
                    AttributeError: 'NoneType' object has no attribute 'lower'

                    ╭─────────────────── Traceback (most recent call last) ────────────────────╮
                    │ C:\spotDL\venv\Lib\site-packages\spotdl\download\downloader.py:495 in    │
                    │ search_and_download                                                      │
                    │                                                                          │
                    │   492 │   │   │   │   │   │   )                                          │
                    │   493 │   │   │                                                          │
                    │   494 │   │   │   # Find song lyrics and add them to the song object     │
                    │ ❱ 495 │   │   │   lyrics = self.search_lyrics(song)                      │
                    │   496 │   │   │   if lyrics is None:                                     │
                    │   497 │   │   │   │   logger.debug(                                      │
                    │   498 │   │   │   │   │   "No lyrics found for %s, lyrics providers: %s" │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\spotdl\download\downloader.py:350 in    │
                    │ search_lyrics                                                            │
                    │                                                                          │
                    │   347 │   │   """                                                        │
                    │   348 │   │                                                              │
                    │   349 │   │   for lyrics_provider in self.lyrics_providers:              │
                    │ ❱ 350 │   │   │   lyrics = lyrics_provider.get_lyrics(song.name, song.ar │
                    │   351 │   │   │   if lyrics:                                             │
                    │   352 │   │   │   │   logger.debug(                                      │
                    │   353 │   │   │   │   │   "Found lyrics for %s on %s", song.display_name │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\spotdl\providers\lyrics\synced.py:62 in │
                    │ get_lyrics                                                               │
                    │                                                                          │
                    │   59 │   │   - The lyrics of the song or None if no lyrics were found.   │
                    │   60 │   │   """                                                         │
                    │   61 │   │                                                               │
                    │ ❱ 62 │   │   lyrics = syncedlyrics.search(f"{name} - {artists[0]}", allo │
                    │   63 │   │                                                               │
                    │   64 │   │   return lyrics                                               │
                    │   65                                                                     │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\syncedlyrics\__init__.py:40 in search   │
                    │                                                                          │
                    │   37 │   lrc = None                                                      │
                    │   38 │   for provider in _providers:                                     │
                    │   39 │   │   logger.debug(f"Looking for an LRC on {provider.__class__.__ │
                    │ ❱ 40 │   │   lrc = provider.get_lrc(search_term)                         │
                    │   41 │   │   if is_lrc_valid(lrc, allow_plain_format):                   │
                    │   42 │   │   │   logger.info(                                            │
                    │   43 │   │   │   │   f'synced-lyrics found for "{search_term}" on        │
                    │      {provider.__class__.__name__}'                                      │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py:30  │
                    │ in get_lrc                                                               │
                    │                                                                          │
                    │   27 │   │   href_match = lambda h: h.startswith("/lyric/")              │
                    │   28 │   │   a_tags_boud = SoupStrainer("a", href=href_match)            │
                    │   29 │   │   soup = generate_bs4_soup(self.session, url, parse_only=a_ta │
                    │ ❱ 30 │   │   a_tag = soup.find_all("a", string=lambda t: text_match(t) > │
                    │   31 │   │   if not a_tag:                                               │
                    │   32 │   │   │   return None                                             │
                    │   33                                                                     │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\bs4\element.py:2030 in find_all         │
                    │                                                                          │
                    │   2027 │   │   if not recursive:                                         │
                    │   2028 │   │   │   generator = self.children                             │
                    │   2029 │   │   _stacklevel = kwargs.pop('_stacklevel', 2)                │
                    │ ❱ 2030 │   │   return self._find_all(name, attrs, string, limit, generat │
                    │   2031 │   │   │   │   │   │   │     _stacklevel=_stacklevel+1, **kwargs │
                    │   2032 │   findAll = find_all       # BS3                                │
                    │   2033 │   findChildren = find_all  # BS2                                │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\bs4\element.py:841 in _find_all         │
                    │                                                                          │
                    │    838 │   │   │   except StopIteration:                                 │
                    │    839 │   │   │   │   break                                             │
                    │    840 │   │   │   if i:                                                 │
                    │ ❱  841 │   │   │   │   found = strainer.search(i)                        │
                    │    842 │   │   │   │   if found:                                         │
                    │    843 │   │   │   │   │   results.append(found)                         │
                    │    844 │   │   │   │   │   if limit and len(results) >= limit:           │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\bs4\element.py:2320 in search           │
                    │                                                                          │
                    │   2317 │   │   # Don't bother with Tags if we're searching for text.     │
                    │   2318 │   │   elif isinstance(markup, Tag):                             │
                    │   2319 │   │   │   if not self.string or self.name or self.attrs:        │
                    │ ❱ 2320 │   │   │   │   found = self.search_tag(markup)                   │
                    │   2321 │   │   # If it's text, make sure the text matches.               │
                    │   2322 │   │   elif isinstance(markup, NavigableString) or \             │
                    │   2323 │   │   │   │    isinstance(markup, str):                         │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\bs4\element.py:2291 in search_tag       │
                    │                                                                          │
                    │   2288 │   │   │   │   │   found = markup                                │
                    │   2289 │   │   │   │   else:                                             │
                    │   2290 │   │   │   │   │   found = markup_name                           │
                    │ ❱ 2291 │   │   if found and self.string and not self._matches(found.stri │
                    │   2292 │   │   │   found = None                                          │
                    │   2293 │   │   return found                                              │
                    │   2294                                                                   │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\bs4\element.py:2352 in _matches         │
                    │                                                                          │
                    │   2349 │   │   │   return markup is not None                             │
                    │   2350 │   │                                                             │
                    │   2351 │   │   if isinstance(match_against, Callable):                   │
                    │ ❱ 2352 │   │   │   return match_against(markup)                          │
                    │   2353 │   │                                                             │
                    │   2354 │   │   # Custom callables take the tag as an argument, but all   │
                    │   2355 │   │   # other ways of matching match the tag name as a string.  │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py:30  │
                    │ in <lambda>                                                              │
                    │                                                                          │
                    │   27 │   │   href_match = lambda h: h.startswith("/lyric/")              │
                    │   28 │   │   a_tags_boud = SoupStrainer("a", href=href_match)            │
                    │   29 │   │   soup = generate_bs4_soup(self.session, url, parse_only=a_ta │
                    │ ❱ 30 │   │   a_tag = soup.find_all("a", string=lambda t: text_match(t) > │
                    │   31 │   │   if not a_tag:                                               │
                    │   32 │   │   │   return None                                             │
                    │   33                                                                     │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py:26  │
                    │ in <lambda>                                                              │
                    │                                                                          │
                    │   23 │   │   # Just processing the `a` tags whose `href` attribute start │
                    │   24 │   │   # and whose text is similar to the query too.               │
                    │      https://github.com/maxbachmann/RapidFuzz#scorers                    │
                    │   25 │   │   _t = lambda s: s.lower().replace("-", "")                   │
                    │ ❱ 26 │   │   text_match = lambda t: rapidfuzz.fuzz.token_sort_ratio(_t(s │
                    │   27 │   │   href_match = lambda h: h.startswith("/lyric/")              │
                    │   28 │   │   a_tags_boud = SoupStrainer("a", href=href_match)            │
                    │   29 │   │   soup = generate_bs4_soup(self.session, url, parse_only=a_ta │
                    │                                                                          │
                    │ C:\spotDL\venv\Lib\site-packages\syncedlyrics\providers\lyricsify.py:25  │
                    │ in <lambda>                                                              │
                    │                                                                          │
                    │   22 │   │                                                               │
                    │   23 │   │   # Just processing the `a` tags whose `href` attribute start │
                    │   24 │   │   # and whose text is similar to the query too.               │
                    │      https://github.com/maxbachmann/RapidFuzz#scorers                    │
                    │ ❱ 25 │   │   _t = lambda s: s.lower().replace("-", "")                   │
                    │   26 │   │   text_match = lambda t: rapidfuzz.fuzz.token_sort_ratio(_t(s │
                    │   27 │   │   href_match = lambda h: h.startswith("/lyric/")              │
                    │   28 │   │   a_tags_boud = SoupStrainer("a", href=href_match)            │
                    ╰──────────────────────────────────────────────────────────────────────────╯
                    AttributeError: 'NoneType' object has no attribute 'lower'
None
[09:40:32] DEBUG    asyncio_1 - Synced failed to find lyrics for Emerson, Lake & Palmer - Piano        downloader.py:358
                    Concerto No. 1 - i. 1st Movement: Allegro Giojoso; ii. 2nd Movement: Andante Molto
                    Cantabile; iii. 3rd Movement: Toccata Con Fuoco; 2017 - Remaster
[09:40:32] DEBUG    asyncio_1 - Genius failed to find lyrics for Emerson, Lake & Palmer - Piano        downloader.py:358
                    Concerto No. 1 - i. 1st Movement: Allegro Giojoso; ii. 2nd Movement: Andante Molto
                    Cantabile; iii. 3rd Movement: Toccata Con Fuoco; 2017 - Remaster
[09:40:33] DEBUG    asyncio_1 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Piano      downloader.py:358
                    Concerto No. 1 - i. 1st Movement: Allegro Giojoso; ii. 2nd Movement: Andante Molto
                    Cantabile; iii. 3rd Movement: Toccata Con Fuoco; 2017 - Remaster
[09:40:36] DEBUG    asyncio_1 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Piano    downloader.py:358
                    Concerto No. 1 - i. 1st Movement: Allegro Giojoso; ii. 2nd Movement: Andante Molto
                    Cantabile; iii. 3rd Movement: Toccata Con Fuoco; 2017 - Remaster
[09:40:36] DEBUG    asyncio_1 - No lyrics found for Emerson, Lake & Palmer - Piano Concerto No. 1 - i. downloader.py:497
                    1st Movement: Allegro Giojoso; ii. 2nd Movement: Andante Molto Cantabile; iii. 3rd
                    Movement: Toccata Con Fuoco; 2017 - Remaster, lyrics providers: Synced, Genius,
                    AzLyrics, MusixMatch
[09:40:36] INFO     asyncio_1 -                                                                        downloader.py:540
None
[09:40:39] DEBUG    asyncio_3 - Synced failed to find lyrics for Emerson, Lake & Palmer - C'est La Vie downloader.py:358
                    - 2017 Remastered Version
None
[09:40:39] DEBUG    asyncio_2 - Synced failed to find lyrics for Emerson, Lake & Palmer - Lend Your    downloader.py:358
                    Love to Me Tonight - 2017 Remastered Version
[09:40:39] DEBUG    asyncio_2 - Genius failed to find lyrics for Emerson, Lake & Palmer - Lend Your    downloader.py:358
                    Love to Me Tonight - 2017 Remastered Version
[09:40:40] DEBUG    asyncio_3 - Genius failed to find lyrics for Emerson, Lake & Palmer - C'est La Vie downloader.py:358
                    - 2017 Remastered Version
[09:40:40] DEBUG    asyncio_2 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Lend Your  downloader.py:358
                    Love to Me Tonight - 2017 Remastered Version
[09:40:41] DEBUG    asyncio_3 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - C'est La   downloader.py:358
                    Vie - 2017 Remastered Version
None
[09:40:42] DEBUG    asyncio_0 - Synced failed to find lyrics for Emerson, Lake & Palmer - Hallowed Be  downloader.py:358
                    Thy Name - 2017 Remastered Version
[09:40:42] DEBUG    asyncio_0 - Genius failed to find lyrics for Emerson, Lake & Palmer - Hallowed Be  downloader.py:358
                    Thy Name - 2017 Remastered Version
None
[09:40:42] DEBUG    asyncio_1 - Synced failed to find lyrics for Emerson, Lake & Palmer - Nobody Loves downloader.py:358
                    You Like I Do - 2017 Remastered Version
[09:40:42] DEBUG    asyncio_0 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Hallowed   downloader.py:358
                    Be Thy Name - 2017 Remastered Version
[09:40:43] DEBUG    asyncio_1 - Genius failed to find lyrics for Emerson, Lake & Palmer - Nobody Loves downloader.py:358
                    You Like I Do - 2017 Remastered Version
[09:40:43] DEBUG    asyncio_2 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Lend     downloader.py:358
                    Your Love to Me Tonight - 2017 Remastered Version
[09:40:43] DEBUG    asyncio_2 - No lyrics found for Emerson, Lake & Palmer - Lend Your Love to Me      downloader.py:497
                    Tonight - 2017 Remastered Version, lyrics providers: Synced, Genius, AzLyrics,
                    MusixMatch
[09:40:43] INFO     asyncio_2 -                                                                        downloader.py:540
[09:40:44] DEBUG    asyncio_3 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - C'est La downloader.py:358
                    Vie - 2017 Remastered Version
[09:40:44] DEBUG    asyncio_3 - No lyrics found for Emerson, Lake & Palmer - C'est La Vie - 2017       downloader.py:497
                    Remastered Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:40:44] DEBUG    asyncio_1 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Nobody     downloader.py:358
                    Loves You Like I Do - 2017 Remastered Version
[09:40:44] INFO     asyncio_3 -                                                                        downloader.py:540
None
[09:40:45] DEBUG    asyncio_3 - Synced failed to find lyrics for Emerson, Lake & Palmer - The Enemy    downloader.py:358
                    God Dances With the Black Spirits - 2017 Remastered Version
[09:40:46] DEBUG    asyncio_3 - Genius failed to find lyrics for Emerson, Lake & Palmer - The Enemy    downloader.py:358
                    God Dances With the Black Spirits - 2017 Remastered Version
[09:40:46] DEBUG    asyncio_0 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Hallowed downloader.py:358
                    Be Thy Name - 2017 Remastered Version
[09:40:46] DEBUG    asyncio_0 - No lyrics found for Emerson, Lake & Palmer - Hallowed Be Thy Name -    downloader.py:497
                    2017 Remastered Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:40:46] INFO     asyncio_0 -                                                                        downloader.py:540
[09:40:46] DEBUG    asyncio_3 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - The Enemy  downloader.py:358
                    God Dances With the Black Spirits - 2017 Remastered Version
[09:40:47] DEBUG    asyncio_1 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Nobody   downloader.py:358
                    Loves You Like I Do - 2017 Remastered Version
[09:40:47] DEBUG    asyncio_1 - No lyrics found for Emerson, Lake & Palmer - Nobody Loves You Like I   downloader.py:497
                    Do - 2017 Remastered Version, lyrics providers: Synced, Genius, AzLyrics,
                    MusixMatch
[09:40:47] INFO     asyncio_1 -                                                                        downloader.py:540
[09:40:48] DEBUG    asyncio_3 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - The      downloader.py:358
                    Enemy God Dances With the Black Spirits - 2017 Remastered Version
[09:40:48] DEBUG    asyncio_3 - No lyrics found for Emerson, Lake & Palmer - The Enemy God Dances With downloader.py:497
                    the Black Spirits - 2017 Remastered Version, lyrics providers: Synced, Genius,
                    AzLyrics, MusixMatch
[09:40:48] INFO     asyncio_3 -                                                                        downloader.py:540
None
[09:40:50] DEBUG    asyncio_2 - Synced failed to find lyrics for Emerson, Lake & Palmer - Closer to    downloader.py:358
                    Believing - 2017 Remastered Version
None
[09:40:50] DEBUG    asyncio_3 - Synced failed to find lyrics for Emerson, Lake & Palmer - Two Part     downloader.py:358
                    Invention in D Minor - 2017 Remastered Version
[09:40:50] DEBUG    asyncio_3 - Genius failed to find lyrics for Emerson, Lake & Palmer - Two Part     downloader.py:358
                    Invention in D Minor - 2017 Remastered Version
[09:40:51] DEBUG    asyncio_2 - Genius failed to find lyrics for Emerson, Lake & Palmer - Closer to    downloader.py:358
                    Believing - 2017 Remastered Version
[09:40:51] DEBUG    asyncio_3 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Two Part   downloader.py:358
                    Invention in D Minor - 2017 Remastered Version
[09:40:52] DEBUG    asyncio_2 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Closer to  downloader.py:358
                    Believing - 2017 Remastered Version
None
[09:40:53] DEBUG    asyncio_0 - Synced failed to find lyrics for Emerson, Lake & Palmer - L.A. Nights  downloader.py:358
                    - 2017 Remastered Version
[09:40:54] DEBUG    asyncio_0 - Genius failed to find lyrics for Emerson, Lake & Palmer - L.A. Nights  downloader.py:358
                    - 2017 Remastered Version
[09:40:54] DEBUG    asyncio_3 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Two Part downloader.py:358
                    Invention in D Minor - 2017 Remastered Version
[09:40:54] DEBUG    asyncio_0 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - L.A.       downloader.py:358
                    Nights - 2017 Remastered Version
[09:40:54] DEBUG    asyncio_3 - No lyrics found for Emerson, Lake & Palmer - Two Part Invention in D   downloader.py:497
                    Minor - 2017 Remastered Version, lyrics providers: Synced, Genius, AzLyrics,
                    MusixMatch
[09:40:54] DEBUG    asyncio_2 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Closer   downloader.py:358
                    to Believing - 2017 Remastered Version
[09:40:54] DEBUG    asyncio_2 - No lyrics found for Emerson, Lake & Palmer - Closer to Believing -     downloader.py:497
                    2017 Remastered Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:40:54] INFO     asyncio_3 -                                                                        downloader.py:540
[09:40:54] INFO     asyncio_2 -                                                                        downloader.py:540
None
[09:40:55] DEBUG    asyncio_1 - Synced failed to find lyrics for Emerson, Lake & Palmer - New Orleans  downloader.py:358
                    - 2017 Remastered Version
[09:40:55] DEBUG    asyncio_1 - Genius failed to find lyrics for Emerson, Lake & Palmer - New Orleans  downloader.py:358
                    - 2017 Remastered Version
[09:40:55] DEBUG    asyncio_1 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - New        downloader.py:358
                    Orleans - 2017 Remastered Version
[09:40:58] DEBUG    asyncio_0 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - L.A.     downloader.py:358
                    Nights - 2017 Remastered Version
[09:40:58] DEBUG    asyncio_0 - No lyrics found for Emerson, Lake & Palmer - L.A. Nights - 2017        downloader.py:497
                    Remastered Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:40:58] INFO     asyncio_0 -                                                                        downloader.py:540
[09:40:58] DEBUG    asyncio_1 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - New      downloader.py:358
                    Orleans - 2017 Remastered Version
[09:40:58] DEBUG    asyncio_1 - No lyrics found for Emerson, Lake & Palmer - New Orleans - 2017        downloader.py:497
                    Remastered Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:40:58] INFO     asyncio_1 -                                                                        downloader.py:540
None
[09:41:02] DEBUG    asyncio_3 - Synced failed to find lyrics for Emerson, Lake & Palmer - Food for     downloader.py:358
                    Your Soul - 2017 Remastered Version
None
[09:41:03] DEBUG    asyncio_2 - Synced failed to find lyrics for Emerson, Lake & Palmer - Tank - 2017  downloader.py:358
                    Remastered Version
[09:41:03] DEBUG    asyncio_3 - Genius failed to find lyrics for Emerson, Lake & Palmer - Food for     downloader.py:358
                    Your Soul - 2017 Remastered Version
[09:41:04] DEBUG    asyncio_2 - Genius failed to find lyrics for Emerson, Lake & Palmer - Tank - 2017  downloader.py:358
                    Remastered Version
None
[09:41:04] DEBUG    asyncio_3 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Food for   downloader.py:358
                    Your Soul - 2017 Remastered Version
[09:41:04] DEBUG    asyncio_0 - Synced failed to find lyrics for Emerson, Lake & Palmer - Pirates -    downloader.py:358
                    2017 Remastered Version
[09:41:05] DEBUG    asyncio_0 - Genius failed to find lyrics for Emerson, Lake & Palmer - Pirates -    downloader.py:358
                    2017 Remastered Version
[09:41:05] DEBUG    asyncio_2 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Tank -     downloader.py:358
                    2017 Remastered Version
[09:41:06] DEBUG    asyncio_0 - AzLyrics failed to find lyrics for Emerson, Lake & Palmer - Pirates -  downloader.py:358
                    2017 Remastered Version
[09:41:07] DEBUG    asyncio_3 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Food for downloader.py:358
                    Your Soul - 2017 Remastered Version
[09:41:07] DEBUG    asyncio_3 - No lyrics found for Emerson, Lake & Palmer - Food for Your Soul - 2017 downloader.py:497
                    Remastered Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:41:08] INFO     asyncio_3 -                                                                        downloader.py:540
[09:41:08] DEBUG    asyncio_2 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Tank -   downloader.py:358
                    2017 Remastered Version
[09:41:08] DEBUG    asyncio_2 - No lyrics found for Emerson, Lake & Palmer - Tank - 2017 Remastered    downloader.py:497
                    Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:41:08] INFO     asyncio_2 -                                                                        downloader.py:540
[09:41:09] DEBUG    asyncio_0 - MusixMatch failed to find lyrics for Emerson, Lake & Palmer - Pirates  downloader.py:358
                    - 2017 Remastered Version
[09:41:09] DEBUG    asyncio_0 - No lyrics found for Emerson, Lake & Palmer - Pirates - 2017 Remastered downloader.py:497
                    Version, lyrics providers: Synced, Genius, AzLyrics, MusixMatch
[09:41:10] INFO     asyncio_0 -                                                                        downloader.py:540
[09:41:10] ERROR    MainThread - https://open.spotify.com/track/3i5bc53F2glMZC7GFXZQ7T -               downloader.py:258
                    AttributeError: 'NoneType' object has no attribute 'lower'

Other details
No response

imo that's an issue with synced lyrics library. this function should return None instead of raising an exception

https://github.com/rtcq/syncedlyrics/pull/8
v4.1.8 will catch such exceptions and will debug print them

authors: azumukupoe xnetcat

repo: spotDL/spotify-downloader


text:
Migration fails on 'add_bootstrap_location' for postgres

I started seeing the following error when trying to run the backend on the latest code:
migration file "20200809202832_add_bootstrap_location.js" failed
migration failed with error: insert into "locations" ("id", "target", "type") values ($1, $2, $3) - invalid input syntax for type uuid: "bootstrap"

It seems related to #1890, however, I thought postgres migrations were run in CI now, so not sure how this crept through.

Yep, the bug is fixed by #1935, but I'll do a proper fix of the actual issue of tests not being run for that code :grin:

authors: Rugvip andrewthauer

repo: spotify/backstage


text:
Report when a time series is filtered out during ingest, try#2

Report dropped-by-filter when a filter drops a time series during ingest. This fixes a "XXX:" comment.
Also updated tests.

Current coverage is 45.18% (diff: 16.66%)

Merging #124 into master will decrease coverage by 0.01%

@@             master       #124   diff @@
==========================================
  Files           598        598
  Lines         15421      15427     +6
  Methods           0          0
  Messages          0          0
  Branches       1585       1585
==========================================
+ Hits           6970       6971     +1
- Misses         7989       7994     +5
  Partials        462        462

Powered by Codecov. Last update eef540e...472eea2

Thanks! Decrease in test coverage is due to the added lines in the semantic module. Ignoring them for now.

authors: codecov-io gabrielgerhardsson udoprog

repo: spotify/heroic


text:
Highlight root tasks in UI

There's currently no easy way of knowing which tasks in the web UI are root tasks. So if one task creates tens of sub tasks, there will be a large number of tasks in the UI but only one will actually show the entire task graph.
It'd be very useful if there was a way to filter the web UI to only show tasks that are root nodes so as to be able to easily see entire task graphs without specifically needing to know the names of those tasks.

Agreed. Want to send a PR? :)

If I get chance I'll look into it (but can't make any promises...)

closing this issue.

[x] It has been inactive for +4 months.
[ ] It's not about luigi core, so not as many users are affected about this.
[ ] The change seems quite big, it's unlikely to be sporadically picked up.
[x] The owner haven't responded or disappeared.
[ ] I don't understand what this issue is about.
[ ] There exists a reasonable workaround for this.
[ ] We need to check if this hasn't been fixed by now (for old issues).
[ ] This is kind of by design and not a bug.
[ ] Resolving this would probably add a lot of complexity.

Every open issue adds some clutter, and we try to make the issues fewer and make it easier for new collaborators to find. Currently we try to close any issue that meets the first checkbox + one other.
Feel free to reopen this issue at any point if you have the intent to continue to work this. :)

authors: Tarrasch boosh mfcabrera

repo: spotify/luigi


text:
[Spark Configuration] Can't pass Spark property using the conf when the option contains an equals

In spark configuration, the conf can't have some options with an equals inside:
[spark]
conf: spark.executor.extraJavaOptions=-Darchaius.deployment.environment=dev
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/luigi/worker.py", line 162, in run
new_deps = self._run_get_new_deps()
File "/usr/local/lib/python2.7/dist-packages/luigi/worker.py", line 113, in _run_get_new_deps
task_gen = self.task.run()
File "/usr/local/lib/python2.7/dist-packages/luigi/contrib/spark.py", line 235, in run
args = list(map(str, self.spark_command() + self.app_command()))
File "/usr/local/lib/python2.7/dist-packages/luigi/contrib/spark.py", line 214, in spark_command
command += self._dict_arg('--conf', self.conf)
File "/usr/local/lib/python2.7/dist-packages/luigi/contrib/spark.py", line 138, in conf
return self._dict_config(configuration.get_config().get("spark", "conf", None))
File "/usr/local/lib/python2.7/dist-packages/luigi/contrib/spark.py", line 261, in _dict_config
return dict(map(lambda i: i.split('='), config.split('|')))
ValueError: dictionary update sequence element #0 has length 3; 2 is required
As a fix, it could be split only on the first equals :
luigi/contrib/spark.py
  def _dict_config(self, config):
        if config and isinstance(config, six.string_types):
            return dict(map(lambda i: i.split('=',1), config.split('|')))

great if you want to submit a PR!

authors: MezianeMehdi erikbern


text:
Make task.disable_window be default source of window int

Description

When accessing task.disable_window property, use the property directly. Do not call the deprecated task.disable_window_seconds.
Motivation and Context

Fixes #3029
Current implementation gives a deprecation warning even when accessing the correct task.disable_window property, which is confusing. Fixing this means that the deprecation warning becomes more meaningful - it will only appear when task.disable_window_seconds is incorrectly accessed.
Have you tested this? If so, how?

Have ran this change locally for me in my employers' Luigi test suite and removed 21 warnings when running a small test on a single task. However, that test suite does not use disable_window.

Please could one of @dlstadther , @Tarrasch or another maintainer approve the workflow so that tests run?


text:
Improve addition of Java to Docker image

Previously, the Docker file added a bunch of utilities and downloaded the version of Java directly in the image that was eventually created.  This left a bunch of unnecessary and potentially vulnerable packages on the image that was used in production.  This change makes the build a multi-stage build and ensures that the network utilities required for downloading only exist on a disposed stage.
In addition to the change to a multi-stage build, this change also swaps from the Pivotal Distribution of OpenJDK to AdoptOpenJDK as part of our commitment to move to an industry standard distribution.  It also swaps from Java 8 to Java 11.  Trust me, you'll be fine.

I run few manual checks of the ./wait-for-it.sh script and can confirm that it is working as expected.
LGTM

@nebhale Per some internal team discussions, we feel like we need more time to do testing with the apps that run on SCDF to ensure that they run on both JDK 8 and 11 without any issues. We are going to do such testing in the next release cycle. Can we keep this PR with all the cleanup that you added, but use the latest JDK 8 that has all the patches applied?

@sobychacko You should feel free to make that change before merging.

Should the 11 version be updated to the latest?

@sobychacko - Can we merge this to a JDK11 branch for now?

Closing in lieu of #5

authors: dturanski nebhale sobychacko tzolov

repo: spring-cloud/openjdk-docker


text:
client with multiple profiles

Describe the bug
I try to start my client application with multiple profiles by setting the active profile to: stage, support-stage
I even see this line in the log:
2021-07-01 17:20:20.236  INFO [springboot-no-ui-tests,,]  [  restartedMain] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=springboot-no-ui-tests, profiles=[stage,support-stage], label=null, version=e9e9fda77b46039faa314499ba133f7af8411ef9, state=null
but for some reason only the first profile is being updated from the config server.
Does Spring cloud config knows how to handle multiple profiles coming from clients?

The reason behind it, is because we want to have two repositories in GIT one for developers to change for their configuration and one for support people. When the client will run we want it to read the configuration from both places for the same app.
for example here is the Spring cloud config server configuration for that:
repos:
cloud-waf-support:
pattern:
- 'springboot-no-ui-tests/support*'
cloneOnStart: true
uri: https://.../v1/repos/cloud-waf-support
default-label: main
cloud-waf:
pattern:
- 'springboot-no-ui-tests/*'
cloneOnStart: true
uri: https://.../v1/repos/cloud-waf
default-label: main
If you have another idea that will solve the issue it can be great

Eventually I solve this requirement: repo for support and repo for dev with the composite solution:
cloud:
config:
server:
composite:
-
type: git
uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/fallback-repo
default-label: main
force-pull: true
username: yyy
password: xxx
clone-on-start: true
repos:
springboot-no-ui-tests:
clone-on-start: true
pattern:
- 'springboot-no-ui-tests'
clone-submodules: false
uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/dev-repo
force-pull: true
default-label: main
username: yyy
password: xxx
-
type: git
uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/fallback-repo
default-label: main
force-pull: true
username: xxx
password: yyy
clone-on-start: true
repos:
springboot-no-ui-tests:
clone-on-start: true
pattern:
- 'springboot-no-ui-tests'
clone-submodules: false
uri: https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/support-repo
force-pull: true
default-label: main
username: yyy
password: xxx

This can be closed.

repo: spring-cloud/spring-cloud-config