text:
Wrong number of inputs reported

After running  a few times in to sounddevice.PortAudioError: Error opening InputStream: Invalid number of channels [PaErrorCode -9998] I noticed sounddevice reports the wrong number of inputs in some cases. It should be 4 in 4 out for the Behringer, as PyAudio reports.
sounddevice: 1246720 PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4
pyaudio    : 1246720 PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4

Sounddevice devices:
   0 Microsoft Sound Mapper - Input, MME (2 in, 0 out)
>  1 Line (2- BEHRINGER UMC404HD 192, MME (2 in, 0 out)
   2 Microsoft Sound Mapper - Output, MME (0 in, 2 out)
<  3 Speakers (2- BEHRINGER UMC404HD, MME (0 in, 2 out)
   4 Digitale audio (S/PDIF) (High D, MME (0 in, 2 out)
   5 Primary Sound Capture Driver, Windows DirectSound (2 in, 0 out)
   6 Line (2- BEHRINGER UMC404HD 192k), Windows DirectSound (2 in, 0 out)
   7 Primary Sound Driver, Windows DirectSound (0 in, 2 out)
   8 Speakers (2- BEHRINGER UMC404HD 192k), Windows DirectSound (0 in, 2 out)
   9 Digitale audio (S/PDIF) (High Definition Audio Device), Windows DirectSound (0 in, 2 out)
  10 UMC ASIO Driver, ASIO (4 in, 4 out)
  11 Speakers (2- BEHRINGER UMC404HD 192k), Windows WASAPI (0 in, 2 out)
  12 Digitale audio (S/PDIF) (High Definition Audio Device), Windows WASAPI (0 in, 2 out)
  13 Line (2- BEHRINGER UMC404HD 192k), Windows WASAPI (4 in, 0 out)
  14 Speakers (UMC404HD 192k Output), Windows WDM-KS (0 in, 4 out)
  15 Line (UMC404HD 192k Input), Windows WDM-KS (4 in, 0 out)
  16 SPDIF Out (HD Audio SPDIF out), Windows WDM-KS (0 in, 2 out)

PyAudio:
 0   Microsoft Sound Mapper - Input    2 in, 0 out
 1   Line (2- BEHRINGER UMC404HD 192   4 in, 0 out
 2   Microsoft Sound Mapper - Output   0 in, 2 out
 3   Speakers (2- BEHRINGER UMC404HD   0 in, 4 out
 4   Digitale audio (S/PDIF) (High D   0 in, 2 out
 5   Primary Sound Capture Driver      2 in, 0 out
 6   Line (2- BEHRINGER UMC404HD 192k) 4 in, 0 out
 7   Primary Sound Driver              0 in, 2 out
 8   Speakers (2- BEHRINGER UMC404HD 192k) 0 in, 4 out
 9   Digitale audio (S/PDIF) (High Definition Audio Device)   0 in, 2 out
10   UMC ASIO Driver   4 in,  4 out
11   Speakers (2- BEHRINGER UMC404HD 192k)   0 in,  2  out
12   Digitale audio (S/PDIF) (High Definition Audio Device)   0 in, 2 out
13   Line (2- BEHRINGER UMC404HD 192k)   4 in,  0 out
14   Speakers (UMC404HD 192k Output)   0 in,  4 out
15   Line (UMC404HD 192k Input)   4 in,  0 out
16   SPDIF Out (HD Audio SPDIF out)   0 in,  2 out

This is indeed very strange.
Since they appear to be using the same PortAudio version, I'd expect the sounddevice module and PyAudio to report exactly the same information ...
Did you run the two queries in exactly the same way?
Same command line, same device permissions?

The script, manually edited the out put too look as above.
import sounddevice
import pyaudio
p = pyaudio.PyAudio()

version, description = sounddevice.get_portaudio_version()
print("sounddevice: {} {}".format(version, description))
print("pyaudio    : {} {}".format(pyaudio.get_portaudio_version(), pyaudio.get_portaudio_version_text()))

print("\nSounddevice devices:\n",sounddevice.query_devices())

print("\nPyAudio")
print("Host API")
for host in enumerate(range(0, p.get_host_api_count())):
    info = p.get_host_api_info_by_index(host[1])
    print (f"\t{host[0]}: {info['name']}")
    print("\t\tAudio device")
    for device in enumerate(range(0, p.get_device_count())):
        info = p.get_device_info_by_index(device[1])
        if host[0] == info["hostApi"]:
            print (f"\t\t{device[0]}: {info['name']}\tmax in: {info['maxInputChannels']} max out: {info['maxOutputChannels']}")
    print("")

Thanks for the update.
I still have no clue, though.
I must assume that different DLLs are in use.
Can you somehow find out which DLLs are used?
For sounddevice, you can check sounddevice._libname.

All I could find was a _portaudio.cp38-win_amd64.pyd I assume is for PyAudio.
I tried a bit more and got more confused. Running sounddevice:
python wire -i1 -o3 -c4
PortAudioError: Error opening Stream: Invalid number of channels [PaErrorCode -9998]

1 & 3 are windows MME and should only support 2 channels i.i.r.c. So the error is right.
Running a similar wire with PyAudio works fine. Recording 4 channels with the MME works fine too.
Had a look at Audacity, with the same PortAudio version, only two channels for MME.
My guess would be there's a problem in PyAudio, not in Sounddevice.

How did you install the sounddevice module?
There are at least 3 ways described at https://python-sounddevice.readthedocs.io/en/0.4.1/installation.html, which all should lead to different DLLs to be used.
Can you please try all three and report the sounddevice._libname for each of them?

Installed with pip.
Just created two new venv's and tried pip again and Gohlke. Don't have Anaconda or WinPython installed.
pip:
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sounddevice
>>> print(sounddevice._libname)
...\Documents\Python\envSD_pip\lib\site-packages\_sounddevice_data\portaudio-binaries\libportaudio64bit.dll
>>> version, description = sounddevice.get_portaudio_version()
>>> print("sounddevice: {} {}".format(version, description))
sounddevice: 1246720 PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4

Gohlke:
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sounddevice
>>> print(sounddevice._libname)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'sounddevice' has no attribute '_libname'
>>> version, description = sounddevice.get_portaudio_version()
>>> print("sounddevice: {} {}".format(version, description))
sounddevice: 1246720 PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4

In the Gohlke venv the library is at ...\Documents\Python\envSD_gohlke\Lib\site-packages\portaudio.dll

Thanks for the update!
So the Gohlke package works as expected, right?

Yes, it works like the PyAudio version.

OK, good to know, you should use the Gohlke package then!
AFAIK, the main difference is that the Gohlke DLL was created on Windows with the MS compiler, while the DLL in the "official" package was cross-compiled on Linux using MXE/MinGW-w64/GCC, see https://github.com/spatialaudio/portaudio-binaries.
If anybody has an idea how to fix the problem in the cross-compilation, please let me know!

Thanks for your effort.
Guten Rutsch!

Hello,
I'm having the same problem, @Moaneschien could you please clarify how did you solve this issue?
thanks!

authors: Moaneschien davircarvalho mgeier

repo: spatialaudio/python-sounddevice


text:
Docs - DOC-1332 - Adds VS 2019 support

Tested with improbadoc

There's still a 2017 reference here.

Also, someone should run the set-up to ensure that our recommended Workloads ares still accurate.

I think that reference is ok because we still offer 2017 as an option?

source: gharchive/pull-request

authors: ernopp oblm

repo: spatialos/UnrealGDK

license: MIT

license_type: permissive

license_source: github-api


text:
add Philippine national holidays

I have based the data I have gathered from several official documents of the Philippine Government

Holidays of the Philippines for 2024 link
Araw ng Kagitingan Legislation link
National Heroes Day link

rebased and squashed!

Thanks!

authors: freekmurze kndrckjvr

repo: spatie/holidays


text:
Keep Ray on top missing for Windows

Describe the bug
It seems that https://spatie.be/docs/ray/v1/the-ray-desktop-app/discovering-the-ray-app#keep-ray-on-top is missing for the Windows app.
It clearly seems like a bug to me if you advertise a feature (without mentioning exceptions) and not having the functionality available.
I have no option to disable the window being always on top which can be frustrating.
Versions
Ray version (you can see this in "About Ray"): 1.14.7.0
Expected behavior
I'd expect the toggle feature on the view menu like shown in the screenshot.
Screenshots

Desktop (please complete the following information):

OS: [e.g. macOS] Windows 10 20H2 (Build 19042.928)

Hey @Morgy93, you can toggle the float-on-top using the second button from the right in the menubar (screenshot is on Mac, but it should be the same on Windows).

Great idea to also have it in the View menu, we'll consider adding this!

Holy... I'd never ever thought that the icon would be responsible for that.
@AdrianMrn You should also update the docs, because then it's not well documented + other screenshots show way different functionality / ui elements.

Thanks for clarification anyway!

Yeah, the app went through some UI changes since those were made 😅
I've written it down and will improve this!

source: gharchive/issue

repo: spatie/ray


text:
Event on opening and closing a tab

Is there a way to trigger an event when you switch to and switch back from a tab?

I got it solved. Took the changed event emitted during the tab switch.

repo: spatie/vue-tabs-component


text:
Design Issue on Submit License Page

On the page Submit New License, there is a patch of white space between the above toolbar and the start of the textbox as we can see in the image. This should not be there as it is a wastage of space as well as not solving anything

Upon further inspection, the reason why this is occurring is that we have intentionally left space for the modal messages above, instead what we can do is that we will need to change the way we handle the modal messages in the base.html which will automatically solve this issue

I have modified this and the new page would look something like this

Resolved with PR #459

repo: spdx/spdx-online-tools


text:
fixed 404 in documentation

links were pointing to ./commandApp, however
the generated page is ./commandapp

Merged! Thank you for your contribution. Much appreciated! 👍

authors: nils-a patriksvensson


text:
Allow a JSON configuration file for configuring the SpeedmentLifecycle

Useful for different deployment scenarios. This could make all the .with() methods configurable via a file.

This might be related to #30.

authors: Pyknic minborg

repo: speedment/speedment

license_source: bigquery


text:
docs: Add sourceRelativeLinks to Blackfriday options

Needs docs for this commit: https://github.com/spf13/hugo/commit/0f6b334b6715253b030c4e783b88e911b6e53e56

@SvenDowideit,
I was looking at documenting these changes from back in September, but I have a question:  what does the sourceRelativeLinks have to do with Blackfriday?
All of the config options under blackfriday have to do with toggling Blackfriday extensions.  Your sourceRelativeLinks option does not.  Should the sourceRelativeLinks feature be decoupled from the Blackfriday options?

Already work in progress in #1977

I'm not sure :)
at this point, its a blackfriday only - it hasn't been implemented when using the MMark renderer.
setting the option overrides the blackfriday link and file render functions.

Fixed by #1977

authors: SvenDowideit moorereason


text:
Improve README

A mostly minor commit improving the readability (source) of the README
and making it slightly better for users, too (HTTPS links, syntax highlighting,
correct links).
Commit message:
All-around improvements to the README.
Each sentence now gets its own line to eliminate ultra-long lines in the
source (expect in bullet points--separating a bullet point into multiple
lines looks weird and decreases readability).
HTTPS links are now preferred to HTTP links and used where applicable.
Link to Mercurial's website corrected to www.mercurial-scm.org
Link to Hugo's docs is now accessible with [Hugo Documentatio][] since
it is used more than once in the README.
Multiline code blocks are now fenced rather than indented and include
syntax highlighting.

Signed-off-by: Ethan Madison <ethan@ethanmad.com>

Visually much better. Merged in 5cf1475

authors: bep ethanmad


text:
fix(docs/reference): Expand description of several pipeline stages.

Specifically: Clone Server Group, Enable Server Group, Find Image From
Cluster, Jenkins, and Manual Judgment.

@duftler would mind you taking another look at this?

license: mit


text:
[WIP] Add ByteBoolArray type and fixe a bug in BoolArray

The main two contributions in this PR are:

A new ByteBoolArray type (closes #335)
An important fix for slicing and initialization of BoolArray

The other high-level comment, we should move this encoding into a crate. The main vortex-array encodings are (mostly) only those that convert 1:1 to Arrow. Anything else is an "extension"

Seems like one of my recent changes broke tests, looking into

authors: AdamGS gatesn

repo: spiraldb/vortex


text:
New term suggestions

I just cleaned out my spam, here's some suggestions:
gginza
louisvuitton
calvinklein
kakaku
rolex
wtobrand
gowatchs
eevance
brandiwc
bestevance
ooobag
ooobrand
ooowatch
laferradura

Thanks for the suggested additions.
I will add most of these to the blacklist (along with their variations), with the exception of a few …
Terms such as louisvuitton, calvinklein, laferradura, kakaku, and rolex are common brand names used in legitimate website addresses. Including them in the blacklist would filter legitimate websites such as rolex.com or calvinklein.ca.
If these terms appear with other “spammy” words or phrases, then I would include them. Is it possible to provide a sample of the entire comment text where some of these terms appeared?

New terms added per commit: 622e924a2bf29328abcee45b7aeb59fca5439cef

authors: gummylick splorp

repo: splorp/wordpress-comment-blacklist


text:
Error: Enable splunk boot-start via systemd failing

The configure_splunk_boot.yml  playbook fails with the error below for a new VM:
fatal: [lm-host]: FAILED! => {"changed": true, "cmd": ["/srv/splunk/splunk/bin/splunk", "enable", "boot-start", "-user", "splunk", "-systemd-managed", "1", "--answer-yes", "--auto-ports", "--no-prompt", "--accept-license"], "delta": "0:00:00.890923", "end": "2021-10-06 15:42:54.740175", "msg": "non-zero return code", "rc": 4, "start": "2021-10-06 15:42:53.849252", "stderr": "Copying '/srv/splunk/splunk/etc/openldap/ldap.conf.default' to '/srv/splunk/splunk/etc/openldap/ldap.conf'.\nbtool returned something in stderr: 'Couldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied\nCouldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied\nCouldn't open \"/srv/splunk/splunk/etc/splunk-launch.conf\": Permission denied\ncouldn't run \"(null)/bin/splunkd\" \"btool\": Permission denied\n'\nCouldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied\nCouldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied\nCouldn't open \"/srv/splunk/splunk/etc/splunk-launch.conf\": Permission denied\n\n\nAn error occurred: Could not create audit keys (returned 8).\nFirst-time run failed!", "stderr_lines": ["Copying '/srv/splunk/splunk/etc/openldap/ldap.conf.default' to '/srv/splunk/splunk/etc/openldap/ldap.conf'.", "btool returned something in stderr: 'Couldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied", "Couldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied", "Couldn't open \"/srv/splunk/splunk/etc/splunk-launch.conf\": Permission denied", "couldn't run \"(null)/bin/splunkd\" \"btool\": Permission denied", "'", "Couldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied", "Couldn't open \"/srv/splunk/splunk/bin/../etc/splunk-launch.conf\": Permission denied", "Couldn't open \"/srv/splunk/splunk/etc/splunk-launch.conf\": Permission denied", "", "", "An error occurred: Could not create audit keys (returned 8).", "First-time run failed!"], "stdout": "ERROR: Couldn't read \"/srv/splunk/splunk/etc/splunk-launch.conf\" -- maybe $SPLUNK_HOME or $SPLUNK_ETC is set wrong?\n\nThis appears to be your first time running this version of Splunk.", "stdout_lines": ["ERROR: Couldn't read \"/srv/splunk/splunk/etc/splunk-launch.conf\" -- maybe $SPLUNK_HOME or $SPLUNK_ETC is set wrong?", "", "This appears to be your first time running this version of Splunk."]}
Any thoughts on what the problem or how it can be fixed?

@morufajibike How did you solve this issue?

I did successfully solved this problem by running this command since it was the problem with permissions.
chmod -R ug+rwX /opt/splunk

authors: karbovski morufajibike

repo: splunk/ansible-role-for-splunk


text:
Suspiciously named exe

PR Template for new Detections
For Authors:

[x] Make sure that CI/CD detection-testing and build-and-validate jobs passed ✔.

For Reviewers:

[ ] Verify CI/CD jobs have passed without errors.
[ ] Validate SPL logic.
[ ] Validate tags, description, and how to implement.
[ ] Validate name matches <platform>_<mitre att&ck technique>_<short description>
[ ] Verify references match analytic.
[ ] Is there an Atomic Test?

Closing for now confirmed with Glory detection building process is being built to addresses these questions.


text:
Nterl0k - T1087.002 - 4662 EventID AD Enumeration

Pending https://github.com/splunk/attack_data/pull/816
Details
Showcases when attackers enumerate AD in mass or for specific high priv (default) groups.
Must allow 4662 events to be ingested by Splunk, which is typically advised against/filtered by the Windows TA.
Checklist

[ ] Validate name matches <platform>_<mitre att&ck technique>_<short description> nomenclature
[ ] CI/CD jobs passed ✔
[ ] Validated SPL logic.
[ ] Validated tags, description, and how to implement.
[ ] Verified references match analytic.

Hello @nterl0k :We are getting some errors for windows_ad_abnormal_object_access_activity.yml. Here is how the given attack data looks in Splunk : you can see that the ObjectName_count is NOT greater than the limit. Do you suspect an issue with the search/ the dataset ? You may need to dump a new dataset for this detetion
Screenshot without the | where ObjectName_count > limit condition

In our production environment the stddev*3 math seemed to be the sweet spot to avoid the detection being overly sensitive.
I can artificially inflate the dataset to make it work as expected.
If I make a PR to the attack dataset can that be expedited to help with this?
Regards,
Steven.
-------- Original message --------
From: Bhavin Patel @.>
Date: 9/12/23 7:56 PM (GMT-05:00)
To: splunk/security_content @.>
Cc: Steven Dick @.>, Mention @.>
Subject: Re: [splunk/security_content] Nterl0k - T1087.002 - 4662 EventID AD Enumeration (PR #2716)
Hello @nterl0khttps://github.com/nterl0k :We are getting some errors for windows_ad_abnormal_object_access_activity.yml. Here is how the given attack data looks in Splunk : you can see that the ObjectName_count is NOT greater than the limit. Do you suspect an issue with the search/ the dataset ? You may need to dump a new dataset for this detetion
Screenshot without the | where ObjectName_count > limit condition
[image]https://user-images.githubusercontent.com/7771446/267493748-a79c1a71-2249-4c33-840e-c9e92f426589.png
—
Reply to this email directly, view it on GitHubhttps://github.com/splunk/security_content/pull/2716#issuecomment-1716731385, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJIYP7UP45GKRC5DFJVJLILX2DZBHANCNFSM6AAAAAAZEYZPUA.
You are receiving this because you were mentioned.Message ID: @.***>

@patel-bhavin - I've built an updated dataset that will trigger the detection as expected. - See PR https://github.com/splunk/attack_data/pull/833 for the updated data.

awesome! i merged the attack_data PR and looks like this passes our detection testing now! Thank you for being so prompt as always. Closing this PR and we will be merging the updated PR that contains your content and will be released in ESCU 4.12.0
https://github.com/splunk/security_content/pull/2810


text:
Add custom field to the component image schema

Add custom data property to component image #134

Current coverage is 92.67% (diff: 100%)

Merging #157 into master will increase coverage by <.01%

@@             master       #157   diff @@
==========================================
  Files            62         62
  Lines          4504       4508     +4
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           4174       4178     +4
  Misses          330        330
  Partials          0          0

Powered by Codecov. Last update 6760c7b...5329900

@zhubofei Now I'm going to be extremely picky, so forgive me 😅 But can you please fix the last commit message: "nit" doesn't say anything about what the commit contains. Something like "Remove unwanted space" or something would be a lot better. The reason I'm a bit picky about this, is because when bisecting the git history (or working with the log in another way), commit messages like this make it very hard to know what's going on 😄 Another option is for you to just squash your two commits together and force push onto this PR.

authors: JohnSundell codecov-io zhubofei

repo: spotify/HubFramework


text:
Logo click reloads page

Clicking the logo should not reload page, but instead work as the other navigation elements:

Fixed in https://github.com/spotify/backstage/pull/1178

repo: spotify/backstage


text:
problem pushing specific tag

Description
When trying to push a specific tag it is dropping the port from the image name and failing to push the private registry due to the image not existing.
How to reproduce
$ mvn docker:build -DpushImageTag -DdockerImageTag=latest
<..snip..>
Successfully built cfe4766f18d5
[INFO] Built docker-lab.example.com:5000/testwebapp
[INFO] Tagging docker-lab.example.com:5000/testwebapp with latest
[INFO] Tagging docker-lab.example.com:5000/testwebapp with 1.0-SNAPSHOT
[INFO] Tagging docker-lab.example.com:5000/testwebapp with test
[INFO] Pushing docker-lab.example.com:5000/testwebapp
The push refers to a repository [docker-lab.example.com:5000/testwebapp]
30aee42f43f3: Layer already exists
30aee42f43f3: Layer already exists
30aee42f43f3: Layer already exists
00ceab1d04a2: Layer already exists
f59b7e59ceaa: Layer already exists
null: null
null: null
[INFO] Pushing docker-lab.example.com:latest
The push refers to a repository [docker.io/library/docker-lab.example.com]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.427s
[INFO] Finished at: Thu Aug 11 15:10:38 EDT 2016
[INFO] Final Memory: 18M/947M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.11:build (default-cli) on project TestWebApp: Exception caught: An image does not exist locally with the tag: docker-lab.example.com -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
pom.xml plugin configuration:
 <plugin>
    <groupId>com.spotify</groupId>
    <artifactId>docker-maven-plugin</artifactId>
    <version>0.4.11</version>
    <configuration>
      <imageName>docker-lab.example.com:5000/testwebapp</imageName>
      <dockerDirectory>docker</dockerDirectory>
      <resources>
        <resource>
          <targetPath>/</targetPath>
          <directory>${project.build.directory}</directory>
          <include>${project.build.finalName}.war</include>
        </resource>
      </resources>
      <imageTags>
        <imageTag>latest</imageTag>
        <imageTag>${project.version}</imageTag>
        <imageTag>test</imageTag>
      </imageTags>
      <serverId>docker-lab</serverId>
      <registryUrl>https://docker-lab.example.com:5000</registryUrl>
    </configuration>
  </plugin>

What do you expect
Push docker-lab.example.com:5000/testwebapp:latest
What happened instead
It tries to push docker-lab.example.com:latest which does not exist.
Software:

docker version: 1.12.0 build 8eab29e
docker-maven-plugin version: 0.4.11
maven version: 3.0.5

Full backtrace
$ mvn docker:build -DpushImageTag -DdockerImageTag=latest
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building TestWebApp Maven Webapp 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- docker-maven-plugin:0.4.11:build (default-cli) @ TestWebApp ---
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] Copying /home/tsaxon/TestWebApp/target/TestWebApp.war -> /home/tsaxon/TestWebApp/target/docker/TestWebApp.war
[INFO] Copying docker/Dockerfile -> /home/tsaxon/TestWebApp/target/docker/Dockerfile
[INFO] Building image docker-lab.example.com:5000/testwebapp
Step 1 : FROM centos:7
---> 970633036444
Step 2 : MAINTAINER systemengineering@example.com
---> Using cache
---> 9871bd89844f
Step 3 : ENV TOMCAT_MAJOR 8 TOMCAT_VERSION 8.0.36 CATALINA_HOME /usr/local/tomcat JAVA_HOME /usr/local/jdk PATH $CATALINA_HOME/bin:$PATH
---> Using cache
---> 38843add5425
Step 4 : ENV TOMCAT_TGZ_URL https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz
---> Using cache
---> b3ebab892804
Step 5 : RUN set -x     && yum -y update     && yum clean all     && yum -y install wget openssl     && wget -O /tmp/tomcat.tar.gz "$TOMCAT_TGZ_URL"     && wget --no-cookies --no-check-certificate -O /tmp/jdk.tar.gz --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u102-b14/jdk-8u102-linux-x64.tar.gz"     && mkdir -p /usr/local/tomcat && mkdir -p /usr/local/jdk     && tar xvf /tmp/tomcat.tar.gz -C $CATALINA_HOME --strip-components=1     && tar xvf /tmp/jdk.tar.gz -C $JAVA_HOME --strip-components=1     && rm -rf /tmp/*.tar.gz
---> Using cache
---> 66ac8693136c
Step 6 : ADD TestWebApp.war /usr/local/tomcat/webapps/
---> Using cache
---> 9fafdb39d756
Step 7 : CMD /usr/local/tomcat/bin/startup.sh && tail -f /usr/local/tomcat/logs/catalina.out
---> Using cache
---> cfe4766f18d5
ccessfully built cfe4766f18d5
▽INFO] Built docker-lab.example.com:5000/testwebapp
[INFO] Tagging docker-lab.example.com:5000/testwebapp with latest
[INFO] Tagging docker-lab.example.com:5000/testwebapp with 1.0-SNAPSHOT
[INFO] Tagging docker-lab.example.com:5000/testwebapp with test
[INFO] Pushing docker-lab.example.com:5000/testwebapp
The push refers to a repository [docker-lab.example.com:5000/testwebapp]
30aee42f43f3: Layer already exists
30aee42f43f3: Layer already exists
30aee42f43f3: Layer already exists
00ceab1d04a2: Layer already exists
f59b7e59ceaa: Layer already exists
null: null
null: null
[INFO] Pushing docker-lab.example.com:latest
The push refers to a repository [docker.io/library/docker-lab.example.com]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.466s
[INFO] Finished at: Thu Aug 11 15:01:07 EDT 2016
[INFO] Final Memory: 18M/947M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.11:build (default-cli) on project TestWebApp: Exception caught: An image does not exist locally with the tag: docker-lab.example.com -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Problem exists between keyboard and chair.... needed to use -DpushImageTags not -DpushImageTag

repo: spotify/docker-maven-plugin


text:
Added BigQuery chunking logic for huge uploads

Motivation and Context
Some people are uploading huge amounts of data (15+ TB) and they are hitting BQ Timeouts. The easiest solution is to chunk data and upload partially.
Description
We added two new Luigi Parameters:

Boolean chunking-enabled - when True (default is False) it will chunk your source data and upload synchronously with a default chunk size (1 TB)
Int chunk-size-gb - user can adjust (optionally) chunk size

If chunking is disabled, then the upload logic is not changed. If chunking is enabled, we will upload chunks to separated temp tables, then we will atomically merge them and store result in one final table. All the temp tables are removed after successful run.
We have a heuristic calculating the size of the input data to be chunked, so chunk size can vary. We are also assuming that files within a a folder to be chunked are of a similar size. So we can handle let's say 22000 files 90MB each or 500 files 17GB each.
Have you tested this? If so, how?
We have included unit tests and did some stress and manual tests in house.
We will be cleaning up some unnecessary printouts, we are looking for some architectural and technical feedback.
Thanks!

One general comment on the scope of this PR:
I struggle to see which exact problem this PR is trying to solve. Loading data into separate tables and merging them together feels like a highly customized solution for a very specific problem.
If timeouts are a problem that should be addressed shouldn't we rather look at resumable uploads as per https://cloud.google.com/bigquery/loading-data-post-request#resumable ?

authors: BartoszJanota dmohns

repo: spotify/luigi


text:
Add support for Data Type

Related roadmap page

Support for four family types:

[ ] object including Map types
[ ] scalar

[ ] built-in
[ ] enums
[ ] custom scalars

[ ] array
[ ] external

Supported functionality:

[ ] Inheritance for objects; mindful of restrictions
[ ] Type Expression
[ ] Union types
[ ] Shortcuts & syntatic sugar
[ ] Runtime polymorphism
[ ] Examples
[ ] Inline declaration

Data Types support for:

[ ] URI Parameters
[ ] Query parameters
[ ] Request & Response Headers
[ ] Request & Response Body

JSON
XML
Form

[ ] Documentation
[ ] Test coverage

I take this bug, can you assign it to me?

@tardyp done! but I think I shouldn't have needed to add you to the spotify org - I just added you as a collaborator, but let me double check sine github org permissions are not that easy to immediately understand.

Hi  @econchick & @tardyp, not sure what the progress is but the links to the different sections inside the spec are out of date. I also like to offer you as much help as possible from our experts as needed. Let me know.

Hi Christian,
Both myself and @econchick had a very busy second quarter, so this is why this is not progressing as we would like. For my side, I am still very interrested in raml potencial for python webapps, and willing not make it move forward as time allows.

@sichvoge hey Christian!
You're right that the links are outdated - but (FYI @tardyp 😄 ) I have been working on it! and current progress can be seen in the v.0.2.0-dev branch (although I have yet to push latest commits from the past couple of days which really work on this particular ticket).
I've updated this issue with appropriate links (and I will do that for the other issues at some point) and current status of progress.

I just forked, are the issues you have listed for this repo the only issues to tackle in order to accomplish the upgrade to 1.0?

I think @econchick and maybe you should review the todos in this issue, and maybe others with the current spec version.

authors: dethbird econchick sichvoge tardyp

repo: spotify/ramlfications


text:
Added Spree::Tracker validation on active field - only one active tra…

…cker per store permitted

@nnande could you please rebase with main branch?


text:
Update README.md

Fix gemfile instructions

Thanks

authors: JDutil MisinformedDNA


text:
Unable to load general settings [WARNING] No :original defined for 'multiple-currency'

The page admin/general_settings/edit doesn't get load and server show following error
  ↳ /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/bundler/gems/spree_admin_roles_and_access-b1fbacfc01df/app/models/spree/ability_decorator.rb:16
  Spree::Permission Load (0.5ms)  SELECT "spree_permissions".* FROM "spree_permissions" WHERE "spree_permissions"."id" = $1 ORDER BY "spree_permissions"."priority" ASC  [["id", 1]]
  ↳ /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/bundler/gems/spree_admin_roles_and_access-b1fbacfc01df/app/models/spree/ability_decorator.rb:16
Deface: 2 overrides found for 'spree/admin/general_settings/edit'
Deface: 'add_favorite_products_per_page_configuration' matched 1 times with '#preferences fieldset .form-actions'
Deface: 'multiple-currency' matched 2 times with '.form-group'
Deface: [WARNING] No :original defined for 'multiple-currency', you should change its definition to include:
 :original => 'fc7a9c51741e715dbefca394dac231931eb37782'
Deface: [WARNING] No :original defined for 'multiple-currency', you should change its definition to include:
 :original => 'a088b96e7cfcddb5a80d89f2b3341275d566125b'
Completed 500 Internal Server Error in 212ms (ActiveRecord: 90.9ms)

exception reentered excluded from capture: DSN not set

fatal (exception reentered):

#<Thread:0x000055d89ca82af8@puma 002@/home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/thread_pool.rb:89 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
/home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `catch': exception reentered (fatal)
/home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `catch': can't modify frozen fatal (FrozenError)
	80: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/thread_pool.rb:135:in `block in spawn_thread'
	79: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/server.rb:334:in `block in run'
	78: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/server.rb:474:in `process_client'
	77: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/server.rb:660:in `handle_request'
	76: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/configuration.rb:227:in `call'
	75: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/engine.rb:524:in `call'
	74: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/sentry-raven-2.9.0/lib/raven/integrations/rack.rb:51:in `call'
	73: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/rack-2.0.7/lib/rack/sendfile.rb:111:in `call'
	72: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/static.rb:127:in `call'
	71: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
	70: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/rack-2.0.7/lib/rack/runtime.rb:22:in `call'
	69: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/rack-2.0.7/lib/rack/method_override.rb:22:in `call'
	68: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/request_id.rb:27:in `call'
	67: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/request_store-1.4.1/lib/request_store/middleware.rb:19:in `call'
	66: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
	65: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
	64: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/rack/logger.rb:26:in `call'
	63: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:71:in `tagged'
	62: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:28:in `tagged'
	61: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:71:in `block in tagged'
	60: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/rack/logger.rb:26:in `block in call'
	59: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/rack/logger.rb:38:in `call_app'
	58: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
	57: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `call'
	56: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `catch'
	55: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:30:in `block in call'
	54: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:135:in `call_app'
	53: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
	52: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:71:in `rescue in call'
	51: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/sentry-raven-2.9.0/lib/raven/integrations/rails/overrides/debug_exceptions_catcher.rb:11:in `render_exception'
	50: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/extensions.rb:28:in `render_exception_with_web_console'
	49: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:87:in `render_exception'
	48: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:102:in `render_for_browser_request'
	47: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:46:in `render'
	46: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger.rb:63:in `block (2 levels) in broadcast'
	45: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger_silence.rb:21:in `silence'
	44: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger.rb:65:in `block (3 levels) in broadcast'
	43: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger_silence.rb:21:in `silence'
	42: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:46:in `block in render'
	41: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/helpers/rendering_helper.rb:33:in `render'
	40: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/renderer.rb:25:in `render'
	39: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/renderer.rb:44:in `render_template'
	38: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:16:in `render'
	37: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:52:in `render_template'
	36: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:61:in `render_with_layout'
	35: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:53:in `block in render_template'
	34: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/abstract_renderer.rb:43:in `instrument'
	33: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `instrument'
	32: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
	31: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `block in instrument'
	30: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/abstract_renderer.rb:44:in `block in instrument'
	29: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:54:in `block (2 levels) in render_template'
	28: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/deface-1.4.0/lib/deface/action_view_extensions.rb:41:in `render'
	27: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/template.rb:157:in `render'
	26: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/template.rb:354:in `instrument_render_template'
	25: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications.rb:170:in `instrument'
	24: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/template.rb:159:in `block in render'
	23: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/templates/rescues/diagnostics.html.erb:15:in `_4669a2f402a75154621ed2ec014a59a8'
	22: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:46:in `render'
	21: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger.rb:63:in `block (2 levels) in broadcast'
	20: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger_silence.rb:21:in `silence'
	19: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger.rb:65:in `block (3 levels) in broadcast'
	18: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/logger_silence.rb:21:in `silence'
	17: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/debug_exceptions.rb:46:in `block in render'
	16: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/helpers/rendering_helper.rb:33:in `render'
	15: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/renderer.rb:25:in `render'
	14: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/renderer.rb:44:in `render_template'
	13: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:16:in `render'
	12: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:52:in `render_template'
	11: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:61:in `render_with_layout'
	10: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:53:in `block in render_template'
	 9: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/abstract_renderer.rb:43:in `instrument'
	 8: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `instrument'
	 7: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
	 6: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/notifications.rb:168:in `block in instrument'
	 5: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/abstract_renderer.rb:44:in `block in instrument'
	 4: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/renderer/template_renderer.rb:54:in `block (2 levels) in render_template'
	 3: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/deface-1.4.0/lib/deface/action_view_extensions.rb:41:in `render'
	 2: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/template.rb:156:in `render'
	 1: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/template.rb:162:in `rescue in render'
/home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionview-5.2.3/lib/action_view/template.rb:323:in `handle_render_error': can't modify frozen fatal (ActionView::Template::Error)
	28: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/thread_pool.rb:135:in `block in spawn_thread'
	27: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/server.rb:334:in `block in run'
	26: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/server.rb:474:in `process_client'
	25: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/server.rb:660:in `handle_request'
	24: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/puma-3.12.1/lib/puma/configuration.rb:227:in `call'
	23: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/engine.rb:524:in `call'
	22: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/sentry-raven-2.9.0/lib/raven/integrations/rack.rb:51:in `call'
	21: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/rack-2.0.7/lib/rack/sendfile.rb:111:in `call'
	20: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/static.rb:127:in `call'
	19: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/executor.rb:14:in `call'
	18: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/rack-2.0.7/lib/rack/runtime.rb:22:in `call'
	17: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/rack-2.0.7/lib/rack/method_override.rb:22:in `call'
	16: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/request_id.rb:27:in `call'
	15: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/request_store-1.4.1/lib/request_store/middleware.rb:19:in `call'
	14: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
	13: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
	12: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/rack/logger.rb:26:in `call'
	11: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:71:in `tagged'
	10: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:28:in `tagged'
	 9: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/activesupport-5.2.3/lib/active_support/tagged_logging.rb:71:in `block in tagged'
	 8: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/rack/logger.rb:26:in `block in call'
	 7: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/railties-5.2.3/lib/rails/rack/logger.rb:38:in `call_app'
	 6: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/actionpack-5.2.3/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
	 5: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `call'
	 4: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:20:in `catch'
	 3: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:30:in `block in call'
	 2: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:134:in `call_app'
	 1: from /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:137:in `rescue in call_app'
/home/er1604140/.rvm/gems/ruby-2.6.3@tfc/gems/web-console-3.7.0/lib/web_console/middleware.rb:137:in `throw': uncaught throw :app_exception (UncaughtThrowError)

If I copy override multi_currency_admin_settings.rb and set original as suggested, it give following error
↳ /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/bundler/gems/spree_admin_roles_and_access-b1fbacfc01df/app/models/spree/ability_decorator.rb:16
  Spree::Permission Load (0.3ms)  SELECT "spree_permissions".* FROM "spree_permissions" WHERE "spree_permissions"."id" = $1 ORDER BY "spree_permissions"."priority" ASC  [["id", 1]]
  ↳ /home/er1604140/.rvm/gems/ruby-2.6.3@tfc/bundler/gems/spree_admin_roles_and_access-b1fbacfc01df/app/models/spree/ability_decorator.rb:16
Deface: 2 overrides found for 'spree/admin/general_settings/edit'
Deface: 'add_favorite_products_per_page_configuration' matched 1 times with '#preferences fieldset .form-actions'
Deface: 'multiple-currency' matched 2 times with '.form-group'
Deface: [ERROR] The original source for 'multiple-currency' has changed, this override should be reviewed to ensure it's still valid.
Completed 500 Internal Server Error in 452ms (ActiveRecord: 128.1ms)

@AmitPatel-BoTreeConsulting , I faced the same issue and found that the issue is with wicked_pdf latest version. Here is the mode details : https://github.com/mileszs/wicked_pdf/pull/574. Downgraded to tag 1.1.0 solved it.

Awesome. It worked. Thanks :1st_place_medal:


text:
Improve tasklauncher-dataflow sink to launch tasks using --URI (without creating task definition manually)

From @majorisit on 10/29/2019:
We have successfully implemented tasklauncher-dataflow sink in K8S environment. But, it required to have task to be created first in order for us to either launch or schedule it.
Since we have humongous SCDF tasks (> 5000 tasks), we are looking for a solution to deploy and launch tasks automatically (without creating task definition manually) at runtime using tasklauncher-dataflow sink by sending Task Launch Request.

Important: FYI, we already have a solution in place using TaskLauncher Kubernetes Sink for dynamically create and launch SCDF tasks using --uri parameter. It's working fine as per our requirement, but this Sink module is no longer recommended by Pivotal because it is not maintained with upstream version upgrades anymore.

Since tasklauncher-dataflow sink is the replacement, however if this sink doesn't meet our requirements, we could collaborate to improve it together.

Please treat this case as high priority

Since the sink-application makes RESful calls against SCDF to launch the desired "tasks" for all the upstream events, the task definition is a requirement in this situation. Today, that's by design.
We could consider deriving and creating the task-definition based on the properties associated with each of the received TaskLaunchRequest objects; however, what if the requests are identical? How will we be able to create the task-definitions first dynamically, and then launch the created definition afterward?

Perhaps it is fine to assume if/when the requests have identical properties, then the first request would create + launch the created definition, and all the subsequent requests will directly launch the existing definition.

We can, in fact, expose a new REST endpoint at the SCDF server controller to launch the task which would also create a task definition if it doesn't exist. This REST endpoint can be used in a situation like this.
This way, we don't have to modify the existing REST endpoints as well as the task launcher dataflow to check for the existence of task definition when launching.

Thanks @sabbyanandan, I will share all the technical details very soon.  I am working on it.

@sabbyanandan,  we may ingest around 100+ data sources in the targeted production environment.  We are planning to keep a seperate SCDF task application for each data source. So, we may end up having 100+ scdf apps.
Let's take an RDBMS / Oracle source for an example, and assume we have 100 tables.  Each table will have a separate task definition, it runs in regular interval based on the cron expression.  The cron condition is different for each table.
We have defined the task definition for each table along with cron condition in the spring-cloud config server like this....
{\
		"table": "performancetable0",\
		"commandLineArgs" : "--recordscount=10000 --fetchsize=1000",\
		"environmentProperties": "",\
		"deploymentProperties": "",\
		"cron": "0 0 * * * ?",\
		"active": "Y"\
	},{\
		"table": "performancetable1",\
		"commandLineArgs" : "--recordscount=10000 --fetchsize=1000",\
		"environmentProperties": "",\
		"deploymentProperties": "",\
		"cron": "0 1 * * * ?",\
		"active": "Y"\
	}

We will have seperate JSON file for each data source.
We will be running a single stream for each data source. The stream source will read this JSON file  and converted as TriggerTaskSchedulerBean for each table, which implements org.quartz.Job.  This TriggerTaskSchedulerBean will send TaskLaunchRequest to Sink module when cron condition met.
Currently, this whole design is working as expected in the non-prod environment by using task-launcher-kubernetes-triggertask-source and task-launcher-kubernetes-sink-kafka-10
Here is our stream definition, we pass some common args here, but rest of the args will be picked up from JSON file before sending TaskLaunchRequest
stream create stream-smoketest --definition "task-launcher-kubernetes-triggertask-source --uri=docker://corp/scdf-smoketest:latest --spring.application.name=smoketest-smokedb-performanceschema --spring.cloud.config.uri=http://config-server:8888 --command-line-args='--datasourcename=smoketest --databasename=smokedb --schemaname=performanceschema'  | task-launcher-kubernetes-sink-kafka-10"
stream deploy stream-smoketest
Since tasklauncher-dataflow sink is the replacement for task-launcher-kubernetes-sink-kafka-10,  however if this tasklauncher-dataflow sink  doesn't meet our requirements, we could collaborate to improve it together.
I will add more details after running some samples in my lower environment.

The tasklauncher-dataflow-sink is easier to use than the platform specific task launcher. Currently there are two manual steps:

register your task applications. That associates the application resource, in this case docker://corp/scdf-smoketest:latest with a logical name.
create a task definition that references the registered application.

We can consider automating the 2nd step. In most cases the task definition contains only the name of the app.  Command line args and deployment properties can be included in the launch request.  The first step is still required.  Does this meet your requirement?

Thanks @dturanski. We are okay with step 1.  Automating step 2 will help a lot.

FYI, this is the sample TaskLaunchRequest  received by task-launcher-kubernetes-sink-kafka-10.  This will ensure that it launches a new instance all the time
TaskLaunchRequest{uri='docker://geico/scdf-smoketest:latest', commandlineArguments=[--batchjobname=ssp-performancetable22-20191031192200, --datasourcename=smoketest, --databasename=smokedb, --schemaname=performanceschema, --table=performancetable22, --recordscount=10000, --fetchsize=1000, random=-761055128]}

@majorisit: Just to close the loop on this, we have added the enhancement to derive and automatically also create the task definition based on the app name. Going forward, all this can be automatically achieved when launching the tasks, so hopefully, this simplifies the workflow for your requirements.
Here are the PR and the documentation for your reference.
We are gearing up for SCDF 2.2.2 and 2.3.0 RC1 releases, but in the meantime, you can already try this on the latest SCDF image from Docker Hub.

Thanks @sabbyanandan.  Can I use the master branch from git clone https://github.com/spring-cloud-stream-app-starters/tasklauncher-dataflow.git ?
Also, I understand that I have specify the app name instead of task name in the below sample. correct?
{
"name":"foo",
"deploymentProps": {"key1":"val1","key2":"val2"},
"args":["--debug", "--foo", "bar"]
}

The fix is not in the app but in SCDF itself. I pointed to the PR ^^ that' why. You could try SCDF's 2.3 BUILD-SNAPSHOT or latest tags from Docker Hub. Or, even, build SCDF source from the repo directly.
@dturanski: Could you clarify the launch with a sample POST request?

Yes you will use the app registration name.
And you can see how to enable the feature in Spring Cloud Data Flow here: https://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#_automating_the_creation_of_task_definitions

@majorisit : That is correct.

Could you please guide me where do I exactly set the property spring.cloud.dataflow.task.autocreate-task-definitions=true.  I am getting an error task definition does not exist with  latest version dataflow server

This is a property on the SCDF server. So if you are running in PCF, you can do cf set-env SPRING_CLOUD_DATAFLOW_TASK_AUTOCREATE_TASK_DEFINITIONS true or put it in the manifest, or if testing locally, something like java --jar spring-cloud-dataflow-server-<version>.jar --spring.cloud.dataflow.task.autocreate-task-definitions=true

Ours in Kubernetes. Could it be under container.env? Shoud I use  SPRING_CLOUD_DATAFLOW_TASK_AUTOCREATE_TASK_DEFINITIONS?

The typical K8s install creates a config-map, see https://github.com/spring-cloud/spring-cloud-dataflow/blob/master/src/kubernetes/server/server-config.yaml . I would edit this file and put autocreate-task-definitions under task: and kubectl apply server-config.yaml or edit the config map directly with kubectl edit ... .   @chrisjs might have a better suggestion.

That looks right.  with this property set and if you are running the updated version of SCDF, you should see the output of  logger.info("Creating a Task Definition {} for registered app name {}", taskName, taskName); in the logs.

It's working after adding the property at config-map for scdf server. I will post here any issues if I found during my testing.  Awesome freinds!!

authors: cppwfs dturanski ilayaperumalg majorisit sabbyanandan


text:
UnknownContentTypeException when client authentication to server fails

When the server has Spring Security, like a basic auth username and password, and the client is misconfigured you get this error:
Caused by: org.springframework.web.client.UnknownContentTypeException: Could not extract response: no suitable HttpMessageConverter found for response type [class org.springframework.cloud.config.environment.Environment] and content type [text/html;charset=UTF-8]
This is very misleading to the user. Since the endpoint does return a 403, it should be simple to improve the error handling for the user. Here is an example of someone running into the error on StackOverflow https://stackoverflow.com/questions/59941635/spring-boot-2-2-2-release-could-not-locate-propertysource-could-not-extract

The problem arises when the config server forwards unauthorized requests to its login page as the page is delivered in a response with status code 200 and a payload in HTML, which the client does not know how to deal with.
One solution for users is to simply disable form login in the config server. The client then still fails but with an HttpClientErrorException$Unauthorized and an insightful stacktrace.
More interestingly, whether the config server forwards or not depends on the accept header of the client's request. For the default application/vnd.spring-cloud.config-server.v2+json, it forwards. But when the property spring.cloud.config.media-type is set to e.g. */* or application/json, everything works with the correct credentials and a 401 stacktrace for incorrect credentials. At least for bare server and client fresh from Spring Initializr and Spring Boot 2.5.3. 😄

How to solve this?

This problem usually appears when form login is enabled in the config server, which is often the case as the auto-configured SecurityFilterChain has form login enabled. When the config client sends the wrong credentials, the server will then respond with the HTML for the login page, which the client does not know how to handle.
If you don't need form login in the config server, you can disable it to solve the issue.

authors: eduanb hpoettker sharkdeng

repo: spring-cloud/spring-cloud-config


text:
Git always wins in a composite version

Having the following composite setup for config server (2021.0.0, Boot 2.6.3)
$ curl https://start.spring.io/starter.tgz -d dependencies=cloud-config-server,cloud-starter-vault-config \
-d language=java -d type=maven-project -d baseDir=spring-cloud-config-server | tar -xzvf -

Annotated the class with @EnableConfigServer
.application.yml
spring:
  cloud:
    vault:
      # [Vault] token
      token: vault-plaintext-root-token
    config:
      server:
        composite:
          - type: vault
            kv-version: 2
          - type: git
            uri: https://github.com/marcingrzejszczak/config-server-properties
  profiles:
    # [Vault & git]
    active: composite

# default port
server.port: 8888

Inserting such vault entries
$ vault kv put secret/bar,foo a="bar with foo profile value from Vault"

Having such git repo
.bar-foo.properties
a=default with foo profile value from git

When sending a request to retrieve the properties for app bar with profile foo I get back a from git not from vault
curl localhost:8888/bar-foo.properties -H "X-Config-Token: vault-plaintext-root-token"

When I remove git from the composite setup I get a equal to bar with foo profile value from Vault

@marcingrzejszczak Please try this:
spring:
  cloud:
    config:
      server:
        vault:
          order: 1
          ...
        git:
          order: 2
          ...

and use profile git,vault.

I did try it out and the only difference was that I had vault,git as profile orders. I'll need to double check my setup in general.

Yeah, there was some glitch with a profile order. Try git,vault instead.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


text:
Behavior of webhooks on CF (PWS)

I'm not sure I'll be able to get all of the details right in the issue, so we may need to discuss this in chat.
Reference applications for this issue:

Config Server - https://github.com/mstine/config-server-encrypt
Config Client - https://github.com/mstine/config-client-encrypt
Config Repo - https://github.com/mstine/config-repo

(Note that no encryption is in play here, I'm just reusing those applications)
LOCALLY - EVERYTHING WORKS

I added the spring-cloud-config-monitor dependency to config-server-encrypt, and ran it on port 8888.
I set up Ngrok to point to my locally running Config Server.
I set up a webhook on config-repo to point to the Ngrok link.
I ran 2 instances of config-client-encrypt on ports 8080 and 8081.
I committed changes to the foo property in crypto.yml and pushed to GitHub.
The webhook POST to /monitor was received, and the applications received the update.

PWS - BROKEN FOR MOST UPDATES

Push the same applications to PWS. config-server-encrypt becomes http://ms-cs-enc.cfapps.io and config-client-encrypt becomes http://ms-cc.cfapps.io.
Scale out ms-cc to 5 instances.
I committed changes to the foo property in crypto.yml and pushed to GitHub.
The webhook POST to /monitor was received, but no applications received the update.

When you look at trace, you see the following:
{
timestamp: 1452037045656,
info: {
signal: "spring.cloud.bus.sent",
type: "RefreshRemoteApplicationEvent",
id: "f1f4c0a5-9844-4615-b0a1-624765aa82a9",
origin: "ms-cs-enc:cloud:0",
destination: "crypto:**"
}
},

So the destination for the message is "all applications named crypto regardless of profile." Note that spring.application.name is set to crypto in both application.yml and bootstrap.yml for ms-cc.
Next experiment:

I committed changes to a property in application.yml and pushed to GitHub.
The webhook POST to /monitor was received, and the applications received the update.

Now when you look at trace, you see the following:
{
timestamp: 1452037279185,
info: {
signal: "spring.cloud.bus.sent",
type: "RefreshRemoteApplicationEvent",
id: "30a17615-150c-47ad-92a5-344b6f2fefcd",
origin: "ms-cs-enc:cloud:0",
destination: "*:**"
}
}

So the destination for the message is "all applications regardless of name and profile." Now we can see acks from the client apps:
{
timestamp: 1452037280765,
info: {
signal: "spring.cloud.bus.ack",
type: "RefreshRemoteApplicationEvent",
id: "30a17615-150c-47ad-92a5-344b6f2fefcd",
origin: "ms-cc:cloud:1",
destination: "*:**"
}
},

Note that the origin of the ack is an application named ms-cc, not crypto.
If you look at /env for ms-cc, you'll see that spring.application.name is consistently still set to crypto. However, cloud.application.name is ms-cc, which seems to be sourced from VCAP_APPLICATION.
Another experiment:

Add a file called crypto-cloud.properties to config-repo and push it. The value of foo in this file should override all others, as it specifies both application name (crypto) and profile name (cloud).
Restart ms-cc. Looking at http://ms-cc.cfapps.io/foo endpoint, this is in fact true.
Now, push an update to crypto-cloud.properties.
The webhook POST to /monitor was received, but no applications received the update.

Now when you look at trace, you see the following:
{
timestamp: 1452037732498,
info: {
signal: "spring.cloud.bus.sent",
type: "RefreshRemoteApplicationEvent",
id: "8476219d-ba94-47e5-a400-66615eebb071",
origin: "ms-cs-enc:cloud:0",
destination: "crypto-cloud:**"
}
}

So, instead of the destination being crypto:cloud as I expected, it is crypto-cloud:**. This seems like a bug.

Looks like 2 issues to me, so we can deal with one of them here, and create a new issue for the other (#318).
Summary: destination: "crypto:**" looks good to me, i.e. the message generated by the monitor is correct, which makes it a Bus problem if it doesn't get delivered.
I don't like the looks of origin: "ms-cs-enc:cloud:0" though, and that comes from Spring Boot. I'll raise an issue there because I think that's actually the problem (in ContextIdApplicationContextInitializer).
The workaround is to use a vcap.application.name that is the same as your spring.application.name.

authors: dsyer mstine


text:
Serving plain text file using a default label is not working in Finchley M9

Using a config server at version Finchley.M9 with the following properties
spring:
  application:
    name: testapp
  profiles:
    active: native
  cloud:
    config:
      server:
        default-label: default-label-test

and config directory layout src/main/resources/config/default-label-test/default-label-nginx.conf
When I request the contents of  default-label-nginx.conf with a request
curl http://localhost:9999/testapp/native/default-label-nginx.conf\?useDefaultLabel, the mapped controller is https://github.com/spring-cloud/spring-cloud-config/blob/master/spring-cloud-config-server/src/main/java/org/springframework/cloud/config/server/environment/EnvironmentController.java#L99
but should be
https://github.com/spring-cloud/spring-cloud-config/blob/master/spring-cloud-config-server/src/main/java/org/springframework/cloud/config/server/resource/ResourceController.java#L81
Using Edgware.SR2 the correct controller method is mapped
@snicoll mentioned that request mapping has changed in Boot 2, see https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide#spring-mvc-path-matching-default-behavior-change

hey @ojhughes - can we close it?

Let's document this as a note.

authors: marcingrzejszczak ojhughes spencergibb


text:
HttpClientCustomizer function no access to micrometer tracing context

Describe the bug
When creating a http client logger I have found that the trace & span id are not propagated for certain methods.
Sample
start.spring.io micrometer tracing & spring cloud gateway
@Bean
  public HttpClientCustomizer httpClientResolverCustomizer(HttpClientLogger httpClientLogger) {
    return httpClient ->
        httpClient
            .doAfterRequest((req, conn) -> logRequest(req, httpClientLogger))
            .doAfterResponseSuccess((res, conn) -> logResponse(res, httpClientLogger));
  }

2023-10-03T23:00:31,971 INFO  [reactor-http-nio-6][app,651c8effe9d86a59bad37df2b701910e,728be67b030f67cf] HttpClientLogger: Sending Request [GET] to [http://localhost/path/to/example] with headers [{}]
2023-10-03T23:00:32,853 INFO  [reactor-http-nio-6][app,,] HttpClientLogger: Received Response [200] from Request to [GET http://localhost/path/to/example] with response headers [{}], duration [N/A]ms

I'm not sure how it could. Maybe @jonatan-ivanov might have an idea.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

I think this can happen because in the instrumentation, the Observation.Scope is closed before doAfterResponseSuccess is called. You might be able to verify this either with a debugger or by registering ObservationTextPublisher that will print out the events that are happening during an observation. If closing the scope is happening before calling doAfterResponseSuccess, this behavior is what you get.
Do you know who creates the Observation here? Is it SC Gateway or Reactor Netty? If the latter, maybe this issue should be closed and a new one in Reactor Netty should be opened.
/cc @violetagg

In both doAfterRequest and doAfterResponseSuccess , you can obtain the current reactive context i.e. currentContextView(). If the Observation is stored in the context then it is available.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

authors: M-Whitaker jonatan-ivanov spencergibb spring-cloud-issues violetagg

repo: spring-cloud/spring-cloud-gateway


text:
UnsupportedOperationException if modifyRequestBody uses LinkedMultiValueMap

  .route("user_route",r->r.path("/user")
                        .filters(
                                f->f.setRequestHeader("Content-Type", MediaType.APPLICATION_FORM_URLENCODED_VALUE)
                                .modifyRequestBody(MultiValueMap.class,MultiValueMap.class,((serverWebExchange, user) -> {
                                    MultiValueMap<String, String> formData = new LinkedMultiValueMap<>();
                                    formData.add("username", "John");
                                    formData.add("nickName", "qwerty");
                                    return  Mono.just(formData);
                                }))
                        )
                        .uri("lb://smweb"))

java.lang.UnsupportedOperationException: null
	at java.util.Collections$UnmodifiableMap.put(Collections.java:1457) ~[na:1.8.0_144]
	at org.springframework.http.HttpHeaders.set(HttpHeaders.java:1451) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
	at org.springframework.http.HttpHeaders.setContentType(HttpHeaders.java:854) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
	at org.springframework.http.codec.FormHttpMessageWriter.write(FormHttpMessageWriter.java:132) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$write$1(MultipartHttpMessageWriter.java:188) ~[spring-web-5.0.7.RELEASE.jar:5.0.7.RELEASE]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:118) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1083) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:115) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:104) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1083) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.MonoCollectList$MonoBufferAllSubscriber.onComplete(MonoCollectList.java:117) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:130) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:245) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:130) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.ipc.netty.channel.FluxReceive.terminateReceiver(FluxReceive.java:377) ~[reactor-netty-0.7.8.RELEASE.jar:0.7.8.RELEASE]
	at reactor.ipc.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:202) ~[reactor-netty-0.7.8.RELEASE.jar:0.7.8.RELEASE]
	at reactor.ipc.netty.channel.FluxReceive.request(FluxReceive.java:110) ~[reactor-netty-0.7.8.RELEASE.jar:0.7.8.RELEASE]
	at reactor.core.publisher.FluxMap$MapSubscriber.request(FluxMap.java:149) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxPeek$PeekSubscriber.request(FluxPeek.java:130) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxMap$MapSubscriber.request(FluxMap.java:149) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.MonoCollectList$MonoBufferAllSubscriber.onSubscribe(MonoCollectList.java:90) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxMap$MapSubscriber.onSubscribe(FluxMap.java:86) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onSubscribe(FluxPeek.java:163) ~[reactor-core-3.1.8.RELEASE.jar:3.1.8.RELEASE]

Please learn how to properly format code and logs.

@qcastel willing to submit a PR based on your comments?

From the stacktrace it's evident that your HttpHeaders are wrapping an unmodifiable map implementation, and it most likely has nothing to do with modifyRequestBody(). Could you provide a minimal reproducible example? I made my own that doesn't trigger any exception
package com.example.gatewaydemo.routeLocator;

import org.junit.jupiter.api.Test;
import reactor.core.publisher.Mono;
import reactor.test.StepVerifier;

import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.reactive.function.client.WebClient;
import org.springframework.web.reactive.function.server.RouterFunction;
import org.springframework.web.reactive.function.server.RouterFunctions;
import org.springframework.web.reactive.function.server.ServerResponse;

import static org.assertj.core.api.Assertions.assertThat;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
public class RouteLocatorTest {
	@LocalServerPort
	int port;

	@Test
	public void test() {
		Mono<ResponseEntity<Void>> responseEntityMono = WebClient.builder()
				.baseUrl("http://localhost:" + port)
				.build()
				.post()
				.uri("/api/user")
				.body(Mono.just(body), String.class)
				.retrieve()
				.toBodilessEntity();
		StepVerifier.create(responseEntityMono)
				.expectNextMatches(r -> r.getStatusCode().is2xxSuccessful())
				.verifyComplete();
	}

	@Configuration
	@EnableAutoConfiguration
	static class RouteLocatorTestConfig {
		@Bean
		RouterFunction<ServerResponse> postRouterFunction() {
			return RouterFunctions.route()
					.POST("/user", request -> request.bodyToMono(MultiValueMap.class)
							.flatMap(body -> {
								assertThat(body.get("username")).asList().containsExactly("John");
								assertThat(body.get("password")).asList().containsExactly("qwerty");
								return ServerResponse.ok().build();
							})
							.onErrorResume(AssertionError.class, t -> ServerResponse.badRequest().build()))
					.build();
		}

		@Bean
		RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
			return builder.routes()
					.route(p -> p
							.path("/api/user")
							.filters(f -> f.rewritePath("/api", "")
									.setRequestHeader(HttpHeaders.CONTENT_TYPE,
											MediaType.APPLICATION_FORM_URLENCODED_VALUE)
									.modifyRequestBody(MultiValueMap.class,
											MultiValueMap.class,
											(serverWebExchange, body) -> {
												MultiValueMap<String, String> formData = new LinkedMultiValueMap<>();
												formData.add("username", "John");
												formData.add("password", "qwerty");
												return Mono.just(formData);
											}))
							.uri("http://localhost:8080"))
					.build();
		}
	}
}

authors: NadChel spencergibb wxiangliang


text:
Allows the ACK_MODE to be configurable when using Stream Binders

This allows the user to configure the ACK_MODE when using the Pub/Sub Spring Cloud Stream binder.
Fixes #2079

Oh! One thing -- bring "Copyright 2017-2018" into the roaring twenties.

Lol, omg we're in the roaring twenties.

repo: spring-cloud/spring-cloud-gcp


text:
Missing observability support in RestTemplateTransportClientFactory

Describe the bug
I have just spent some hours debugging why RestTemplate is not propagating traces to Eureka Server. Turns out RestTemplateTransportClientFactory creates a RestTemplate instance by calling new RestTemplate().
To fix this, RestTemplate.Builder should be used to create a RestTemplate in RestTemplateTransportClientFactory and we get observability support for free.
Now consider the WebFlux WebClient equivalent
public WebClientTransportClientFactories webClientTransportClientFactories(
		ObjectProvider<WebClient.Builder> builder) {
	return new WebClientTransportClientFactories(builder::getIfAvailable);
}

Here, as expected, the WebClient.Builder is used, and produced TransportClientFactory has observability support ...
Version: Spring Cloud Starter 2023.0.0

Thanks for reporting the issue, @ZIRAKrezovic. Makes sense.

Reopening as changes reverted due to broken tests.

authors: OlgaMaciaszek ZIRAKrezovic

repo: spring-cloud/spring-cloud-netflix


text:
Zuul, Spring MVC and trailing slash

Hello.
I'm building an application is based on the Microservice architecture. I have a microservice with Spring MVC that is behind Zuul and accessed by http://localhost/ui/ . The microservice has a page with a link that references to http://localhost/order (of course, this page is not found because the correct link is http://localhost/ui/order), but when I'm on http://localhost/ui/ (trailing slash has been added) the link has a correct address (http://localhost/ui/order).
For my microservice I'm using Spring Boot (1.3.2) :

spring cloud Brixton (snapshot)
spring-boot-starter-web,
spring-boot-starter-thymeleaf,
spring-boot-starter-security (oauth, jwt)
eureka, feign

I don't understand, this is bug or just I need to set some config properties?
Thank you for your help.

Not sure it's a problem or configuration in zuul. Your ui app needs to know how to properly build links behind a proxy. Zuul adds a number of headers you could use:
X-Forwarded-Host
X-Forwarded-Port
X-Forwarded-Proto
X-Forwarded-Prefix

This would happen behind any proxy.

Thank you for answer. Yes, Zuul inserts these headers in every request.
If I add server.contextPath in the application.yml then Thymeleaf template resolver adds the context path to every link on my page. Support this feature for  X-FORWARDED-PREFIX? I mean, can Thymeleaf template resolver gets this header from a request and adds to a link? Or I have to do it manually.

I think you just have to add the trailing slash. Or install an endpoint that redirects to the one with the slash.

I have similar setup as @opavlov24. For me Thymeleaf or Spring Web can not handle X-Forwarded-Prefix. All URLs generated by Thymeleaf and all redirects done by Spring all have no Prefix. The other X-Forwarded-* headers are used in org.apache.catalina.valves.RemoteIpValve to replace URLs hostname, port and protocol (if using Tomcat as embedded container). Shouldn't RemoteIpValve be aware of that header too? Trailing slash doesn't help in my case.

That's a different question. X-Forwarded-Prefix is a spring thing. If the links are generated by Thymeleaf then you should ask them to fix it (by using the Sprin URI builder utilities).

Thank you, I will reach out to thymeleaf guys. But redirect inside an spring controller does not work for me either. Host, port and protocol are replaced, but the prefix gets lost.
Example
http://publichost:80/api/application -> http://localhost:9001/ -> (redirect:home) -> http://localhost:9001/home -> http://publichost:80/home
Should I build redirect urls manually (e.g. with ServletUriComponentsBuilder) or will Spring ever handle this prefix header correctly?

Somebody has to use ServletUriComponentsBuilder but that should include pretty much everything in Spring MVC at least.

Closing this due to inactivity. Please re-open if there's more to discuss.

I have the same error.
zuul:
    routes:
      security-view-angular:
        path: /security-view-angular/**
        service-id: security-view-angular
        strip-prefix: false

When I go to: http://localhost:8080/security-view-angular/, everything is ok.
But, when I go to:  http://localhost:8080/security-view-angular (without last slash) zuul redirect me to http://172.16.5.79:9080/security-view-angular/

As I said already, the trailing slash is mandatory. This is the behaviour of the backend container, e.g. Tomcat, not Zuul, so we can't change it.

I have the same problem like @marvindaviddiaz  https://github.com/spring-cloud/spring-cloud-netflix/issues/1475
It's similar to the server_name_in_redirect in nginx properties. Can I solve this problem in zuul such as server_name_in_redirect ?

@matrixforest please open a new issue

@brnhrdt did you ever get thymleaf link rendering to work? I checked the thymeleaf code (https://github.com/thymeleaf/thymeleaf/blob/3.0-master/src/main/java/org/thymeleaf/linkbuilder/StandardLinkBuilder.java) and it doesnt seem to consider the X-FORWARDED-PREFIX header, nor does the thymeleaf-spring integration module seem to have anything.

Hi
I have a similar problem. I have a Spring boot application MVC (Thymeleaf) using Spring Security that runs fine with the default login page and inMemory users. But when I set the application to work behind a Zuul reverse-proxy using Eureka registry, it behaves and redirects in a strange way. For example:
http://zuulapp_ip/myapp       redirects to       http://nonproxiedapp_ip/login
As I know I have to use x-forward-headers but they need to be intercepted before Spring Security filters so that when Spring security redirects to /login, it can do the redirect behind the Zuul context.
I am not using server.context-path because it makes myapp no to be found in the Eureka server (the app is registered but when invoking behind Zuul, it is not found).
Some help would be great. Thanks

@rburgst I'm not sure, as I don't have access to the project and sources anymore. If I remember correctly I ended up using the ServletUriComponentsBuilder to manually create links for redirects.

I found a different solution by adding the following bean:
    @Bean
    FilterRegistrationBean forwardedHeaderFilter() {
        // we need to use the ForwardedHeaderFilter in order to make sure
        // redirects and
        // thymleaf urls work properly
        // https://github.com/spring-projects/spring-framework/wiki/Upgrading-to-Spring-Framework-5.x#forwarded-headers
        // https://github.com/spring-projects/spring-boot/issues/15046
        // https://github.com/qaware/x-forwarded-filter
        FilterRegistrationBean frb = new FilterRegistrationBean();
        frb.setFilter(new ForwardedHeaderFilter());
        // must run as first filter
        frb.setOrder(Ordered.HIGHEST_PRECEDENCE);
        return frb;
    }

authors: ELIASIVANCARRILLO brnhrdt dimitrijber dsyer marvindaviddiaz matrixforest opavlov24 rburgst ryanjbaxter spencergibb


text:
Allow configuration of redirect following

Default value is now DefaultClientConfigImpl.DEFAULT_FOLLOW_REDIRECTS
(false), it was true before (because it's true by default in HttpClient)
Also fixed the setting of connectTimeout and readTimeout who were equal
to zero before.
fixes gh-691

@snussbaumer NP. When I get a bit more time, I'll add some comments. We'll need you to make some changes. Have you signed the contributors agreement?

@spencergibb : I've just signed the contributors agreement. NP for the changes, go ahead !

Closed via 0ddfb42d4df41132774459bcd1eebfa5e9bd5408

authors: snussbaumer spencergibb


text:
provisioning changes

This PR depends on https://github.com/spring-cloud/spring-cloud-stream/pull/787

Separate Rabbit exchange/queue provisioning from the binder using the SPI provided in core
Refactor the common entities needed for Rabbit into a new core module
Refactor binder code to reflect the SPI changes

Fixes #40

PR comments addressed.

@artembilan Are your issues all resolved? I thought I was just approving my review but it seems GH approved yours too.

Merged as d9c404402741f5308f31f93acb1207bfcd41d2fc

authors: garyrussell sobychacko


text:
Upgrade Spring Boot to the latest version

Update the guide to use the most recent Spring Boot version.
Files that require changes are:
initial/build.gradle
initial/pom.xml
complete/build.gradle
complete/pom.xml

I did make a pull request for upgrading this repo.
But I am wondering, why not using dependapod or renovate for this ?

Hi @bodote .  Thanks for the comment and the PR!  I merged the PR.
I agree there are more automated ways to keep our dependencies up to date.  For this project I want to keep the effort manual so we can provide issues for first time contributors.  We've found a large audience that would like to contribute to OSS but aren't sure how to navigate the process.  We're using this issue (and others in the spring-guides org) largely for educational purposes.  Depending on the interest we may look at using an automated workflow in the future.
I will close this issue as the PR is merged.  There will likely be new opportunities to upgrade the Boot version in the future.

authors: bodote robertmcnees

repo: spring-guides/gs-rest-service


text:
"What You Need" section is broken

It's broken with the following errors:
Unresolved directive in <stdin> - include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/main/prereq_editor_jdk_buildtools.adoc[]

Unresolved directive in <stdin> - include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/main/how_to_complete_this_guide.adoc[]

This has been reported in https://github.com/spring-projects/spring-restdocs/issues/727 originally by @iggykarpov.

Looks OK to me now. There was a stale cache entry.

authors: dsyer izeye

repo: spring-guides/gs-testing-web


text:
Determine charset from contentType if any

There is no guarantee that the consumer uses Jackson to decode the content.
We can consider moving the charset to the contentType; please open a new issue for that.

Originally posted by @garyrussell in https://github.com/spring-projects/spring-amqp/issues/1251#issuecomment-700028191
Affected version: spring-amqp 2.4.2
Recently we were affected by this as the charset was provided in contentType header like e.g.
content-type: application/xml; charset=utf-16 and causes deserialization problems.
For the sending part: non spring-amqp consumers don't expect to find charset at contentEncoding header.
Please consider moving the charset to the contentType.

This might also impact the compression post processors, including the DelegatingDecompressingPostProcessor.

authors: bjmi garyrussell

repo: spring-projects/spring-amqp


text:
The SimpleMessageListenerContainer does not shutdown properly

We can see that from logs when we close an application normally:
2024-01-09T10:56:32.795-05:00 DEBUG 24368 --- [    container-1] o.s.a.r.listener.BlockingQueueConsumer   : Received message: (Body:'[B@756a9bb3(byte[5])' MessageProperties [headers={}, contentType=application/octet-stream, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=, receivedRoutingKey=test-supplier-queue, deliveryTag=2, consumerTag=amq.ctag-sbd2qMhyoUNhBz69Y9isVQ, consumerQueue=test-supplier-queue])
2024-01-09T10:56:32.797-05:00 DEBUG 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Shutting down Rabbit listener container
2024-01-09T10:56:32.806-05:00  INFO 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Waiting for workers to finish.
2024-01-09T10:56:32.807-05:00 DEBUG 24368 --- [pool-3-thread-5] o.s.a.r.listener.BlockingQueueConsumer   : Received cancelOk for tag amq.ctag-sbd2qMhyoUNhBz69Y9isVQ (test-supplier-queue); Consumer@35a9e665: tags=[[amq.ctag-sbd2qMhyoUNhBz69Y9isVQ]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest@127.0.0.1:53456/,1), conn: Proxy@2cdb5974 Shared Rabbit Connection: SimpleConnection@6d4502ca [delegate=amqp://guest@127.0.0.1:53456/, localPort=53462], acknowledgeMode=AUTO local queue size=0
2024-01-09T10:56:37.819-05:00  INFO 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Workers not finished.
2024-01-09T10:56:37.821-05:00  WARN 24368 --- [    container-1] o.s.a.r.l.SimpleMessageListenerContainer : Closing channel for unresponsive consumer: Consumer@35a9e665: tags=[[amq.ctag-sbd2qMhyoUNhBz69Y9isVQ]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest@127.0.0.1:53456/,1), conn: Proxy@2cdb5974 Shared Rabbit Connection: SimpleConnection@6d4502ca [delegate=amqp://guest@127.0.0.1:53456/, localPort=53462], acknowledgeMode=AUTO local queue size=0

Pay attention to 5 sec delay between Waiting for workers to finish. and Workers not finished..
This is reported by the code in the SimpleMessageListenerContainer:
				boolean finished = this.cancellationLock.await(getShutdownTimeout(), TimeUnit.MILLISECONDS);
				if (finished) {
					logger.info("Successfully waited for workers to finish.");
				}
				else {
					logger.info("Workers not finished.");

So, the cancellationLock is not released from a consumer which has just reported Received cancelOk for tag.
The BlockingQueueConsumer must be fixed in its handleCancelOk() to add:
BlockingQueueConsumer.this.activeObjectCounter.release(BlockingQueueConsumer.this);

NOTE: the DirectMessageListenerContainer already has such a logic:
		private void closeChannel() {
			RabbitUtils.setPhysicalCloseRequired(getChannel(), true);
			RabbitUtils.closeChannel(getChannel());
			RabbitUtils.closeConnection(this.connection);
			DirectMessageListenerContainer.this.cancellationLock.release(this);
		}

which is called from the mentioned handleCancelOk().

@artembilan
This issue seems to be still unresolved, by using spring-rabbit 3.15, I try to shutdown SimpleMessageListenerContainer, but also get "Workers not finished" warnning
2024-06-17T15:26:32.411Z INFO [] [SpringApplicationShutdownHook] [][][][][] [o.s.a.r.l.SimpleMessageListenerContainer.lambda$shutdownAndWaitOrCallback$2(693)] - Workers not finished.
2024-06-17T15:26:32.413Z WARN [] [SpringApplicationShutdownHook] [][][][][] [o.s.a.r.l.SimpleMessageListenerContainer.lambda$shutdownAndWaitOrCallback$1(697)] - Closing channel for unresponsive consumer: Consumer@6e18d3f9: tags=[[amq.ctag-zUuqxb9iGhy0RJvcL5ghxQ, amq.ctag-hKRvkNR0o2FzFCqw-Q06IQ]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest@10.43.32.93:5672/,1), conn: Proxy@4063af6 Shared Rabbit Connection: null, acknowledgeMode=AUTO local queue size=0
I'm trying to understand the commit https://github.com/spring-projects/spring-amqp/commit/37d96412a1fc92d349f05976eb179cb7d8da0bd6#diff-537549efc15fbf10a5941ac7f1a8648a24ffc56cccc7c46f190ccff45351b44b and but I'm worried that he might not work because if a consumer is cancelled, the mainLoop() will not be executed and no chance to release itself?
try { 	initialize(); 				while (isActive(this.consumer) || this.consumer.hasDelivery() || !this.consumer.cancelled()) { 					mainLoop(); 				} 			}

I think you mean like this:
	finally {
				SimpleMessageListenerContainer.this.cancellationLock.release(this.consumer);

after that while with mainLoop().
Yes, if we don't enter to that loop, we don't release this cancelled consumer.
Please, confirm that we are on the same page, and I'll provide the fix shortly.

@artembilan
Thank you very much for your quick reply, I think your fix should work. Please help to fix it at your convenience.

BTW, @haloxinyu , we have just released version 3.1.6. Give it a shot with your project!

@artembilan
It works good! Amazing release speed, thanks a lot!

authors: artembilan haloxinyu


text:
GH-280: Polishing: Consumer Message Header, Docs

See: https://github.com/spring-projects/spring-kafka/issues/280
Add consumer as a mapped header.

Merged as https://github.com/spring-projects/spring-kafka/commit/a71054273cd3e9d0327fbf05da6bfff47b38e59b

authors: artembilan garyrussell

repo: spring-projects/spring-kafka


text:
[Sprint 41 Review Only - Do not merge yet] XD-2358-2359-2360-2549 Kafka source using the Simple Consumer API

[Do not merge - dependency in staging]

Upgrade to Spring Integration Kafka 1.0.0.M3
Replace the Kafka Source with one based on the KafkaMessageDrivenChannelAdapter
Add support for listening to a subset of the original set of partitions ('partitions' module parameter)
Evenly split the listened partitions between module instance and coordinate via Zookeeper
Add support for starting reading from a series of initial offsets ('initialOffsets' parameter)
Pluggable strategy for storing offsets during consumption (supported: in-memory/transient and Redis)

No tests on matter.
Or do existing tests for Kafka source cover this well?

Re: testing - the core components have been tested as part of Spring Integration Kafka. There currently are is KafkaSinkSourceTest that exercises the source. Added https://jira.spring.io/browse/XD-2624 to cover the more comprehensive testing before RELEASE.

That's all.
@mbogoevici , be sure to comment in the PR in the future after adding commits.
Unfortunately GH doesn't notify us, when there is just only commit added...

Ok. Fixed.

squashed, rebased, merged as a3e40c2979a882a017c1394abcd057a73325c0b2

authors: artembilan markpollack mbogoevici

repo: spring-projects/spring-xd