text:
Enhancement: Data Connector Release Candidate: Spice.ai Cloud Platform

Goal-State/What/Result
The Spice.ai Cloud Platform Data Connector meets or exceeds the release candidate criteria.
Why/Purpose
Part of our stated Roadmap: https://github.com/spiceai/spiceai/blob/trunk/docs/ROADMAP.md
By When
A target date for done-done completion of the entire enhancement.
Issue/Spec written and reviewed:
Done-Done:
Done-Done

[x] First Principles Driven
[x] The Algorithm
[x] PM/Design Review
[x] DX/UX Review
[ ] Threat Model / Security Review
[ ] Tests
[ ] Performance / Benchmarks
[ ] Documentation
[ ] Samples/Quickstarts/Tutorials
[ ] Release Notes

The Algorithm

[x] Every requirement questioned?
[x] Delete (Scope) any part you can.
[x] Simplify.
[x] Break down into smaller iterations/milestones.
[x] Opportunities for automation.

Spec
All Connectors
These requirements are imposed on every connector, regardless of the selected features/tests in the above table.
The RC release criteria expand on and require that all Beta release criteria continue to pass for the connector.

[ ] All Beta release criteria pass for this connector.
[ ] For exceptions where a Core Connector Data Type is not supported, print a warning and ignore the column instead of crashing or failing the query.
[ ] All known Major Bugs are resolved.

Documentation
Documentation criteria should be re-checked on every release, to ensure the documentation is still accurate for the connector.

[ ] Documentation includes all information and steps for a user to set up the connector.
[ ] Documentation includes any mappings from Core Connector Data Types to Core Arrow Data Types types.
[ ] Documentation includes all known issues/limitations for the connector.
[ ] Documentation includes any exceptions made to allow this connector to reach RC quality (e.g. if a particular data type cannot be supported by the connector).
[ ] The connector has an easy to follow quickstart.
[ ] The connector status is updated in the table of connectors in spiceai/docs.

Conditional Criteria
The following features/tests are dependent on the required features/tests for the specified connector, from the Connector criteria table
Streaming

[ ] Data is streamed when reading/performing queries from this connector.

Schema Inference
Support for schema inference in a connector is classified as:

✅: Full coverage. The connector supports native schema inference.
⚠: Partial coverage. The connector does not support native schema inference, but it requires no inference (e.g. static schema, etc).
❌: No coverage. The connector does not support native schema inference, and typically infers schema from the first row of results.

Full Coverage

[ ] The schema for data returned from queries is determined using a native CLI/library method provided by the connector.

Federation
Support for federation in a connector is classified as:

✅: Full coverage. The connector supports full federation and query push down.
⚠: Partial coverage. The connector supports partial federation and query push down.
❌: No coverage. The connector does not support federation or query push down.

Full Coverage

[ ] The connector supports full federation within a single dataset (e.g. select * from my_dataset)
[ ] The connector supports federation push down across multiple datasets within the same connection source (e.g. select * from first_dataset, second_dataset)

Test Coverage
Indexes are not required for test coverage, but can be introduced if required for tests to pass (e.g. due to performance characteristics, etc).
TPC-H

[ ] End-to-end test to cover connecting to TPC-H SF1 for the connector type and benchmarking TPC-H queries (official and simple).

[ ] Connectors should run all queries with no Major Bugs.

[ ] A test script exists that can load TPC-H data at the designated scale factor into this connector.
[ ] The connector can load TPC-H at the designated scale factor, and can run all queries with no Major Bugs.
[ ] All Minor Bugs for TPC-H are raised as issues.
[ ] TPC-H queries that execute successfully on Datafusion, should execute successfully on the connector.

TPC-DS

[ ] End-to-end test to cover connecting to TPC-H SF1 for the connector type and benchmarking TPC-DS queries (official and simple).

[ ] Connectors should run all queries with no Major Bugs.

[ ] A test script exists that can load TPC-DS data at the designated scale factor into this connector.
[ ] The connector can load TPC-DS at the designated scale factor, and can run all queries with no Major Bugs.
[ ] All Minor Bugs for TPC-DS are raised as issues.
[ ] TPC-DS queries that execute successfully on Datafusion, should execute successfully on the connector.

ClickBench

[ ] A test script exists that can load ClickBench data into this connector at the designated scale factor.
[ ] All queries are attempted on this connector. No bug fixes are required for ClickBench.
[ ] All ClickBench bugs are raised as issues.

Data Correctness
Data correctness can only be validated where the connector has a native CLI to replicate identical results. For connectors that do not have a native CLI to generate identical results, skip these tests.
Connectors that are excluded from these tests does not indicate that data is incorrect - it indicates only that we do not perform automated testing of data correctness.

[ ] TPC-H queries at the designated scale factor return identical results in Spice and the native connector CLI.
[ ] TPC-DS queries at the designated scale factor return identical results in Spice and the native connector CLI.

@ewgenius  to follow up if he needs help with this WI

authors: phillipleblanc sgrebnov

repo: spiceai/spiceai


text:
Publish development documents through github actions.

What type of PR is this?

release/none
kind/doc

What this PR does / why we need it:
Supports the latest branch and can generate development documents. The design is as follows:
In the github interface, you can pass in the parameter main to determine whether your document is published as the latest.If the value is main, development documentation will be generated based on the main branch. Its format is dev (main)，If the value is not main, it will be triggered based on tag.
Which issue(s) this PR fixes:

Fixes #2482
Special notes for your reviewer:
Signed-off-by: ty-dc tao.yang@daocloud.io

/hold

The effect looks like this:

不需要 0.8（latest），只需要latest。否则 没有tag或者 version没更新，就变成 0.7 latest

不需要 0.8（latest），只需要latest。否则 没有tag或者 version没更新，就变成 0.7 latest

是这样做的。只要是main 分支，都默认打到 dev 目录下，界面只会显示 dev。不会变为 0.7、0.8 这类的。
下列的命令 -t ，只是给 dev 从新取了个标题。大概告诉了一下，该 dev 是基于main 分支。
mike deploy --rebase -b ${{ env.MERGE_BRANCH }} dev -t "dev (${{ env.REF }})"

实际效果：

source: gharchive/pull-request

authors: ty-dc weizhoublue

repo: spidernet-io/spiderpool

license_type: permissive

license_source: github-api


text:
Cancel open the node on click or make it can be controlled by click listener

update the zoom function in flamegraph.js
function zoom (d) {
        if (tooltip) tooltip.hide()
        const updateChart = () => {
            hideSiblings(d)
            show(d)
            fadeAncestors(d)
            update()
        }
        if (scrollOnZoom) {
            const chartOffset = svg._groups[0][0].parentNode.offsetTop
            const maxFrames = (window.innerHeight - chartOffset) / c
            const frameOffset = (d.height - maxFrames + 10) * c
            window.scrollTo({
                top: chartOffset + frameOffset,
                left: 0,
                behavior: 'smooth'
            })
        }
        if (typeof clickHandler === 'function') {
            if (clickHandler.length >= 2) {
                clickHandler(d, updateChart)
            } else {
                const res = clickHandler(d)
                if (res !== false) {
                    updateChart()
                }
            }
        } else {
            updateChart()
        }
    }

can use like this
var chart = flamegraph()
        .width(960)
        .cellHeight(18)
        .onClick(onClick);

function onClick(d, zoom) {

        if (window.event.ctrlKey) {
            history.pushState({id: d.id}, d.data.name, `#${d.id}`);
            zoom();
        } else {
            console.info(`Click on ${d.data.name}, id: "${d.id}"`, d);
        }

    }

update the zoom function in flamegraph.js

function zoom (d) {

        if (tooltip) tooltip.hide()

        const updateChart = () => {

            hideSiblings(d)

            show(d)

            fadeAncestors(d)

            update()

        }

        if (scrollOnZoom) {

            const chartOffset = svg._groups[0][0].parentNode.offsetTop

            const maxFrames = (window.innerHeight - chartOffset) / c

            const frameOffset = (d.height - maxFrames + 10) * c

            window.scrollTo({

                top: chartOffset + frameOffset,

                left: 0,

                behavior: 'smooth'

            })

        }

        if (typeof clickHandler === 'function') {

            if (clickHandler.length >= 2) {

                clickHandler(d, updateChart)

            } else {

                const res = clickHandler(d)

                if (res !== false) {

                    updateChart()

                }

            }

        } else {

            updateChart()

        }

    }

can use like this

var chart = flamegraph()

        .width(960)

        .cellHeight(18)

        .onClick(onClick);

function onClick(d, zoom) {

        if (window.event.ctrlKey) {

            history.pushState({id: d.id}, d.data.name, `#${d.id}`);

            zoom();

        } else {

            console.info(`Click on ${d.data.name}, id: "${d.id}"`, d);

        }

    }

update the zoom function in flamegraph.js

function zoom (d) {

        if (tooltip) tooltip.hide()

        const updateChart = () => {

            hideSiblings(d)

            show(d)

            fadeAncestors(d)

            update()

        }

        if (scrollOnZoom) {

            const chartOffset = svg._groups[0][0].parentNode.offsetTop

            const maxFrames = (window.innerHeight - chartOffset) / c

            const frameOffset = (d.height - maxFrames + 10) * c

            window.scrollTo({

                top: chartOffset + frameOffset,

                left: 0,

                behavior: 'smooth'

            })

        }

        if (typeof clickHandler === 'function') {

            if (clickHandler.length >= 2) {

                clickHandler(d, updateChart)

            } else {

                const res = clickHandler(d)

                if (res !== false) {

                    updateChart()

                }

            }

        } else {

            updateChart()

        }

    }

can use like this

var chart = flamegraph()

        .width(960)

        .cellHeight(18)

        .onClick(onClick);

function onClick(d, zoom) {

        if (window.event.ctrlKey) {

            history.pushState({id: d.id}, d.data.name, `#${d.id}`);

            zoom();

        } else {

            console.info(`Click on ${d.data.name}, id: "${d.id}"`, d);

        }

    }

source: gharchive/issue

authors: Ilovetaxi sliyoxn

license_source: bigquery


text:
Reporting a vulnerability

Hello!
I hope you are doing well!
We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly.  GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.
Can you enable it, so that we can report it?
Thanks in advance!
PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository

@igibek Please review the existing SECURITY.md.

authors: igibek ljstella


text:
Optimize fluentd buffers and guidance

What would you like to be added:
The Fluentd buffers should be optimized and guidance and simple test results should be shared.
https://github.com/splunk/splunk-connect-for-kubernetes/blob/e680eb1d6fe53c0346ad66ddb22b3d8ee0703226/helm-chart/splunk-connect-for-kubernetes/values.yaml#L635-L643
  buffer:
    "@type": memory
    total_limit_size: 600m
    chunk_limit_size: 10m
    chunk_limit_records: 10000
    flush_interval: 3s
    flush_thread_count: 1
    overflow_action: block
    retry_max_times: 5
    retry_type: periodic

Why is this needed:
chunk_limit_size has tested best at 10-20MB in high volume environments when customers have been doing scale/tress tests, plus we want to keep thruput high and chunk size low! We should try never to be sending a payload that big.
retry_type keeps us away from fluentd's exponential backoff timer, which can cause fun lag issues. We should lean to always firing out data as fast as we can. dont want to be holding buffers. Saw an instance where after failed attempts fluentd would wait 24 hours to send

Good points!
May I ask what's the perf bottleneck of your system? fluentd or splunk? As perf perspective, the best practice may vary in different cases.
If the bottleneck is Splunk, yes I think we can use small chunk_limit_size but exponential backoff retry - this is recommended by Splunk HEC and implemented in many other Addons; However, if the bottleneck is fluentd, bigger chunk_limit_size can reduce the communication cost, and periodic retry is better.
Thanks for raising these questions. I think we can verify and then document them in future release.

Bottleneck is almost always network or Splunk side (indexing pipeline optimizations/config or intermediate forwarding and Load Balance tuning)
These settings are from my work with customers in the wild when they stress test us before use.
Similar results in On-prem OpenShift and Azure k8s, where reducing chunk size and adjustung retries reduced lag (_time vs index_time )
Happy to help set up the test use cases or re-create if the team wants!

Thanks for your info. @rockb1017  will follow up

The current default in values.yml  is using retry_max_times of 5 with 1 second retry_wait (default according fluentd manual). This gives 5 seconds until the chunk is discarded and logs are lost.
https://github.com/splunk/splunk-connect-for-kubernetes/blob/79d17645af594762c9421cc184d59201a7ccb0a6/helm-chart/splunk-connect-for-kubernetes/values.yaml#L373-L382
So if you are in a situation where you have a network issue on the path from fluentd to splunk server you loose logs in almost no time.

You're right, retry should wait like 30 sec probably. We saw the lower retries cause trouble with Splunk Cloud too. retry should be periodic but give at least 30 sec (hec backoff tells client to try again later when saturated, for example.

the current defaults also prevent the slow flush threshold (default: 20s) warning  ever to kick in due to discarding the chunk much earlier.
I think a sane default retry value would be something that tries to prevent loosing logs.
something like:
retry_forever: true
retry_type: exponential_backoff
retry_wait: 1s
retry_max_interval: 60s

retry after 1,2,4,8,16,32 and then every 60 seconds.
Regarding @type: memory : the logs already moved to buffer will be lost once something kills (or restarts) the fluentd pod . there should IMHO at least be a concrete example on how to configure a file backed buffer.

periodic was added cause customers didn't like the math in exponential backoff...it can lead fluentd to wait an entire day before retrying a chunk. I think you nailed it, we just need explicit retry wait rather than default 1s.
I suggest simply starting with a long retry interval. 60s sounds reasonable.

having set the retry_max_interval to something reasonable should workaround the silly 1d retry wait intervals seen with default config and also cope with very short issues better than just a periodic 60s wait time.

@vinzent we had some network/splunk side issue and wanted to tune the fluentd config to avoid loosing logs. Below is the config that I am planning to use. Do you think this is a good config or any recommendations? Plan is to wait for approx. 12 hourse(retry_max_times: 720).
buffer:
    "@type": memory
    total_limit_size: 1000m
    chunk_limit_size: 20m
    chunk_limit_records: 100000
    flush_interval: 5s
    flush_thread_count: 1
    overflow_action: block
    retry_type: exponential_backoff
    retry_timeout: 72h
    retry_forever: false
    retry_max_times: 720
    retry_wait: 1s
    retry_max_interval: 60s

authors: cheneysplunk johnr84 matthewmodestino vinzent

repo: splunk/splunk-connect-for-kubernetes


text:
feat: Replace filter with store-matches with parser

This PR replaces the use of filter + store-matches with a parser to improve readability and reduce copy of variable operations. This PR also cleanups up file naming conventions so all app-parser files and function names will match

:tada: This PR is included in version 2.0.0-next-major.10 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

authors: rfaircloth-splunk srv-rr-github-token

repo: splunk/splunk-connect-for-syslog


text:
Add support for Draco compression

Is exists way to read files with draco compression?

Any updates here? I prefer this library's API over tinygltf. This is the only thing I need that isn't supported yet.

Same here, loving the lib so far :D

I've added initial support for KHR_draco_mesh_compression. I've only tested it against the BrainStem.gltf on two meshes, but I think this should do it.
This marks the last Khronos extension asides from animation_pointer and that weird XMP extension, which would be hard/impossible to support nicely, I think.

authors: SirBob01 edunad spnda tigrazone

repo: spnda/fastgltf

license: MIT


text:
fix: Fix FlagApplier async behaviour

Issues with flaky tests seem to be caused by the async apply function in FlagApplier not really waiting for the entire triggerBatch operation to finish (including network + updating the cache for apply status), before returning.
Note: older context around the existing asynchronous setup: https://github.com/spotify/confidence-openfeature-provider-swift/pull/35 (not sure how much this is relevant for this PR)

Looks good 👍

authors: fabriziodemaria nicklasl


text:
add RegistryAuthSupplier interface

This completes #749 - David and I worked on this together in-person.
The new RegistryAuthSupplier interface replaces the existing RegistryAuth support in DefaultDockerClient so that users can customize the authentication behavior much more closely. On each operation that requires authentication, DefaultDockerClient will invoke the appropriate method on the RegistryAuthSupplier instance, allowing the implementation to supply up-to-date authentication information/tokens.
This change is inspired by our attempts to integrate with Google Container Registry, where ~/.docker/config.json file contains short-lived access tokens that expire shortly after creation. This means that a long-lived DefaultDockerClient instance cannot store static RegistryAuth information from construction-time.
Also included is an implementation of RegistryAuthSupplier that refreshes the access tokens supplied by gcloud docker -a prior to each authFor(..) operation, as well as an implementation of RegistryAuthSupplier that returns static data.
Closes #740.

@davidxia @mavenraven
one thing we decided to punt on for the moment is customizing the GCR RegistryAuthSupplier to be able to do something like gcloud --account <accountname> docker ... to allow the user to choose which account/credentials to use. What is here now uses the "default" credentialed account in gcloud, which seems to static to rely on long-term. We'll come back to this soon, in a new PR.

👍

Codecov Report

Merging #759 into master will increase coverage by 0.17%.
The diff coverage is 68.64%.

@@             Coverage Diff              @@
##             master     #759      +/-   ##
============================================
+ Coverage     65.56%   65.74%   +0.17%
- Complexity      655      668      +13
============================================
  Files           153      158       +5
  Lines          2878     2925      +47
  Branches        335      335
============================================
+ Hits           1887     1923      +36
- Misses          844      851       +7
- Partials        147      151       +4

authors: codecov-io mattnworb mavenraven

repo: spotify/docker-client


text:
Support include of directories

Description
When creating an image I want to include a jar as well as a directory of dependencies.
<includes>
    <include>${project.build.finalName}.jar</include>
    <include>dependencies</include>
</includes>

The second line for dependencies is ignored.  However if I specify an expression it creates a Dockerfile with 1 line per jar file within dependencies.
<includes>
    <include>${project.build.finalName}.jar</include>
    <include>dependencies/*</include>
</includes>

Since there are more than 120 jar files in that folder, the Dockerfile has more than 120 ADD statements.  Eventually at "Step 119" the plugin generates this error in maven: [ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.10:build (default-cli) on project cubeone-server-jar: Exception caught: max depth exceeded -> [Help 1]
What do you expect
It would be great if the plugin supported include of directories to add.  Docker supports it and this plugin can as well.
When using dependencies/* in docker command line, it works as expected, but using it in the pom.xml the plugin expands it out to a list of files instead.  Don't expand out the list.
Software

docker version: 1.12
docker-maven-plugin version: 0.4.10
maven version: 3.x

+1

authors: gonciarz nkavian

repo: spotify/docker-maven-plugin


text:
Add drag&drop the Upload Dialog

check how hard it is to replace
https://ui5.sap.com/#/api/sap.m.upload.UploadSet%23overview

i tried to implement with PR #455 but i dont like UploadSet especially because UploadSet is for a different use case, mainly showing backend files and uploading more.
One of the reasons was to have drag and drop support, but that is currently not worth tweaking UploadSet to make it work for my use case.
It maybe possible to add drag and drop to FileUpload

instead i added drag and drop to the dialog itself


text:
How to import multiple config maps

I am about to use the feature of this project to import Kubernetes configmap into my deployment.
https://github.com/spring-cloud-incubator/spring-cloud-kubernetes#configmap-propertysource
This allows me to specify a single a single configmap to import. I would like to specify a configmap for multiple things for example Postgres Settings, RabbitMq Setting, Logging settings that can be combined into a single yml file that my Spring boot app can import.
Is this possible?

Is this your question https://stackoverflow.com/questions/46179113/kubernetes-spring-cloud-multiple-config-maps-being-used?

That question was one of my colleagues yes. It seems that you can import multiple secrets using this tool, but not configmaps. Is there a reason for this, or just not implemented?

+1 - it would be nice to have something like spring.config.location in place for spring-cloud.
I went to the code a little - this class defines the Interface which loads only a PropertySource. If this could be extended somehow to return a List it would be possible to implement.
https://github.com/spring-cloud/spring-cloud-commons/blob/master/spring-cloud-context/src/main/java/org/springframework/cloud/bootstrap/config/PropertySourceLocator.java
Has to be somewhat backwards-compatible.

I list is not needed since you can create a CompositePropertySource and add as many property sources as needed.

authors: computerlyrik spencergibb tomhughesnice


text:
configclient

I have configserver deployed working, able to encrypt and decrypt passwords using url localhost:8888/encrypt and localhost:8888/decrypt. I encrypted password used in the config server security.user.password.
I added encryted securitykey to config client bootstrap properties. Now bootsttrap for config client as follows.
encrypt:
key: caoConfigserverspring
application:
name: xyz
cloud:
config:
uri: http://localhost:8888
enabled: true
fail-fast: true
username: my_user
password: '{cipher}1234567890hjdbcbvnhfg'
server:
port: 8080
I tried password without single quote and application fails with parsing error. I tried with plain as password and the application works fine. When I use encrypted value, while deploying I see following error. What am I missing when using encrypted password?
2019-05-17 13:03:53 - Application run failed
java.lang.IllegalStateException: Cannot decrypt: key=cloud.config.password
at org.springframework.cloud.bootstrap.encrypt.EnvironmentDecryptApplicationInitializer.doDecrypt(EnvironmentDecryptApplicationInitializer.java:238) ~[spring-cloud-context-2.1.0.RELEASE.jar!/:2.1.0.RELEASE]
at org.springframework.cloud.bootstrap.encrypt.EnvironmentDecryptApplicationInitializer.decrypt(EnvironmentDecryptApplicationInitializer.java:169) ~[spring-cloud-context-2.1.0.RELEASE.jar!/:2.1.0.RELEASE]
at org.springframework.cloud.bootstrap.encrypt.EnvironmentDecryptApplicationInitializer.initialize(EnvironmentDecryptApplicationInitializer.java:96) ~[spring-cloud-context-2.1.0.RELEASE.jar!/:2.1.0.RELEASE]
at org.springframework.cloud.bootstrap.BootstrapApplicationListener$DelegatingEnvironmentDecryptApplicationInitializer.initialize(BootstrapApplicationListener.java:407) ~[spring-cloud-context-2.1.0.RELEASE.jar!/:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:649) [spring-boot-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:373) [spring-boot-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at com.verizon.cao.hcm.shared.security.AuthorizationApplication.main(AuthorizationApplication.java:15) [classes!/:0.1.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_172]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_172]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_172]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_172]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [auth-0.1.0.jar:0.1.0]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [auth-0.1.0.jar:0.1.0]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [auth-0.1.0.jar:0.1.0]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [auth-0.1.0.jar:0.1.0]
Caused by: java.lang.IllegalStateException: Unable to invoke Cipher due to bad padding
at org.springframework.security.crypto.encrypt.CipherUtils.doFinal(CipherUtils.java:142) ~[spring-security-crypto-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.security.crypto.encrypt.AesBytesEncryptor.decrypt(AesBytesEncryptor.java:128) ~[spring-security-crypto-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.security.crypto.encrypt.HexEncodingTextEncryptor.decrypt(HexEncodingTextEncryptor.java:40) ~[spring-security-crypto-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.cloud.bootstrap.encrypt.EnvironmentDecryptApplicationInitializer.doDecrypt(EnvironmentDecryptApplicationInitializer.java:230) ~[spring-cloud-context-2.1.0.RELEASE.jar!/:2.1.0.RELEASE]
... 17 common frames omitted
Caused by: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:991) ~[sunjce_provider.jar:1.8.0_171]
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847) ~[sunjce_provider.jar:1.8.0_171]
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446) ~[sunjce_provider.jar:1.8.0_171]
at javax.crypto.Cipher.doFinal(Cipher.java:2164) ~[na:1.8.0_171]
at org.springframework.security.crypto.encrypt.CipherUtils.doFinal(CipherUtils.java:135) ~[spring-security-crypto-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]

Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

authors: ryanjbaxter spring-issuemaster ubalaji

repo: spring-cloud/spring-cloud-config


text:
Session hijacking with Spring Cloud Gateway, Token Relay filter and OAuth2 authorization server

Describe the bug
I have RESt backends behind Gateway with TokenRelay filter, OAuth2 client spring-boot-starter-oauth2-client and spring-cloud-starter-security. When I login in two different browsers (browser 1 and browser 2) under the same user, they get different session cookies. After I logout in browser 1 and relogin under other user after I refresh the page in browser 2 with - I get the session of other user currently logged in browser 1.
Sample
Currently making a project reproducing this behaviour, probably I'm missing something very simple and obvious, so creating this issue now.

** Sample **
session branch in this repo:
https://github.com/looksworking/gw-oauth/tree/session
To start:
cd deploy && ./do.sh build && ./do.sh start
Next open http://172.10.16.1:8030/res in one browser and use user1:pass1 credentials to authenticate
Next in the other browser the same url and the same credentials
Next in the first browser use exitAuth button to logout and relogin with credetials user2:pass1
Next refresh page in second browser and should see how Resource accessed by line is changed from user1 to user2.

Yep, at least I was right - I was missing something simple and obvious. user_info endpoint returned hardcoded value, no issue with Gateway, closing.

repo: spring-cloud/spring-cloud-gateway


text:
how to deal with: ByteBuf.release() was not called before it's garbage-collected

Describe the bug
Gateway version used 2.1.5.RELEASE。This exception happens occasionally。
I want to find out the cause of this exception！please help me。How to deal with it
Exception Example
LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Created at:
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:363)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:71)
org.springframework.core.io.buffer.NettyDataBufferFactory.allocateBuffer(NettyDataBufferFactory.java:39)
org.springframework.core.codec.CharSequenceEncoder.lambda$encode$1(CharSequenceEncoder.java:85)
reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:144)
reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73)
reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73)
reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121)
reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onNext(FluxContextStart.java:103)
reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onNext(FluxContextStart.java:103)
reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:287)
reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:331)
reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:160)
reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1624)
reactor.core.publisher.FluxCallable.subscribe(FluxCallable.java:49)
reactor.core.publisher.MonoCollect.subscribe(MonoCollect.java:68)
reactor.core.publisher.MonoFilterFuseable.subscribe(MonoFilterFuseable.java:44)
reactor.core.publisher.MonoMapFuseable.subscribe(MonoMapFuseable.java:56)
reactor.core.publisher.MonoSubscriberContext.subscribe(MonoSubscriberContext.java:47)
reactor.core.publisher.MonoSubscriberContext.subscribe(MonoSubscriberContext.java:47)
reactor.core.publisher.MonoMapFuseable.subscribe(MonoMapFuseable.java:59)
reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44)
reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44)
reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
reactor.core.publisher.FluxSourceMonoFuseable.subscribe(FluxSourceMonoFuseable.java:39)
reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
reactor.core.publisher.MonoSingle.subscribe(MonoSingle.java:58)
reactor.core.publisher.MonoSwitchIfEmpty.subscribe(MonoSwitchIfEmpty.java:44)
reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
reactor.core.publisher.Mono.subscribe(Mono.java:3879)
reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:172)
reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56)
reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67)
reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121)
reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onNext(FluxContextStart.java:103)
reactor.core.publisher.FluxContextStart$ContextStartSubscriber.onNext(FluxContextStart.java:103)
reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:287)
reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:331)
reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1623)
reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:160)
reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:136)
reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:252)
reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:136)
reactor.netty.channel.FluxReceive.terminateReceiver(FluxReceive.java:427)
reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:211)
reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:369)
reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:369)
reactor.netty.http.server.HttpServerOperations.onInboundNext(HttpServerOperations.java:496)
reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:92)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
reactor.netty.http.server.HttpTrafficHandler.channelRead(HttpTrafficHandler.java:214)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe$1.run(AbstractEpollChannel.java:387)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)

We can't help you without more information.  It is not even clear if this is an issue with gateway at all.  Please provide more information and reopen the issue.

authors: janecms ryanjbaxter


text:
How do you prevent two people from configuring the same predicates path, but different downstream services lead to one person routing not working?

How do you prevent two people from configuring the same predicates path, but different downstream services lead to one person routing not working?
Such as：
route1:
spring:
  cloud:
    gateway:
      routes:
      - id: route1
        uri: http://123.book.com
        predicates:
        - Path=/book/**
        filters:
        - RewritePath=/book/(?<segment>.*), /$\{segment}

route2:
spring:
  cloud:
    gateway:
      routes:
      - id: route2
        uri: http://456.book.com
        predicates:
        - Path=/book/**
        filters:
        - RewritePath=/book/(?<segment>.*), /$\{segment}

The above configuration comes from two people who don't know each other.It can result in a certain person's configuration invalid.
How do you prevent this, such as giving friendly reminders or reporting errors directly when the same path configuration is saving? Is the spring cloud gateway supported?

Nothing can be done from you improperly configuring the gateway.  We could do something to analyze the predicates of the routes and see if there is overlap, but that could become quite complex.

Nothing can be done from you improperly configuring the gateway. We could do something to analyze the predicates of the routes and see if there is overlap, but that could become quite complex.

But it's dangerous and necessary to add routing dynamically in a production environment, or to provide a mechanism that provides early route matching before a route is added, such as when a person add route2 and before route2 is actually added to the system, we can find that There is an overlap between route1 and it, which can change route2 to prevent this situation at this time.

I am not saying its impossible to do, just quite a bit of (brittle) logic IMO.

I'd saw overlapping routes is a feature allowing earlier defined routes to match before later ones. I don't think there is anything we can do to say that one overlapping route is an error.

authors: ryan-shi ryanjbaxter spencergibb


text:
Finchley RC2 Config Server with missing mandatory parameters throws NPE instead of assertion failure

Just noticed a minor usability change in behaviour between Finchley RC1 and RC2 in the case that a Config Server application is started without a mandatory parameter. In Finchley RC1 (and in previous versions to the best of my knowledge) the application will fail to start with an IllegalStateException informing the user that a property is missing.
e.g.
Caused by: java.lang.IllegalStateException: You need to configure a uri for the git repository

With Finchley RC2 the application instead fails to start because of a NullPointerException...
. . .
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository]: Factory method 'defaultEnvironmentRepository' threw exception; nested exception is java.lang.NullPointerException
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:579) ~[spring-beans-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        ... 111 common frames omitted
Caused by: java.lang.NullPointerException: null
        at org.springframework.cloud.config.server.environment.HttpClientConfigurableHttpConnectionFactory.addHttpClient(HttpClientConfigurableHttpConnectionFactory.java:60) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at org.springframework.cloud.config.server.environment.HttpClientConfigurableHttpConnectionFactory.addConfiguration(HttpClientConfigurableHttpConnectionFactory.java:43) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepositoryFactory.build(MultipleJGitEnvironmentRepositoryFactory.java:59) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at org.springframework.cloud.config.server.config.DefaultRepositoryConfiguration.defaultEnvironmentRepository(EnvironmentRepositoryConfiguration.java:209) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at org.springframework.cloud.config.server.config.DefaultRepositoryConfiguration$$EnhancerBySpringCGLIB$$527ad0e1.CGLIB$defaultEnvironmentRepository$0(<generated>) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at org.springframework.cloud.config.server.config.DefaultRepositoryConfiguration$$EnhancerBySpringCGLIB$$527ad0e1$$FastClassBySpringCGLIB$$a0f12160.invoke(<generated>) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:361) ~[spring-context-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        at org.springframework.cloud.config.server.config.DefaultRepositoryConfiguration$$EnhancerBySpringCGLIB$$527ad0e1.defaultEnvironmentRepository(<generated>) ~[spring-cloud-config-server-2.0.0.RC2.jar:2.0.0.RC2]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        ... 112 common frames omitted

Seems to me that the former behaviour is more helpful to the user.
To see the new behaviour please follow these steps:

Clone https://github.com/georgeharley/no-config.git
Run ./gradlew clean build
Edit file src/main/resources/application.properties and comment out all of the external configuration properties (they were only needed in order to allow the test to pass)
Run ./gradlew bootRun. The application will fail to start up because of the NPE described above

To see the former behaviour exhibited by Finchley RC1 (and previous releases):

Restore the properties in src/main/resources/application.properties
Edit build.gradle and change the value of the springCloudVersion property on line 29 from "Finchley.RC2" to "Finchley.RC1"
Run ./gradlew clean build
Comment out the properties in src/main/resources/application.properties once again
Run ./gradlew bootRun. The application fails to start up but this time rather than an NPE occurring the cause is a failed assertion because of a missing mandatory property:

. . .
Caused by: java.lang.IllegalStateException: You need to configure a uri for the git repository
        at org.springframework.util.Assert.state(Assert.java:73) ~[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.afterPropertiesSet(JGitEnvironmentRepository.java:224) ~[spring-cloud-config-server-2.0.0.RC1.jar:2.0.0.RC1]
        at org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository.afterPropertiesSet(MultipleJGitEnvironmentRepository.java:72) ~[spring-cloud-config-server-2.0.0.RC1.jar:2.0.0.RC1]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1765) ~[spring-beans-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1702) ~[spring-beans-5.0.6.RELEASE.jar:5.0.6.RELEASE]
        ... 109 common frames omitted

@georgeharley I think you meant to put this in Spring Cloud Config, not Netflix ;)

authors: georgeharley ryanjbaxter

repo: spring-cloud/spring-cloud-netflix


text:
FeignClient url placeholder

Using spring-cloud-openfeign version 2.1.5-RELEASE
set the placeholder in url attribute.
@FeignClient(name = "${feign.name}", url = "${feign.url}")
public interface StoreClient {
    //..
}

I specify the feign.url in the configServer.and It always adds "http://"prefix to the left of the string,the final url is like:http://https://example.com. But if I specify the url in a properties file, it works just fine.
I checked the source code in FeignClientsRegistrar class and I found that
    static String getUrl(String url) {
        if (StringUtils.hasText(url) && (!url.startsWith("#{") || !url.contains("}"))) {
            if (!url.contains("://")) {
                url = "http://" + url;
            }

            try {
                new URL(url);
            } catch (MalformedURLException var2) {
                throw new IllegalArgumentException(url + " is malformed", var2);
            }
        }

        return url;
    }

How can I fix it?

https://github.com/spring-cloud/spring-cloud-netflix/issues/2684

repo: spring-cloud/spring-cloud-openfeign


text:
Expose isSampled() in TraceContext

Is your feature request related to a problem? Please describe.
I'm trying to log a JSON field "sampled": true when the current trace is sampled.
Describe the solution you'd like
It would be great if org.springframework.cloud.sleuth.TraceContext could expose the isSampled() Method found in BraveTraceContext.
Describe alternatives you've considered
Im 3.0.0-M5 i casted the TraceContext to BraveTraceContext but this is no longer possible, as BraveTraceContext is now package protected.

Makes perfect sense!

Just a quick note: in your commit d547386cb51cabdf114d6179fa7844402018b455 you used a boolean, the isSampled() from Brave returns a Boolean (nullable) - this could lead to a NPE.

Oh, great catch! Let me fix this

There you go https://github.com/spring-cloud/spring-cloud-sleuth/commit/7fee2670c73cc7f5f196917573cd8b584cd4e675

I've renamed the method to sampled from isSampled to be consistent with lack of get / set / is

Perfect, many thanks!

authors: marcingrzejszczak phxql

repo: spring-cloud/spring-cloud-sleuth


text:
Wrong property name used in spring-cloud-sleuth-sample-zipkin

The sample project spring-cloud-sleuth-sample-zipkin contains an application.yml with the property "sample.zipkin.enabled".
Shouldn't that property be "spring.zipkin.enabled" instead?
The file location is:
https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-samples/spring-cloud-sleuth-sample-zipkin/src/main/resources/application.yml

Nope - it's related to this property - https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-samples/spring-cloud-sleuth-sample-zipkin/src/main/java/sample/SampleZipkinApplication.java#L51 . To have a custom reporter in the sample.

Oooops! Thank you.

authors: jorgenota marcingrzejszczak


text:
Asterisk characters are not properly escaped in README.adoc

in the text. But was not able to make a pull request.
2 asteriks are not properly escaped in the  text.

Hi @kiuxfr. I see what you mean. In the README.adoc:
The two form fields, expressed with `th:field="*{id}"` and
`th:field="*{content}"`, correspond to the fields in the `Greeting` object.

Which renders to:

The two form fields, expressed with th:field="{id}" and th:field="{content}", correspond to the fields in the Greeting object.

Because of not escaping the *, the word and is bolded in our documentation. Another tip something isn't working.
I'm no expert in the matter, but played around and I think that escaping in this way should do the trick:
The two form fields, expressed with `th:field="+*+{id}"` and
`th:field="+*+{content}"`, correspond to the fields in the `Greeting` object.

Perhaps @Buzzardo has a better way. It isn't as simple as escaping with a /.
Seems like a good change for the community. I'll mark it as good for first time contributor.
It's been a while since you opened this issue, but I do hope you try again to submit a PR. In addition to the link @Buzzardo provided, I usually link people to this video by two Spring Engineers for an overview of the OSS process.
If anyone runs into problems submitting a PR let's continue the conversation.

Hi @robertmcnees, I will like toopen a PR for this issue. I see where the issue is in the documentation.

I created a Pr at #31
However I noticed that there is a single asterik befor "{id}"and "{content}"as seen below
th:field="*{id}" and th:field="*{content}"
Is this correct?
Please take a look at this @Buzzardo @robertmcnees

Looks good! I can see that the live documentation now includes this fix. Closing this issue. Thanks for the PR.

authors: Aysha-py kiuxfr robertmcnees


text:
Unknown token_type_hint should be ignored

Describe the bug
IMO we're using the unsupported_token_type response wrongly. Currently, we're retrieving this error response if we pass a token_type_hint request parameter value different to access_token or refresh_token. As I understand the specs, the Authorization Server should just ignore the token_type_hint param if it doesn't match the token type:

If the server is unable to locate the token using the given hint, it MUST extend its search across all of its supported token types.  An authorization server MAY ignore this parameter, particularly if it is able to detect the token type automatically.

An invalid token type hint value is ignored by the authorization server and does not influence the revocation response.

Furthermore, the unsupported_token_type error response should be used only if the authorization server explicitly doesn't allow revoking a particular token type:

unsupported_token_type:  The authorization server does not support the revocation of the presented token type.  That is, the client tried to revoke an access token on a server not supporting this feature.

Also note that this is not really related to the token_type_hint parameter, for example, we could pass a refresh_token as token_type_hint request parameter, but find that the token actually corresponds to an access_token. At that point, if the server is configured not to revoke access_tokens, then it would retrieve the corresponding error response.
To Reproduce
Send a request as follows:
curl --location --request POST 'http://localhost:9000/oauth2/revoke' \
--header 'Authorization: Basic bWVzc2FnaW5nLWNsaWVudDpzZWNyZXQ=' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'token=<token>' \
--data-urlencode 'token_type_hint=whatever'

Server responds with 400 Bad Request:

{
    "error": "unsupported_token_type"
}

Expected behavior
Token should be revoked anyway, and the service should respond with 200 OK.

Thanks for the report @rozagerardo ! This is now fixed via 7f8aff7

authors: jgrandja rozagerardo


text:
GH-1237: RCFB Close key/trust store input streams

Resolves https://github.com/spring-projects/spring-amqp/issues/1237
cherry-pick to 2.2.x, 2.1.x, 1.7.x

... and cherry-picked to 2.2.x & 2.1.x.
Back-ported to 1.7.x as https://github.com/spring-projects/spring-amqp/commit/efcba3a83af50b458e3561ce8af3e0b0a87c6b59 after fixing conflicts.

authors: artembilan garyrussell

repo: spring-projects/spring-amqp


text:
Java 21 virtual thread support with spring-graphql

I was wondering if we have support to use virtual threads, for batch-loaders as opposed to current use?

See https://github.com/graphql-java/graphql-java/issues/3331

@vyj7 are you aware that SchemaMapping and BatchMapping methods support returning Callable that's invoked through the configured Executor, e.g. VirtualThreadTaskExecutor?
The discussion that Brian referenced would be needed for deeper support, but as it is today, you can use virtual threads.

We recently migrated from java-kickstart. I have explored spring-graphql very minimal so that migration can be smooth. But this looks promising to me.
Thanks @bclozel @rstoyanchev . I will try doing it.

I'll update the documentation a little to make this easier to find.

authors: bclozel rstoyanchev vyj7

repo: spring-projects/spring-graphql


text:
A spring boot application server should start up regardless of whether Apache Kafka server is available tor not

I have a spring boot application then I need to run regardless of whether Apache Kafka server is available or not. At present what happens If Apache Kafka server is down I am not able to start my spring boot application and I am not able to find out a way by which I can make sure my spring boot application start up without the availability of Apache Kafka server.
Basically I need a way by which I can override KafkaMessageDrivenChannelAdapter.doStart()
I am using spring-kafka-integration: spring-integration-java-dsl

Set autoStartup to false and start()/stop() it yourself.

Thanks a ton !!
I am trying to achieve this but it's giving me an error:
@Bean(name = "testKafkaMessageDrivenChannelAdapterListenerContainerSpec")
@DependsOn({ "connectionFactory", "mediaConsumerOffsetManager" })
public KafkaMessageDrivenChannelAdapterSpec.KafkaMessageDrivenChannelAdapterListenerContainerSpec getKafkaMessageDrivenChannelAdapterListenerContainerSpec() {
KafkaMessageDrivenChannelAdapterListenerContainerSpec kafkaMessageDrivenChannelAdapterListenerContainerSpec = Kafka.messageDriverChannelAdapter(connectionFactory(), "topic1");
	return kafkaMessageDrivenChannelAdapterListenerContainerSpec;
}

this configuration is giving me
Exception in thread "main" java.lang.IllegalArgumentException: Cannot instantiate 'IntegrationConfigurationInitializer': org.springframework.integration.dsl.config.DslIntegrationConfigurationInitializer
at org.springframework.integration.config.IntegrationConfigurationBeanFactoryPostProcessor.postProcessBeanFactory(IntegrationConfigurationBeanFactoryPostProcessor.java:51)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:265)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:177)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:606)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:462)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:118)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:686)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:320)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:957)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:946)
at com.bcgdv.kaishi.media.Application.main(Application.java:18)
Caused by: org.springframework.beans.factory.BeanCreationException: 'IntegrationComponentSpec' beans: '[testKafkaMessageDrivenChannelAdapterListenerContainerSpec]' must be populated to target objects via 'get()' method call. It is important for @Autowired injections.
at org.springframework.integration.dsl.config.DslIntegrationConfigurationInitializer.checkSpecBeans(DslIntegrationConfigurationInitializer.java:65)
at org.springframework.integration.dsl.config.DslIntegrationConfigurationInitializer.initialize(DslIntegrationConfigurationInitializer.java:52)
at org.springframework.integration.config.IntegrationConfigurationBeanFactoryPostProcessor.postProcessBeanFactory(IntegrationConfigurationBeanFactoryPostProcessor.java:48)
... 10 more
Is this not the right way to do it.

You need to show your DSL snippet where you are wiring the adapter into your flow.

This is the configuration that I am using
@Bean(name = "connectionFactory")
public ConnectionFactory connectionFactory() {
ZookeeperConnect zookeeperConnect = new ZookeeperConnect();
zookeeperConnect.setZkConnect(zookeeperAddress);
	ZookeeperConfiguration zookeeperConfiguration = new ZookeeperConfiguration(zookeeperConnect);
	return new DefaultConnectionFactory(zookeeperConfiguration);
}

@Bean(name = "testConsumerOffsetManager")
@DependsOn("connectionFactory")
public OffsetManager offsetManager(ConnectionFactory connectionFactory) {
	MetadataStoreOffsetManager offsetManager = new MetadataStoreOffsetManager(connectionFactory);
	offsetManager.setConsumerId("testConsumerGroupId");
	offsetManager.setReferenceTimestamp(OffsetRequest.LatestTime());
	return offsetManager;
}

/*
 *
 */
@Bean(name = "testKafkaMessageDrivenChannelAdapter")
@DependsOn({ "connectionFactory", "testConsumerOffsetManager" })
public KafkaMessageDrivenChannelAdapterSpec.KafkaMessageDrivenChannelAdapterListenerContainerSpec  getKafkaMessageDrivenChannelAdapter() {
	return Kafka.messageDriverChannelAdapter(connectionFactory(), mediaConsumerTopic).autoCommitOffset(false).payloadDecoder(String::new).keyDecoder(b -> Integer.valueOf(new String(b))).configureListenerContainer(c -> c.offsetManager(offsetManager(connectionFactory())).maxFetch(100));
}

/*
 * Configuration for fetching the messages from kafka server.
*/
@Bean(name = "testConsumerIntegrationFlow")
public IntegrationFlow mediaConsumerIntegrationFlow() {
	return IntegrationFlows.from(getKafkaMessageDrivenChannelAdapter()).channel(c -> c.queue(mediaConsumerListeningFromKafkaResults)).handle(getConsumerService(), consumeMessageMethodName).get();
}

@Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {
	PollerMetadata pollerMetadata = new PollerMetadata();
	pollerMetadata.setTrigger(new PeriodicTrigger(10));
	return pollerMetadata;
}

The configuration works if testKafkaMessageDrivenChannelAdapter is not created as a bean and it is just a part of 'testConsumerIntegrationFlow' bean.
However since I want to start the kafka server myself I tried to split that as a separate bean.

Your getKafkaMessageDrivenChannelAdapter() is really bad.
If you'd like to get access to the bean by its name using IntegrationFlow definition, you should do something like this:
@Bean(name = "testConsumerIntegrationFlow")
    public IntegrationFlow mediaConsumerIntegrationFlow() {
return IntegrationFlows
					.from(Kafka.messageDriverChannelAdapter(connectionFactory(), mediaConsumerTopic)
							.id("testKafkaMessageDrivenChannelAdapter")
							.autoCommitOffset(false)
							.payloadDecoder(String::new)
							.keyDecoder(b -> Integer.valueOf(new String(b)))
							.configureListenerContainer(c -> c.offsetManager(offsetManager(connectionFactory()))
									.maxFetch(100)))

Pay attention to the .id() definition and try to follow with the StackTrace recommendation! :smile:

Hi,
I guess this is a bit complex for me to understand. I need a bean  for 'KafkaMessageDrivenChannelAdapterListenerContainerSpec' and with the above configuration how will I get the bean. I am sorry for this but I am not able to understand.

No problem! Definitely it is worth to explain one more time.
The IntegrationFlow definition (bean) is just a configuration level container which says the Spring Integration infrastructure to pick up components from it and treat them as beans.
So, in your case you should just add the .id() method for the Kafka.messageDriverChannelAdapter() definition, which will become as a KafkaMessageDrivenChannelAdapter bean with that id. Actually it is always as a bean, but by default with generated id.
The next your step is to @Autowired that KafkaMessageDrivenChannelAdapter bean to your custom service and start() it on demand!
No reason to populate one more bean, or try to introduce some more complex logic: just follow with the existing Spring Integration Java DSL procedure: https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference.
Please, read it and let us know how it is clear for you.
Any other questions are welcome!
Thank you for the attention to the stuff!

Thanks a ton !!. That was help full :)

Deleted the comment by mistake.
Thanks Artem for providing further information but this is still not working for me.

Ok, no problem! Please, share the stacks race then or anything else what's going on

Finally I got through it.
To summarize my initial problem was that when I start my code server and Kafka server is down then my server will not come up.
Solution it to was to set autoStartUp as false 'kafkaMessageDrivenChannelAdapterListenerContainerSpec.autoStartup(false)' and start/stop the server on your own.
Next problem that I faced was how to get bean for 'KafkaMessageDrivenChannelAdapter' so that I can start/stop the server on my own.
This requires me to define the adapter as a bean id and then auto wired it.
Kafka.messageDriverChannelAdapter(connectionFactory(),consumerTopic).id("kafkaMessageDrivenChannelAdapter");
@Autowired
protected KafkaMessageDrivenChannelAdapter kafkaMessageDrivenChannelAdapter;
Make sure you autowired it properly, in my case I was having circular dependency issue. So I have to move this dependency to another bean.
Once again thanks Artem and Gary.

authors: artembilan garyrussell suchitgupta


text:
Rebalance triggered under load

Today I discovered an issue with the KafkaMessageListenerContainer (running 1.0.2) and manual acknowledgement that triggers rebalances and prevents offsets from being committed.
What I believe is happening is the following sequence:

A batch is received on thread A and sends the batch to thread B and goes back to polling
Thread B invokes the listener for each record
When the record is processed the record is acknowledged which adds the offset to the ack queue
The batch is now fully processed and because it is manual ack, the consumer is woken up
Thread A throws the WakeupException and goes to commit the offset
Meanwhile thread B finished another batch and now calls wakeup again. This interrupts the commitSync() and the offset is not committed.
Thread A goes back to polling, but now another batch is finished so the consumer is woken up again
and so on

Because the consumer is woken up all the time, it does not get to send its heartbeat nor commit its offset. This sequence of events can continue until the session timeouts and a rebalance happens. This is a fairly complicated sequence in order for an error to occur, but after have been running this in staging for some time, we see it happen fairly often with our workload. The image below is the "Time since last heartbeat" for the consumer which should be below 30s (the red area) and preferable below 10s (the yellow area).

When I changed the consumer to remove the wakeup on line 623 (https://github.com/spring-projects/spring-kafka/blob/9ec3a780155e73a47aaf93807a51e4718523e777/spring-kafka/src/main/java/org/springframework/kafka/listener/KafkaMessageListenerContainer.java#L623) the heartbeats succeeds, the commits are performed correctly and the overall performance is improved. Please have a look and get back to me - for now, I will be running it with my fix as we had 4-6 rebalances per hour per consumer before.

Thanks for reporting this - I think we need to do a best-effort commit - i.e. perhaps a boolean indicating the consumer is in poll() and only wakeup if it's true. Not entirely reliable (we might miss a wake up in the small race condition between polls, but I think it's better than waiting for the poll to end to do the commits.

I think that it should be an option whether the wakeup should be used or not. Using it will have a big impact of performance and is frankly not needed unless you are using an immediate manual commit.

But isn't it the case that, if you are using BATCH acks (commit after batch is processed), and you have a long poll timeout, we could end up losing the commit if the JVM fails while waiting for the poll to end?

Yes that is true with the current implementation. But that should be up to the user to tune the consumer correctly in my opinion.
To add to my last comment, if you interrupt the poll() again and again, you would not be sending heartbeats which could trigger a rebalance. So using the wakeup() is tricky to get right I think

Makes sense.

Pushed the fix I have been running for some days. Please apply it on the 1.0.x branch as well

See https://github.com/spring-projects/spring-kafka/pull/200

I wonder if this SO is related: http://stackoverflow.com/questions/40406638/spring-kafka-batch-not-updating-offsets-with-larger-batches

@artembilan It looks a lot like the issue I was seeing

authors: artembilan garyrussell martindam

repo: spring-projects/spring-kafka


text:
Update Reactor to 6.2.0-M7

Closed via https://github.com/spring-projects/spring-pulsar/commit/14cf303f54d649d1f0aacbb7f0c9c81e419bcb11

repo: spring-projects/spring-pulsar


text:
Don't include a license object when both license and licenseUrl are e…

Implements the proposed fix in issue #1264.

@wgreven-ibr thank you for the PR! :metal: for the included tests!

authors: dilipkrish wgreven-ibr

repo: springfox/springfox


text:
Cannot use 0 as a UUID

When you pass 0 as the UUID to AccoridanItem it gets ignored.
An example of this causing an issue:
<Accordion
  onChange={(selectedItems) => setSelected(selectedItems)}
  allowMultipleExpanded
  allowZeroExpanded
>
  {faqs.map((faq, index) => (
    <AccordionItem uuid={index} key={faq.question}>
      <AccordionItemHeading aria-level="3">
        <AccordionItemButton className={styles.faqQuestion}>
          <Avatar className={styles.avatar}>
            {selected.includes(index) ? (
              <KeyboardArrowDownIcon className={styles.icon} />
            ) : (
              <ChevronRightIcon className={styles.icon} />
            )}
          </Avatar>
          {faq.question}
        </AccordionItemButton>
      </AccordionItemHeading>
      <AccordionItemPanel>
        <div
          className={styles.faqAnswer}
          dangerouslySetInnerHTML={{ __html: faq.answer }}
        ></div>
      </AccordionItemPanel>
    </AccordionItem>
  ))}
</Accordion>

In the above example for the first element faqs will pass 0 to AccordianItem but then it gets ignored and the UUID is set to the default RAA one. This then causes the chevron later on not to get flipped.
This looks like the problematic line of code. Due to 0 being falsy it uses instanceUuid. Maybe it should not do a falsy check but a more explicit undefined and null check?

My temporary fix is to use index + 1 rather than index but the ideal fix for me would be an explicit undefined and null check

I agree. That deffo sounds like a bug
On Thu, 29 Jul 2021, 20:56 Gavin Henderson, @.***>
wrote:

My temporary fix is to use index + 1 rather than index but the ideal fix
for me would be an explicit undefined and null check
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/springload/react-accessible-accordion/issues/324#issuecomment-888935422,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEXQJCNH6UDPFDZEDAV2STT2EJSPANCNFSM5BGAIZ3A
.

authors: gavinhenderson holloway

repo: springload/react-accessible-accordion

license: mit


text:
TE-8567: Updated resolve-url-loader dependency

Ticket - https://spryker.atlassian.net/browse/TE-8567
PR Description
Updated resolve-url-loader dependency 2.3.2 -> 3.1.2.
Checklist

[ ] I agree with the Code Contribution License Agreement in CONTRIBUTING.md

Not actual.

authors: andrew-maslov tolerants

repo: spryker/oryx-for-zed


text:
Fix downloader

This patch changes two things in install.pp

Switches the download tool from wget to curl to allow the use of
the 'file://' scheme for the download URL (wget doesn't support file://)
Checks $::gitlab::puppet_manage_packages before declaring
Package['curl']. This follows the behaviour of the other dependencies
and Puppet best practices.

The latest commits moved the package declaration to packages.pp as requested

this has been sitting around for long enough that gitlab has made it obsolete by now offering apt/yum repositories.

authors: fvoges igalic

repo: spuder/puppet-gitlab


text:
search job definition by name only in batch submit job method

In the real aws, you can call the boto3 batch.submit_job method with a job definition name, without including the entire arn, nor the revision. If you include only the name, the latest revision is assumed. IN moto, in the BatchBackend class, the job definition name is searched by arn, or by name and revision. It should also be searched just by name, as the real aws does.

Thanks; setting this as a feature request.

authors: dedominicisfa spulec

repo: spulec/moto


text:
mock_s3 on inherited classes doesn't work

What I've observed is that if I added the decorator mock_s3 to a parent class and not add the decorator to the child class, the child class s3 instances won't be mocked. Is this expected? Couldn't find anything related to this. So, posting it here.
Below is the code snippet I used.
import boto3
from moto import mock_s3

@mock_s3
class Parent:
    def create(self):
        print("Parent Create started")
        conn = boto3.resource("s3", region_name="us-east-1")
        print(list(conn.buckets.all()))
        conn.create_bucket(Bucket="asd")
        print(list(conn.buckets.all()))
        print("Parent Create ended")

class Child(Parent):
    def create(self):
        super().create()
        print("Child Create started")
        conn = boto3.resource("s3", region_name="us-east-1")
        print(list(conn.buckets.all()))
        conn.create_bucket(Bucket="pqr")
        print(list(conn.buckets.all()))

Child().create()

This is the following console log I get in the response
Parent Create started
[]
[s3.Bucket(name='asd')]
Parent Create ended
Child Create started
[s3.Bucket(name='api-v2-zappa-dev')]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/saichander/sixsense/repos/sixsense-api-v2/moto_test.py", line 20, in create
    conn.create_bucket(Bucket="pqr")
  File "/Users/saichander/.pyenv/versions/3.8.9/envs/sixsense-api-v2/lib/python3.8/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/Users/saichander/.pyenv/versions/3.8.9/envs/sixsense-api-v2/lib/python3.8/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/Users/saichander/.pyenv/versions/3.8.9/envs/sixsense-api-v2/lib/python3.8/site-packages/botocore/client.py", line 337, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/saichander/.pyenv/versions/3.8.9/envs/sixsense-api-v2/lib/python3.8/site-packages/botocore/client.py", line 656, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (IllegalLocationConstraintException) when calling the CreateBucket operation: The unspecified location constraint is incompatible for the region specific endpoint this request was sent to.

As you can see above, after the line Child Create started, it is actually using my original access key and secret key and fetching the actual buckets.
I'm wondering if this is the expected behaviour and it's mandatory to add mock_s3 decorator to both parent and child class?
I'm guessing since you mock every single method of the class on which decorator is used, this is expected but just wanted to confirm again.
Moto version = 3.1.4
boto3 version = 1.14.29
botocore version = 1.17.63

Hi @saichander17, the decorator adds the mock-behaviour at compile-time. At compile-time we only know about Parent::create, so the mock will be started before this method executes, and ends after this method has finished.
Any code executed afterwards is no longer mocked.
In your example, the mock only needs to be added to the Child-class, to ensure that the mock is active around the Child::create-method.
(It is possible to add it to the Parent-class as well, but if the mock is already active for the Child-class, the mock for the Parent-class is ignored.)
Does that solve your problem?

Yes, Thanks for the detailed answer. I think this info should be available somewhere in the documentation to avoid the confusion.


text:
Release Moto 4.x

The release of Moto 4.x is due soon now released, and will contain the following changes:
Multi-Account support (see the documentation)
Removal of the mock_dynamodb2 and mock_rds2 decorators
The ThreadedMotoServer will be considered out of beta, and the API considered stable
The following dependencies will get a minimum version:
responses >= 0.13.0
Timeline
A new development release will be created for every change, with some time in between every release to catch any bugs.
The table below contains the approximate exact timeline:

Feature
Release Date
Version

Multi-Account support
13/08/2022
4.0.0.dev3

Removal deprecated decorators
16/08/2022
4.0.0.dev4

Set minimum version of responses
17/08/2022
4.0.0.dev5

ThreadedMotoServer
18/08/2022
4.0.0.dev10

Full Release
20/08/2022
4.0.0

Feedback
We encourage everyone to test these changes as early as possible, and let us know about any bugs you may encounter by raising a new issue.
Breaking Changes
Introducing the Multi-Account support means changing the complete data structure, to ensure the data for each account is logically separated.
We know a lot of users access the internal API to have better control over the mocked data. Before, you may have done something like this to access internal models:
from moto.ec2.models import ec2_backends

ec2_backend = ec2_backends["us-west-1"]

As of verion 4, you will also have to specify which Account ID you want to access:
from moto.ec2.models import ec2_backends

ec2_backend = ec2_backends[ACCOUNT_ID]["us-west-1"]

The variable moto.core.ACCOUNT_ID is also removed, in favor of moto.core.DEFAULT_ACCOUNT_ID.
Issue Change Log

Date
Change

09 Aug 22
Added this table & Added the note on removal of ACCOUNT_ID

13 Aug 22
Added release-date for MultiAccount support

16 Aug 22
Added the Removal of Deprecated Decorators release

20 Aug 22
Full v4 release

MultiAccount support is now available as of Moto >= 4.0.0.dev3

The deprecated decorators are now removed as of Moto >= 4.0.0.dev4

Moto 4 is now released! Please raise a new issue if you encounter any bugs/problems with this version, as usual.

@bblommers Please mention how to replace usages of the removed deprecated mock_dynamodb2 decorator. All I can find by googling is an issue that proposed how to ignore deprecation warnings... instead of upgrading to whatever the current practice should be, which is the only option as of 4.0.0.

Is it as simple as replacing mock_dynamodb2 with mock_dynamodb?

Yes, it is @berzi  - I'll mention it above as well for clarity. Thanks for bringing this to our attention.
FYI, the full history/reasoning behind this change can be found here: https://github.com/spulec/moto/issues/4526

authors: bblommers berzi


text:
Spyder crashes while typing docstrings

Description
What steps will reproduce the problem?

Type docstrings (""" """) of classes after 4.0 update. Kite extension is being used also. Not sure if that is what is causing it.
Traceback
  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\plugins\editor\panels\codefolding.py", line 256, in paintEvent

QBackingStore::endPaint() called with active painter on backingstore paint device
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\plugins\editor\panels\codefolding.py", line 256, in paintEvent
    self._draw_fold_region_background(block, painter)
  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\plugins\editor\panels\codefolding.py", line 300, in _draw_fold_region_background
    end = self.folding_regions[start]
KeyError: 842
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\plugins\editor\panels\codefolding.py", line 256, in paintEvent
    self._draw_fold_region_background(block, painter)
  File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\plugins\editor\panels\codefolding.py", line 300, in _draw_fold_region_background
    end = self.folding_regions[start]
KeyError: 842

Versions

Spyder version: 4.0.0
Python version: 3.6.8
Qt version: 5.9.6
PyQt5 version: 5.9.2
Operating System: Windows 10

Dependencies
cloudpickle >=0.5.0          :  1.2.2 (OK)
pygments >=2.0               :  2.5.2 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
nbconvert >=4.0              :  5.6.1 (OK)
sphinx >=0.6.6               :  2.3.0 (OK)
pylint >=0.25                :  2.4.4 (OK)
psutil >=0.3                 :  5.6.7 (OK)
qtawesome >=0.5.7            :  0.6.0 (OK)
qtpy >=1.5.0                 :  1.9.0 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
zmq >=17                     :  18.1.0 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
numpydoc >=0.6.0             :  0.9.1 (OK)
spyder_kernels >=1.8.1;<2.0.0:  1.8.1 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
atomicwrites >=1.2.0         :  1.3.0 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
intervaltree                 :  None (OK)
watchdog                     :  None (OK)
keyring                      :  None (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pympler                      :  None (OK)
sympy >=0.7.3                :  1.5 (OK)
cython >=0.21                :  0.29.13 (OK)
IPython >=4.0                :  7.10.2 (OK)
matplotlib >=2.0.0           :  3.1.1 (OK)
pandas >=0.13.1              :  0.25.1 (OK)
numpy >=1.7                  :  1.17.4 (OK)
scipy >=0.17.0               :  1.3.2 (OK)
pyls >=0.31.2;<0.32.0        :  0.31.2 (OK)
rtree >=0.8.3                :  0.8.3 (OK)

This is fixed in our latest version. Please update by opening the Anaconda Prompt and running there
conda install spyder=4.0.1

It'll probably take a couple of days for the package to be available, please be patient.

repo: spyder-ide/spyder


text:
Python processes linger after quitting Spyder macOS standalone application

Python processes started by Spyder do not seem to be killed upon quitting Spyder, as seen in the Activity Monitor and in the Dock.
I'll have to investigate this further to determine what is causing this issue, but I do not see the same behavior when launched from conda or bootstrap; I think it is only the macOS standalone application.

See comment: https://github.com/spyder-ide/spyder/issues/17731#issuecomment-1105938824


text:
Menu view/toolbar has empty contents

on my installs with spyder 3.0.1 and 3.0.2 the menuoption for toolbars is not showing.
the installs are fresh anaconda installs on windows 64-bit, and then an upgrade of spyder.
I think it worked with the 3.0.0, since I did change the toolbars on one computer,
but that is no longer possible.
did not directly find this mentioned anywhere - if it is a duplicate just close.

This is right. I don't know what happened with that menu.
@dalthviz, please solve this one.

just tried to paste the added code into my qthelpers.py,
and the menu content was back.
Thanks!!


text:
OK Button does not always dismiss Preferences Dialog

From steve.f....@gmail.com on 2010-11-29T12:08:43Z
Similar to #462
What steps will reproduce the problem?
Not sure how to reproduce. I was just browsing the preferences dialog and hit OK to dismiss it. What version of the product are you using? On what operating system? spyder 2.0.0rc3, python 2.6.1, Windows XP SP3
Please provide any additional information below
. Traceback info from Internal Console:
>>> Traceback (most recent call last):
File "C:\Python26\lib\site-packages\spyderlib\widgets\editor.py", line 1356, in _refresh_outlineexplorer
update=update)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 577, in set_current_editor
self.treewidget.set_current_editor(editor, fname, update)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 308, in set_current_editor
self.populate_branch(editor, item, tree_cache)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 436, in populate_branch
item = TreeItem(data.text, line_nb, parent, preceding)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 123, in init
while preceding.parent() is not parent:
AttributeError: 'NoneType' object has no attribute 'parent'
Traceback (most recent call last):
File "C:\Python26\lib\site-packages\spyderlib\plugins\configdialog.py", line 146, in button_clicked
configpage.apply_changes()
File "C:\Python26\lib\site-packages\spyderlib\plugins\configdialog.py", line 67, in apply_changes
self.apply_callback()
File "C:\Python26\lib\site-packages\spyderlib\plugins\configdialog.py", line 169, in <lambda>
self.apply_settings(self.changed_options))
File "C:\Python26\lib\site-packages\spyderlib\plugins\editor.py", line 1837, in apply_plugin_settings
self.outlineexplorer.update()
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 595, in update
self.treewidget.update_all()
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 339, in update_all
self.populate_branch(editor, item, tree_cache)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 436, in populate_branch
item = TreeItem(data.text, line_nb, parent, preceding)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 123, in init
while preceding.parent() is not parent:
AttributeError: 'NoneType' object has no attribute 'parent'
Traceback (most recent call last):
File "C:\Python26\lib\site-packages\spyderlib\plugins\configdialog.py", line 137, in accept
configpage.apply_changes()
File "C:\Python26\lib\site-packages\spyderlib\plugins\configdialog.py", line 67, in apply_changes
self.apply_callback()
File "C:\Python26\lib\site-packages\spyderlib\plugins\configdialog.py", line 169, in <lambda>
self.apply_settings(self.changed_options))
File "C:\Python26\lib\site-packages\spyderlib\plugins\editor.py", line 1837, in apply_plugin_settings
self.outlineexplorer.update()
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 595, in update
self.treewidget.update_all()
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 339, in update_all
self.populate_branch(editor, item, tree_cache)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 436, in populate_branch
item = TreeItem(data.text, line_nb, parent, preceding)
File "C:\Python26\lib\site-packages\spyderlib\widgets\editortools.py", line 123, in init
while preceding.parent() is not parent:
AttributeError: 'NoneType' object has no attribute 'parent'
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=474

From pierre.raybaut on 2010-11-29T23:11:45Z
Status: Fixed


text:
Automatic error report

<!---   *** BEFORE SUBMITTING: PASTE CLIPBOARD HERE TO COMPLETE YOUR REPORT ***   ---The connecting to kernel keeps appearing at the IPython console

Closing because this is empty.


text:
PR: Create a Comm socket

Description of Changes
Calling do_one_iteration is not a good idea as it might process messages out of order.
Here, a new socket is created. It is a one way socket designed to skip the message queue in the kernel, and runs in a thread. This means that:

Only messages that are explicitely sent to this socket are processed out of order, instead of all messages as it is the case now.
Any thread can make a blocking call to the frontend without needing to start an event loop.
We can set breakpoints while executing!

[x] Wrote at least one-line docstrings (for any new functions)
[x] Added unit test(s) covering the changes (if testable)

[ ] Included a screenshot or animation (if affecting the UI, see Licecap)

Issue(s) Resolved

Fixes #
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:

retrigger tests

This is ready for review


text:
MacApp: Use login-shell option on spyder_kernel launch

Description of Changes
Use login shell option in shell script to start spyder_kernel for both internal and external iPython console interpreters.

[x] Wrote at least one-line docstrings (for any new functions)
[x] Added unit test(s) covering the changes (if testable)

[x] Included a screenshot or animation (if affecting the UI, see Licecap)

By adding the login shell option, a user's local environment variables will be passed to the iPython console interpreter environment. This is required when using external applications such as LaTeX or MDSPlus.
This is an issue when Spyder is not launched from a login-shell environment, such as with a stand-alone Mac application.
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:
@mrclary

@mrclary, I'm going to cancel your builds in Azure for now because I'm trying to fix failures in our tests on Windows.
I'll restart them once I managed to do that. Also, it'd be great if you could wait before pushing more commits until I let you know things are good to go again, so I don't have to keep doing this manually.
Thanks!

Hello @mrclary! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file spyder/utils/programs.py:

Line 145:13: E129 visually indented line with same indent as next logical line

Will look at why the tests are failing

@mrclary which PR is next in line and has passing tests (to review and merge) ?

#12011 and #12271 pass and should be next. Then #12236 and #12237, but these still don't pass

@goanpeca, the only failure here is test_conda_env_activation which passes on my local Ubuntu, so I don't know what the issue could be.

This PR should follow #12236.

This still passes on my local VM, but now test_auto_backend is failing on the server !?
I fixed test_auto_backend in #12236; why is it showing up again here?
This PR is the lowest priority of group for the Mac app, so we can defer.

This still passes on my local VM, but now test_auto_backend is failing on the server !?
I fixed test_auto_backend in #12236; why is it showing up again here?

Sometimes the changes make subtle changes in the pytest runs, so I would not worry too much about this. As soon as the other PRs are in we can look again into this one, maybe changing the order in which the failing test is ran.
Thanks again.

@mrclary, I think we can close this one, right?

Yep.


text:
PR: Fix cog menu not showing in the variable explorer when more than one IPython console is opened

Pull Request Checklist

[x] Read and followed this repo's Contributing Guidelines
[x] Based your PR on the latest version of the correct branch (master or 3.x)
[x] Followed PEP8 for code style
[x] Ensured your pull request hasn't eliminated unrelated blank lines/spaces,
modified the spyder/defaults directory, or added new icons/assets
[x] Wrote at least one-line docstrings for any new functions
[ ] Added at least one unit test covering the changes, if at all possible
[x] Described your changes and the motivation for them below
[x] Noted what issue(s) this pull request resolves, creating one if needed
[x] Included a screenshot, if this PR makes any visible changes to the UI

Description of Changes

The layout of the VariableExplorer plugin consists mainly of a StackWidget.  When an IPython console is created, a corresponding NamespaceBrowser is instantiated and added to the stack.
The cog menu button (cmb) is created by the variable explorer plugin and passed as a class argument when instantiating a new namespace browser (nsb ), which is later added to its layout during the setup. So this means that the cmb is shared among all the nsb of the stack.
The problem is that, when the current IPython console change, and so the current nsb of the stack widget, there is no mechanism to update the cmb  actions and to move it from the previous current nsb layout to that of the new current nsb. In order to solve this issue, this PR:

Add the code infrastructure to update the cmb actions when the current nsb of the stack change.
Add the code infrastructure to move the cmb to the current nsb layout.

The changes at the plugin layer are minimal and use the public interface already in place for plugins. Almost all changes were made at the widget layer.
Issue(s) Resolved

Fixes #7704

@ccordoba12 If you want a test to cover this, could you please tell me where this test should go?

If you want a test to cover this

Don't worry, I don't think a test is necessary (or even doable) for this.


text:
Gulp output files

I keep accidentally editing the gulp outputs instead of the inputs. Can we make the outputs be set read-only (so you can't save over them) and have a one-line comment header at the top that says they're machine-generated and shouldn't be edited?
I looked through the gulp docs and couldn't see an easy way to programmatically do either of these things to all the output files.

Will look on it. For now we probably should exclude links to generated files from projects.

authors: iskiselev kg

repo: sq/JSIL


text:
Fix Database Status check & Add instance/computer to it blocks

A New PR
THANK YOU - We love to get PR's and really appreciate your time and help to improve this module
Accepting a PR
Before we accept the PR - please confirm that you have run the tests locally to avoid our automated build and release process failing. You can see how to do that in our wiki
https://github.com/sqlcollaborative/dbachecks/wiki
Please confirm you have 0 failing Pester Tests
[x] There are 0 failing Pester tests
Changes this PR brings

Fixes https://github.com/sqlcollaborative/dbachecks/issues/566
Add instance/computer name to "It" blocks where it was missing. It was hard reading the Power BI grid to know some errors belongs to which instance.

@SQLDBAWithABeard can you take a look? Should I submit a dummy commit to start the checks again?

repo: sqlcollaborative/dbachecks


text:
PlanCache updates, renames and tests

/cc @tboggiano for fysa

this has a file i need to have fixed in dev so i merged 👎

repo: sqlcollaborative/dbatools


text:
0.9.518

breaking changes as described at https://dbatools.io/breaking-changes-update/

wait no, this was ssome small updates whoops, i was wondering!


text:
Pssa/new dba db data generator config

Type of Change

[x] Bug fix (non-breaking change, fixes #)
[ ] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose
It'd be unfortunate to overwrite a perfectly good (albeit old) config, so in this case I think ShouldProcess should be used. It's once for db and only if a previous file is found so not-too-nagging.

@sanderstad will this cause issues with your upcoming PR? if so, can you manually add his changes to your file set?

fixed help 👍 and no issues with sander's commands

authors: niphlod potatoqualitee


text:
Silence notification when file/path is in .sqlfluffignore

When the extension is configured to run automatically upon saving a file, and that file/path is in .sqlfluffignore, it keeps showing this notification:

Would it be possible to add a toggle to disable this notification? We know that the file shouldn't get linted, but the extension tries to do it anyway. 🤷♂
And thank you for the amazing extension! 🙏 💙

Does adding the argument to your extension setup in settings.json disable the notification? "sqlfluff.linter.arguments": ["--disregard-sqlfluffignores"]. You might have to add that to the format arguments as well if you are trying to format it.
You could also try disabling the notification popups with "sqlfluff.suppressNotifications": true

Yeah.... I was hoping to:

Not ignore .sqlfluffignore.
Not disable all notifications.

It's really this specific notification which doesn't bring much value. If the extension took .sqlfluffignore into consideration before running, then it wouldn't even trigger for those files. But since it triggers on every .sql file when it's saved... we end up here.
I'll try your recommendation of "sqlfluff.linter.arguments": ["--disregard-sqlfluffignores"] for now. At least that will get rid of the notification, but now these files will be full of errors (since they shouldn't be linted in the first place). 😞

What version of the extension are you on? I think that error message has been hidden in the current version of the extension, but not sure about the 2.x versions of the extension.

I'm on v3.2.0. It's set to auto update.

Hmm, looks like they changed the message, you must be on a later sqlfluff version than I am. I just published v3.2.1 which should hide that error message.

It works! 🎉 🍾 🚀
Thank you!! 🙏 🙌
And you did it so quick! 🏎 💪  💙

authors: RobertOstermann mroy-seedbox

repo: sqlfluff/vscode-sqlfluff