text:
Migrating to Connexion 3: Issues with current_app and g for Authentication and Request Lifecycle

Description
I am currently in the process of migrating a Flask application to Connexion 3 and have encountered an issue related to the use of current_app and the g object, which I previously utilized extensively in my Connexion 2 FlaskApp setup. In the older version, I used to set several variables in g during the @before_request hook, and these variables were accessible throughout the lifespan of a request, including in my authentication methods and in methods decorated with @after_request.
This setup was crucial for maintaining request-scoped data, such as user authentication details and other contextual information needed across different parts of the application during the processing of a single request.
However, with the upgrade to Connexion 3, I am finding it challenging to replicate this behavior. The documentation and migration guides I've consulted so far do not provide clear instructions on how to achieve similar functionality with the updated framework version. Specifically, I am looking for guidance on:

How to properly set and access request-scoped variables using g or an equivalent mechanism in Connexion 3.
Ensuring that these variables are accessible in the authentication methods and throughout the request lifecycle, including @before_request and @after_request hooks.
Any changes or updates in Connexion 3 that affect the use of current_app in the context of request handling and authentication.

Could you provide examples or guidance on how to adapt my existing FlaskApp code to work with Connexion 3, particularly focusing on the use of current_app and g for request-scoped data handling and authentication?
Output of the commands:

Python 3.10.9
Connexion 3.0.5
Flask 2.2.5
Werkzeug 2.3.8

Thanks @gaurav-triverus.
I submitted a PR which makes the request available to the security handlers. In combination with a custom before/after-request middleware, you should be able to implement this flow.
class CustomMiddleware:

    def __init__(self, next_app: ASGIApp):
        self.next_app = next_app

    def before_request(scope: Scope):
        # Code here. State can be stored on the scope.

    def after_request(scope: Scope):
        # Code here. State can be stored on the scope.

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        self.before_request(scope)
        await self.app(scope, receive, send)

app = connexion.FlaskApp(__name__)
app.add_middleware(CustomMiddleware, position=connexion.middleware.MiddlewarePosition.BEFORE_SECURITY)

You can then access the state via request.scope in your security handler.

Thanks @RobbeSneyders
Just to confirm, even after those changes flask g object is not accessible in security layer.
we will have to create a custom middleware to set variables to be used in throughout the lifespan?

Yes indeed @gaurav-triverus
g is only available within the context of the Flask application. Since Connexion 3's security functionality is performed by middleware wrapped around the Flask application, it's not possible to have access there.

Hi, I have some issues implementing this custom Middleware.
this is my code:
`class CustomMiddleware:
def init(self, next_app):
self.next_app = next_app
def before_request(self, scope):
    print('test app  -> CustomMiddleware before_request')
    print("scope => " + str(scope))

def after_request(scope):
    print('test app  -> CustomMiddleware after_request')

async def __call__(self, scope, receive, send) -> None:
    print('test app -> CustomMiddleware __call__')
    self.before_request(scope)
    await self.next_app(scope, receive, send)

def get_project_root_dir() -> str:
return os.path.dirname(os.path.dirname(file))
def decode_jwt(token):
print('test app--> in decode_jwt')
decoded_token = jwt.decode(
    token,
    "secret",
    algorithms=["HS256"],
    options=dict(require_exp=True, require_sub=True),
)

return decoded_token

def hellow_world():
print('test app -->', 'in hellow_world ')
stores = [{"name": "local flask app", "description": "local flask app to test connexion "}]
return {"stores": stores}
def create_app():
print('test app --> in create_app')
options = SwaggerUIOptions(swagger_ui_path="/docs")
app = connexion.FlaskApp(
name, specification_dir=get_project_root_dir(), swagger_ui_options=options
)
app.add_api(
    'openapi.yaml',
    swagger_ui_options=options,
    strict_validation=True,
    validate_responses=True
)

app.add_middleware(CustomMiddleware, position=connexion.middleware.MiddlewarePosition.BEFORE_SECURITY)

flask_app = app.app

@flask_app.before_request
def push_db():
    print('test app --> in @flask_app.before_request')

@flask_app.teardown_request
def close_db(exception=None):
    print('test app --> in @flask_app.teardown_request')

@flask_app.after_request
def access_log_end(response):
    print('test app --> in @flask_app.after_request')
    return response

app.run(host = "0.0.0.0",
        port=5000,
        )

if name == "main":
create_app()`
when running the code and testing the endpoint hellowworld I have 1 issue:

Printing steps in the code shows that before_request in customMiddleware is called 2 times with different value for scope param.

Results:
test app --> in create_app
test app -> CustomMiddleware call
test app  -> CustomMiddleware before_request
scope => {'type': 'lifespan', 'asgi': {'version': '3.0', 'spec_version': '2.0'}, 'state': {}, 'app': <connexion.middleware.main.ConnexionMiddleware object at 0x7fd76318a100>}
test app -> CustomMiddleware call
test app  -> CustomMiddleware before_request
scope => {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.22.0.2', 5000), 'client': ('172.22.0.1', 44230), 'scheme': 'http', 'root_path': '', 'headers': [(b'host', b'localhost:5000'), (b'connection', b'keep-alive'), (b'sec-ch-ua', b'"Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"'), (b'accept', b'application/json'), (b'sec-ch-ua-mobile', b'?0'), (b'authorization', b'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiYWRtaW4ifQ.LmuLCvLRfzvDVYK_iUBmwL3-5K9N0QLFrHwhXVb5TPU'), (b'user-agent', b'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36'), (b'sec-ch-ua-platform', b'"Windows"'), (b'sec-fetch-site', b'same-origin'), (b'sec-fetch-mode', b'cors'), (b'sec-fetch-dest', b'empty'), (b'referer', b'http://localhost:5000/docs/'), (b'accept-encoding', b'gzip, deflate, br, zstd'), (b'accept-language', b'en-US,en;q=0.9,es-US;q=0.8,es;q=0.7'), (b'cookie', b'_ga=GA1.1.1667569712.1706132845; _ff={%22enableLocalizeVietnameseLanguage%22:false}; _ga_SW2TVH2WBY=GS1.1.1708531929.9.1.1708540204.0.0.0')], 'state': {}, 'method': 'GET', 'path': '/hellowworld', 'raw_path': b'/hellowworld', 'query_string': b'', 'app': <connexion.middleware.main.ConnexionMiddleware object at 0x7fd76318a100>, 'starlette.exception_handlers': ({<class 'starlette.exceptions.HTTPException'>: <function ExceptionMiddleware.http_exception at 0x7fd7634a0e50>, <class 'starlette.exceptions.WebSocketException'>: <bound method ExceptionMiddleware.websocket_exception of <connexion.middleware.exceptions.ExceptionMiddleware object at 0x7fd762d2c5b0>>, <class 'connexion.exceptions.ProblemException'>: <function ExceptionMiddleware.problem_handler at 0x7fd762d3ab80>, <class 'Exception'>: <function ExceptionMiddleware.common_error_handler at 0x7fd762d3ac10>}, {}), 'path_params': {}, 'extensions': {'connexion_routing': {'api_base_path': '', 'operation_id': 'app.hellow_world'}}}
test app--> in decode_jwt
test app --> in @flask_app.before_request
test app --> in hellow_world
test app --> in @flask_app.after_request
test app --> in @flask_app.teardown_request
INFO:     172.22.0.1:44230 - "GET /hellowworld HTTP/1.1" 200 OK

authors: JoelPelaezP RobbeSneyders gaurav-triverus oyeamit

repo: spec-first/connexion


text:
oneOf does not work on requestBody with content type application/x-www-form-urlencoded

Description
With oneOf :
"/oauth/token" : {
      "post" : {
        "tags": [
          "oauth"
        ],
        "summary": "Get Token",
        "description": "Use this endpoint to:\nGet an Access Token in order to call an API. You can, optionally, retrieve an ID Token and a Refresh Token as well.\nRefresh your Access Token, using a Refresh Token you got during authorization.",
        "operationId": "orbital_user_management.api.oauth.token",
        "requestBody": {
          "description": "token parameters",
          "content": {
            "application/x-www-form-urlencoded": {
              "schema": {
                "oneOf": [
                  {
                    "$ref": "#/components/schemas/resource_owner_request"
                  },
                  {
                    "$ref": "#/components/schemas/refresh_token_request"
                  }
                ]
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Successfully created",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/token_response"
                }
              }
            }
          },
          "401": {
            "description": "Unauthorized",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/error"
                }
              }
            }
          }
        }
      }
    }

validation error: Extra formData parameter(s) client_id, refresh_token, grant_type not in spec
without oneOf
"/oauth/token" : {
      "post" : {
        "tags": [
          "oauth"
        ],
        "summary": "Get Token",
        "description": "Use this endpoint to:\nGet an Access Token in order to call an API. You can, optionally, retrieve an ID Token and a Refresh Token as well.\nRefresh your Access Token, using a Refresh Token you got during authorization.",
        "operationId": "orbital_user_management.api.oauth.token",
        "requestBody": {
          "description": "token parameters",
          "content": {
            "application/x-www-form-urlencoded": {
              "schema": {
                "$ref": "#/components/schemas/resource_owner_request"
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Successfully created",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/token_response"
                }
              }
            }
          },
          "401": {
            "description": "Unauthorized",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/error"
                }
              }
            }
          }
        }
      }
    }

no validation error
Additional info:
Output of the commands:

python --version
pip show connexion | grep "^Version\:"

Closing in favor of https://github.com/spec-first/connexion/issues/1569

source: gharchive/issue

authors: RobbeSneyders stephane-tagheuer

license_type: permissive

license_source: github-api


text:
Proposal: Support JWT

Would be great if this helper supported grabbing a JWT and making that available to the app. I propose adding a couple of fields to the config spec, jwtPath where the file should be written (this would respect the cert_dir already there, and jwtENV, an environment variable to inject the JWT into.
Signal handling would remain the same, etc...
If folks agree with the proposal I'm happy to implement and PR.

OK scratch this, looks like without the new API this is going to be a huge refactor. Might as well wait ;)

repo: spiffe/spiffe-helper


text:
Add support for Nextjs #151

updated carbon charts to latest version
updated to latest react, react-dom, added latest next

This PR addresses #151
Opening up for review at this point but please note with the removal of @wojtekmaj/enzyme-adapter-react-17 which was used in two test files cluster-create.test and cluster-list.test those two tests might need to be removed altogether.
Fully migrating to next.js will probably require discussion on how to handle structure and which components need to remain client components vs the default server components in next. I believe also those components which are designated server will need to be migrated away from class components.

Moved to https://github.com/spiffe/tornjak/pull/317

source: gharchive/pull-request

repo: spiffe/tornjak


text:
Implement the same functionality as SSLPersonas

"SSLPersonas shows you how secure a web page is in a very obvious way: It changes the Firefox theme according to the SSL status of a website." - SSLPersonas

https://github.com/ylecuyer/envify/issues/18

I'm conflicted. Last time someone asked me to make functionality for site-specific color, I turned it down and closed the issue with "out of scope" and "wontfix". And then Envify came around, which did exactly that, and I came second in the competition... to Envify. So while I would consider this out of scope, I am hesitant to not add this feature because I don't want to be kicking myself again for missed opportunities.

I accept your decision. Do you want me to close the issue or should it be kept open?

Well that's the problem I don't know what my decision is. Do you really think this fits the purpose of the extension well enough to be included, or do you think it's important enough to rethink the purpose of the extension to fit a broader scope?

https://github.com/SSLPersonas/SSLPersonas/issues/10

authors: davidhedlund spikespaz

repo: spikespaz/firefox-nativedark

license: MIT


text:
Prepare to 1.6.3 release

@wolfy-j BORS is ready to help you :)

bors +

bors r+

repo: spiral/roadrunner


text:
#82 downgrade fluentd docker version, allow fluentd minor version upg…

…rades, update readme on building gem files and docker image
Proposed changes
Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue.
Types of changes
What types of changes does your code introduce?
Put an x in the boxes that apply

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist
Put an x in the boxes that apply.

[x] I have read the CONTRIBUTING doc
[x] I have read the CLA
[ ] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

There's no need to downgrade the Dockerfile version; simply updating the dependency to be ~>rather than = should suffice IMO.

Fixed as part of https://github.com/splunk/fluent-plugin-splunk-hec/pull/87

authors: MrMarshall atkinsj chaitanyaphalak

repo: splunk/fluent-plugin-splunk-hec


text:
Nterl0k T1548.002 - I'm down with UAC, yeah know me.

Details
Contains 2 generic UAC behavior detections based on process names commonly used for UAC bypass techniques. These detections are partially overlap existing ESCU content, however they also pull forward a number of suspect UAC behaviors not currently accounted for.
Pending https://github.com/splunk/attack_data/pull/864

Checklist

[ ] Validate name matches <platform>_<mitre att&ck technique>_<short description> nomenclature
[ ] CI/CD jobs passed ✔
[ ] Validated SPL logic.
[ ] Validated tags, description, and how to implement.
[ ] Verified references match analytic.

Thanks @nterl0k : detections are reviewed and tested

Huzzah!

Shipped in 4.19.0 :)


text:
Anyone success retrieve AKS master node log and API Server logs ?

Hi ALL,
Anyone success pulling AKS master logs such as host log (e.g. /var/log/auth.log) or container logs (e.g. kube-apiserver log) using splunk connect for kubernetes ? I am playing with 1.4.10 and able to receive all logs from node including host log and container log, but no luck with any AKS master node log. Any suggestions are welcome. Thanks a lot.

I am using AKS, the Kubernetes v1.20.9. I cannot find a way to ssh into AKS master node. However, from Azure Log Analytics Workspace, "AzureDiagnostics" shows the log path of API server is something like "/var/log/pods/<pod_uid>/kube-apiserver/0.log".

so you are not getting container logs nor host logs from master node? is the daemonset deployed to the master node?

Anything from AKS control plane would have to come from Event Hub, I believe. AKS does not let you access the control plane directly. We will only pick up logs off the node.

Check out: https://docs.microsoft.com/en-us/azure/aks/monitor-aks#collect-resource-logs

@lastcoolnameleft Tommy!! Good to bump into you here virtually! Hope to get to see ya at a Kubecon again in the future! Thanks for the share!

@lastcoolnameleft @matthewmodestino Thanks for your information. I can grab the host log (e.g. /var/log/auth.log) for VM, Virtual Machine Scale Set and AKS non-autoscale set. However, seems it does not apply to AKS autoscale set. We have tried Azure policy, deploy Log Analytics agent for Linux VMs & deploy Log Analytics agent for Linux virtual machine scale sets, custom log. Seems all those cannot applied to AKS autoscale set.
Also, I have tried to override the default Azure Log Analytics, install Splunk agent(https://github.com/CyberSecThreat-Corporation-Limited/Splunk) or Splunk-connect-for-kubernetes. All these methods seems can collect host log on worker node, but not AKS master node.
I haven't tried "EventHub" way, but it should works just like send to "Log Analytics workspace". Just want to check if anyone has successfully grab "/var/log/auth.log" on AKS autoscale set master node.
Thanks again.

Do you have direct access to the master node? my understanding is that MSFT runs the control plane in AKS, can you point me to what exactly the "master node" appears like to you?

@matthewmodestino Thanks for your reply again. I cannot find any documentation how can I access to "master node". What I refer to "master node" is actually the "control plane" usually running kube-api, etcd, kube-scheduler and kube-controller. For on-premise environment, we will install those components to a "master node" to interact with, while other worker nodes will run application service workloads. I was asked to check if there are any "dedicated machines" if an AKS auto-scale set is provisioned. If that control plane is shared among other customer, I think there are no needs monitor the host log. Just want to check if anyone had experience regarding this. Thanks again.

Yeah, because AKS is a managed control plane, MSFT will expose what logging is available as seen in the document above.
The only host logs you will need are from the worker nodes.

@matthewmodestino Thanks a million! I will close this question.

authors: kkkelvinkk lastcoolnameleft matthewmodestino rockb1017

repo: splunk/splunk-connect-for-kubernetes


text:
Documentation for multiline separator is missing

Helm chart allows providing separator in multiline config. But, it is not documented.
Related to https://github.com/splunk/splunk-connect-for-kubernetes/issues/791

It is used here:
https://github.com/splunk/splunk-connect-for-kubernetes/blob/22740cd246999a88bc807fa14237451f3b1be43a/helm-chart/splunk-connect-for-kubernetes/charts/splunk-kubernetes-logging/templates/configMap.yaml#L222


text:
Added a Flake8 check for slience PEP standard checks

Description:

Added a Flake8 check to silence the PEP standards

Tests:

Added the tests along with it

AFAIC, putting a noqa comment on the first line doesn't do anything - you have to put it at the end of each line for flake8 to recognise it.
The new --add-noqa option, just released in version 1.3, does this. It also handles existing noqa and other comments.
Thanks for the point about the git branch (develop -> master) - I've fixed that now.

Thanks since you have added the noqa change, I am closing this PR :+1:

repo: spookylukey/table-format


text:
Add an option to pass youtube URLs to mpv player

Requested Feature
Can you include an option that allows you to display the YouTube URL of a Spotify song or playlist in the terminal and call mpv to play this track? Something like: mpv [URL 1] [URL 2] [URL3]
Possible implementation
spotdl play Get YouTube URL of a Spotify song or playlist and play it in mpv

mpv $(spotdl url test abc xddd hehe --simple-tui)
update to dev

repo: spotDL/spotify-downloader


text:
Add Currency (Symbol)

Currency name:
Symbol:
Website:
Theme color:
Logo (SVG format):

Kuzz

repo: spothq/cryptocurrency-icons


text:
Training code

Hello! Thank you for your work! The results sound interesting, but sometimes not perfect. I would like to train my model or use yours for fine tuning. Do you provide code for this?

We're working on this! Please see https://github.com/spotify/basic-pitch/pull/38 for in-progress work regarding training code

repo: spotify/basic-pitch


text:
Drop metric with row key length bigger then BT limit

This change will drop metric with row key length longer than 4096 bytes.
It will also create counter metric so we could alert on high numbers.

Seems like checkstyle is still failing :(

repo: spotify/heroic

license_source: bigquery


text:
what's "assistant"?

hii, guys.
is it has something to do with "worker"?

it's an experimental feature to allow tasks to be run on a host separate from where it's scheduled. basically assistants pull all types of tasks and try to run them

Is assistant used while scheduling tasks from a different host in central scheduler configuration? or it has a different use case?

authors: Sahilshetye erikbern leafjungle

repo: spotify/luigi


text:
put temp file into temporary directory and make it relative

fixes #1368

great. btw I have some ideas on how to refactor the temp file stuff, but will do separately

This definitely needs a test case documenting the behaviour. I'm intending to refactor and generalize  this eventually, and I don't want this to break then =)

I assumed there is already a test case making sure that a file is written.
On Fri, Jan 15, 2016, 6:25 PM Arash Rouhani notifications@github.com
wrote:

This definitely needs a test case documenting the behaviour. I'm intending
to refactor and generalize this eventually, and I don't want this to break
then =)
—
Reply to this email directly or view it on GitHub
https://github.com/spotify/luigi/pull/1515#issuecomment-172135603.

The purpose of the test would be to make sure that if the implementation change is reverted, the test fails.

I fixed the existing test case to work, but it is an integration test with an ftp server (not my favorite). Then I added a line to make sure that the file can be opened and read properly. I do not think that the actual location that the file is put should be considered part of the test, but since the file is at an absolute location the test cannot pass since most machines do not let you write into /test/ directory location. (open to suggestions on a better way to test this)

Can you resolve the flake8 errors?
./luigi/contrib/ftp.py:118:30: E261 at least two spaces before inline comment
./luigi/contrib/ftp.py:124:24: E261 at least two spaces before inline comment
./test/_test_ftp.py:191:66: E261 at least two spaces before inline comment

okay.

you put the space in the wrong place :) should be before the # not after :)

fyi you can run the tests locally by running tox -e flake8

oops. sorry about that. thanks for the tip. they look good now.

thanks. will merge on travis!

authors: Tarrasch a86c6f7964 erikbern


text:
Allows the usage of Hive Metastore

The usage of Hive Metastore greatly speeds up table and partition lookup in luigi.
Although the MetastoreClient was already implemented, the code always looked for one of two alternatives of the hive command, which is slow.
This PR's settings are only active if
[hive]
release: metastore

is set

thanks!


text:
dynamic demo fix

Tied to Issue #2297 @erikbern . No testing really necessary.

nice!


text:
Can't load config from nacos

Nacos Config
env:
nacos：0.8 standalone
spring-boot：2.0.6.RELEASE
spring-cloud：Finchley.SR2
spring-cloud-alibaba-dependencies：0.2.1.RELEASE

I know what's wrong~
spring.cloud.nacos.discovery.server-addr
spring.cloud.nacos.config.server-addr
too similar！


text:
Adding spring-cloud-services-starter-config-client as a dependency

Resolves https://github.com/spring-cloud-stream-app-starters/app-starters-release/issues/80

LGTM, built http and jdbc locally and they ran fine. Also tested on k8s without problems.

Adding any of the spring-cloud-services starter as a dependency to the core would bring in spring-cloud-core into app's classpath. This could potentially cause some side effect from the Spring Cloud Service connectors especially in CF environment.
One such example would be, the Spring Cloud Stream Rabbit binder has the conditional configuration that will be active only if spring-cloud-core is in it's classpath.
But adding the spring-cloud-services-starter-config-client dependency would bring in following dependencies:
\- io.pivotal.spring.cloud:spring-cloud-services-starter-config-client:jar:1.4.1.RELEASE:compile
[INFO] |     +- org.springframework.security.oauth:spring-security-oauth2:jar:2.0.13.RELEASE:runtime
[INFO] |     |  +- commons-codec:commons-codec:jar:1.10:compile
[INFO] |     |  \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:runtime
[INFO] |     |     \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:runtime
[INFO] |     +- io.pivotal.spring.cloud:spring-cloud-services-cloudfoundry-connector:jar:1.4.1.RELEASE:compile
[INFO] |     |  +- io.pivotal.spring.cloud:spring-cloud-services-connector-core:jar:1.4.1.RELEASE:compile
[INFO] |     |  |  \- org.springframework.cloud:spring-cloud-core:jar:1.2.3.RELEASE:compile
[INFO] |     |  \- org.springframework.cloud:spring-cloud-cloudfoundry-connector:jar:1.2.3.RELEASE:compile
[INFO] |     +- io.pivotal.spring.cloud:spring-cloud-services-spring-connector:jar:1.4.1.RELEASE:compile
[INFO] |     |  +- org.projectlombok:lombok:jar:1.16.14:compile
[INFO] |     |  \- org.springframework.cloud:spring-cloud-spring-service-connector:jar:1.2.3.RELEASE:compile
[INFO] |     \- io.pivotal.spring.cloud:cloudfoundry-certificate-truster:jar:1.0.1.RELEASE:compile

which means the spring-cloud-core is always in the app's classpath.

I found one issue. This is added to the config:
"springCloudServicesBootstrap": {
   "spring.application.name": "${vcap.application.name:}"
},

This means that in a non-cf environment spring.application.name is set to "" as we can see in the config server lookup log message:
2017-04-05 09:57:35.312  WARN 77753 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Could not locate PropertySource: I/O error on GET request for "http://localhost:8888//default": Connection refused; nested exception is java.net.ConnectException: Connection refused

Re: https://github.com/spring-cloud-stream-app-starters/core/pull/18#issuecomment-291797078 , following a brief discussion with @markpollack
Applications using the Spring Cloud Stream RabbitMQ binder that run in Cloud Foundry are expected to bind to a Rabbit service, and as a result they would normally get spring-cloud-core on their classpath anyway - so it is unclear how this would change things from that point of view.
Under normal use cases the configuration will ensure that the application uses the Rabbit bound service. The current configuration options for the Rabbit binder also allow superseding the Cloud connectors ( spring.cloud.stream.overrideCloudConnectors)  and defaulting to Spring Boot properties where the use of a Rabbit service is not desired - one such examples is in the Hystrix Stream client of the SCS tile that uses a completely independent binder.

I think the simplest solution for the spring.application.name is to just add a bootstrap.properties file to every generated stream app with this content:
spring.application.name=${vcap.application.name:log-sink}

where log-sink is the name of the app we generate this for

LGTM now, merging ...

authors: ilayaperumalg mbogoevici sobychacko trisberg

repo: spring-cloud-stream-app-starters/core


text:
spring.cloud.config.uri no effect

Spring cloud config client property "spring.cloud.config.uri" has no effect, I see the source in class org.springframework.cloud.config.client.ConfigClientProperties
    public ConfigClientProperties override(
            org.springframework.core.env.Environment environment) {
        ConfigClientProperties override = new ConfigClientProperties();
        BeanUtils.copyProperties(this, override);
        override.setName(
                environment.resolvePlaceholders("${" + ConfigClientProperties.PREFIX
                        + ".name:${spring.application.name:application}}"));
        if (environment.containsProperty(ConfigClientProperties.PREFIX + ".profile")) {
            override.setProfile(
                    environment.getProperty(ConfigClientProperties.PREFIX + ".profile"));
        }
        if (environment.containsProperty(ConfigClientProperties.PREFIX + ".label")) {
            override.setLabel(
                    environment.getProperty(ConfigClientProperties.PREFIX + ".label"));
        }
        return override;
    }

I see "profile" and "label" will override the config, but uri does not, is it right?

Are you setting the property in bootstrap.yml?  A sample project might help.

You haven't explained what "no effect" means. Please describe your problem first with steps to reproduce or a sample project before trying to point out a bug.

My application.yml
`server:
port: 8080
management:
context-path: /
logging:
level:
com.netflix.discovery: 'OFF'
org.springframework.cloud: 'DEBUG'
eureka:
instance:
leaseRenewalIntervalInSeconds: 10
statusPageUrlPath: /info
healthCheckUrlPath: /health
client:
serviceUrl:
defaultZone: ${eureka.addr}
feign:
hystrix:
enabled: true
#spring:
cloud:
config:
server:
native:
searchLocations: classpath:/
spring:
application:
name: demo-service
cloud:
config:
label: master # optional
uri: http://localhost:8889 # optional
profile: ${spring.profiles.active}
zipkin:
baseUrl: ${zipkin.addr}
enabled: true
compression:
enabled: false
flush-interval: 1
sleuth:
feign:
enabled: true
`
spring.cloud.config.uri is set to localhost:8889, but when I startup, it still connnect to localhost:8888

Is it enough to find out the problem?

You need to set spring.cloud.config.uri in bootstrap.yml not application.yml.
From the documentation:

To modify the startup behaviour you can change the location of the config server using bootstrap.properties (like application.properties but for the bootstrap phase of an application context)

Thanks.I will try it.

You need to set spring.cloud.config.uri in bootstrap.yml not application.yml.
From the documentation:

To modify the startup behaviour you can change the location of the config server using bootstrap.properties (like application.properties but for the bootstrap phase of an application context)

It solved my issue

what needs to be done if I don't want to create a separate file and provide entry only in application.properties

From the documentation:

To modify the startup behaviour you can change the location of the config server using bootstrap.properties (like application.properties but for the bootstrap phase of an application context)

repo: spring-cloud/spring-cloud-config


text:
Not able to register module on data flow server deployed on cloud foundry

I am trying connect to spring cloud data flow server deployed on CF from local data flow shell
This is how I go and could connect to data flow REST end point from local shell
server-unknown:>dataflow config server https://spring-cloud-dataflow-server.xyz.com
Successfully targeted https://spring-cloud-dataflow-server.xyz.com
Then when I tried register a module its throws error -
dataflow:>module register --name abcSource --type source --coordinates com.xyz.cloud:abc-soure:jar:exec:1.0.0-SNAPSHOT
Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: Required String parameter 'uri' is not present
Required String parameter 'uri' is not present
org.springframework.cloud.dataflow.rest.client.DataFlowClientException: Required String parameter 'uri' is not present
at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:52)
I think module command do not have uri option.
This works good with cloud data flow server local.

Hi, @sachkk76:
We have moved away from --coordinates and it is replaced with --uri property across all the server variants. Though SCDF's CF server is at 1.0.0.M2, SCDF's core bits (including shell) is at 1.0.0.M3 release - check out the release trains in the project site. So, make sure you're running 1.0.0.M3 release of the shell locally.
Now, for your situation, you'd have to register the apps as described in the docs here. The qualifier :exec is also not necessary anymore. Depending on the message bus of choice, you'd need to register the respective apps. Alternatively, you could bulk import using the module import command.
For instance, if you're using rabbit as the message, you'll be registering the app like:
module register --name jdbc --type sink --uri maven://org.springframework.cloud.stream.app:jdbc-sink-rabbit:1.0.0.M1 --force

(notice the binder type: jdbc-sink-**rabbit**)

Thanks @sabbyanandan
I would upgrade to newer version but for now I am rolling around same issue as could see below
dataflow:>module register --name abcSource --type source --coordinates com.xyz.cloud:abc-soure:jar:exec:1.0.0-SNAPSHOT
Command failed org.springframework.cloud.dataflow.rest.client.DataFlowClientException: Required String parameter 'uri' is not present
dataflow:>module register --name abcSource --type source --uri maven://com.xyz.cloud:abc:jar:1.0.11-SNAPSHOT
You should specify option (--coordinates, --force) for this command
dataflow:>module register --name abcSource --type source --uri maven://com.xyz.cloud:abc:1.0.11-SNAPSHOT --force true
You should specify option (--coordinates) for this command
Now if add coordinate to above command it says -
Option 'uri' is not available for this command. Use tab assist or the "help" command to see the legal options
I am not sure why for spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-SNAPSHOT.jar version
command
module register --name abcSource --type source --coordinates com.xyz.cloud:abc-soure:jar:exec:1.0.0-SNAPSHOT is not working when it worked well in local.
is this URI mean to be data flow sever REST end point in CF?
Sorry for long message.

If you're on SNAPSHOT, please use SNAPSHOT version of local/cf servers and as well as the shell. Make sure you either build the project or download the latest SNAPSHOT versions. The compatibility between servers and the shell is the key.
Also, as pointed out in previous comment, you're still using :exec token while registering the app and that is wrong. See here.

I downloaded 1.0.0.M3 for shell it worked. So I have spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-SNAPSHOT.jar on CF and spring-cloud-dataflow-shell-1.0.0.M3.jar as local shell.
Now getting error in stream creation -
OUT Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.springframework.cloud.stream.module:log-sink:jar:1.0.0.BUILD-SNAPSHOT from/to repository1 (https://xyz.com/nexus/content/repositories/snapshots/): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
So where do I install certificate on CF? Do I have to specify in env? Does Pivotal App manager UI provides me a way to install certificates?

@sachkk76 While the https requires auth in this case which needs to be addressed either by specifying authentication for the xyz.com , do you have problem using http:// instead for your maven repository?

My repo do require authentication and could pass certs path when I did setup this on local.. so I was trying similar on cloud foundry.. so not sure I how do I set up this on CF

@sachkk76: Is this still a problem?

It was certs issue, so we got build pack with certs within resolved the issue. So now could create stream on CF. will close this issue

authors: ilayaperumalg sabbyanandan sachkk76


text:
how to change response httpcode

Question
how to change response httpcode

Have a look at Set status

@xvjialing does this solve your issue?

it solved

authors: Aloren OlgaMaciaszek xvjialing

repo: spring-cloud/spring-cloud-gateway


text:
Update Spring Cloud GCP Home Page

Describe the bug
https://spring.io/projects/spring-cloud-gcp can use an update to document the newest modules.
Sample
E.g., Firestore is missing.

@artembilan We have already updated this in our source. Can we run the update on the web site? Thanks!

Sure! Give me a text and I'll post it on the project page!

@artembilan the updated text is here: https://github.com/spring-cloud/spring-cloud-gcp/blob/master/docs/src/main/asciidoc/sagan-index.adoc

Fixed: https://spring.io/projects/spring-cloud-gcp

authors: artembilan dmitry-s meltsufin saturnism

repo: spring-cloud/spring-cloud-gcp


text:
Pageable argument is handled in FieldQueryMapEncoder

When I define a method in my feign client with only a Pageable argument, this is handled in the FieldQueryMapEncoder.
PagedModel<EntityModel<Foo>> getFooPaged(Pageable pageable).
This throws an IllegalStateException because in the method feign.querymap.FieldQueryMapEncoder#encode it tries to put 2 serialVersionUI keys in a map.

This is broken when going from version 4.0.1 to 4.0.2.
Caused by: java.lang.IllegalStateException: Duplicate key serialVersionUID (attempted merging values -4541509938956089562 and 1232825578694716871)
	at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
	at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182)
	at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at feign.querymap.FieldQueryMapEncoder.encode(FieldQueryMapEncoder.java:49)
	at feign.ReflectiveFeign$BuildTemplateByResolvingArgs.toQueryMap(ReflectiveFeign.java:267)
	at feign.ReflectiveFeign$BuildTemplateByResolvingArgs.create(ReflectiveFeign.java:248)
	at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:67)
	at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:98)
	at jdk.proxy2/jdk.proxy2.$Proxy262.getTmcLocationsPaged(Unknown Source)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.validation.beanvalidation.MethodValidationInterceptor.invoke(MethodValidationInterceptor.java:141)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:218)
	at jdk.proxy2/jdk.proxy2.$Proxy263.getFooPaged(Unknown Source)

Same here, but it also happens with any combination of parameters. The problem is the hierarchy of AbstractPageRequest as @steinmaerivoet indicated.
As a workaround, in our project we added a reimplementation of PageRequest with a different serialVersionUID and then used it as Pageable. I hope this issue will be solved soon.

Hello @steinmaerivoet @rubenqba does the solution suggested by @orange-guo solve your issue? If not, please provide a minimal, complete, verifiable example that reproduces the issue.

In my case it does. I use a client manually created, so @orange-guo solution is the correct one in my case.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

authors: OlgaMaciaszek rubenqba spring-cloud-issues steinmaerivoet

repo: spring-cloud/spring-cloud-openfeign


text:
Problem occurs when adding to gradle plugin

This appears when testing.
When using the gradle wrapper, the above error continues to occur even if you set it in properties.
--

Exception in thread "Daemon client event forwarder" java.lang.OutOfMemoryError: Java heap space
Task :generateTestAot FAILED
FAILURE: Build failed with an exception.
What went wrong:
Execution failed for task ':generateTestAot'.
Java heap space

Could you try with --no-daemon and let us know if that works (to be sure that's not a leak in another plugin, like Kotlin 1.5.0 Gradle plugin has)?
Do you have any customization in ~/.gradle/gradle.properties?
Could you share a repro project?

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Dear
Try running it about 3 times while sout changing the getIssue TEST method in the test package.
The error is reproduced.

project : https://github.com/rhqnrehf/spring-native-issue-temp

Indeed, I can reproduce, this is a pretty bad one cc @bclozel @aclement.
First time I run it:
Failed verification check: this type was requested to be added to configuration but is not resolvable: io.netty.channel.socket.nio.NioSocketChannel it will be skipped
Failed verification check: this type was requested to be added to configuration but is not resolvable: org.springframework.web.reactive.socket.server.upgrade.TomcatRequestUpgradeStrategy it will be skipped

Second time I run it:
Failed verification check: this type was requested to be added to configuration but is not resolvable: io.netty.channel.socket.nio.NioSocketChannel it will be skipped
Failed verification check: this type was requested to be added to configuration but is not resolvable: io.netty.channel.socket.nio.NioSocketChannel it will be skipped
Failed verification check: this type was requested to be added to configuration but is not resolvable: org.springframework.web.reactive.socket.server.upgrade.TomcatRequestUpgradeStrategy it will be skipped
Failed verification check: this type was requested to be added to configuration but is not resolvable: org.springframework.web.reactive.socket.server.upgrade.TomcatRequestUpgradeStrategy it will be skipped

Third time I run it:
Gradle build daemon has been stopped: JVM garbage collector thrashing and after running out of JVM memory

I am going to move 0.10.1 earlier than expected to solve this.


text:
Function calling - FunctionCallingOptions doesn't work with OpenAiChatClient/AzureOpenAiChatClient

Bug description
Function calling doesn't happen when using FunctionCallingOptions with OpenAiChatClient/AzureOpenAiChatClient
Environment
spring-ai-core-1.0.0-20240420.063719-118.jar
spring-ai-openai-1.0.0-20240420.063719-101.jar
spring-ai-azure-openai-1.0.0-20240420.063719-112.jar
Steps to reproduce
    // kotlin code
    val functionOptions = FunctionCallingOptions.builder()
        .withFunction("weatherFunction")
        .build()

    val response: ChatResponse = chatClient.call(
        Prompt(
            listOf(UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?")),
            functionOptions
        )
    )

    println(response.result.output.content)

OpenAiChatClient:
Sorry, as an AI, I don't have real-time capabilities to provide current weather updates. Please check a reliable weather forecasting website or app for the most accurate and up-to-date information.

AzureOpenAiChatClient:
Sorry, as an AI, I don't have real-time capabilities to provide current weather updates. Please check a reliable weather forecasting website or app for this information.

Expected behavior
Function calling should happen and response should contain function call result.
Analysis
AzureOpenAiChatClient.toAzureChatCompletionsOptions and OpenAiChatClient.createRequest don't use FunctionCallingOptions.getFunctions() for building the request,  so function call didn't happen.

@tha2015 where is your weatherFunction defined? Not clear from your snippets above?
As the documentation suggests you can [register your functions] as plain Function Bean or FunctionCallback wrapper.
Here is a working integration tests for FunctionCallback wrapper: AzureOpenAiChatClientFunctionCallIT.
And you can find more working ITs here: https://github.com/spring-projects/spring-ai/tree/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/azure/tool
Let me know if this helps

Also mind that the property spring.ai.azure.openai.chat.options.model has been renamed to spring.ai.azure.openai.chat.options.deployment-name.

I've been having the same issue with OpenAI. Just to test something, I copied OpenAiChatClient class and changed only one thing.
I actually didn't have the time to understand what is the purpose of IS_RUNTIME_CALL, although this tweak helped me make it work.
if (this.defaultOptions != null) {
            // I removed '!' that was modifying IS_RUNTIME_CALL
            Set<String> defaultEnabledFunctions = this.handleFunctionCallbackConfigurations(this.defaultOptions,
                    IS_RUNTIME_CALL);

            functionsForThisRequest.addAll(defaultEnabledFunctions);

            request = ModelOptionsUtils.merge(request, this.defaultOptions, OpenAiApi.ChatCompletionRequest.class);
        }

I could not setup the spring-ai project to run the tests, and when I copying the test to my project it failed. I'm attaching the screenshot of the code where it failed for your reference. My understanding is FunctionCallingOptions was handled like a normal ChatOptions (without tools property) and so the tools property was not used.

@tzolov I found out why the FunctionCallingOptions  IT test passed. It is because that test case functionCallWithPortableFunctionCallingOptions  doesn't have assertions like other test cases. I believe that if you add assertThat(response.getResult().getOutput().getContent()).contains("30", "10", "15"); , the test will fail.

authors: edumaxsantos tha2015 tzolov

repo: spring-projects/spring-ai


text:
Async @RabbitListener does not handle AmqpRejectAndDontRequeueException

Bug report
spring-amqp version: 2.1.7-RELEASE
The documentation for asynchronous @RabbitListener states:

If the async result is completed with an AmqpRejectAndDontRequeueException, the message will not be requeued.

The following example listener instead requeues the message and loops indefinitely:
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.util.concurrent.SettableListenableFuture;

@Component
public class Listener {

  @RabbitListener(queues = "example")
  public ListenableFuture<Void> receive(User user) {
    System.out.println(user);
    SettableListenableFuture<Void> future = new SettableListenableFuture<>();
    future.setException(new RuntimeException());
    return future;
  }
}

The problem seems to lie on the AbstractAdaptableMessageListener. The requeue flag is true regardless of the exception type:
https://github.com/spring-projects/spring-amqp/blob/8336d4ed3f86519d2a9870217b6bb84a9b99bf3f/spring-rabbit/src/main/java/org/springframework/amqp/rabbit/listener/adapter/AbstractAdaptableMessageListener.java#L380-L388
Moreover, not an issue for me now, but the documentation also states:

By default, the message will be requeued, unless the container’s defaultRequeueRejected property is set to false

And from the snippet above, it doesn't seem to be true either.

That was a really quick fix! I was actually interested in contributing here. In the future, should I clearly state it in the bug report?

Right, typically we are very open for Contribution, but right now it is a rush to make as much fixes as possible for upcoming release on Monday. Thank you for proposal anyway: there are still some issues you can pick up for Contribution!

Ok, I will take a look at those soon. This one just looked like such an easy one for my first contribution :) ...

In the future, should I clearly state it in the bug report?

Thanks; yes, in general, it's best to note that you intend to contribute a fix.
As Artem said, though, this time we are so close to the next release I picked it up right away.

I understand, it was better that way for me too, I needed this fix ASAP for a few projects using spring-amqp.

I released it today; 2.1.8 is in maven central.

@garyrussell
Can you point out how to configure this? Within .properties file? I googled everywhere but was not able to find the point where I can set it to false

Does this work for you: spring.rabbitmq.listener.simple.default-requeue-rejected = false?
See https://docs.spring.io/spring-boot/docs/2.2.0.M6/reference/html/appendix.html#integration-properties

repo: spring-projects/spring-amqp


text:
there is none verification of dates

Date visit must be greater than birth date of pet.

https://github.com/yurimokhna/spring-petclinic/commit/5e153c1f4eb0658018317744e806ccc3bd123fa2

authors: viruslemon yurimokhna

repo: spring-projects/spring-petclinic


text:
Configure an inline m2e lifecycle mapping for wro4j-maven-plugin

Signed-off-by: Jinbo Wang jinbwan@microsoft.com
Fixes #750

@testforstephen Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@testforstephen Thank you for signing the Contributor License Agreement!

@dsyer Could you help a look at this PR?

authors: pivotal-issuemaster testforstephen


text:
Update to Spring 4.3.24.RELEASE

Backport of #1685

Fixed via f922b0a6f9af842854373163f4b0a6e9537aca78

repo: spring-projects/spring-security-oauth


text:
Fix Apple's mess

Apple POSTs user first name, last name, and email directly to the callback URL, as opposed to embed them in the ID token like any sane company would do.

You merged the prs together?

repo: sproutfi/kratos


text:
Upgrade to didkit/ssi v0.4.

Upgrading to didkit and ssi v0.4 leads to a compilation error for libp2p-noise:
error[E0282]: type annotations needed
   --> /home/jward/.cargo/registry/src/github.com-1ecc6299db9ec823/libp2p-noise-0.32.0/src/protocol/x25519.rs:221:45
    |
221 |         curve25519_sk.copy_from_slice(&hash.as_ref()[..32]);
    |                                        -----^^^^^^--
    |                                        |    |
    |                                        |    cannot infer type for type parameter `T` declared on the trait `AsRef`
    |                                        this method call resolves to `&T`
    |
    = note: type must be known at this point

For more information about this error, try `rustc --explain E0282`.
error: could not compile `libp2p-noise` due to previous error
warning: build failed, waiting for other jobs to finish...
error: build failed

FYI it's unrelated to didkit, you can check the tree of dependencies with cargo tree

closed by #83 as part of the spruceid/rust-ipfs update

authors: chunningham cobward sbihel

repo: spruceid/kepler


text:
KMS - describe keys doesn't work with alias for boto3

When using the describe_key function of the KMS client, it seems that moto's behavior around looking up aliases differs from boto3.
Environment info:
Python: 2.7.10
Moto: moto-1.0.1
Boto3: boto3-1.4.4
Botocore: botocore-1.5.76
OS: macos and ubuntu
Moto behavior
In [1]: from moto import mock_kms
In [2]: mock = mock_kms()
In [3]: mock.start()
In [4]: import boto3
In [5]: client = boto3.client('kms', region_name='us-east-1')
In [6]: response = client.create_key(Description="mock", KeyUsage="ENCRYPT_DECRYPT", Origin="AWS_KMS")
In [7]: print(response)
Out[7]:
{u'KeyMetadata': {u'AWSAccountId': u'0123456789012',
  u'Arn': u'arn:aws:kms:us-east-1:0123456789012:key/270fc203-b23a-4ccb-8447-be48d7036a60',
  u'CreationDate': datetime.datetime(2015, 1, 1, 0, 0),
  u'Description': u'mock',
  u'Enabled': True,
  u'KeyId': u'270fc203-b23a-4ccb-8447-be48d7036a60',
  u'KeyUsage': u'ENCRYPT_DECRYPT'},
 'ResponseMetadata': {'HTTPHeaders': {'connection': 'close',
   'content-length': '296',
   'content-type': 'text/plain; charset=utf-8',
   'date': 'Wed, 28 Jun 2017 13:48:45 GMT',
   'server': 'Python/HTTPretty',
   'status': '200'},
  'HTTPStatusCode': 200,
  'RetryAttempts': 0}}

In [8]: cmk_key_arn = response['KeyMetadata']['Arn']
In [9]: client.create_alias(AliasName='alias/54321', TargetKeyId=cmk_key_arn)
Out[9]:
{'ResponseMetadata': {'HTTPHeaders': {'connection': 'close',
   'content-length': '4',
   'content-type': 'text/plain; charset=utf-8',
   'date': 'Wed, 28 Jun 2017 13:50:33 GMT',
   'server': 'Python/HTTPretty',
   'status': '200'},
  'HTTPStatusCode': 200,
  'RetryAttempts': 0}}

In [10]: client.describe_key(KeyId='alias/54321')
---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
<ipython-input-14-e7cdd337c83a> in <module>()
----> 1 client.describe_key(KeyId='alias/54321')

/Users/ryan.hartje/Library/Python/2.7/lib/python/site-packages/botocore/client.pyc in _api_call(self, *args, **kwargs)
    249                     "%s() only accepts keyword arguments." % py_operation_name)
    250             # The "self" in this scope is referring to the BaseClient.
--> 251             return self._make_api_call(operation_name, kwargs)
    252
    253         _api_call.__name__ = str(py_operation_name)

/Users/ryan.hartje/Library/Python/2.7/lib/python/site-packages/botocore/client.pyc in _make_api_call(self, operation_name, api_params)
    535
    536         if http.status_code >= 300:
--> 537             raise ClientError(parsed_response, operation_name)
    538         else:
    539             return parsed_response

ClientError: An error occurred (404) when calling the DescribeKey operation:

I've validated that the alias exists here:
In [15]: client.list_aliases()
Out[15]:
{u'Aliases': [{u'AliasArn': u'arn:aws:kms:us-east-1:012345678912:alias/aws/ebs',
   u'AliasName': u'alias/aws/ebs'},
  {u'AliasArn': u'arn:aws:kms:us-east-1:012345678912:alias/aws/s3',
   u'AliasName': u'alias/aws/s3'},
  {u'AliasArn': u'arn:aws:kms:us-east-1:012345678912:alias/aws/redshift',
   u'AliasName': u'alias/aws/redshift'},
  {u'AliasArn': u'arn:aws:kms:us-east-1:012345678912:alias/aws/rds',
   u'AliasName': u'alias/aws/rds'},
  {u'AliasArn': u'arn:aws:kms:us-east-1:012345678912:alias/54321',
   u'AliasName': u'alias/54321',
   u'TargetKeyId': u'18d9183e-e7db-4005-955f-dda7adaebd62'}],
 'ResponseMetadata': {'HTTPHeaders': {'connection': 'close',
   'content-length': '572',
   'content-type': 'text/plain; charset=utf-8',
   'date': 'Wed, 28 Jun 2017 13:51:56 GMT',
   'server': 'Python/HTTPretty',
   'status': '200'},
  'HTTPStatusCode': 200,
  'RetryAttempts': 0},
 u'Truncated': False}

We should be able to set the KeyId to alias and have it return our blob about our key:
Boto3 Behavior
In [1]: import boto3
In [2]: client = boto3.client('kms', region_name='us-east-1')
In [3]: client.describe_key(KeyId='alias/54321')
Out[3]:
{u'KeyMetadata': {u'AWSAccountId': u'redacted',
  u'Arn': u'arn:aws:kms:us-east-1:u'redacted:key/65fa2df1-834f-476f-9c7a-05dff40d6e52',
  u'CreationDate': datetime.datetime(2017, 5, 25, 13, 16, 10, 753000, tzinfo=tzlocal()),
  u'Description': u'test key',
  u'Enabled': True,
  u'KeyId': u'65fa2df1-834f-476f-9c7a-05dff40d6e52',
  u'KeyState': u'Enabled',
  u'KeyUsage': u'ENCRYPT_DECRYPT',
  u'Origin': u'AWS_KMS'},
 'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',
   'content-length': '335',
   'content-type': 'application/x-amz-json-1.1',
   'date': 'Wed, 28 Jun 2017 14:25:52 GMT',
   'server': 'Server',
   'x-amzn-requestid': '303d7478-002f-4855-b7a0-777e2e650976'},
  'HTTPStatusCode': 200,
  'RequestId': '303d7478-002f-4855-b7a0-777e2e650976',
  'RetryAttempts': 0}}

I see from the source that this function should be able to process the alias, but I didn't immediate see why a 404 would be returned instead:
https://github.com/spulec/moto/blob/5807a38092553360cf7024214453ba5c56ee7a8e/moto/kms/models.py#L81

Thanks for opening!
I'm having trouble reproducing this. I'm on the same version of boto3, botocore, moto. Python 2.7.13 and OSX 10.12. The following code works for me:
import boto3
from moto import mock_kms

mock = mock_kms()
mock.start()

client = boto3.client('kms', region_name='us-east-1')
response = client.create_key(Description="mock", KeyUsage="ENCRYPT_DECRYPT", Origin="AWS_KMS")

cmk_key_arn = response['KeyMetadata']['Arn']
client.create_alias(AliasName='alias/54321', TargetKeyId=cmk_key_arn)
res = client.describe_key(KeyId='alias/54321')
assert res['KeyMetadata']['Arn'] == cmk_key_arn

A very similar thing is running on Travis though that is boto, not boto3. Can you think of anything else that might be different about your environment?

Closing for now, I can't replicate on Ubuntu now, assuming the issue is on my end. Thank you for your help.

authors: RyanHartje spulec

repo: spulec/moto


text:
Fix wrong type if exists

cast_value s are always "S" (string type) value
Because cast_value does not in expression_attribute_values

Coverage decreased (-0.2%) to 92.387% when pulling cf157287e707117447ccb637d8ffbf642eaad240 on uncovertruth:fix/wrong_type_if_exists into dfa7935e13e6065810d6fc4e1edbf3e86499aea5 on spulec:master.

Coverage decreased (-0.2%) to 92.387% when pulling cf157287e707117447ccb637d8ffbf642eaad240 on uncovertruth:fix/wrong_type_if_exists into dfa7935e13e6065810d6fc4e1edbf3e86499aea5 on spulec:master.

Thanks!

authors: Surgo coveralls spulec


text:
Correct group inline policy rendering

GetAccountAuthorizationDetails returns an invalid response in group inline policies.
PolicyDocument should be an IAM policy document JSON, however the current implementation returns a Python dict expression.
Similar implementation can be found just below the code, in role inline policies, which seems correct.
https://github.com/spulec/moto/blob/cb600377b48ca676fc6a29a6690aeb51e702da43/moto/iam/responses.py#L2123
This PR fixes group inline policy rendering just like role inline policy.
How to ensure issue
The small code below is a PoC with a group and a policy.
Backend data are correct: you can see a normal JSON.
#!/usr/bin/env python

from moto import mock_iam
from moto.iam.models import iam_backend
import boto3

@mock_iam
def t():
    policy_document = """
{
  "Version": "2012-10-17",
  "Statement":
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::example_bucket"
    }
}
"""

    iam = boto3.client("iam")
    iam.create_group(GroupName="test-role")
    iam.put_group_policy(
        GroupName="test-role",
        PolicyName="s3",
        PolicyDocument=policy_document,
    )
    ret = iam.get_group_policy(GroupName="test-role", PolicyName="s3")
    d = iam_backend.get_account_authorization_details(["Group"])
    for group in d["groups"]:
        # like in template
        print(f"Group: {group.name}")
        for policy in group.policies:
            print(f"  p: {policy}")
            print(f"  d: {group.get_policy(policy)}")
        # backend data
        print(group.policies["s3"])

t()

Output
Group: test-role
  p: s3
  d: {'policy_name': 's3', 'policy_document': '\n{\n  "Version": "2012-10-17",\n  "Statement":\n    {\n      "Effect": "Allow",\n      "Action": "s3:ListBucket",\n      "Resource": "arn:aws:s3:::example_bucket"\n    }\n}\n', 'group_name': 'test-role'}

{
  "Version": "2012-10-17",
  "Statement":
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::example_bucket"
    }
}

Coverage remained the same at 94.386% when pulling 315063cfe239525fe88d8676e1a6ce292fd6552b on limitusus:bugfix/iam-get-account-authorization-details into 475f022b7839420ffacb940abe6b0214bfa10279 on spulec:master.

@bblommers Sure I'll try that

@bblommers
I added tests for user/role/group inline policies.
While running tests, I found user inline policies are not included in the response, so I fixed that issue as well.
NOTE: UserPolicyList element is omitted if empty, while RolePolicyList and GroupPolicyList is not.

authors: coveralls limitusus


text:
Cognito IdP: Raise UsernameExistsException from sign_up when user exists

Similar to admin_create_user (#2357), the sign_up method in the Cognito identity provider client should raise a UsernameExistsException if a user already exists within the user pool

Codecov Report

Merging #3765 (c86bbfc) into master (f96ac40) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3765      +/-   ##
==========================================
- Coverage   94.80%   94.77%   -0.03%
==========================================
  Files         437      437
  Lines       48116    48118       +2
==========================================
- Hits        45614    45605       -9
- Misses       2502     2513      +11

Flag
Coverage Δ

servertests
56.00% <0.00%> (-0.01%)
:arrow_down:

unittests
94.64% <100.00%> (-0.03%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/cognitoidp/models.py
88.28% <100.00%> (+0.03%)
:arrow_up:

moto/batch/models.py
82.84% <0.00%> (-1.46%)
:arrow_down:

moto/iam/models.py
95.13% <0.00%> (-0.15%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f96ac40...c86bbfc. Read the comment docs.

This is now part of moto >= 2.0.3.dev4

authors: bblommers codecov-io sawyerh


text:
Improved the ELBBackend

This change implements subnet and tag handling for the CreateLoadBalancer call.

when subnets are given, the corresponding VPC is assigned to the ELB
tags can also be set on creation

Before, subnets and tags passed on creation where silently ignored.

Coverage decreased (-2.2%) to 93.244% when pulling 105ba838353b4f8588d46c3188420bf6e48a9625 on dhepper:elb_vpcs into 100ec4e7c8aa3fde87ff6981e2139768816992e4 on spulec:master.

Thanks!

authors: coveralls dhepper spulec


text:
'Decodable' protocol is now ambiguous in Swift 4

Hey,
Simple fix for 'Decodable' is ambiguous for type lookup in this context error caused by new 'Decodable' protocol in Swift 4.
Tested on Swift 4  (Xcode 9 Beta) and Swift 3 (Xcode 8.3.2).
Only tested the core (not RxSwift or ReactiveSwift).
Regards,
Yonic

thanks for the contribution, @saeros!
this will be backwards compatible with Swift 3 too, correct?

Hello @spxrogers,
Yes it's fully backwards compatible with Swift 3.
I've just prefixed Decodable with Gloss.
In Swift 3 it has no impact, but in Swift 4 (iOS 11) it helps to make the difference between Gloss' Decodable protocol and the new Swift 4 Decodable protocol (which was causing errors and preventing compilation in Xcode 9 beta).
Regards,
Yonic

LGTM, thanks @saeros for the update and contribution to help Moya-Gloss get prepped for Swift 4.

you can except a new CP+Carthage release tomorrow evening at the latest. 👍

updating the status on the last comment above about a new release: life happened yesterday and i'm trying to consolidate some release targets right now for all iOS/macOS/tvOS/watchOS. building a prospective RC now, but if i see some Carthage errors i will defer that build work to quickly release the Swift 4 compatibility fix tomorrow and focus on Xcode targets later this week.
cc: @saeros et al.

authors: saeros spxrogers

repo: spxrogers/Moya-Gloss

license: mit


text:
plugin = mod.PLUGIN_CLASS(self) AttributeError: '_ModuleMock' object has no attribute 'PLUGIN_CLASS'

Description
What steps will reproduce the problem?

What is the expected output? What do you see instead?
Please provide any additional information below
plugin = mod.PLUGIN_CLASS(self)

AttributeError: '_ModuleMock' object has no attribute 'PLUGIN_CLASS'
Version and main components

Spyder Version: 3.2.1
Python Version: 3.6.8
Qt Versions: 5.9.7, PyQt5 5.9.2 on Windows

Dependencies
pyflakes >=0.6.0 :  2.1.1 (OK)
pycodestyle >=2.3:  2.5.0 (OK)
pygments >=2.0   :  2.3.1 (OK)
pandas >=0.13.1  :  0.24.2 (OK)
numpy >=1.7      :  1.16.2 (OK)
sphinx >=0.6.6   :  2.0.1 (OK)
rope >=0.9.4     :  0.12.0 (OK)
jedi >=0.9.0     :  0.13.3 (OK)
nbconvert >=4.0  :  5.4.1 (OK)
sympy >=0.7.3    :  1.4 (OK)
cython >=0.21    :  0.29.7 (OK)
qtconsole >=4.2.0:  4.4.3 (OK)
IPython >=4.0    :  7.4.0 (OK)
pylint >=0.25    :  2.3.1 (OK)

This is fixed in our latest Spyder 3 version. Please update by opening the Anaconda Prompt and running there
conda install spyder=3.3.6

repo: spyder-ide/spyder


text:
Debugger panel is not shown for pyw files

Description
What steps will reproduce the problem?

In main file with exension pyw, space for breakpoint near line number is missing.
I still can set brakpints and I can see them on right, but I cannot see the red point.
Versions

Spyder version: 4.1.3
Python version: 3.8.2
Qt version: 5.12.8
PyQt5 version: 5.12.3
Operating System: Windows 10

Dependencies

# Mandatory:
atomicwrites >=1.2.0           :  1.4.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.4.1 (OK)
diff_match_patch >=20181111    :  20181111 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.14.0 (OK)
jedi =0.15.2                   :  0.15.2 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  1.0.0 (OK)
paramiko >=2.4.0               :  2.7.1 (OK)
parso =0.5.2                   :  0.5.2 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.0 (OK)
pygments >=2.0                 :  2.6.1 (OK)
pylint >=0.25                  :  2.5.2 (OK)
pyls >=0.31.9;<0.32.0          :  0.31.10 (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  0.7.2 (OK)
qtconsole >=4.6.0              :  4.7.4 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
sphinx >=0.6.6                 :  3.0.4 (OK)
spyder_kernels >=1.9.1;<1.10.0 :  1.9.1 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.1 (OK)

# Optional:
cython >=0.21                  :  None (OK)
matplotlib >=2.0.0             :  3.2.1 (OK)
numpy >=1.7                    :  1.18.4 (OK)
pandas >=0.13.1                :  None (OK)
scipy >=0.17.0                 :  1.4.1 (OK)
sympy >=0.7.3                  :  None (OK)

Hi @Archi712, thanks for reporting this. I tried to reproduce this issue but I don't see any problem in files with pyw extension. Can you post a screenshot of the problem?

Hi Juanita.
Here I add 2 screenshot of the same file with different extensions.
I restart spyder after I renamed the file.
As you can see in the evidenced part, in pyw file there is no space for breakpoints.
Regards.
Riccardo.
Da: Juanita Gomez notifications@github.com
Inviato: martedì 23 giugno 2020 01:50
A: spyder-ide/spyder spyder@noreply.github.com
Cc: Carassale Riccardo Riccardo.Carassale@ansaldoenergia.com; Mention mention@noreply.github.com
Oggetto: [EXTERNAL] Re: [spyder-ide/spyder] Breakpt and pyw (#13069)
Hi @Archi712https://urldefense.com/v3/__https:/github.com/Archi712__;!!JZsaRFfwqOYHaA!lh9GLZqMxMdUwCJp3tkJNTFyrOI8zF5sgI08qirE2eM5652Uv_-cCPVPXW_r9SG500n5m8BFy6A$, thanks for reporting this. I tried to reproduce this issue but I don't see any problem in files with pyw extension. Can you post a screenshot of the problem?
[Screen Shot 2020-06-22 at 6 48 13 PM]https://urldefense.com/v3/__https:/user-images.githubusercontent.com/18587879/85345754-f07e1b80-b4b8-11ea-8905-edc52e60ee55.png__;!!JZsaRFfwqOYHaA!lh9GLZqMxMdUwCJp3tkJNTFyrOI8zF5sgI08qirE2eM5652Uv_-cCPVPXW_r9SG500n5cnZOAbk$
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/spyder-ide/spyder/issues/13069*issuecomment-647825291__;Iw!!JZsaRFfwqOYHaA!lh9GLZqMxMdUwCJp3tkJNTFyrOI8zF5sgI08qirE2eM5652Uv_-cCPVPXW_r9SG500n5NfxozAg$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AQAN4LP65AXD4I7XJLTIY3LRX7U3TANCNFSM4OC7QQLQ__;!!JZsaRFfwqOYHaA!lh9GLZqMxMdUwCJp3tkJNTFyrOI8zF5sgI08qirE2eM5652Uv_-cCPVPXW_r9SG500n5q4pPSc4$.
P
Prima di stampare questa comunicazione consideratene, per favore, l'impatto ambientale
Please consider the environment before printing this email.

Questo messaggio e i suoi allegati sono confidenziali e destinati ad essere inviati esclusivamente ai destinatari effettivi per gli scopi e gli usi previsti. Qualsiasi uso, distribuzione, riproduzione o divulgazione da e verso qualsiasi persona diversa dagli utenti previsti é severamente vietata. Se hai ricevuto questo messaggio per errore, ti invitiamo a informare immediatamente il destinatario e ad eliminarlo dal tuo sistema. Ansaldo Energia S.p.A. tratta i dati per ﬁnalità connesse alla richieste formulate e al corretto svolgimento del rapporto in essere. Informazioni dettagliate sono riportate nell'informativa privacy sul sito internet della società, www.ansaldoenergia.com, e potranno essere richieste via e-mail a privacy@ansaldoenergia.com, a cui poter indirizzare anche eventuali richieste di accesso.
This message and its attachments are conﬁdential intended to be sent exclusively to the actual recipients for the intended purposes and uses. Any use, distribution, reproduction or disclosure to and from any person other than the intended users is strictly prohibited. If you have received this message by mistake, we invite you to notify the recipient immediately and to delete it from your system. Ansaldo Energia S.p.A. processes the data for purposes related to the request formulated and the correct execution of the existing relationship. Detailed information can be found in the privacy policy on company's website, www.ansaldoenergia.com, and can be requested by e-mail to privacy@ansaldoenergia.com , to which any access requests may also be addressed.

Hi @Archi712, it seems that your screenshots couldn't get uploaded via email, can you upload them here in the issue tracker https://github.com/spyder-ide/spyder/issues/13069.
You can just drag them into a comment box

Here are my screenshot.
Sorry for the problem.

@Archi712, I found the cause of this bug when trying to fix another problem. The fix will be part of our next version (4.1.4), to be released next week.

Thank you.
Da: Carlos Cordoba notifications@github.com
Inviato: giovedì 25 giugno 2020 02:41
A: spyder-ide/spyder spyder@noreply.github.com
Cc: Carassale Riccardo Riccardo.Carassale@ansaldoenergia.com; Mention mention@noreply.github.com
Oggetto: [EXTERNAL] Re: [spyder-ide/spyder] Debugger panel is not shown for pyw files (#13069)
@Archi712https://urldefense.com/v3/__https:/github.com/Archi712__;!!JZsaRFfwqOYHaA!lxsjG-sqEknn8DZGScSjxkqOV410MkKZ41h7-jOoyCF7_BUuWl6YkEcOBtc3-XmauUj8AEIqeUc$, I found the cause of this bug when trying to fix another problem. The fix will be part of our next version (4.1.4), to be released next week.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/spyder-ide/spyder/issues/13069*issuecomment-649146801__;Iw!!JZsaRFfwqOYHaA!lxsjG-sqEknn8DZGScSjxkqOV410MkKZ41h7-jOoyCF7_BUuWl6YkEcOBtc3-XmauUj8lT_MXPI$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AQAN4LNS5UK4IR5YEGF4BK3RYKMILANCNFSM4OC7QQLQ__;!!JZsaRFfwqOYHaA!lxsjG-sqEknn8DZGScSjxkqOV410MkKZ41h7-jOoyCF7_BUuWl6YkEcOBtc3-XmauUj880es60Y$.
P
Prima di stampare questa comunicazione consideratene, per favore, l'impatto ambientale
Please consider the environment before printing this email.

Questo messaggio e i suoi allegati sono confidenziali e destinati ad essere inviati esclusivamente ai destinatari effettivi per gli scopi e gli usi previsti. Qualsiasi uso, distribuzione, riproduzione o divulgazione da e verso qualsiasi persona diversa dagli utenti previsti é severamente vietata. Se hai ricevuto questo messaggio per errore, ti invitiamo a informare immediatamente il destinatario e ad eliminarlo dal tuo sistema. Ansaldo Energia S.p.A. tratta i dati per ﬁnalità connesse alla richieste formulate e al corretto svolgimento del rapporto in essere. Informazioni dettagliate sono riportate nell'informativa privacy sul sito internet della società, www.ansaldoenergia.com, e potranno essere richieste via e-mail a privacy@ansaldoenergia.com, a cui poter indirizzare anche eventuali richieste di accesso.
This message and its attachments are conﬁdential intended to be sent exclusively to the actual recipients for the intended purposes and uses. Any use, distribution, reproduction or disclosure to and from any person other than the intended users is strictly prohibited. If you have received this message by mistake, we invite you to notify the recipient immediately and to delete it from your system. Ansaldo Energia S.p.A. processes the data for purposes related to the request formulated and the correct execution of the existing relationship. Detailed information can be found in the privacy policy on company's website, www.ansaldoenergia.com, and can be requested by e-mail to privacy@ansaldoenergia.com , to which any access requests may also be addressed.


text:
AttributeError: 'SpyderKernelManager' object has no attribute 'kernel'

<!---   *** BEFORE SUBMITTING: PASTE CLIPBOARD HERE TO COMPLETE YOUR REPORT ***   ---!>

Hi @DoRmAmMu1997 could you give us more information regarding your issue (OS, Python version, Spyder version, dependencies version, steps to reproduce the issue, etc)? Without more information there is not much we can do to help you, sorry :/

This is a problem with the latest jupyter-client (i.e. version 7.0 or higher). For now, please downgrade to jupyter_client 6.1.12.
This will be fixed in our next version (5.1.2), to be released shortly.


text:
Running code hangs in stand alone 5.1.5 MacOS 12.0.1

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[ ] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[ ] Could not reproduce inside jupyter qtconsole (if console-related)
[ ] Tried basic troubleshooting (if a bug/error)

[x] Restarted Spyder
[ ] Reset preferences with spyder --reset
[x] Reinstalled the latest version of Anaconda
[ ] Tried the other applicable steps from the Troubleshooting Guide

[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
Hello world hangs in my console window after I installed the stand alone version 5.1.5 on a MacBook Pro running MacOS 12.0.1.
In [1]: runfile('/Users/duane/hello.py', wdir='/Users/duane')
What steps reproduce the problem?

Tried keeping and deleting opt/anaconda.  Still hangs.
Downloaded spyder.dmg and installed in applications folder.
launch spyder
hit the run-flle green arrow
In [1]: runfile('/Users/duane/hello.py', wdir='/Users/duane')
No hello world

@author: duane
"""
"""Print "Hello World" and return None."""
def hello():
print("Hello World")
Main program starts here
hello()
What is the expected output? What do you see instead?
Should have seen 'hello world', but it just hangs
Paste Traceback/Error Below (if applicable)

PASTE TRACEBACK HERE

Versions

Spyder version: 5.1.5
Python version: 3.9.5 64 bit
Qt version: 5.12.10
PyQt version: 5.12.3
Operating System name/version: MacOS 12.0.1

Dependencies

# Mandatory:
applaunchservices >=0.1.7     :  0.2.1 (OK)
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0               :  7.27.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.0 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.2.1 (OK)
nbconvert >=4.0               :  6.1.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
parso >=0.7.0;<0.9.0          :  0.8.2 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.10.0 (OK)
pylint >=2.5.0;<2.10.0        :  2.9.6 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.2.2;<1.3.0          :  1.2.2 (OK)
pylsp_black >=1.0.0           :  None (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.0.3 (OK)
qtconsole >=5.1.0             :  5.1.1 (OK)
qtpy >=1.5.0                  :  1.11.1 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  58.0.4 (OK)
sphinx >=0.6.6                :  4.2.0 (OK)
spyder_kernels >=2.1.1;<2.2.0 :  2.1.1 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.5 (OK)
zmq >=17                      :  22.2.1 (OK)

# Optional:
cython >=0.21                 :  0.29.24 (OK)
matplotlib >=2.0.0            :  3.4.3 (OK)
numpy >=1.7                   :  1.21.2 (OK)
pandas >=1.1.1                :  1.3.3 (OK)
scipy >=0.17.0                :  1.7.1 (OK)
sympy >=0.7.3                 :  1.8 (OK)```

Hey @tduane3579, thanks for reporting. Other users have reported our Mac app is running code successfully for them in 12 (see issue #16793).
@mrclary, whenever you have the chance to update and test this, could check if we have problems in Mac 12? Thanks!

@tduane3579, did you set a different graphics backend in Preferences > IPython Console > Graphics? That could be the cause too.

Hi Carlos:
Spyder 4.2.5 works fine from Anaconda Navigator.  Here's my IPython > Graphics from it:
Here's my IPython > graphics from 5.1.5 stand alone.  It's the same:
Thanks,Duane
On Wednesday, November 10, 2021, 06:54:54 PM PST, Carlos Cordoba ***@***.***> wrote:

@tduane3579, did you set a different graphics backend in Preferences > IPython Console > Graphics? That could be the cause too.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

I also get this complier error message from 5.1.5:
Thanks,Duane
On Wednesday, November 10, 2021, 07:31:07 PM PST, Duane Takahashi ***@***.***> wrote:

Hi Carlos:
Spyder 4.2.5 works fine from Anaconda Navigator.  Here's my IPython > Graphics from it:
Here's my IPython > graphics from 5.1.5 stand alone.  It's the same:
Thanks,Duane
On Wednesday, November 10, 2021, 06:54:54 PM PST, Carlos Cordoba ***@***.***> wrote:

@tduane3579, did you set a different graphics backend in Preferences > IPython Console > Graphics? That could be the cause too.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

@tduane3579 , I don't see the compiler error message; did you forget to paste into the comment?
Can you verify the status of the Python environment? Preferences -> Python interpreter. The "Default" or "Use the following Python interpreter:". If you're not using the default, make sure that the Python environment exists and is working properly by testing in a terminal.
@ccordoba12, I'm currently on vacation and will take a look at it next week.

Ok, no prob. Sorry for the constant pings, but don't worry about them. You can answer them when you come back from vacations.

Sorry.  Here's the compiler error message:

I'm using the default interpreter:
:

Hmm. I cannot reproduce the OP. I've got the same specs as the @tduane3579, although I'm on an Apple M1 chip.

@ccordoba12, as an aside, I think the evidence presented by @tduane3579 contradicts my assertion that ipython=7.27.0 would (temporarily) resolve #16828. I've seen this on local builds, and on recent GitHub artifacts, and thought it was revealed with ipython=7.28.0, but the above screen shot from @tduane3579 proves that is not the case. This indicates that #16828 may be machine dependent (yikes!).
@tduane3579, do you have Xcode installed on your system? Xcode command-line tools? Are you on the new Apple M1 chip?

I think the evidence presented by @tduane3579 contradicts my assertion that ipython=7.27.0 would (temporarily) resolve #16828

You're right, that's bad news.

@mrclary I don't have Xcode installed, but I can.  I don't have the M1:
Model Name:	MacBook Pro
Model Identifier:	MacBookPro15,1
Processor Name:	6-Core Intel Core i7
Processor Speed:	2.6 GHz
Number of Processors:	1
Total Number of Cores:	6
L2 Cache (per Core):	256 KB
L3 Cache:	9 MB
Hyper-Threading Technology:	Enabled
Memory:	32 GB

@tduane3579, is the following exactly what is in your hello.py file?
def hello():
    print('Hello world!')

hello()

Can you open the Internal console (View->Panes->Internal console), then run hello.py and tell me if anything is reported in the Internal console? I'm hoping there is an error message.
If that is not fruitful, please restart Spyder in debug mode, run the file, and attach the log file.

File->Restart in debug mode
In the pop-up window, click "Verbose" button
After Spyder restarts, run your file
Quit Spyder
Attach the file ~/.spyder-py3/spyder-debug.log

Yes, that was the code I was trying to run.
I tried View > Panes, but I didn't see that option.  Sorry, I don't know how to run python under Xcode.  I tried this:
Connecting Python 3.8 with Xcode 11 | Apple Developer Forums
|
|
|
|  |  |
|
|
|
|  |
Connecting Python 3.8 with Xcode 11 | Apple Developer Forums
|
|
|
Thanks,Duane
On Tuesday, November 16, 2021, 07:05:09 AM PST, Ryan Clary ***@***.***> wrote:

@tduane3579, is the following exactly what is in your hello.py file?
def hello():
print('Hello world!')
hello()
Can you open the Internal console (View->Panes->Internal console), then run hello.py and tell me if anything is reported in the Internal console? I'm hoping there is an error message.
If that is not fruitful, please restart Spyder in debug mode, run the file, and attach the log file.

File->Restart in debug mode
In the pop-up window, click "Verbose" button
After Spyder restarts, run your file
Quit Spyder
Attach the file ~/.spyder-py3/spyder-debug.log

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You should not run code under Xcode. I was just wondering if you had it installed, but my idea about it is now obsolete.
The View->Panes->Internal console should be in Spyder's menubar.

@tduane3579, could you please post your comments directly on Github's website? Posting via email reply seems to garble things. I have no idea what this is supposed to be:

Connecting Python 3.8 with Xcode 11 | Apple Developer Forums
|
|
|
|  |  |
|
|
|
|  |
Connecting Python 3.8 with Xcode 11 | Apple Developer Forums
|
|
|

OK.  hello.py runs on the internal console:

run hello.py
Hello World

I started Spyder in verbose debug mode and got this missing dependency message:
You have missing dependencies!
Mandatory:
rtree >=0.9.7 : None (NOK)
Please install them to avoid this message.
Note: Spyder could work without some of these dependencies, however to have a smooth experience when using Spyder we strongly recommend you to install all the listed missing dependencies.
Failing to install these dependencies might result in bugs. Please be sure that any found bugs are not the direct result of missing dependencies, prior to reporting a new issue.

You should not run the code in the internal console. Only run the code in the IPython console. The internal console may display an error (a stderr output) if your code does not run properly in IPython console.

@tduane3579 , is this still a problem for you? Have you tried installing the latest Spyder 5.2.2?

It works!
Thank you.  You can close this.
Regards,Duane
On Sunday, March 20, 2022, 08:38:47 PM PDT, Ryan Clary ***@***.***> wrote:

@tduane3579 , is this still a problem for you? Have you tried installing the latest Spyder 5.2.2?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you were mentioned.Message ID: @.***>

@tduane3579, thanks for responding. I'm glad it's working for you now. If this issue resurfaces, please feel free to reopen this issue or create a new one. Happy coding!


text:
Python seems to be incorrectly compiled for macOS installer

Problem Description
The macOS standalone application reports the following warning in the IPython console when the Python interpreter is set to internal (same as Spyder):
This version of python seems to be incorrectly compiled
(internal generated filenames are not absolute).
This may make the debugger miss breakpoints.
Related bug: http://bugs.python.org/issue1666807

What steps reproduce the problem?

Launch macOS standalone Spyder application with Python interpreter set to internal

Versions

Spyder version: 5.2.0.dev0 None
Python version: 3.9.5 64-bit
Qt version: 5.12.10
PyQt5 version: 5.12.3
Operating System: Darwin 20.6.0

Dependencies

# Mandatory:
applaunchservices >=0.1.7     :  0.2.1 (OK)
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0               :  7.29.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.0 (OK)
jellyfish >=0.7               :  0.8.9 (OK)
jsonschema >=3.2.0            :  4.2.1 (OK)
keyring >=17.0.0              :  23.2.1 (OK)
nbconvert >=4.0               :  6.3.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
parso >=0.7.0;<0.9.0          :  0.8.2 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.10.0 (OK)
pylint >=2.5.0                :  2.9.6 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.2.2;<1.3.0          :  1.3.0.dev0 (OK)
pylsp_black >=1.0.0           :  None (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.0 (OK)
qtconsole >=5.2.0;<5.3.0      :  5.2.0 (OK)
qtpy >=1.5.0                  :  1.11.2 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  58.5.3 (OK)
sphinx >=0.6.6                :  4.3.0 (OK)
spyder_kernels >=2.1.1;<2.2.0 :  2.2.0.dev0 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.6 (OK)
zmq >=17                      :  22.3.0 (OK)

# Optional:
cython >=0.21                 :  None (OK)
matplotlib >=2.0.0            :  None (OK)
numpy >=1.7                   :  None (OK)
pandas >=1.1.1                :  None (OK)
scipy >=0.17.0                :  None (OK)
sympy >=0.7.3                 :  None (OK)

Just a quick note: I wonder if this could be a problem with the latest Py2app (in case we're using it).

The provided link, http://bugs.python.org/issue1666807, indicates an issue with absolute paths in pyc files that was fixed over 10 years ago (https://bugs.python.org/issue1180193). Nevertheless, the same symptoms manifest in the IPython console when using the internal Python interpreter.
In [1]: import inspect

In [2]: inspect.getabsfile(inspect)
Out[2]: '/Users/rclary/Applications/spyder-dev/Spyder.app/Contents/Resources/lib/python39.zip/inspect.py'

In [3]:inspect.getabsfile(inspect.iscode)
Out[3]: '/Users/rclary/inspect.py'

In [4]: inspect.__file__
Out[4]: '/Users/rclary/Applications/spyder-dev/Spyder.app/Contents/Resources/lib/python39.zip/inspect.pyc'

In [5]: import jedi

In [6]: jedi.__file__
Out[6]: '/Users/rclary/Applications/spyder-dev/Spyder.app/Contents/Resources/lib/python3.9/jedi/__init__.py'

In [7]: inspect.getabsfile(jedi)
Out[7]: '/Users/rclary/Applications/spyder-dev/Spyder.app/Contents/Resources/lib/python3.9/jedi/__init__.py'

In [8]: inspect.getabsfile(jedi.get_system_environment)
Out[8]: '/Users/rclary/Applications/spyder-dev/Spyder.app/Contents/Resources/lib/python3.9/jedi/api/environment.py'

Above you can see that importing the module from the zip folder inside the application resource path (a pyc file) results in inspect using paths relative to the current working directory rather than relative to the module path. This is not the case when the module is loaded from a python source file (py file).
I don't know if this issue is malignant or benign. I'm not aware of any adverse behavior in our macOS standalone applications when using the internal interpreter for the IPython console; so it would seem, so far, that the only real issue is the warning message displayed on IPython console startup. This issue does not manifest when using an external interpreter.
This warning message does not appear in our 5.1.5 release, but does appear in the artifact for PR #16322. As of this investigation, this was the oldest merged PR for which artifacts still remained on the server (GitHub only keeps them for 30 days).

This warning message does not appear in our 5.1.5 release

That's why I thought that maybe this could be related to a newer version of Py2app (or perhaps one of its dependencies).

but does appear in the artifact for PR #16322

I really doubt that PR introduced this problem.

Two additional questions:

If everything seems to be working with the internal interpreter, perhaps we should patch Python to avoid showing that warning?
Do you have time to do that (or keep investigating) this week? Our plan is to release 5.2.0 next Monday.

This warning message does not appear in our 5.1.5 release

That's why I thought that maybe this could be related to a newer version of Py2app (or perhaps one of its dependencies).

but does appear in the artifact for PR #16322

I really doubt that PR introduced this problem.

I agree that our source code did not introduce this problem; that was merely the first artifact available on the server for which the issue manifested. I think it was revealed with IPython 7.28.0.
We are still using py2app==0.22, so this is not the issue. There are several updates to py2app but it has introduced a Qt plugin issue that prohibits us from updating (ronaldoussoren/py2app#380). In my local patches for py2app's latest commits, this warning message in the IPython console does not appear, although the symptoms exhibited by inpsect shown above still manifest. This local build uses ipython==7.29.0 and jupyter-core==4.9.1.

Two additional questions:
* If everything seems to be working with the internal interpreter, perhaps we should patch Python to avoid showing that warning?

* Do you have time to do that (or keep investigating) this week? Our plan is to release 5.2.0 next Monday.

It appears that the warning message began manifesting upon ipython>=7.28.0, and limiting it to 7.27.0 eliminates the warning. I recommend the following:

For now, limit ipython<7.28.0 for just the macOS installer.
This constraint will be removed upon updating py2app>=0.27
It still remains to be determined whether this is merely covering up the issue and whether this issue is malignant or benign. I don't think we can know until a concrete behavioral issue is reported by a Spyder user.

Ok, thanks a lot for your help with this one @mrclary!

Per conversations in #16795 and #14025, reopening this issue.

Update
I've discovered the following:

The warning message originates in the module debugpy._vendored.pydevd.pydevd_file_utils.py at the module level.
The message is a pydev_log.critical logger message conditional on whether os.path.realpath.__code__.co_filename is an absolute path.

os.path is loaded from Spyder.app/Contents/Resources/lib/python39.zip/posixpath.pyc, but os.path.realpath.__code__.co_filename is just posixpath.pyc which is not an absolute path, resulting in pydev_log printing its critical message.
Conclusions
First, while I don't know exactly why the critical message is displayed in the IPython console in some circumstances and not others, I believe that #14025 robustly revealed the issue by improving the stdout/err capture. I have tested whether os.path.realpath.__code__.co_filename is an absolute path for several of our releases and local builds under various versions, etc; they all indicate that it is not an absolute path and therefore trigger the pydev_log.critical message. I believe the only explanations for the different IPython console results are: (a) some idiosyncrasy with pydev_log (log level?); or (b) some idiosyncrasy with how Spyder captures and/or displays these messages. Perhaps it is (b) and #14025 removes the idiosyncracy, thus ensuring consistent capture of these log messages. The evidence presented in #16795 supports this conclusion.
Second, while anything imported from modules in Spyder.app/Contents/Resources/lib/python39.zip/ will likely have the same absolute path status, I think this is benign. The macOS Spyder application has been running, so clearly these packages (and their modules, submodules, functions, etc.) have been found, so no problem there. The message originates from debugpy and indicates that breakpoints may not be found by the debugger. Since anything imported from Spyder.app/Contents/Resources/lib/python39.zip/ will be byte-compiled code, it is impossible for the user to set breakpoints in these codes, so the warning should not apply to us. Note that this issue should not affect breakpoint discovery in anything outside the zip folder, including a user's source code.
I'll propose a solution in a forthcoming comment.

Perhaps it is (b) and #14025 removes the idiosyncracy, thus ensuring consistent capture of these log messages. The evidence presented in #16795 supports this conclusion.

Yeah, I think that's definitely the case.

I'll propose a solution in a forthcoming comment.

Thanks @mrclary!

As I see it, we have two objectives:

Ensure that this particular pydev_log message does not cause the installer test to fail.
Hide this particular pydev_log message from users since it is benign.

I do not see any way to ensure that os.path.realpath.__code__.co_filename is an absolute path, so I believe the only option is to filter it on our end. Note that after merging #14025, pydev_log message is displayed in the IPython console, but only briefly since the kernel did not actually fail to start. The following gif illustrates this behavior, and the still shot is provided for clarity.

@impact27, for the general case, should these types of logging messages from the stderr file be cleared from the IPython console like that?

@mrclay these messages are printed to the console as they come. It's just that qtconsole clears the console when the kernel "restarts". I'll see it there is something to do.

See https://github.com/spyder-ide/spyder/pull/16850