text:
Workload Identity with KEDA

Report
No response
Expected Behavior
We have KEDA setup on several clusters and are trying to use workload identity with it.  We also have other tools setup on these same clusters that use workload identity just fine.  One of our clusters is working just fine with KEDA and workload identity, but all the others are not. We install KEDA exactly the same way on all clusters.   If I install my KEDA test setup to another cluster we get the error message below in both the scaledobject events and in the logs for the keda-operator.
Does anybody have any Idea how to debug this?
I have gone through the managed identities comparing everything I can find between the two setups and I can find no difference and the Federated credentials all match.
Actual Behavior
KEDA is throwing ERRORs in the log and ScaledObject.  KEDA is not scaling up the workload.
Steps to Reproduce the Problem
Unknown how to have someone else reproduce since I do not even understand the problem in the first place.  Everything has been checked by multiple people on the team and no one can see any issues.
Logs from KEDA operator
(combined from similar events): ChainedTokenCredential authentication failed POST https://login.microsoftonline.com/<TENANT ID>/oauth2/v2.0/token -------------------------------------------------------------------------------- RESPONSE 401 Unauthorized -------------------------------------------------------------------------------- { "error": "unauthorized_client", "error_description": "AADSTS70021: No matching federated identity record found for presented assertion. Assertion Issuer: '<OIDC-URL>'. Assertion Subject: 'system:serviceaccount:keda:keda-operator'. Assertion Audience: 'api://AzureADTokenExchange'. https://docs.microsoft.com/en-us/azure/active-directory/develop/workload-identity-federation\r\nTrace ID: 2c9b6265-8984-4d03-b502-cabf8aab5d01\r\nCorrelation ID: 64bd287f-80ee-45ea-b6ab-087abb31ab53\r\nTimestamp: 2023-05-31 21:43:25Z", "error_codes": [ 70021 ], "timestamp": "2023-05-31 21:43:25Z", "trace_id": "2c9b6265-8984-4d03-b502-cabf8aab5d01", "correlation_id": "64bd287f-80ee-45ea-b6ab-087abb31ab53", "error_uri": "https://login.microsoftonline.com/error?code=70021" } --------------------------------------------------------------------------------

KEDA Version
2.10.1
Kubernetes Version
1.25
Platform
Microsoft Azure
Scaler Details
Azure Service Bus
Anything else?
No response

Hi
It looks like the pod hasn't been mutated with the needed environment variables. If you see AZURE_AUTHORITY_HOST, AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE as environment variables, the problem could be inside KEDA.
If you don't see those variables, your should review if the pod is correctly labeled and&or the service account is correctly labeled and annotated.
If both are well, restart KEDA and check the pods envs again.
Workload identity federation works as a mutating webhook, adding the needed info during the pod scheduling if the service account is correctly annotated and labeled and also the pod is labeled.

Hi,
It looks like a wrong configuration. Is the managed identity federated with the correct OIDC url? Is the KEDA service account annotated with the correct clientId which has been federated with that cluster OIDC url?
Based on the error, KEDA is trying to use the provided managed identity, but the provided clientId doesn't have the federation registered. That could mean some things, but I'd check typos/misconfigurations:

subject in the federation section in azure
oidc url in the federation section in azure
clientId in the service account

So the configuration is correct.  I pulled the oidc url out of the error message on purpose and everything looks correct.  I have had several people double check it.  The keda-operator has the 4 workload identity env variables injected into it.  The managed identity does have the federated credentials in it.
I can use the same managed identity with a different federated credential in another application and it works just fine.

Picture of the Federated Credential from the managed Identity

Could you have any login restriction policy at AAD level? For example: at work, we have some restrictions for limiting MSI/ServicePrincipal logins from some locations

I mean, if the subject, the oidc, the clientId and the audience are correct, the login restrictions are the only things left that I can image

@tomkerkhove , do you know who could help us here?

I mean, if the subject, the oidc, the clientId and the audience are correct, the login restrictions are the only things left that I can image

So I asked the person that manages that for my company and set most of our subscriptions up and he said there are no restrictions setup for MSI or SPs.
We have also submitted an Azure Help Desk ticket since our company has full MS Support but we have not heard back yet.
I just got done setting up a Test case where I have KeyVault CSI SecretProviderClass and Keda using the same Managed Identity and both using the same serviceaccount and thus the same federated credentials.  The KeyVault CSI works just fine and KEDA is getting the above error message.  So I know the cluster is successfully pulling the OIDC for that Federated Credential.
In my POD that is from the Deployment with the ScaledObject I see a constant stream of error messages not from my code.  I am assuming they are from the KEDA scaledobject since it matches the event in there.  But it does not provide me any more info than above.

In my POD that is from the Deployment with the ScaledObject I see a constant stream of error messages not from my code. I am assuming they are from the KEDA scaledobject since it matches the event in there. But it does not provide me any more info than above.

What info would you like to have? I mean, we will release the next version in a few weeks and any improvement is welcomed. Would you be willing to contribute with this extra info?

Yeah I am not sure what info I would like...  This type of problem is a really hard problem to solve.
I will tell you that I have now isolated the issue to the managed identity we are using associated with the KEDA install.  Today I was able to hand create a Manage Identity in the same subscription that the Cluster is running in and it allowed KEDA to work properly.  The one I was trying to use was generated in our Bicep File that installs the cluster and lots of other resources.   I now need to start digging through the Managed Identity to figure out why the one I hand created worked and the one Bicep generated does not.  As far as I can tell right now they both have the Same exact RBAC roles setup and they have the same federated credentials setup.

Would you think that printing things like the used clientId could be useful? As KEDA supports to use other clientIds than the default (just providing them in the TriggerAuthentication), maybe it could help 🤔
Apart from that, I'm not totally sure about what info we have in KEDA side that could help during the troubleshooting, because we are already printing the whole error

Yeah, I agree with the extra complexity of the chained credential. I'd like to add Okteto for improving development experience and also to improving troubleshooting but until that, the only way that I have found to debug scalers with identities is that chained credential :/

Hi @kstedman9
Have you solved the issue? if yes would you like to share what was happening?

@JorTurFer   We are still waiting on Microsoft Tech Support to figure it out.  On one of the clusters, I was able to just recreate the Managed Identity using the same exact bicep to create it and it worked the second time.  On a different cluster I have tried recreating the managed identity several times and it still has not fixed it on it.   I will update here when I finally get any resolution from them.

It'd be nice!
As KEDA maintainer I'm interested, but as KEDA user who is integrating managed identities and workload identity everywhere, I'm really interested xD

So we finally figured it out.  The workloads managed identity they are using must have the federated credentials for KEDA Service Account in it along with the workload required federated credentials.  Not sure why this is the case.   I just happen to have it on all my test cases but when it came to using it on the production clusters I did not do that.

Oh, I thought that you already federated them. This is mandatory because the working way of workload identity federation is based on federating k8s OIDC with Azure AD, this means that you have to register in Azure the cluster OIDC to trust on it, but also you have to identify the service account because it's part of the token from k8s (it's the subject).
KEDA can just access to its own service account token (with its subject), if you don't add KEDA's service account to the managed identity, KEDA can't login as it because AAD doesn't trust on KEDA.
Is it now clearer?

Actually no.  Because of chaining of identities this should not be required correct?  The managed identity associated with the keda service account has the federated credential in it.  Why would I then need to add that same federated credential to every workloads managed identity also?

KEDA service account has to be federated with all the identities that KEDA has to use. During the "login" process in AAD, KEDA uses its own k8s token, that is generated for KEDA's service account. It's not enough just federating the workloads, because KEDA doesn't have access to those workloads tokens (they are mounted directly into those pods).
If you are using a single identity for KEDA, and it's that identity who has the accesses, you just need to federate KEDA with that, but if you are using different managed identities (overriding the default value in the TriggerAuthentication) to reuse workloads' managed identities for KEDA, KEDA service account must be federated there too.
It depends on how you are managing the identities and the accesses, but KEDA must be federated with any managed identity that is used during KEDA operation.
Let's put some examples
Case 1
You have an identity for KEDA, with access to ServiceBus A, ServiceBus B and ServiceBus C, also you have other identities for the workloads, so you have:

KEDA identity with access to ServiceBus A, ServiceBus B and ServiceBus C (identity set during installation and not overrided)
Workload A identity with access to Service Bus A
Workload B identity with access to Service Bus B
Workload C identity with access to Service Bus C

In this case, KEDA has to be federated only with KEDA msi.
Case 2
To avoid stacking too many permissions on KEDA identity, you have an identity for KEDA with any access (not related, for example, to the Key Vault), also you have other identities for the workloads, so you have:

KEDA identity without access to any Service Bus
Workload A identity with access to Service Bus A
Workload B identity with access to Service Bus B
Workload C identity with access to Service Bus C

In this case, you are overriding the default identity set during the installation thanks to the option in TriggerAuthentication (.spec.podIdentity.identityId), so each ScaledObject uses its own TriggerAuthentication, and each TriggerAuthentication specifies an override (the TriggerAuthentication for the workload A, sets the idntityId of the workload A, the B for the B, etc...).
Thanks to this, you don't need to stack a lot of accesses on KEDA identity, but in this case, KEDA has to be federated with all the identities that KEDA will try to assume.

TriggerAuthentication without overrides will use KEDA identity (for accessing the Key Vault for example)
TriggerAuthentications with overrides will use the identity set as part of TriggerAuthentication (so KEDA's service account must be federated on them).

Is it clearer now? I guess that your case is the case 2, am I right?

Yes Case 2 is what we are using. Yes I understand more now.  Is there anyway this can be documented better in the KEDA docs?

KEDA service account has to be federated with all the identities that KEDA has to use. During the "login" process in AAD, KEDA uses its own k8s token, that is generated for KEDA's service account. It's not enough just federating the workloads, because KEDA doesn't have access to those workloads tokens (they are mounted directly into those pods).
If you are using a single identity for KEDA, and it's that identity who has the accesses, you just need to federate KEDA with that, but if you are using different managed identities (overriding the default value in the TriggerAuthentication) to reuse workloads' managed identities for KEDA, KEDA service account must be federated there too.
It depends on how you are managing the identities and the accesses, but KEDA must be federated with any managed identity that is used during KEDA operation.
Let's put some examples
Case 1
You have an identity for KEDA, with access to ServiceBus A, ServiceBus B and ServiceBus C, also you have other identities for the workloads, so you have:
* KEDA identity with access to ServiceBus A, ServiceBus B and ServiceBus C (identity set during installation and not overrided)

* Workload A identity with access to Service Bus A

* Workload B identity with access to Service Bus B

* Workload C identity with access to Service Bus C

In this case, KEDA has to be federated only with KEDA msi.
Case 2
To avoid stacking too many permissions on KEDA identity, you have an identity for KEDA with any access (not related, for example, to the Key Vault), also you have other identities for the workloads, so you have:
* KEDA identity without access to any Service Bus

* Workload A identity with access to Service Bus A

* Workload B identity with access to Service Bus B

* Workload C identity with access to Service Bus C

In this case, you are overriding the default identity set during the installation thanks to the option in TriggerAuthentication (.spec.podIdentity.identityId), so each ScaledObject uses its own TriggerAuthentication, and each TriggerAuthentication specifies an override (the TriggerAuthentication for the workload A, sets the idntityId of the workload A, the B for the B, etc...). Thanks to this, you don't need to stack a lot of accesses on KEDA identity, but in this case, KEDA has to be federated with all the identities that KEDA will try to assume.
* TriggerAuthentication without overrides will use KEDA identity (for accessing the Key Vault for example)

* TriggerAuthentications with overrides will use the identity set as part of TriggerAuthentication (so KEDA's service account must be federated on them).

Is it clearer now? I guess that your case is the case 2, am I right?

@JorTurFer would be nice to add this to documentation.

We could add this the the docs an example, but honestly, I'd not do it because it's how the workload identity works itself, if MSFT changes something, we'll be updated. This isn't related with KEDA but any workload in general, but as I said, we can added a reference to this comment or directly copy and paste this into docs.

Gotcha. Can we at least link relevant Msft MSFT?

Yeah, we can add the whole example from my comment too. I mean, it's clear the concept without entering in implementation details, but this is strongly attached to how Azure Workload Identity works, for example, the same idea in AWS side, works in a totally different way

Yeah, but I mean the more docs the better :) So I think that adding this with a note that it works with Azure Workload Identity isn't going to do any harm.

https://github.com/kedacore/keda-docs/pull/1247

Can this be reopened?
I'm trying to understand a bit more about how this works with workload identity, especially when using the AKS add-on that's now GA
Am I safe to assume that for this to work as you're describing (using this scenario for example), we still need to install KEDA via the helm chart - as we then have an identity to use for role assignments (as noted in the above service bus scenario)?
I think something's just not computing with me on how the role assignments should be configured

You have to assign the role, via UI, az-cli, or however you prefer: https://github.com/kedacore/sample-dotnet-worker-servicebus-queue/blob/main/workload-identity.md#creating-a-new-azure-service-bus-namespace--queue

❯ az role assignment create --role 'Azure Service Bus Data Receiver' --assignee  --scope /subscriptions//resourceGroups//providers/Microsoft.ServiceBus/namespaces/

authors: JorTurFer kstedman9 thepaulmacca zroubalik


text:
why keda doesn't scale up extra pods after trigger for a long time.

Report
Here is my ScaledObject yaml config:
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  namespace: maicedata
  name: scaled-auth
spec:
  scaleTargetRef:
    name:  auth         # Mandatory. Must be in the same namespace as the ScaledObject
  pollingInterval:  5                            # Optional. Default: 30 seconds
  cooldownPeriod:   360                              # Optional. Default: 300 seconds
  idleReplicaCount: 0                                # Optional. Default: ignored, must be less than minReplicaCount
  minReplicaCount:  1                                # Optional. Default: 0
  maxReplicaCount:  10                              # Optional. Default: 100
  fallback:                                          # Optional. Section to specify fallback options
    failureThreshold: 3                              # Mandatory if fallback section is included
    replicas: 2                                      # Mandatory if fallback section is included
  advanced:                                          # Optional. Section to specify advanced options
    restoreToOriginalReplicaCount: false        # Optional. Default: false
    horizontalPodAutoscalerConfig:                   # Optional. Section to specify HPA related options
      behavior:                                      # Optional. Use to modify HPA's scaling behavior
        scaleDown:
          stabilizationWindowSeconds: 300
          policies:
          - type: Percent
            value: 100
            periodSeconds: 15
        # 减少扩容抖动
        scaleUp:
          stabilizationWindowSeconds: 15
  triggers:
    - type: prometheus
      metricType: Value
      metadata:
        serverAddress: https://prom.idatatlas.com
        namespace: maicedata
        metricName: auth-prom-metric
        activationThreshold: "-1"
        threshold: '0.8'
        # worker使用率
        query: sum(gunicorn_busy_workers{env="prod", exported_instance="k8s", role="auth"})/sum(gunicorn_total_workers{env="prod", exported_instance="k8s", role="auth"})
        authModes: "basic"
      authenticationRef:
        name: keda-prom-creds

The upper graph represents the gunicorn worker utilization.
The bottom graph represents the current pod's number

When the query metric value is greater than threshold, It scales up one pod, but the query metric value lasts for a long time that is greater than threshold, keda doesn't scale up extra pods. Why???? I hope that keda continues to scale up extra pods when the query metric value is greater than threshold for 10s or a customized time
Expected Behavior
I hope that keda continues to scale up extra pods when the query metric value is greater than threshold for 10s or a customized time
Actual Behavior
keda doesn't continue to scale up pods.
Steps to Reproduce the Problem

Logs from KEDA operator
example

KEDA Version
None
Kubernetes Version
None
Platform
None
Scaler Details
No response
Anything else?
No response

Hello
What KEDA version are you using?
Could you share KEDA operator logs?

keda:2.9.1
operator has no special logs.

Hello What KEDA version are you using? Could you share KEDA operator logs?

keda:2.9.1
operator has no special logs.

something in KEDA metrics server logs?
Are you scrapping prometheus metrics exposed by KEDA? Could you shared the chart for keda_scaler_metrics_value during the same period?

the chart is

I scrap the metric that is exposed by my Prometheus on-premises. The yellow line is the threshold.
  triggers:
    - type: prometheus
      metricType: Value
      metadata:
        serverAddress: https://xx.xxx.xxx

I specific the URL in the triggers config zone.

No no, I meant if you scrape the metrics that KEDA exposes about its own information: https://keda.sh/docs/2.9/operate/prometheus/

I find the metric doesn't suit the metricType Value. It works using metricType averageValue, and change the promQL from sum(gunicorn_busy_workers{env="prod", exported_instance="k8s", role="auth"})/sum(gunicorn_total_workers{env="prod", exported_instance="k8s", role="auth"}) to sum( gunicorn_busy_workers{env="prod", exported_instance="k8s", role="map-maker"} / gunicorn_total_workers{env="prod", exported_instance="k8s", role="map-maker"} )

source: gharchive/issue

repo: kedacore/keda

license_type: permissive

license_source: github-api


text:
Migrate e2e test for MySQL Scaler to Go

Added Golang based e2e test for MySQL Scaler
Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)
[x] Tests have been added

Fixes #3243

/run-e2e mysql

source: gharchive/pull-request

authors: JorTurFer yokawasa


text:
kedro-telemetry failing to send telemetry when project UUID not found

Description
Short description of the problem here.
Context
How has this bug affected you? What were you trying to accomplish?
Steps to Reproduce
$ mkdir kedro-telemetry-test && cd kedro-telemetry-test
$ uv init
$ git init && git add . && git commit -m 'First commit'
$ cat pyproject.toml
[project]
name = "kedro-telemetry-test"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = []

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
$ uv add kedro

Expected Result
I didn't expect a successful Kedro run, because this reproducer doesn't create a valid Kedro project, but at least I expected an error an that's it.
Actual Result
Tell us what happens instead.
$ uv run kedro registry list
[08/23/24 08:53:47] INFO     Using                                                                     __init__.py:249
                             '/private/tmp/kedro-telemetry-test/.venv/lib/python3.12/site-packages/ked
                             ro/framework/project/rich_logging.yml' as logging configuration.
[08/23/24 08:53:49] ERROR    Failed to retrieve UUID: badly formed hexadecimal UUID string                plugin.py:81
Usage: kedro [OPTIONS] COMMAND [ARGS]...
Try 'kedro -h' for help.

Error: No such command 'registry'.
                    INFO     Kedro is sending anonymous usage data with the sole purpose of improving    plugin.py:233
                             the product. No personal data or IP addresses are stored on our side. If
                             you want to opt out, set the `KEDRO_DISABLE_TELEMETRY` or `DO_NOT_TRACK`
                             environment variables, or create a `.telemetry` file in the current working
                             directory with the contents `consent: false`. Read more at
                             https://docs.kedro.org/en/stable/configuration/telemetry.html
                    WARNING  Failed to send data to Heap. Response code returned: 400, Response reason:  plugin.py:336
                             Bad Request

Kedro project not found in this directory. Project specific commands such as 'run' or 'jupyter' are only available within a project directory.

Hint: Kedro is looking for a file called 'pyproject.toml, is one present in your current working directory?

This issue is about finding out why the Heap request fails. Several things not specifically related to it:

Having the INFO and WARNING messages after the error make it more difficult to spot.

Maybe the INFO should be on top?
And, if the request to Heap ever fails, I'm not sure the user cares. Maybe we should make it DEBUG

It was surprising to read "is there a pyproject.toml in your cwd?" message when there was indeed one. Maybe we could tweak the hint so it says "is there a pyproject.toml with a [tool.kedro] table there?"

repo: kedro-org/kedro-plugins


text:
fix(telemetry): remove double execution by moving to after catalog created hook

Description
This PR fixes #382 by moving the sending of the heap to the after_catalog_created hook instead of the after_context_created hook. This prevents the creation of the catalog twice and thus double execution of after_catalog_created hooks as part of any downstream project.
Checklist

[x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
[x] Updated the documentation to reflect the code changes
[x] Added a description of this change in the relevant RELEASE.md file
[x] Added tests to cover my changes

Thanks a lot @fdroessler! Could you please signoff your commit so the DCO check passes? If you click on it you will find instructions. If it was only one commit you can do git commit --amend --signoff --no-edit, if you had more, git rebase --signoff. After rewriting the history you will need to do git push --force.
And if you keep working on this branch, please use the -s/--signoff flag when committing, for example git commit -s -m 'Add feature'.

authors: astrojuanlu fdroessler


text:
Chemlambda, graph theory etc

Hello Guy, ive been doing lot of expetimentation lately on parsers, grammar and graph theory, and started to dig the relationship between graph theory and computation theory.
Deep down, all those algorithm of parser combinators, ll/lr parser etc seem to always come down to a tree representation, and i started to dig the relationship between graph theory and computation model.
Just yesterday i found this article
https://blog.reverberate.org/2013/07/ll-and-lr-parsing-demystified.html

There are three ways of traversing a binary tree, as explained on Wikipedia: in-order, pre-order, and post-order. They differ in whether you visit the parent node before (pre-order), after (post-order), or in between (in-order) the children of that parent. These three correspond exactly to infix, Polish, and Reverse Polish notation:

1 + 2 * 3 // Infix expression; in-order traversal.

1 * 2 3 // Polish (prefix) expression; pre-order traversal.
1 2 3 * + // Reverse Polish (postfix) expression; post-order traversal.

So Polish and Reverse Polish notation fully encode a tree structure and the steps you would take to traverse it. The main difference between these encodings and an actual parse tree is that Polish and Reverse Polish encodings are not random access. With a real tree you can choose to follow an interior node to its right subtree, its left subtree, or even (for many trees) its parent. With these linear encodings there is no such flexibility: you have to follow the traversal as it is already encoded

Thats an example of how tree theory and the way to process a tree has also implication into computation model.
While digging this,i found this this thing called chemlambda, who seems To be a model of computation based on graph theory, with graph rewrite operations that allow to modify the graph, which allow a way of formal self modifying code.
Apparently chemlambda is inspired from this model
http://tuvalu.santafe.edu/~walter/AlChemy/Statement/organization.html
Which i find very inspiring, and i found it has implication in both complexity theory, and distributed systems.
Its extremely interesting To me, especially this part

A central theme of this project is our emphasis on the action of "agents" as consisting in a mapping from "agents" to "agents". The foundational distinction between this project and others sharing similar emphasis is that none of the others have the feature of being interpretable as mathematical structures. To bootstrap a theory of biological organization, requires a formalization of chemistry that is useful for biology. Ultimately this means defining "interaction" and "structure of an entity" mutually. In our case "structure of an entity" will specifically come to mean a formula, expression, or term within a syntactical system resulting from a theory about the possible interactions characterizing that class of entities. To be illuminating, a linkage between structure and interaction must connect to a mathematical theory.

Stable relational structures are what conventional dynamical systems take as a given. If we aim at a description of their genesis, we must extend our description of the world to include a sound and sufficiently general dynamics of relationships capable of generating stable relational structures as stationary outcomes. What enables any such dynamics from within, we submit, is at the very minimum a notion of entities that lead this subtle double-life: entities that simultaneously act as relationships between entities (by being interpretable as functions on them).

Which remove the barrier between "code" and "data" in sort that you can have functions that take a function as an input and output another function, which allow for this kind of algebra needed for self organising structures. It seem to be a solution for parallelism, decentralised distributed system, and self modifying code, which look like an holy grail to me :)
I started to experiment with this and exploring this relationship between graph theory and computation model, but cant seem to find lot of information on this. It seems to be a given that a parser can output parse tree, and that those parse tree can be turned to turing complete system but i cant really find real deep formal theory on this.
I have this hunch that for example sticking to what can be represented as a DAG without backward references prevent loop and infinite recursion.
As i use a blockchain system to store the tree, with nodes referenced by their hash, it naturally prevent cycles. I just added some instruction to have equivalent of map working on lists, and recursive operations on data tree like to compute skeletal animation based on quaternion, and it seems to works well.
With a lr parsing style it even seem to allow very clean handling of asynchronous function, as all the computation that has To be done on the result is on the stack when the asynchronous function is called, so it can easily be added to the handler code and executed asynchronously when the function is done, and all the dependancy graph is explicit.
In theory using this sort of algebra can also allow runtime polymorphism, as if you can have graph operation working on the computation graph, it can allow to generate monomorphised function based on the polymorphic version, while still keeping in the realm of formal system.
In theory it can be done at compile time, load time or runtime, and i think can even elucidate the interface problem as function can be checked for their type with graph operations. And in theory graph rewrite operation like in chemlambda seem to be a good way to represent polymorphism.
And in theory it can accomodate with self modifying code, and all kind of dynamic system modeling, even if it seems to escape common computation rules.
But i cant seem to find lot of formal definition of how tree properties translate to computation model, even if it seem to me computation model can always be mapped to a tree, and its very easy to have a system of formal operations working on graph, which i think can be a way to have runtime polymorphism.
Its a paradigm that got stuck in my mind and it seems i cant really nail this sort of relationship or find formal systems describing this.

hello :)
I found out as well it has a connection with physics, even if that was not what i was seeking in the first place.
I could talk to a good physicist who told it looked close to prigogine thermodynamics
https://en.wikipedia.org/wiki/Ilya_Prigogine
As i'm also quite into edelman and whiteheads it would not suprise me that there are connections in physics. I would just answer this How is Natural Science Possible?: Whitehead's 1st Lecture at Harvard (1924) :)
I couldn't find a direct connection between wolfram model and this one, but it's close in the principle, it's likely one inspired the other or they have same implication with physics.
https://arxiv.org/pdf/1811.04960.pdf

The references and links in the text are left as they were and much improvement could be made concerning references pointing to Interaction Nets [2], for example; Berry and Boudol CHAM [1] is not explicitely mentioned here, but the initial version of this formalism, i.e. ”chemical concrete machine”[13], uses it. The Algorithmic Chemistry of Fontana and Buss [3] [4] appears here via a link to their research, perhaps this link is better. The same happens with the work [5] by Christoph Flamm, for which, in the body of the article, a link to his papers page is provided. Part of this was due to the ignorance of the author at the moment (a mathematician). A later hope was to keep the article as is and to link it with critical public reviews, as a weaker form of validation than, for example, an alternative Haskell implementation of chemlambda [5]. However, the main idea of the article is original, as are the self-imposed constraints, namely to restrict to only local rewrites on graphs, regardless their interpretation as lambda terms (or lack of such interpretation), restrict to only two algorithms of reduction (deterministic or random), the most simple possible.

This sort of model is more relevant for dealing with living organism than with inanimate objects like usual physics.
But it was not my original intent to get into physics.
Found this article that make a short summary of different way to achive polymorphism :
https://dev.to/jmcclell/inheritance-vs-generics-vs-typeclasses-in-scala-20op

Subtype Polymorphism (Inheritance)
subtyping allows a function to be written to take an object of a certain type T, but also work correctly, if passed an object that belongs to a type S that is a subtype of T (according to the Liskov substitution principle) [2]

Parametric Polymorphism (Generics)
Parametric polymorphism allows a function or a data type to be written generically, so that it can handle values uniformly without depending on their type [3]

Ad-hoc Polymorphism (Type Classes)
the term ad hoc polymorphism [refers] to polymorphic functions that can be applied to arguments of different types, but that behave differently depending on the type of the argument to which they are applied [4]

I will skip the first one based on subtyping because it still depend on static code, and some invariant among a category of different datatypes.
The 2 others are more relevant to where i want to get at, which is that you have a computation defined in a form that allow it to be combined with a type to yeld a version of the computation unique to that type.
So they both encode this notion of 'variable functions', but if i want to make a comparison on classic lambda function level, it's like if instead of computing the result from a value, you would pick an output value in a set of precomputed values based on the input. Much like you would use a table to compute a sinus if the cpu doesn't have sinus. The idea of dispatch table with a switch case is similiar to this principle. You have a set of predefined functions that are selected based on the input type. Whereas some computation rules applied to a graph could achieve the same thing. And it can also deal very well with asynchronous / simultaneous operations (even all of them are not executed in parrallel on the hardware).
My goal for the moment is not to have the whole range of operation that you would have in a real molecular computer, or to get to a physic model.
More to have some computation that can modify graph to alter the computation being done that can still fit into a formal model.

For those interested, i found the abstraction i was looking for in stepanov book 'elements of programming', he calls this 'coordinated structure' and gets into all the points i'm searching, in case someone end up with the same questions :)
I'm still digesting the two chapters but at least i feel less in uncharted territory :)

A concept is a coordinate structure if it consists of one or more coordinate
types, zero or more value types, one or more traversal functions, and zero or more
access functions. Each traversal function maps one or more coordinate types and/or
value types into a coordinate type, whereas each access function maps one or more
coordinate types and/or value types into a value type. For example, when considered
as a coordinate structure, a readable indexed iterator has one value type and two
coordinate types: the iterator type and its distance type. The traversal functions
are + (adding a distance to an iterator) and — (giving the distance between two
iterators). There is one access function: source.

Fun thing when i look for coordinated structure in google i find mostly result from chemistry =)

@NodixBlockchain I think Stepanov's "Elements of Programming" is one of the best books on Generics I have read. I recommend reading it from start to finish several times. It is not an easy book, but once you get the idea of what is going on, the early chapters make a lot more sense.
I also recommend reading "From Mathematics to Generic Programming" by Stepanov & Rose as it presents the same material in a different way, which can help with understanding, plus some really interesting history of mathematics and programming.

Yes there are all the algorithm to find isomorphism between "coordinated structures" and all those, couldn't find much reference to this term in the context of programming outside of stepanov book though.
The other book is also on my list, found some video on youtube as well.

@NodixBlockchain Try "Data Structues and Network Algorithms" by Robert Endre Tarjan, which is a book on algorithms recommended by Stepanov.

I just found this issue, thank you for interest. There is now a new page of experiments and technical notes on chemlambda: https://chemlambda.github.io/index.html
which I hope can answer many questions. There is a lot of information which one can play with, spread over several repositories.
For more please let me know, I'd be happy to discuss more, because the project started from a question in geometry and it now returns to it, but there are many directions which I would like to develop, like the "made of money" idea from the repository hapax. or which I would like a real programmer to magically do it, like a graphical horoscope app which would allow chemlambda molecules to live and circulate between many phones :)

https://youtu.be/2RMXujk5AhM?t=1543
Gerald Edelman - Brain Dynamics to Consciousness (Full Lecture)
the kind of theory i'm into :)

I just found this issue, thank you for interest. There is now a new page of experiments and technical notes on chemlambda: https://chemlambda.github.io/index.html
which I hope can answer many questions. There is a lot of information which one can play with, spread over several repositories.
For more please let me know, I'd be happy to discuss more, because the project started from a question in geometry and it now returns to it, but there are many directions which I would like to develop, like the "made of money" idea from the repository hapax. or which I would like a real programmer to magically do it, like a graphical horoscope app which would allow chemlambda molecules to live and circulate between many phones :)

originally i found this project looking into relationship between graph and computation model, looked for things like 'graph theory + lambda calculus' and found this project :)
what i would have in mind is really some kind of computation model with self modifying code that would follow a formal computation theory, and i think going through graph theory is really the way to go, because chemlambda show how graph manipulation can lead to a whole computation model, that can be equivalent to lambda calculs, but also add more possibilities :)
i'm still digging the whole thing :)

Oh thank you. I'm  more into applied biochemistry. Many links are available from the page of chemlambda, but maybe relevant for the situation we are in now is this short story https://archive.is/Ir9GG .
I would search in databases of biochemical reactions for patterns akin to those I use (and everybody else does, in so various fields of math and computation).
There is also the problem that graph rewrite systems (GRS) and term rewrite systems (TRS) are very different beasts. There is no clear way to define an isomorphism between a GRS and a TRS. Indeed, you can turn a term of a TRS into it's abstract syntactic tree, which is a graph. But you can't always turn a term rewrite into a graph rewrite on the tree graph, in such a way as to preserve the locality of the rewrite.
In the opposite direction, it is sometimes possible to define a function which decorates the edges of a graph with terms, such that if the graph is the abstract syntactic tree of a term then the root edge of the graph is decorated with that term. However, if you pass to graph rewrites, even if the graph rewrite is local (like the graphical version of the beta rewrite say), the decoration of the graph changes globally.
So I think that  TRS and GRS are not comparable without the introduction of external hypotheses, which may destroy the meaning of the seeked isomorphism.
An implication is that GRS-es with only local rewrites (like in nature) are not only a unnecessary graphical decoration of things which can be explained with TRS only (like lambda calculus), they are intrinsically interesting outside their (undefined properly) isomorphism with TRS-es.
But you'll say that any GRS for which there is a program is equivalent with the TRS (the program). Agree! but the TRS will consist mainly of a generator of randomness and a destroyer of the supplimentary structure (like random shuffles of lists), all run on a machine whose code can't be controlled. On the other side a GRS like real chemistry uses real space and probability distributions which are function of the state, so again is not at all clear in what sense the two systems are really comparable.
If though there is a low hanging fruit, is is that maybe in real chemistry we can identify a GRS which works (not perfectly but) well enough so we can use ideas from functional programming to get real chemical effects. And damned be the search for perfect semantics.
----- Original Message -----
From: "NodixBlockchain" notifications@github.com
To: "keean/zenscript" zenscript@noreply.github.com
Cc: "mbuliga" Marius.Buliga@imar.ro, "Comment" comment@noreply.github.com
Sent: Tuesday, August 25, 2020 1:17:50 PM
Subject: Re: [keean/zenscript] Chemlambda, graph theory etc (#48)
https://youtu.be/2RMXujk5AhM?t=1543
Gerald Edelman - Brain Dynamics to Consciousness (Full Lecture)
the kind of theory i'm into :)
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/keean/zenscript/issues/48#issuecomment-679937649

Thanks for the input :) I can see from your answer it doesn't seem there is a very clear formal system to get from GRS to TRS or vice versa.
Weirdly enough, edelman doesn't even really get into neurochemistry at all, he get his whole brain theory based on a mixture of philosophy, darwinism and computation model.
In theory it sort of lead to a category / type system, he made a robot that he called darwin, following this sort of principles

He make a statement like programs make mistake, mistake are a full part of darwinist process, but they don't crash from mistake, they learn from mistake.
I already started to get into computational model close to lambda calculus, based on blockchain which use a graph system, it's more limited than full lambda calculs/turing machine, but i'm pretty sure it can get close, even if i'm quite in the dark regarding the computation model it entails.
Already from the stepanov book i get that you need at least a tree/graph parsing order algorithm additionally to the graph itself, because the way it's processed change the result. He give all the algorithm to formalise how two 'coordinated structures' can be tested for isomorphism.
But it's already pretty cool, because the graph can be represented very easily graphically, so it allow to completely edit the computation visually automatically from the graph, and then the code to execute the graph looks very much like a parser combinator in the end.
It's not completely operational yet, i already have skeletal animation with a 3D scene represented using a graph like this. Using this system i think i'm already pretty close to have a full match between modification in the graph and term/functional programming representation.
But it's very experimental, i'm not sure where it's going to lead to, but so far i get pretty interesting result from it.

As long as your GRS is restricted to tree traphs, it is indeed equivalent with a TRS, where you translate terms to their AST. But what if the GRS does act on a class of graph larger than trees? Then the equivalence breaks (in interesting ways).
Here is, related to the video of Edelman lecture, the part about neural groups, an experiment with two "neurons".

available also in the collection here https://chemlambda.github.io/collection.html#159
These graphs are encodings of two linear systems with some inputs and outputs connected. Imagine something like having two lambda terms, say T1 and T2, and you mix their inputs (free variables) and outputs (values of the terms) in the sense that you feed some outputs into some inputs. Then you reduce this mix graphically.

This is what electronics design tools like Symplectic do. The problem is
that designs like this are very hard to reason about and verify. When I
worked on VLSI designs we used to write a prototype in 'C' as it was much
easier to validate, and then compare the data from the electronic system
with the prototype for validation. The system I worked on was for MPEG
encoding, so you would apply the same configuration settings to both
systems and then compare the inputs and outputs bit for bit to validate the
design.
The point I am making is that designing free input/output systems is much
harder than writing sequential code. So a language like this will not get
used much, because not many people will be smart enough to use it, and
those that do will spend a lot of time validating designs.
On Wed, 26 Aug 2020, 09:33 mbuliga, notifications@github.com wrote:

As long as your GRS is restricted to tree traphs, it is indeed equivalent
with a TRS, where you translate terms to their AST. But what if the GRS
does act on a class of graph larger than trees? Then the equivalence breaks
(in interesting ways).
Here is, related to the video of Edelman lecture, the part about neural
groups, an experiment with two "neurons".
[image: 2neurons]
https://user-images.githubusercontent.com/39514942/91279759-95092c00-e78e-11ea-8d41-32bf778d8740.gif
available also in the collection here
https://chemlambda.github.io/collection.html#159
These graphs are encodings of two linear systems with some inputs and
outputs connected. Imagine something like having two lambda terms, say T1
and T2, and you mix their inputs (free variables) and outputs (values of
the terms) in the sense that you feed some outputs into some inputs. Then
you reduce this mix graphically.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/keean/zenscript/issues/48#issuecomment-680740471, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAD4DYA5LC6Y3VQLNURQJBDSCTCFRANCNFSM4N6EKMMA
.

This is what electronics design tools like Symplectic do. The problem is that designs like this are very hard to reason about and verify. When I worked on VLSI designs we used to write a prototype in 'C' as it was much easier to validate, and then compare the data from the electronic system with the prototype for validation. The system I worked on was for MPEG encoding, so you would apply the same configuration settings to both systems and then compare the inputs and outputs bit for bit to validate the design. The point I am making is that designing free input/output systems is much harder than writing sequential code. So a language like this will not get used much, because not many people will be smart enough to use it, and those that do will spend a lot of time validating designs.
…
On Wed, 26 Aug 2020, 09:33 mbuliga, @.***> wrote: As long as your GRS is restricted to tree traphs, it is indeed equivalent with a TRS, where you translate terms to their AST. But what if the GRS does act on a class of graph larger than trees? Then the equivalence breaks (in interesting ways). Here is, related to the video of Edelman lecture, the part about neural groups, an experiment with two "neurons". [image: 2neurons] https://user-images.githubusercontent.com/39514942/91279759-95092c00-e78e-11ea-8d41-32bf778d8740.gif available also in the collection here https://chemlambda.github.io/collection.html#159 These graphs are encodings of two linear systems with some inputs and outputs connected. Imagine something like having two lambda terms, say T1 and T2, and you mix their inputs (free variables) and outputs (values of the terms) in the sense that you feed some outputs into some inputs. Then you reduce this mix graphically. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#48 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4DYA5LC6Y3VQLNURQJBDSCTCFRANCNFSM4N6EKMMA .

yes ! it's not easy to write code like this, but the idea is that you don't build the graph by writing code, because the good thing about graph is that they can be very easily represented visually, and this sort of system works pretty well.
I had the chance to meet eric wenger, who is the author of meta synth/voyager etc for example, and this whole system works with graph that are edited visually and plugged into each others.

I already applied this principle like to build audio synth, dynamic web page, and 3D scenes, i think the mind still can cop very well with 'recursive graph' system, when they are presented visually.
For it to be meaningul it needs some amount of prebuilt basic functional/logic brick that can be semantically meaningful to begin with, as well as their composition/combination.
The meta synth system from eric wenger show it's possible to have this sort of functions that can work on audio/video/3d (depth  field rendering with some algebra), that can be combined together. Didn't dig the software too much, but he showed me some quick demo of the system and the possibilities are amazing :)
Well that's mostly step 1, already if i manage to get there it would already be a thing :)
But then the step 2 to go in the edelman kind of direction is that you don't actually write the whole tree, but it's more 'goal oriented', and the 'system' will work by trial and error to find the graph solution to the problem.
In the video there is also this slide

And i think that's sort of the problem is that you can't really 'proove' too much why and how the solution work, or even if it's the best possible solution to the problem.
It would be similar to neural network, except neural network are complete black box, and they don't allow to extract meaningfull generalisation or categories by themselves, and often it require some pre step to prepare the data for the neural network to works well.

I agree, these GRS are not for humans, they are for machines or for understanding nature (biochemistry in this case).
On the other hand, at first there were only (few) mathematicians playing with bits in the emerging computer architectures.
----- Original Message -----
From: "Keean Schupke" notifications@github.com
The point I am making is that designing free input/output systems is much
harder than writing sequential code. So a language like this will not get
used much, because not many people will be smart enough to use it, and
those that do will spend a lot of time validating designs.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/keean/zenscript/issues/48#issuecomment-680764900

When i look at the definition of complex system, i think it touch on the point i'm looking into.
Like this definition of a complex system as in it cannot be explained only looking at separate elements on their own.
I'm not a chemistry buff, but i think i'm not wrong if i say you can't really account for the properties of complex molecular structure, at the most complex level let say living organism, only by looking at the set of atoms that compose it.
In term of stepanov element of programming, just looking at the set of atom would correspond to iterators, that are like a flat set of elements that cannot capture the particular organisation and how atoms are connected to form more complex molecules of which proparties not only  depend on the set of atoms composing it, but also how those atoms are connected together.
Coordinate structure can capture this idea of molecular organisation, and stepanov consider those coordinate structure as a form of iterator or set, but they can capture more complex configuration between the elements of the set.
But then it would need a form of type reflection to be able to capture the properties of the said coordinate structure, because it's the type of the object/structure tree that will capture the information of how each elements is related to the other, and i can completely get kean points on the flaw of type reflection approach.
That said, inside of a compiler, it still have to hold 'type variables' in order to build the AST, so a compiler would be able to capture this information about how each elements of a set relate to another in a dynamic system, but once compiled this informations is lost.
It's why i started to look into GRS as a possible way to be able to analyze those relationship.
In a way there is already a form of this in direct show, as the graph builder is able to automatically connect a file input to an output in real time, but it has 2 layers above the c++ typing, there is a layer of COM for the base module interface encapsulation, and it adds also a 'media type' for each 'pin', which allow the graph builder to connect a file input to an output with all the combination of codecs/filters/image-audio format converters etc, depending on what is installed or not on the system, and can add new codec/filter/converters without having to recompile anything, which allow to build graph that are specific to the particular combination of input source/graphic-audio device and codecs present on the system that are not known at compile time.
It's this kind of things that would not be easy to do with languages like haskel. I don't see a really good way to do this with any language i know of.
Direct show is still a basic example, the system of eric wenger is more developped, i'm not sure if it could still be able to model 'complex systems', i think it's still a bit limited on this regard, but it already give an infinite number of combination possibility with a modular functional composition approach.
With an approach based on typed module interface, and runtime/loadtime graph building mechanism, i think it could still give more opportunity to have graph analysis that would allow to capture the feature of 'complex system' defined as the whole exhibit properties that cannot be captured only by analysis the set elements that compose it.
I'm not sure if the way i put it is extremely clear though :) but it's something that is bugging me or obsessing me for while and i can't seem to find the good silver bullet system to explain this.
Chemlambda seems to be close, especially in the lense of the alchemy of fontanna and buss that i found on chemlambda related sites.

I'm not sure if the way i put it is extremely clear though :) but it's something that is bugging me or obsessing me for while and i can't seem to find the good silver bullet system to explain this.

Maybe process mining? https://en.wikipedia.org/wiki/Process_mining

I'm not a chemistry buff, but i think i'm not wrong if i say you can't really account for the properties of complex molecular structure, at the most complex level let say living organism, only by looking at the set of atoms that compose it.

I think the contrary (with some caveats). After all there exist universal Turing machines, we all use them, we can even build them. The tower of software we put on them is for our human convenience, to be able to easily program or interact with them as humans, not because we need all the huge pile for universal computation. Likewise, a cell is nothing more than a very special pattern of chemical reactions. When we explain (what we understand about) a cell, we invent, for our human understading purpose, all sort of names and analogies, but the cell functions well without the tower of semantic we inflict to it.

Chemlambda seems to be close, especially in the lense of the alchemy of fontanna and buss that i found on chemlambda related sites.

ALCHEMY is a very strange beast. Historically is very important, but otherwise is a system which disregards exactly what's the most interesting: computation.  Indeed, in ALCHEMY is used a soup (multiset of) lambda terms in_normal_form and there is a template for an infinity of possible chemical reactions:
A + B = NormalForm(A B)
Therefore the most interesting part from the point of view of computation, namely NormalForm(), is just ignored.
Later Fontana used a pi-calculus, which is another fashionable way (along with category theory) to ignore how computation really happens. It's magical. Who cares? they say, because they are interested in the meaning of the computation, not in the computation itself. I believe that computation, concretely, is all that happens, semantics attached or not. But how it happens in nature?
I'm not even that strange, there is an old fight here: TOC vs TOP
https://pron.github.io/posts/what-we-talk-about-when-we-talk-about-computation
I am TOC not TOP :)
Differently from ALCHEMY, chemlambda uses graphs, some of them can be obtained with a parser graph = lambda2mol(term), but in general it does not matter, and purely local graph rewrites (i.e. they can be executed by local machines, like enzymes) as chemical reactions. The computation is central here.
What is left, and you may consider this a critique, is how the graphs are built, before they engage in random chemical reactions.

Who cares? they say, because they are interested in the meaning of the computation, not in the computation itself. I believe that computation, concretely, is all that happens, semantics attached or not. But how it happens in nature?

A quick tangential answer, but somehow that can be also a valid concern, i remember a while back i was argueing with a free mason around pythagora, trying to make a sort of point like this, that computation is all what matter and he answered me something like computation like pythagora is a solution to an old problem that is as old as mankind and not a thing of its own.
I think it's also a bit the point that edelman makes all along, and even rieman i think made that sort of argument, that even computation or any kind of theorical science is maybe more a solution to a problem of the mind than really something reality needs to function.
It's more the question of how useful those concept are to solve humans problems that matter rather than how 'real' it is, i think it's a bit related to the degeneracy thing edelman talks about as well.
Maybe at neurological level, not two persons will have the same neural pattern that is used to represent the 'computation' or whatever is going on.
Very interesting answer otherwise, i'm going to dig all this more :)

For quite a while, I’ve been disturbed by the emphasis on language in computer science… Thinking is not the ability to manipulate language; it’s the ability to manipulate concepts. Computer science should be about concepts, not languages. … State machines… provide a uniform way to describe computation with simple mathematics. The obsession with language is a strong obstacle to any attempt at unifying different parts of computer science.

That seems like something stepanov would agree with :)

Computer scientists collectively suffer from what I call the Whorfian syndrome — the confusion of language with reality…Many of these formalisms are said to be mathematical, having words like algebra and calculus in their names. … Despite what those who suffer from the Whorfian syndrome may believe, calling something mathematical does not confer upon it the power and simplicity of ordinary mathematics.

good point as well :)
Well originally i started to get into this because i started to develop some micro kernel architecture, and i needed something to smash different modules/drivers together, compiled indifferently from windows/visual studio, or gcc, or any other compiler, and add new modules/drivers without having to recompile the whole tree. Like with the pci bus modules are matched with vendor/device ID, and that's all. Even using precompiled NDIS modules for network device or such.
But then i noticed the same structure works for filesystems, for windows system, for lot of things actually, even at the core web applications like say joomla or wordpress can be similar in that way of adding modules and configuring the system to load them together.
With also this possibility to log/debug the particular configuration used at some point when lot of loading and configuration is made more or less automatically, or can become relatively complex without being directly specified in a program.
But then i pushed it further to have decentralised/distributed computation.
Thanks for the answer, lot of food of thoughts :)

I'm not even that strange, there is an old fight here: TOC vs TOP
https://pron.github.io/posts/what-we-talk-about-when-we-talk-about-computation
I am TOC not TOP :)

The conclusion i reached is if you want to stick to a pure computation model, there are 2 things you are going to miss, which are streams, open/read/write/close are not really computation, and have not really much equivalent in mathematics, natural science or physics, and interupts in that sense of having procedures that can be executed at random moments, such as input devices for UI, or other hardware devices that work asynchronously with the main cpu. Can even add threads to this if targetting multi core, but if avoiding shared mutable states, i guess it can also come down to an interupt paradigm.
There can be way to get around those, but to me it looks all the ways that 'programming langages' departs from a pure computation models are either to handle streams or interupts. Those are not described in lambda calculs or computation models, and it's where all the clumpsyness comes from in most languages.

@NodixBlockchain The computational model for streams is called "codata". Haskell deals with this very nicely - lazy computation and codata go together very well, just like eager computation goes with data.
The problem is, like with most pure computational models, efficiency is difficult to achieve, and hard for programmers to understand.
One nice solution that keeps everything eager is 'coroutines'.  Coroutines can be modelled at a high level, see JavaScripts 'generator functuons' and Go's 'goroutines'.
Things like open/read/write can be modelled at the type system level by something I would call a 'protocol'.
Fundamentally the difference between 'logic' (mathematics) and computation, is that mathematics has no 'time'. A proof is either true, false or unprovable, there is no before and after. In computation 2 + 2 is not 4 until the expression is reduced, so there is a before "2 + 2" and and after "4". This is what Turing contributed to mathematics.
So a pure representation of something must be "true for all time" and declarative. We represent the state machine in the type system: the type of a new file is Unopened, after opening its Open, and after closing its Closed. The transitions are triggered by functions (messages) but we want to tie these to the File type.
data Unopened
data Open
data Closed
class File a
instance File Unopened
instance File Open
instance File Closed
open :: Unopened -> Open
read :: Open -> (Open, Buffer)
write :: Open -> Buffer -> Open
close :: Open -> Closed

I am not sure this is the best we can do in Haskell, but it has the problem that the functions are not really tied to the typeclasses. We could imagine writing a protocol like this:
protocol File (Unopened | Open | Closed) where
   open :: File Unopened -> File Open
   read :: File Open -> (Buffer, File Open)
   write :: File Open -> Buffer -> File Open
   close :: File Open -> File Closed

Which is sort of a cross between a type-class and a Generalised Abstract Data Type (GADT).

Am i right if i say codata and corecursion will work mostly if there is a sort base case that will be the same for all values generated ? Like if the stream is essentially an iterator ?
Because the main problems is that streams are not necessarily defined following a type definition, and there are all the serialisation format, like ASN, protobuf or BNF, that can become even a grammar of their own, or streams can just be random junk.
Even beside this, when dealing for example with sockets, the behavior of read/write will depend on the socket protocol, or the operating system, and there can be many way to handle a situation. To me it still seems it lacks the kind of concept like stepanov would put it to make the compiler able to check if the behavior of the socket manipulation algorithm is 'correct', or if the algorithm actually implement the desired behavior.
Even more so if you get asynchronous stream answers.
Even with goroutine if the channel is actually a socket, there are many way to handle a block, or a read/write return value, many flags/options that can be used alongside with sockets that will change the behavior and how the algorithm should handle their return value and what there will be in the buffer.

To get a bit more in depth, the problem as i see it is almost all file format or network protocol will have first an header that will contain a least the type of the data and its length, then the data itself.
Which mean you will have at least two successive read operation that depend on each other.
Eventually you'll have a layer of compression or encryption on top of it, or things like chunked transfer (Each chunk is preceded by its size in bytes. The transmission ends when a zero-length chunk is received. )
Eventually there is the system like in avi/mp4/ogm where there are "virtual streams" multiplexed inside a physical stream.
The idea would be alongside being able to define stream layout, and a processing algorithm, with something that allow to control the progress, let say in byte.
And then being able to use the stream definition as an input for computation.
I'm pretty sure with a system like this, the open/close points could be implicit and determined by the compiler, alongside with some error management, integrated into a 'concept' for the whole stream processing algorithm, in sort that the whole algorithm can be compile-time checked.

@NodixBlockchain

There can be way to get around those, but to me it looks all the ways that 'programming langages' departs from a pure computation models are either to handle streams or interupts. Those are not described in lambda calculs or computation models, and it's where all the clumpsyness comes from in most languages.

If I understand well, in the background of this discussion is the hope that we (somebody) will make some day a truly decentralized computer. Which has I/O and it is a true computer in the sense that it can do anything (computable).
The discussion is then that somehow an attempt like this may lead or led to some block, where there is no solution or there are solutions which turn a decentralized system into a centralized one, in time or in space. The discussion then turns to details and after a while the participants or the readers are lost.
This comment is to make a (perhaps well known) statement that a decentralized computer based on a term rewrite system (like lambda calculus is) is simply not possible from first principles. You can't mend it, nobody can.
On the contrary, a decentralized computer based on a local graph rewrite system (like Lafont Interaction Combinators, the pure version, or chemlambda or chemSKI, etc) is possible. The problem here is that there are no programmers to efficiently use it (yet).
So, like it or not, the goal to make a decentralized computer by using what programmers like, it is as possible as the transmutation of elements were for alchemists.

I like clojure's approach to exactly these kind of systems, which it calls an "Epochal timeline" I think.
Clojure is dynamically typed, but they have developed "spec" to specify wire protocols. I also note that the creator of Clojure says that the internals of programs should reflect their externals (paraphrased), so to me this implies that modules should also have APIs defined with "spec".
The state model I have been proposing to investigate for my language turns out to be the same as Clojure's Epochal timeline, except I want to allow controlled mutability (different topic however). The main difference in approach is that I want to capture the typing of protocols and modules in a type system rather than a tool.
Basically in my model/terminology a 'protocol' is a type that represents the exchange of data between two programs. Calling an API method in a local library is a very simple example of a protocol. Calling a remote procedure call, or a RESTful API with JSON data is a other slightly more complex protocol. The most complex protocols involve more than two programs, and each is maintaining an internal state that affects the messages it can send it receive.

https://www.academia.edu/27047824/Emergence_Oriented_Programming
Emergent properties arise from the numerousinteractions of many simple agents. It is difficult toencapsulate emergence in a traditional programminglanguage. An object-oriented language such as Java iscapable of encapsulating the properties and behavior of asingle agent within a class instance, or object. Thus, Javaprovides a specific language mechanism for modularizingagents. However, emergent structure does not exist withina single agent, nor does it result from the execution of asingle agent behavior; rather it arises from the interactionsof many agents over the course of time. Thus, toencapsulate emergence, we need a program languageconstruct that can express modularity across a set of objectsand a set of object interactions. This requires encapsulationof a set of classes and class method invocations. Object-oriented languages do not support such modularity.A new programming language paradigm has beenintroduced in recent years to address such “cross-cutting”concerns[7]. Aspect-oriented programming (AOP) is alanguage model that supports the encapsulation of concernsthat cross the object-oriented class boundary. AOP providesdirect language support for modularizing emergentstructure and behavior. For example, an aspect may bewritten that describes a set of method invocations, whichmay occur in multiple classes. Emergent behavior maythus be recognized when such a set of method invocationsoccurs during a program execution.
4.1

Using Aspect-Oriented Programming
Aspect-oriented programming allows theimplementation to accurately reflect the conceptual layersfrom the cognitive model by creating modular structuresthat encapsulate the layers on the software side[7]. Itsupports modularized code that can be mechanicallyinserted into executable code at specific, user-definablelocations. The key point is the set of locations can not beencapsulated modularly at the lower layer because theprogramming language at the lower level can not provide aconstruct for describing the set of execution points. Forexample, if we want to perform some logging operation thatshould occur directly before a particular set of invocationsof the method foo( ), we can either manually insert codeprior to each invocation of foo( ) throughout the Javaprogram, or use a higher level language such as AOP thathas a structure to express all invocations of foo( )simultaneously. By intertwining the new AOP modulesexpressed at the higher level into the executable bytecodes(in the case of Java) the original source is untouched, yetintegrates the desired functionality in the resultingexecutable. Many aspects can be applied to the same code,and aspects can be applied to other aspects. This powerfulmechanism provides a direct, tangible representation of thelayers within the emergence model described in section 3.We use the AspectJ compiler[1] to develop all code relatedto the visualization and implementation of downwardcausation of emergence, thus implementing the emergenceframework without tangling the agent and swarm models

authors: NodixBlockchain keean mbuliga

repo: keean/zenscript

license: mit

license_source: bigquery


text:
feat: add fast trigonometric functions

This PR adds fast trigonometric functions (sin, cos, tan) to the library. Compared to standard trigonometric functions, fast trigonometric functions are computational and spacial efficient, with minor (acceptable errors).
Pull Request type

Please check the type of change your PR introduces:

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Add fast trigonometric functions (sin, cos, tan) to the library
All tests are passed

Does this introduce a breaking change?

[ ] Yes
[x] No

Other information

why not i64 everywhere ?

why not i64 everywhere ?

Thanks @LucasLvy for taking time to review this PR!
It seems that i64 does not implement core::traits::Div and core::traits::Rem, which is crucial for the fast trigonometric algorithm. Consequently, I decided to firstly implement sin(x) for x > 0, and thereby obtaining sin(x), cos(x) and tan(x) for x < 0 and x > 0 using symmetry properties.

authors: LucasLvy Soptq

repo: keep-starknet-strange/alexandria

license: MIT


text:
Add failing test case for #249

Subscribing to a circular dep should cause everything to
deactivate but doesn't.
I don't have a solution to this yet, but if anyone has ideas why this might be happening, we at least have a test case we can use to verify the solution.

This seems like something impossible to fix without some kind of global supervision that would analyze all observables and detect loops. I mean from the point of view of individual observables there is no way to notice a loop and deactivate itself, i.e. notice that the only subscriber that left is effectively the same observable.

notice that the only subscriber that left is effectively the same observable.

I'd probably have to dig into the internals a bit more, but if the only subscriber isn't active, we don't necessarily need to detect a loop, do we?

but if the only subscriber isn't active

A subscriber isn't an observable it's just a callback. The callback is added by an observable, but we have access only to a callback. But even if we had access to the observable it's in fact active because it also have a subscriber, (basically from the first observable, via another intermediate observable).

If this isn't a bug, should we consider closing this issue?

authors: bpinto mAAdhaTTah rpominov

repo: kefirjs/kefir


text:
Node engine >= 7.6?

Is that really necessary? I'm on 6.11.3 LTS.

That is really necessary now because it uses async functions available on v7.6.0 or later. v7.6.0 updated V8 to 5.5 which supports async functions. I've created a new branch to test it. Here's the results:

https://travis-ci.org/kei-ito/rollup-plugin-glob-import/builds/275715410

Thank you for your question. IMO, the plugin should support older versions until EOL. I'll update it on this branch to support versions listed in the results.

I've published v0.0.8 which works on Node>=6.

Thank you.

authors: kei-ito tkambler

repo: kei-ito/rollup-plugin-glob-import


text:
Ignore commented lines when checking indentation, attached replication - to explore

test.txt
source - internal gtm chat group

Above test.txt works (after correcting indentation to spaces instead of mix tabs + spaces) but below fails when it's on 2nd line.
b.txt

Normally programming best practice comment should be aligned with indentation for the block.
This is not straightforward to implement unless exclude comments from generated JS script.
But that's bad for debugging and tracing issues if problem happens during compilation.
Dropping this to prioritise on other items.

repo: kelaberetiv/TagUI


text:
Rename repository to uuid

Given that this module is cross-platform (node + browser) and that the npm module has been renamed to uuid long time ago, I suggest renaming this repository to kelektiv/uuid as well.
Since GitHub will keep redirects from the old name to the new name this should not cause any real trouble.
@broofa @defunctzombie opinions?

Works for me.

Moving to #338

authors: broofa ctavan

repo: kelektiv/node-uuid


text:
Plex not secure with Traefik

This seems to still be an issue with Traefik Plex not Secure Media being played back on all clients does not show as secure in Plex.  Certain clients will fail playback with this.  In Gooby with Nginx this wasn't an issue.  Are others seeing this issue?  Are there any solutions?

Are you connecting to Plex via the Traefik proxy or did you open up a port and connect to Plex directly?
Unless you've changed from the defaults, the Plex server should be setup with remote access looking like this:

The public IP is your host server's IP address and 443 tells it to route through Traefik.  I connect to my Plex server this way using https://plex.mydomain.com (no need to specify the port since 443 is implied by https).
Now, the top part will often say "unreachable" but that's because Plex doesn't know that Traefik exists.  But that's OK.
You'll also want to configure the "customer server access URL" on the network tab of settings (towards the bottom).  Again, if setup using the defaults, this would just be https://plex.mydomain.com.
Since all access to and from Plex is going through https, it is indeed secure.

Setup with all the defaults exactly as you mention above:

Modifying plex yaml to open a port for Plex and not connecting through traefik:

I think indeed it is technically secure but Plex/Plex servers don't think so.   So configured with Traefik if you change inside Plex settings from Secure Connection : preferred to required nothing will work and you can't connect to Plex server.  Also some Plex clients will fail playback (Namely Android TV, FireTV) if the connection isn't secure.  Interestingly this was not an issue with NGINX in Gooby.

Setup with all the defaults exactly as you mention above:

Modifying plex yaml to open a port for Plex and not connecting through traefik:

I think indeed it is technically secure but Plex/Plex servers don't think so. So configured with Traefik if you change inside Plex settings from Secure Connection : preferred to required nothing will work and you can't connect to Plex server. Also some Plex clients will fail playback (Namely Android TV, FireTV) if the connection isn't secure. Interestingly this was not an issue with NGINX in Gooby.

I'm seeing the same behaviour as you with the red arrows and remote access error (expected) though I hadn't noticed any issues with client playback so to be honest hadn't questioned it!

One of my family members uses ubiquiti equipment and they would get playback fail on FireTV  when I was running Gooby but adding port to "customer server access URL" from  https://plex.mydomain.com to https://plex.mydomain.com:443 fixed it on Gooby and plex had green arrows and he had no more problems.  Now problems are back with OmniStream and adding 443 doesn't help with traefik.  Don't change Plex setting to require secure connections on remote sever you will lose access and have to ssh tunnel into your sever to access it again.  Maybe there is a way to change traefix to fix this ... I am no expert.

This post seems to be a possible solution?  Seems connection from Plex docker to traafik might be the issue?  I am noob on Traefik can someone guide me here and I will do the work to test this?  thanks

Has anyone remedied this error? I've tried a few things but to no avail.

There really isn't any issue here.  https to Traefik = secure.  Traefik to Plex is happening on the same server and is literally two areas of memory communicating with each other (whether encrypted or unencrypted).  Plex may not "think" it is secure because it sees this data exchange and not the exchange between the true client (Plex's app or Infuse or a web browser) and Plex.

authors: Simmy-Simalia christeasdale kelinger thunderstorm654

repo: kelinger/OmniStream


text:
Wrong Unicode conversion for "dash within numbers" and "range of numbers"

I tried the following conversion in an annotation field with the "Guess Unicode Punctuation" script:
Before conversion (all three lines use hyphen minus (U0012) from the standard keyboard)
Figure Dash (U2012)	Used as a dash within numbers (e.g. 555-1212).
En Dash     (U2013)	Indicates a range of numbers (e.g. 1989-90).
Hyphen 	    (U2010)     Joins words and syllables of a word (e.g. co-operate) and used within dates (e.g. 2022-01-01 or 2021-31)
After conversion
Figure Dash (U2012)	Used as a dash within numbers (e.g. 555–1212).                                                                 actual result: U2013   expected result: U2012
En Dash     (U2013)	Indicates a range of numbers (e.g. 1989‐90).                                                                   actual result: U2010   expected result: U2013
Hyphen 	    (U2010)     Joins words and syllables of a word (e.g. co‐operate) and used within dates (e.g. 2022‐01‐01 or 2021‐31)       actual result: U2010
So according to https://wiki.musicbrainz.org/User:Jacobbrett/English_Punctuation_Guide the first two lines seem to be wrong after conversion. Is this a supported case by the script ?

Thanks for your tests, currently numbers other than dates are not really handled well by the simple RegEx replacement transformations that the script is doing (as you have noticed).
I had not put much thought into figure dashes since they are very rarely to be found in song and album titles. Also it is often impossible to tell ranges and grouped digits apart. Your examples perfectly show why even a more complex logic can't do these correctly in all cases: 555-1212 could also be a range (since 1212 is greater than 555) while 1989-90 was unlikely to be a range if 90 was not an abbreviated year. So I had decided to keep it simple and interpret all remaining hyphen-minus between digits as range specifiers (en dash) 😅
The only ways I see how the script does not return wrong results for grouped digits is to...

ignore all hyphen-minus between digits which are not valid dates. I would rather avoid this as ranges in titles are much more common.
add special cases for e.g. ISBNs and (some) phone numbers (if even possible). It is probably overkill to have dedicated additional rules for these rare things which then have to be run needlessly for all titles. Currently I am very happy that I have condensed most cases into just 11 rules, but I might consider this in the future if there is demand.

I could probably use figure dashes if there are three or more groups of digits, I will have a look.
So far I was at least able to make the script handle the 1989-90 case correctly, previously it interpreted that one as a partial ISO 8601 date. Now it tries to validate the date before doing the replacement and leaves invalid dates for the other transformation rules, which leads to 1989-90 interpreted as range (see d0ff0142fc944ef181dcf2cf018f1c963570f2c1).

Thanks for your tests, currently numbers other than dates are not really handled well by the simple RegEx replacement transformations that the script is doing (as you have noticed).
I had not put much thought into figure dashes since they are very rarely to be found in song and album titles. Also it is often impossible to tell ranges and grouped digits apart. Your examples perfectly show why even a more complex logic can't do these correctly in all cases: 555-1212 could also be a range (since 1212 is greater than 555) while 1989-90 was unlikely to be a range if 90 was not an abbreviated year. So I had decided to keep it simple and interpret all remaining hyphen-minus between digits as range specifiers (en dash) 😅
The only ways I see how the script does not return wrong results for grouped digits is to...

ignore all hyphen-minus between digits which are not valid dates. I would rather avoid this as ranges in titles are much more common.
add special cases for e.g. ISBNs and (some) phone numbers (if even possible). It is probably overkill to have dedicated additional rules for these rare things which then have to be run needlessly for all titles. Currently I am very happy that I have condensed most cases into just 11 rules, but I might consider this in the future if there is demand.

I could probably use figure dashes if there are three or more groups of digits, I will have a look.
So far I was at least able to make the script handle the 1989-90 case correctly, previously it interpreted that one as a partial ISO 8601 date. Now it tries to validate the date before doing the replacement and leaves invalid dates for the other transformation rules, which leads to 1989-90 interpreted as range (see d0ff0142fc944ef181dcf2cf018f1c963570f2c1).

I could probably use figure dashes if there are three or more groups of digits (then it is definitely not a range and dates have already been converted before), I will have a look.

Done in c8348ac5f61c71d698739138c90413fae9e59721, now it only interprets two groups of digits as range (correctly or mistakenly).
I have released these changes in version 2022.1.1.

I could probably use figure dashes if there are three or more groups of digits (then it is definitely not a range and dates have already been converted before), I will have a look.

Done in c8348ac5f61c71d698739138c90413fae9e59721, now it only interprets two groups of digits as range (correctly or mistakenly).
I have released these changes in version 2022.1.1.

cool thx ... and happy new year

authors: kellnerd vzell

repo: kellnerd/musicbrainz-bookmarklets


text:
[Error] FATAL empty src template with Consul as backend

Hi, guys:
I met the following issue when i use consul as confd's backend.
the dir tree listed below:
./
|-conf.d
|     |-confd.toml
|     |-hosts.toml
|-templates
|-hosts.conf.tmpl
the confd is configuration file for confd:
backend = "consul" confdir = "/home/confd/etc" log-level = "debug" onetime = false interval = 15 nodes = [     "http://ip1:8500",     "http://ip2:8500", ]
the hosts.toml:
[template] src = "hosts.conf.tmpl" dest = "/tmp/hosts" user = "confd" keys = ["/test/master/hosts"]
the template file:hosts.conf.tmpl
[hosts] {{getv "/test/master/hosts"}}
Starting commandline:
confd -config-file="/home/confd/etc/conf.d/confd.toml"
I can get the key directly from consul webpage and api, but information remains "empty src template" after starting with confd commandline, is there syntax error in conf file or somewhere else went wrong?

It looks like your hosts.conf.tmpl file is located in ./ directory instead of ./templates directory. Try moving it to ./templates directory instead.

Formatting mistake，has been put under templates, also try move to /etc/confd, use default config file setting, seems still doesn't work. @okushchenko

@ChuckNg I noticed that you have 2 config files (confd.toml and hosts.toml), but only 1 template file (hosts.conf.tmpl). This may be the source of your error, as confd expects each config to have a corresponding template.

authors: ChuckNg okushchenko

repo: kelseyhightower/confd


text:
Add new function ProcessFile to read from a .env file

The file is read and environment variables added to the
environment, then the existing Process function is called.
Not sure if you want this functionality in the code, but it's useful to me.

Hi @robvdl, thanks for the PR.
Yeah, as you intimated I don't really see this as a part of envconfig's wheelhouse. ProcessFile is effectively processing a config file into a struct, and there are better file formats (and much better libraries) for accomplishing that.
If on the other hand this is externally imposed constraints (your organization already uses this kind of configuration approach) then there's nothing keeping you from using your ProcessFile implementation in your own package -- it's just not a great fit for here upstream.

Fair enough, I had a feeling about that.  I will just have to put that function in my projects instead which is fine.
It isn't quite "just a config file" though, as it basically is a shell parseable file containing just env variables that you can source in bash just as well, that is why the format is intentionally simple.
Only reason I thought I would try is because other similar libraries implement this, for example https://github.com/joho/godotenv

authors: robvdl teepark

repo: kelseyhightower/envconfig


text:
endpoints access for worker nodes fail

When I setup a worker node.. It can't fetch from any urls from apiserver. f.ex. https://lbip:6443/api/v1/endpoints?limit=500&resourceVersion=0"
It seems to be an rbac problem.
If I try to curl:
curl --cert /var/lib/kubernetes/worker01.pem --key /var/lib/kubernetes/worker01-key.pem .. I get:
"message": "endpoints is forbidden: User "system:node:worker01" cannot list endpoints at the cluster scope",

I get the same if I try to use the kubernetes client cert
"message": "endpoints is forbidden: User "kubernetes" cannot list endpoints at the cluster scope",
This works fine:
kubectl get componentstatuses --kubeconfig admin.kubeconfig

cert is created using cfssl in your guide and looks fine?
 Subject: C = DK, ST = Copenhagen, L = Copenhagen, O = system:nodes, OU = Infrastructure, CN = system:node:worker01

I fixed this, by altering this line in kube-api-server systemd:
--runtime-config=api/all=true
to this:
--runtime-config=api/all=true,admissionregistration.k8s.io/v1alpha1=true

That should probably be updated in your guide ?

This don't solve the same problem at my side. But the following command solves it:
kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

please check RBAC file or kubeconfig for node Authorization https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md

authors: KlavsKlavsen hareeshrao4839 palbiez

repo: kelseyhightower/kubernetes-the-hard-way


text:
Nothing interesting at all

Well, maybe one thing. For empty commits do:
git commit -m "message" --allow-empty

Great fix. Can I get a permission from you to cherry-pick it to my project?

repo: kelseyhightower/nocode


text:
Add problem solving status table based on difficulties (List Page)

List Page に Difficulty 区分の進捗状況を表示するテーブルを実装しました (これを Difficulty Status と呼称します)
List Page の Point Status と Difficulty Status にワンクリックでフィルタを設定できるリンクを追加しました
List Page の {from / to} Difficulty filter の挙動を変更しました
フィルタを未設定の場合は全ての問題を表示しますが、 {from / to} いずれかが有効になっている場合には difficulty が設定されている問題のみを表示します
List Page のフィルタを一括でリセットするボタンを設置しました
User Page の Recommends の {Easy, Moderate, Difficult} それぞれで表示する問題を Solve Probability の値で限定しました

Easy: 50 - 100 %
Moderate: 20 - 80 %
Difficult: 0 - 50 %

これにより重複表示をある程度抑制できますが、 #267 を直接解決はしません。 上記の List Page のフィルタ機能の改善により、高難度の問題をより簡単に一覧表示できるようになったため、 問題を解きすぎてしまった方の要望にはある程度応えられていると思います。(そういう人は上級者なので difficulty や点数を見て自分で解く問題決めればいいと思う)

Codecov Report

Merging #270 into master will decrease coverage by 0.58%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #270      +/-   ##
==========================================
- Coverage   23.72%   23.14%   -0.59%
==========================================
  Files          66       67       +1
  Lines        1766     1815      +49
  Branches      238      244       +6
==========================================
+ Hits          419      420       +1
- Misses       1277     1326      +49
+ Partials       70       69       -1

Impacted Files
Coverage Δ

...ms-frontend/src/pages/UserPage/Recommendations.tsx
0% <0%> (ø)
:arrow_up:

...der-problems-frontend/src/pages/ListPage/index.tsx
0% <0%> (ø)
:arrow_up:

...ms-frontend/src/pages/ListPage/DifficultyTable.tsx
0% <0%> (ø)

...roblems-frontend/src/pages/ListPage/SmallTable.tsx
0% <0%> (ø)
:arrow_up:

atcoder-problems-backend/src/utils.rs
85% <0%> (+2.5%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 961d535...7513511. Read the comment docs.

SUGOII

authors: codecov-io kenkoooo koyumeishi

repo: kenkoooo/AtCoderProblems


text:
Cookies never returned in response object

I have been trying to get the cookies from a response, as your docs suggest (http://docs.python-requests.org/en/master/user/quickstart/#cookies [first example]), but it does not seem to be working.
When I dug through the sessions.py (https://github.com/kennethreitz/requests/blob/master/requests/sessions.py#L655), it looks like cookies are extracted to the cookie jar on the Session object, but never set on the response.
I had to do the following to get my intended behavior (which is probably not entirely correct, since it will copy all cookies on the Session, not just the ones from the most recent request):
>>> url = 'http://example.com/some/cookie/setting/url'
>>> s = requests.Session()
>>> r = s.get(url)
>>> r.cookies = s.cookies

>>> r.cookies['example_cookie_name']
'example_cookie_value'

Hey @dubhunter, I'm actually unable to reproduce this in either Python 2 or 3 for Requests 2.13.0.
Could you please provide the version of Requests you're using and verify for me that the script below returns <RequestsCookieJar[]>? No need to provide the contents if the RequestsCookieJar is populated.
import requests

s = requests.Session()
resp = s.get('https://github.com/kennethreitz/requests')
print(resp.cookies)

@nateprewitt You are correct. After further digging, it looks like I am having trouble because the CookiePolicy set on the Session object is not copied to Response and there is no way to set the CookiePolicy for the Response.
We can close this issue and open another, or reword this one.
Thoughts?

I was able to work around this with the following code in my adapter:
    def build_response(self, req, resp):
        response = super().build_response(req, resp)
        response.cookies = cookiejar_from_dict({})
        response.cookies.set_policy(self.cookie_policy)
        extract_cookies_to_jar(response.cookies, req, resp)
        return response

    @property
    def cookie_policy(self):
        return CookiePolicy()

I think the cookie policy concern is fundamentally part of #3669. However, you may find a better workaround is to patch requests.cookies.requests_cookie_policy to return whatever it is you want. That function is used to initialize all RequestsCookieJar objects.

It seems like this may be more closely related to #3416 and #3463. @dubhunter, would you mind clarifying a bit further your desired use case/behaviour here? It seems like you want a Response specific cookie policy propagated by the Session?
#3416 was asking for the user to be able to persist a Session-wide policy that would scope what cookies were allowed into the Session CookieJar. While this would still allow the user to inspect ALL cookies returned on an individual response, it would only carry policy-approved cookies to the next request, which still seems reasonable to me. Is this anywhere inline with your use case?

Yes, #3416 seems to cover my use case. It would also be nice to be able to override RequestsCookieJar (since the cookie policy is never consulted when sending cookies), but that has lower utility.

@Lukasa @sigmavirus24, do either of you have any further thoughts on persisting cookie policies? I'm still not ecstatic about the implementation in #3463 but it seems a step in the right direction for cases like this. I only ask since #3998 will make doing this more difficult.
Either way we can probably close this out as a dupe of #3416.

Ok, let's close this as a dupe. I think persisting cookie policies would solve the problem in #3998, so I'm open to just preferring doing that.

authors: Lukasa dubhunter nateprewitt

repo: kennethreitz/requests


text:
Install & Configuration Problem

Hello
I have some problem installation & configuration on Ubuntu 14.04 ec2.
I followed instruction and I change security rules in ec2  (ports etc.) but unfortunately when I check my ip on browser I could not connect to rails server.
Does anyone try to install it ec2 ? or do you have any recommendation ?
Cheers,
Oz.

Refer this tutorial once it may help you for installation & configuration on Ubuntu 14.04 ec2.
https://www.youtube.com/watch?v=imdrYD4ooIk&list=PLcyjBWSnKz8aHip8pFVTE4pA6iloUx3fs&index=2&t=1488s
But easy way of doing  this is using elasticbeanstalk.

authors: dinesh-pallapa okaragoz

repo: kenny-hibino/stories