text:
incomplete comment in proto_format.sh

Title: Incomplete comment found in script proto_format.sh
Description: incomplete comments in proto_format.sh script it says "# Generate //versioning:active_protos."
However it produces both active and frozen_protos as well.
url: https://github.com/envoyproxy/envoy/blob/master/tools/proto_format/proto_format.sh#L27

Describe the issue.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

though it is very trivial bug to report but i thought it should be corrected for clarity of code.

@dio. please reopen it and assign to me. as per your latest comment i will push pr.

/no stalebot


text:
QUIC Connection Migration

We support QUIC upstream, but there's currently not support for migration.  I think this would have to be done in upstream Envoy even if it's likely only going to be used for mobile.

Connection migration and port migration is implemented in https://github.com/envoyproxy/envoy/pull/18103. But they are behind a default false flag --quic_remove_connection_migration_connection_option. We turned off --quic_remove_connection_migration_connection_option in QUICHE to make the server to advertise connection migration to quic clients only implemented by QUICHE. To enable cronvoy to indicates it used QUICHE client implementation, we need to hard code a QUICHE connection option tag RVCM in the client connection.

/assign @RenjieTang

source: gharchive/issue

repo: envoyproxy/envoy

license_type: permissive

license_source: github-api


text:
Matching api to Compose Wasm filter failed

Matching api to Compose Wasm filter failed
Description:
I try to use match api and compose filter to compose wasm filter, it runs ok but it seems like filter is not working;
I checked stats and it failed:

Is Compose filter not supported to compose wasm filter or something else?
config is here:
admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
  listeners:
    - name: main
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 18000
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: auto
                route_config:
                  name: httpbin
                  virtual_hosts:
                    - name: httpbin
                      domains:
                        - "*"
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: httpbin
                http_filters:
                - name: composite
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.common.matching.v3.ExtensionWithMatcher
                    extension_config:
                      name: composite
                      typed_config:
                        "@type": type.googleapis.com/envoy.extensions.filters.http.composite.v3.Composite
                    matcher:
                      matcher_tree:
                        input:
                          name: request-headers
                          typed_config:
                            "@type": type.googleapis.com/envoy.type.matcher.v3.HttpRequestHeaderMatchInput
                            header_name: switch
                        exact_match_map:
                          map:
                            "test":
                              action:
                                name: composite-action
                                typed_config:
                                  "@type": type.googleapis.com/envoy.extensions.filters.http.composite.v3.ExecuteFilterAction
                                  typed_config:
                                    name: envoy.filters.http.wasm
                                    typed_config:
                                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                                      value:
                                        config:
                                          name: "my_plugin1"
                                          root_id: "my_root_id1"
                                          vm_config:
                                            vm_id: "my_vm_id1"
                                            runtime: "envoy.wasm.runtime.v8"
                                            code:
                                              local:
                                                filename: "./log.wasm"
                                            allow_precompiled: true
                            "wasm":
                              action:
                                name: composite-action
                                typed_config:
                                  "@type": type.googleapis.com/envoy.extensions.filters.http.composite.v3.ExecuteFilterAction
                                  typed_config:
                                    name: envoy.filters.http.wasm
                                    typed_config:
                                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                                      value:
                                        config:
                                          name: "param-exchange"
                                          root_id: "param-exchange"
                                          vm_config:
                                            runtime: "envoy.wasm.runtime.v8"
                                            code:
                                              local:
                                                filename: "./param-exchange.wasm"
                                            allow_precompiled: true
                                          configuration:
                                              "@type": "type.googleapis.com/google.protobuf.StringValue"
                                              value: |
                                                {"update_param_list": [
                                                    {
                                                        "old_request": {
                                                            "location": "query_parameter",
                                                            "key": "source1"
                                                        },
                                                        "new_request": {
                                                            "location": "query_parameter",
                                                            "key": "replace1"
                                                        }
                                                    },
                                                    {
                                                        "old_request": {
                                                            "location": "query_parameter",
                                                            "key": "source2"
                                                        },
                                                        "new_request": {
                                                            "location": "query_parameter",
                                                            "key": "replace2"
                                                        }
                                                    }
                                                ],
                                                "delete_param_list": [
                                                    {
                                                        "request": {
                                                            "location": "cookie",
                                                            "key": "destination3"
                                                        }
                                                    },
                                                    {
                                                        "request": {
                                                            "location": "cookie",
                                                            "key": "destination4"
                                                        }
                                                    }
                                                ],
                                                "add_param_list": [
                                                    {
                                                        "request": {
                                                            "location": "header",
                                                            "key": "add1"
                                                        },
                                                        "value": "true"
                                                    },
                                                    {
                                                        "request": {
                                                            "location": "header",
                                                            "key": "add2"
                                                        },
                                                        "value": "true"
                                                    },
                                                    {
                                                        "request": {
                                                            "location": "header",
                                                            "key": "add3"
                                                        },
                                                        "value": "true"
                                                    }
                                                ]}
                - name: envoy.filters.http.router

  clusters:
    - name: httpbin
      connect_timeout: 5000s
      type: strict_dns
      lb_policy: round_robin
      load_assignment:
        cluster_name: httpbin
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 0.0.0.0
                      port_value: 9001

layered_runtime:
  layers:
    - name: static-layer
      static_layer:
        envoy:
          reloadable_features:
            experimental_matching_api: true

[optional Relevant Links:]

Any extra documentation required to understand the issue.

cc @snowp

Can you run this with debug logs? That should output the cause of the failure

Looking at the config my first guess is that it would be the use of TypedStruct causing the issue

I run envoy with envoy -c compose.yaml -l debug I can not see the output of the failure

[2021-05-29 01:27:31.956][1928132][debug][filter] [source/extensions/filters/http/composite/filter.cc:88] failed to create delegated filter [INVALID_ARGUMENT: cannot delegate to filter that adds an access log handler]<br class="Apple-interchange-newline">

The problem seems to be that the WASM filter adds an access logger, which isn't supported by the composite filter. Unfortunately this is a limitation of the composite filter for now, and would require a bit of code change to fix.

I see... Will this be fixed in future?

Someone will need to do the work, it's not on my personal roadmap

Can you take a look at this log : https://github.com/Xunzhuo/tmp/blob/main/log2 ? I turn out accesslog in WASM filter

Are you still seeing the failure stats? I don't see a log line here

Sorry, I check it again which caused by the same problem as you said, it seems like WASM adds access logger no matter if I use log abi... Maybe I could try to give a PR...

No problem! The problem with supporting access loggers is that normally access loggers are set up when the request first starts, we'd need support for adding an access logger during the request (or somehow be okay with the access log not being added, but that seems risky in case filters depend on this)

Hi guys, we have bumped into this issue as well, even in WASM filters that are only implementing on_http_request_headers. Can you think of any workaround to avoid it?

I don't believe there are any possible workarounds besides adding support to the composite filter to support filters that add an access logger. Most likely this can be done by having the composite filter add its own delegating access logger, but this will require code change.

authors: Xunzhuo balbifm phlax snowp


text:
Envoy forward headers

I am using envoy.ext_authz filter with OPA as my external authorization service provider. The OPA policy executes successfully however it is returning 403 UAEX in both success and error scenarios. I tested the policy in Rego playground and it returns true in success scenario and false in error scenario. I am suspicion whether envoy is sending x-custom-header in my use-case.
Questions :

Will envoy forward request headers to OPA or will we need to explicitly pass header to OPA with some Envoy configuration? Please advise.
Please advise on how i can troubleshoot this error. Is there a way i can log input headers in envoy or in OPA?

Policy:
package demo

default allow = false

allow = {
        input.headers["x-custom-header"] == "foo"
}

Input:
{
       "headers": {
              "x-custom-header" :"foo"
        }
}

Rego playground output:
{
     "allow": [
           true
     ]
}

/cc @esmet @gsagula @dio

@gituserjava I believe authorization request headers need to be allowlisted https://github.com/envoyproxy/envoy/blob/e98f39b1b3729c8ee507cf4b64cbbe72e3991457/api/envoy/extensions/filters/http/ext_authz/v3/ext_authz.proto#L241 though certain ones like Authorization, etc are included automatically.

authors: esmet gituserjava rojkov


text:
No stack info when use admin API cpuprofiler.

I complie envoy use centos7.2,and collect the cpuprofiler with admin api(curl -X POST localhost:15000/cpuprofiler?enable=y).But thers is no stack info about envoy.
Here is the output.
File: envoy
Type: cpu
Showing nodes accounting for 10ms, 100% of 10ms total
      flat  flat%   sum%        cum   cum%
      10ms   100%   100%       10ms   100%  [envoy]
         0     0%   100%       10ms   100%  [libc-2.17.so]

I have builded envoy with no strip ,and even complie dwp file in pprof work directory.
Any help will be appreciated!!

https://github.com/envoyproxy/envoy/blob/main/bazel/PPROF.md#collecting-cpu-or-heap-profile-for-a-full-execution-of-envoy
Did you build Envoy with --define tcmalloc=gperftools?

Yes，I have define tcmalloc=gperftools,or the cpuprofiler admin api can not be posted.

I'm facing the same issue, are there any workarounds?

authors: fxposter lizan pyrl247


text:
ext-authz with http_service: fail-close doesn’t work for grpc traffic

Title: ext-authz with http_service: fail-close doesn’t work for grpc traffic
Description:
The ext-authz http_service is configured with failure_mode_allow set to false and status_on_error set to 406.
This configuration operates correctly for incoming HTTP requests. However, for incoming gRPC requests, if a timeout occurs for the ext-authz request sent to the ext-authz service over http occurs, the request is erroneously sent to the upstream.
Instead, these requests should be identified as failures and should not be forwarded to the upstream.
Repro steps:

Use httpstatus image as the authrizor. It lets you simulate timeout based on the value in a header 'x-httpstatus-sleep'
docker run -d --rm -p 8081:80 --name httpstatus ghcr.io/aaronpowell/httpstatus:main
Run a simple grpc server as described here https://grpc.io/docs/languages/python/quickstart/#run-a-grpc-application
python greeter_server.py
Run the grpc client
Run envoy with --net=host with the below Config
docker run --net=host -d -v "$PWD"/envoy.yaml:/etc/envoy/envoy.yaml --name my-envoy-instance envoyproxy/envoy-dev:latest envoy -c /etc/envoy/envoy.yaml --log-level debug
change the server port in greeter_client.py to 50000 and run it.
python greeter_client.py
To simulate a 3 seconds timeout by setting the header value:
a. Add the header to the greeter_client.py code as metadata: response = stub.SayHello(helloworld_pb2.HelloRequest(name="you"), metadata=[('x-httpstatus-sleep', '3000')])
b. Or use grpcurl
grpcurl -proto python/grpc/examples/protos/helloworld.proto -H "x-httpstatus-sleep: 3000" -plaintext -d '{"name": "You"}' 10.26.30.232:50000 helloworld.Greeter/SayHello

Admin and Stats Output:
attached
admin_output.zip
Config:
admin:
  address:
    socket_address: { address: 0.0.0.0, port_value: 50080 }
static_resources:
  listeners:
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 50000
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: AUTO
          stat_prefix: grpc_json
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                  grpc: {}
                route:
                  cluster: grpc_service
          http_filters:
          - name: envoy.filters.http.ext_authz
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
              http_service:
                server_uri:
                  uri: 127.0.0.1:8081/200
                  cluster: ext_authz
                  timeout: 2.5s
                path_prefix: /200
                authorization_request:
                  #headers_to_add:
                  #  - key: "x-httpstatus-sleep"
                  #    value: "2000"
                  allowed_headers:
                    patterns:
                    - safe_regex:
                        google_re2: {}
                        regex: .*
              failure_mode_allow: false
              status_on_error:
                code: 406
          - name: envoy.filters.http.lua
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
              inline_code: |
                function envoy_on_request(request_handle)
                  request_handle:headers():add("te", "trailers")
                end
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: grpc_service
    connect_timeout: 2s
    type: strict_dns
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: grpc_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 50051
    typed_extension_protocol_options:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        explicit_http_config:
          http2_protocol_options: {}
  - name: ext_authz
    type: STRICT_DNS
    connect_timeout: 0.25s
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: ext_authz
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 8081

Logs:
[2024-01-30 15:11:03.711][1][debug][upstream] [source/common/upstream/upstream_impl.cc:437] transport socket match, socket default selected for host with address 127.0.0.1:8081
[2024-01-30 15:11:03.712][1][debug][upstream] [source/common/upstream/upstream_impl.cc:437] transport socket match, socket default selected for host with address 127.0.0.1:50051
[2024-01-30 15:11:03.789][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-01-30 15:11:06.641][15][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"14"] new connection from 10.26.30.232:51938
[2024-01-30 15:11:06.641][15][debug][http2] [source/common/http/http2/codec_impl.cc:1618] [Tags: "ConnectionId":"14"] updating connection-level initial window size to 268435456
[2024-01-30 15:11:06.642][15][debug][http] [source/common/http/conn_manager_impl.cc:393] [Tags: "ConnectionId":"14"] new stream
[2024-01-30 15:11:06.642][15][debug][http] [source/common/http/conn_manager_impl.cc:1192] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] request headers complete (end_stream=false):
':method', 'POST'
':scheme', 'http'
':path', '/helloworld.Greeter/SayHello'
':authority', '10.26.30.232:50000'
'content-type', 'application/grpc'
'user-agent', 'grpcurl/dev-build (no version set) grpc-go/1.57.0'
'te', 'trailers'
'grpc-accept-encoding', 'gzip'
'x-httpstatus-sleep', '3000'

[2024-01-30 15:11:06.642][15][debug][connection] [./source/common/network/connection_impl.h:98] [Tags: "ConnectionId":"14"] current connecting state: false
[2024-01-30 15:11:06.642][15][debug][router] [source/common/router/router.cc:514] [Tags: "ConnectionId":"0","StreamId":"1353047819360458962"] cluster 'ext_authz' match for URL '/200/helloworld.Greeter/SayHello'
[2024-01-30 15:11:06.642][15][debug][router] [source/common/router/router.cc:731] [Tags: "ConnectionId":"0","StreamId":"1353047819360458962"] router decoding headers:
':method', 'POST'
':path', '/200/helloworld.Greeter/SayHello'
':scheme', 'http'
':authority', '10.26.30.232:50000'
'content-length', '0'
'x-request-id', '81a8db2e-0bc8-4aee-b651-fd1c6fed0dfd'
'user-agent', 'grpcurl/dev-build (no version set) grpc-go/1.57.0'
'content-type', 'application/grpc'
'x-forwarded-proto', 'http'
'grpc-accept-encoding', 'gzip'
'x-httpstatus-sleep', '3000'
'x-envoy-internal', 'true'
'x-envoy-expected-rq-timeout-ms', '2500'

[2024-01-30 15:11:06.643][15][debug][pool] [source/common/http/conn_pool_base.cc:78] queueing stream due to no available connections (ready=0 busy=0 connecting=0)
[2024-01-30 15:11:06.643][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:291] trying to create new connection
[2024-01-30 15:11:06.643][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:145] creating a new connection (connecting=0)
[2024-01-30 15:11:06.643][15][debug][connection] [./source/common/network/connection_impl.h:98] [Tags: "ConnectionId":"15"] current connecting state: true
[2024-01-30 15:11:06.643][15][debug][client] [source/common/http/codec_client.cc:57] [Tags: "ConnectionId":"15"] connecting
[2024-01-30 15:11:06.643][15][debug][connection] [source/common/network/connection_impl.cc:1009] [Tags: "ConnectionId":"15"] connecting to 127.0.0.1:8081
[2024-01-30 15:11:06.643][15][debug][connection] [source/common/network/connection_impl.cc:1028] [Tags: "ConnectionId":"15"] connection in progress
[2024-01-30 15:11:06.643][15][debug][http] [source/common/http/conn_manager_impl.cc:1175] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] request end stream
[2024-01-30 15:11:06.643][15][debug][connection] [source/common/network/connection_impl.cc:746] [Tags: "ConnectionId":"15"] connected
[2024-01-30 15:11:06.643][15][debug][client] [source/common/http/codec_client.cc:88] [Tags: "ConnectionId":"15"] connected
[2024-01-30 15:11:06.643][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:328] [Tags: "ConnectionId":"15"] attaching to next stream
[2024-01-30 15:11:06.643][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"15"] creating stream
[2024-01-30 15:11:06.643][15][debug][router] [source/common/router/upstream_request.cc:569] [Tags: "ConnectionId":"0","StreamId":"1353047819360458962"] pool ready
[2024-01-30 15:11:06.643][15][debug][client] [source/common/http/codec_client.cc:141] [Tags: "ConnectionId":"15"] encode complete
[2024-01-30 15:11:08.713][1][debug][upstream] [source/common/upstream/upstream_impl.cc:437] transport socket match, socket default selected for host with address 127.0.0.1:8081
[2024-01-30 15:11:08.714][1][debug][upstream] [source/common/upstream/upstream_impl.cc:437] transport socket match, socket default selected for host with address 127.0.0.1:50051
[2024-01-30 15:11:08.790][1][debug][main] [source/server/server.cc:239] flushing stats
[2024-01-30 15:11:09.140][15][debug][router] [source/common/router/router.cc:1070] [Tags: "ConnectionId":"0","StreamId":"1353047819360458962"] upstream timeout
[2024-01-30 15:11:09.140][15][debug][router] [source/common/router/upstream_request.cc:476] [Tags: "ConnectionId":"0","StreamId":"1353047819360458962"] resetting pool request
[2024-01-30 15:11:09.140][15][debug][client] [source/common/http/codec_client.cc:158] [Tags: "ConnectionId":"15"] request reset
[2024-01-30 15:11:09.140][15][debug][connection] [source/common/network/connection_impl.cc:146] [Tags: "ConnectionId":"15"] closing data_to_write=0 type=1
[2024-01-30 15:11:09.140][15][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"15"] closing socket: 1
[2024-01-30 15:11:09.140][15][debug][client] [source/common/http/codec_client.cc:107] [Tags: "ConnectionId":"15"] disconnect. resetting 0 pending requests
[2024-01-30 15:11:09.140][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:495] [Tags: "ConnectionId":"15"] client disconnected, failure reason:
[2024-01-30 15:11:09.140][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:463] invoking 1 idle callback(s) - is_draining_for_deletion_=false
[2024-01-30 15:11:09.140][15][debug][http] [source/common/http/async_client_impl.cc:106] async http request response headers (end_stream=true):
':status', '200'
'content-type', 'application/grpc'
'grpc-status', '14'
'grpc-message', 'upstream request timeout'

[2024-01-30 15:11:09.140][15][debug][lua] [source/extensions/filters/common/lua/lua.cc:39] coroutine finished
[2024-01-30 15:11:09.140][15][debug][router] [source/common/router/router.cc:514] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] cluster 'grpc_service' match for URL '/helloworld.Greeter/SayHello'
[2024-01-30 15:11:09.140][15][debug][router] [source/common/router/router.cc:731] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] router decoding headers:
':method', 'POST'
':scheme', 'http'
':path', '/helloworld.Greeter/SayHello'
':authority', '10.26.30.232:50000'
'content-type', 'application/grpc'
'user-agent', 'grpcurl/dev-build (no version set) grpc-go/1.57.0'
'grpc-accept-encoding', 'gzip'
'x-httpstatus-sleep', '3000'
'x-forwarded-proto', 'http'
'x-request-id', '81a8db2e-0bc8-4aee-b651-fd1c6fed0dfd'
'te', 'trailers'
'x-envoy-expected-rq-timeout-ms', '15000'

[2024-01-30 15:11:09.140][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:265] [Tags: "ConnectionId":"5"] using existing fully connected connection
[2024-01-30 15:11:09.140][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:182] [Tags: "ConnectionId":"5"] creating stream
[2024-01-30 15:11:09.140][15][debug][router] [source/common/router/upstream_request.cc:569] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] pool ready
[2024-01-30 15:11:09.140][15][debug][client] [source/common/http/codec_client.cc:141] [Tags: "ConnectionId":"5"] encode complete
[2024-01-30 15:11:09.140][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"15"] destroying stream: 0 remaining
[2024-01-30 15:11:09.141][15][debug][router] [source/common/router/router.cc:1505] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] upstream headers complete: end_stream=false
[2024-01-30 15:11:09.141][15][debug][http] [source/common/http/conn_manager_impl.cc:1869] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'application/grpc'
'grpc-accept-encoding', 'identity, deflate, gzip'
'x-envoy-upstream-service-time', '1'
'date', 'Tue, 30 Jan 2024 15:11:08 GMT'
'server', 'envoy'

[2024-01-30 15:11:09.141][15][debug][client] [source/common/http/codec_client.cc:128] [Tags: "ConnectionId":"5"] response complete
[2024-01-30 15:11:09.141][15][debug][pool] [source/common/conn_pool/conn_pool_base.cc:215] [Tags: "ConnectionId":"5"] destroying stream: 0 remaining
[2024-01-30 15:11:09.141][15][debug][http] [source/common/http/conn_manager_impl.cc:1899] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] encoding trailers via codec:
'grpc-status', '0'
'grpc-message', ''

[2024-01-30 15:11:09.142][15][debug][http] [source/common/http/conn_manager_impl.cc:1974] [Tags: "ConnectionId":"14","StreamId":"6135348612711403180"] Codec completed encoding stream.
[2024-01-30 15:11:09.142][15][debug][http2] [source/common/http/http2/codec_impl.cc:1363] [Tags: "ConnectionId":"14"] stream 1 closed: 0
[2024-01-30 15:11:09.142][15][debug][http2] [source/common/http/http2/codec_impl.cc:1427] [Tags: "ConnectionId":"14"] Recouping 0 bytes of flow control window for stream 1.
[2024-01-30 15:11:09.142][15][debug][http2] [source/common/http/http2/codec_impl.cc:1363] [Tags: "ConnectionId":"5"] stream 3 closed: 0
[2024-01-30 15:11:09.142][15][debug][http2] [source/common/http/http2/codec_impl.cc:1427] [Tags: "ConnectionId":"5"] Recouping 0 bytes of flow control window for stream 3.
[2024-01-30 15:11:09.144][15][debug][connection] [source/common/network/connection_impl.cc:278] [Tags: "ConnectionId":"14"] closing socket: 0
[2024-01-30 15:11:09.144][15][debug][conn_handler] [source/common/listener_manager/active_stream_listener_base.cc:136] [Tags: "ConnectionId":"14"] adding to cleanup list

@yairrw this is because the you're configuring ext_authz server as HTTP service, while the service is returning gRPC, which always HTTP 200 so Envoy will treat it is OK. The log in these line shows:
[2024-01-30 15:11:09.140][15][debug][http] [source/common/http/async_client_impl.cc:106] async http request response headers (end_stream=true):
':status', '200'
'content-type', 'application/grpc'
'grpc-status', '14'
'grpc-message', 'upstream request timeout'

You'll need to configure ext_authz to talk to a gRPC server if your ext authz is a gRPC service.

Thanks for your response @lizan
In our use case the ext_authz server is an HTTP service that serves both http requests and grpc requests.
We'd like to be able to detect ext_authz failures for both.

Any suggestions on how to handle grpc failures with an ext_authz that serves both http requests and grpc requests?

authors: lizan yairrw


text:
uneven request distribution when using weighted LR LB

Title: uneven request distribution when using weighted LR LB
Description:

When LR LB is used with weights, each upstream tends to send a large series of requests to one host before moving onto the next.  For example, we've seen 50 sequential requests into a single box when using a weight of 50. It seems like the LB strategy is to send 'n' requests to a given host before moving onto the next one.

Repro steps:

Setup a weighted cluster and observe the request distribution into

I'm not completely sure how to fix this in the power of 2 load balancer. The thing that comes to mind (needs some simulation to check) is to use both weight and number of requests to create a psuedo-weight for the two chosen hosts, then pick based on that.

Also, I should clarify as the title of this jira is vague. The request distribution is actually even it's just that hosts get bombarded with requests serially.

Why not use an EDF scheduler and weight based on # of requests?

@htuch what do you mean? Use an EDF weight schedule like for RR and pick 2 off the top and select the one with the least requests?

@mattklein123 I think picking the min of (number of requests) / weight will work - if the weights are equal, it's the same as unweighted LR, and in the steady state, the number of requests for each host would be proportional to the weights.
For comparing 2 hosts, you can avoid the division and do e.g. min_index = requests[i] * weights[j] < requests[j] * weights[i] ? i : j;

I will fix this.

I'm trying to fix this and the math turns out to be incredibly non-intuitive. Here is what I have so far:
https://github.com/envoyproxy/envoy/commit/2f8114543394fc61fdfa6f03cba52a61c507c0dd
It's not a finished change, but there is some simulation code. Roughly, the algorithm I'm trying to use looks like this:

Pick 2 random hosts (I added some temporary code to try to make sure we actually pick 2 different hosts).
Calculate a scaled synthetic request count by dividing number of active requests (min 1) by the weight.
Calculate what % of requests host 1 should get of both hosts.
Use a random number to pick between host 1 and host 2 based on the target request %.

This actually seems to work correctly w/ 2 hosts. But it doesn't work correctly with more than 2 hosts. I think what is happening is that when hosts with the same weight are picked, it's somehow skewing the results, but the math here is beyond me.
I also found this paper which is directly applicable but I don't understand it either:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.85.1379&rep=rep1&type=pdf
I will play around with it a bit more tomorrow, but my current plan is to potentially do a full scan in the weighted case, and put in a TODO for a better algorithm. If anyone out there has any thoughts please speak up!

We could look into an EDF scheduler also, per @htuch, but that would require having a callback for when active requests go up and down for every host. The code changes to do this would be non-trivial.

Quick update here. @htuch and I met with the researcher who wrote http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.85.1379&rep=rep1&type=pdf this morning.
It turns out that paper and similar papers don't apply, as they are about the inversion of the problem. Basically, when requests have weight but all backends are the same weight. Apparently from a mathematical perspective this makes a big difference and there is no way to easily make P2C work with different backend weights.
The researcher did have some good ideas:

Duplicate all of the hosts based on weight, and then do straight P2C. This will work, at the expense of extra memory.
Use consistent hashing for weighting, but instead of actually hashing, just pick 2 random inputs, and then do P2C on those. So basically we could use a Maglev table with weights and then choose out of that. This is a really interesting observation.

The third option is the one I already started working on which is a modified EDF approach. I think I'm going to keep going with the EDF approach for now as it is very straightforward and almost all the code can be shared with RR. I will leave some comments in the code of other options and we can revisit in the future if necessary.

authors: chriselion htuch mattklein123 wgallagher


text:
fix clang-tidy reported errors on master

Description:
clang-tidy runs against diffs in CI, but it will caught all errors on the lines the PR author touched. Ideally we should make a full clang-tidy run error free.

I am very interested in, I can come to fix the issue

@zyfjeff Great, I think you can just run clang-tidy with -fix flag with the checks marked as error.

Not directly relevant, but would be nice for instructions on clang-tidy to be added in the same places as instructions on fix_format, e.g.
https://github.com/envoyproxy/envoy/blob/15706964a29e595c045a5d7ef0646d04f347dcc1/support/README.md

@auni53 Definitely, do you want to take that part?

Can do :)

Reporting this presumably because others will run into the issue. I have no idea how to run the clang-tidy script locally. Documenting here to try and prevent others running into the same...
$ ./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.clang_tidy'

No remote cache bucket is set, skipping setup remote cache.
ENVOY_SRCDIR=/source
fatal: Not a git repository (or any parent up to mount point /build)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).```

ah, rm -rf /tmp/envoy-docker-build did the trick.

@auni53 'ci/run_clang_tidy.sh' is written for running from do_ci.sh so it may depend on env vars from there. Usually I do tools/gen_compilation_database.py --run_bazel_build --include_header && run-clang-tidy-7, let me know (or ping me on Slack) when you face any issue.

hah, took a day to realize that should be run-clang-tidy -7 (in case anyone else comes along this)

@auni53 no run-clang-tidy-7 (without space) is the right one if you're in envoy-build-docker or have llvm installed via official apt repository.

latest status of master errors
dereka@dev-dereka-2:~/code/envoy-oss$ make clang-tidy 2>&1 | grep " error: " | grep -v "chromium_url"
/home/dereka/.cache/bazel/_bazel_dereka/b4e08c1eaa8adbb40db2e2ca7ea0cbf2/execroot/envoy/include/envoy/server/admin.h:124:53: error: invalid case style for parameter 'access_log_path_' [readability-identifier-naming,-warnings-as-errors]
/home/dereka/.cache/bazel/_bazel_dereka/b4e08c1eaa8adbb40db2e2ca7ea0cbf2/execroot/envoy/include/envoy/server/admin.h:124:53: error: invalid case style for parameter 'access_log_path_' [readability-identifier-naming,-warnings-as-errors]
bazel-out/k8-fastbuild/bin/source/extensions/quic_listeners/quiche/_virtual_includes/envoy_quic_server_session_lib/extensions/quic_listeners/quiche/envoy_quic_server_session.h:16:10: error: 'extensions/quic_listeners/quiche/quic_filter_manager_connection_impl.h' file not found [clang-diagnostic-error]
bazel-out/k8-fastbuild/bin/source/extensions/quic_listeners/quiche/_virtual_includes/envoy_quic_server_session_lib/extensions/quic_listeners/quiche/envoy_quic_server_session.h:16:10: error: 'extensions/quic_listeners/quiche/quic_filter_manager_connection_impl.h' file not found [clang-diagnostic-error]
source/extensions/quic_listeners/quiche/platform/quic_stack_trace_impl.h:16:22: error: implicit instantiation of undefined template 'std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >' [clang-diagnostic-error]
bazel-out/k8-fastbuild/bin/source/extensions/quic_listeners/quiche/_virtual_includes/envoy_quic_server_session_lib/extensions/quic_listeners/quiche/envoy_quic_server_session.h:16:10: error: 'extensions/quic_listeners/quiche/quic_filter_manager_connection_impl.h' file not found [clang-diagnostic-error]
bazel-out/k8-fastbuild/bin/source/extensions/quic_listeners/quiche/_virtual_includes/envoy_quic_server_session_lib/extensions/quic_listeners/quiche/envoy_quic_server_session.h:16:10: error: 'extensions/quic_listeners/quiche/quic_filter_manager_connection_impl.h' file not found [clang-diagnostic-error]
source/extensions/quic_listeners/quiche/platform/quic_stack_trace_impl.h:16:22: error: implicit instantiation of undefined template 'std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >' [clang-diagnostic-error]
bazel-out/k8-fastbuild/bin/source/extensions/quic_listeners/quiche/_virtual_includes/envoy_quic_server_session_lib/extensions/quic_listeners/quiche/envoy_quic_server_session.h:16:10: error: 'extensions/quic_listeners/quiche/quic_filter_manager_connection_impl.h' file not found [clang-diagnostic-error]
bazel-out/k8-fastbuild/bin/source/extensions/quic_listeners/quiche/_virtual_includes/envoy_quic_server_session_lib/extensions/quic_listeners/quiche/envoy_quic_server_session.h:16:10: error: 'extensions/quic_listeners/quiche/quic_filter_manager_connection_impl.h' file not found [clang-diagnostic-error]
test/server/filter_chain_benchmark_test.cc:174:20: error: initializer for base class 'Envoy::Server::FilterChainBenchmarkFixture' is redundant [readability-redundant-member-init,-warnings-as-errors]
test/server/filter_chain_benchmark_test.cc:183:20: error: initializer for base class 'Envoy::Server::FilterChainBenchmarkFixture' is redundant [readability-redundant-member-init,-warnings-as-errors]
/home/dereka/.cache/bazel/_bazel_dereka/b4e08c1eaa8adbb40db2e2ca7ea0cbf2/execroot/envoy/test/common/upstream/cluster_manager_impl_test.cc:49:16: error: using decl 'ByRef' is unused [misc-unused-using-decls,-warnings-as-errors]
dereka@dev-dereka-2:~/code/envoy-oss$

hi, anyone working on this issue ?

Did anyone meet this issue when running full clang-tidy under docker?
$ ./ci/run_envoy_docker.sh "RUN_FULL_CLANG_TIDY=1 ./ci/do_ci.sh bazel.clang_tidy"
3de483a98c5e24973e710b4f97b2dabcd3cb621f: Pulling from envoyproxy/envoy-build-ubuntu
Digest: sha256:a0dfdec963cec68c0636578ec1587cc542f9839584f32bb56165284d64cbafe6
Status: Image is up to date for envoyproxy/envoy-build-ubuntu:3de483a98c5e24973e710b4f97b2dabcd3cb621f
docker.io/envoyproxy/envoy-build-ubuntu:3de483a98c5e24973e710b4f97b2dabcd3cb621f

real    0m3.934s
user    0m0.029s
sys     0m0.003s
No remote cache is set, skipping setup remote cache.
ENVOY_SRCDIR=/source
ENVOY_BUILD_TARGET=//source/exe:envoy-static
ENVOY_BUILD_ARCH=x86_64
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Starting local Bazel server and connecting to it...
WARNING: info command does not support starlark options. Ignoring options: [--@com_googlesource_googleurl//build_config:system_icu=0]
Skip setting up Envoy Filter Example.
building using 8 CPUs
building for x86_64
clang toolchain with libstdc++ configured
Generating compilation database...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
INFO: Analyzed 3758 targets (1466 packages loaded, 92970 targets configured).
INFO: Found 3758 targets...
INFO: Elapsed time: 36.237s, Critical Path: 0.58s
INFO: 1 process: 1 internal.
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
WARNING: info command does not support starlark options. Ignoring options: [--@com_googlesource_googleurl//build_config:system_icu=0]
Running a full clang-tidy
python3: can't open file '/opt/llvm/share/clang/run-clang-tidy.py': [Errno 2] No such file or directory

authors: arvyshka auni53 derekargueta gyohuangxin lizan zyfjeff


text:
[test] Fix handling of Timer::disableTimer when using simulated time

Commit Message: test: Fix handling of Timer::disableTimer when using simulated time on timers that are disabled between the time they are found to have fired and when the actual event_activate triggers.
Additional Description:
Risk Level: n/a test library changes
Testing: unit
Docs Changes: n/a
Release Notes: n/a

/assign @jmarantz

Alyssa -- you have some familiarity with sim-time, especially its limitations. Do you want to take a look at this?

Senior reviewer should arguably  be a non-googler, but I'm going to assume no one cares because it's a buffix and only test code.  I'm up for taking a look after master merge.

Senior reviewer should arguably be a non-googler, but I'm going to assume no one cares because it's a buffix and only test code. I'm up for taking a look after master merge.

Merged
Pushed without verify since 'tools/code_format/check_format.py fix' seems to be making formatting changes in random places which seem unrelated to this PR.

source: gharchive/pull-request

authors: alyssawilk antoniovicente jmarantz


text:
[fuzz] fix router filter bugs

Commit Message: Fixes two router filter related fuzz bugs.
Additional Description:

The first doesn't catch empty type urls in the protos, which leads to a null dereference. This PR uses config utility functions to get the factory for InternalRedirectPredicateFactory which will properly handle (throw) the error in the constructor.
The second is a case where router filter returns StopIteration due to a bad request and sends a local reply to stop decoding, but the fuzzer did not handle direct responses and continued to decode data.

Risk Level: Low
Testing: Corpus entries added as regression.
Fixes:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=23161
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=23045
Signed-off-by: Asra Ali asraa@google.com

/cc @penguingao for the InternalRedirectPredicateFactory change -- just in case the choice not to use config utility was intentional

friendly ping @htuch


text:
Windows: Fix ssl_socket_test

Commit Message:
Windows: Fix ssl_socket_test

SmallReadsIntoSameSlice test: use write method on connection instead
of transport socket so SSL_write failures will be properly retried,
also otherwise empty writes from connection will trigger failed
assertions in buffer::linearize
When SSL shutdown is not appropriate, perform basic socket shutdown;
relevant when client certificate is invalid and connection should be
dropped by server, however we still need to ensure TLS alerts are
able to be read by client, shutdown on socket ensures this can happen
Replace \r with "" when comparing test fixtures to results to ensure line
endings are ignored

Windows needs to open in Binary mode, text mode can cause Windows to
stop reading a file if 0x1a is seen

Risk Level: Low
Testing: Locally on Windows
Docs Changes: N/A
Release Notes: N/A
Fixes https://github.com/envoyproxy/envoy/issues/13191

cc @envoyproxy/windows-dev

/assign @PiotrSikora

Need to make binary mode open optional/refactor some tests
/wait

Looks good besides a small fix. Thanks a lot for getting this one to work!

I believe the openssl fork errors are either caused by the Windows update service, or MSYS2 issues, hopefully addressed by one of: https://github.com/envoyproxy/envoy-build-tools/pull/101 or https://github.com/envoyproxy/envoy-build-tools/pull/100

See https://github.com/envoyproxy/envoy/pull/13276
Looks like https://github.com/envoyproxy/envoy-build-tools/pull/100 is required to solve this issue in the build container/RBE

/wait
on https://github.com/envoyproxy/envoy-build-tools/pull/100

/wait
waiting on https://github.com/envoyproxy/envoy/pull/13367

Failing due to same reason this test is being marked as flaky for the time being: https://github.com/envoyproxy/envoy/pull/13407

/wait
still waiting on resolving the openssl msys2 fork issue, may continue with just checking in certs etc.

Blocked on whichever https://github.com/envoyproxy/envoy/pull/13702 or https://github.com/envoyproxy/envoy-build-tools/pull/107 is chosen as the solution

unblocked by https://github.com/envoyproxy/envoy/pull/13702

/retest

@PiotrSikora this should be ready to go again, macos CI failure looks like one of the systematic ones we've been having generally

merged master again to hopefully get the non-windows flakes out, @PiotrSikora have you had a chance to take a look at this yet?

authors: davinci26 dio sunjayBhatia wrowe


text:
For invert header match rule, when the header is not present and header

For invert header match rule with any header match type other than present, and the header is not present, always consider it as a match.
Signed-off-by: Yuhao Liu yuhaoliu@google.com

Commit Message:
Additional Description:
Risk Level: Low
Testing: Unit test
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue] #21828
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

@envoyproxy/api-shepherds  in case of this API change need to be discussed, seems like this undefined behavior in the past.

Please fix the format. I suggest you running the format script ./tools/code_format/check_format.py fix && ./tools/code_format/format_python_tools.sh fix according to https://github.com/envoyproxy/envoy/blob/main/support/README.md

/wait on CI and review comments :-)

I think this makes sense based on what was implied by the existing field comments, but I'd like to check with @markdroth @dfawley @ejona86 that the gRPC set of clients have/want the same semantics here.

Yes for the last commit I ran the ./ci/run_envoy_docker.sh './ci/do_ci.sh fix_proto_format script. Does it fix all the format issues?
Update: I think I found why the script didn't work. Let's try it again

@yuhaoliu77 It failed due to runtime flag required the a changelog https://dev.azure.com/cncf/envoy/_build/results?buildId=111549&view=logs&j=55bc2ab9-36c1-5ab8-58a6-8c258728eeff&t=33b57bf9-ef13-58c8-bc2b-f804aeb16d39&l=64
you can add changelog here https://github.com/envoyproxy/envoy/blob/main/changelogs/current.yaml

@yuhaoliu77 It failed due to runtime flag required the a changelog https://dev.azure.com/cncf/envoy/_build/results?buildId=111549&view=logs&j=55bc2ab9-36c1-5ab8-58a6-8c258728eeff&t=33b57bf9-ef13-58c8-bc2b-f804aeb16d39&l=64
you can add changelog here https://github.com/envoyproxy/envoy/blob/main/changelogs/current.yaml

Ah, thanks for the pointer!

Thanks for the update! This LGTM, waiting for the maintainer and API people to feedback.

(I see Mark has replied while I wrote this. I've written this in parallel.)
Given that HeaderMatch is used in RBAC, it seems inaccurate to label this as "Risk Level: Low".
gRPC duplicated Envoy's current behavior:

If a header is not present, HeaderMatch will not match except for present_match when present_match == invert_match. This is because HeaderMatcher.invert_match inverts the comparison operation so something like exact_match changes from an == comparison to a != comparison, and both fail to match if the header is not present.

I wouldn't make this change willy-nilly. This is the sort of thing that the expected behavior can differ per-person and even per-situation. This is a common problem with ternary logic (I'm looking at you SQL and NaN). I wasn't necessarily a fan of the current behavior, but it is conceptually consistent.
I will note that with either interpretation of invert the matcher is equally powerful as you can do a header presence check before whatever match operation. This is more about convenience and expectations.

I suggest the following alternative: Leave the behavior of invert_match as-is, but add the following new field:
// If true, a missing header will be treated as if it was present and its value is the empty string.
bool treat_missing_as_empty = N;

I think this gives us the best of both worlds:

If treat_missing_as_empty is false (the default), we continue to get the current behavior: If the header is not present, then invert_match is used only for the present_match matcher; all other matchers return false, regardless of the value of invert_match.
If treat_missing_as_empty is true, then we get the new behavior: We treat the header as present with value being the empty string, and then apply the matcher (regardless of type) to that value, and then we apply invert_match to that result.

I think the only real difference here is that if the matcher does actually match the empty string (e.g., a regex match allowing the empty string), then the matcher will match and invert_match will make it not match, but that may actually be a feature instead of a bug.
WDYT?

I think this is a great idea. This way we can keep the default behavior to make it backward compatible, also allows easy switching to the new behavior. So I plan to go ahead and make a change to use a feature flag called treat_missing_header_as_empty_for_invert_match, and make it default to disabled. WDYT?

So I plan to go ahead and make a change to use a feature flag called treat_missing_header_as_empty_for_invert_match, and make it default to disabled. WDYT?

To be clear, I think this should be a new API field, not a runtime flag.  Note that runtime is an Envoy-specific concept; xDS clients like gRPC do not have any equivalent, but we'll presumably want this new behavior to be possible for all clients.  Also, it's important for clients to know what a control plane is instructing them to do, which means that the control plane needs a way to encode its intent in the xDS resources it sends.
Also, I think the field should not be specific to invert_match.  I think it should just be treat_missing_as_empty, and if true, it should be honored even if invert_match is false.  That way, it can also be used to (e.g.) use a regex match that matches against either an empty value or a specific value.

So I plan to go ahead and make a change to use a feature flag called treat_missing_header_as_empty_for_invert_match, and make it default to disabled. WDYT?

To be clear, I think this should be a new API field, not a runtime flag. Note that runtime is an Envoy-specific concept; xDS clients like gRPC do not have any equivalent, but we'll presumably want this new behavior to be possible for all clients. Also, it's important for clients to know what a control plane is instructing them to do, which means that the control plane needs a way to encode its intent in the xDS resources it sends.

Thanks, good to learn this!
Also checks the doc of headers again:
A match will happen if all the headers in the route are present in the request with the same values (or based on presence if the value field is not in the config).
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-msg-config-route-v3-routematch
It seems like the current behavior is documented if I didn't parse it wrong.
So the envoy API doc should be the truth of the source. For anything is not document or undefined in the API doc, we shouldn't use the runtime flag. We only should use runtime flag when the envoy implements a wrong behavior compared to the API doc. Did I understand that correctly?

Also, I think the field should not be specific to invert_match. I think it should just be treat_missing_as_empty, and if true, it should be honored even if invert_match is false. That way, it can also be used to (e.g.) use a regex match that matches against either an empty value or a specific value.

For both treat_missing_as_empty and present_match are true, I guess it will consider the header always present, right?

For both treat_missing_as_empty and present_match are true, I guess it will consider the header always present, right?

This is a good point. If we strictly follow the definition of treat_missing_as_empty, any header would be present so it is always a present_match? Thoughts on the preference here?

I just committed a version that strictly follows the description of "treat_missing_header_as_empty", which means the header will always be considered as "present" when this option is set to true. Let me know your thoughts and we can change the behavior if needed.

Good to have @markdroth help to ensure the API again.

cc @htuch for API pass
/retest

@markdroth can you take a look from gRPC side and LGTM if this makes sense?

/retest

/lgtm api
Thanks, this looks great!

/wait

/retest

/retest

authors: alyssawilk bryanwux ejona86 htuch markdroth soulxu yuhaoliu77


text:
mobile: allow overriding dns cache key value save interval

In Lyft's usage, we have up to 14 calls per second to save DNS results to the key value store. Buffering these to save at most once per second should be sufficient.
Commit Message: mobile: set dns cache key value save interval to 1s
Additional Description: In Lyft's usage, we have up to 14 calls per second to save DNS results to the key value store. Buffering these to save at most once per second should be sufficient.
Risk Level: Low
Testing: In Lyft's apps
Docs Changes: Added
Release Notes: Added
Platform Specific Features: N/A

Oops, marked as ready to review too quickly, tests are failing. I'll update that.
I'll expose a builder parameter to set the interval since different apps might want different values. For us, 1s is fine and 60s would be too long, since storing the results in a sqlite db should be very efficient with our kinds of data, and we have app sessions shorter than 60s.

@RyanTheOptimist are we using the DNS cache yet for YT? If not I'd be inclined to suggest we switch the time interval now, and TODO the builder change for 2 weeks just because there's a lot of builder changes in flight
if it'd affect us now, I'll deal with the merge conflicts :-P

I've wired this through all the builder paths to keep things in sync as much as possible and minimize thrash with the proto builder changes.

Looks great! Should we have a test for the C++ builder too?

Yes, I'll add that.

authors: alyssawilk jpsim


text:
add StarryVae to go extension code owner

Commit Message: add StarryVae to go extension code owner
Additional Description: N/A
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A

@doujiang24 could i apply to become a code owner of Go Filter? thanks. ❤ cc @wbpcode @mattklein123

thanks!


text:
[balsa] Backport line folding support to 1.28

Commit Message: [balsa] Backport line folding support to 1.28
Additional Description: Backport #31080 (test-only) and #31168 to release 1.28 to support obsolete line folding in BalsaParser
Risk Level: low
Testing: test/common/http/http1:codec_impl_test
Docs Changes: n/a
Release Notes: Enable obsolete line folding in BalsaParser.
Platform Specific Features: n/a
Balsa implementation tracking issue: #21245

/assign @yanavlasov

@bencebeky could you add the changelog on main also please

@bencebeky could you add the changelog on main also please

Thanks for the reminder. In progress at #32423.

authors: bencebeky phlax


text:
mobile: Remove HTTP server code from TestJni

This PR removes the HTTP server code from TestJni. The tests have been updated to use either HttpTestServer or HttpProxyTestServer.
Risk Level: low (tests only)
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: mobile

/retest

/retest

/retest


text:
bug_fix: modified HTTP and gRPC clients to support response status on client error

Description: Ext_Authz HTTP client has been modified so that 5xx errors received from the authorization server will set the filter response status to error instead of denied and HTTP status code field to Forbidden. The gRPC client has been also modified in order to return HTTP status code Forbidden whenever an error between the client and the authorization server occurs.
Risk Level: low
Testing: unit tests, manual tests.
Docs Changes: not needed.
Fixes issue: https://github.com/envoyproxy/envoy/issues/4124.
Signed-off-by: Gabriel gsagula@gmail.com
cc @dio

@dio I added some more details to the docs and did some optimations around the authorization response. Please feel free to take another look when you get a chance. Thanks for reviewing it!

@ramaraochavali Thanks for reviewing it. Please, feel free to take another look at when you get a chance.

@dnoe yes, please. Thanks!

@dnoe @ramaraochavali Whenever you get a chance, please let me know if you have any question or if there is anything else that I should change. Thanks!

@gsagula I am good with the changes. Thanks.

@dnoe, should we invite @htuch to review this? 🙂

@htuch Thanks for looking at the code. I will work on it this weekend.

@htuch Sorry for the delay on this one. Feel free to take another look. Thanks!

authors: dio gsagula ramaraochavali


text:
OIDC filter randomly failing in v1.2 due to missing information in token request

Hi. With envoy gateway version 1.1.2 (envoy version 1.31.2) all my OIDC filters were working fine. I havent touched any of my SecurityPolicies and updated to 1.2.1 (envoy version 1.32.1) and now I get a OAuth flow failed error in the browser with oauth.missing_credentials in the envoy logs. When I enable debug logging I additionally see my IDP (Microsoft Entra ID fka AAD) complaining about a missing client_id in the POST body during the OIDC token request: AADSTS900144: The request body must contain the following parameter: 'client_id'.
I further troubleshooted this by doing a fresh envoy gateway installation in version 1.2.1 on a separate cluster and gradually applying configuration to it. While doing this I noticed that apparently OIDC is working fine in the beginning and then fails at some point. Furthermore, this seems to happen quite randomly: Sometimes a certain OIDC filter is working, then after some time of usage or after applying some unreleated envoy resources to the cluster it stops working and after some time it might even recover and be working again. Generally I have the feeling that the more different OIDC filters are in place on a cluster the more likely it is to cause this issue.
This issue might be related to #4625 which describes similar behavior with a different IDP, but suspects the nonce as the cause.
#4706, which is about a general instability of OIDC (in conjunction with JWT authorization), could be related as well.

@jaynis Could you please share your SecurityPolicy (Sensitive info can be extracted)? Do you use OIDC with JWT?

@zhaohuabing I use the following configuration which I dont consider special and which has been working fine in previous envoy gateway versions. Also, as I have mentioned before, this configuration initially works even with version 1.2.1 and then stops working after some point. Havent figured out yet when exactly, but adding further SecurityPolicys and HTTPRoutes might cause this.
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: dashboard-security-policy
  namespace: tools
spec:
  oidc:
    clientID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    clientSecret:
      kind: Secret
      name: dashboard-security-policy-oidc-secret
    provider:
      issuer: https://sts.windows.net/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    redirectURL: https://dashboard.example.com/oidc/callback
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: dashboard-route
---
apiVersion: v1
data:
  client-secret: cmVkYWN0ZWQK
kind: Secret
metadata:
  name: dashboard-security-policy-oidc-secret
  namespace: tools
type: Opaque
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: dashboard-route
  namespace: tools
spec:
  hostnames:
  - dashboard.example.com
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: envoy-gateway-https
    namespace: default
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: dashboard-service
      port: 8080
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /

@jaynis Thanks for providing the SecurityPolicy. Also if you can use egctl config envoy-proxy all to get the Envoy configuration form the time the OIDC is failing, it would be  helpful for finding the root cause.

It may be caused by this: https://stackoverflow.com/questions/68838592/azure-aadsts900144-the-request-body-must-contain-the-following-parameter-clie

we are facing same issue in multiple clusters OAuth flow failed
with egctl I can see following
          "dynamicWarmingSecrets": [
            {
              "lastUpdated": "2024-11-18T07:55:08.679Z",
              "name": "oauth2/client_secret/securitypolicy/demo/demo-auth",
              "secret": {
                "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
                "name": "oauth2/client_secret/securitypolicy/demo/demo-auth"
              },
              "versionInfo": "uninitialized"
            },
            {
              "lastUpdated": "2024-11-18T07:55:08.679Z",
              "name": "oauth2/hmac_secret/securitypolicy/demo/demo-auth",
              "secret": {
                "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
                "name": "oauth2/hmac_secret/securitypolicy/demo/demo-auth"
              },
              "versionInfo": "uninitialized"
            }
          ]

then in envoyproxy logs
[2024-11-18 08:00:30.500][1][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret
[2024-11-18 08:00:30.500][1][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret

rollbacking envoy gateway makes everything working.

@jaynis could you check do you see similar behaviour in your installation?

@plnordquist as you did that other ticket, can you check your setup as well. Do you see similar in envoyproxy logs and with egctl?

we are facing same issue in multiple clusters OAuth flow failed. From debug logs I can see that envoyproxy tries to fetch token from token endpoint using credential that does not contain client_secret at all client_id is there. So I was wondering why secret is not there:
with egctl I can see following
          "dynamicWarmingSecrets": [
            {
              "lastUpdated": "2024-11-18T07:55:08.679Z",
              "name": "oauth2/client_secret/securitypolicy/demo/demo-auth",
              "secret": {
                "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
                "name": "oauth2/client_secret/securitypolicy/demo/demo-auth"
              },
              "versionInfo": "uninitialized"
            },
            {
              "lastUpdated": "2024-11-18T07:55:08.679Z",
              "name": "oauth2/hmac_secret/securitypolicy/demo/demo-auth",
              "secret": {
                "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
                "name": "oauth2/hmac_secret/securitypolicy/demo/demo-auth"
              },
              "versionInfo": "uninitialized"
            }
          ]

then in envoyproxy logs
[2024-11-18 08:00:30.500][1][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret
[2024-11-18 08:00:30.500][1][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret

rollbacking envoy gateway makes everything working.

@zetaab This looks like different than the one reported by @jaynis and caused by a failure to initialize xDS secret. Are there any errors/warnings in the EG logs when this happened?

@zhaohuabing I went through the commit list in release/v1.2 and tracked the breaking commit which is https://github.com/envoyproxy/gateway/pull/4227
before that everything works fine, and with that pr commit I start to see errors in logs and oidc will not work.

Hi @zetaab . I can see the same things with egctl and in the logs. However, as I described before, in my case the identity provider complains about a missing client_id and not client_secret. What is the error message you get from your identity provider (and which one do you use)?
@zhaohuabing I have a config dump created with egctl config envoy-proxy all. Is there anything you are particular interested in? If I need to provide the entire file I have to go through 4700 lines of JSON and redact sensitive information.

@jaynis I use cognito and will get back that my client_id is not working. However, with debug log I can see that client_id is there and client_secret is empty

@zetaab Thanks for digging into this. https://github.com/envoyproxy/gateway/pull/4707

but yes, with 1.2.1 I can see
% kubectl logs envoy-gateway-59849c8649-rwr7c|grep "xds cluster exists"
2024-11-18T09:03:30.919Z        ERROR   xds-translator  runner/runner.go:85     failed to translate xds ir      {"runner": "xds-translator", "error": "xds cluster exists"}
2024-11-18T09:03:30.920Z        ERROR   watchable       message/watchutil.go:56 observed an error       {"runner": "xds-translator", "error": "xds cluster exists"}

with last working commit 5375cf0
% kubectl logs envoy-gateway-79dfddcf7b-v8czx|grep "xds cluster exists"|wc -l
       0

breaking commit a351c4b (PR #4227)
% kubectl logs envoy-gateway-8688459cbc-ptkn5|grep "xds cluster exists"|wc -l
       2

This should be fixed by https://github.com/envoyproxy/gateway/issues/4706.

I can confirm that #4707 will fix this issue at least for me

@zhaohuabing I have a config dump created with egctl config envoy-proxy all. Is there anything you are particular interested in? If I need to provide the entire file I have to go through 4700 lines of JSON and redact sensitive information.

@jaynis You can only share the output egctl config envoy-proxy listener.

The missing client ID is the same issue I faced as well @zhaohuabing

@zhaohuabing I have attached the egctl config envoy-proxy listener output here. This dump has been created with v1.2.1 and the issue being present.
Seemingly #4707 fixes the issue for me as well. But as already described initially, there is some randomness involved and therefore it is hard to say this for certain.
Is there any ETA for merging #4707 respectively for v1.2.2?

Is there any ETA for merging https://github.com/envoyproxy/gateway/pull/4707 respectively for v1.2.2?
We'll try to relese v1.2.2 by the end of this week.

@jaynis @zetaab @missBerg Could you please confirm if the latest EG image resolves your issue?
State encoding isse has been fixed by https://github.com/envoyproxy/envoy/pull/37473 .

@zhaohuabing yeah we are using 1.2.3 without issues. However, there is/was another issue which prevented us running multiple controllers. Its maybe fixed already, cannot find it now

@zhaohuabing yeah we are using 1.2.3 without issues. However, there is/was another issue which prevented us running multiple controllers. Its maybe fixed already, cannot find it now

I think it's this one, and it should be fixed by  #16306.

authors: jaynis missBerg zetaab zhaohuabing

repo: envoyproxy/gateway


text:
decision: Segmenting API Gateway features into PolicyAttachment CRDs

Description:
Adding options for how API Gateway features such as ratelimiting can be expressed as PolicyAttachment CRDs. Please dont focus on the names, just added placeholders.

one generic policy CRD with features as top level fields

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: Policy
 .....
 spec:
   authentication:
   .....
   ratelimiting:
   ....

one generic policy CRD with networking  (shapes traffic) and security (authenticates & authorizes traffic) as top fields

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: Policy
 .....
 spec:
   security:
     authentication:
     ....
   networking:
     ratelimiting:
     .....

2 Policy CRDs - one for networking and the other for security

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: SecurityPolicy
 ....
 ....
 spec:
   authentication:

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: NetworkingPolicy
 ....
 spec:
   ratelimiting:
   ....

1 CRD per API Gateway feature

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: Authentication
 ....
 spec:
   jwt:
   ....

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: Ratelimiting
 ....
 spec:
   ....

@Xunzhuo @danehans @youngnick @skriss @LukeShu @AliceProxy can you please vote on this issue

I prefer 3 or 4, they have more independence and loose coupling between features.

I vote for 2. or 3. and here's why

1 or 2 CRDs are easier to maintain for the project than multiple
The user can express their intent in 1 instantiated resource (useful when team is small) or multiple ( useful when multiple teams author intent)

I think 3 would be preferable over 1 and 2 although I prefer 4 the most. Allows for clear separation of resources depending on what the user is trying to accomplish. Prioritize keeping things self contained. While you could group two features into a single group like networking / security I feel that when a user has the goal of setting up a ratelimt they shouldn't need to worry about what other resources exist and then check to see if they have ratelimits configured on them and need to be edited vs. extended versus creating a new resource.
IE: I'm a cluster admin and I know that I want to setup authentication, I shouldn't have to use kubectl for some other resource and then check which of those resources configure authentication vs. a different security feature. I should just be able to quickly list all the resources in the cluster by the feature I'm trying to configure and get a rough idea from the info via kubectl get that I dont need to dig into any thing more in depth to figure out what my cluster config is.
Makes it easier for us to iterate over each individual feature separately. Not sure that the further grouping of a bunch of different things together is really benefiting the user but I'd like to hear any counter arguments.

Also, see the Policy Boilerplate section which uses MUST language around the default and override fields for Policies. The hierarchical nature and defaulting/override behavior is a core concept of the Policy resource. Using more focused resources IMO leads to a lot less complexity for implementers and users when taking this into account.

Policy Boilerplate

I think the complexity of merging individual policies is the same for 3. and 4.

just gives the flexibility of merging personas into one (a small independant team) or allowing
individual personas to author and instantiate many policy resources

apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: SecurityPolicy
 ....
 ....
 spec:
   authentication:
   .....
   cors:
   .....
   default:
     ......
     authentication:
     .....
     cors:
     ....

I think 3 would be preferable over 1 and 2 although I prefer 4 the most. Allows for clear separation of resources depending on what the user is trying to accomplish. Prioritize keeping things self contained. While you could group two features into a single group like networking / security I feel that when a user has the goal of setting up a ratelimt they shouldn't need to worry about what other resources exist and then check to see if they have ratelimits configured on them and need to be edited vs. extended versus creating a new resource.
IE: I'm a cluster admin and I know that I want to setup authentication, I shouldn't have to use kubectl for some other resource and then check which of those resources configure authentication vs. a different security feature. I should just be able to quickly list all the resources in the cluster by the feature I'm trying to configure and get a rough idea from the info via kubectl get that I dont need to dig into any thing more in depth to figure out what my cluster config is.
Makes it easier for us to iterate over each individual feature separately. Not sure that the further grouping of a bunch of different things together is really benefiting the user but I'd like to hear any counter arguments.
IE: kubectl get ratelimits let's me roughly see the ratelimiting config for the cluster and then I can dive into a specific one if needed based on the previous info.
Vs. kubectl get policy I can see there are a bunch of policies configured for the cluster but I have to go into more depth with each of them to figure out what all is being configured.

thanks for raising the argument around retrieving current intent.

from a production point of view, the intent will version controlled / git and checked before being edited
from a live state perspective, I can understand the case when the user only wants to retrieve the ratelimit block having a command like kubectl get ratelimit is easier than kubectl get trafficpolicy, but there are also cases (and I think this is more of a common case then edge case) where this same user owns and cares about multiple other traffic/networking properties  such as loadbalancing and prefers all features being clubbed into the same field.

there's also the case where the user doesnt remember the names of all the CRDs, and wants to remember the bare minimum to achieve their goal, so they might not be able to / or want to remember the exact names of each CRD such as ratelimiting, loadbalancing, waf, cors

xref https://github.com/envoyproxy/gateway/issues/675 as @sunjayBhatia has spent considerable time researching this topic for Contour. As I stated here, I suggest EG focus on extensions that can be implemented as implementation-specific filters. EG can add Policy Attachment support at a future date to support overrides and defaulting.

Regarding https://github.com/envoyproxy/gateway/pull/529#issuecomment-1299536714, my understanding is that EG maintainers achieved consensus for expressing extended functionality through separate resource kinds. Let's use this comment to formalize the vote. Maintainers, please use a thumbs-up or thumbs-down emoji for expressing extended functionality through separate resource kinds, e.g. separate Custom Resource Definitions.

+1 on 4

I would also +1 on 4 (non-maintainer), that gives each feature independence and they can evolve in their own pace as well.

+1 for 2 and 3 as they align with well with networking and security persona

2 and 3 as they align with well with networking and security persona

Note that 4 meets this requirement while supporting more granular RBAC.

xref #675 as @sunjayBhatia has spent considerable time researching this topic for Contour. As I stated here, I suggest EG focus on extensions that can be implemented as implementation-specific filters. EG can add Policy Attachment support at a future date to support overrides and defaulting.

for features like auth it might need to be enforced, so policy fits better for that case imho
the deprecation for policy and extension filter will look similar once the features have been graduated into GAPI, so no difference from that perspective

for features like auth it might need to be enforced, so policy fits better for that case IMHO

True, but AuthN "enforcement" should not be conflated with providing EG users with a request authentication mechanism. I agree that "enforcement" should be achieved through GWAPI Policy Attachment but as a separate backlogged feature.

the deprecation for policy and extension filter will look similar once the features have been graduated into GAPI, so no difference from that perspective

IMHO GWAPI Policy Attachment is still evolving (see xrefs) while implementation-specific filters are far more mature and ready to implement today.
xrefs:

https://docs.google.com/document/d/1p_kmAOmqebX8zyHnXW5I58cD8ZhNX4QNSa_gw8uAYxw/edit?usp=sharing
https://github.com/kubernetes-sigs/gateway-api/discussions/1498

thanks everyone for sharing your votes and comments, realized that chatting about PolicyAttachment might be too premature, and will require us to first agree on which extension points we should be using and when, hoping we can use #675 for aligning together about which extension type to use.

So, seems like I missed the discussion here. I'm a strong +1 for option 4, even with the cost it will impose on EG development, because (in addition to @AliceProxy and @sunjayBhatia's excellent arguments):

I think the Policy objects should be called _Envoy_RatelimitingPolicy, _Envoy_AuthNPolicy, _Envoy_TimeoutPolicy, etc. We are the official Envoy Gateway API implementation, and work we do to define Envoy-specific policies can then be useful for other projects that are Envoy based. Using an Envoy prefix makes this a bit clearer.
As I think I mentioned before, it's much easier to take a single object and discuss it upstream, since each object is tightly focussed.

That said, I think that having custom Filters or other extensions that can be configured in a hierarchical way using Policy objects seems like the most flexible option. (I'm going to remove the restriction on this in the Gateway API docs very soon).

+1 on 4

thanks everyone for sharing your preference, the majority of the maintainers  have voted for option 4  (1 CRD per API Gateway feature)

@youngnick reg your comment about naming, my opinion is that we should not add any Envoyisms into the name or the feature API. The ideal goal would be to build platform agnostic APIs that can be moved into the Gateway API repository in the future. Any Envoyisms should be part of the advanced mode Envoy API

Thanks @arkodg, but I disagree. Almost everything we do for Ratelimiting, Auth etc will be heavily influenced by how Envoy configures things anyway, so we should lean into making ours the Envoy versions, and then taking those to the community and seeing what can be made generic.
Rob and I spent days looking into how to write a generic Timeout policy across implementations, and it is impossible. Each datapath implementation thinks of timeouts very differently, and they are a very simple use case by comparison to ratelimiting etc.
Doing Envoy-ism versions first lets us both show how the Policy framework can be used, and gives our users (and the users of other Envoy datapath implementations) quicker access to this functionality. It also prevents name collisions later, in the happy case that we can make a generic version.

^ would also be consistent with existing EG API naming, EnvoyGateway and EnvoyProxy.

If we do move forward with PolicyAttachment, separate CRDs should be used.

authors: AliceProxy Xunzhuo arkodg danehans lizan sunjayBhatia youngnick zinuga zirain


text:
ENYO-2265 _observeDisappearance() is called multiple time with one 5 …

…way move
Add condition to compare old and new object before calling _observeDisappearance()

Closing in favor of https://github.com/enyojs/spotlight/pull/196.

authors: aarontam yeram

repo: enyojs/spotlight

license_source: bigquery


text:
Update scalafmt-core to 3.3.1

Updates org.scalameta:scalafmt-core from 2.7.5 to 3.3.1.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "scalafmt-core" } ]

labels: library-update, early-semver-major, semver-spec-major, commit-count:1

Superseded by #179.

repo: enzief/recursionz


text:
Finetuning eole with OpenNMT-py model checkpoints

Hi,
I am experimenting with eole training for bilingual MT models. I can successfully train a model with eole from scratch using wmt17 recipe. However I also want to see if I can fine-tune my legacy opennmt-py models with eole. With opennmt, it was possible to load a model checkpoint using train_from parameters. But for eole it seems from code that it expects a model directory with checkpoint, config.json and vocab.json files. Is there a way to convert opennmt checkpoint files (.pt) to this required format?
Thanks

not so easy but read this: https://github.com/eole-nlp/eole/discussions/76

repo: eole-nlp/eole

license: MIT


text:
Change link text to readme template source code

What does this PR do?

makes for an SEO friendly link to markdown source code page

Steps to test

Go To https://guias.eoscostarica.io/docs/herramientas/plantilla-readme
See link

for guides, do the same?

for guides, do the same?

yes please!


text:
Docs/flight methology

What does this PR do ?
Document flight methodology

@Mike3563  the name of the file is incorrect docs/flight-methology.md

@Mike3563  we are also going to need to link to this page from another page in order for people to find it , it can also be included as a sidebar menu item


text:
Performance Rig Large Runners

From engineering issue 62, this epic is to obtain or create "large runners" (beefy computers) that meet engineering's needs to perform reliable and reproducible performance benchmarks.

The full set of steps.

Need to work with engineering leadership on the proper hosting solution for large runner
Setup the new hosting solution
Automate via cron
Store stats from runs

New runners are a significant body of work and should be tracked as an independent epic.

When you have rough estimates on cost and frequency of runs, we should meet to discuss impacts.

repo: eosnetworkfoundation/engineering


text:
It's not possible to select parts of different structures using the 'Shift' key.

Steps to Reproduce

Ketcher is launched.
Several structures are created on the canvas
Press the 'Shift' key and with any 'Select' tool select parts of the created structures.

Expected behavior
Parts of different structures are highlighted in green.
Actual behavior
The selection is reset when a part of the next structure is selected.
Screenshots

Desktop:

OS: Windows 10
Browser: Chrome
Version: 100.0.4896.127

Ketcher version:

v2.4.2-219-g11bc5160

Additional information:
The issue is reproducible in 2.4. Behavior is as expected in 2.3.

Verified as fixed on Release 2.5.0.

authors: Svetlana-Asinovskaia ZhannaNovikova

repo: epam/ketcher


text:
Calcium atom is rendered incorrectly to and from SMILE string

Steps to Reproduce:

Create the structure from smile-string: Ca(C)C
Click "Save As..." button.
Select "Daylight SMILE" in drop down list.

Expected Result:
p.1 - Correct structure with Ca atom in center displayed.
p.3 - Ca(C)C displays in text field.
Actual Result:
Center of structure displays "A" instead "Ca".
When User try to save structure in .smi file "C*(C)C" displays.

Issue is not a bug, format of SMILE string in steps incorrect.
If User convert correct string Ca(C)C - application works correctly.
Was approved by @AlexanderSavelyev 11/1/2021


text:
Doesn't Work on my website

Hello Dear,
I added it to my website. http://www.videoondemands.club/faucet/ after completing installation process I got the admin passphrase.
Then I deleted install.php and install.sql files.
Then I change the permissions of
ads to 777
template_c to 777
configs to 777
but now when i continue further it redirect me to http://www.videoondemands.club/faucet/adm/install.php
instead of http://www.videoondemands.club/faucet/adm/login.php or http://www.videoondemands.club/faucet/adm/index.php
and also when I visit http://www.videoondemands.club/faucet/ it redirects me toward http://www.videoondemands.club/faucet/install.php
but the install.php file I already deleted from hosting.
Please help me regarding this issue

Did you change permissions recursive ?

I have the same problem on  What should be done?

license: mit


text:
Device rescope

This change reduces the scope of pvi to just producing UIs of PVs scraped from template files, regrouped based on existing UIs and then tweaked and maintained by hand.
The produce is left in place for possible future work, but is now only used to extract class and parent class names from a header file to populate the device.yaml. The tests have been removed for now.
The process will now be that driver code and templates will be left untouched and the device.yaml used and maintained in parallel. The steps to use pvi for a driver will be:

convert device <header> <template(s)> ...

-> device.yaml

regroup device.yaml <UI(s)> ...

-> Regrouped device.yaml

Check in and maintain device.yaml
format device.yaml <formatter> ...

-> device.bob

@coretl I think this is ready

repo: epics-containers/pvi


text:
CI ranges for "doubling/halving times" don't appear to include point estimates.

See (for example): https://epiforecasts.io/covid/posts/subnational/united-states/maryland/

Doubling/halving time (days): -17 (29 – -6.9)

This looks to be reporting a CI of [-6.9,29], which does not include the point estimate of -17. It appears that the true CI that should be reported is (-Inf,-6.9] U [29,Inf), and that the naive view of the "reported" CI is actually a 5% CI.
This mostly looks like a typographical issue, arising from a denominator which isn't behaving nicely.

Hi @cdowd,
Thanks for this. As you note this is because the doubling time is a slightly odd metric in that when it inverts to the halving time it goes through infinity. If you read the CI with that in mind it is technically correct but I agree confusing.
I'm very open on other ways to represent this. Could you just clarify your suggestion with a few examples?
(I've opened an issue in the package we use to produce these estimates as any change needs to be made there).

I'm afraid I don't have any suggestions much better than using standard set union notation. E.g. as I wrote above

(-Inf,-6.9] U [29,Inf).

I suppose an alternative would be to split out doubling and halving times separately, e.g.

CI: Halving time (-Inf,-6.9], Doubling time [29,Inf).

If those are both unpalatable, a simple solution would be to throw an asterisk on the CI being given, and note that it goes through infinity. E.g.

Doubling/Halving time: -17 (29 - -6.9)†
† Intervals where the order is reversed (e.g. 1 - -1 instead of -1 - 1), include everything except what is between the numbers shown.

This would perhaps be the simplest of the solutions, as it can be tacked on purely at the end and only need affect the doubling/halving time CI calculations. Though the adjacent dash and minus symbols are still somewhat confusing. Not sure how easy it would be to replace the dash with the word "to" (in both the text and the examples of the footnote), but that would then completely resolve things.

authors: cdowd seabbs

repo: epiforecasts/covid


text:
Strict config option; match exception type to access pattern

Maybe eventually pypiper can leverage the AttributeDict currently in looper.models; for now, this is a simple patch to match/use some of its behavior.

ok, great. did you run python test_pypiper.py ?
no need to convert pypiper unit tests at this point but at least use what we already have I guess

Did now! Sorry, I don't think I'd ever run 'em for pypiper before, and I hadn't thought to look. :ok_hand:

Closing in light of https://github.com/epigen/pypiper/issues/41

authors: nsheff vreuter

repo: epigen/pypiper


text:
Move self-hosted runners to GCP

...this is action item from 05/05/23 retro
to prevent SUSE R&D network rate limiting issues for various services (gh, registry...) it would be nice to run our runners outside of this network.
https://confluence.suse.com/pages/viewpage.action?spaceKey=CECO&title=Auto-scaling+GitHub+Self-Hosted+Runners
Given that Sebi invested an effort to do runners cleanup it should be discussed with him first.

After reviewing the elemental docs and our epinio repositories, I came to the following conclusion:
We were using self-hosted runners more extensively, since they were considered free, fast, aso.
However, when we switch those to use GCP resources, we need to pay for, it doesn't make sense for lean jobs, e.g. linter.
So the plan is to

Narrow down self-hosted runner usage and switch to use ubuntu-latest where possible, before
Prepare VM/runner images according to elemental docs
Have a first workflow using autoscaled GCP runners in parallel to test it

Narrow down self-hosted runner usage within epinio repositories - created:

https://github.com/epinio/epinio/pull/2297
https://github.com/epinio/helm-charts/pull/425
https://github.com/epinio/ui-backend/pull/26

authors: svollath thehejik

repo: epinio/epinio


text:
Display better message when not finding default epinio settings.yaml file

Issue
Currently if default epinio settings.yaml file is not found the message displayed is:
❌  error retrieving Epinio environment information: making the request: Get "/api/v1/info": unsupported protocol scheme ""

Request
It would be nice to say something as  "settings.yaml" file not found in /home/mmartin/.config/epinio/ or similar
To reproduce:

remove /home/mmartin/.config/epinio/settings.yaml  from your respective path
run epinio info

Verified in v1.10.0-rc1
Before
$ rm -r ~/.config/epinio/
$ epinio-1.9 info

❌  error retrieving Epinio environment information: making the request: Get "/api/v1/info": unsupported protocol scheme ""

After
$ epinio-1.10 info

❌  error retrieving Epinio environment information: Client settings not found. Please ensure that the cluster is running, Epinio is installed, and the client is logged in.