text:
MSID: 91359 Version: 1 DOI: 2023.09.12.557326

[PLACE MANUSCRIPT AND EDITOR DETAILS HERE WHEN AVAILABLE] See step 3
[PLACE PDF URL HERE WHEN AVAILABLE] See step 7
Step 1. Inform bioRxiv
Who can help: @QueenKraken, @nlisgo, @scottaubrey

[x] Manuscript is in data hub index (https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=91359)

Step 2. Create preview of manuscript
Who can help: @fred-atherden, @nlisgo, @scottaubrey

[ ] Manuscript is available for preview (https://prod--epp.elifesciences.org/preview/91359/v1)

[ ] Trigger import for staging: https://prod--epp.elifesciences.org/import
[ ] Sync data folder with s3://prod-elife-epp-data/data
[ ] Trigger import for staging: https://staging--epp.elifesciences.org/import
[ ] Sync data folder with s3://staging-elife-epp-data/data
[ ] Pull request created to enhanced-preprint-data
[ ] Post link to pull request below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-manuscript
Step 3: Awaiting public reviews and QC
Who can help: Production team

[ ] QC passed or QC not needed

Who can help: Editorial team

[ ] Public reviews available on sciety (https://sciety.org/articles/activity/10.1101/2023.09.12.557326)
[ ] Manuscript and editor details at the top of this issue (Supplied by Editorial team)
[ ] Reviews returned by EPP - https://prod--epp.elifesciences.org/api/reviewed-preprints/91359/v1/reviews

Example
"msas": "Genetics and Genomics", "Neuroscience"
"msid": "84628"
"version": "1"
"preprintDoi": "10.1101/2022.10.28.514241"
"articleType": "Reviewed Preprint"
"status": "Published from the original preprint after peer review and assessment by eLife."

"Reviewed Preprint posted": "2023-01-02"
"Sent for peer review": "2022-10-28"
"Posted to bioRxiv": "2022-11-21" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2022.10.28.514241v1")

Editors:
Reviewing Editor
Michael B Eisen
University of California, Berkeley, United States
Senior Editor
Michael B Eisen
University of California, Berkeley, United States

Step 4: Modify manuscripts.json (no PDF)

[ ] Reviewed preprint is published to EPP (https://elifesciences.org/reviewed-preprints/91359)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[ ] Request that a doi be registered

[ ] Post link to #enhanced-preprint comment below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]

Instructions to modify manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/publish-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/334/files
Once the pull request is merged in it should be available a few minutes later.

Request that a doi
Post the following in #enhanced-preprint:
@Fred can you register a doi for https://elifesciences.org/reviewed-preprints/91359

Step 5: Awaiting search reindex

[ ] Reviewed preprint is avaliable on journal homepage (https://elifesciences.org)

The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.

Additional info
If needed, the jenkins pipeline to reindex search can be triggered sooner.
https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 6: Published! Request PDF generation

[ ] Post the link to the PDF url at the top of the issue

[ ] Post link to #enhanced-preprint comment below

#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]
Post the following to the #enhanced-preprint on slack:
@Ryan Dix-Peek please can you generate a PDF for https://elifesciences.org/reviewed-preprints/91359

Step 7: Introduce PDF to data folder and git repo

[ ] PDF is avaliable at https://github.com/elifesciences/enhanced-preprints-data/raw/master/data/91359/v1/91359-v1.pdf

Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-pdf
Step 8: Add PDF url to manuscripts.json

[ ] Reviewed preprint PDF is available for download (https://prod--epp.elifesciences.org/reviewed-preprints/91359)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[PLACE LINK TO PULL REQUEST HERE]

Instructions to add PDF url to manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/add-pdf-url-to-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/397/files
Once the pull request is merged in it should be available a few minutes later.

Step 9: Done!

[ ] Kettle is on!

(Issues to resolve)

authors: acollings nlisgo


text:
AgGrid filtered row extraction

how could be possible to extract data from and AgGrid only those rows that are still visible after the user set a column filter, or a combination of them?

like how is it described here for the javascript framework:
https://stackoverflow.com/questions/43542998/how-to-get-the-number-of-filtered-rows-in-ag-grid

Please see the example below:
import justpy as jp
import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/fivethirtyeight/data/master/alcohol-consumption/drinks.csv', encoding="ISO-8859-1")

def grid_change(self, msg):
    print(msg.data)
    msg.page.df = jp.read_csv_from_string(msg.data)
    c = msg.page.df.jp.plot(0, [1,2,3,4], kind='column', classes='m-2 p-2 w-2/3 border', title='Alcohol Consumption per Country')
    msg.page.c.options = c.options

def grid_test():
    wp = jp.WebPage()
    wp.df = df
    wp.c = df.jp.plot(0, [1,2,3,4], kind='column', a=wp, classes='m-2 p-2 border', title='Alcohol Consumption per Country')
    grid = df.jp.ag_grid(a=wp)
    for event_name in ['sortChanged', 'filterChanged', 'columnMoved', 'rowDragEnd']:
        grid.on(event_name, grid_change)
    return wp

jp.justpy(grid_test)

msg.data contains the filtered grid in CSV format

impressive!
Thank you!

source: gharchive/issue

authors: elimintz giodegas

repo: elimintz/justpy

license_type: permissive

license_source: github-api


text:
Support migrations in subdirectories

Hello, it gets pretty quick, that you end up with lots and lots of files within the priv/repo/migrations directory.. while the application gets bigger, changes are happening, so the single directory approach does not scale well
It might be helpful, when subdirectories were supported.
priv/repo/migrations/accounts/
priv/repo/migrations/dependency_migration/
ect.
This PR includes

finding migrations within subdirectories
define the subdirectory (domain) with the (-d, --subdir) option to generate a migration inside the subdir

Could you add some more context what's exactly the problem you're facing? The sheer number of migration files? Too many migrations added at the same time? (e.g. many developers working concurrently on the project.)
For the first problem, I'd suggest to regularly (e.g. every 3-6 months) collapse multiple migration files into a single one - I've used this a couple of times and it worked well. For the second problem, come up with naming conventions to organise it (e.g. <timestamp>_<subsystem>_<name of the migration>)
Thoughts?

The other option is completely ditch migrations and use ecto.load to load a
dump of the production database structure.
José Valim
www.plataformatec.com.br
Skype: jv.ptec
Founder and Director of R&D

@wojtekmach my specific use case, is that I have an umbrella project, where I have a repo application, and different applications which are using it. I think there are possibilities to move each migration into the corresponding app, but for now, it would be already helpful, if I could group the migrations by the corresponding app/context/domain. So I looked at the code, and found, thats only working the flat directory.
Sure, there are many possibilities to organize migrations like naming conventions. Still you have a lot of files ordered by the timestamp only. My suggestion is, to put them inside domains, so this would make it also easier to squash them into a single file later, or create a dump like suggested by José. But still keep them inside domains. Not forced, but to have that option would be nice, I dont see any downside.
Further, if some other bright minds like the idea, we could create phoenix schemas and put the migrations into the corresponding domain (by the context), which I think makes sense.

The other option is completely ditch migrations and use ecto.load to load a
dump of the production database structure.

@josevalim the data structure mutates over time, how to handle updates?

@webdeb in the same way, whenever you want to discard more migrations, you generate a new dump. This is also very useful in development because if you have too many migrations going with a dump will may speed up the process.

@josevalim its ok to ditch the -d option, its an API thing..
But, would you be ok with this change:
https://github.com/elixir-ecto/ecto_sql/pull/67/files#diff-c160bf108a9ad548fa1f2a8280bad4e2R395
- query = Path.join("migrations", "*")
+ query = Path.join(["migrations", "**", "*.exs"])

Just to make it possible to structure/group migrations inside the migrations folder.

source: gharchive/pull-request

authors: josevalim webdeb wojtekmach


text:
Fix documentation and validation improvements

While reading the docs I've found some misleading documentation from column conditional creation.
Refactor precision type validation logic after finding a similar situation from this refactor.
The above point lead me to uncovered code, so I added a little test to ensure validation on add_if_not_exists.

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:

authors: josevalim pmargreff


text:
Binary encoding bit and geometry types on MariaDB

Currently we can't encode bit and geometry types on the binary protocol on MariaDB servers, when we attempt that we get null as the result.
Looks like it's a problem on the server, because if we encode these types as binaries (as opposed to their respective types):
diff --git a/lib/myxql/protocol/values.ex b/lib/myxql/protocol/values.ex
index 7250e57..b64e5c7 100644
--- a/lib/myxql/protocol/values.ex
+++ b/lib/myxql/protocol/values.ex
@@ -212,7 +212,7 @@ defmodule MyXQL.Protocol.Values do
     size = bit_size(bitstring)
     pad = 8 - rem(size, 8)
     bitstring = <<0::size(pad), bitstring::bitstring>>
-    {:mysql_type_bit, encode_string_lenenc(bitstring)}
+    {:mysql_type_var_string, encode_string_lenenc(bitstring)}
   end
␠
   def encode_binary_value(true) do
@@ -246,7 +246,7 @@ defmodule MyXQL.Protocol.Values do
     defp encode_geometry(geo) do
       srid = geo.srid || 0
       binary = %{geo | srid: nil} |> Geo.WKB.encode!(:ndr) |> Base.decode16!()
-      {:mysql_type_geometry, encode_string_lenenc(<<srid::uint4, binary::binary>>)}
+      {:mysql_type_var_string, encode_string_lenenc(<<srid::uint4, binary::binary>>)}
     end
   end

it works!
The problem with this patch however is because we're encoding them as binaries in certain queries where the type is unknown we lose the type information that we otherwise already had:
Before this patch:
iex> MyXQL.query!(pid, "SELECT ?", [%Geo.Point{coordinates: {1, 1}}]).rows
[[%Geo.Point{coordinates: {1.0, 1.0}, properties: %{}, srid: nil}]]

After this patch:
iex> MyXQL.query!(pid, "SELECT ?", [%Geo.Point{coordinates: {1, 1}}]).rows
[
  [
    <<0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 240, 63, 0, 0, 0, 0, 0, 0,
      240, 63>>
  ]
]

Of course this is not a problem when the query does have the column information, e.g. we select
data from a given column.
I'm a bit hestitant to move forward with this. But maybe losing this behaviour is not gonna be a
big issue in practice?
I also submitted this as a bug upstream.

Since the bug is reported upstream, I'm gonna close this for now.

See my comment upstream: https://jira.mariadb.org/browse/MDEV-21150?focusedCommentId=171564&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-171564.

repo: elixir-ecto/myxql


text:
Update the readme (filetype?)

I am using neovim and nvim-treesitter
I followed the advanced setup for installing a non-maintained parser

Appears installed:

Treesitter lua conf:
local parser_config = require "nvim-treesitter.parsers".get_parser_configs()
parser_config.iex = {
  install_info = {
    url = "~/projects/tree-sitter-iex", -- local path or git repo
    -- url = "git@github.com:elixir-lang/tree-sitter-iex.git", -- local path or git repo
    files = {"src/parser.c"}
  },
--  filetype = "zu", -- if filetype does not agrees with parser name
--  used_by = {"bar", "baz"} -- additional filetypes that use this parser
}

I think the problem has to do with the filetype, once we sort this I would be happy to add some extra docs into the readme about how to install/use this.

:wave: hello!
I tried out integrating the grammar with nvim with those instructions and I think there are some pitfalls you may want to check.
The first from that adding parsers section:

Note that neither :TSInstall nor :TSInstallFromGrammar copy query files from the grammar repository. If you want your installed grammar to be useful, you must manually add query files to your local nvim-treesitter installation.

So I copied queries/highlights.scm from this repository into ~/.local/share/nvim/plugged/nvim-treesitter/queries/iex/
The next issue I ran into is that nvim-treesitter does injections very differently from tree-sitter-cli, so the injections.scm in this repository won't work. Instead I tried some like
(evaluation_block (prompt_line (expression) @elixir))
(result) @elixir

which works reasonably well, but has some limitations: nvim-treesitter does not currently implement "combined" injections, so multi-line expressions like
iex> if true do
...>   :ok
...> end
:ok

will show the end as unhighlighted. Based on a recent PR, it looks like one of the primary nvim-treesitter maintainers is interested in solving that problem soon.
The last issue is filetype as you mention. You'll need some vimscript so nvim knows that *.iex files should use tree-sitter-iex. Here's how it works for elixir, but according to these docs (the third paragraph), you can try :set filetype=iex and that should get the highlights working. Since iex isn't really a file-type, though, I'm not sure it makes sense to setup a rule to mark *.iex as iex. This gammar is mostly meant for injections like in markdown blocks.

I would be happy to add some extra docs into the readme about how to install/use this.

In general I would prefer to keep installation/usage instructions for specific editors out of the readme since the grammar can be used by multiple editors. IMO troubleshooting/installation should be in the editor/plugin's docs since it varies so much between editors (as in this issue, :point_up: all of those pitfalls are nvim-treesitter specific).
Ideally it'd be great to have this grammar added to nvim-treesitter. I don't use nvim regularly so I probably shouldn't be the maintainer though. Would you be interested in opening a PR in nvim-treesitter to integrate this grammar?

Wow, thanks for your messages that was quick!  I really appreciate your answers 💯 .
Regarding your second comment, yeah it makes sense, specifics should not be added in this repo's readme.
I am happy to help, I will come back to you 😉 .

I am going to park this for the time being, I use nix to configure my environment and I am struggling to test locally. So far, if someone is interested:

I installed neovim and it's plugins with home-manager. Fins here my dot-files.
Then I tried to find the derivation where tree-sitter modules are installed.

nix-store --query --requisites /run/current-system

Home-manager installs nvim treesitter plugin in a derivation that looks like this:
/nix/store/kw4m0krl18kh2wfdhjnkazd3kd8dqnvg-vim-pack-dir/pack/home-manager/start/vimplugin-nvim-treesitter/

Iex appears as installed:

But when I set the filetype it does not parse as I would expect 😄.

authors: gilacost the-mikedavis

repo: elixir-lang/tree-sitter-iex


text:
Filter sensitive data from inspect(Tesla.Client.t)

Hi :wave:
We recently discovered - luckily before going to production - that we leaked secret tokens for an API integration, as we were doing something like this
{:ok, _} =
  [{Tesla.Middlewares.BearerAuth, token: "supersecret"}]
  |> Tesla.client()
  |> Tesla.post(...)

The resulting MatchError in case of network failures etc. would include the inspected Tesla.Client struct which in turn included the supersecret token.
We thought implementing Inspect within Tesla itself and filtering some options would be a more sensible default for all users, hence this PR.
Best,
malte
Todos

[ ] Add Tesla.Middleware.BasicAuth.Options struct
[ ] Add Tesla.Middleware.BearerAuth.Options struct
[ ] Add Tesla.Middleware.DigestAuth.Options struct

that we leaked secret tokens for an API integration

Do you mind expanding on that, do you mean you leaked to some 3rd party provider?

I am not opposed to the idea.
The curious part is that this issue would be all over Elixir code bases, so I am not sure what is the right call here.
As far as I can tell you would modify your logger to hide sensitive information, but I may be wrong.
Gonna lean on @teamon for some feedback about it.

I tend to have a custom implementation of Inspect protocol on a per-project basis, often something as simple as:
defimpl Inspect, for: Tesla.Client do
  @moduledoc """
  Implement custom Inspect protocol for Tesla.Client
  to prevent leaking secrets added with Tesla.Middleware.Headers
  to AppSignal or logs.
  """

  import Inspect.Algebra

  def inspect(client, opts) do
    attrs = client |> Map.take([:adapter]) |> Map.to_list()
    concat(["#Tesla.Client<", to_doc(attrs, opts), ">"])
  end
end

The problem with implementing it directly into tesla package is that as far as I know one wouldn't be able to override it anymore.

A solid argument is that Ecto does the same https://github.com/elixir-ecto/ecto/blob/d52039037ae8d3ba66034c35c08fc4d56c621447/lib/ecto/changeset.ex#L3134-L3172 (from Kyle Steger)

A solid argument is that Ecto does the same https://github.com/elixir-ecto/ecto/blob/d52039037ae8d3ba66034c35c08fc4d56c621447/lib/ecto/changeset.ex#L3134-L3172 (from Kyle Steger)
but then as @teamon pointed out, what happens when we have other middleware that wants to accomplish the same? We can't redact anymore something?

It's possible to write another middleware, Tesla.RedactValues or something of the like that takes a list of keys to redact. It looks like Tesla.post|patch|etc accept [option], which may be an OK place to add redact_keys: [:a, :b, :c].
I think what is proposed here provides immediate benefit and prevents known secret data from being leaked in logs. Gets a thumb in my book

@yordis

Do you mind expanding on that, do you mean you leaked to some 3rd party provider?

Leaked to our application monitoring as part of its exception handling.

it directly into tesla package is that as far as I know one wouldn't be able to override it anymore

How about hiding it behind a compile-time flag? I'd assume that the vast majority of users do not have a custom implementation, so defaulting it to true would make sense, IMO. Safety first.
config :tesla, implement_inspect_protocol: false

Also, as a side note: We have since discovered that we also use the KeepRequest middleware in one case, so we would also need to purge the req_headers from the opts field of Tesla.Env :/ - We're currently debating whether we want to implement Inspect for Tesla.Env as well (and optionally upstream it, too), or whether we can get rid of KeepRequest in this case. Implementation of Inspect for Tesla.Env feels a bit brittle in this case. What do others think?

@yordis

As far as I can tell you would modify your logger to hide sensitive information, but I may be wrong.

That's unfortunately not possible in this case. Exception is raised, Appsignal inspect()s it and we can't really filter it anymore from the resulting string.

Safety first.

Agree with this, but having a solution when opting out without so much rework should be taken into consideration as well.

What do others think?

What if the input is an struct instead of a keyword list, then you implement the redacting for such struct, example:
{MyMiddlewareName, %MyMiddlewareName.Config{token: "..."}}

So you implement the protocol for MyMiddlewareName.Config
Would that work?
If that strategy work, we could decide to commit to what we have, and maybe recommend people to do such a thing if they want to control redacting things.
Thoughts?

Gentlemen, any update on this? I'm happy to rework it in any way you conceive, but I do believe the status quo is an unnecessary pitfall / security risk for unaware people.

@maltoe sorry it took me this long to get back to this issue, but I have other duties.
From my perspective, the tricky situation with this PR is that we are taking over control of the Inspect for the entire Tesla.Client.
From the library perspective,
How do we ensure that this implementation of the Inspect protocol will work for all middleware (including the ones we did not write)?
That may not be a concern, but I am unsure.
This leads us to two possible solutions,

In your application code, you define the Inspect protocol (less ideal)
Use a struct as configuration (see my changes)

IMPORTANT
I pushed some changes showcasing the usage of Tesla.Middleware.BasicAuth.Options, which I made the rookie mistake of force-push because I rebased (my bad).
If you wish to continue helping in the subject, we must do the same for Tesla.Middleware.BearerAuth and Tesla.Middleware.DigestAuth thus far.
Let me know if you can help with the task. Otherwise, I can take over.

@yordis

sorry it took me this long to get back to this issue, but I have other duties.

Sorry if my last comment came across too pushy. I'm sure you have and thank you for your work on tesla, it's an awesome tool :hammer_and_wrench: :purple_heart:  !

From my perspective, the tricky situation with this PR is that we are taking over control of the Inspect for the entire Tesla.Client.
From the library perspective, how do we ensure that this implementation of the Inspect protocol will work for all middleware (including the ones we did not write)?

Personal opinion: I'm wondering whether we're "overthinking" it - the original code in this PR was my attempt at fixing the "leak" while leaving some useful information in the inspect output. We have since refined it in our codebase and even expanded it to Tesla.Env (req_headers in case KeepRequest is used). But I'm more sceptical about its inclusion in Tesla itself now, as it has become a non-negligible amount of code, still doesn't cover all cases (e.g. third-party middlewares as you mentioned), and especially as I can't remember ever having needed the information in the inspect output at all.
Hence, how about going back to what @teamon originally said in https://github.com/elixir-tesla/tesla/pull/534#issuecomment-1164551535 and providing an extremely basic default implementation (e.g. hiding all middlewares including the internal ones) with an opt-out? That way you prevent leaks even from external middlewares, protect the innocents, and people don't have to rework much at all when they opt-out :grimacing:
Otherwise, I also like your struct approach (though req_headers in Tesla.Env needs to be considered as well). Unfortunately, I don't know how much time I'll be given to work on this any time soon, so can't promise much.
Thanks again for considering & best wishes.

Coming back to this,
We should refrain from doing anything about this, which is a fundamental problem for Elixir. In the best case, a scapegoat exists, and changing the behavior of the Inspect is good enough; adding [redacted] is the tricky situation here, as anything in the stack could be printed out.
This is also why Logger filters exist: You can clean up and remove things there to ensure that you do not leak information.
I am not opposed 100% yet to trying to fix it, but looking at the code and whatnot, I am somewhat thinking of pushing the responsibility to copy+paste @teamon example or simply use the existing tool from Elixir, such as the Logger filters to deal with the situation

I've recently ended up with something like this:
defimpl Inspect, for: Tesla.Client do
  @moduledoc """
  Implement custom Inspect protocol for Tesla.Client
  to prevent leaking secrets added to headers
  """

  @conceal [
    {Tesla.Middleware.BasicAuth, [:password]},
    {Tesla.Middleware.BearerAuth, [:token]},
    {Tesla.Middleware.DigestAuth, [:password]}
  ]

  def inspect(client, opts) do
    client =
      Map.update!(client, :pre, fn middlewares ->
        Enum.map(middlewares, fn {module, fun, [opts]} ->
          case @conceal[module] do
            nil ->
              {module, fun, opts}

            attrs ->
              {module, fun,
               Enum.reduce(attrs, opts, fn attr, acc ->
                 acc
                 |> Access.get_and_update(attr, fn val -> {val, "*****"} end)
                 |> elem(1)
               end)}
          end
        end)
      end)

    Inspect.Map.inspect(client, opts)
  end
end

Maybe there is a room for some Tesla.conceal function that could be used as the implementation if one want's to change the Inspect behaviour? (which is not the only way to do it as discussed and should not be changed by default)

authors: kyleVsteger maltoe teamon yordis

repo: elixir-tesla/tesla

license: MIT


text:
Crash on large traffic

Get Timeout of GenServer error when handling large traffic (thousands of queries). Without UAInspector.parse/1 it works fine.

Do you have some more details on that? Something like the actual timeout message you are seeing?

@mneudert
At this moment it falls with logs like:
[error] #PID<0.9719.0> running Test.Endpoint terminated
Server: 0.0.0.0:4000 (http)
Request: GET /test
** (exit) exited in: :gen_server.call(:ua_inspector_pool, {:checkout, #Reference<0.4279952074.2567176194.71000>, true}, 5000)
    ** (EXIT) time out

To reproduce it I made ~7k requests async with task for parsing user agents.

That message looks like an empty worker pool to me.
I suggest you try one of two possible solutions:

Circumvent pooling by directly calling UAInspector.Parser.parse("your user agent"). This should execute the lookup inside your own process. Please be aware that (at least for the moment) this call should be treated as internal and might change unexpectedly, so definitely include several tests to ensure you don't break your application on an update!
Pump up the pool worker numbers (you indirectly made me aware of a chunk of missing documentation on that...). Adding config :ua_inspector, pool: [max_overflow: 100, size: 1000] might help you. I can't give you any actual numbers but seeing you have a testing environment for the overload you should be able to get a good estimation on what might/should work. See the poolboy documentation for available options, they are passed unmodified.

Both of these should ease the problems you are seeing as long as the real issue is not down at the storage process. There are calls taking place to lookup an ets table identifier (currently not using an outside callable named table) but I don't expect those to issue the message you are seeing.

@mneudert
Thanks a lot!
After changing pool configuration seems logs are much better.
But sometimes there are errors also.
The first option doesn't make sense, because we'll need to write our own GenServer in that case. But it will be the same as yours.

The first option eliminates the need for a GenServer as it performs the lookup inside your process currently communicating with the lookup pool. The GenServer itself carries no state so it is only an artificial boundary used for logical separation and more reliable/expectable behaviour. Removing that boundary might work for you but also introduce other unexpected problems...

Btw, how to you test that overload scenario?
Is it "complicated" or as simple as something like this:
for i <- 1..10_000 do
  Task.async(fn -> UAInspector.parse("some user agent") end)
end
|> Enum.map(&Task.await/1)

The problem with that is that if your load is high enough there are far too many processes to execute within the (yet to be made configurable) 5 second timeout. If you happen to drain all the resources you have then skipping the pool might be the only way to handle the pressure...
Note: I assume you are already reducing the number of parsing requests by only doing it once for every new session and not on every single request. If there is no session you might want to keep the parsing results in a separate cache to not look up the exact same user agent multiple times.

Thanks for explaining, now we solved this problem just changing UAInspector.parse to UAInspector.parse_client. We don't need to save any info if it is a bot.

Btw, how to you test that overload scenario?

It's really "complicated" way to test, but we isolated it from other processes (as possible).

About cache: yes, we see that it's needed and will add it to project.
(We thought it already made inside UAInspector)

Closing due to age and as the removal of process pooling in v1.0.0 should have "some time ago" solved this issue.

authors: ivan-safonov mneudert pups3s

license_source: bigquery


text:
Error stack trace

If we add recovery-panic snippet like on example:
	// Error stack trace
	defer func() {
		if err := recover(); err != nil {
			err = fmt.Errorf("transpileToNode\nnode = %#v\nError: %v\n", node, err)
			panic(err)
		}
	}()

Maybe we will have a lot of error clarification.
Let's take for example of C source - kilo.c https://github.com/Konstantin8105/c2go-rating/blob/master/SingleCcode/kilo.c
Example of error output after adding in file transpiler.go:
Before:
Error: panic: invalid identity: 'Not[]erow'

After
Error: panic: invalid identity: 'Not[]erow' [recovered]
	panic: transpileToExpr
node = &ast.UnaryOperator{Address:"0x2cd8748", Position:"line:688:9, col:10", Type:"int", IsLvalue:false, IsPrefix:true, Operator:"!", Children:[]ast.Node{(*ast.ImplicitCastExpr)(0xc42058f020)}}
 [recovered]
	panic: transpileToStmt
node = &ast.IfStmt{Address:"0x2cd8a58", Position:"line:688:5, line:691:5", Children:[]ast.Node{ast.Node(nil), (*ast.UnaryOperator)(0xc42058ee40), (*ast.CompoundStmt)(0xc4203e19c0), ast.Node(nil)}}
 [recovered]
	panic: transpileToStmts
node = &ast.IfStmt{Address:"0x2cd8a58", Position:"line:688:5, line:691:5", Children:[]ast.Node{ast.Node(nil), (*ast.UnaryOperator)(0xc42058ee40), (*ast.CompoundStmt)(0xc4203e19c0), ast.Node(nil)}}
 [recovered]
	panic: transpileToStmt
node = &ast.CompoundStmt{Address:"0x2cd9088", Position:"col:30, line:699:1", Children:[]ast.Node{(*ast.DeclStmt)(0xc4202915c0), (*ast.DeclStmt)(0xc420017240), (*ast.DeclStmt)(0xc4200c5880), (*ast.IfStmt)(0xc4203e1000), (*ast.BinaryOperator)(0xc42058f9e0), (*ast.CallExpr)(0xc42000ecd0), (*ast.IfStmt)(0xc4205a2180), (*ast.UnaryOperator)(0xc4202e0d20)}, BelongsToSwitch:false}
 [recovered]
	panic: transpileToStmts
node = &ast.CompoundStmt{Address:"0x2cd9088", Position:"col:30, line:699:1", Children:[]ast.Node{(*ast.DeclStmt)(0xc4202915c0), (*ast.DeclStmt)(0xc420017240), (*ast.DeclStmt)(0xc4200c5880), (*ast.IfStmt)(0xc4203e1000), (*ast.BinaryOperator)(0xc42058f9e0), (*ast.CallExpr)(0xc42000ecd0), (*ast.IfStmt)(0xc4205a2180), (*ast.UnaryOperator)(0xc4202e0d20)}, BelongsToSwitch:false}

So, let`s analyze:

We have panic with body : Error: panic: invalid identity: 'Not[]erow'
Before that panic we in func transpileToExpr with input data:

panic: transpileToExpr
node = &ast.UnaryOperator{Address:"0x2cd8748", Position:"line:688:9, col:10", Type:"int", IsLvalue:false, IsPrefix:true, Operator:"!", Children:[]ast.Node{(*ast.ImplicitCastExpr)(0xc42058f020)}}

and more...
Is acceptable to adding that error stack trace?
I hope it is helpful for analasing the real code error in project https://github.com/Konstantin8105/c2go-rating .
Minus

Error is very large
Error stack trace don't clarify errors from function func (p *Program) String() string{...} https://github.com/elliotchance/c2go/blob/master/program/program.go#L161

Plus

We have large error stack trace for analyse

We should remove anywhere that a panic happens when processing code. These were left over from how lazy error handling used to be done. The goal now is to always produce Go output and try to provide as useful warnings as possible in the output file.
How about we look into why these panics are occurring in the first place?

Now, we don't use panic-recovery pattern (in according to our last discussions), so, that issue is not actual for myself.
If it is not actual for you, then please close.

license: mit


text:
Set a session-level statement timeout per connection

Tunable via POSTGRES_STATEMENT_TIMEOUT and defaulted to 1 minute to match prod.
Counterpart to https://github.com/ello/ello/pull/2109.
[#146710477]

Oh, @jayzes the environment variable needs to be added here so we have access to it: https://github.com/ello/apex/blob/master/elixir_buildpack.config

@alanpeabody ah right - done in https://github.com/ello/apex/pull/123/commits/ed9a8415002ab3db977db5c9c6e2cd86c8493257

authors: alanpeabody jayzes

repo: ello/apex


text:
add support for modules in the site.json

We can do modular theme work the same way we do the dynamic imports stuff. This is the same paradigm / data structure except this will just load and append elements of that type to the DOM when everything is setup. This will allow for slapping on / enabling data analytics (for example) and beginning the path towards modular front end functionality.

merging with #204

repo: elmsln/HAXcms


text:
Pretty TimeStamp

Is there any way to change default timestamp from epoch to date string
1486381070403|D|X-LOG|Simple message
to
Mon, 06 Feb 2017 11:37:50.403 GMT|D|X-LOG|Simple message

@Isanderthul Take a look at ClassicFlattener or PatternFlattener, and specify the log flattener using FilePrinter.Builder.logFlattener(Flattener). With Flattener, you can totally customize the format of your log.

Can we create an API or param to easily change the format of the timestamp without subclassing PatternFlattener?
We just want to set the format of the timestamp, the rest of the implementations are ok.

authors: Isanderthul elvishew iadcialim

repo: elvishew/xLog


text:
Could not read a texture of the Data URI Scheme form in Safari.

Safari で Data URI Scheme 形式のテクスチャが読み込めない
Safari で下記のようにテクスチャを読み込もうとするとエラーとなる。
var texture = new GLBoost.Texture('data:image/jpeg;base64,/9j/4AA...');

実行結果
Cross-origin image load denied by Cross-Origin Resource Sharing policy.

glTF ファイルの Embedded ファイル形式の時も、同様のエラーが出ていた為、恐らく同様の対策が必要と思われる。
参考：_loadBinaryFile()

https://github.com/emadurandal/GLBoost/issues/94

対応したのでクローズします。

repo: emadurandal/GLBoost


text:
2 general requests | 2 bugs

Hey,
thx for providing a working  owl carousel directive.
Some general requests

could you add a licence please?
could you add this to npm please?

... and now some bugs

currently you cannot attach callbacks (missing values in owlProperties list)
you cannot trigger  an event programmatically (no access to owl carousel api)

Pullrequest is on the way :)
update:
Pull request #2

@ManticSic hey there , thank you for pull request. i also added Mit licence and npm support.

authors: ManticSic emalikterzi


text:
Sha version in the output index.html?

I'd like a quick way to figure out what version is really showing up on the server and maybe add it to any bug reports from my users.
Is there a way to export the sha of the deployed version to the index.html file?

This could be pretty trivially implemented as a plugin in the upcoming 0.5.0 version. In general, I'm not sure where responsibility for this should lie between ember-cli-app-version and ember-cli-deploy.

Nevermind. It seems it is already listed in the ember inspector info tab. So I must have access to it in some way. Thanks!

authors: juggy lukemelia

repo: ember-cli/ember-cli-deploy


text:
update ember-string-helpers

This gets rid of the error:
DEPRECATION: ember-cli-babel 5.x has been deprecated. Please upgrade to at least ember-cli-babel 6.6. Version 5.2.8 located: your-addon -> ember-cli-addon-docs -> ember-cli-tailwind -> ember-string-helpers -> ember-cli-babel

Nevermind, this is using ember-string-helpers and not ember-cli-string-helpers.

repo: embermap/ember-cli-tailwind


text:
Cache jar files to build JavaDoc and RDoc on Travis CI

On Travis CI, downloaded dependent jar files are cached after script step before after_success step (see .travis.yml file). Thus jar files downloaded at after_success steps are not cached. after_success step downloads many large jar files including jruby-complete every time. This is inefficient.
A possible solution is to download the jar files at script step.

Travis CI has gone.

authors: dmikurube frsyuki

repo: embulk/embulk


text:
Can't run examples

In order to run examples I had to edit gradle wrapper from version 6.1 to 6.3 and I also had to update polkaJ version from 0.4.0-snapshot to 0.3.0 because the snapshot is not being hosted in bintray.

If you have a different version of the Gradle just use it as is. I.e. just gradle <command>.
Bundled ./gradlew wrapper script is useful, in my opinion, only when you don't have any Gradle installed so it installs it on the first run. Otherwise just use any version you already have.
You can run gradle install in root directory of the project to build and install snapshot locally, and 0.4.0-SNAPSHOT would work after that.

Thanks okay yes running gradle install now I can run examples with snapshot.  Running my installed gradle does work, but I do believe the wrapper files need to be updated to version 6.3 to work. At least for my configuration they do. I have java 15.0.1 on Big Sur and gradle 6.8

@splix any idea on the error for balance I posted?

Have resolved examples not running will be included in next pr

I ran into the same
This examples\common.gradle is misleading:
If we use SNAPSHOT versions, that requires running gradle install,
then repository entry is not needed.
//
// Common config for Polkaj Examples
//
ext {
   polkajVersion = "0.4.0-SNAPSHOT"
}

repositories {
   // polkaj public repo
   maven { url  "https://dl.bintray.com/emerald/polkaj" }
}

authors: nschwermann paulvi splix

repo: emeraldpay/polkaj


text:
Ask storage plugin for tarball URL when getting packages

When getting packages, full URL to tarball must be provided. Currently, it is provided by config and applied here. However, some packages might benefit from changing tarball download url, for example, @emeralt/storage-gcs, which may provide url to download directly from Google Cloud Storage.

It seems like there's no good solution for this. Asking storage plugin about download url on every package get would drastically slow down get endpoint, especially in the case of storage plugins like gcs that need to create a signed url.

repo: emeralt/emeralt


text:
Fix multiple partial updates of the same texture

introduces a queue for texture changes

Closes https://github.com/emilk/egui/issues/1337.

I think we should make it a Vec<(TextureId, ImageDelta)> for simplicity sake. Are you still planning on working on this, or should I take it over?

Oof. Vec::retain_mut seems to be a rather new thing. @emilk what do you propose?

I suggest:
self.delta.set.retain(|(x, _)| x != id);
self.delta.set.push((id, delta));

@emilk Please take a look again :)

repo: emilk/egui


text:
Issue with Drawer UI Freezing on iOS When Navigating Back via Gesture

Hello,
I've recently started exploring frontend development and decided to use https://github.com/suyalcinkaya/onur.dev as a base for my website. I've encountered an issue specific to iOS devices where the UI freezes momentarily after navigating to a different link using the drawer component and then returning to the original page via a swipe gesture. This freeze lasts about 2~3 seconds (see the attached screen shot) and is not observed on Android devices, suggesting it might be related to how iOS caches the page before navigation.
Here's a brief overview of the problem:
Platform Affected: iOS only.
Description: After using the drawer UI to navigate to a new page and then using a swipe gesture to return to the previous page, the UI freezes for approximately 5 seconds. This suggests that iOS might be caching the page's last state in a way that interferes with the smooth restoration of the UI.
Reproduction Steps:

Open the drawer UI.
Click on a link to navigate to a new page.
Use a swipe gesture to return to the original page.
Observe the UI freeze.

Here is the link to the drawer component's source code for reference.
Any insights, suggestions for fixes, or similar experiences would be greatly appreciated. Thank you for your time and help!

I had a similar problem for my PWA, I am using react so as a workaround solution, i useEffect a cleanup function that closer the drawer when component unmounts. So that when i navigate back the drawer is not opened

I had a similar problem for my react PWA, so as a workaround solution, i used useEffect's cleanup function to close the drawer whenever the component unmounts. So that when i navigate back the drawer is not opened.

Thanks, are you doing something like following? I try this, but it seems not working. BTW, I noticed that https://ui.shadcn.com/docs has the exact issue. the drawer UI will freeze when use the left menu drawer to navigate to new page then switch back to original page on ios.
import { useEffect, useState } from 'react';
import { Drawer } from 'vaul';
import { CommandIcon } from 'lucide-react';
import { MenuContent } from '@/components/menu-content';
import { Button } from '@/components/ui/button.jsx';

export function MobileDrawer() {
  const [isOpen, setIsOpen] = useState(false);

// use useEffect to clean up function to close the draw
  useEffect(() => {
    // This function gets called when the component unmounts
    return () => {
      setIsOpen(false); // Close the drawer
    };
  }, []); // Empty dependency array means this effect runs only on mount and cleanup runs on unmount

  const handleOpenChange = (open) => {
    setIsOpen(open);
  };

  return (
    <Drawer.Root shouldScaleBackground value={isOpen} onOpenChange={handleOpenChange}>
      <Button variant="ghost" size="icon" title="Toggle drawer" asChild onClick={() => setIsOpen(!isOpen)}>
        <Drawer.Trigger>
          <CommandIcon size={16} />
        </Drawer.Trigger>
      </Button>
      <Drawer.Portal>
        <Drawer.Overlay className="fixed inset-0 bg-black/40" />
        <Drawer.Content className="fixed bottom-0 left-0 right-0 mt-24 flex h-[80%] flex-col rounded-t-lg bg-gray-100">

            <MenuContent />

        </Drawer.Content>
      </Drawer.Portal>
    </Drawer.Root>
  );
}

Can you provide a demo with reproduction?

@emilkowalski
this is the screenshot of using https://ui.shadcn.com/docs as example. When open the menu drawer and navigate to a new page. If using gesture to switch back to previous page, the menu will appear and uncontrollable for about 1 second. My site has exact same behavior when using drawer to navigate.
https://github.com/emilkowalski/vaul/assets/9373761/75e5d162-7312-4dcf-bfd2-f730d6d25b5c

authors: aakashbhadana emilkowalski quickpanda

repo: emilkowalski/vaul


text:
Homework2 Grading

Please grade my homework!!! Thank you!!! @ryarza

Graded!

thanks


text:
Support pod security standard (PSS) on created namespace in emissary-ingress chart

Please describe your use case / problem.
Pod security standard (required to replace PSP since Kubernetes 1.25) should be able to be applied to the namespace created by the chart.
Describe the solution you'd like
Make namespaceLabels configurable alongside pod and deployment labels.
Additional context
Doc: https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-namespace-labels/
Willing to PR if desired.

HI @jasonstitt, thanks for highlighting this.  The #emissary-dev channel in our Slack is a good place to sound this out.
We also have a monthly Contributors meeting so that would be a great place to discuss this as well with our engineers.  We welcome contributions!

authors: cindymullins-dw jasonstitt

repo: emissary-ingress/emissary


text:
dtest_k3s: Work with newer cgroups

Description
If something's going wrong, don't silently try to retry repeatedly until go test finally times out and panics. Detect errors! Log things!
And that error detecting and logging gets us to the real fix: Work with Linux kernels with newer cgroups.
v2:

Rebased to be closer to master

Testing
When I stick a replace line in the Telepresence go.mod to point to this, the tests once again work on my laptop.
Checklist

[x] I made sure to update CHANGELOG.md. - no applicable changes
Remember, the CHANGELOG needs to mention:

Any new features
Any changes to our included version of Envoy
Any non-backward-compatible changes
Any deprecations

[x] This is unlikely to impact how Ambassador performs at scale. - should only affect the tests
Remember, things that might have an impact at scale include:

Any significant changes in memory use that might require adjusting the memory limits
Any significant changes in CPU use that might require adjusting the CPU limits
Anything that might change how many replicas users should use
Changes that impact data-plane latency/scalability

[x] My change is adequately tested. - no, but that's an existing problem with dtest being under-tested
Remember when considering testing:

LEGACY MODE TESTS DO NOT RUN FOR EVERY PR. If your change is affected by legacy mode, you need
to run legacy-mode tests manually (set AMBASSADOR_LEGACY_MODE=true and run the tests).
(This will be fixed soon.)
Your change needs to be specifically covered by tests.

Tests need to cover all the states where your change is relevant: for example, if you add a behavior that can be enabled or disabled, you'll need tests that cover the enabled case and tests that cover the disabled case. It's not sufficient just to test with the behavior enabled.

You also need to make sure that the entire area being changed has adequate test coverage.

If existing tests don't actually cover the entire area being changed, add tests.
This applies even for aspects of the area that you're not changing – check the test coverage, and improve it if needed!

We should lean on the bulk of code being covered by unit tests, but...
... an end-to-end test should cover the integration points

[x] I updated DEVELOPING.md with any any special dev tricks I had to use to work on this code efficiently. - no tricks

Any idea what's going on with the CodeQL-Build CI?

Now lives separately at https://github.com/datawire/dtest


text:
[v2.3] Prepare for all of Envoy's v2 APIs to go away in the future

Description
In the next major Emissary version 3.0.0, all of Envoy's v2 APIs will be going away due to upgrading Envoy.  This PR is making sure that both us and our users are ready for that.
The highlights are:

Rewrite t_logservice.py to actually test behavior, so that we can actually know whether a change we make changes Emissary's behavior. (40cf4607dfb80ed606873a365460a99046fa753f)
Add a bunch of changelog text about breaking changes. (6da9b4f665ba3541b0675af5ed370535e78f3946, c12dcf00644e64bad92c6fb5ce502775e6b01414)
Fix xDS-v3's support for Zipkin-v1, so that users who need the Zipkin-v1 protocol can get to Emissary 2.3-without-forcing-xDS-v2 in order to have a path to upgrade. (a41b100e68a7c897fd3f49e13d2a03cceba1e35e)
Some of the tests were being skipped for xDS-v3. Don't do that, it'll be the only version soon. The exception to this is HostCRDRootRedirectECMARegexMapping because regex_type: unsafe is just absolutely unimplementable with xDS-v3. (dae6ff165e1722117f0528a86d9ecbade3e6725d)
Just as AuthService and RateLimitService users need a version where where they can switch between transport proto v2 and v3 in order to have a smooth migration path to Emissary 3.0.0's v3-only, so do LogService users. Implement that. (39253f0eebfccbfd7c66dfad782465c8732f8229)

This doesn't actually drop xDS v2, since that'd break regex_type: unsafe users. But this makes sure that users will have a migration path (for everything but regex_type: unsafe) when we do.
As usual for me, I suggest a commit-by-commit review.
Related Issues
None
Testing
See the added/edited tests in the PR.
Checklist

[x] I made sure to update CHANGELOG.md. - yes
Remember, the CHANGELOG needs to mention:

Any new features
Any changes to our included version of Envoy
Any non-backward-compatible changes
Any deprecations

[x] This is unlikely to impact how Ambassador performs at scale.
Remember, things that might have an impact at scale include:

Any significant changes in memory use that might require adjusting the memory limits
Any significant changes in CPU use that might require adjusting the CPU limits
Anything that might change how many replicas users should use
Changes that impact data-plane latency/scalability

[ ] My change is adequately tested. - pretty good, but it could be better
Remember when considering testing:

Your change needs to be specifically covered by tests.

Tests need to cover all the states where your change is relevant: for example, if you add a behavior that can be enabled or disabled, you'll need tests that cover the enabled case and tests that cover the disabled case. It's not sufficient just to test with the behavior enabled.

You also need to make sure that the entire area being changed has adequate test coverage.

If existing tests don't actually cover the entire area being changed, add tests.
This applies even for aspects of the area that you're not changing – check the test coverage, and improve it if needed!

We should lean on the bulk of code being covered by unit tests, but...
... an end-to-end test should cover the integration points

[x] I updated DEVELOPING.md with any any special dev tricks I had to use to work on this code efficiently. - no tricks

[ ] The changes in this PR have been reviewed for security concerns and adherence to security best practices. - what does that even mean? This bullet item isn't a good addition to the checklist, it's not clear.

The lukeshu/release-notes branch has the release notes changes.


text:
Samouczek

Do prezentacji musimy przygotować samouczek

Rozbijmy to na frontend i UX

popatrzyłem na figmie też na desktop i jest git tylko brakuje wytłumaczenia przycisków na desktopie

A potrzebne są? Na desktopie domyślnie przyciski mają opis

a no w sumie niepotrzebne

spoko jest

authors: Michal-Czekanski emkarcinos mattyl006

repo: emkarcinos/WMIAdventure


text:
WHY

WHY
WHY WOULD YOU CREATE SUCH A HORRENDOUS CODE LANGUAGE?? WHO THE FUCK USES IT?? IN WHAT WAY, SHAPE, OR FORM, WOULD THIS BE A PRACTICAL LANGUAGE TO USE???
God has left the game.

I do. https://git.procrastination.life/Pako/Cancer

authors: MagnificentPako jennasisis

repo: emojicode/emojicode


text:
Plugins, Views and Data separation

This is a bit of "random" ramblings to figure out a design for this.
Right now several parts are being a bit glued together which is problematic extension wise.
There are no plugins for "views" which is something should be be fixed. This is to allow custom visualization plugins for example.
No view in HippoPlayer should be "special" unless there is a really good reason. For example right now the PlaylistView owns the PlaylistData. It would be nicer if the Playlist could be a global concept that plugins can both consume and mutate if wanted. In the general case consuming would be more common (to show the playlist in some different way for example) while the regular playlist view would be responsible for most of the common operations such as adding and removing files.
Another thing is output of audio playback is hard-coded to a single backend. This would be good to separate as well to make it easier to have plugins writing to other sources (such as to file or streaming to some service)
The main application should be responsible to handing out data / fetching data from the plugins but some of the data may only be consumed by the plugins themselves even if main application just sends them around.
So this brings us to have something like this
* Output plugins (audio hw, file, streaming, etc)
* View plugins (show stuff, has access to the UI, visualization)
* Decoder plugins (decode stuff to a format (right now f32) for nr of channels (2 right now)). Decoder plugins also need a way to show a config UI.
* File plugins (for packed data, the way I see this is there is a global VFS (virtual file system) where files can get "mounted" into (think *nix style) This should all be transparent to decoders

Some global concepts
Playlist
Audio frames
.. likely more stuff

Also some kind of message passing between the plugins. Not sure how this should work. I think starting explicit and not trying to be very general would be a good start.

This has more or less been implemented now

repo: emoon/HippoPlayer


text:
Main.py doesn't launch

Yooo
Got everything setup, pyton is installed to path, but when I launch main.py it just opens as a cmd and then close. Lemme know how to repair it.

saw some closed issues and its same as some people

repo: emppu-dev/roblox-decal-mass-uploader


text:
Change default for xdirect

Setting xdirect=False in bipole and dipole (calculating the direct field in the wavenumber-frequency domain) is much faster than having it at the default xdirect=True. Additionally, it is more consistent, as everything is calculated in the same domain.
Before change check if there are any issues setting xdirect=False by default.

The possible issues are edge-cases, but generally there shouldn't be a problem. Only if someone expects that dipole/bipole returns the analytical fullspace-solution for a fullspace then now xdirect has to be set to True, otherwise it is calculated in the wavenumber-frequency domain and transformed.

repo: empymod/empymod


text:
topic reflection

After https://github.com/nanomq/NanoNNG/pull/692 and sync nng

Better use msg dup

Better use msg dup.
Will have a deep look tmw

I took a look at the implementation of msg_dup and found that msg_dup would be alloc multiple times, causing a noticeable performance degradation
The current approach is to modify the packet topic information, which is simple and efficient
So both schemes have their advantages and disadvantages
If there is any benefit that I don't notice that only using msg_dup yields, wish to point it out

@wanghaEMQ Thanks for your suggestions, which are helpful.
I fix them one by one, please check again :P

authors: JaylinYu RanMaoyi

repo: emqx/nanomq


text:
Build failed on Python 3.6.5 - Gentoo

Hi,
I am getting error after issuing $pip install --user storm:
zangetsu@venus /var/lib/layman/archenroot/dev-python/ez_setup $ pip install --user storm
Collecting storm
  Using cached https://files.pythonhosted.org/packages/a7/4c/e5dbc818c432cd80d2e1aa1576f911eda49a3c206c6697db4070f402cda9/storm-0.20.tar.bz2
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-vddhlj_n/storm/setup.py", line 5, in <module>
        import ez_setup
      File "/tmp/pip-build-vddhlj_n/storm/ez_setup.py", line 106
        except pkg_resources.VersionConflict, e:
                                            ^
    SyntaxError: invalid syntax

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-vddhlj_n/storm/

Is this version supported or should I install older python alongside as well?
Thx.

Mistake, different package I issued storm instead of stormssh -> it works.

repo: emre/storm


text:
ncurses: include terminfo database in package

This modifies the ncurses recipe to include terminal database information, in other words runtime configuration files, in the share directory. This is needed for the upcoming vim recipe which needs to use the terminfo/x/xterm-256color to operate correctly. Rather than just building and shipping that one file, I'm including all of the database files as I expect more will be needed at some point in the future.
The changes to the recipe are at the configure stage, removing the --disable-database flag and specifying the --build and --host platforms. The latter are needed to ensure that it knows it is cross-compiling so that it uses the build platform's tool (called tic) to build the terminfo rather than the host's.
The packages bin directory now includes a couple more utilities and a new share directory which is 7.2 MB.
As far as I can tell ncurses is only used in the lua recipe so I don't think the additions here will affect any other recipes. But it would be good to get a second opinion that these changes are OK.

I originally packaged ncurses to use it to build readline, but there are no plans to build readline anymore so these changes should be fine.

Thanks @IsabelParedes, I will merge then.

repo: emscripten-forge/recipes


text:
Not able to find a download for pdfresurrect

I want to use pdfresurrect, but i am not getting any proper link to download, can you help me get that downloaded.

Hi @Hiteshsaai, I do not provide any built versions of pdfresurrect.  However, you might be able to find a version specific to your platform, via a package manager.  For instance apt-get install pdfresurrect

got it @enferex  thanks 👍

authors: Hiteshsaai enferex

repo: enferex/pdfresurrect


text:
🛑 Eirik Rolland Enger is down

In 79867db, Eirik Rolland Enger (https://eirikenger.xyz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Eirik Rolland Enger is back up in e71ccca.

repo: engeir/upptime


text:
Remove React.createFactory call on new React versions since React thr…

…ows warning about no render method defined when component is wrapped with createFactory
Warning: Component(...): No render method found on the returned component instance: you may have forgotten to define render in your component or you may have accidentally tried to render an element whose type is a function that isn't a React component.

@rikukissa 100% sure about that? When I was testing it earlier the rendering didn't work without that...

React.createFactory is a necessary wrapper for those not using JSX - however, the module consumer should be responsible for calling createFactory if they need to. Otherwise those using JSX will run into problems. Per Facebook, exporting factories is an anti-pattern that should be avoided, the class should be exported instead.

Yeah, when used without JSX everything surely works fine with createFactory, but with JSX you'll get the error I mentioned earlier. So it would probably be logical to export the barebones version of the component and let non-JSX users wrap the component with createFactory

Thank you!

authors: dandelany rikukissa yaycmyk

repo: enigma-io/react-input-placeholder


text:
Simplify test and getIndex fix for triply nested case

I'm late to your #723 party, @MartijnR and @FrankApiyo but did want to take a moment to understand what was going on since I know repeats are complex.
This PR is mostly meant to check my understanding. I think the simplifications proposed might also help us in the future if/when this code needs to be revisited but will be interested to hear whether others agree.
I also moved the test. In the repeats tests, there's a cloning test block that this new test was previously in but then there are many tests that appear related to cloning outside of that block, too. @MartijnR is there meaning to that organization or is it just that the block was forgotten about at some point? I moved the new test to after 'In an empty form it creates the first repeat instance automatically because there are other tests around there about nested repeats and the issue turned out to be about missing repeat instances on outer repeat creation.
Some additional clarifications:

My understanding of the bug is that because the info nodes were not being counted correctly (the count was increasing even if there wasn't an info node there), there were nodes that were supposed to be cloned that weren't. Is that a reasonable one sentence explanation? If not, what would one be?
getIndex computes an absolute index of repeat instances or repeat info of a particular name across the entire primary instance? What's the high-level reason for it? E.g. is it used to index into linear collections that keeps track of all instances of one repeat regardless of their nesting structure?
In the conversation around this issue, @MartijnR you mentioned some kind of recursive algorithm for getIndex. I don't see any recursion there, am I missing it? It seems it walks the node tree iteratively. The inner loop counts repeat instance that are siblings of the node that was passed in and the outer loop does the same count in any outer repeats that may exist. Did I get that roughly correctly?
I'm having trouble understanding the initialization of the count in getIndex.  I would expect that initializing using repeat-number and then counting previousElementSiblings would double count but clearly that's not the case. What am I missing?
@MartijnR mentioned that there might be value in splitting getIndex. This feels related to some questions we've run into around set/update for the records queue and saveRecord/submitRecord for online and offline views. My sense is that it's dangerous to have two functions that have largely duplicated code with subtle differences. It requires a lot of concentration to consistently make updates in both in parallel and it can be difficult for the call site to make the right choice. I've proposed alternate simplifications that hopefully increase readability without needing two functions.
Doesn't getInfoIndex do exactly the split you describe? Except I think it wouldn't work in nested cases? I feel it illustrates what I described above -- it has a subtly different signature (can return null), and clearly wasn't used everywhere it could have been. I can confirm that using its approach in getIndex would lead to all tests passing except those having to do with nested repeats. I propose removing the duplication.

or is it just that the block was forgotten about at some point?
Yes
I don't see any recursion there, am I missing it? It seems it walks the node tree iteratively.
Yes, indeed. I used the wrong word to describe that the code should work for any level of nesting (to explain that the workaround proposed in the previous PR seemed not right

Thanks for this PR and reviewing!

Thanks so much for taking a look, @MartijnR, and for putting up with my wall of questions. My spidey-sense tells me we'll be back in this code so I think it's a good investment to get context now.
I feel like I'm at least confident I understand what getIndex does and how it works. I don't have a sense of why this index is used (as opposed to e.g. an index within a parent). If you can give us a high-level sense or a hint on where to look, @MartijnR, I think that'd good to know. But also not critical at the moment. We have a lot more context and breadcrumbs now.
In response to @eyelidlessness's comment about count vs. index, I've tried changing the accumulator's name to predecessorCount and I find that helpful. That also led me to move the adjustment to go from a 1-based index to a 0-based index up in the initialization which I think makes more sense. That change helped me understand why the initialization works: the inner while loop doesn't add to the count in the case of a non-nested repeat. The loop condition is checking for repeat siblings but then the querySelectorAll call looks for children of the element being checked that match the original element's name. If I got all that right, the one unfortunate thing is that there is iteration over every instance of a repeat that adds 0 every time even if it's not nested. I'm not immediately coming up with a way to remove that and it doesn't feel like a big deal.

This whole process has been extraordinarily clarifying. Thank you @lognaturel both for revisiting it and helping me get a better understanding of what’s happening here. The only thing I want to add for posterity, which was additionally clarifying in our check in today, is that the thing that helped us understand what happens in this bit of code is the “absolute” index is an index of all repeated/nested references as if they had been flattened into a single list, and that that term flatten helped make it clear.

 I don't have a sense of why this index is used (as opposed to e.g. an index within a parent). If you can give us a high-level sense or a hint on where to look, @MartijnR, I think that'd good to know. But also not critical at the moment. We have a lot more context and breadcrumbs now.

This was a very early choice based on easy querySelectorAll (or jQuery) searches, as opposed to determining indices at every nested step. Not sure if that still makes sense, but it was never revisited by me.

Thanks so much to you both. This looks like a big improvement! 🎉

This was a very early choice based on easy querySelectorAll (or jQuery) searches

Ah, got it. Makes perfect sense, thank you!
@eyelidlessness it might also help to consider that repeats in general but especially nested repeats are not a big priority for web form users. Nested repeats in particularly generally imply an enumerator-mediated context and even then they're relatively rare. Most of the time when forms with nested repeats end up in Enketo they were authored for Collect and are being used for edits.
I still wouldn't be surprised if we did work in this area sooner than later but major refactoring might not be so compelling unless it benefits no-repeat or single-repeat cases significantly.

authors: MartijnR eyelidlessness lognaturel

repo: enketo/enketo-core


text:
newsapi kotlin: refactor connectivity check again

Made the same changes as java version

I added two more commits:
-I replaced ObservableField with LiveData, since according to this article published today: “Android Data Binding Library — From Observable Fields to LiveData in two steps” by Jose Alcérreca https://link.medium.com/C6b1SZ6WlV it is now what Google advises.
-I also reorganized packages.

repo: enpassio/Databinding


text:
Clean repo files

We should remove:

ensime_bridge under /bin
everything under /lib since it's not useful anymore
ensime_bridge spec
test suite (and replace it by one that works)

WDYT? @yazgoo

repo: ensime/ensime-vim


text:
Write the log in XML format suitable for VisualVM 'UI Actions'

Pull Request Description
Start project-manager with following options to provide first 20s of the startup sequence:
$ project-manager --profiling-events-log-path=start.log --profiling-path=start.npss --profiling-time=20

once the start.log and start.npss files are generated (next to each other), open them in GraalVM's VisualVM:
$ graalvm/bin/jvisualvm --openfile start.npss

analyze.
Checklist
Please include the following checklist in your PR:

[ ] The documentation has been updated if necessary.
[x] All code conforms to the
Scala,
Java,
style guides.
All code has been tested:

[x] Manually tested

VisualVM shows timeline with "stackdepth" and also "UI Actions". Hovering over boxes in "UI Actions" shows the messages describing what happens in the engine. One can then select an interval and get profiling information for that interval:

This picture shows that 2.7s is spend in compiler.

Can we have these steps documented, please?

Here is the documentation for your consideration, @hubertp.

repo: enso-org/enso


text:
How to remove Entando?

i installed Entando with
https://multipass.run/#install
curl -sfL https://get.entando.org | bash
now I want to remove it from my machine.
how can I do that?
thanks in advancece

Hi, @sinanasiriamirbande.
You have a couple of options. If you want to completely remove the VM created by Multipass then you can use multipass delete entando (where the last option is the VM name) and then you can use multipass purge to recover the resources. If you just want to shutdown Entando but keep the VM you can use multipass shell entando to shell into the VM and then remove the namespace via sudo kubectl delete namespace/entando. There are other tips here: https://dev.entando.org/v6.3/docs/reference/local-tips-and-tricks.html#quickstart-management.
I'd encourage you to use the Entando forum or slack for additional questions since typically repository issues are specifically for issues related to that codebase. See https://dev.entando.org/v6.3/docs/community/contributing.html#get-help for more information.
Cheers,
Nathan

authors: nshaw sinanasiriamirbande

repo: entando/entando-docs


text:
Make sure ec is built with network

In #1188 tests were made to run without network. This works if the local golang module cache already contains the required dependencies, i.e can be built offline. When a dependency is not available from the cache, due to the offline nature of the namespace the build fails.
This circumvents that by invoking the equivalent of ec opa version so that the ec command is build outside the offlined namespace.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (c4746f5) to head (07106aa).

Additional details and impacted files

@@            Coverage Diff            @@
##              main     #1192   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          120       120
  Lines         6254      6254
=========================================
  Hits          6254      6254

authors: codecov-commenter zregvart

repo: enterprise-contract/ec-policies


text:
Proposed solution to broken profile pages.

Empty strings for input values caused the functions to fail. Changing the value function to return the placeholder instead of an empty string seemed to fix the problem.
Using "bio" as Id caused function to fail, changed the ID for the text box and the ForHTML for the label to "about" and problem seemed fixed.
Fix tested in Firefox. May need extra tweaking.

Thanks!!

authors: TheLustriVA entmike

repo: entmike/fd-web-ui


text:
Define tables for wind plant metadata and asset metadata (turbines and met towers).

Define and implement table structure and column names for 1) Wind plant metadata
2) Asset metadata (turbines and met towers)
Establish wind plant and wind turbine IDs that are common to metadata and SCADA (and other time series) tables.

To demonstrate AEP analysis, we need:
Wind plant metadata

Plant ID
plant capacity
meter frequency


text:
Editing drink ingredients section doesn't show current ingredients + need a way to re-order ingredients

This area should show the current ingredient + servings provided for the drink with an icon to remove the ingredient.  If it's easy enough, drag/drop to re-order ingredients would be useful. Otherwise we need a way to re-order the ingredients since some drinks require ingredients be added in a specific order.

Make sure you get my latest push to develop as I changed this page a bit.

@sirtastic : Needs some css magic from you. Use git show <revision> to see my changes (in case you want to identify what css to undo).
commit a500872b8e8e0b0a192e01193e0b14d57bfeb3f5
Date:   Wed Aug 5 19:13:35 2015 -0700
Make drink ingredients sortable. Add drag-and-drop for ordering drink ingredients. Closes #46 (github)

@entrity I've remedied the CSS here in latest push.

authors: entrity sirtastic

repo: entrity/tipsy-web-public

license: unlicense


text:
perf: remaining small items

A few more small items for this perf pass before calling it good:

[x] Don't allocate PooledStreamEncoder (class can be merged into Router, not used anywhere else)
[x] Cache date string for date header every 500ms or so
[ ] Store integer headers as integers also to avoid converting back and forth
[x] Optimize buffer callbacks
[ ] Remove locking from libevent base
[x] 0 copy grpc encode
[x] Only do PNRG in runtime when needed
[ ] stat flushing code
[x] Don't copy http/2 static headers when encoding
[ ] remove fmt::format from dynamic stats in router
[x] remove copy from path match route action
[ ] allocate "light" host when doing service discovery so we don't allocate full stat block which we never use.
[ ] std::vector<nghttp2_settings_entry> iv in http2 codec can be a fixed array

https://github.com/lyft/envoy/pull/220

https://github.com/lyft/envoy/pull/221

https://github.com/lyft/envoy/pull/244

I'm closing this out. We can open individual issues for perf things that come up on traces.

repo: envoyproxy/envoy


text:
Envoy filter for outbound https traffic

I am trying to intercept all outbound http/s traffic from a pod and add a custom header to the request. After reading some documentation I came to the understanding that an envoy filter on SIDECAR_OUTBOUND with some custom lua code would do the trick. So this is the configuration that I did:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: outbound-filter
spec:
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_OUTBOUND
        listener:
          filterChain:
            filter:
              name: envoy.http_connection_manager
              subFilter:
                name: envoy.router
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.lua
          typed_config:
            '@type': type.googleapis.com/envoy.config.filter.http.lua.v2.Lua
            inlineCode: |
              function envoy_on_request(request_handle)
                  request_handle:logWarn("Hello World")
                  request_handle:headers():add("origin", os.getenv("ISTIO_META_WORKLOAD_NAME"))
              end

And it works perfectly fine for http requests. However, it seems that the filter does not run at all when the request is over https.
As such I tried to use the Egress gateway for it. This is what I did

Deploy an egress gateway in my k8s cluster
Change the outbound mesh traffic to registry only
Create a service entry for a particular host
Create a gateway kubernetes object which routes the requests from the egress gateway to a particular URL and listens at a particular port
Create a virtualService which serves the purpose of making sure that the routing of all http requests that are going to a particular URL are redirected to the egress gateway service and then the egress gateway can send it to its destination which is the final URL

All of this is good, but this can only be done for one particular remote service outside the mesh and for every service I want to add , I would have to add its url in the preexisting serviceentry and basically go over the full loop again.
Is there a way I can do that for all requests going out of my mesh ?

This question is more appropriate for the Istio organization. But to answer, yes, I think having a dedicated egress gateway for all traffic is probably the right solution to inspect all outgoing traffic. Otherwise, Envoy probably cannot parse encrypted HTTP traffic.

authors: kyessenov solarkaka


text:
Generic attribute bag shared by all filters

I was under the impression that the RequestInfo::FilterState (per request or per connection) is an object that can be modified by multiple (successive) filters. It turns out that its a write-once and read only object. @curiouserrandy any reason why you set the shared requestInfo::Filterstate to be a write-once thing instead of a model where multiple filters can write to same object?
I propose we move towards a generic attribute bag kind of system, such that the filters in a chain can append/update any attributes they want for that connection or request. This will allow all the reporting filters to consume info generated by all downstream filters, and emit access logs/reports with info gathered by everyone. Similarly, this will allow policy engines to act on data (about connection/request) gathered by multiple filters which have their own domain expertise.
For example, consider a stack that has
mongo+rbac+tcp_proxy+access_log
proprietary_decoder+rbac+tcp_proxy+access_log
hcm+oauth_filter+rbac+router+access_log
In all these cases, the RBAC filter does not have to know the semantics of the previous filters. If it simply exposes the ability to act on attributes generated by previous filters, we could do a lot many cool things. Similar idea applies to access logs, allowing any newly introduced protocol codec (or existing mongo/redis/thrift etc.) to emit the attributes to be logged and we can have the access log filter spit them out.
Along the same vein, it might help if we publish this so called object (one for HCM and one for Network stack) as protobuf representations for doc purposes.

Allowing multiple writers seems reasonable to me. It could also probably be expressed as part of the dependency/policy graph. @curiouserrandy thoughts?

[Just to clarify something that I think is understood: FilterState as a whole isn't write-once, but each object stored within FilterState is indeed write-once.]
Our experience at Google was that having a lot of filters all writing to the same objects made dependencies very hard to keep separate as the number of filters grew.  The goal of FilterState wasn't to create state that was shared between filters, it was to create a mechanism to allow information transfer between filters while keeping ownership and data flow of that data clear.  Data source is the (single) filter that sets the data originally, and the data sinks are the various filters that read that data.
Note that the current implementation doesn't make it impossible for different filters to write to the same filter, it just discourages it.  Object methods can implement arbitrary functionality by casting away const; indeed, this was how I imagined lazy evaluation being done for cases in which that was useful.  But the current design is specifically for the purpose of getting people to think carefully about their dataflow between filters, lest they end up in the state that we've experienced here.
In terms of a depencency graph, I suppose if we implemented the "Specifying Dependencies Between Filters" section of https://docs.google.com/document/d/1GNawM59Pp09WZ34zqJYiO_qmERjuGbOUeAqF1GG6v9Q/edit (and made it possible for multiple filters to call "providesData()" for a particular filter name, which the document doesn't forbid but hadn't been my vision) that could provide some level of constraint and documentation on the dependencies.  But I'd still worry that it was relaxing the structure too much for keeping inter-filter interfaces clean.  Maybe if you gave me some examples of the use cases you wanted to support?
@alyssawilk: You want to add to or modify anything in the above?

FWIW, in the PR that @rshriram just put in, I suggested doing a similar, but oppositely directed version: a key that anyone can write to, and that tcp_proxy would consume (if set). But that still maintains the property that it is easy to reason about, because it is (as currently written) a directed message from one filter to another. I'd be curious to hear your (@curiouserrandy) opinion on that use.

@curiouserrandy the use case I have is mentioned in the description above. I would like filters to add one or more attributes to a connection state (e.g., mongo adds some attributes, then tcp proxy adds some more, and finally access log uses it to create a log entry or the RBAC filter uses both to do some policy enforcement.). For example, lets say the RBAC filter looks for  "envoy.rbac.attributes" key in the shared connection state and does policies based on the attributes (a map of key value pairs). I would like all filters before RBAC to set the attributes that they glean from the connection into this shared object.

@ggreenway that was the original idea behind the post box style of the metadata namespace; the filter name was intended as the destination. It seems there are useful cases for both 1:N and N:1 communication.

@rshriram : Ah, sorry, somehow missed that section of your initial writeup.  I think I'm with @htuch / @ggreenway that 1:N and N:1 data flows would both have their use.  I'll note that one concern I have around N:1 is that I still don't think that it makes sense (at least based on what I understand of your use case) for multiple filters to overwrite the  same piece of data; what you want is for them to be able to add new pieces of data to a store/mailbox that the receiving filter will read.
WRT to protobufs, part of the reason for this implementation (using templates to allow arbitrary object storage) was to allow the cost for creation of complex data to be done once, and to allow filters to export logic/intelligence to be used by other filters.  So I'd rather not pull back from the current C++ template implementation to protobufs (though if I've misunderstood your point please correct me).
@htuch : Since I didn't end up replacing RequestInfo::dynamicMetadata(), it would be perfectly reasonable to keep that post office box style (and document it more clearly).  I'm not sure having both mechanisms in parallel for almost the same data transfer (the only difference being 1:N vs. N:1) makes sense as the mechanisms are fairly different (I think of dynamicMetadata() as being for config->filter communication, and FilterState as being filter->filter communication) but it's an option we could consider.

I am not looking for overwrite semantics. Append should do..

I would reasonably strongly prefer that we avoid that solution.  Let's craft a solution that's more targeted towards appending as opposed to allowing mutation.
One easy way to do that, either as another accessor or as a change to the existing situation, is to allow multiple setData<> of the same type on the same name, and have a getData<> that returns a vector of results.  Would that address this use case?
My personal preference would be another accessor so that it's clear when you're doing this different pattern of access.

neither of these would satisfy this use case though.. What I want is a bag of attributes.. Like a map within a map..
e.g.,
  mongoFilter:
     attributeBag = read_callbacks->connection().perConnectionState().getData("attribute_bag")
    attributeBag.addAttribute("foobar", someObject)
  rbacFilter:
    attributeBag = read_callbacks->connection().perConnectionState().getData("attribute_bag")
    iterate over the attributes in the attribute bag and do stuff..

Why wouldn't it satisfy the use case?  You just have objects with the same name (e.g. the destination filter that will consume the information) and those objects have an attributeName() accessor and the FilterState interface allows the ability to iterate through all of those objects.

sorry, i messed up the example.. here is the right one
mongoFilter:
     attributeBag = read_callbacks->connection().perConnectionState().getData("attribute_bag")
    attributeBag.addAttribute("foobar", someObject)
filterX:
    attributeBag = read_callbacks->connection().perConnectionState().getData("attribute_bag")
    attributeBag.addAttribute("filterX", someObject)
rbacFilter:
    attributeBag = read_callbacks->connection().perConnectionState().getData("attribute_bag")
    iterate over the attributes in the attribute bag and do stuff..

Ah I see what you are saying.. return vector of objects instead of a single object..
that could work.

should I change the getData to always return a vector instead of another variant?

We have been wanting a mixer-esque attribute bag for a while. I think it makes a lot of sense.
This proposal will have to decide on issues of attribute vocabulary that mixer had to before. Is it purely defined by the producer of the attribute, are attributes  namespaced ?

As this FitlerState is being implemented, what's the plan for the dynamic metadata? will the dynamic metadata be deprecated and removed sometime? If so when will this happen? Thanks.

@yangminzhu looks like it will be around for some time due to dependencies in JWT/RBAC filters IIRC.

We are planning to remove dynamic metadata from RBAC
and use the filter state instead.

Thanks for the comment, one more thing to confirm, currently the FilterState only has a generic Object interface.
To make it useful for sharing data between filters, should we define some well-known classes sub-classing the Object? For example, StringObject, NumberObject or even StructObject (for the protobuf struct).
Or otherwise should just we let each filter to create their own classes sub-classing the Object? (This seems creating a hard dependency if filter Y wants to receive data generated by filter X)
/cc @qiwzhang

Looking at RBAC filter, its simply a set of string matches isn't it? A string/NumberObject seems like a good start.

@rshriram I'm confused. How can the attributes be known well enough to be useful while the namespaces aren't known? At some point there is a contract over naming between producer and consumer, and that can include a namespace.
In your mixer example, why can't filter emit attributes under istio.mixer for example?

I think this is done. I will file a follow up issue to reconcile FilterState w/ dynamic metadata.

authors: curiouserrandy ggreenway htuch mandarjog mattklein123 rshriram yangminzhu


text:
PGV validations on embedded fields

Hi everyone, I'm running into a protoc-gen-validation issue when using Envoy. It seems like the values of map aren't validated.
The RBAC filter configuration includes a map of policies:
  map<string, Policy> policies = 2;

where in a Policy is defined as:
https://github.com/envoyproxy/envoy/blob/986173ed516dcc1c3dea7db90659ed993d0aad75/api/envoy/config/rbac/v2/rbac.proto#L84-L99
So both embedded repeated fields Permissions and Principals should have at least one item by the PGV constraints.
However, it seems like if we have an empty repeated message field, we can still validate OK. For example, this test crashes and doesn't exit cleanly, despite the PGV validation:
TEST(RoleBasedAccessControlEngineImpl, InvalidConfig) {
  envoy::config::rbac::v2::RBAC rbac;
  rbac.set_action(envoy::config::rbac::v2::RBAC_Action::RBAC_Action_ALLOW);
  envoy::config::rbac::v2::Policy policy;
  policy.add_permissions();
  (*rbac.mutable_policies())["foo"] = policy;
  try {
     // Validated the input successfully, yet did not call an exception that min_items = 1 on the
     // Principals field or required rule in Permissions.
    TestUtility::validate(rbac);
     // Crash now, because create Permissions returns panic: not reached because of missing case.
     RBAC::RoleBasedAccessControlEngineImpl engine(rbac);
  } catch (const EnvoyException& e) {
    // Clean exit.
    ENVOY_LOG_MISC(debug, "EnvoyException: {}", e.what());
  }
}

(I've added this test in engine_impl_test along with including envoy/config/rbac/v2/rbac.pb.validate.h)
Is PGV not running validations on the map values' messages?

cc @rodaine @htuch @akonradi

Upon carefully reading the documentation, the answer is no, PGV doesn't support validation on keys or values in C++, which means no recursive validation:
https://github.com/envoyproxy/protoc-gen-validate/blob/master/rule_comparison.md#maps
This also applies to repeated embedded message fields -- they're not validated in C++. Just FYI, in case we are relying on validations for embedded messages in repeated fields or maps.

Originally (predating RBAC), there were no repeated/map fields in the Envoy config messages that required validation, so the C++ implementation for those rules was kicked down the road. I can't couldn't say if I had forgotten this or if I was being optimistic when implementing RBAC.
There are no blockers to add this to PGV, but I do not currently have the bandwidth to work on it now, so if someone else would like to take a stab, it'd be much appreciated.

@asraa is this within scope of resolving our fuzz backlog? @rodaine it seems pretty reasonably scoped, just an additional recursive traversal in all languages.

@rodaine it seems pretty reasonably scoped, just an additional recursive traversal in all languages.

Not saying it’s technically hard. C++ is the only language right now in PGV that doesn’t support it. We skipped it on your initial pass since it wasn’t a requirement for the configs at the time.

@asraa sounds good, from what @rodaine says we should be able to figure this out, I can take a look at the WiP next week.

I found some time to work on this. I should have a PR out this afternoon that makes PGV recursively handle validations on maps.

https://github.com/envoyproxy/protoc-gen-validate/pull/281

Thank you so much!!!

@rodaine or @akonradi do either of you have time to review https://github.com/envoyproxy/protoc-gen-validate/pull/281?

authors: asraa htuch mattklein123 rodaine zuercher


text:
Minor update of bazel doc

Signed-off-by: Yitao Hu harmonyharmo@users.noreply.github.com

For an explanation of how to fill out the fields, please see the relevant section
in PULL_REQUESTS.md
Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]

/retest


text:
[backport/v1.21] docs: Fix mappings for version history links

Signed-off-by: Ryan Northey ryan@synca.io
Commit Message:
Additional Description:
Minor cleanup to ensure version history links are correct
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/retest


text:
Update QUICHE from 09a54d2f1 to 47aa4cda2

https://github.com/google/quiche/compare/09a54d2f1..47aa4cda2
$ git log 09a54d2f1..47aa4cda2 --date=short --no-merges --format="%ad %al %s"

2023-03-26 vasilvv Untangle dependencies for the default impl of quiche_default_proof_providers
2023-03-25 dschinazi Make MASQUE toy client perform DNS on proxy
2023-03-25 martinduke Store sent ECN codepoint in QuicUnackedPacketMap.
2023-03-24 martinduke Move peer_ack_ecn_counts_ from QuicConnection to QuicSentPacketManager.
2023-03-23 martinduke Make ecn_counts argument to OnAckFrameEnd const.
2023-03-23 quiche-dev Updating PublicMetadataHashWithHKDF function according to the latest IETF draft before we use it in prod. Adding the latest test vectors as well.
2023-03-23 haoyuewang Monitoring the QUIC version used for sessions with successful handshake.
2023-03-23 martinduke Eliminate QuicFramer::OnAckEcnCounts()
2023-03-23 quiche-dev Moving blind_signer and verifier from private_membership/anonymous_tokens/crypto to third_party/anonymous_tokens/cpp/crypto along with some test utilities. No new code is being added.
2023-03-23 haoyuewang Deprecate --qbone_server_defer_control_stream_creation, which has been default true for a long time.

Risk Level: Low
Testing: Existing
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

/assign @alyssawilk

/lgtm deps
Sorry for delay. Out of the country on vacation

Oh gosh! Sorry to bother you on vacation.


text:
Update QUICHE from f326e9346 to 667c58656

https://github.com/google/quiche/compare/f326e9346..667c58656
$ git log f326e9346..667c58656 --date=short --no-merges --format="%ad %al %s"

2023-09-13 birenroy Adds an OgHttp2Session option for the initial assumed value of the remote endpoint's max concurrent streams setting.
2023-09-13 quiche-dev Add missing absl includes
2023-09-13 quiche-dev Allow truncating of binaryhttp request
2023-09-12 vasilvv Set a fixed CC algorithm in quic_generic_session_test
2023-09-11 bnc Make QuicErrorCode underlying type uint32_t, avoid narrowing when parsing.
2023-09-11 quiche-dev Registering a new use case to test out go/at-integration-guide.
2023-09-11 vasilvv Introduce support for basic WebTransport stats.
2023-09-11 vasilvv Add missing QUICHE_EXPORT directives.
2023-09-07 vasilvv Fix QUICHE_EXPORT directives for Chromium merge
2023-09-07 vasilvv Add missing QUICHE_EXPORT annotations

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

I see there is also https://github.com/envoyproxy/envoy/pull/29602, which is marked as draft too. Are these PRs coordinated?

I have no idea.  @birenroy asked me to make sure I got his change in and I did?

Sorry, I decided to work through the Envoy QUICHE import instructions, and forgot that this would end up being publicly visible. I was just practicing, and didn't mean to start a competing PR.

Sorry, I decided to work through the Envoy QUICHE import instructions, and forgot that this would end up being publicly visible. I was just practicing, and didn't mean to start a competing PR.

AH! Thanks for the backstory. If it's any consolation, GH is profoundly terrible about PRs, visibility, reviews, etc. You did everything right... GH just sucks.

authors: RyanTheOptimist alyssawilk birenroy


text:
[bp/1.25] Backport stack (latest)

disable clang-tidy
update rules_rust
update distroless

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/retest


text:
udp: implement recvfrom(2) in os_sys_calls_impl

Signed-off-by: Christopher M. Luciano cmluciano@us.ibm.com
Description: implement recvfrom(2) in os_sys_calls and use in udp_listener_impl
Risk Level: Low - same functionality but different implementation
Testing: bazel test //test/common/network:udp_listener_impl_test
Docs Changes: N/A
Release Notes: N/A

/assign @alyssawilk @conqerAtapple

LGTM.

authors: cmluciano conqerAtapple


text:
Update Quiche tar ball.

Point to new Quiche tar ball: https://quiche.googlesource.com/quiche/+archive/4fbea5de9afdf30611b27afd54c45a596944f9c2.tar.gz;
Remove QuicStringImpl as it's removed from recent Quiche updates;
Wrap some platform api .h files with envoy_select_quiche() if their impl's are wrapped so. This is to make sure quiche targets can be built both with and without --define quiche=enabled;
Change tests in test/extensions/quic_listeners/quiche/platform/quic_platform_test.cc to be under quic namespace instead of Envoy as they will be used under namespace quic;
Modify quic_logging_impl.h to remove quic:: as it is supposed to be used under namespace quic;
Move macro's: EXPECT_LOG_NOT_CONTAINS, EXPECT_LOG_CONTAINS... out of namepsace Envoy because change 4 requires these macro's to be used under namespace quic for testing.

Risk Level: low (code not in use, or test only)
Testing: existing envoy tests passed, and test/extensions/quic_listeners/quiche/platform:quic_platform_test and @com_googlesource_quiche//:quic_platform_test --define passed both with and without quiche=enabled --experimental_remap_main_repo
Part of #2557

/assign @wu-bin

spdy_platform_test.cc and http2_platform_test.cc are still in the Envoy namespace, do you plan to change their namespace to ::spdy and ::http2 as well?
I haven't started working on those tests yet. But I think they should be under their own namespace.

/assign @alyssawilk

/retest

/retest


text:
forward proxy: per host SNI and SAN verification

Implement auto SAN/SNI setting on a per-host basis.
Cleanup some v1 config translation.

Part of https://github.com/envoyproxy/envoy/issues/1606
Risk Level: Low
Testing: Integration test
Docs Changes: Done
Release Notes: N/A

@PiotrSikora @lizan I want to improve test coverage a bit, but can you give me some early feedback on the TLS code when you get a chance? cc @alyssawilk

@alyssawilk updated per comments. I regenerated all of the upstream certs because they all need to be signed by the CA. I could probably hack up the script by hand to reuse the existing CA cert, but I'm not sure the effort is worth it. LMK.

@lizan thanks for the great suggestion. This is much better! PTAL, cc @PiotrSikora also.


text:
router check tool: add flag for only printing failed tests

Description:
For teams that have large numbers of tests, printing all test names can be cumbersome and make it hard to debug. For this reason, this change adds a new command line flag for the router check tool which when set only shows test results for tests that have failed.
When this flag is set alongside the detailed flag, only results for failed tests are printed.
When only the detailed flag is set, all test names are shown and names for failed tests are not printed again.
When only the new flag is set, only test results for failed tests are printed.
When neither flag is set, there is no output.
Risk Level: Low
Testing:
Added unit tests for expected output under different details & only_show_failures flag combinations in route_tests.sh.
Docs Changes: included
Release Notes: included
[Optional Fixes #Issue]
[Optional Deprecated:]

cc @jyotima

We need documentation and version history change like here: https://github.com/envoyproxy/envoy/pull/8058/files

/retest

/wait

/wait

#730

Thank you for working on the comments. Looks good to me now.

authors: LisaLudique jyotimahapatra