text:
PEX_ROOT/unzippped_pexes/<hash>/__main__.py fails to find ~itself in its own stack frame.

See in a Pants CI job here: https://github.com/pantsbuild/pants/actions/runs/3283455197/jobs/5408557053#step:11:11743
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.14/x64/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/hostedtoolcache/Python/3.7.14/x64/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/__main__.py", line 103, in <module>
    bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/pex_bootstrapper.py", line 595, in bootstrap_pex
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/pex_bootstrapper.py", line 662, in _bootstrap_venv
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/pex_bootstrapper.py", line 496, in ensure_venv
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/venv/virtualenv.py", line 217, in create_atomic
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/venv/virtualenv.py", line 200, in create
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/venv/virtualenv.py", line 235, in __init__
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/interpreter.py", line 886, in from_binary
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/interpreter.py", line 865, in _spawn_from_binary
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/interpreter.py", line 806, in _spawn_from_binary_external
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/interpreter.py", line 662, in _create_isolated_cmd
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/third_party/__init__.py", line 591, in expose
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/third_party/__init__.py", line 460, in isolated
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/util.py", line 109, in dir_hash
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/hashing.py", line 213, in dir_hash
  File "/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/hashing.py", line 185, in file_hash
FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/__main__.py'

This job ran using Pex 2.1.108

Much like in #1956, this backtrace has the same origination creating a venv mode PEX from a PEX that has laid itself out (<packed PEX> -> unzipped_pexes/<X> -> <create venv of X>). Maybe even more directly than in #1956, it shows a clear file-exists, a few more lines of code execute, file does not exist:
https://github.com/pantsbuild/pex/blob/bb68814b476c96ae25576620b685122695429bad/pex/hashing.py#L189-L213
The path /home/runner/.cache/pants/named_caches/pex_root/unzipped_pexes/58058b45d45f7fd03762b653bb93f82431caf384/.bootstrap/pex/__main__.py that is found not to exist on line 213 was just gathered via a filesystem walk on line 206 above. That is a clear case of that portion of the filesystem changing out from underneath a single function call's execution scope. This case does not involve the sys.path at all; so I'll close #1956 as a dup if this issue.

Ok, this is a broken Pants cherry pick here: https://github.com/pantsbuild/pants/pull/17200#discussion_r1002093435
Pex is fine.

repo: pantsbuild/pex


text:
Index was out of range

I have an Azure Service Bus namespace that I'm able to use 4.0.57, from the released zip file, and an older 3 version to explore.  However, when I click on any Topic Subscription, I receive the following items in the log:
<13:28:08> Exception: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index

I cannot perform any operations on the subscription, e.g. Receive Messages or Dead Letter Queue items.  Or view any of the subscription configuration details.  My colleague, who is using the very same builds on her machine, does not have these issues.
I'm uncertain what environmental differences might lead to this behavior.

Mmm... this is quite strange. Maybe it could be due to the entity named and regional settings on your machine?

I've had this same issue. Deleting the entire directory and re-building it didn't help. A direct copy of the built application to another machine did work.
For me, the only solution was to use another machine. I believe I remember tracing this error back to something to do with window width, but I wasn't able to solve it.

I was facing the same issue. From my analysis in HandleSubscriptioControl.cs the following method is having an issue.
private void CalculateLastColumnWidth(object sender)
The columns.count for object sender is 0 hence the code breaks at this statement
var width = dataGridView.Width -
dataGridView.RowHeadersWidth -
dataGridView.Columns[1].Width -
dataGridView.Columns[2].Width;
A simple check of if (dataGridView.Columns.Count < 5) { return;}
resolved the issue for me although I am not sure that is the right approach.

Thanks for the heads up. This should be fixed now. I added this line of code:
if (dataGridView.Columns.Count == 0) {return;}

source: gharchive/issue

authors: dberryalliance goanpao paolosalvatori timj92

repo: paolosalvatori/ServiceBusExplorer

license: mit

license_type: permissive

license_source: bigquery


text:
Regression in 0.7.1 for actix web & tail matches

Looks like there has been a change to the way url path parameters are generated between 0.7.0 & 0.7.1.  When using a tail match, which is normally defined by with a {param:.*} in the url, this now has broken output.
Here's an example config:
cfg.service(web::resource("/example/{filename:.*}").route(web::get().to(get_example)))

This matches anything that has the /example/ prefix, including any extra slashes etc..
Here's a diff of a service:
     "/example/{filename:.*}": {
       "get": {
         "operationId": "get_example",
         "parameters": [
           {
             "in": "path",
-            "name": "filename:.*",
+            "name": "filename",
             "required": true,
             "schema": {
               "type": "string"
             },
             "style": "simple"
           },

I would expect in this case the name still is filename:.*

Looks like this was done by #430

@cetra3 it this is what's happening then I think current behaviour is wrong because it leads to invalid openapi (easy to check on openapi website editor)
@snaggen any thoughts here?

Yeah the problem is that the filename on the path differers, so this yields invalid spec :(
Since there is nothing on the spec about regex being valid or other I think we should keep it as it was before.
I also use regex in the openapi at work, and I've yet to see an issue with the openapi tools.
We can add a option to keep or remove regex? But when the option is set to remove, remove it from the path as well.
What do you think?

I would have thought, stripping the regex out of the path and adding it as a pattern in the schema would be most correct.
Keeping the regex on the name causes it to be visible to users in documentation as part of the name. This is not a huge problem, just a bit ugly.

In the example diff in OP , there are a schema section. A schema section can contain a pattern, which should be a regex. So, by moving the regex from the path and name, to the schema (in the pattern property) we should get the same semantics as the actix path syntax.

You mean like:
type: string
pattern: ".*"

Interestingly on the spec Format it says the following:
"The schema exposes two types of fields: Fixed fields, which have a declared name, and Patterned fields, which declare a regex pattern for the field name.
So it seems a field name can be a regex?

Well, under the Path section for Patterned Fields, it doesn't mention anything about regex at all.... it seems to just mean the /some/{path} syntax. And also, the Patterned Fields refer to matching the path to use, not to restrict it like actix does, so it is a very different semantics.
     "/example/{filename}": {
       "get": {
         "operationId": "get_example",
         "parameters": [
           {
             "in": "path",
            "name": "filename",
             "required": true,
             "schema": {
               "type": "string",
               "pattern": ".*"
             },
             "style": "simple"
           },

is how I think the actix route
cfg.service(web::resource("/example/{filename:.*}").route(web::get().to(get_example)))

should be interpreted since in actix this means the path /example/{filename} that matches ".**

Yeah I'd be happy with that. @cetra3 looks good to you too?
How would we do reversal to generate server side code from that yaml? Only if the pattern is explicitly ".*" do we pass that to the actix server?

How would we do reversal to generate server side code from that yaml? Only if the pattern is explicitly ".*" do we pass that to the actix server?
Well, since the pattern is regex shouldn't it just be reversed, so that the path would be /example/{name:pattern} in any case where the schema have a pattern?

Oh ok I thought actix didn't quite support regex, it's been a while since I looked at it.

So to expand upon the issue a little more, we're using the openapi-generator-cli which does a pretty trivial "search & replace" against the path to build up the correct uri.  With 0.7.1 this doesn't work, as filename:.* != filename.
To clarify: in actix-web this syntax is not strictly regex, it's a special way of dictating a path route includes a tail match. That is: the param can include forward slashes.
I.e, /example/{filename:.*} matches on:
/example/file.txt
/example/path/to/file.txt
/example/another/file.txt

Whereas /example/{filename} only matches things like:
/example/file.txt
/example/another_file.txt

So I think this statement might not be entirely correct:

should be interpreted since in actix this means the path /example/{filename} where the filename matches ".**

However: I agree that the example given example looks like it would solve the issue.

I took a stab at fixing the regression, but got stuck. Basically, one way to do it would be to strip the pattern as early as possible. So, that nothing that goes in to path_map contains a pattern. However, then we loose the information about the pattern. So, I guess the pattern should be removed as late as possible, but I'm not sure how to do that in a good way.

I have now pushed a fix for this regression. The pattern in the template is removed when the paths is serialized. And the pattern is also set as a pattern in the parameter information.
I realize that the patterns might not be fully compatible, but it will probably only be a problem in the more advanced cases. So, perhaps just wait and address that when it becomes a problem.

Hey looks like this one is still an issue in 0.8.1 should I raise another issue or can we reopen this one.

Oh then let's reopen it :(

authors: cetra3 snaggen tiagolobocastro

repo: paperclip-rs/paperclip

license_source: github-api


text:
[Reporting - API] Add method to determine breakdown of customer browsers/timezones/OS's

Problem
Users should be able to see a breakdown of their customers' browser/OS/timezone in the Reporting Dashboard
Solution
Let's create a method in the Reporting context (https://github.com/papercups-io/papercups/blob/master/lib/chat_api/reporting.ex) that counts the number of customer browsers/OS's/timezones for a given date range.
Basically this just means grouping the customers table by a given field (e.g. :browser/:os/:time_zone) and counting the number of occurrences.
The method might look something like:
def get_customer_breakdown(account_id, field, filters \\ %{}) do
  # `field` will be `:os`, `:browser`, `:time_zone`, etc (fields on the Customer model)
  # `filters` should include a `from_date` and `to_date`
end

The response should look something like this:
get_customer_breakdown(account_id, :browser, %{from_date: ..., to_date: ...})
# =>
[
  %{browser: "Chrome", count: 25},
  %{browser: "Firefox", count: 12},
  # etc
]

Testing
Please add tests to test/chat_api/reporting_test.exs and make sure they pass with mix test :)
Context
See https://github.com/papercups-io/papercups/issues/252
Questions, or need help getting started?
Feel free to ask below, or ping us on Slack :)
(You can also check out our CONTRIBUTING.md)

You can assign this to me.

hey @shivamls -- any updates?

Does it make sense for me to pick this up? It looks stale.

yup go for it @eikooc :)

@reichert621 I have created a draft pull request at #361. I couldn't for the life of me figure out how to make the field an atom or what is needed to make the key of the map dynamic based on the field given as input, so I chose to go with a generic name called field for now. If you can help me, then it would be appreciated :)

hey @eikooc! thanks for the draft PR :) i'll take a look later, but in the meantime if it's easier feel free to use use Enum.map to convert :field to the name of the field (e.g. :browser)

authors: eikooc reichert621 shivamls

repo: papercups-io/papercups

license: MIT


text:
Remove option-ext as a dependency

Describe the feature
option-ext is required by directories, which is used by confy through this chain:

directories
dirs-sys
option-ext

We should replace confy.
Additional context
No response

Closed in #10290

repo: paradigmxyz/reth


text:
staged-sync: use smaller incremental range for subsequent merkle stage runs

Issue Description
Currently, there are 2 paths for MerkleExecute stage: clean trie reconstruction & incremental trie update. The threshold for discarding all previous trie progress and starting from scratch is quite low due to the entire block range being passed as an argument to incremental trie computation and it being memory intensive. Logic in question at L174 below
https://github.com/paradigmxyz/reth/blob/6d036cd95b68b80c5dc40a768c30784eb84162d6/crates/stages/stages/src/stages/merkle.rs#L172-L177
Desired Behavior
Introduce a new parameter inside MerkleExecute stage called commit_threshold. Change the behavior of the incremental path to keep running incremental root until completed if entire block range doesn't exceed clean_threshold. Increase clean_threshold to much bigger number (e.g. >100k TBD).
Testing requirements
To complete this issue, the contributor is required to perform mainnet sync with new logic, experiment with different values for commit_threshold and report them in the PR description.

Case1: normal

avg time for each block: ~3 hours and 40 minutes / 71488
Case2:  the performance is bad

avg time for each block: ~3 hours and 55 minutes / 5005
There is Rebuilding tire logs for this execution.

I found some weire point that from this check, to_block should be from_block.
let mut entities_checkpoint = if let Some(checkpoint) =                 checkpoint.as_ref().filter(|c| c.target_block == to_block)
because the target_block of checkpoint is the last block of the previous range and it should be from_block of current range.
If my hypothesis is right I would like to make another issue or make PR for this one

authors: forcodedancing rkrasiuk siddharth0a


text:
chore: patch ruint for quantity serialization

Patches ruint with a new repo for U256 serialization fixes

sweet

source: gharchive/pull-request

authors: Rjected gakonst


text:
When prompted with overwrite dialog, user loses "Don't Save" option upon "Cancel"

This is related to #270.  We are unable to tell where the "overwrite" dialog was opened from and whether it had a "Don't Save" button, so you are returned to the save dialog without the "Don't Save" button.

@phated - Good catch!

authors: PropGit phated

repo: parallaxinc/Parallax-IDE


text:
Add ‘Twitter-X’ to brand colors (for hover)

Example on DE:

 Twitter Logo update on web sites and newsletters.


text:
Add wufoo form support to  podcast page

 Podcast contact form changes


text:
Document the update in #45

Id est #45 .

Closed by wiki#d71918f.

repo: paramsiddharth/rema

license: ISC


text:
Update security vulnerabilities

also fix register WebView & name casing
also add prettier config

This change is

lib/sneeze-board.web-view.tsx line 40 at r1 (raw file):

  const [users] = useData.User<AchYouDataTypes, 'User'>('sneezeBoard.sneezes', '*', []);
  // TODO: not sure how to actually get other extensions' types yet paranext-core#69

@tjcouch-sil a fix needs to be applied here since #69 is now complete. I left of doing it because it was out of scope for this PR since the app still works without it.


text:
fix: ability to specify alt kernel config options

Currently, the kernel config is checked for a static
list of kernel config options pertaining to eBPF.
This change allows the developer to specify
alternatives to each option in cases where there are
valid fallbacks.

LGTM.
I've seen only one thing related to style to improve readbility.

Alright, I think the latest change gets it a bit closer. I don't think we can bail any earlier than we do, because we're checking for the existence of any of the alternatives for each option, not all if that makes sense?

Thanks for handling this. Just have one comment, looking good otherwise. :)

:tada:

authors: jnsgruk v-thakkar

repo: parca-dev/parca-agent


text:
unable to encode latest metadata with metadata codec

Given https://gist.github.com/ryanleecode/c0d2b7322f2252378dc50a969bd88772 as the latest metadata. Using metadata from substrate codec to encode it fails with:
file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:350
var VectorEnc = (inner, size) => size >= 0 ? (value) => mergeUint8(...value.map(inner)) : (value) => mergeUint8(compact.enc(value.length), ...value.map(inner));
                                                                                ^

TypeError: Cannot read properties of undefined (reading 'length')
    at Array.<anonymous> (file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:350:131)
    at file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:331:99
    at Array.map (<anonymous>)
    at file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:331:66
    at file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:93:54
    at Array.map (<anonymous>)
    at Array.<anonymous> (file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:350:149)
    at file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:331:99
    at Array.map (<anonymous>)
    at file:///home/ryan/Documents/Repositories/capi/node_modules/scale-ts/dist/scale-ts.mjs:331:66

Node.js v20.3.1

I've tried to reproduce the issue and everything seems to be working fine 🤔.
Please have a look at the test-broken-meta branch, that branch was created after the current main branch and basically what it has is a new package named experiments, which is what I've been using for running different sorts of experiments... Since I already had some code for getting the latest metadata from smoldot, the only thing that I had to do was to try to re-encoded, and everything seems to be working well.
In order to reproduce it, do this: checkout the test-broken-meta branch, run npm i from the root, then run npm run build (also from the root), and then go to packages/experimets and from inside that path run npm start.
Please fee free to make the necessary changes to reproduce the issue.
If I had to make a guess about what could have happened here, is that perhaps that latest-metadata.json file was generated with an old version of the metadata codec. Please keep in mind that the metadata codec has recently changed. So, if that file was created with the prior codec, then it wont work with the current one.
PS: I will be creating a PR later today to add the experiments package into the repo, so that we have a common playground for testing things. That should make everything easier: not just to test stuff, but also to show issues and/or ideas.

If I had to make a guess about what could have happened here, is that perhaps that latest-metadata.json file was generated with an old version of the metadata codec. Please keep in mind that the metadata codec https://github.com/paritytech/capi/pull/20. So, if that file was created with the prior codec, then it wont work with the current one.

Yeah, that's what happened: https://gist.github.com/ryanleecode/c0d2b7322f2252378dc50a969bd88772#file-latest_metadata-json-L9-L24

Yup that was indeed the issue. thx for investigating

authors: josepot ryanleecode

repo: paritytech/capi


text:
nomination-pools: clear pool metadata after a pool is deleted.

In fn dissolve_pool, the corresponding pool metadata has to be removed as well.
Currently, we forget to do this.
Also requires a small migration: iterate over all metadatas, if one does not have a corresponding bonded_pool, delete it. The migration could also be executed manually with providing the right system::kill_storage.

Would like to attempt this

What do you mean by migration?

authors: kianenigma tifecool

repo: paritytech/substrate


text:
DoubleStorageMap No access to keys in iter_prefix_values()

When using iter_prefix_values() of DoubleStorageMap trait, there seems to be no way to retrieve k2. That creates a problem when iterating over prefix and wanting to change values in the storage.
I want to be able to achieve this:
decl_storage! {
    ...
    trait Store for Module<T: Trait> as MyModule {
        pub MyDoubleMap get(fn my_double_map) config(): double_map hasher(blake2_128_concat) MyKey1, hasher(blake2_128_concat) MyKey2 => MyValue;
    }
}

fn some_function_using_storage<T: Trait>(prefix: MyKey1) {
    let items = MyDoubleMap::<T>::iter_prefix_values(prefix);

    // this is the desired effect - in the current implementation `.next()` returns only `my_value`
    while let Some((my_key2, my_value)) = items.next() {
        MyDoubleMap::<T>::mutate(prefix, my_key2, |value| { ... })
    }
}

Is this feature something that can be added, or is there any obstacle? Is there any other way how to achieve walkthrough prefix's items and modify them? The iterator has access to the key, but its type is Vec<u8>, so I'm not sure if it can be converted back to MyKey2. https://crates.parity.io/src/frame_support/storage/mod.rs.html#445
EDIT: source code improved

You are searching for this here: https://docs.rs/frame-support/2.0.0/frame_support/storage/trait.IterableStorageDoubleMap.html#tymethod.iter_prefix

Thx! Missed that one.

authors: bkchr ondratra


text:
cli: disable pruning on validators

Currently GRANDPA needs to import all blocks that are voted on, and therefore requires to have state available for those blocks. During periods of long finality stalls we run the risk of state being pruned due to depth (we prune after 4096 blocks regardless of finality), or e.g. a short-lived fork being pruned out. This PR changes the default pruning mode to be set to archive if the node is a validator, otherwise it defaults to keeping state for the last 256 blocks. The node will refuse to start if the node is a validator and pruning is enabled, unless the flag --unsafe-pruning is given as well.
Currently the node doesn't seem to detect whether the existing database is an archive or pruned. This means that if you start with a pruned database, and restart the node as archive, the node will either crash due to missing state (if you are a validator trying to author), or it just won't import any blocks. The reverse (archive -> pruned) seems to work. This might be a bit confusing to end-users, especially if they just enable/disable --validator which implicitly switches the pruning mode.
cc @arkpar

If it is really a short term patch that will be replaced anyhow (post #3639), and can be done manually via flag, a warning in the case of is_validator || !archive_mode might also be enough.

@kianenigma I would like to make it particularly hard for validators to run with what we consider unsafe settings. A warning is easily ignored IMO, hence why I added the --unsafe-pruning flag, so that this becomes obvious to anyone that has to type it out.

Currently the node doesn't seem to detect whether the existing database is an archive or pruned. This means that if you start with a pruned database, and restart the node as archive, the node will either crash due to missing state (if you are a validator trying to author), or it just won't import any blocks.

we should probably address that in this patch then otherwise we'll have a torrent of validators wondering why their node is crashing. i'd suggest just discarding the (pruned) DB if they are wanting to use archive and then it'll just resync from scratch.

@arkpar Any suggestion on how to verify whether a given state db is an archive or pruned on startup? (in particular something that is migration friendly). I guess we could check for the existence of a pruning journal on the database?

@andresilva Client::block_status can be used to check if the state is available

authors: andresilva arkpar gavofyork kianenigma


text:
Benchmarking for Offchain Phragmén.

Adds some benchmarking and builds on top of https://github.com/paritytech/substrate/pull/4517.

@shawntabrizi I will try to keep this branch up to date for usability, but I don't see any point in merging it. The dependencies are ugly and the internals need to become compatible with the macro, which I am not sure if it is possible for now. So no need to rush into this.

This is beyond saving. I will make a new PR soonish.


text:
Integrate contracts into substrate-demo runtime

Also, the way contract address is calculated is changed.
ext_create now takes one additional parameter: salt (u64), which is ultimately passed into the contract_address_for. You can read about motivation in KIP-4.
Additionally, data (input parameter for constructor) is also passed to the contract_address_for . This was changed to make address calculation mechanism to be similar to pwasm's one, where input data is also accounted for address calculation (since it is a part of the constructor code there).
contract_address_for for substrate-demo has the following formula:
blake2_256(blake2_256(code) + blake2_256(data) + LE(salt) + origin)

not super keen on salt - we can just introduce a "salt" into the init code to avoid this.

Me too. But I'm not keen on the introducing salt into the init code either, for the same reason why we pass init code and data separately (like we are doing in pwasm).
In pwasm we do this because of compatibilty with ethereum, where you don't have separate "data" field. In case of EVM this works fine since it just a bunch of instructions and data, but in case of wasm, which have a strict binary format, it is merely a hack: we just concatenate input data to the end of wasm init code. This trailing data makes the wasm binary invalid and we detect this in pwasm loader and strip all trailing data and treat it as input data. But that said, this is a better solution than trying to modify init code wasm binary: you can't get away with monkey patching the data section because you need to preserve binary format. Moreover, you need to be careful to not overwrite existing data in data section and update the initial memory page count if input data doesn't fit in. There might be other caveats that I can't come up with now. All of this makes me think that modifying init code is not feasible.
Back to salt: we can make it an ABI problem, user can always modify data and just ignore it in their constructor. But this will introduce some burden in the userspace: constructors should provide a way to supply some space for the salt in data they accept.

we can make it an ABI problem

Indeed. I'd prefer to dirty the user space (when needed) than dirty the core protocol specification on which all userspace rests.

Ok, removing salt then!

Removed salt and rebased @gavofyork

more resolving :) @pepyakin

authors: gavofyork pepyakin


text:
Migrate assets pallet to new macros

Converts the Assets pallet to the new pallet attribute macro introduced in #6877.
Part of #7882.
Upgrade guidelines used.
:warning: Breaking Change :warning:
From https://crates.parity.io/frame_support/attr.pallet.html#checking-upgrade-guidelines

storages now use PalletInfo for module_prefix instead of the one given to decl_storage: use of this pallet in construct_runtime! needs careful updating of the name in order to not break storage or to upgrade storage (moreover for instantiable pallet). If pallet is published, make sure to warn about this breaking change.

So users of the Assets pallet must be careful about the name they used in construct_runtime!. Hence the runtime-migration label, which might not be needed depending on the configuration of the Assets pallet.

@ascjones Do you know how we should handle the deprecation of RawEvent? Should we still provide the type alias or rather try to track down uses and update (or both)?

@ascjones Do you know how we should handle the deprecation of RawEvent? Should we still provide the type alias or rather try to track down uses and update (or both)?

we can also have a deprecated alias. This could help other to migrate.
(
before this PR raw event was:
enum RawEvent<AccountId, Balance, AssetId> {...};
type Event<C: Config> = RawEvent<C::AccounId,...>;

)
Maybe we can now do:
#[deprecated(...)]
type RawEvent<T> = Event<T>;

But at the same time this is a breaking RawEvent as its generic are different

But at the same time this is a breaking RawEvent as its generic are different

Right, so people would get a compiler error and also the deprecation message? That seems good.

bot merge

@thiolliere I added the D1-runtime-migration label here but I don't think that is actually needed for pallet-assets (not used in Polkadot). Do you agree?

@thiolliere I added the D1-runtime-migration label here but I don't think that is actually needed for pallet-assets (not used in Polkadot). Do you agree?

Yes I think it is better to have all the commit with potential need for migration labelled as such. It is useful for other builders

authors: dvdplm thiolliere


text:
Update the Grafana dashboards

Updates the dashboards to the latest changes that I've made over the past weeks.
As usual it's kind of hard to review if you're trying to read JSON, so I guess trust me that I haven't done anything wrong :shrug:

There were panels on top showing the node name, and apparently Grafana's export feature doesn't understand that the values come from a variable.

bot merge


text:
DESIGN: what dates do we want to show in the search results?

The current implementation is a bit weird:

One of those dates is the date the thing was created, the other is the date of the last modification.  One is expressed in formal date language, the other in human "x months ago" type language.
I think we lost some fidelity in this process along the way.  We should figure out:

which date matters
whether we need to show authorship and/or last-editor.

In the spirit of wikipedia, and a less cluttered search results page, my suggestion is:

we don't show names of authors or editors  in the search results page
we only show one date, probably the 'last edited' date, and using human language.

Opinions?  @jesicarson as the proxy for the community in particular.

Sure! I agree with Davids suggestions. And... there are some additional notes about search results on latest redpen (https://redpen.io/ar95785df6be9283e4):

Move the content type label (CASE) below the image, above case title.
Bookmark: This move will allow for space for the bookmark icon below the bottom right corner of the image.
Featured: Rather than a star or any featured icon, add the word FEATURED before the content type label for featured items, ie: FEATURED CASE

@davidascher @jesicarson @andreadelrio - Let's also removed the "8 months ago" and replace it with the [last edited date] or [publish date] (Which one do you think Jesi?) - We won't need to indicate time, only the date - I'd recommend this format: "DD Month, YYYY" or "9 May, 2017" because mm/dd/yy can be confusing between date and month.

What about "May 9, 2017"

sure! as long as we have a clear month and date ;)

Cool lets go with that.

Covered in PR #303

authors: andreadelrio davidascher jesicarson samjiangweb

repo: participedia/frontend


text:
Make files display on reader view

For any data question involving form upload, display in the sidebar as the label "Files uploaded" and the value "Yes", and hyperlink "Yes" to jump to the bottom of the narrative, where the header "Files" will display as a list (header will only show if files are attached).
Each file will display the attribution first (hyperlinked to the file, which will open if the link is clicked), followed by a period, followed by the Title/Description for that file as text, followed by a period.
@scottofletcher is "Files uploaded" and "Yes" and "Files" the right text to use here?
@dethe does this make sense or would you like a mockup?
Thanks!

Looks good! Could we use the same 'Files' heading for Final Reports? That way the "Yes" values of both "Files uploaded" and "Final report" would link to the same heading and the narrative wouldn't be so long.
Also wondering if the display should hyperlink the Title/Description rather than the attribution. Or just the whole thing.
And will there be a little icon beside it indicating what file format it's in? Like a little PDF/Acrobat icon like we have on .net:

FYI chatted with Dethe, and he doesn't think we need to link the data sidebar to the bottom of the narrative - simply listing the attached files below the narrative should be sufficient.
And, sure, all file uploads can be listed there, including evaluation reports.

authors: jesicarson scottofletcher


text:
[Feature] Added param_prefix to IIA

Relates to #272 Added param_prefix to Identity initialized ansatz. It defaults to an empty string.

One consequence of having an empty string as the prefix is that in cases where it defaults to the empty string, the parameter names would start with an underscore _.
On the other hand, keeping some other prefix, like "iia" or "phi" is that the names would become unnecessarily too long? Such as phi_alpha_01 or iia_theta_ent_01.
Which of the two is the better alternative?

Thanks @jpmoutinho
I added "iia" as the default string, and taking your suggestion, included the respective Greek characters to shorten the name.

repo: pasqal-io/qadence


text:
chore: add dev container

What's New?

Adds a dev container config file to bootstrap the dev environment
Replaces openapitools npm package with docker image for consistency across sdks

codegen change can be tested with
curl -o openapi.json https://api.swaggerhub.com/apis/passage/passage-mgmt-sdk-api/1
./generate.sh openapi.json

Screenshots (if appropriate):

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have manually tested my code thoroughly
[ ] I have added/updated inline documentation for public facing interfaces if relevant
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing integration and unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Additional context

formatting expected to fail since it'll be addressed in another PR

repo: passageidentity/passage-flex-node


text:
some media content won't play

First of all thank you for developing such an app.
I am running Ubuntu 16.04 32 bit. I cloned twd and ran nmp install subsequently launched it via
nw .  command within twd directory. My nw version is 0.12.3 . It launched successfully, and works well except it fails to play some videos including some youtube videos. I get
  "Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self' http://localhost:* http://localhost.twitter.com:* https://*.twitter.com https://*.twimg.com https://vine.co https://*.vine.co https://*.periscope.tv https://*.pscp.tv".

"
error.
I searched for the error on the net but could not find a fix.
How can I gain that functionality?
Regards,
Kenn.

Hi. This project is not maintained and has been mostly abandoned for at least a year. While it obviously works somewhat, I'm not going to put more time into it myself. However, if patches are produced I will happily merge them in.

authors: kensully passcod

repo: passcod/twd


text:
non-flat (.zip) pkgs don't get friendly names from JSS API

I thought I fixed this but just saw it again when deploying and OSXUpgrade (mpkg).
Investigate.

this should be fixed in the this branch?
https://github.com/patchoo/patchoo/tree/fix-spacesinpkgnames

repo: patchoo/patchoo


text:
WillDisplay doesn't get called?

IMPORTANT - BEFORE CREATING YOUR ISSUE PLEASE READ THE FOLLOWING:

PLEASE STATE THE VERSION NUMBER you are using.
v7.1.5
Are you using the latest version of JTAppleCalendar? Latest version is currently 7.1.5
YES
PROVIDE ENOUGH INPORMATION SO I CAN RECREATE THE PROBLEM.

I run the sample project you attached at issue 667
And I download the  sample project from the repo, it has same issue.
Any idea why?

For you to say that it is not getting called, you must have some expectation of when it gets called.
When do you expect it to be called?

I just want my calendar to scroll smoothly, since I need do some time-consuming jobs for the date before it calls cellForItemAt.

I understand the concern you have.
But i do not understand what you are trying to do in the code to fix your concern, and how is your fix not working.

I think what might be @pandaApe's issue is that when scrolling horizontally between calendar frames, cellForItemAt is not called for the very next cell coming into the view. My guess is this has something to do with tableView pre-fetching?
In the screenshot below, I am scrolling from one week to the next and the 13th is the next date to appear in the view frame, however, cellForItemAt is not called for that cell and is instead called for a cell two indexPath's later. As a result, my configureCell function is not called to update the incoming cell's backgroundView for the appropriate selectedPosition

See my JTAppleCalendarViewDelegate implementation code below:
func calendar(_ calendar: JTAppleCalendarView, willDisplay cell: JTAppleCell, forItemAt date: Date, cellState: CellState, indexPath: IndexPath) {
    let cell = cell as? CalendarDayCell
    cell?.configureCell(for: cellState, date: date)
}

func calendar(_ calendar: JTAppleCalendarView, cellForItemAt date: Date, cellState: CellState, indexPath: IndexPath) -> JTAppleCell {
    let cell = calendar.dequeueReusableCell(ofType: CalendarDayCell.self, for: indexPath)!
    cell.configureCell(for: cellState, date: date)

    return cell
}

func calendar(_ calendar: JTAppleCalendarView, didSelectDate date: Date, cell: JTAppleCell?, cellState: CellState) {
    let cell = cell as? CalendarDayCell
    cell?.configureCell(for: cellState, date: date)
}

func calendar(_ calendar: JTAppleCalendarView, didDeselectDate date: Date, cell: JTAppleCell?, cellState: CellState) {
    let cell = cell as? CalendarDayCell
    cell?.configureCell(for: cellState, date: date)
}

@obianachebe
I am not sure how your calendar is not updating because i do not have your code.
Therefore i just added an example code you can follow to do range selection.
Download the github project and run the sample calendar attached to it. You'll see rangel selection as the 4th option. Let me know how it works.

@patchthecode I think my issue was actually because I'm using a calendar embedded in a UITableViewCell. For my use case, calling tableView.reloadData() updated the selection style correctly. Thanks for all the help and great work on this library!

Awesome. Glad it works. Closing this issue now.

authors: obianachebe pandaApe patchthecode

repo: patchthecode/JTAppleCalendar


text:
Memory spike when programmatically scroll calendar without animation

I'm using the latest version, 7.1.6.
I created a demo project that calls scrollToSegment(.next) in the didSelectDate delegate method and the memory would keep climbing by about 1MB for every new segment and would only drop after I scroll the calendar on the screen. I have also tried this with scrollToDate and it causes the same problem. There doesn't seem to be any issues when I enable scroll animation.
func calendar(_ calendar: JTAppleCalendarView, willDisplay cell: JTAppleCell, forItemAt date: Date, cellState: CellState, indexPath: IndexPath) {
        let cell = cell as! CustomCell
        cell.label.text = cellState.text
    }

    func calendar(_ calendar: JTAppleCalendarView, cellForItemAt date: Date, cellState: CellState, indexPath: IndexPath) -> JTAppleCell {
        let cell = calendar.dequeueReusableJTAppleCell(withReuseIdentifier: "customCell", for: indexPath) as! CustomCell
        cell.label.text = cellState.text

        return cell
    }

    func configureCalendar(_ calendar: JTAppleCalendarView) -> ConfigurationParameters {
        dateFormatter.dateFormat = "dd mm yyyy"
        let startDate = Date()
        let endDate = dateFormatter.date(from: "25 01 2025")
        let parameters = ConfigurationParameters(startDate: startDate, endDate: endDate!)

        return parameters
    }

    func calendar(_ calendar: JTAppleCalendarView, didSelectDate date: Date, cell: JTAppleCell?, cellState: CellState) {
        //print(date.toFormat("MMMM dd yyyy"))
        //calendar.scrollToDate(date.dateAtStartOf(.month).dateAt(.nextMonth), animateScroll: false)
        calendar.scrollToSegment(.next, animateScroll: false)
    }

I don't know how helpful this screenshot will be, sorry if it's not lol

i guess some more investigation is needed here.
Under the hood for this library, i use UICollectionView code :/
I do not do anything fancy under the hood, so I am not sure as the cause.
Is this issue for device? or on simulator.

closing this issue.
I did not notice this on latest build.
You can chip in your thoughts on this.

authors: Sothyrith patchthecode


text:
Adding "Select All" and "De Select All"

It would be really helpful to have methods to be able to select all and select none. I tried to achieve this (see below hacky code), but the information doesn't seem to be saved:
$("#btnSelectAll").on('click',function() {
event.preventDefault();
$("#PromptPanel1_treeviewCommentCodes > option").each(function(){
$(this).attr('selected','selected');
});
$(".tree-multiselect input").each(function(){
$(this).attr('checked','checked');
});
});
            $("#btnDeSelectAll").on('click',function() {
                event.preventDefault();
                $("#PromptPanel1_treeviewCommentCodes > option").removeAttr("selected");
                $(".tree-multiselect input:checked").removeAttr("checked");
            });

Thanks a lot for your support,
Can i set value for every checkboxin the tree?

This will help me if i want to set checked to determine nodes..
Thanks

Could you wrap all your options with a section? Then you could just check/uncheck one box.

I tried to set value attribute inside addCheckboxes() function by the followng line
$checkbox.attr('value', $targets.data('value'));
`function addCheckboxes($selectionContainer, options) {
var $checkbox = $('', { type: 'checkbox', value: '' });
if (options.freeze) {
$checkbox.attr('disabled', 'disabled');
}
var $targets = null;
if (options.onlyBatchSelection) {
$targets = $selectionContainer.find("div.title");
} else if (options.allowBatchSelection) {
$targets = $selectionContainer.find("div.title, div.item");
} else {
$targets = $selectionContainer.find("div.item");
}
    **$checkbox.attr('value', $targets.attr('data-value'));**

    $checkbox.prependTo($targets);
    $selectionContainer.find('input[type=checkbox]').click(function (e) {
        e.stopPropagation();
    });
}`

but didnt get the agood result.
can you help me to put value of every option to related checkbox.
Thank you very much

Thanks for replying patosai!
If possible, I would prefer not to wrap everything in an outer section - I am already using sections and don't want to show all sections open by default (which I would probably have to do, if I had a single outer section, otherwise my users wouldn't see the options available to them).

Ok I can it being useful. Hang on.

f9997b0c8cc878cfc1b74757ab0c6b453d8f0003 should do it. I'm not sure how to best style it; let me know what you think.

Functionally, what you have added is perfect - thank you for being so accommodating! For my application, I would like to style the links as bootstrap btns - maybe the css classes used for the spans can be an option?. Is it possible to make the "Select All" and "Un Select All" text variable (via options)? I would like to localize it so that the appropriate translation may be displayed for the end user's language.

You can modify the source yourself for the bootstrap buttons. I think having the text as an option is a good idea.


text:
Inspection of gradient calculation

Hello!
Is there a tutorial on how to inspect gradient/hessian calculation when using diffrax?
I am trying to calculate gradients and hessians, but the returned values are always nans, and I do not know where to start "debugging".

So there's nothing special about Diffrax -- I'd recommend debugging NaNs in the same way as any JAX program.
First of all, one very common source of NaNs during autodiff is a missing double where trick. (Needing this trick is actually a general fact of autodifferentiation systems! Nothing JAX-specific here.)
Second, another common source is when doing a sqrt or log of a negative number. If your vector field includes either of these operations, then consider checking whether their input is negative. (Note that Diffrax reserves the right to query your vector field with any values for t and y, even those outside the region in which the ODE is solved -- some solvers will make queries outside those ranges. So your vector field must be robust to such queries.) You can easily check this with jax.debug.print.
If it's not any of those common errors, then your main tools to track this down are:

jax.debug.print to print intermediate quantities.
jax.debug.breakpoint to put a breakpoint in the middle of your computation. (Make sure to pass the -s flag if used in conjunction with pytest.)
Set the environment variable JAX_DEBUG_NANS=1 to interrupt the computation when a NaN is detected. This works best if the NaN occurs on the forward pass and outside of any loops. (If the NaN occurs inside a loop then adding JAX_DISABLE_JIT=1 can also help. And once you become more advanced, then JAX_TRACEBACK_FILTERING=off can also help, as it allows you to inspect JAX internals.)
If your gradients only arise on the backward pass, but not the forward pass, then try equinox.internal.debug_backward_nan.

Generally speaking, NaN issues aren't usually too tricky to track down. Just bisect through your code until you find the operation that's producing them.
Moreover I'd encourage you to be willing to place these checks inside your copy of Diffrax's code. You can get the install location via import diffrax; print(diffrax.__file__). (Placing breakpoints within the library is often a quicker way of debugging, as you have finer-grained control over your bisection search.)

That probably doesn't help, I'm afraid!

authors: moesphere patrick-kidger

repo: patrick-kidger/diffrax


text:
can't run if ruby is already installed

I think it is an issue to not let ruby be installed in WSL before being able to run this tool. Is this something that can be fixed easily? or do I need to make a pull request?

Sorry for late reply. The issue is fixed recently(Well, I think...) can you try it again?

license: Unlicense


text:
Addressing vulnerabilities in dependencies

Hi there,
This issue is to raise awareness on a couple of vulnerabilities being identified by yarn audit in a few outdated dependencies, with the hopes to work together on addressing them.
I've included information below on the dependencies at issue, the versions where they get patched, and if there are any additional notes on the dependencies or vulnerabilities. I'll also setup a PR as well.

Package
Installed Version
Addressed In
Latest Available
Release notes
Other notes

jimp
^0.2.28
0.22.09
0.22.12
jimp release notes
The vulnerability isn't actually with jimp, but with a dependency (mkdirp) that jimp used. That dependency was removed in 0.22.09

git-url-parse
^9.0.1
13.0.0
14.0.0
git-url-parse release notes
The vulnerability isn't actually with git-url-parse, but rather with parse-url that is used by git-up. The dependency was addressed in version 7.0.0 of git-up.

conventional-changelog-writer
^4.0.0
5.0.1
8.0.0
conventional-changelog-writer release notes
The vulnerability isn't actually with conventional-changelog-writer, but rather with handlebars and minimist that are used as dependencies.

pkgfiles
^2.3.2
n/a
^2.3.2
pkgfiles releases
The vulnerability isn't actually addressed in pkgfiles. We can fix this by overriding the minimist dependency to >=1.2.6

yarn auditoutput
yarn audit v1.22.22
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.4                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ jimp                                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ jimp > mkdirp > minimist                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097677                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Server-Side Request Forgery in parse-url                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ parse-url                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=6.0.1                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ git-url-parse                                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ git-url-parse > git-up > parse-url                           │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1088918                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Server-Side Request Forgery (SSRF) in GitHub repository      │
│               │ ionicabizau/parse-url                                        │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ parse-url                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=8.1.0                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ git-url-parse                                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ git-url-parse > git-up > parse-url                           │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1092304                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.4                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ git-url-parse                                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ git-url-parse > parse-domain > mocha > mkdirp > minimist     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097677                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ json-schema is vulnerable to Prototype Pollution             │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ json-schema                                                  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.4.0                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ jimp                                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ jimp > request > http-signature > jsprim > json-schema       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1095057                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Remote code execution in handlebars when compiling templates │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ handlebars                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.7.7                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ conventional-changelog-writer                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ conventional-changelog-writer > handlebars                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1095063                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in handlebars                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ handlebars                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.7.7                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ conventional-changelog-writer                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ conventional-changelog-writer > handlebars                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1095465                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.4                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ conventional-changelog-writer                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ conventional-changelog-writer > handlebars > optimist >      │
│               │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097677                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=1.2.6                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ pkgfiles                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ pkgfiles > minimist                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097678                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=1.2.6                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ conventional-changelog-writer                                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ conventional-changelog-writer > meow > minimist              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097678                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.4                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @patrickhulce/lint                                           │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @patrickhulce/lint > eslint > mkdirp > minimist              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097677                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.4                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @patrickhulce/lint                                           │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @patrickhulce/lint > eslint > file-entry-cache > flat-cache  │
│               │ > write > mkdirp > minimist                                  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097677                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Insufficient Entropy in cryptiles                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ cryptiles                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.1.2                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ jimp                                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ jimp > request > hawk > cryptiles                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1095034                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ critical      │ Prototype Pollution in minimist                              │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.2.4                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ pkgfiles                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ pkgfiles > fstream-npm > fstream-ignore > fstream > mkdirp > │
│               │ minimist                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1097677                     │
└───────────────┴──────────────────────────────────────────────────────────────┘
91 vulnerabilities found - Packages audited: 451
Severity: 1 Low | 39 Moderate | 37 High | 14 Critical
Done in 1.14s.

Thanks @neilmispelaar ! To be clear, this is all in devDeps and not affecting any consumers, right?

Just created a PR with the minimum bumps needed to address the critical vulnerabilities: #12
Would you have a set of recommended tests I can run to see if any of the updated packages broke anything? Happy to help where I can / where it's helpful.
Or if there is anything else you need please feel free to reach out.
Thanks!

fixed by #12

authors: neilmispelaar patrickhulce

repo: patrickhulce/hulk


text:
Controls are not correctly activated when filtering on load with a compound selector

E.g.:
var containerEl = document.querySelector('.container');

var mixer = mixitup(containerEl, {
    load: {
        filter: '.pink, .green'
    }
});

Fixed in v3.2.2

repo: patrickkunka/mixitup


text:
At the beginning and at the end the label of bottom axis is cut off if you add a point to the lineChart

How to reproduce
unworking_charts.patch
You can try it patch in Vico repository and run sample. The third graph will be cropped

Observed behavior
I would like to be able to draw a static graph with any number of uncut labels on the screen. Or be able to set this amount yourself. Because now it is impossible to display even all days of the week by bottom axis
Expected behavior
At the beginning and at the end the label of bottom axis is cut off if you add a point to the lineChart
Vico version(s)
Latest stable version
Android version(s)
34
Additional information
No response

Hello! Thanks for the report. The first label is truncated because there isn’t enough room for it. More horizontal padding is required. You can use the parameters of HorizontalLayout.FullWidth.default to add such padding. Vico 1.13.0 Alpha 7 introduced the ability to add unscalable padding, which would be preferable here. Vico 1.13.0 Alpha 9, which we plan on releasing within a couple of hours, will introduce an option that automatically adds the required amount of padding, ensuring that extreme HorizontalAxis labels are visible. Because the behavior described isn’t a bug, and the aforementioned new feature has already been requested, I’ll be closing this issue. I’ll let you know when Vico 1.13.0 Alpha 9 is available.

Vico 1.13.0 Alpha 9 is now rolling out. AxisItemPlacer.Horizontal.default(addExtremeLabelPadding = true) instructs a HorizontalAxis to make room for its extreme labels.

authors: TimoninD patrickmichalik

repo: patrykandpatrick/vico


text:
angular.merge is not a function .controller

Downloaded the latest, saw the above error when employing pfSimpleFilter / pfDataToolbar.  Working on the issue presently, will provide additional information as the issue becomes clearer.

I upped the angular version to 1.4, error went away, is support for 1.3.x by the wayside?

repo: patternfly/angular-patternfly


text:
How to handle word-wrapping

In cases like below where text wraps to the next line should we used the CSS hyphen option to add a hyphen between letters in words that wrap to the next line, like typical text editors do. It would be great to get some guidance from @stacymcauliffe  here.
Related to #1761

@christiemolloy - great question! I would recommend we do not allow text to wrap in the middle of a word. Is it possible to force words that do not fit on a line to get pushed automatically to the next line?
In the example you show, we've got a single 16-letter word that doesn't fit on the line, so I could see how that might force us to decide on hyphen or no hyphen... But in an instance where we're using a 16 letter word in a context like this, I might recommend we select a different word.
We also have defined standards for PatternFly on how to use truncation, although we likely wouldn't want to be forced to truncate a title like this unless we were providing a super long host name or something like that.
Hopefully this helps!

@mcoker I think instead of word-wrap: break-word; we should use hyphen: auto so that if part of an important word does break to the next line a hyphen is used to indicate that part of the word has wrapped.
@stacymcauliffe what do you think about having titles/h1-h6 using hyphen: auto ?

@christiemolloy - yep, totally agree, for the edge case where a word must wrap, I think this makes sense. Thanks!

Closing this issue for now, because we won't be adding hyphens.

authors: christiemolloy stacymcauliffe

repo: patternfly/patternfly-next


text:
Overflow component

Related PR: https://github.com/patternfly/patternfly-next/pull/2126

closed via https://github.com/patternfly/patternfly-next/pull/2126

authors: mcoker rachael-phillips


text:
Simple list and Notification drawer component styles are missing

Styles aren't showing up.

:tada: This issue has been resolved in version 2.58.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: mcoker redallen


text:
Improve AirPLS and ArPLS performance - sparse matrix operations

Description
AirPLS (Adaptive Iteratively Reweighted Penalized Least Squares) and ArPLS (Asymmetrically Reweighted Penalized Least Squares) are powerful algorithms for removing complex non-linear baselines from spectral signals. However, their computational cost can be significant, especially when processing large numbers of spectra. Currently, we use the csc_matrix representation from scipy.sparse to optimize performance, but further improvements are needed.
Improving Attempts
To improve the performance, I have tried just-in-time compilation of some key functions using numba. However, numba does not support the csc_matrix type, and I cannot JIT compile the code. To overcome this issue, I thought of looking for a numba compatible representation of sparse matrices, but could not find one. Therefore, I have created my own, together with some functions to make basic algebra operations with them code to Gist. Unfortunately, this did not improve the performance over the current implementation.
Hacktoberfest Challenge
We invite open source developers to contribute to our project during Hacktoberfest. The goal is to improve the performance of both algorithms
Here are some ideas to work on:

Find a more efficient way to JIT compile the code using tools like numba.
Investigate parallel or distributed computing techniques to speed up the processing of multiple spectra.

How to Contribute
Here is the contributing guidelines
Contact
We can have the the conversation in the Issue or the Discussion
Resources
Here are some relevant resources and references for understanding the theory and implementation of the AirPLS and ArPLS algorithms:

Paper on AirPLS: Z.-M. Zhang, S. Chen, and Y.-Z. Liang, Baseline correction using adaptive iteratively reweighted penalized least squares. Analyst 135 (5), 1138-1146 (2010).
Paper on ArPLS: Sung-June Baek, Aaron Park, Young-Jin Ahn, Jaebum Choo Baseline correction using asymmetrically reweighted penalized least squares smoothing

Hello,
Suspected Main Problem
As far as I can judge, the problem also affects the WhittakerSmooth-class because all of them rely on the sparse matrix representation of the involved weighting and penalty matrices. This is way more performant than using the pure dense NumPy-Arrays, yet dense NumPy-Arrays are part of the solution here.
With scipy.sparse most of the functions (especially the spsolve) do not distinguish between general sparse matrices that can have nonzeros at any position, e.g.,
A = [[ a    0    0    0    b]
     [ 0    c    0    d    0]
     [ 0    e    f    0    g]
     [ 0    0    0    h    0]
     [ 0    0    0    i    j]]

and the highly structured sparse format encountered in all the described functionalities (I'll call them Whittaker-Like) which is banded (only a few central diagonals are non-zero) and symmetric, e.g.,
B = [[ a    b    0    0    0]
     [ b    c    d    0    0]
     [ 0    d    e    f    0]
     [ 0    0    f    g    h]
     [ 0    0    0    h    i]]

Usually sparse solvers rely on LU-decomposition, but as far as I'm aware, the LU-decomposition of a sparse matrix like A is not necessarily sparse as well but dense, so the gain is not as much as it could be even when using a high-performance sparse solver like spsolve from pypardiso.
Proposed solution
In contrast to this, B has an LU-decomposition that is (almost) as sparse as B itself with (almost) the same banded structure, so the performance can be increased quite a lot. The LAPACK-wrapper solve_banded from scipy.linalg exploits this together with a special banded storage as dense Array which makes the excecution way faster.
Actually, one can go down even further. Since for Whittaker-Like problems, the matrix to invert is  I + lam * D.T @ D where I is the identity matrix and  D.T @ D is the square of the finite difference matrix D (which is of order difference), the problem simplifies. D.T @ D has difference zero eigenvalues and if one adds I to it, one basically lifts the main diagonal of D.T @ D. So, all the resulting eigenvalues are lifted to be > 0 (at least mathematically, but not numerically in all cases). This allows to rely on Cholesky decomposition rather than LU-decomposition which is again faster because it only computes a lower triangular matrix L instead of a lower and an upper triangular matrix L and U (but the banded structure (almost) remains for both).
Again, SciPy offers the LAPACK-wrapper solveh_banded for doing so.
Typically what I do is I first attempt to use solveh_banded and upon a numpy.linalg.LinalgError which indicates that the Cholesky decomposition was not possible, I fall back to the LU-decomposition via solve_banded which is more stable due to the fact that it also uses partial pivoting (row swapping) to ensure stability.
a_banded = ... # matrix to invert in banded storage with `n_upp` super- and `n_upp` sub-diagonals above or below the main diagonal
b = ... # right hand side vector
try:
    # Attempt banded Cholesky decomposition
    x = solveh_banded(
        a_banded[0 : n_upp + 1, ::], # taking only the superdiagonals and the main diagonal since symmetric
        b,
        lower = False, # only the superdiagonals and main diagonal were taken
    )

except np.linalg.LinalgError:
    # Fall back to banded LU decomposition
    x = solve_banded(
        l_and_u = (n_upp, n_upp), # number of super- and subdiagonals
        a_banded,
        b,
    )

Further tweaks
What I just described is as far as one can get for the general case of arbitrary difference order of D. However, there are special cases that allow for further enhancement:

difference = 1: this results in a tridiagonal I + lam * D.T @ D which can be solved even faster. This is already included in scipy.linalg.solve_banded for LU-decomposition, but unfortunately, there is no equivalent for scipy.linalg.solveh_banded for Cholesky-factorisation
difference = 2: this results in a pentadiagonal I + lam * D.T @ D for whose solution the Python package pentapy is highly optimised. As an added bonus differences = 2 is used in many baseline correction algorithms, so I'd say this will be the best speedup for the Air- and ArPLS.

pentapy also gives an overview over the performances of a variety of banded solvers which indicates that there is still a lot of performance to gain when moving away from SciPy's spsolve:

Besides, there is another added bonus.  All these functions can solve (I + lam * D.T @ D)Z = Y for Z also when Y has multiple columns, i.e., spectra to smooth. So, the same expensive factorisation - after computed only once - can be used for solving each column in Y for the corresponding column in Z which is way less expensive since the solving is only a forward and a backward substitution. On top of that, the loops then also happen in low-level languages and not in Python anymore. Consequently, the Python loops in all the Whittaker-Like functions of chemotools could simply be dropped for performance.

Hi @IruNikZe thanks a lot for the fantastic explanation and it looks super promising! I will turn this issue into a branch and you can add your changes there!!

@IruNikZe  I have assigned you to that issue, and this is the branch you can add your changes too :smile: :smile:
44-improve-airpls-and-arpls-performance-sparse-matrix-operations

Hi-hi,
It's been a while, so I wanted to give a quick update.
From testing I figured out two things:
1) Numerical instability of Whittaker smoother
Problem
For difference orders m > 4 and large signal n > 1000 solving the linear system (W + lamba * D^T @ D)z = Wy becomes instable. Unfortunately, these sizes are in the order of magnitude of the typical size of spectra and m = 6 is required for choosing lambda based on the signal in a spatially adaptive fashion (which is the next step I would want to tackle in a later issue).
The problem affects both the banded Cholesky decomposition as well as the more robust (but more expensive) banded pivoted LU-decomposition. pentapy fails as well because it uses the same matrix to solve the system.
Cause
The problem comes from squaring the linear system before solving because (W + lamba * D^T @ D) = (sqrt(W)^T @ sqrt(W) + sqrt(lambda) * sqrt(lambda) * D^T @ D, so

the weights sqrt(W) are being squared which boosts any imbalanced weight distributions even further and also causes more imbalances in normal weights (compare 1 and 1.5 which are close, but squared 1 and 2.25 are more apart, and here the order of magnitude was the same, so it can get way more worse)
sqrt(lambda) which can be arbitrarily small or large is also squared and this affects
the squaring of D which is the most critical in this context. Since D does not have full row rank (less rows than columns in this case), D^T @ D has m zero eigenvalues, so solving a system where lambda is so large that basically D^T @ D is the only numerically present summands causes a breakdown of the system.

All in all, this blows up the condition number of A = (W + lamba * D^T @ D) since it is squared. High condition numbers indicate that the system is ill-conditioned and slightest perturbations have strong effects on the final solution. With the limited precision of float64, this causes build-up of rounding errors until the system - even though solvable from a mathematical point of view - cannot be solved numerically (see this and related Wikipedia articles).
Partial solution
While it's not possible to properly cure the case when lambda is so large that the system becomes too ill-conditioned to be solved (at least this problem should not and cannot be tackled from within the solver), all the other cases can be handled way by avoiding the squaring just described.
This requires some insight in how the solver works, but basically A is decomposed in two uniquely defined triangular matrices L and L^T that have the same entries, but are transposed, i.e., A = L @ L^T. Triangular matrices are easy to solve, but we dont need uniquely defined matrices because A = S @ S^T can be solved in the exact same way as for L as long as S is lower triangular.
One such decomposition can be derived when writing A = B^T @ B where B is a full matrix with no special structure. From the above, it is obvious that B = [[sqrt(W)], [sqrt(lambda) * D]] (if you multiply it out one by one, you get A). Now, B needs to be triangularized, which can be achieved by QR-decomposition B = Q @ R. Here, Q is an orthonormal matrix (Q^T = Q^(-1); Q^T @ Q = I), and R is upper triangular.
With this A = B^T @ B = R^T @ Q^T @ Q @ R = R^T @ I @ R = R^T @ R, i.e., we get a triangular decomposition, but A was never formed by squaring which leaves the condition number relatively low (it is not squared like before), i.e., the system can be solved with a strongly improved numerical accuaracy. Since Q cancels out, only R needs to be formed. To keep the computations accurate, I resorted to a floating point exact sparse Givens rotation as well as fused-multiply-add operations (FMA).
For this I went down to Cython (1) that I combined with SciPy's cython_lapack for solving the triangular system after the QR-decomposition.
I'll go for a draft pull request first for this because we need to think about packaging in more detail then and probably first need to set up the packaging in the CI pipeline (if we wanna go down this road at all). If not, I also can re-write it in NumPy which will be slower and less accurate since it has no FMA, or Numba which also does not easily offer an FMA.
(1) and also a Rust implementation which would probably make the package more scalable as functionality grows and new external functionalities are required which can be kind of a nightmare in Cython.
Added bonus
Since the QR-decomposition was some effort to code and also optimize, I went a step further and extended the system of the Whittaker smoother to solve (K^T @ W_1 @ K + lambda * D^T @ W_2 @ D)z = K^T @ W_1 @ y where W_2 will be need later for spatial adaptiveness.
The added bonus is the kernel matrix K which also needs to be banded. Assume your spectrum was blurred by, e.g., a Lorentzian peak (like the natural linewidth), this can deconvolved out and z will not become a smooth version of y, but a smooth version with the peak width reduced since the Lorentzian is "missing". Of course other point spread functions can also be deconvolved out with this.
The original smoothing solver just has K = I.
2) Performance of baseline algorithms
Problem
The baseline algorithms can take some time for converging.
Cause
The initialisation of baseline algorithms is key here. Right now, they are initialised with a constant vector, but with a more elaborate initialisation, the convergence can be sped up dramatcially.
Solution
Typically, one would divide the signal in like 20 windows, find the minima of these windows and interpolate them. This estimate is often very very close to the baseline (given that lambda is well chosen).
Alongside this, I would also give the baseline two modes:

the baseline is below the peaks (e.g., absorbance spectroscopy)
the baseline is above the peaks (e.g., transmittance spectroscopy)

Implications
I would have to adapt the tests for ArPLS and AirPLS then because the different initialisation will change the results. Thus, the tests will fail because they just reference to the result of the very initial implementation (at least that's my guess).
Any feedback is highly welcome, but I guess when the draft pull request is submitted, things will get clearer 😁

Hello,
Since I took over from my account @IruNikZe, I want to give an update here.
So far I

unified the implementations of WhittakerSmooth, ArPls, and AirPls
replaced the sparse solvers and even all sparse matrices which come with an incredible performance penalty due to the initialisation overhead (magnitudes longer than what an optimized solver takes). Now, everything is computed in compact dense storage which was a bit more involved in terms of logic but significantly faster
added weights and an automated lambda selection to the WhittakerSmooth. The automated selection is designed modular. Currently, it relies on the log marginal likelihood, but something like (Generalized) Cross-Validation would be a nice addition here, so I left it more in a plug-and-play fashion from an implementation point of view
tested all new functionalities (1 test for the automated lambda yet to be implemented)
parallelized the tests to make them run in a fraction of the original time

None of the changes broke the API. However, it is extended quite a bit for the WhittakerSmooth where lam now allows fixed and automatically fitted smoothing.
Besides that there are some TODOs where we need to improve the numerical stability for large signals, but I will open two separate issues for this: one for the implementation itself and one for extending the packaging by Cython-builds because we will need access to some LAPACK functions that don't have fancy high level wrappers in SciPy and need to be accessed via Cython, namely dgbcon and dpbcon. These functions will allow us to run the Whittaker solver even faster because they offer a sanity check for the results of the current and a faster method. Such a check is currently missing and made me resort to scipy.linalg.solve_banded (current method mentioned) which can be too conservative and thus slow in some cases, but also to instable in other scenarios.
So right now, we have a good level of safety which can still be improved by enabling more performance (scipy.linalg.solveh_banded (faster method mentioned)) and also an even more safe fallback.
Now, I still need to

merge the current main into the respective branch
finalise the docstrings and comments (@paucablop  I might need some helps regarding the package docs)
change the baseline initialisation in ArPls and AirPls because the first iterations from a zero baseline can be far away from the true baseline which can be estimated better

Since the Pull Request will be quite massive already, I would for now not touch the baseline initialisation (which will require new classes) and open a Pull Request with what is there this week (as a draft due to the package docs).
@paucablop Should we leave the branch open and then do another Pull Request when the new baseline initialisation is finished to make sure everything is updated by the branch linked to this issue?
I wish a great start into the week 👋 😸

I'd already like to add this tiny teaser for the automated smoothing as a result of the tests.
I tried to mimic two peaks with Poisson-noise sitting on a baseline. I drew 10 repetitions of random Poisson noise and used the new weighting functionality to show that regions with high noise (lower weights) are smoothed stronger and vice versa 😸

Even though the smoothing itself is repeated quite often to find the optimum lambda (like 40 times or so), this still runs in less than 5 milliseconds on my relatively old machine 🏃

@paucablop In case you already want to see the new functionalities of WhittakerSmooth, you can try it out in this notebook:
whittaker_show.txt
After you pulled the current version of the issue/feature branch, all you need to do is:

replace the .txt-file-ending of the notebook by .ipynb
start the notebook in an environment that has jupyter installed (Python >= 3.9)
select that environment as the kernel
replace the inital path by the path where chemotools is installed on your machine (keep the \ on Windows in mind)

Then, you can go through the notebook. It has short documentations as Markdown cells whenever required.
It also provides you timings and actually I need to correct what I said. The automated smoothing takes 10 ms on my machine 🙃

I timed the implementation against this Rust-based package.
I found that:

on a single smooth level, we are slower (450 µs in the notebook vs. 60 µs)
however, for automatic lambda-fitting, we are faster (10 ms µs in the notebook vs 30 ms)

So, I guess we are on a good way here. In the future, this could be further improved by moving the main logic to Cython or also Rust.

Hi @MothNik
I went through the branch, and it looks really good, very nice job, and also very well documented :smile: Super exciting to start testing it out. Also thanks a lot for enabling a nice testing environment. I really like the auto lambda functionality :star_struck:! I have started testing it out on my end (I could not wait :stuck_out_tongue_closed_eyes: ) and will keep doing so during this week (I have so many use cases I want to test the functions :yum:).
I think that it would be nice to start the PR process when you are ready. I could give you a hand with the documentation (eventually, I would like to change the documentation site, but I think for now we keep it as it is).
I usually like to keep the branches short, but since this would be a very related development, I think it is a good idea to leave the branch open and do another PR later on.
I am very much looking forward to the PR, and once again, thank you very much for your brilliant contribution!

@paucablop
After I've seen #56, I renamed the window_size of the function to window_length as for SciPy's savgol_filter.
Here is an updated notebook that works with the current branch version:
whittaker_show_v3_noise_estim.txt

@MothNik Just saw the PR, very exciting!!
I have been checking the three functions with some of use-cases, and it is working really good, it is significantly faster, on the data I used to benchmark these two versions, I could see around an order of magnitude improvement 🚀🚀, and numerical stability (spectra of around 3500 datapoints) 🦾🦾. Very good job!!!
I indeed have the opportunity to estimate the weights from the inversed standard deviation of several measurements, but I know it is not always the case, so I am supper happy you suggested a method to do so, I did not know this method before, but after reading the attached paper you included, it seems like a very smart idea, and brilliant addition to the methods 💡💡Also, having the function estimate_noise_stddev accessible from utils makes it quite nice to have, I think there can also for other interesting use cases for it, next week I would like to spend some more time playing with it. Thanks for being so diligent on the documentation and referencing of the functions! 🤓🤓
Regarding the windows_size, windows_length, I have no strong opinions, I think that it makes sense calling it windows_size as you also point out it is also used within scipy!
I will start jump to the PR right away!!! 😄😄 Thanks again for the great job @MothNik !!

@paucablop You're welcome! 😸
I will quickly rename it back to window_size then 💪

@paucablop Renamed and tested 👍

authors: IruNikZe MothNik paucablop

repo: paucablop/chemotools


text:
Make into proper lib add additional dependencies and CLI

…nltk punkt

@thiswillbeyourgithub I think I address all your comments. Lemme know what you think

Thanks for the improvements. I added a few more questions.

repo: paulbricman/autocards


text:
Edit cell when pressing .

Hello Paul!
One of my user reported to me that you can't edit a cell when you press the dot sign
on the numpad on a cell.

It has been done, updates on the master repository.

the feature is working as asked thank you for the work!

authors: felixvandal paulhodel

repo: paulhodel/jexcel


text:
Update sbt to 1.4.6

Updates org.scala-sbt:sbt from 1.4.3 to 1.4.6.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

labels: library-update, semver-patch

Superseded by #464.

Superseded by #464.

repo: pauljamescleary/scala-pet-store


text:
🛑 Taqueria Bonjour is down

In 805b1d6, Taqueria Bonjour (https://bonjour.paulynomial.com) was down:

HTTP code: 503
Response time: 375 ms

Resolved: Taqueria Bonjour is back up in 76c5a5c.

repo: paulkiernan/paulynomial-uptime


text:
🛑 Paulynomial (index) is down

In 5a8d969, Paulynomial (index) (https://paulynomial.com) was down:

HTTP code: 530
Response time: 53 ms

Resolved: Paulynomial (index) is back up in c5b608f after 7 minutes.


text:
npm install fails: getaddrinfo ENOTFOUND artprod.dev.bloomberg.com

After cloning the repo and running npm install, the install fails with:
npm ERR! code ENOTFOUND
npm ERR! syscall getaddrinfo
npm ERR! errno ENOTFOUND
npm ERR! network request to http://artprod.dev.bloomberg.com/artifactory/api/npm/npm-repos/webpack-sources/-/webpack-sources-1.3.0.tgz failed, reason: getaddrinfo ENOTFOUND artprod.dev.bloomberg.com
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/ben/.npm/_logs/2021-03-17T17_09_32_609Z-debug.log

Some packages in package-lock.json appear to have references to this domain in their resolved attribute. Removing package-lock.json and rerunning npm install seems to fix this.

Fixed in latest release

authors: bentorkington paulmg


text:
relative error is too big for Math.expm1 and Math.log1p

Math.expm1(-2e-17)
Math.log1p(-2e-17)

The spec doesn't tend to define precision very well for its Math operations. Can you link to something in the spec that's incorrect here? What values do you expect? In what browser(s) are you getting which values that you don't expect?

@ljharb , so why not to use Math.exp(x) - 1 or Math.log(1 + x) ?
You can define Math.expm1 = function (){} according to the spec, but does this make your shim good?
https://github.com/kangax/compat-table/issues/392

I'm still not understanding what this issue is about. All three of the variables referenced in the compat-table issue return true for me in Chrome with or without the es6-shim (since they're native). In which browsers are you seeing any of these return false?
Also, if the spec does not explicitly say that your error criteria must be applied, then they need not be applied by this shim or by any browsers. I'm definitely open to making them more precise, but the shims are "good" so long as they provide any guarantees the spec requires of them, nothing more.

Thanks for your examples - I'm sorry we continue to have misunderstandings in these issues, I hope we can chalk it up to language barriers.
I've added failing tests for log1p, and corrected them, in the es6-shim. While our number 1 priority is to comply with the spec, number 2 is, when it doesn't conflict with the spec, to comply with the majority of implementations - in this case, both of these math functions aren't returning the same values as their native counterparts.
I can't locate a polyfill for expm1 that's not exp(1) - 1, and that polyfill has the error you described. Do you have a suggested implementation?

I have only this(using Taylor series expansion):
Math.expm1 = function (x) {
  if (x !== x || x === 0) {
    return x;
  }
  if (Math.abs(x) > 1 / 2) {
    return Math.exp(x) - 1;
  }
  var t = x;
  var s = 0;
  var n = 1;
  while (s + t !== s) {
    s += t;
    n += 1;
    t *= x;
    t /= n;
  }
  return s;
};

@Yaffle Thanks, that seems to do it! Can you suggest more meaningful variable names than "t", "s", and "n"?

authors: Yaffle ljharb


text:
why secp256k1.utils.randomPrivateKey() produce a 32 bytes key?

And so do the secp256k1.getPublicKey().
Shouldn't they produce keys of 64 bytes?
BTW: I tried it in noble/curves, guess it's the same here.

Because keys are 32 byte, not 64.

I see, for my case, I need 64 bytes of the uncompressed public key to talk to another program that uses another library in another language..
So, is there a way in this lib to uncompress it?

authors: paulmillr zzzh


text:
web3@1.3.5

PS C:\Windows\System32> npm install --save web3@1.3.5
npm WARN deprecated mkdirp-promise@5.0.1: This package is broken and no longer maintained. 'mkdirp' itself supports promises now, please switch to that.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated multicodec@0.5.7: stable api reached

bufferutil@4.0.3 install C:\Windows\System32\node_modules\bufferutil
node-gyp-build

keccak@3.0.1 install C:\Windows\System32\node_modules\keccak
node-gyp-build || exit 0

secp256k1@4.0.2 install C:\Windows\System32\node_modules\secp256k1
node-gyp-build || exit 0

utf-8-validate@5.0.4 install C:\Windows\System32\node_modules\utf-8-validate
node-gyp-build

web3-bzz@1.3.5 postinstall C:\Windows\System32\node_modules\web3-bzz
echo "WARNING: the web3-bzz api will be deprecated in the next version"

"WARNING: the web3-bzz api will be deprecated in the next version"

web3-shh@1.3.5 postinstall C:\Windows\System32\node_modules\web3-shh
echo "WARNING: the web3-shh api will be deprecated in the next version"

"WARNING: the web3-shh api will be deprecated in the next version"

web3@1.3.5 postinstall C:\Windows\System32\node_modules\web3
echo "WARNING: the web3-shh and web3-bzz api will be deprecated in the next version"

"WARNING: the web3-shh and web3-bzz api will be deprecated in the next version"
npm WARN saveError ENOENT: no such file or directory, open 'C:\Windows\System32\package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open 'C:\Windows\System32\package.json'
npm WARN System32 No description
npm WARN System32 No repository field.
npm WARN System32 No README data
npm WARN System32 No license field.

web3@1.3.5
added 359 packages from 325 contributors and audited 359 packages in 40.434s

Hi @emiratay! Please use StackOverflow for generic questions about node.js development.

authors: emiratay paulrberg

repo: paulrberg/create-eth-app