text:
Opt-In js.Any reflective call support

We could support reflective calls on subtypes of js.Any if we are "allowed" to leak prototype members of the objects in question.
To some projects this could be acceptable. Should we add opt-in support for this? (Note that this will require us to generate code for RawJS types).

This feature can improve the interoperability with TypeScript, because TypeScript's interfaces are structural types.
Currently we convert a TypeScript interface to a Scala.js trait that extends js.Object. As a result, compatible TypeScript types will not be able to convert to each other in Scala.js.

You really wouldn't want to declare every single interface coming from TypeScript as a structural type. Structural types in Scala are a second-class citizen to begin with. For example, they cannot be recursive. They have no support in the compiler to cache recurring subtyping checks, so each check has to go through the whole member list, increasing compile times. They cannot declare concrete members, so the = js.undefined optional members of JS traits wouldn't be an option. Etc.

You are right. Scala's type system is almost a superset of Java's, while TypeScript's type system is more natural for JavaScript.

authors: Atry gzm0 sjrd


text:
Remove the hack to fix percentage signs in Node.js.

Back when that hack was introduced in 1fc8d0526ba9b46049c0deefc3fe3fa93d3271b0, Node.js' console.log deduplicated consecutive % signs in its argument, even if it was a single argument. At the time we thought that was intentional, and we added our hack to have it behave in a more standard way for Scala.js.
The deduplication with only 1 argument turned out to be a bug, recognized as such in https://github.com/nodejs/node/issues/3396, when it was accidentally fixed in Node.js 4. It is now the documented behavior, see https://nodejs.org/api/util.html#util_util_format_format_args which is referenced from https://nodejs.org/api/console.html#console_console_log_data_args.

Hum, I should probably have elaborated more on the commit message.
It's not in 0.6.x because it is a breaking change. Our hack was too eager, and was doing too much. In trying the solve the case where only one string argument with %% is passed to console.log, it "solved" a few other things:

It was deduplicating %% signs also when more than one argument was given
It was forcing the conversion of the first argument to be a string, preventing Node.js' default console.log behavior of displaying non-strings in a more elaborate way than just their toString().

So removing the hack altogether restores those two behaviors to the Node.js defaults. Since the tests were effectively testing our modified behavior, the tests need to change (and in the same commit).
All we really care about is that if we pass in 1 string argument, that it is sent to stdout as-is, without processing any %. Cases where we pass more than 1 argument, or where we pass a non-string argument, are of no concern to Scala.js per se, and should be left to the particular engine to deal with as they wish.

Makes sense.

source: gharchive/pull-request

repo: scala-js/scala-js

license_type: permissive

license_source: github-api


text:
How to check if a Symbol has the override modifier ?

Hi !
I am rewriting the ExtractAPI phase of dotty using TASTy Query and I need to query if a Symbol has the override modifier.
Here is what the original dotty source code looks like (available in full here):
def apiModifiers(sym: Symbol): api.Modifiers = {
  val absOver = sym.is(AbsOverride)
  val abs = absOver || sym.isOneOf(Trait | Abstract | Deferred)
  val over = absOver || sym.is(Override)
  new api.Modifiers(abs, over, [...])
}

My current solution boils down to doing
sym.flags.is(Override)

However, Symbol.flags is protected, so I invoke it using reflection, which does not seem like a great solution.
Does someone know a better way of checking if a Symbol has the override modifier ?
Thanks in advance for your assistance !

advice from @sjrd - ignore the override flag altogether - its not information that could ever require a downstream recompilation

source: gharchive/issue

authors: alexishorner bishabosha

repo: scalacenter/tasty-query


text:
Index class primary constructor parameters

Fixes #134. I noticed that the new "clear index" command doesn't cleanup the in-memory indexedJars map forcing me to restart the server to re-index.
This is my first PR developed (almost) entirely through vscode 🎉

I noticed that the new "clear index" command doesn't cleanup the in-memory indexedJars map forcing me to restart the server to re-index.

Good point, I remind thinking of that and I eventually decided that clearing and restarting was good enough for that PR. We may want to add a command on-top-of that to also trigger re-indexing.

authors: gabro olafurpg

repo: scalameta/language-server


text:
Cross-compile stuff for 2.11 too

This should be needed to embed that in almond (which supports both 2.11 and 2.12 - no plan to drop 2.11 support in the short term).

About https://github.com/scalameta/mbrowse/pull/110/commits/b45acd63792898db1a7fd0ae6e873a1bf9545770, there should have been no problem I think. cli doesn't depend on js.

I had initially disabled 2.11 for the js module, as it was taking too long to compile locally.

I added b45acd6 only to make +publish work, I tried +publishLocal and it failed with resolution errors. It's not strictly necessary but I like the simplicity of using 2.11-only artifacts for the 2.11 cross-build.

I had initially disabled 2.11 for the js module, as it was taking too long to compile locally.

Yeah, bundling the optimized client for the cli takes a really long time 😢 Fortunately, it's not as bad with fastOptJS when iterating on the webpack dev server.

We'll see if release works with the new cross-build for merge 🍿 https://travis-ci.org/scalameta/mbrowse/jobs/445668791

authors: alexarchambault olafurpg

repo: scalameta/mbrowse


text:
Update sbt-scalajs, scalajs-compiler, ... to 1.9.0

Updates

org.scala-js:sbt-scalajs
org.scala-js:scalajs-compiler
org.scala-js:scalajs-junit-test-runtime
org.scala-js:scalajs-library
org.scala-js:scalajs-library_2.13
org.scala-js:scalajs-test-bridge
org.scala-js:scalajs-test-bridge_2.13
org.scala-js:scalajs-test-interface

from 1.6.0 to 1.9.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-js" } ]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #507.

repo: scalameta/munit


text:
Ref.symbol for unattributed trees

DISCLAIMER: Ref.symbol is only going to be available in scala.meta.semantic.v1. The concept of symbols has a benefit of being easy to implement, but has a bunch of known issues discovered in the scala.reflect experiment. Therefore, in scala.meta.semantic.v2, we're planning to remove symbols and Ref.symbol and replace them with something else (#610).

Prerequisites:

[x] Online mirror with support for Ref.symbol (#606)

The next step after supporting attributed trees in #606 and #607 is implementing Ref.symbol for unattributed trees. We'd like to shield the user from understanding the distinction between unattributed and attributed trees, so we'll perform typechecking under the covers.
Usage example:
implicit val m = Mirror(...)
t"Int".symbol // _root_.scala.Int#
q"implicitly".symbol // _root_.scala.Predef.implicitly(Ljava/lang/Object;)Ljava/lang/Object;.

Implementation plan:

[ ] Track positions for quasiquotes, so that we have easy locations for things like t"Int" (#46, #159 and #510 will likely be relevant)
[x] Implement on-the-fly typechecking for unattributed trees

Simple quasiquotes should be easy, because they should have positions by now
Dynamically parsed strings should be easy as well by the same reasoning
Unattributed trees created manually or thrown together via unquoting/splicing will probably be hard, but we should try to support them too
Reject mixed unattributed/attributed trees crashing with an error message referring to #621

[x] Some previous work in this domain:

https://github.com/scalameta/scalameta/blob/7d6f917de39f8bfdbeb1f87d145f9797803d9ebb/scalahost/src/main/scala/scala/meta/internal/hosts/scalac/contexts/Adapter.scala#L54
Think whether #286 applies here

[x] Upgrade implementations of #606 and #607 to use on-the-fly typechecking in cases when there's a semantic db miss

I didn't need to implement position tracking for quasiquotes in order to hack up this functionality. For unpositioned trees, I'm simply prettyprinting and then immediately reparsing, obtaining a structurally-equivalent, but now well-positioned tree. Seems to work well for the original use cases of t"Int" or q"List". Closing this issue and leaving the rest for #621.

As described in #808, we're giving up on runtime typechecking. As a result, this API becomes impossible to implement, so #808 removes it.

repo: scalameta/scalameta

license_source: bigquery


text:
Add UI for creating reminders

Right now it's hardcoded...

[ ] Remind me at specific point in time
[ ] Remind me every x minutes

repo: scanf/Awesom-O


text:
Support for "application/vnd.ms-office.chartstyle+xml"

After creating a basic Powerpoint file via Microsoft Office (OSX) with an embedded Excel chart, I attempted to open the file with python-pptx and save it just to ensure opening and saving it would still work in Powerpoint. However, I get this error.
  File "power-point-excel.py", line 6, in <module>
    presentation = Presentation('/Presentation1-test.pptx')
  File "/work/python-ppt/pptx/api.py", line 29, in __init__
    self._package = _Package(file)
  File "/work/python-ppt/pptx/presentation.py", line 96, in __init__
    self.__open(file)
  File "/work/python-ppt/pptx/presentation.py", line 219, in __open
    pkg = pptx.packaging.Package().open(file)
  File "/work/python-ppt/pptx/packaging.py", line 100, in open
    part._load(fs, partname, cti, parts_dict)
  File "/work/python-ppt/pptx/packaging.py", line 256, in _load
    target_part._load(fs, target_partname, ct_dict, parts_dict)
  File "/work/python-ppt/pptx/packaging.py", line 256, in _load
    target_part._load(fs, target_partname, ct_dict, parts_dict)
  File "/work/python-ppt/pptx/packaging.py", line 256, in _load
    target_part._load(fs, target_partname, ct_dict, parts_dict)
  File "/work/python-ppt/pptx/packaging.py", line 240, in _load
    self.typespec = PartTypeSpec(content_type)
  File "/work/python-ppt/pptx/packaging.py", line 517, in __init__
    raise KeyError(tmpl % content_type)
KeyError: "no content type 'application/vnd.ms-office.chartstyle+xml' in pptx.spec.pml_parttypes"

It seems that the content type produced by my version of Office's Powerpoint isn't supported?
Application information: Microsoft Powerpoint for Mac Version 15.27 (161010)
The Powerpoint document does have an embedded Excel chart in it.

What PowerPoint version is that in terms of what it shows on the screen or whatever? Like PowerPoint 2011? Maybe PowerPoint 2015?
Also, what version is the Excel you're using? Anything distinctive about the chart? Do you have chart styles defined on your machine that you might be using in the chart (perhaps located in the ~/Library/Application Support/Microsoft/Office/..something templates  directory, possibly a .crtx file)?
I think we can fix this just with adding a content type, but I'd like to have an idea why it's showing up suddenly.
Can you provide a small-as-possible example .pptx file that exhibits this behavior? I'd like to look under its hood and see what type of parts the package contains. There must be a chart style part or something in there.

Okay, some research seems to indicate this is a new part type that has been added (without benefit of documentation :). If you send over the example I'll investigate further. There's a new revision of the ISO/IEC 29500 spec (2016 version). It doesn't mention it but perhaps the .xsd and other "electronic inserts" it comes with will shed some light.

@Emn1ty Any response on this one? I'll be pushing a new release here in a few days and if this turns out to be simple (as I think it may) I'd like to include getting this working as it's bound to come up for other folks as they get to newer versions.

Hey @scanny, my apologies as it has been quite hectic at work recently and just haven't noticed this. To answer your request I the sample files here: SamplePPT.zip
I was looking into this because I was attempting to use python-pptx to embed linked charts from excel into powerpoint. Hope these files help though :)

Okay, for some reason I can't reproduce this error, but I did add a small handful of Microsoft extension content types I found, including this Chart Style part type. We'll see how we go :)
This will be in the release later today, v0.6.2.

repo: scanny/python-pptx

license: MIT


text:
Fail: AttributeError: module 'collections' has no attribute 'Container'

Hi, I try to start "python-pptx", but i have Error:
AttributeError: module 'collections' has no attribute 'Container'
in File "...\lib\site-packages\pptx_init_.py", line 14, in
Use Python 3.10, python-pptx (0.6.21), lxmx (4.6.3).
Please, help me.

Thanks @scanny. Looking at the Python docs this code looks like it should work on 3.10. I checked the 3.10 docs and there's no sign even of a deprecation.
(The docs got overhauled sometime between 3.8 and 3.9 - expanding with examples.)
I'm not - at least for the moment - going to flag 3.10 as problematic for python-pptx or my package that relies on it.

authors: MartinPacker Vladzrabbit


text:
platform Ubuntu Server 14.04

this is not an issue
just wanted to say, that script
./X11rdp-o-matic.sh –justdoit
works fine on Ubuntu 14.04.2 Server

working fine on Ubuntu 15.05, but with some exceptions.

Need to use xfce de to be able to login. For that, apt-get install xfce4
echo xfce4-session >~/.xsession

additionally xrdp and xrdp-sessman will not start out of box via systemd =\ But working fine with -ns params

can you explain about which -ns switch you are talking about? @hapylestat

the -ns switch stands for "no service" or "no daemon"
It means run in foreground and not daemonize.

@jsorg71 correct, systemd do the job for making process to be daemon, otherwise he think that process die (not really known how daemonization is realized on xrdp).

license: isc


text:
Error when loading database file

When using the example code in Python 3.8 i get this error :
Traceback (most recent call last):
  File "C:/Users/loicn/backend_db.py", line 7, in <module>
    db = connection.my_tiny_database
  File "C:\Users\loicn\AppData\Local\Programs\Python\Python38\lib\site-packages\tinymongo\tinymongo.py", line 73, in __getattr__
    return TinyMongoDatabase(name, self._foldername, self._storage)
  File "C:\Users\loicn\AppData\Local\Programs\Python\Python38\lib\site-packages\tinymongo\tinymongo.py", line 73, in __getattr__
    return TinyMongoDatabase(name, self._foldername, self._storage)
  File "C:\Users\loicn\AppData\Local\Programs\Python\Python38\lib\site-packages\tinymongo\tinymongo.py", line 73, in __getattr__
    return TinyMongoDatabase(name, self._foldername, self._storage)
  [Previous line repeated 996 more times]
RecursionError: maximum recursion depth exceeded

Process finished with exit code 1

What version of Python are you running?  I am seeing this on Python 3.8.2, but not on Python 3.6.9.

I figured it out.
TinyDB which TinyMongo wraps around has slightly different behavior in Python 3.8.  So there will need to be some changes to TinyMongo to compensate for this.
There is a class "property" from tinydb that is not working in the newest versions of Python.   It requires a fix of the class TinyMongoClient() property _storage.
If you want a temporary work-around you can override the class
import tinymongo as tm
import tinydb

class TinyMongoClient(tm.TinyMongoClient):
    @property
    def _storage(self):
        return tinydb.storages.JSONStorage

If you put that code at the top of your module, you can now use TinyMongoClient in the default JSON storage mode.

Maybe you could try the master branch which seems to be TinyDB v4 adopted in this fork: https://github.com/cjboyle/tinymongo

That fork works, thanks @davidlatwe . I had hacked the module to work. But my version "locked" TinyMongo to the JSONstorage and would not have supported other middlewares. So your version is definitely welcome.
I did not run the suite of tinymongo tests, but the tests in my project associated with its usage of tinymongo passed on all versions of Python 3.5+
    def _storage(self):
        """comments removed"""
        return TinyDB.default_storage_class

What version of Python are you running? I am seeing this on Python 3.8.2, but not on Python 3.6.9.

I was using 3.8.2 but with the workaround it works

I have the same problem using python3.7.5

Thanks for the information.
On Sun, May 31, 2020 at 12:37 AM Marcos Diniz notifications@github.com
wrote:

I have the same problem using python3.7.5
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/schapman1974/tinymongo/issues/58#issuecomment-636419963,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA6KUBCKL7IE6GK5H6PV7X3RUHNIXANCNFSM4M75KOOQ
.

--
Jeff Muday
Instructional Technology Analyst
Wake Forest University
1834 Wake Forest Rd.
Winston Hall Rm. 27
Winston Salem, NC 27106
mudayja @ wfu . edu
p 336.758.6171
f  336.758.6004

I actually got this on 3.7.6
The class TinyMongoClient(tm.TinyMongoClient) solution didn't seem to work for me, but I can't 100% ensure that I didn't mess something else up :)

I have used this extensively-- so I have a high level of confidence it
works and will continue to work even if Schapman upgrades the code.
Make sure you are importing tinymongo and tinydb.  In this case we are
"overriding" the TinyMongoClient class property import to ensure we have
the correct return.  It might look a little awkward, but it achieves the
same effect as Conner Boyle's fork (Thank you David Lai for pointing this
out)
######### code starts here
import tinymongo as tm
import tinydb
class TinyMongoClient(tm.TinyMongoClient):
@property
def _storage(self):
return tinydb.storages.JSONStorage
############ code ends here
ALTERNATELY... You could just work with the forked version from Conner
Boyle's github repository.
$ pip uninstall tinymongo
$ pip install git+https://github.com/cjboyle/tinymongo
The only "gotcha" of doing it this way is that you cannot uninstall
CJBoyle's version with a "pip uninstall", you have to remove it the old
fashioned way of going into the environment packages and deleting the
directory.
On Wed, Jun 17, 2020 at 2:59 PM Selwyn-Lloyd McPherson <
notifications@github.com> wrote:

I actually got this on 3.7.6
The class TinyMongoClient(tm.TinyMongoClient) solution didn't seem to
work for me, but I can't 100% ensure that I didn't mess something else up :)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/schapman1974/tinymongo/issues/58#issuecomment-645561061,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA6KUBCOTUYWTKDVDZFO3MLRXEHBDANCNFSM4M75KOOQ
.

--
Jeff Muday
Instructional Technology Analyst
Wake Forest University
1834 Wake Forest Rd.
Winston Hall Rm. 27
Winston Salem, NC 27106
mudayja @ wfu . edu
p 336.758.6171
f  336.758.6004

Still works for me FYI

How would one support custom serialization such as the DateTimeSerializer provided with tinymongo using this work around?
The suggested method of supporting datetime objects in tinymongo already overrides this property:
class TinyMongoClient(tm.TinyMongoClient):     @property     def _storage(self):         serialization = SerializationMiddleware()         serialization.register_serializer(tm.serializers.DateTimeSerializer(), 'TinyDate')         return serialization

authors: JRileyH davidlatwe frankcarey jefmud lnoest marcos-diniz mypolopony

license: mit


text:
chore: Deduplicate collected errors that don't come from failed checks

Forgot to implement it in #976
Ref: #975

Codecov Report

Merging #977 (7a06dd2) into master (441f825) will decrease coverage by 0.17%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #977      +/-   ##
==========================================
- Coverage   99.78%   99.60%   -0.18%
==========================================
  Files          66       66
  Lines        4573     4580       +7
  Branches      751      753       +2
==========================================
- Hits         4563     4562       -1
- Misses          3        8       +5
- Partials        7       10       +3

Flag
Coverage Δ

unittests
99.60% <100.00%> (-0.14%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/schemathesis/runner/impl/core.py
98.83% <100.00%> (+0.03%)
:arrow_up:

src/schemathesis/_compat.py
50.00% <0.00%> (-50.00%)
:arrow_down:

src/schemathesis/extra/pytest_plugin.py
97.77% <0.00%> (-2.23%)
:arrow_down:

src/schemathesis/cli/callbacks.py
98.19% <0.00%> (-1.81%)
:arrow_down:

src/schemathesis/cli/cassettes.py
98.30% <0.00%> (-1.70%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 441f825...7a06dd2. Read the comment docs.

repo: schemathesis/schemathesis


text:
Replaced renamed pep8 with pycodestyle and added install dependency

The pep8 module was renamed to pycodestyle. flake8 already switched to pycodestyle 2.0.0. Because of that using flake8 and flake8-todo results in a dependency glitch. I also added a install dependency on pycodestyle. Until now this dependency was only indirectly given because flake8 had a dependency on pep8.

Thanks. Can you please pin the dependency as flake8 is doing ('pycodestyle >= 2.0, < 2.1').

Done!

any chance of getting this merged and having a new release of the package?

Yes, I'll try to find time for a release at the weekend.

Thanks!

authors: ferraith hvelarde schlamar


text:
Remove 'overweight'

This commit removes the word 'overweight' from the corpus. This is an
insensitive term that implies that there is a certain weight that one should
be, rather than 'over' or 'under'.

Thanks, I'll publish a new release soon.

authors: mwaggett schmich

repo: schmich/spicy-proton


text:
🛑 averagedrecipe is down

In caed34a, averagedrecipe (https://averagedrecipe.com/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: averagedrecipe is back up in 0eafd9c after 8 minutes.

repo: schollz/upptime


text:
Minor documentation error

Should this param be of type Shape instead of Body?

Yes! Fixed it. Thanks!

authors: schteppe xeolabs


text:
Collider completly within zone

Hello is there a way to check if a collider is completly within a zone.
Like I have a cylinder and I want to check if it is inside a rectangle.

Currently no. Do other physics engines offer this?
Perhaps you could solve it in some other way, like making a box "surface" out of 6 boxes and check when the cylinder gets inside and stops overlapping them... Does this make sense?

Well if that's how it is, i would rather use raycasts then.
My shape is a cylinder which height's is 4 times bigger than its radius. So I just need to put raycasts around the rectangular shape and some other raycasts inside.
If the collider is touched by the outer rays, it's not inside the zone.
If it collide with the inner rays only, than it's within the zone.

Sounds legit!

authors: harraps schteppe


text:
Inclui preprints na listagem de coleções

Este PR inclui preprints na listagem de coleções. Para que a coleção aparece de fato na listagem será necessário adicionar um registro correspondente no endpoint http://articlemeta.scielo.org/api/v1/collection/identifiers/, do ArticleMeta.
Onde a revisão poderia começar?
n/a
Como este poderia ser testado manualmente?
Sugiro que crie uma instância local apontando para o backend de homologação.
Algum cenário de contexto que queira dar?
n/a
Screenshots
n/a
Quais são tickets relevantes?
#147
Referências
n/a

Marquei como draft pois entendo que é interessante atualizarmos o endpoint do ArticleMeta antes de incorporar este código.


text:
ci: Add more integration tests in Fedora

[ ] Run all/more pytest tests
[ ] Run freethreading tests

Note-to-self: Separate the plans with and without CMake/ninja installed. Could cover #863

repo: scikit-build/scikit-build-core


text:
Tell a point with its uncertainty

In the long term, it would be nice if one could tell a pair (x, y) along with uncertainty on y.
For example, when y is a cross-validation estimate of the performance of a ML model parameterized by x, then we also have an uncertainty estimate of y. This piece of information is currently not leveraged, which is too bad!
Not that this is however not a trivial change, since none of the regression models we have support that. The old version of gaussian processes in scikit-learn supported a nugget argument though, through which this information could be fed.

alpha does the same work as what nugget does no? (it can be array as well). I opened an issue here (https://github.com/scikit-learn/scikit-learn/issues/7821) to set it at fit time but was unable to review the Pull Request.

Good point. While I am not sure about what will happen with regards to adding sample_alpha at fit (see scikit-learn/scikit-learn#7985), we could backport such change in skopt.

authors: MechCoder glouppe

repo: scikit-optimize/scikit-optimize


text:
RFC: seeking thoughts on rewriting special functions written in Cython into C++

At the moment we have a number of functions in special where we have provided our own implementation using Cython. I think Cython was chosen for its ergonomics and because it may be easier to maintain for developers used to Python. Its use comes with some downsides though, and using C++ brings upsides.
@izaid has pointed out that if we can supply C++ implementations in header files, than other array computing libraries such as CuPy, PyTorch, or Jax could potentially use these implementations. https://github.com/scipy/scipy/pull/19023#issuecomment-1689050946. @mdhaber, izaid and myself have been discussing proposing an array API extension for special functions, and moving to C++ could help facilitate that.
Lack of interoperability affects us internally too. Because of how the build system works currently for special functions, the compiled kernels for special functions written in Cython can not be called from compiled kernels written in C or C++. This is blocking https://github.com/scipy/scipy/pull/19287 at the moment, cc @rlucas7.
Since the special function infrastructure already handles generating the ufunc boilerplate on the NumPy C API side, we don't get the primary benefit of using Cython in special. Where Cython really shines is in writing extensions without needing to work out the somewhat tedious C API boilerplate.
@izaid has generously offered to help review PRs I submit for rewriting special functions from Cython into C++, and I would be happy to review any PRs he submits. We would prioritize functions that we think should be included in an array API extension.
My thought here is that we should try to write these in the style of Boost special functions, making use of Boost's fundamental building blocks when possible, with the idea that perhaps we could eventually contribute these upstream to Boost. At the moment, the biggest thing missing from Boost is complex support for special functions where that is relevant. cc @mborland. This is the approach @tirthasheshpatel and I are looking at for our SDG to work on special infrastructure; making use of Boost's internal building blocks, and writing our own building blocks in a similar style if needed.
As part of this, we would also avoid writing any new special functions in Cython, recommending instead C++ in the development docs.

As long as we don't deal with the nasty bits of C++, and I'm not implying a C++ style guide like Google or Spotify, I think this would be really nice. Is there any specific reason for not choosing plain C?
With C++, a typical situation is that the project becomes a grenade-driven development (GDD) waiting for us in the build stage. Especially, if MSVC will be involved, I'd be terrified to use complex numbers (all past traumas) but I'm no C++ expert by any measure maybe things are much better now with the later versions.
One definite advantage of Cython is memoryviews and NumPy interplay otherwise the usual footgun shananigans apply. But it becomes harder to detect hence there is some reduction in the available maintainer resource but nevertheless the interplay with other package maintainers might make it more robust over time.

I think @steppi hit all the right points here! But, to be 100% clear, let me just emphasize that the idea is to write only the actual implementations of the scalar functions themselves in C++. And it's C++ only in the simplest sense, nothing fancy and no complicated features, just typed arguments. Importantly though, they do need to be headers so as to support JITs (like for CuPy). This can then be exposed to other libraries as something like sp.special.get_include(), analogous to np.get_include().
There is no need to touch the Python C API or anything like that. How the special functions become ufuncs is a choice for SciPy, I know you have various ways you already to do that. One could use the NumPy machinery for ufuncs, or still wrap these with Cython, whatever. That's a choice for SciPy and is an independent thing. Other array libraries have their own ways to create ufuncs from C++ source, for instance CuPy's ElementwiseKernel (https://docs.cupy.dev/en/stable/reference/generated/cupy.ElementwiseKernel.html).
The Array API and getting special functions onto all devices is the motivation for us, but I think this is sort of a complementary problem that should be solved independently. I think a lot of people now look to the special functions in SciPy as some sort of reference implementation, and this is also a way to spruce it up and move in that direction.

No objection from my side but just curiosity and nice to learn about it.
The reason I mentioned these are the compiler differences and the Microsoft's (maybe by-now-outdated) notoriety of nonconforming implementations. Windows platform is always a major pain and I'm hoping that we stay away from stuff such as np.complex256/complex128/complex64 vs. long double complex/double complex conversion. If the our C++ folks are OK with it then please ignore my comments :)

I think you've covered everything @izaid, but would just like to mention I share your distrust of the more complicated parts of C++ @ilayn and emphatically support keeping to only a minimal subset of the language.

It makes sense to me to expand upon the building blocks that have existed for quite a long time in Boost.Math as opposed to starting your own work in say C. @mdhaber, @WarrenWeckesser, and @mckib2 have been using Boost.Math in scipy.stats so we have worked out a number of bugs that impact Scipy. You would also not be the first one to use Boost.Math to provide special functions. As of 2-3 years ago the MSVC STL ships our implementations for everything listed here: https://en.cppreference.com/w/cpp/numeric/special_functions.
For the concerns related to non-conformance we test in CI GCC 5-13, Clang 5-13, and MSVC 14.0 - 14.3. I expect you will have very few issues with doing math at the C++14 language standard. @jzmaddock and I can assist as we have been.
To simplify things a little bit wouldn't a fair number of the Boost.Math special functions already be available through the Boost.unfucs machinery used in scipy.stats? Please correct me if I am wrong @mckib2.

@mborland Indeed, I certainly wouldn't want to duplicate tons of effort. A question though: what's the current state of things with regard to CUDA in Boost.Math? Has anyone wanted or tried to turn Boost.Math functions into __device__ functions that can be called in a CUDA kernel? The other thing is that NVCC in JIT mode is very particular and restrictive on include files.
This is potentially the issue using the Boost implementations for what is described here.

@mborland Indeed, I certainly wouldn't want to duplicate tons of effort. A question though: what's the current state of things with regard to CUDA in Boost.Math? Has anyone wanted or tried to turn Boost.Math functions into __device__ functions that can be called in a CUDA kernel? The other thing is that NVCC in JIT mode is very particular and restrictive on include files.
This is potentially the issue using the Boost implementations for what is described here.

We have an ancient open PR to add support: https://github.com/boostorg/math/pull/127. There's been no demand signal for it so that's likely why it wasn't pursued further.

Thanks @mborland! In that case, what might make sense here what @steppi described. We use Boost where we can, but where can't - either for complex arguments or device (CUDA) support - we do it in SciPy with the intention of Boost hopefully adopting it. And eventually Boost may even push it into the C++ standard, as you did with the many of those special functions.

We have an ancient open PR to add support: https://github.com/boostorg/math/pull/127. There's been no demand signal for it so that's likely why it wasn't pursued further.

Right, I experimented with it.  At the time the one CUDA device I had available to me, didn't show any performance gains doing things this way, rather than using (the much faster) CPU serially.  And now my current laptop has no CUDA support at all :(
Most of the changes are relatively trivial: just adding markup to the function definitions so that they are usable on CUDA devices.  BTW it's a right pain that CUDA needs special function markup with non-standard C++ just to compile for the device.
Were there particular functions that you wanted to CUDA-enable?
As for complex special functions, we reasonably often get requests for complex Bessel functions (for example), but I've deliberately avoided them as most of the evaluation methods look rather simplistic to me with no investigation of where they work and where they explode (numerically), plus the complex domain is a huge space to explore... I'm not even sure it's possible to devise comprehensive tests over such a large space?  But for sure folks do need them.

@ilayn Thanks for the helpful comments! The idea is simple C++, that's it. It will certainly look pretty close to C. It could be made to work with C, but a few reasons why we may want C++ are:

C does not support simple overloading on argument types, C++ does. This is relevant for many of the special functions where the implementation differs if an argument is an integer or a float or a complex number (Bessel functions, polynomials, etc).
C++ has now added a lot of very important special functions into the standard library: https://en.cppreference.com/w/cpp/numeric/special_functions. We'd probably prefer to make use of those when / where available?
Built-in complex number support.

Just a small point. C has complex number support too.

@jzmaddock Totally agree, things would be a lot easier if no special markup was needed for CUDA. But, as you said, it actually is quite easy! Something like the following is all we need here.
Put this in a common header file:
#ifdef __CUDACC__
#define SPY_SPECFUN __host__ __device__
#else
#define SPY_SPECFUN
#endif

And for each function going forward define it as:
SPY_SPECFUN RetType my_specfun(ArgType0 arg0, ArgType1 arg1, ...) {
   // implementation
}

Just speaking for what I personally have needed, among other things I've ended up writing implementations of the Bessel functions (both cylindrical and spherical), spherical harmonics, and various orthogonal polynomials for CUDA over the years. For many other functions, I've copied implementations from Boost, the C++ standard library, Slatec, and SciPy where needed, then did something similar to the SPY_SPECFUN above. More widely, you can see that CuPy's special includes a lot of what is in SciPy: https://docs.cupy.dev/en/stable/reference/scipy_special.html. Am sure the other array libraries would like to not reimplement all of that.
Great that you've played around with the Boost PR! Since what we are talking about is elementwise computation, it really should be faster but you might not see that unless you are calling the function on arrays of a sufficient size (1000?). It's just like calling sin(x) on your CUDA array x, but with a much more complicated computation per element.
@lesshaste You are correct, of course. I find C complex numbers difficult to use, but they do exist.

Would of course be awesome if we could delegate to Boost as much as we can!

if we can supply C++ implementations in header files, than other array computing libraries such as CuPy, PyTorch, or Jax could potentially use these implementations

So this would bring us to the realm of ABI compat, which we so far managed to avoid even with cython_{blas,lapack}.pxd  being public. Would be great to hash out implications of that and formulate what exactly SciPy guarantees. Also would be great to have an okay from the build/RM folks, @rgommers, @tylerjereddy @h-vetinari  .

Would of course be awesome if we could delegate to Boost as much as we can!

if we can supply C++ implementations in header files, than other array computing libraries such as CuPy, PyTorch, or Jax could potentially use these implementations

So this would bring us to the realm of ABI compat, which we so far managed to avoid even with cython_{blas,lapack}.pxd being public. Would be great to hash out implications of that and formulate what exactly SciPy guarantees. Also would be great to have an okay from the build/RM folks, @rgommers, @tylerjereddy @h-vetinari .

For reference Boost.Math offers no ABI stability guarantees, and I am not aware of any complaints about that.

the compiled kernels for special functions written in Cython can not be called from compiled kernels written in C or C++.

This statement confuses me. Cython functions are regular C functions, and C++ should easily be able to call those. I don't know if there are more specific bits in the cython documentation, but the following looks relevant -- say_hello_from_python is defined in Cython and then called from an external C program. Perhaps @da-woods or @matusvalo can correct me if I'm saying something wrong.
Pythran can also generate generic C++ code, but I see you ran into some issues with that already.

Just a small point. C has complex number support too.

Except in MSVC. Well, they do actually have all the relevant functions, but they operate on different types (long story). It makes no sense to me, but it hasn't changed for a long time (go upvote this if you have a MSFT account).

In general, I think using C++ code in scipy is perfectly fine. Exporting C++ functions (however simple we set out to keep them) is a whole different ballpark. I'm not saying it's impossible or not worth it, but it opens up a whole new - and complicated - interface we'd have to take care of.

The cython_special extension module is only usable from Cython - we ship the relevant .pxd Cython headers for that. They aren't really usable from C/C++, since we don't install complete headers. We do happen to install _ufuncs_defs.h/_ufuncs_cxx_defs.h more or less by accident, but those contain (a) npy_double & co types rather than C types, and (b) #include statements to other header files that we do not install.

but the following looks relevant -- say_hello_from_python is defined in Cython and then called from an external C program.

That does a PyImport_ImportModule("embedded") type thing that other projects aren't going to want to deal with I'm sure.

In general, I think using C++ code in scipy is perfectly fine.

Agreed - using as in "I'm copying this stuff over".

Exporting C++ functions (however simple we set out to keep them) is a whole different ballpark. I'm not saying it's impossible or not worth it, but it opens up a whole new - and complicated - interface we'd have to take care of.

Yes, that's a separate discussion from whether kernel can be (re)written in C++.

So this would bring us to the realm of ABI compat

There's no ABI involved here - the proposal touched on shipping as headers, not as an installed shared library.

to be 100% clear, let me just emphasize that the idea is to write only the actual implementations of the scalar functions themselves in C++.

This seems perfectly fine to me. And I agree that C++ is a good / the best choice here.

This can then be exposed to other libraries as something like sp.special.get_include(), analogous to np.get_include().

This "expose as headers" part seems like a second or third step. It would require some careful thought about what can be exposed and how we'd deal with API stability guarantees. Re the mechanism, I still have it on my TODO list to implement pkg-config and a numpy-config tool support for dependency detection, because get_include() is problematic. So we'd want to go straight there for SciPy, rather than adding any get_include() function.

For reference Boost.Math offers no ABI stability guarantees, and I am not aware of any complaints about that.

Indeed.  However, the special functions are just that - functions - so their ABI and API (which we do keep stable) are effectively one and the same.
BTW If you wanted to be sure of a stable ABI then wrapping them in a thin extern "C" function and exporting that is trivial, and is of course then language neutral as just about anything will import those.  That's what we do for the TR1 special functions, and likewise what MSVC does when it uses and re-exports our code for their std math special functions.  I wouldn't even contemplate exporting a template instance - that was lies madness! ;)
There is one caveat to the above: there are some functions which are only really super fast if they are expanded inline - an example would be the Elliptic Integral K: I recently went to some lengths to implement a new "faster" method only to discover that the performance improvement was completely swallowed up by the cost of a function call, so then I had to jump through all kinds of hoops to ensure there was nothing in there that would inhibit inline expansion.  And BTW that is one function which apparently benefits from being evaluated on massively parallel hardware for some physics (orbital?) calculations.  The benefit would likely only be seen though, if you can inject the C++ code inline right into the main CUDA procedure.

would be great to have an okay from the build/RM folks

To keep it short, not a fan of C++ (to say the least) but happy to support whatever delivers the greatest value to our ecosystem.

I don‘t know if it’s a good idea to use (part of) scipy as C/C++ lib. If there is a need of several projects for a common C/C++ lib of special functions, isn’t it better to create a new project or extend existing ones like boost?
Another possibility would be to make scipy sf array api compatible. Nanobind might be a possibility, see https://nanobind.readthedocs.io/en/latest/ndarray.html.

Perhaps @da-woods or @matusvalo can correct me if I'm saying something wrong.

I did not read the whole thread hence I don't know fully the background.
Cython allows calling cdef functions from C when declared as api/public:
https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#using-cython-declarations-from-c
When you compile .pyx file containing api/public function, cython will generate also C header file.
But not sure if this is useful for you or not. I think this is the only reasonable way how to call cython functions from C. If this does not fit you, you will need to use C/C++

Just commenting here because I was tagged and offering a few observations (no real conclusions):
There were a few comments about complex and MSVC - I think they're mostly related to C (where it does something very non-standard). I think for C++ it follows the standard. Recent MSVC has pretty good conformance with the C++ standard in my experience.

Since the special function infrastructure already handles generating the ufunc boilerplate on the NumPy C API side, we don't get the primary benefit of using Cython in special. Where Cython really shines is in writing extensions without needing to work out the somewhat tedious C API boilerplate.

Cython can generate ufunc boilerplate for you now (probably not relevant for this discussion though)

C does not support simple overloading on argument types, C++ does

This does make it hard of you want to provide an extern "C" interface (since such an interface doesn't allow overloading). It doesn't sound like that's a big concern right now.

As @matusvalo and @h-vetinari say: you can get Cython to generate a C header file with cdef public. As @rgommers notes - that generates an extension module that you need to import (i.e. it isn't just a stand-alone C/C++ library). That probably makes it less useful for people that want to use them from C or C++.

Lack of interoperability affects us internally too. Because of how the build system works currently for special functions, the compiled kernels for special functions written in Cython can not be called from compiled kernels written in C or C++. This is blocking #19287 at the moment, cc @rlucas7.

Thanks @steppi , I don't have anything to add that isn't repetitive of what's already said, so I'll just say +1.

Thanks everyone for your comments. It seems like there's no real objections to @izaid and I beginning to work on rewriting special function scalar kernels that are currently in Cython into C++.
I'll try to summarize some of the discussion above iand respond to anything I think hasn't been addressed.
There were some concerns about the complexity of C++, but fortunately we need only a small subset of the language. As @izaid said, we just want to be able to leverage the rich ecosystem for mathematical functions that already exists in C++, to benefit from the possibility overloading on argument types, and to benefit from built in complex number support which is conformant across platforms.
I think there was also some concern about scarcity of maintainers with C++ knowledge, but since the Cython were talking about touches neither the Python or NumPy C API, the use of Cython here doesn't actually cut down on any complexity. The only differences from C are basically in syntax, and the C++ we will be using is basically just C with some templates and calls to Boost primitives.
This RFC issue only regards rewriting the scalar kernels in C++, but there was a lot of good discussion regarding sharing the resulting headers among different projects. This is something that would be planned for further down the line; with rewriting these scalar kernels as only an initial step. For know, I see this as creating reference versions of these functions which could in principle be used in the other projects, even if just by copy and pasting.
As @rgommers said, the idea is only to ship headers, not to have a shared library, so we don't need to worry about ABI compatibility. @jzmaddock had suggested wrapping everything in a thin extern "C" function as a means to ensure ABI compatibility, but noted that this would interfere with the ability to inline the functions. For performance reasons we'd like to be able to inline everything, which shipping only headers should allow.
@rgommers pointed out that exposing headers through get_include() is problematic, but he has plans to work on a better mechanism to do this.

I don‘t know if it’s a good idea to use (part of) scipy as C/C++ lib. If there is a need of several projects for a common C/C++ lib of special functions, isn’t it better to create a new project or extend existing ones like boost?

@lorentzenchr, I think you have good point here.  When we get to that stage, I don't think it matters where exactly the headers are shipped from, just that they are available to all of the relevant projects. Eventually I would like to see these things make their way into Boost, and I plan to write them using Boost primitives as if they were going into Boost. Much of special is in a bad state though, and I think there is a lot of room for improvement before things get in good enough shape to make it to Boost. Trying to get things into Boost in one go seems like a more difficult path.

Another possibility would be to make scipy sf array api compatible. Nanobind might be a possibility, see https://nanobind.readthedocs.io/en/latest/ndarray.html.

In a way, we're already working on array API compatibility for special, https://github.com/scipy/scipy/pull/19023#, basically by just dispatching to the implementation in the correct array compute library based upon the input array type. @lorentzenchr, are you saying that instead of shipping headers which can be used by other libraries, we could maintain implementations of the functions that end up in the special array API extension for Jax, PyTorch etc... within SciPy and dispatch to those from scipy.special? It seems like it would be taking on more work; I'd hope to ship some of the maintenance burden to maintainers of the other array compute libraries.

Cython can generate ufunc boilerplate for you now (probably not relevant for this discussion though)

@da-woods, I've been keeping an eye on this new Cython 3 feature, and have considered replacing our infrastructure to do this once Cython 3.x becomes the minimal supported version SciPy.

As for complex special functions, we reasonably often get requests for complex Bessel functions (for example), but I've deliberately avoided them as most of the evaluation methods look rather simplistic to me with no investigation of where they work and where they explode (numerically), plus the complex domain is a huge space to explore... I'm not even sure it's possible to devise comprehensive tests over such a large space? But for sure folks do need them.

@jzmaddock, yeah, he ability to test comprehensively drops dramatically as the number of parameters in a special function increases. I hope to devote effort to comprehensively testing special functions of a complex variable. Very roughly speaking, the plan is to use known structure to cut down on the number of test cases by eliminating some which are essentially the same as others. As an example, here's the test suite I've worked out for the Gaussian hypergeometric function hyp2f1, though this is still a little rough. I hope to write technical reports with details of the testing strategies for each function, along with the results.

It looks like this discussion has ran its course. I'll leave this open a couple more days in case anyone else wants to chime in.

@steppi A C++ header only lib, starting off inside scipy, with long term vision inside boost seems like a good idea to me.
The inline argument is a really good one. Once, I wanted to use xlogy from scipy in a performance critical Cython loop in scikit-learn and ended up with my own copy of that function because it couldn’t be inlined when cimported.

are you saying that instead of shipping headers which can be used by other libraries, we could maintain implementations of the functions that end up in the special array API extension for Jax, PyTorch etc... (at least those they don't already have) within SciPy and dispatch to those from scipy.special?

What I meant is that with nanobind, AFAIU, you can call my_special_fun(x) on numpy, jax and pytorch arrays x. The function understands array data from all those frameworks. So there would be no need anymore to have special functions (reimplemented) in all different array frameworks.
This was just a suggestion and food for thought what’s possible today.

I see @lorentzenchr. I'm not familiar with nanobind, it seems like its meant as a replacement for pybind11 by the same author?  Can you elaborate on how it could help. We're dealing with scalar functions written in C or C++, and different libraries have their own means of turning these into ufuncs which can operate on arrays.
In NumPy, we use 'PyUFunc_FromFuncAndData` from the C API as described [here]. SciPy has its own infrastructure for generating the right C API boilerplate.
CuPy has its own API and its own infrastructure. Here's an example PR, https://github.com/cupy/cupy/pull/7520/files, adding spherical_yn.  Here's an example of what goes into adding a special function to PyTorch, https://github.com/pytorch/pytorch/pull/78900. What we would be supplying is the scalar kernel that goes into the file Math.h here. I'm not familiar with how things are done in Jax.
Looking at those CuPy and PyTorch PRs, what we'd be supplying is the scalar kernel, and then the package developers would add the boilerplate as needed. Are you saying that we could use nanobind to create a streamlined common replacement for the ufunc creation APIs of the different packages?

I'm going to go ahead and close this but will be creating new issues for discussion of more specific aspects related to this effort. Feel free to create a new more specific issue if you have more to say that doesn't fall under a relevant open issue.

authors: da-woods ev-br h-vetinari ilayn izaid jzmaddock lesshaste lorentzenchr matusvalo mborland rgommers rlucas7 steppi tylerjereddy

repo: scipy/scipy


text:
BUG: stats.skewnorm.ppf returns wrong values with moderately high skew parameters

Describe your issue.
I'm trying to compute the inverse CDF (i.e. PPF) of several skewnorm distributions. I am getting nonsense answers when the skew parameter is moderately large.
Reproducing Code Example
from scipy.stats import halfnorm, skewnorm

#  This is the correct ppf for a standard skewnorm with skew 500
q = 0.012533469508013

#  As confirmation of the above result, the skewnorm with skew 500 is
#  extremely close to a halfnorm distribution.
#  In fact, the ppf of the halfnorm agrees with q to 13 decimal places.
halfnorm.ppf(0.01, 0, 1)
#  0.012533469508069276

#  This should theoretically be 0.01, and it is very close.
skewnorm.cdf(q, 500, 0, 1)
#  0.009999999999999898

#  This should theoretically be q, but it's not at all.
#  This is the problem.
skewnorm.ppf(0.01, 500, 0, 1)
#  4.833181656374989e+142

Error message
/opt/conda/lib/python3.11/site-packages/scipy/stats/_continuous_distns.py:9141: RuntimeWarning: overflow encountered in _skewnorm_ppf
  return _boost._skewnorm_ppf(x, 0, 1, a)

SciPy/NumPy/Python version and system information
1.12.0 1.26.3 sys.version_info(major=3, minor=11, micro=7, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /opt/conda/include
    lib directory: /opt/conda/lib
    name: blas
    openblas configuration: unknown
    pc file directory: /opt/conda/lib/pkgconfig
    version: 3.9.0
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /opt/conda/include
    lib directory: /opt/conda/lib
    name: lapack
    openblas configuration: unknown
    pc file directory: /opt/conda/lib/pkgconfig
    version: 3.9.0
  pybind11:
    detection method: pkgconfig
    include directory: /opt/conda/include
    name: pybind11
    version: 2.11.1
Compilers:
  c:
    args: -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -march=nocona, -mtune=haswell,
      -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections,
      -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include, -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem,
      /opt/conda/include
    commands: /home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/_build_env/bin/x86_64-conda-linux-gnu-cc
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,--allow-shlib-undefined, -Wl,-rpath,/opt/conda/lib,
      -Wl,-rpath-link,/opt/conda/lib, -L/opt/conda/lib, -Wl,-O2, -Wl,--sort-common,
      -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now, -Wl,--disable-new-dtags, -Wl,--gc-sections,
      -Wl,--allow-shlib-undefined, -Wl,-rpath,/opt/conda/lib, -Wl,-rpath-link,/opt/conda/lib,
      -L/opt/conda/lib, -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -march=nocona, -mtune=haswell,
      -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections,
      -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include, -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem,
      /opt/conda/include
    name: gcc
    version: 12.3.0
  c++:
    args: -fvisibility-inlines-hidden, -fmessage-length=0, -march=nocona, -mtune=haswell,
      -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections,
      -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -fvisibility-inlines-hidden,
      -fmessage-length=0, -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC,
      -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections, -pipe, -isystem,
      /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include, -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem,
      /opt/conda/include
    commands: /home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/_build_env/bin/x86_64-conda-linux-gnu-c++
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,--allow-shlib-undefined, -Wl,-rpath,/opt/conda/lib,
      -Wl,-rpath-link,/opt/conda/lib, -L/opt/conda/lib, -Wl,-O2, -Wl,--sort-common,
      -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now, -Wl,--disable-new-dtags, -Wl,--gc-sections,
      -Wl,--allow-shlib-undefined, -Wl,-rpath,/opt/conda/lib, -Wl,-rpath-link,/opt/conda/lib,
      -L/opt/conda/lib, -fvisibility-inlines-hidden, -fmessage-length=0, -march=nocona,
      -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt,
      -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -fvisibility-inlines-hidden,
      -fmessage-length=0, -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC,
      -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections, -pipe, -isystem,
      /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include, -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem,
      /opt/conda/include
    name: gcc
    version: 12.3.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.7
  fortran:
    args: -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
    commands: /home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/_build_env/bin/x86_64-conda-linux-gnu-gfortran
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,--allow-shlib-undefined, -Wl,-rpath,/opt/conda/lib,
      -Wl,-rpath-link,/opt/conda/lib, -L/opt/conda/lib, -Wl,-O2, -Wl,--sort-common,
      -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now, -Wl,--disable-new-dtags, -Wl,--gc-sections,
      -Wl,--allow-shlib-undefined, -Wl,-rpath,/opt/conda/lib, -Wl,-rpath-link,/opt/conda/lib,
      -L/opt/conda/lib, -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1705882420314/work=/usr/local/src/conda/scipy-split-1.12.0,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
    name: gcc
    version: 12.3.0
  pythran:
    include directory: ../../_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.11/site-packages/pythran
    version: 0.15.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /opt/conda/bin/python
  version: '3.11'

Additional information:
I have isolated the error to Boost. I found a reported regression which has been fixed in https://github.com/boostorg/math/pull/1080. Thus I suspect that this can be solved by updating Boost.
from scipy.stats._boost.skewnorm_ufunc import _skewnorm_ppf

_skewnorm_ppf(0.01, 0, 1, 500)
#  4.833181656374989e+142

x-ref gh-19348 - looks like we want to try updating to Boost 1.85 once it's out

authors: lucascolley maresb


text:
ENH: stats.wrap_rv_continuous: add wrapper to convert rv_continuous to ContinuousDistribution

What does this implement/fix?
This adds a function that accepts an instance of stats.rv_continuous and returns a subclass of stats.ContinuousDistribution. The ContinuousDistribution uses overridden methods of the rv_continuous class where possible (e.g. pdf, logpdf) but uses its own generic method implementations (which are often faster and more accurate).
As an example inspired by gh-18799:
from scipy import stats
GH = stats.wrap_rv_continuous(stats.gausshyper, support=(0, 1))
X = stats.gausshyper(a=1.5, b=2.5, c=2, z=0)
Y = GH(a=1.5, b=2.5, c=2, z=0)

X is a "frozen" distribution object representing the Gauss hypergeometric distribution and Y is a corresponding instance of a ContinuousDistribution. Since gausshyper_gen defines _pdf, the pdf methods of the two objects behave identically:
X.pdf(0.3), Y.pdf(0.3)
# (np.float64(1.6337206456367985), np.float64(1.6337206456367985))

but because gausshyper_gen does not define _cdf, Y will use the ContinuousDistribution framework to evaluate the CDF, which tends to be much faster and more accurate than the generic implementation in the rv_continuous framework.

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

import numpy as np
from scipy import stats
from scipy import special as sc
from time import perf_counter
from mpmath import mp
mp.dps = 50

a, b, c, z = 1.5, 2.5, 2, 0

def _pdf_mpmath(x):
    a, b, c, z = 1.5, 2.5, 2, 0
    a, b, c, z = mp.mpf(a), mp.mpf(b), mp.mpf(c), mp.mpf(z)
    Cinv = mp.gamma(a) * mp.gamma(b) / mp.gamma(a + b) * mp.hyp2f1(c, a, a + b, -z)
    return mp.one / Cinv * x ** (a - mp.one) * (mp.one - x) ** (b - mp.one) / (mp.one + z * x) ** c

X = stats.gausshyper(a=a, b=b, c=c, z=z)
GH = stats.wrap_rv_continuous(stats.gausshyper, (0, 1))
Y = GH(a=a, b=b, c=c, z=z)

x = np.linspace(0, 1, 1000)[1:]
tic = perf_counter()
cdf_vals = X.cdf(x)
toc = perf_counter()
t1 = toc-tic
print(f"Time for 1000 CDF value (old): {t1}")

tic = perf_counter()
cdf_vals2 = Y.cdf(x)
toc = perf_counter()
t2 = toc-tic
print(f"Time for 1000 CDF value (new): {t2}")

cdf_vals3 = np.asarray([mp.quad(_pdf_mpmath, (mp.mpf(0), mp.mpf(xi))) for xi in x], dtype=np.float64)

rerr = abs((cdf_vals - cdf_vals3)/(cdf_vals3))
rerr2 = abs((np.asarray(cdf_vals2) - cdf_vals3)/(cdf_vals3))
rerr = rerr[rerr != 0]
rerr2 = rerr2[rerr2 != 0]

print(f"Speedup: {t1/t2}")
print(f"gmean of relative error (old): {stats.gmean(rerr)}")
print(f"gmean of relative error (new): {stats.gmean(rerr2)}")

print(f"max relative error (old): {np.max(rerr)}")
print(f"max relative error (new): {np.max(rerr2)}")

Time for 1000 CDF value (main): 0.737669125024695
Time for 1000 CDF value (main): 0.006104291998781264
Speedup: 120.84433791371256
gmean of relative error (old): 8.034300408110537e-16
gmean of relative error (new): 4.553789318107634e-16
max relative error (old): 1.1235272457145405e-07
max relative error (new): 1.445515587446478e-13

And we get all the bells and whistles of ContinuousDistribution:
(Y + 1).mode()  # np.float64(1.2500000068559416)
Y.plot()

Additional information
Questions:

What should this be called? (Maybe see https://github.com/scipy/scipy/pull/21707#issuecomment-2412636465.)
Should it be public? I think so, because I don't think we want to create public versions of all old distributions with the new interface.

Thoughts:

Not all distributions will "just work" as written because some call other private methods of rv_continuous I think I can work around this.
Currently, the user has to pass the support of the distribution as the second argument. (If they don't, they don't get a truncated distribution... just something that is wrong sometimes.) We could eliminate this need by defining it for all instances of rv_continuous. (The domain of the parameters is given by _shape_info.)
We could offer a public interface like this for creating custom distributions so users don't need to delve into the private methods of ContinuousDistribution to create their own distributions.

Perhaps I should change this to make ContinuousDistribution public and expose the functionality via that name. The constructor (__new__) could accept an instance of rv_continuous and return a subclass of ContinuousDistribution (preferred, I think), or (if that is "wrong") the initializer could probably accept a frozen distribution and return an instance of ContinuousDistribution.

I like the idea of making ContinuousDistribution public and exposing the functionality of this PR through it. Maybe something like ContinuousDistribution.from_rv_generic(stats.gamma) to return a (dynamic) subclass of ContinuousDistribution, as the PR does. This would be mirrored by DiscreteDistribution.from_rv_discrete, for example.
The benefit of taking an (old) distribution object rather than a frozen distribution object is that you could easily “port” the old distributions into the new framework, like you demonstrated with Gamma = wrap_rv_continuous(gamma).

Yeah, I think I will give it a from_ method. That is what we did with Covariance and somehow I didn't learn from it. I suppose these methods might be best in a superclass (e.g. StatisticalDistribution.from_rv_continuous), but then they would be inherited anyway, so I suppose it would be OK to add at the subclass level initially. I'll go ahead and do that.

Btw, I have the impression there could be some issues related to object lifecycle etc. when a locally defined class (or anything that takes a closure) is returned. But I can’t recall the details or construct an example to demonstrate any issue.
Would it be possible, if not too clumsy, to define CustomDistribution as a regular (module level) class? That might simplify the logic somewhat, and might move closer to the idea of “offering a public interface like this for creating custom distributions so users don't need to delve into the private methods of ContinuousDistribution to create their own distributions.”

There is already a lifecycle issue with the old distributions: gh-13986. I thought about serialization issues, too, but there are workarounds if that is a problem for someone.
There are certainly ways of doing something similar without returning a class, but there seem to be tradeoffs in terms of simplicity and/or functionality. Let's wait for a second maintainer to get involved.

This is going to difficult to type 😅

It looks like the linter added a new rule which got picked up after merging main. After the offending line is fixed, I think this is ready to merge.

It is UP31, for which the detection in ruff isn't perfect but seems to improve frequently with releases. Probably makes sense to go through the process of disabling it (or ignoring it) until we deal with them all in one go, like https://github.com/scipy/scipy/pull/21029.

authors: fancidev jorenham lucascolley mdhaber


text:
CI: On Travis matrix "OPTIMIZE=-OO" flag ignored

In order to address #9373, we need a working (in this case failing) build however Travis doesn't seem to enable the -OO flag. This PR is for testing different options in the yaml file.

Ooops this is creating a scipy branch.


text:
Error return improvements

Relates #690, #525, possibly others. Raised via support conversation with @cgc.

Every error response needs to be constrained to the same struct, with optional fields. For example, as per #690 the validation error could be a field that is left blank for non-validation errors.

Contain the request ID as per #525.

Megan & I have been talking about error codes for awhile - in addition to / supplanting the HTTP status code, we need permanent integers that describe the problem. My guess is these could be associated with various Exception types under the hood, and be shipped next to human-meaningful messages for the category, as well as the current instance-specific message.

More?

With the improvements to our Exceptions, such as the addition of our own internal codes, any thoughts on moving away from the self.abort and calling the webapp2 Exceptions to our own?

I don't have an opinion at the moment, but in general the less ties to webapp2 we have, the better. It's hopefully not long for this world.

repo: scitran/core


text:
Optional arg to force upgrade from previous version

And a quick fix for upgrade 30.
Review Checklist

Tests were added to cover all code changes
Documentation was added / updated
Code and tests follow standards in CONTRIBUTING.md

Logic looks good, the table I tested the logic with (by reading, not through running the updates) was:

ok
ok
error
error
error

current db_version
29
30
30
30
31

database.py version
30
30
30
30
30

force_from
28
29
30
31
30


text:
Downloads

Based on the initial development of GBIF-download by @jlegind, this pull request provides the ability to post download requests to GBIF. This provides an answer to issue #16:

GbifDownload class to construct and combine a number of predicates and post this as a request to GBIF, getting back the identifier as provided by Gbif
the current inability of using in with different taxonkeys, is solved by the provision of a add_iterative_predicate function
Adaptation of the download function that was already available  (previously as comments). The statements are processed and the predicate handling and posting of the requests relies on the GbifDownload class
(the iterative_predicate is not yet supported by the download function)
tests are added for the GbifDownload class and the download functions in a separate file
documentation is adapted, incorporating the download function

Coverage increased (+7.6%) to 71.129% when pulling ddd6301686178ac6d8aaa5d6fd6b1e1e1f98f86e on stijnvanhoey:downloads into bfb9ca115e819243e41ce56d4acea36f028079a3 on sckott:master.

I'll do further testing with python 2.7 and update the pull request...

Coverage increased (+7.7%) to 71.279% when pulling eb2bd06aae10bc52cb36bbbcf7a51bac95bffe79 on stijnvanhoey:downloads into bfb9ca115e819243e41ce56d4acea36f028079a3 on sckott:master.

Thanks @jlegind @stijnvanhoey @peterdesmet !
Looks good, tested locally.
merging,

authors: coveralls sckott stijnvanhoey

repo: sckott/pygbif


text:
Check Error updates

Last PR for a bit on this. Adding the capability for exceptions to returned errorcodes coming from MOAB. I'm hoping there is some discussion behind this one. I'd particularly like some input from @vijaysm. I'd like to make sure the user knows that the option for adding error exceptions is available as MOAB is not designed s.t. MB_SUCCESS is a required return behavior for the ErrorCode.
Maybe a part of this PR should be documentation for the existing functions so users are easily aware of this capability?

I am waiting for @vijaysm

Patrick, what is the motivation behind an exception options ? Is this to allow known failure behaviors that can be caught and processed, without erroring out ? As you said, MB_SUCCESS is not a required return of many routines and not every error code is fatal either. So the error behavior is context specific and I can see exceptions being handy in that case. Was this your intent ?
Documentation with some examples will be helpful. A simple use case where exception option would be useful is when say you are querying for a tag and get TAG_NOT_FOUND error. You could die or create a new tag with the same name. This is context specific since for visualization, this is fatal but for simulation, its part of the workflow.
Or did I completely misunderstand the idea here ?

@vijaysm
p1- Yes. That is exactly my intent and you're spot on.
p2 - I agree. I should add a couple of examples (maybe even within the yt2moab example querying for a tag value I know isn't there.)

p2 - I agree. I should add a couple of examples (maybe even within the yt2moab example querying for a tag value I know isn't there.)

Yes, this can be one of the cases. We can probably also create some unit tests that provide use cases where exception will be useful.
On a separate note, we should talk about adding PyMOAB to the buildbot suite at ANL. Perhaps after thanksgiving ?!

That's probably a better idea. And yes. Unit tests are my next move here. Fortunately, this should be pretty straight forward with what we have available in the interface right now.

@vijaysm @scopatz With unit tests incoming (started on some simple ones today), are we ok with what's in this PR?

Looks good to me.

@scopatz good to go here?

Sorry about the delay @Pshriwise

Just the one small issue!

@scopatz on it! Thank you!

Cool, looks great to me.  Is this ready to merge?

Its all set as far as I'm concerned. Unit tests on the way!

Woohoo Thanks!

authors: Pshriwise scopatz vijaysm

repo: scopatz/pymoab


text:
What is the data-structure to be considered for identified data points?

As explained in the architecture, RDD can used to pass data-points between components, RDD is kind of immutable data-sets that are generated from any type of data objects. But do we have to identify a generic data structure to represent objects, or do we maintain desperate objects per data-source. That should be resilient for future plug-in data sources as well. come up with such

First, we thought to go with index/type/document Elasticsearch data-schema, but I have doubt whether we actually need that ES layer, since the burden of keeping an universal data-structure is difficult at document layer. We can use RDD in-common and do all operations on top of RDD, Succinct does seem a good candidate on providing search optimized storage on compressed RDDs, also GraphX can be operated. But we need to know the fields we're going to capture at RDD, so graph construction can perform easily across RDDs.

repo: scorelab/OpenADS


text:
Proposing seperate markdown documentation for components

This is a feature request
It would be good to have documentation for the project currently, both as a whole and component-wise as many new contributors face a little difficulty in keeping pace with how different components are rendered and how can we customize them using props. Therefore, I propose to have separate markdown documentation for each component which would make it really easy for contributors to see follow along with successful imports and use of the components, also altering the props.
Having well-structured documentation support for components will support more contributions, even from the newer contributors :)

I would request @charithccmc and @wdevon99 to give their insights regarding this request :)

Doing as we create and modify components

repo: scorelab/Webiu


text:
No such file or directory: 'requirement.txt'

Expected Behavior
In Readme in the 'setup flask server', there is a sub-part 'install your requirements'.
Current Behavior
There is no requirement.txt present in that directory

During setup for this project I am getting this error. Please guide me how should I approach to the error.

You can find requirement.txt in fact-bounty-flask folder. Make sure you are running pip install -r requirements.txt command in above mentioned folder.

Thank you. It worked.

repo: scorelab/fact-Bounty


text:
Webdriver

Selenium 2 (webdriver) is the latest automation tool and the product of merging Selenium Remote Control (RC) and Google's Webdriving project in 2009

Good detail.


text:
LED not working

I managed to get the visualization.py to work, but the LED's don't seem to light up when I run the program. The power supply is on, the wires are connected, and IP's match. My only guess is that the hardware may be messed up. Does anyone have any other ideas on how I can test my LEDs? I'm running this on windows 10, esp8266.
Here's the links to what I am using.
esp8266:
https://www.amazon.com/gp/product/B071YPS666/ref=ppx_yo_dt_b_asin_title_o09_s00?ie=UTF8&psc=1
LED:
https://www.amazon.com/gp/product/B01LSF4Q0A/ref=ppx_yo_dt_b_asin_title_o09_s00?ie=UTF8&psc=1

Make sure PRINT_FPS is defined as 1 and check the output, are you receiving any fps?

DISPLAY_FPS in config.py is set to true, and I am receiving an FPS. I'm running this through IDLE, I figured I'd mention it though I don't think it is the complication.

+1 for same problem here, tested with pi only and works well like that, but failed with nodemcu or wemos d1 mini + esp8266 config settings doesn't seem to work, i also am recieving an fps in the serial monitor.

I removed Hash.h, I meant to remove that a while ago. I also removed the mention to websocketsserver as it doesn't exist anymore and only serves to confuse people. Neither of those are required.

@joeybab3 Thanks man, Appreciate the super quick update. I'll test it out on asap and report back :)

@joeybab3 Fixed my issues- now working really well!
thanks again

Perfect, thanks for notifying me of that.

Woops, I didn't realize that this was someone else's issue. @MackRhine does the current code work for you any better with the imports removed?

If you're receiving frames than the issue is most likely on the Arduino/hardware side.
Are you able to run strand test or one of the adafruit ws2812 examples successfully?

authors: MackRhine SealedJoy joeybab3

repo: scottlawsonbc/audio-reactive-led-strip


text:
Support dict items as spider output

The current version ignores simple dict items produced by the spiders. This PR adds support for dict objects in order to handle them like standard Scrapy items.

Thanks @jesuslosada! :+1:

authors: jesuslosada vshlapakov

repo: scrapy-plugins/scrapy-pagestorage


text:
🛑 Material Snake is down

In aec3d1c, Material Snake (https://snake.atanas.info) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Material Snake is back up in 32235b7.

repo: scriptex/uptime


text:
🛑 SCSS Goodies is down

In c362261, SCSS Goodies (https://scss-goodies.atanas.info) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SCSS Goodies is back up in 0967930.


text:
Convert Author Names to Links

We have saved author websites/portfolios in our scriptified-cms. We will need to add those links in author names from our API. Easy!
Tasks

[ ] Add tip source in Tip Of The Week
[ ] Add hyperlinks in author name in Articles
[ ] Add hyperlinks in author name in Tools
[ ] Add author name in Tech Talks
[ ] Add Quiz source
[ ] Add GIF source

Reference

Hey @gupta-ji6  I can work on this issue :) .

sure @contracamilo, go ahead! I've assigned it to you :)
mention @scriptified/core-team if you need help with understanding the requirements or anything else 🙌🏻


text:
FulIncomingSlipLetter: Don't include "Available items" for article copies

In these cases, we get a list of all the items/issues associated with the relevant holdings.. which can be several pages.
Not sure how to best filter these, perhaps on request type PHYSICAL_NON_RETURNABLE or DIGITAL...

Added a filter on material_type != 'Article' in https://github.com/scriptotek/alma-slipsomat/commit/319de859b75bfa5ba6ff1c2ba31bca551da26c84 . Might not catch all cases, but most.

repo: scriptotek/alma-slipsomat


text:
Dropped network connections

Describe the bug
Logs from the Reth node syncing contained a lot of connections being dropped. These dropped connections were for bootnodes and ended with Received disconnect command for session reason=Some(SubprotocolSpecific) remote_peer_id=0x.... This led to a very long completion time for the header stage of the sync pipeline, even needing to be restarted 4 times (first 3 times stopped fetching headers after about 20% completion).

Based on the below screenshot, it looks like a lot of connections are in "Pending Outgoing" and only up to 3 concurrent connections are ever established.

Investigate why connections are being dropped and why outgoing connections are stuck in pending state.
Steps to reproduce
RUST_LOG=debug cargo run --bin scroll-reth --features scroll --manifest-path crates/scroll/bin/Cargo.toml -- node --chain scroll-mainnet
Node logs
No response
Platform(s)
Linux (x86)
What version/commit are you on?
reth-scroll-cli Version: 1.1.2
Commit SHA: bb82d3b10960f953c363c2946b5876340a7b0ad9
Build Timestamp: 2024-12-20T09:20:35.578132888Z
Build Features: scroll
Build Profile: release
What database version are you on?
Current database version: 2
Local database version: 2
Which chain / network are you on?
scroll-mainnet
What type of node are you running?
Archive (default)
What prune config do you use, if any?
Default setting
If you've built Reth from source, provide the full command you used
RUST_LOG=debug cargo run --release --bin scroll-reth --features scroll --manifest-path crates/scroll/bin/Cargo.toml -- node --chain scroll-mainnet --debug.tip 0x60025906c0192f1ad2f3c9f5bbf79c9c03eed3271a258cdb278e8aefd768060f --metrics 0.0.0.0:9001
Code of Conduct

[X] I agree to follow the Code of Conduct

I think you are right, NewBlock and NewBlockHashes are only supported in PoW, and lead to a disconnect.

@frisitano propose to add a function within_pow_or_log (similiar to within_pow_or_disconnect) to simply log the incoming peer message, without disconnecting.

@frisitano propose to add a function within_pow_or_log (similiar to within_pow_or_disconnect) to simply log the incoming peer message, without disconnecting.

Sounds reasonable to me. Alternatively, we can feature gate the disconnect behind scroll and add the option to log if scroll feature is active.

repo: scroll-tech/reth


text:
post mainnet development

routine developments

upgrade deps: ff / group / halo2-libs / snark-verfier / halo2 / revm / ethers-rs / rust (check whether we can use stable)...
https://github.com/scroll-tech/zkevm-circuits/issues/884 Use halo2 circuit-params feature (by using this, we can enable ecc/sig circuits testing inside CircuitTestBuilder more efficiently
re-unify with upstream: at least check whether we should merge the hi-lo refactor
https://github.com/scroll-tech/zkevm-circuits/issues/882 refactor rw: restrict read/write must happen through rw
https://github.com/scroll-tech/zkevm-circuits/issues/840 refactor: use preStateTracer
fast & accurate ccc, especially for poseidon

more https://www.notion.so/scrollzkp/Status-and-Future-of-zkevm-circuits-2023-07-f2643cf83ee64709858d83665e6f9221
docs
big new features

the 3 hash precompile
remove usage of RLC inside shared tables, paving road for future hybrid circuit.
eip4844: how to use this to reduce l1 finialization gas, not support eip4844 inside zkevm

random ideas

is it possible to use beacon chain randao as the fiat-shamir randomness?

bug fixes & soundness issues

decide whether & how should we constrain tx receipts

toward scroll-zkevm 2.0
rewrite of constrain system and whole circuits design

the GitHub actions also need upgrade, the action-rs is no longer maintained.
we should deprecate usage of lazy_static in favour of once_cell

We may also setup cargo deny ci to check our dependency licenses especially when new deps are added.

We may also setup cargo deny ci to check our dependency licenses especially when new deps are added.

great. we need this.

authors: lightsing lispc

repo: scroll-tech/zkevm-circuits


text:
Cancel button on Spotter report form does not work

Cancel button does not even need to appear on the Spotter form unless document.getElementById('appName')=='runlocal' in which case it can be a Javascript back() function.  Add code to set the button's display style to none and Javascript code during page initialization can set it to inline if the proper conditions are met.

Fixed.  @rbarden you will want to view the diff in commit dev/9c42089 and apply the same changes to your NCO form.