text:
Open discussions tab for contact request

Hi,
Please open discussions on this repo, or accept my friend req on discord Kendall#9903
Thanks

??
On Thu, 17 Feb 2022, 12:58 am Tom Schmelzer, @.***>
wrote:

Closed #22 https://github.com/schmelto/schmelto/issues/22.
—
Reply to this email directly, view it on GitHub
https://github.com/schmelto/schmelto/issues/22#event-6085281755, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AUGJUTLFZQMVNVAVR3WNHZ3U3P3FDANCNFSM5OR7QXIQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

@KendallDoesCoding Ive accepted your friend request on discord that's why I closed this issue here. Further I don't want to open discussion on this repo due its only my personal GitHub profile. If you want to reach out to me you could connect with me on LinkedIn or twitter. Further on my portfolio website is an email form.
Regards

Thanks will do!
On Thu, 17 Feb 2022 at 11:55, Tom Schmelzer @.***>
wrote:

@KendallDoesCoding https://github.com/KendallDoesCoding Ive accepted
your friend request on discord that's why I closed this issue here. Further
I don't want to open discussion on this repo due its only my personal
GitHub profile. If you want to reach out to me you could connect with me on
LinkedIn or twitter. Further on my portfolio website is an email form.
Regards
—
Reply to this email directly, view it on GitHub
https://github.com/schmelto/schmelto/issues/22#issuecomment-1042615666,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AUGJUTNMNL5KESYWVIEZSV3U3SIGNANCNFSM5OR7QXIQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

authors: KendallDoesCoding schmelto

repo: schmelto/schmelto


text:
Mark .ward file as read-only by default

.ward file at rest should be read-only. Running ward editing commands should 1) reset read-only flag, 2) do the operation, 3) set read-only flag

We shouldn't do this due to Dropbox (et al) syncing.

source: gharchive/issue

repo: schmich/ward

license: mit

license_type: permissive

license_source: bigquery


text:
After updating to 1.1.0, sensors show Unavailable

After updating to 1.1.0 via Community, Pending updates, then restarting Home Assistant Core (running in Docker on Ubuntu 18.04), my Eero sensors show Unavailable.
This log file entry appears at HA startup:
2021-01-29 06:58:09 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry My_Name for eero
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 236, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/config/custom_components/eero/__init__.py", line 135, in async_setup_entry
    conf_activity = options.get(CONF_ACTIVITY, data[CONF_ACTIVITY])
KeyError: 'activity'

When I revert to 1.0.3, sensors appear again.

Sorry about that, I'll have to push a quick fix for that this weekend.
Another option in the meantime is to delete and re-setup the integration within Home Assistant.

Sorry about that, I'll have to push a quick fix for that this weekend.
Another option in the meantime is to delete and re-setup the integration within Home Assistant.

No worries, thanks for your efforts!

No worries, thanks for your efforts!

@waltlillyman I just pushed an update that should fix the problem. Let me know if you still encounter problems.

@waltlillyman I just pushed an update that should fix the problem. Let me know if you still encounter problems.

Update did not work for me, but delete/reinstall worked fine.

Update did not work for me, but delete/reinstall worked fine.

What errors did you get after updating to 1.1.1?

What errors did you get after updating to 1.1.1?

Sorry, I didn't save the logs.  I had the same problem as @waltlillyman : all of the entities showed as offline.  When I would try to investigate the integration in the UI, it would spin off with a message like "Please wait while installing" but then never got anywhere.
It seemed like it was "crashing" (for lack of a better word) on startup.
I'm new here, so if there is some log I can grab for you I'm willing to.

Sorry, I didn't save the logs.  I had the same problem as @waltlillyman : all of the entities showed as offline.  When I would try to investigate the integration in the UI, it would spin off with a message like "Please wait while installing" but then never got anywhere.
It seemed like it was "crashing" (for lack of a better word) on startup.
I'm new here, so if there is some log I can grab for you I'm willing to.

My upgrade from 1.0.3 to 1.1.1 worked perfectly, thanks, @schmittx !

My upgrade from 1.0.3 to 1.1.1 worked perfectly, thanks, @schmittx !

Good, glad to hear it. I'll close the issue now but please re-open or a submit a new one if you still have problems.

Good, glad to hear it. I'll close the issue now but please re-open or a submit a new one if you still have problems.

authors: DrGomek schmittx waltlillyman

repo: schmittx/home-assistant-eero

license: MIT

license_source: github-api


text:
Implement class mtmp::Process (cpp)

Processes are not Threads: they do not share memory space. A dead process does not kill the whole processing while a blocked thread blocks the whole processing.

Finally, won't do. avt will only use multithreading.

repo: schmouk/ArcheryVideoTraining


text:
joining the legion unable to be completed

the one fort that actually needs bandits active, doesnt have bandits active
fort hraggstad doesn't have bandits enabled and as such can't be completed
no map marker either

Hmmm... I don't think that this is CWO. I made a fix to have remove bandits but not until after the Battle for Whiterun.

i don't have any mods that edit forts, additionally there is no map marker

yeah this may just be a map marker issue cause i found some bandits

update killed all bandits, and nothing

update markers have appeared, outside the fort

update they are citizens of dragons bridge?

update; can't fast travel cause "enemies nearby"

hold up let me retest this this could be a me thing

unable to reproduce, it was a me thing

authors: LABWAZHERE schofida

repo: schofida/CWORR


text:
Enable unlimit scanning and notify function

Enabling unlimitted scanning by setting TimeLimit to a negative value and setting Notify to a function, consuming a Discovered struct, will let the peerdiscovery library run forever. Each discovery, regardless of wether known before or not, will be passed to the Notify function. This implementation is oriented by the observer pattern.
I needed the described change of the library for my use case. This enables scanning for new peers in the background and letting me decide if I want to establish a connection to those hosts.
Please give me a feedback, if this is helpful for you or this library at all?

This is awesome! I've always wanted to do this kind of thing but didn't know how! Thanks!

Thanks for merging this PR and adding an example :-)

source: gharchive/pull-request

authors: geistesk schollz

repo: schollz/peerdiscovery


text:
🛑 findabooktoread.com is down

In f47881e, findabooktoread.com (https://findabooktoread.com/) was down:

HTTP code: 502
Response time: 213 ms

Resolved: findabooktoread.com is back up in e85e569.

repo: schollz/upptime


text:
Use setup-python pip caching

Close #83.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d12dd01) 95.19% compared to head (3711edb) 95.19%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #87   +/-   ##
=======================================
  Coverage   95.19%   95.19%
=======================================
  Files           1        1
  Lines         104      104
=======================================
  Hits           99       99
  Misses          5        5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: codecov-commenter jarrodmillman


text:
ENH get rid of our implementation of power method

It seems it exists in scipy: https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.interpolative.estimate_spectral_norm.html
let's just check that their implementation is faster and robust, but I'd be surprised if it wasn't :)

That could have been great if we were able to use scipy, namely to reduce the code base leverages our dependencies.
I afraid,

it would not be possible as we call spectral_norm inside jitted classes
timing comparisons don't yield surprising results

scipy uses a random estimate which doesn't yield a tight estimate of the spectral norm

authors: Badr-MOUFAD mathurinm

repo: scikit-learn-contrib/skglm


text:
Bug report on BayesSearchCV

Steps to reproduce:
from skopt import BayesSearchCV
from sklearn.datasets import load_digits
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split

X, y = load_digits(10, True)
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.75, test_size=.25, random_state=0)

# log-uniform: understand as search over p = exp(x) by varying x
opt = BayesSearchCV(
    SVC(),
    {
        'C': (1e-6, 1e+6, 'log-uniform'),
        'gamma': (1e-6, 1e+1, 'log-uniform'),
        'degree': (1, 8),  # integer valued parameter
        'kernel': ['linear', 'poly', 'rbf'],  # categorical parameter
    },
    n_iter=32,
    cv=3
)

got this error:
Traceback (most recent call last):
  File "<stdin>", line 11, in <module>
  File "C:\ProgramData\Anaconda3\lib\site-packages\skopt\searchcv.py", line 288, in __init__
    return_train_score=return_train_score)
TypeError: __init__() got an unexpected keyword argument 'fit_params'

Tested on linux and windows. Got same error.
Reason of the bug: scikit-learn 0.21.0
How to solve: pip install scikit-learn==0.20.3

I get the same error.
this is due to the depreciation of self.fit_params to the constructor of BaseSearchCV in scikit-learn 0.21.0 ( https://github.com/scikit-learn/scikit-learn/commit/#13519 ).
A simple way to solve it is simply to remove fit_params = fit_params in super(BayesSearchCV, self).__init__(...) and add self.fit_params = fit_params. That's what I did for now.
Not sure if this is the best way to do (skopt might want to also remove fit_params to be consistent)  with sklearn), but it's a good way of using the latest version of scikit-learn.
@iaroslav-ai might be worth solving this before putting v0.6 on pip.

HEllo @YannDubs , I use colab google and I have the same problem.  But I can't edit  SearchCV file for applying your solution. It is available just for read-only. How Can I make the modification??

@LaymounaLhoulwa , you can import and override class methods by inheritance, like this

thanks a lot @escherba for your reply, It is helpful

any progress on fixing this to make scikit-optimize compatible with the latest scikit-learn?

@darenr I presume we are waiting for this: https://github.com/scikit-optimize/scikit-optimize/pull/772

But you should restrict version of sklearn to scikit-learn==0.20.3 waiting for sklearn fix

Is there an ETA for this issue? the PR seems ready to merge

@darenr I presume we are waiting for this: #772

please affecting us here too

I forked the project, fixed this issue and cherry picked some upstream pull
requests too
https://github.com/darenr/scikit-optimize
I'll continue to pull in any upstream changes that improve this project.
On Tue, Aug 13, 2019 at 6:53 PM Paul Steven Conyngham <
notifications@github.com> wrote:

@darenr https://github.com/darenr I presume we are waiting for this:
#772 https://github.com/scikit-optimize/scikit-optimize/pull/772
please affecting us here too
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scikit-optimize/scikit-optimize/issues/762?email_source=notifications&email_token=ABJ52NIU6TSFXHCXB7BTXJ3QENQR3A5CNFSM4HMJUNJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4HO53I#issuecomment-521072365,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABJ52NOWOED73DLJKRFWPBLQENQR3ANCNFSM4HMJUNJA
.

Hi, I am experiencing a similar problem.

Am working in an Ubuntu environment
scikit-learn==0.20.3
-e git://github.com/YannDubs/scikit-optimize.git@d8a5aefe7d472b7c7b929ef9135d71a263af7717#egg=scikit_optimize

The last two bullet points are based on the discussion above. Thoughts on how to handle this problem?

I'm also experiencing this bug!

I get the same error.
this is due to the depreciation of self.fit_params to the constructor of BaseSearchCV in scikit-learn 0.21.0 ( https://github.com/scikit-learn/scikit-learn/commit/#13519 ).
A simple way to solve it is simply to remove fit_params = fit_params in super(BayesSearchCV, self).__init__(...) and add self.fit_params = fit_params. That's what I did for now, if you're just doing a few tests you can use my fork for now (only changed this problem): pip install -e git://github.com/YannDubs/scikit-optimize.git#egg=scikit-optimize (and then uninstall it once this is solved on the main branch, as mine will not be updated).
Not sure if this is the best way to do (skopt might want to also remove fit_params to be consistent) with sklearn), but it's a good way of using the latest version of scikit-learn.
@iaroslav-ai might be worth solving this before putting v0.6 on pip (+ it's only 2 lines to change).

I get the same error.
this is due to the depreciation of self.fit_params to the constructor of BaseSearchCV in scikit-learn 0.21.0 ( https://github.com/scikit-learn/scikit-learn/commit/#13519 ).
A simple way to solve it is simply to remove fit_params = fit_params in super(BayesSearchCV, self).__init__(...) and add self.fit_params = fit_params. That's what I did for now, if you're just doing a few tests you can use my fork for now (only changed this problem): pip install -e git://github.com/YannDubs/scikit-optimize.git#egg=scikit-optimize (and then uninstall it once this is solved on the main branch, as mine will not be updated).
Not sure if this is the best way to do (skopt might want to also remove fit_params to be consistent) with sklearn), but it's a good way of using the latest version of scikit-learn.
@iaroslav-ai might be worth solving this before putting v0.6 on pip (+ it's only 2 lines to change).

what files should be modified?

So whats the verdict regarding this issue. I see there are some active PR that seem to have fixed the bug. Are there any plans to release a new version or at least merge the fixes on the master branch?

Use https://github.com/darenr/scikit-optimize for a fork incorporating the simple fix to this bug.
conda uninstall scikit-optimize OR pip uninstall scikit-optimize
git clone https://github.com/darenr/scikit-optimize
cd scikit-optimize
pip install .

Use https://github.com/darenr/scikit-optimize for a fork incorporating the simple fix to this bug.
conda uninstall scikit-optimize OR pip uninstall scikit-optimize
git clone https://github.com/darenr/scikit-optimize
cd scikit-optimize
pip install .

I'm already using that fork. I am however curious about why it is taking long to merge the fix to the main repo. Has the author abandoned this project?

Use https://github.com/darenr/scikit-optimize for a fork incorporating the simple fix to this bug.
conda uninstall scikit-optimize OR pip uninstall scikit-optimize
git clone https://github.com/darenr/scikit-optimize
cd scikit-optimize
pip install .

I'm already using that fork. I am however curious about why it is taking long to merge the fix to the main repo. Has the author abandoned this project?

I'm also worried that this project might be abandoned.
@iaroslav-ai, thanks for creating such a useful tool. Is this actively maintained by anyone? If not, would it be possible to give @darenr merge permissions?

Hi! This project is indeed no longer maintained unfortunately. We are looking for people to take over.

For this issue:
pip install git+https://github.com/darenr/scikit-optimize
@glouppe I'm volunteer

@glouppe I'm also happy to volunteer.

Hi! This project is indeed no longer maintained unfortunately. We are looking for people to take over.

Hi! This project is indeed no longer maintained unfortunately. We are looking for people to take over.

I'd be happy to volunteer too.

Would be good to see if we can get Conda forge to point at the fixed version really.

Hello I've tried some of the solutions proposed here but I am still getting the error related to fit_params.
I am using scikit-learn 0.21.0

authors: Gonzalo933 LaymounaLhoulwa PaulConyngham SimonHStats XT-Tahoe YannDubs darenr dmvieira escherba exiledtothetropics glouppe oli5679 sousablde underscrew v-borges zoj613

repo: scikit-optimize/scikit-optimize


text:
Scipp widgets generalization

Update based on comments in https://github.com/scipp/scipp-widgets/pull/1
This updates the design and documentation to be more generic.  See the intro and concepts notebooks in docs for design details.

I'm not sure what i'm doing wrong but i can't seem to be able to use these widgets. I keep getting the same error
TypeError: callable() takes no keyword arguments

I'm not sure what i'm doing wrong but i can't seem to be able to use these widgets. I keep getting the same error
TypeError: callable() takes no keyword arguments

As I think you've already spotted an input name got mangled should be corrected now.

authors: Matthew-Andrew nvaytet

repo: scipp/scipp-widgets


text:
pip3 installation error

My issue is about ...
I have installed scipy using command pip3 install scipy. But when I run a command from scipy import stats, it was giving an error.
Screenshot:

Hi @rpotter12. Make sure you don't have any other Python 3 versions, say 3.7,  installed that pip3 points to.

In general, it's better practice to install packages like this:
python -m pip install scipy

Then you'll be certain you're installing it in the site packages of the python executable you're using.

@ilayn @jakevdp thanks for the information :)

authors: ilayn jakevdp rpotter12

repo: scipy/scipy


text:
Problem with step and TransferFunction

When I run the code:
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import TransferFunction, step, impulse

class MyTF(TransferFunction):

    def _convert_to_lti_if_needed(self, system):
        if type(system) is int or type(system) is float:
            other = tf([system], [1])
        return system

    def __add__(self, other):
        system = self._convert_to_lti_if_needed(other)

        num = np.polyadd(np.polymul(self.num, system.den), \
                         np.polymul(system.num, self.den))
        den = np.polymul(self.den, system.den)
        return tf(num, den)

    def __sub__(self, other):
        system = self._convert_to_lti_if_needed(other)

        num = np.polysub(np.polymul(system.num, self.den), \
                         np.polymul(self.num, system.den))
        den = np.polymul(self.den, system.den)
        return tf(num, den)

    def __mul__(self, other):
        system = self._convert_to_lti_if_needed(other)
        num = np.polymul(self.num, system.num)
        den = np.polymul(self.den, system.den)

        return tf(num, den)

    def __truediv__(self, other):
        system = self._convert_to_lti_if_needed(other)

        num = np.polymul(self.num, system.den)
        den = np.polymul(system.num, self.den)
        return tf(num, den)

G = MyTF([1], [1, 1])
t, y = step(G)

I get the following error:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-151-ecd0b76107c8> in <module>
----> 1 t, y = step(G)

/usr/local/lib/python3.5/dist-packages/scipy/signal/ltisys.py in step(system, X0, T, N)
   2281                              'systems.')
   2282     else:
-> 2283         sys = lti(*system)._as_ss()
   2284     if N is None:
   2285         N = 100

TypeError: type object argument after * must be an iterable, not MyTF

Do you have any idea why this is happening? If I run the step with the following line, it works just fine:
t, y = step((G.num, G.den))

In this case, you are trying to expand system as an iterator but it is not hence fails. So even if that would have passed then what you would get after that TypeError: 'MyTF' object is not iterable. You probably want to check what the subclass is and convert if it is a Transfer function.
In the step case there is no problem because you already provided the correct arguments as a tuple.
If you want this specific syntax regardless, then you need to implement __iter__ and __next__ methods in your class.

authors: colombojrj ilayn


text:
min_only=true djisktra infinite loop with duplicate indices

My issue is about scipy.sparse.csgraph.djisktra with the min_only=True option.
Seems that if you pass duplicate indices into this function you can induce an infinite loop.
Solution would maybe be to raise an exception in this case?
Or is there a way to a handle it? Take the first unique index as the true target and throw out the others so that no paths lead to them?
Reproducing code example:

Sample code to reproduce the problem

Error message:
code never returns..
Scipy/Numpy/Python version information:
1.1.0 1.14.3 sys.version_info(major=3, minor=6, micro=6, releaselevel='final', serial=0)

@fcollman thanks for the report. Can you add a self-contained code example to reproduce the issue you're seeing?

sure working on it now... I think this is likely a bug with code i submitted so was going to follow it up with a PR that at least has a test with a failure, but hopefully a fix too.

@fcollman that would be great, thanks for looking into it

authors: fcollman larsoner rgommers


text:
Remove all usages of numpy.compat

numpy.compat isn't public API and we shouldn't be using it. This should be an easy code cleanup, we no longer need py2-py3 compatibility code.
$ rg "numpy.compat"

scipy/spatial/qhull.pyx
22:from numpy.compat import asbytes

scipy/io/netcdf.py
46:from numpy.compat import asbytes, asstr

scipy/io/mmio.py
18:from numpy.compat import asbytes, asstr

scipy/io/matlab/mio5.py
84:from numpy.compat import asbytes, asstr

scipy/io/matlab/mio5_utils.pyx
34:from numpy.compat import asbytes, asstr

scipy/io/matlab/mio4.py
7:from numpy.compat import asbytes, asstr

scipy/io/matlab/tests/test_mio_funcs.py
8:from numpy.compat import asstr

scipy/io/idl.py
34:from numpy.compat import asstr

scipy/linalg/tests/test_build.py
7:from numpy.compat import asbytes

#13884
cleaned up the code.

@rgommers  can you please suggest where to implement asstr and asbytes functionality after removeing numpy.compat from .py files

scipy/_lib/_util.py would make sense.

@rgommers please help to review https://github.com/scipy/scipy/pull/13889

Please help review #14009


text:
DOC: Remove L from longs

This is a small cleanup before I try to upstream some changes into refguide_check.py.
This is my first issue in SciPy; I've been doing some Python2 cleanup in NumPy and was asked to try and upstream some changes here. I'm excited to help out. If I'm doing things wrong please let me know so I can try and improve. Thanks --Seth

Thanks @sethtroisi! If you'd like your name added to THANKS.txt, please make an entry here.

authors: mdhaber sethtroisi


text:
ENH: Add single-sided p-values to t-tests

Reference issue
Part of work being done for gh-12506.
What does this implement/fix?
This enhancement implements the "alternative" parameter in SciPy's t-tests to allow the selection and return of single-sided p-values.
Additional information
•Implementation closely follows the method found in the brunnermunzel test with modifications made to support arrays. •Selection is done by setting the alternative parameter to one of three strings ("two-sided", "less", or "greater") with the default value being "two-sided" for backwards-compatibility.
•I'm still in the process of writing unit tests. They will be implemented as a method in the "TestStudentTest" class in test_stats.py, if it's the best place.

At the right, click on "Details" to see what went wrong in the continuous integration tests. The first of these (ARM - Test Group 1) doesn't look like it could be caused by this PR. The ARM builds fail intermittently, so we don't really need to be concerned unless it's happening consistently with this PR and not others. Click on the second (Refguide-check, Latest NumPy) and scroll to the bottom. These failures look like they were caused by doctests, which are intolerant of even the most minute changes in output. Just find where those doctests are written and update the numbers; they should go away.

Just a quick note: I haven't given the code a careful review, but I did install this PR and compare it to R's t.test, including the various combinations of the t.test parameters alternative, equal_var and paired, and I got agreement between the R code and the results computed using this PR.

@DominicChm, another formatting change you'll need to make is to ensure that any lines that you add or edit have a length of at most 79 characters.  This is a PEP-8 requirement, and with the recently merged https://github.com/scipy/scipy/pull/12734, this will be checked as part of the CI tests for each pull request.

@DominicChm, another formatting change you'll need to make is to ensure that any lines that you add or edit have a length of at most 79 characters. This is a PEP-8 requirement, and with the recently merged #12734, this will be checked as part of the CI tests for each pull request.

@WarrenWeckesser Got it, I'll get right on making the necessary changes. As far as code style when breaking up longer statements, is the following acceptable? Or is there a more elegant way to force the 79 character limit than splitting lines with a '\'?
...
        raise ValueError("alternative should be 'less', 'greater' or 'two-sided'")
        # becomes:
        raise ValueError("alternative should be "
                         "'less', 'greater' or 'two-sided'")

...
    elif (alternative, root_tail) in {("greater", "less"), ("less", "greater")}:
    # becomes:
    elif (alternative, root_tail) in \
            {("greater", "less"), ("less", "greater")}:

is there a more elegant way to force the 79 character limit

I've been told that it is preferable to use natural groupings where possible:
elif (alternative, root_tail) in {("greater", "less"),
                                  ("less", "greater")}:

Parentheses, square brackets, and curly braces all let you continue on new lines without explicit breaks.

I've been told that it is preferable to use natural groupings where possible:
elif (alternative, root_tail) in {("greater", "less"),
                                  ("less", "greater")}:

Got it, will push a commit soon fixing these.

@WarrenWeckesser Looks like all tests are passing, is this ready to merge?

@DominicChm, I'm taking a closer look at the code this afternoon.  More soon...

@WarrenWeckesser I think the plan was to use _convert_symmetric_p_value in all tests with a symmetric p-value, not just here. You would prefer for the same logic to be written out for each test?

@mdhaber, yeah, it's just a few lines of straightforward code.   It also avoid the loss of precision that occurs in the expression 1 - p when p is very small.

@DominicChm - @WarrenWeckesser and I have discussed this. As I wrote originally:

And maybe someone else will have something better and ask you to change it again : )

So go ahead and change as Warren suggests. Sometimes you end up right where you started, and that's not necessarily a bad thing. It can be tough to identify the best solution without seeing the alternatives. And hey - after copying and pasting this in a few other PRs, we might even decide to revisit this again!

Maybe eventually we'll want a function with a signature:
def statistic_to_p(statistic, distribution, alternative):

That would evaluate distribution's cdf or sf, depending on alternative at the value of statistic.
But yeah, for now, let's just keep it simple.

@DominicChm, the latest changes look good, thanks.
I was running through a bunch of different permutations of inputs, and I realized there is one big problem remaining.  If nan_policy is 'omit' and the input contains nan, the functions pass the arguments off to the mstats versions, and those versions don't handle alternative.  This means an example such as
In [71]: x = np.array([1, 1, 3, 5, 8, 5, 4, -3, np.nan])

In [72]: ttest_1samp(x, 1, nan_policy='omit', alternative='less')
Out[72]: Ttest_1sampResult(statistic=1.694637091472647, pvalue=0.13396903280382275)

gives an incorrect result; the correct result is
In [74]: ttest_1samp(x[:-1], 1, alternative='less')
Out[74]: Ttest_1sampResult(statistic=1.694637091472647, pvalue=0.9330154835980886)

I've worked on removing the use of the mstats functions for handling nan_policy='omit' in other pull requests, so I know from experience it gets a little messy, especially when we also have an axis argument like we do here.  Eventually I think we'll end up with a collection of tools that will make this much easier, but we're not there yet.
@mdhaber (and anyone else): what do you think of temporarily disallowing the combination of nan_policy='omit' and alternative being either 'less' or 'greater' (e.g. raise an exception in that case), with the plan to fix this before 1.6 is released?
We could also work on adding alternative to the mstats functions, but ultimately, calling the mstats versions when nan_policy='omit' still isn't what we want, so it doesn't really solve the immediate problem.

We could also work on adding alternative to the mstats functions, but ultimately, calling the mstats versions when nan_policy='omit' still isn't what we want, so it doesn't really solve the immediate problem.

On the other hand, maybe if we use the new _ttest_finish function in the mstats versions of ttest_*, it will be easy to add alternative to these functions.  Then we could--for now--continue to call the mstats version, and leave that clean up for another time.

P.S. The idea to create _evaluate_distribution_p to handle the alternative p-values more generically looks good, but lets wait for the next PR to add that generalization. (The name should also have symmetric in it somewhere.)

I don't have any experience with mstats so please make that call @WarrenWeckesser.
_evaluate_distribution_p doesn't seem to rely on the distribution being symmetric ,does it? I think it just needs a distribution for which the different alternatives are meaningful.

Hmm, IIUC it seems either way is a little messy. Is this right?
I’d prefer this to throwing an exception given that we’d want the other behavior in before 1.6 release and the release will likely be upon us sooner than we now think.
Does the finishing function work as in on the masked arrays? I haven’t seen that used yet but I might’ve missed a PR or 3, this summer’s been busier than I anticipated.
-L

On Aug 25, 2020, at 9:56 PM, Warren Weckesser notifications@github.com wrote:

We could also work on adding alternative to the mstats functions, but ultimately, calling the mstats versions when nan_policy='omit' still isn't what we want, so it doesn't really solve the immediate problem.
On the other hand, maybe if we use the new ttest_finish function in the mstats versions of ttest*, it will be easy to add alternative to these functions. Then we could--for now--continue to call the mstats version, and leave that clean up for another time.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

@WarrenWeckesser Ah, I completely didn't notice that little check. At the moment my plan was to leave the masked array tests for later because they seem to use unusual methods to generate their P-values ( masked ttest_1samp for example calculates its pvalue using prob = special.betainc(0.5*df, 0.5, df/(df + t*t)) ).  Is this is because the CDF and SF distribution functions don't support masked arrays? It's not clear to me because other functions like the masked skewtest seem to have no problem using the standard (and easily changeable) 2 * distributions.norm.sf(np.abs(...)) method.
@rlucas7 Doing some quick testing seems to indicate that the T sf and cdf throw an invalid value error when passed a NaN inside a masked array, but the current masked implementation seems to expect and ignore these errors given that part of the calculation is wrapped with with np.errstate(divide='ignore', invalid='ignore'):.
To test a bit more, I removed the call to the masked version inside the standard function and adjusted how n is calculated in the normal ttest_1samp to handle masked arrays, finding that it seems to work correctly with a (very simple) test array = [1, 2, float("nan"), 4, 5]
def ttest_1samp(a, popmean, axis=0, nan_policy='propagate',
                alternative="two-sided"):

    a, axis = _chk_asarray(a, axis)

    contains_nan, nan_policy = _contains_nan(a, nan_policy)

    if contains_nan and nan_policy == 'omit':
        a = ma.masked_invalid(a)
        n = a.count(axis) #+
        #return mstats_basic.ttest_1samp(a, popmean, axis)
    else :
        n = a.shape[axis] #+

    df = n - 1

    d = np.mean(a, axis) - popmean
    v = np.var(a, axis, ddof=1)
    denom = np.sqrt(v / n)

    with np.errstate(divide='ignore', invalid='ignore'):
        t = np.divide(d, denom)
    t, prob = _ttest_finish(df, t, alternative)

    return Ttest_1sampResult(t, prob)

There is certainly more nuance to it (proxying the masked version to it fails some tests), but could a possible solution be to simply merge the masked and standard versions of the function?

@DominicChm,   sorry for the long gap since my last comment!  Your suggested modification to ttest_1samp works for the examples that I've tried.  For example, this session is run with your suggested changes:
In [99]: a
Out[99]:
array([[ 8.,  7.,  1.,  0.,  6.,  4.,  7.,  4.],
       [ 7.,  6.,  9.,  0.,  1., nan, nan, nan],
       [ 6.,  3.,  5.,  0.,  6.,  5.,  6., nan]])

In [100]: stat, p = ttest_1samp(a, 5, axis=1, nan_policy='omit', alternative='less')

In [101]: stat
Out[101]:
masked_array(data=[-0.36266239, -0.22866478, -0.67936622],
             mask=False,
       fill_value=1e+20)

In [102]: p
Out[102]: array([0.36377991, 0.41517215, 0.26111822])

In [103]: for b in a:
     ...:     print(ttest_1samp(b[~np.isnan(b)], 5, alternative='less'))
     ...:
Ttest_1sampResult(statistic=-0.36266238893870795, pvalue=0.3637799139221004)
Ttest_1sampResult(statistic=-0.228664780190012, pvalue=0.415172151852777)
Ttest_1sampResult(statistic=-0.6793662204867571, pvalue=0.26111822031871)

It doesn't eliminate the behavior that results in the surprising use of masked arrays for the return values when nan_policy is 'omit', but that is existing behavior that we don't have to change in this PR.   Can you go ahead and implement that, and include some unit tests?  Something like the above example could be used as a test--that is, compare the result for an input that contains nan to the result obtained when the nans are removed before calling ttest_1samp.
Have you tried something similar for ttest_ind and ttest_rel?
Then

@WarrenWeckesser I haven't yet tried it with _rel or _ind. I should also mention that I tried to entirely replace the masked version of this test with a call to my above test implementation and a few of the masked tests failed. I'll see if I can work around it in some way, and try to add this functionality to the other tests.

@DominicChm, all my suggestions were minor changes, so I've committed them.  Once the tests finish, I think we can merge this.

@DominicChm, we'll need the same change for ttest_rel and ttest_ind that you made for ttest_1samp to avoid dropping down to masked arrays when alternative is not two-sided.  For example,
In [63]: xn
Out[63]: array([-10. ,   1. ,   2. ,   2.5,   9. ,   nan])

In [64]: yn
Out[64]: array([-13.,  -4.,   0.,   1.,   3.,  nan])

For comparison, here's the default two-sided ttest_ind result:
In [65]: ttest_ind(xn, yn, nan_policy='omit')
Out[65]: Ttest_indResult(statistic=0.8373780870523444, pvalue=0.4266863609716408)

In this case, the one-sided alternatives are ignored, so the result is incorrect:
In [66]: ttest_ind(xn, yn, nan_policy='omit', alternative='less')
Out[66]: Ttest_indResult(statistic=0.8373780870523444, pvalue=0.4266863609716408)

In [67]: ttest_ind(xn, yn, nan_policy='omit', alternative='greater')
Out[67]: Ttest_indResult(statistic=0.8373780870523444, pvalue=0.4266863609716408)

The same is true for ttest_rel:
In [68]: ttest_rel(xn, yn, nan_policy='omit')
Out[68]: Ttest_relResult(statistic=4.041451884327381, pvalue=0.015585300188062097)

In [69]: ttest_rel(xn, yn, nan_policy='omit', alternative='less')
Out[69]: Ttest_relResult(statistic=4.041451884327381, pvalue=0.015585300188062097)

In [70]: ttest_rel(xn, yn, nan_policy='omit', alternative='greater')
Out[70]: Ttest_relResult(statistic=4.041451884327381, pvalue=0.015585300188062097)

Thanks for the review, sorry I didn't have time to get to it earlier. I've gone ahead and added the same ValueError to the ttests I didn't get to in the first commit.

@DominicChm, thanks, I've run this through a bunch of tests of my own and it looks good.
This is a useful improvement as is; handling the masked/nan-containing inputs for the less & greater alternatives can be done in a follow up PR.

@WarrenWeckesser was this ready?

@WarrenWeckesser would you agree this close gh-10595?

@mdhaber, yes and yes.  Merged.

I added a note to the release notes.

authors: DominicChm WarrenWeckesser mdhaber rlucas7


text:
BLD, BUG: add c99 compiler flag to HiGHS basiclu library

Reference issue

Related to issues:

https://github.com/scipy/scipy/issues/13085
https://github.com/scipy/scipy/issues/13088

What does this implement/fix?

Explicitly adds -std=c99 flag to basiclu library to fix compilation issues encountered on MacOS builds

Additional information

@mdhaber @tylerjereddy

Thanks, @mckib2, I hope this does the trick..


text:
MAINT: Update ascent.dat file hash

Reference issue
xref https://github.com/scipy/dataset-ascent/pull/3
xref https://github.com/scipy/scipy/pull/17606
What does this implement/fix?
This change is required because the source file has been updated in https://github.com/scipy/dataset-ascent/pull/3
I'd expect the CI to fail until https://github.com/scipy/dataset-ascent/pull/3 is merged.
cc @tylerjereddy @rgommers

Ohh sorry, I could have skipped CI for this change.

Let's see if CI on the merged commit is happy again.

authors: AnirudhDagar rgommers


text:
ENH: Cython api for lapack and blas

[x] Add interface for lapack and blas routines.
[x] Add working example in tests
[x] Review signatures for correctness
[x] Add function to get directory of scipy.linalg (similar to numpy.get_include)
[x] Finish wrappers for 'gees' and 'gges' functions.
[x] Add interfaces to the functions added in https://github.com/scipy/scipy/pull/3984
[x] Add this api to the documentation

Okay, this appears to be working fine on all the platforms I have available for testing now. Hopefully the Travis build agrees.
A few things I'd like to clear up before this is merged:

These wrappers do not check that the pointers passed to the routines are aligned, nor do they do any sort of implicit copying. I feel like that should be pretty obvious to anyone using these routines. Should that still be documented anywhere?
There should probably be a short example in the documentation somewhere. Is there any preference as to where? Would the module-level docstring be the best place?
I'm unsure of the difference between ctypedef int dgem(...) and ctypedef void dgemm(...) Is there any functional difference here?

AFAIK all of the subroutines should have a void return type.  I don't have any strong opinion about the docs and it is easy enough to move them around if necessary.

These wrappers do not check that the pointers passed to the routines are aligned, nor do they do any
sort of implicit copying. I feel like that should be pretty obvious to anyone using these routines.

Definitely worth to document explicitly. Module docstring would be good, along with a short example. Definitely worth a mention in the release notes.

Alright, I'll be out of town this week, but I'll take care of the remaining issues when I get back.

I'd perhaps prefer avoiding f2py completely:
https://github.com/pv/scipy-work/compare/pr-4021
.
"get_linalg_dir" doesn't seem to be really needed, because "cimport
scipy.linalg.cython_blas" appears to just work when running Cython when
Scipy is installed.
.
I'd also like to have "cython" in the names of the interface modules,
because they are only useful for Cython. I'm not sure if it's better to
have a separate module with function pointers, or just add some suffix
or prefix to them. From usage point of view, it probably doesn't matter,
and for maintenance POV it's nicer that everything would be in the same
module.
.
It might also make sense to just have only one "cython_blaslapack"
module with all the functions...
.
Note that there was work done by Dag Seljebotn a some years back on
writing nice wrappers in Cython, so that they are compatible with the
f2py generated wrapper functions... That work may be useful if you
intend to write Cython-friendly wrapper functions for all of the functions.

@pv That looks great. The version without f2py is much more direct. I hadn't figured out how to use macros like that in Cython. FWIW, I prefer having the two as seperate modules cython_blas and cython_lapack. These changes look great to me. I do have a few questions though.
I'm not finding the header "fortran_defs.h". Did you mean to add it in git somewhere? I haven't been able to use these wrappers without including the scipy.linalg source directory, if I can figure out what is going on with the header though, I'd be happy to test it.
If we're going to use a macro to make external declarations at the C level that properly handle the Fortran name mangling, why not avoid using function pointers entirely? If I understand correctly, that should make it easier for the C compiler to inline them.
Also, how would you like me to work with the git history in including these changes?

I forgot fortran_defs.h. It contains the boilerplate stuff from f2py,
and is the same file as scipy/spatial/qhull_blas.h but without the
three #define lines at the bottom.
.
We want to have the Cython module export the pointers to other Cython
modules, via the Cython-specific __pyx_api__ export magic, so that 3rd
party Cython modules do not need to link directly against BLAS/LAPACK.
This seems to require that we assign the external symbols explicitly to
function pointers --- tried to do it without, but it doesn't seem to
work otherwise.
.
To get my changes, do
git remote add pv https://github.com/pv/scipy-work.git
git fetch pv
git checkout cyblas
git merge --ff-only pv/pr-4021

Please also feel free to squash my commits into a single commit, my
commit messages seem to be crap there.

@pv Thanks! I'll work on it.

Ok, I guess this is nearing completion? Thanks a lot for the hard work!
I have one remaining concern --- I would like to veto the "friendly wrappers" for now, as it's not clear to me at the moment that the design is right. I think these would be best removed from this PR, and added in a separate PR. The concerns:

The BLAS wrappers are not complete --- I don't really like the situation we have with the f2py wrappers where some things are missing, and new functions were added later on with little regard for consistency. It would be best to add everything in one go so that we don't end up releasing Scipy versions with only a partial API coverage.
dgemm wrapper requiring GIL seems undesirable
There should be a plan on what to do with C vs. Fortran order, and whether automatic copies are to be done internally or not

So the suggestion: prefix the functions with _ since they are used for testing, and remove them from the documentation. The function pointers added in this PR are already very useful by themselves, so I'd like to not delay adding them.

@pv Thanks, yes, this is getting close to done.
I'll go ahead and make the existing memory view friendly wrappers private again.
There is one bug left that I'm trying to sort out. In testing I ran up against a Cython bug on windows where extern declarations are not handled properly. See the report. A good workaround is to just make a C header file with the corresponding C function declarations. I'm currently trying to figure out a nice way to macro-around it at the Cython/Tempita level to avoid further replication of the function signatures.
Once that is done I'll make another final pass to update the documentation and test on what platforms I can and then this should be finished.

On Windows you need to link with an import library which exports the symbol if you have __attribute__((dllimport)). On Unix the DL loader will resolve the symbol and you don't get a link error is not found.

I agree with @pv on omitting the "friendly wrappers". They must be more thought through, particularly the GIL issue and error handling.

Since this doesn't depend on having wrapped the subroutines using f2py first, It's probably worth thinking about other functions we might want.  For instance, the remaining BLAS functions.  One way to get the signature and other needed information (and maybe this is what you did @insertinterestingnamehere?) is to call f2py.crackfortran(filename) on the files in question, say from the reference BLAS, and then process the resulting dict to write out the cython we want.  This could be done as a one-off kind of script since we'll really only need to do it once.
In this way, it shouldn't be hard to support every function from both the BLAS and a particular LAPACK version.
Some example output:
In [25]: f2py.crackfortran.crackfortran('./dgeqp3.f'
   ....: )
Reading fortran codes...
        Reading file './dgeqp3.f' (format:fix,strict)
Post-processing...
        Block: dgeqp3
Post-processing (stage 2)...
Out[25]:
[{'args': ['m', 'n', 'a', 'lda', 'jpvt', 'tau', 'work', 'lwork', 'info'],
  'block': 'subroutine',
  'body': [],
  'entry': {},
  'externals': [],
  'from': './dgeqp3.f',
  'interfaced': [],
  'name': 'dgeqp3',
  'sortvars': ['tau', 'jpvt', 'n', 'lwork', 'info', 'a', 'work', 'm', 'lda'],
  'vars': {'a': {'attrspec': [],
    'dimension': ['lda', '*'],
    'typespec': 'double precision'},
   'info': {'typespec': 'integer'},
   'jpvt': {'attrspec': [], 'dimension': ['*'], 'typespec': 'integer'},
   'lda': {'=': 'shape(a,0)',
    'attrspec': ['optional'],
    'check': ['shape(a,0)==lda'],
    'depend': ['a'],
    'typespec': 'integer'},
   'lwork': {'typespec': 'integer'},
   'm': {'typespec': 'integer'},
   'n': {'typespec': 'integer'},
   'tau': {'attrspec': [], 'dimension': ['*'], 'typespec': 'double precision'},
   'work': {'attrspec': [],
    'dimension': ['*'],
    'typespec': 'double precision'}}}]

I suspect using the python hooks in GCC would also let you do something like this, but that's just a guess.

Here it's probably good to remind that Cythonization of Scipy was done once before, including lapack/blas wrappers:
https://github.com/pv/scipy-work/blob/jason-refactor/scipy/linalg/flapack.pyx.in
Not merged due to the status of fwrap and limited advantages over f2py at that time, however.

Perferably I would like to see that all of BLAS and LAPACK is wrapped.
fwrap requires Fortran 2003. numpy.distutils do not support that yet.

Another issue here is the g77 ABI in Apple's Accelerate Framework. We should have wrappers for this too. The F2py wrappers do, but not the Cython wrappers here as far as I can tell.

There are two ABI issues: One is calling Fortran functions from C. The other is g77 vs. gfortran. The latter is not dealt with correctly.
@insertinterestingnamehere did you understand this? I think you might have misunderstood @pv last time this was discussed.

Let's try to keep the discussion from exploding to all direction, I'm having trouble following you :)

Discussion on the "friendly wrappers" should continue at gh-4516, it's decided now we leave them out from this PR
The set of function pointers currently here is complete, right?
OSX g77 issue is already taken into account here, AFAIK

@sturlamolden: note that the fortran function calls call to wsdot not to sdot, so they go through the usual Accelerate/OSX wrappers

Right, my bad. I see setup.py includes the g77 wrappers.

2: As far as I can tell this PR wraps the same set of functions that the f2py wrappers cover. Neither BLAS nor LAPACK are completely wrapped, but the PR covers the same subsets as the f2py layer.

The extra least-squares solvers included in https://github.com/scipy/scipy/pull/4478 should perhaps be added too?

In #4478 @ev-br asked if we should have a method to avoid that the f2py and Cython wrappers diverge too much. I guess that belongs here.

I think consistency is not needed, as the f2py wrappers have quirks (sometimes returning work arrays etc) that are frozen due to backward compatibility.
The function pointer wrappers in this PR are by construction consistent with the BLAS/LAPACK API

Okay, sorry for the lag in response time, I'm working on my masters thesis right now.
@sturlamolden

The length-1 fortran arrays are a relic of the f2py wrappers. IIRC, I added them because I couldn't get the f2py wrappers to work without them. I'd be happy to remove them on my next code cleanup pass.
The int return type should be void. I haven't changed that yet either. I'll take care of that in some more code cleanup too.
The cython-friendly wrappers are used to separate the testing logic from the compiled modules right now. The functions used for testing need to be accessible from the Python level, so they must be declared cpdef.
There are various places in the cython-friendly wrappers that could be marked "nogil", but I originally added those functions to move the testing logic out of the compiled Cython modules. These versions should be a private part of the testing code. I'll prepend the necessary underscores in the next code cleanup pass. The intricacies of making them suitable for use in nogil sections of Cython can be discussed later on in https://github.com/scipy/scipy/issues/4516. I'd be all for having gil-free versions, but I don't think that needs to be a part of this PR. At this point, these are just intermediate testing functions that happen to provide a proof of concept for a bigger idea.
I am aware of the issues with the abi mismatch. I don't know all the details of how that fix works, but I have linked against the proper libraries to work around it. I've tested this on OSX Mountan Lion, but additional testing is welcome. I don't have other versions on hand.
The extra wrappers added in other PRs should be added here, I can do that here or in a later PR (see discussion below on wrappers for the rest of BLAS/LAPACK)
The Cython bug is a separate issue that I'd rather discuss on the PR there.
The tests to verify if the same routines are wrapped shouldn't be necessary since it is a lot easier to add routines here than to the f2py wrappers.

With regards to working around the Cython bug, What I'm trying to do about the Cython bug is to figure out a way to trick Cython into generating the correct code using cnames for typedefs and such. If I stuck on that for very long, the next-best thing would be to separate the signatures into a text file, read them in in the templated cython files and auto-generate a C header inside the setup.py for scipy.linalg. Either method makes it so that the signatures are only stored in one place so they are easier to add and modify.
@ewmoore @pv Yes, the routines currently wrapped here are only the ones wrapped by f2py. That was because this was piggybacking off of the f2py machinery. At this point, adding more routines should be nothing more than adding more function signatures to these files.
I originally did these by hand, and then verified them by writing a Python script that would generate the signatures from lapacke.h. I used the hand-written and the generated versions to debug one another. Getting the proper declarations out of lapacke.h wasn't very hard. Getting the rest of the declarations there should be easy. I had more trouble getting a proper machine-generated version for BLAS since the format in the header is less standardized. IIRC, I wrote two versions by hand, compared them against one another, then did some checking in a python script to check for standardized formatting. If f2py.crackfortran gives the proper signatures, that could be a quick way to autogenerate the declarations for BLAS as well.
Given that I can autogenerate all the remaining signatures, it shouldn't be terribly hard to add them. I'd be fine doing that in this PR or a later one.
The one concern I have is whether the various versions of LAPACK and BLAS required for scipy might have some differences that I'll need to account for. What versions of LAPACK and the various third party BLAS libraries can I safely assume as a minimum? Any ideas on which functions should be available in all of them?

Lapacke also suffers from not covering all of lapack, even some essential solvers, or at least it did before.

If f2py needs the length-1 arrays than just leave it as it is. I write a lot of Fortran, but I hardly ever use f2py. It just looked weird, like written by someone who doesn't understand Fortran :-)

Which LAPACK version to support is indeed a good question. Last time I checked Apple was still only providing LAPACK 3.1 because of clapack. The current version in Netlib is 3.5.
I am not sure what ATLAS, MKL and OpenBLAS provides right now. We should probably also check ACML, Cray libsci and IBM.
However, LAPACK is just a Fortran 77 library on top of BLAS. With a BLAS library we can always compile lapack from source (Netlib) and link with BLAS without any performance penalty to care about. (MKL, OpenBLAS and ATLAS only overload a handful of subroutines.)
BLAS has had a stabile API since forever.
Also we use a version of LAPACK which is indexed with C int. This is hardcoded in numpy.linalg and numpy.dot (through the included version of cblas.h) and also assumed in the f2py wrappers and ABI wrappers in SciPy. We also assume gfortran/ifort ABI (g77 ABI is wrapped to gfortran ABI) and we only call Fortran subroutines from C (i.e. we never call Fortran functions from C).
Thus the only thing we really need to worry about is the minimum LAPACK version to support.

Maybe we then require LAPACK 3.1? That's probably a relatively safe bet.
.
This seems to be the list: https://gist.github.com/pv/89dbed57a071e94adf81

As for BLAS: http://netlib.org/blas/
.
Currently missing are: xGBMV, xHBMV, xHPMV, xSBMV, xSPMV, xTBMV, xTPMV,
xTRSV, xTBSV, xTPSV, xHPR, xHER2, xHPR2, xSPR, xSPR2, xTRSM

I think consistency is not needed, as the f2py wrappers have quirks (sometimes returning work arrays etc) that are frozen due to backward compatibility.

So that idea is to declare f2py wrappers legacy and effectively freeze them after merging the PRs which are currently open?

I don't think there's a special need to declaret them frozen, except in
the sense that strict backward compatibility needs to be maintained, and
that effort spent on improving them might be more productively spent
elsewhere.

All I'm saying is that it would be good to communicate what what you just said to potential contributors somehow. ATM the draft roadmap only says "wrap more lapack functions" and we have at least five f2py PRs in the last couple of months.
Anyway, this is OT for this PR :-)

Okay, this is working on all the platforms I have on hand at the moment (64 bit windows with mingw, 64 and 32 bit linux, and FreeBSD). I'll test it on a mac on Monday when I get back to the office.
This version includes all the requested routines from BLAS and LAPACK with the exception of scabs1 and sisnan. Those aren't currently included in the wrappers for ABI compatibility and it didn't seem like it was worth the effort to wrap them since there are equivalent macros in the C standard library.
I opted to extract all the needed signatures from the blas/lapack source then generate the .h, .f, .pyx, and .pxd files from the signatures. The generated files are included here. An alternate approach would be to add another utility script that runs before cythonize, but that seemed like an unnecessary special case for this.
Testing/feedback would be greatly appreciated.

Could you add the scripts that generate these files to the repository too? These are the "preferred form for editing" in case some changes need to be made later on.

Alright, I've posted a portion of the code-generating files.
What I've done locally is:

one script that generates signature txt files from the lapack/blas source
A set of signature txt files that contain the signatures needed to generate all the wrappers.
A script that generates the wrappers based on the generated signatures.

The first script still needs some code clean up, but I can get it posted in the next day or two.
The last two parts are now included in this PR.

You should note somewhere the version of lapack that this wraps.  I think 3.1 was suggested above. This should probably go in the lapack pxd file.

@ewmoore Thanks, I've updated that.
The tests for this are passing on all platforms now. I've added a full set of code generation scripts as well. What else needs to be done here?
Thus far I've tested on:
32 bit linux with Python 2.7
64 bit linux with Python 2.7
64 bit windows with Python 2.7 and the MinGW-W64 compilers
64 bit BSD with Python 2.7
64 bit OSX with Python 3.4

Tested on OS X 10.10 with Python 2.7.8, Clang clang-600.0.51, gfortran 4.9.1 and Cython 0.22 - works like a charm.

@insertinterestingnamehere there's over 1500 warnings like this:
scipy/linalg/_lapack_subroutines.h:855:74: note: passing argument to parameter 'a' here
void F_FUNC(zgetrf,ZGETRF)(int *m, int *n, _scipy_linalg_double_complex *a, int *lda, int *ipiv, int *info);
                                                                         ^
scipy/linalg/cython_lapack.c:25326:36: warning: incompatible pointer types passing '__pyx_t_double_complex *' to parameter of type '_scipy_linalg_double_complex *' [-Wincompatible-pointer-types]
  F_FUNC(zgetri,ZGETRI)(__pyx_v_n, __pyx_v_a, __pyx_v_lda, __pyx_v_ipiv, __pyx_v_work, __pyx_v_lwork, __pyx_v_info);

Do you think these can be either avoided or silenced?

I think the wrappers are repeating typedefs that are already taken care of in NumPy. We can use npy_*. What is the purpose of _scipy_linalg_double_complex when we already have npy_cdouble? (Not to mention the former is too verbose.) Also, we can cimport numpy as cnp to access the NumPy typedefs in Cython, which is how this will be used anyway. In numpy.pxd they have names that consequently map to the Python side dtype names.
I suggest the wrappers should consequently use NumPy typedefs for all their data array arguments (or perhaps for all their arguments?) in the C headers. In the Cython files we use the corresponding dtype_t ctypedef names from Cython's numpy.pxd.
http://docs.scipy.org/doc/numpy/reference/c-api.dtype.html

@rgommers I've cleaned up the complex type declarations. That should eliminate the warnings.
@sturlamolden Thanks for the suggestion. I came to the same conclusion when writing the fix. The declarations I threw in originally were unnecessarily messy.

Comment for Accelerate: If you want to go for LAPACK 3.2 just add the extra Fortran files (from Netlib) above the 3.1 version, similar to what I did for the SGEMV fix. The only thing that will happen is that the linker overshadows the version in Accelerate framework if Apple implements more than version 3.1. So don't worry about name collisions; just let them collide and the linker will pick the symbols from our auxillary *.f files instead of the identical symbols exported from the framework. But as long as Apple say they only support clapack we should just assume that they don't provide more than 3.1.
(Since 99 % of the performance comes from BLAS (particularly *GEMM), this doesn't even matter for the performance.)

Perhaps we should consider to always compile LAPACK from source and just depend on BLAS, whether we are using MKL, OpenBLAS, ATLAS or Accelerate. The way LAPACK and BLAS (except reference BLAS) are constructed, the only thing that really matter for performance is *GEMM.

@sturlamolden Regarding Accelerate, I suppose I can move it back to just the routines in 3.1. The only real difference as far as this code is concerned is that zcgesv has a different signature between 3.1 and 3.2, but Accelerate (at least as of OSX 10.7) provides the newer interface. I can just remove it for now. Really the challenge is to include as many routines as possible that are available across all the different LAPACK releases. As it stands, what is here should be compatible with LAPACK 3.2 onward and Accelerate.
@sturlamolden Regarding compiling LAPACK from source: I'm not sure about that. That seems like something that would be better covered in a different pull request though. It would need extensive discussion on the mailing list too. It would significantly add to the compile time, but it would allow us to standardize which routines are provided. If that were ever adopted, it would be good to at least allow users to use their own lapack libraries to save on compilation time. MKL and AMCL also export their own LAPACKs. Those could contain patches/optimizations that are worth keeping.
All: What I'd prefer to do here is to include all possible routines that have a common interface between all current versions of LAPACK (3.1.0 through 3.5.0), with the exception of the ones that take string arguments, since they really should be wrapped with Fortran's ISO C bindings rather than trying to fiddle with all the different compiler standards for strings. I'd also like to exclude scabs1 from BLAS and sisnan from LAPACK since similar things are already included in the C standard library and these aren't currently a part of our ABI wrappers.
Here's a summary I put together of the differences between versions:
https://gist.github.com/insertinterestingnamehere/baf36ce759b1712f2dfb
It uses the same set of cython types as the signature files included here.
I excluded testing routines and routines with string arguments from that list.
csrot, zdrot, dlazq3, dlazq4, slazq3, slazq4, dlasq3, dlasq4, slasq3, slasq4, zcgesv, dlasq5, slasq5 are the only routines present in 3.1.0 that are missing/changed later on.

Alright, the current version exactly matches 3.1.0 (with zcgesv removed) and only wraps the fixed-interface subroutines from that release. I also removed xerbla since it expects a string argument.

The Travis error seems unrelated to this PR.

I am not sure why XERBLA is any worse than *GEMM, which also takes string arguments. I think we can just assume that a Fortran string maps to char* in C. That is what we have assumed in f2py as well.

Yep, the Travis failures are unrelated. A restart would be appreciated.

restarted

agreed that compiling LAPACK should be out of scope of this PR. That would be a big discussion, and we'd like to get this PR merged at some point:)

Ok, we should probably discuss that in an issue or on the mailing list first. It would be good to do some measurements too, both performance and extra compile-time. It is almost orthogonal to this PR anyway.

Okay, the wrappers are all here now. The ones in the last commit are the auxiliary routines that haven't been explicitly requested yet.
AFAICT, which routines to include is the only item still needing discussion here. Either way we go though, all the code is done. I can remove the last commit if we decide not to include the other auxiliary routines. Any more thoughts?

@insertinterestingnamehere can you take over https://github.com/rgommers/scipy/commit/d6601de69a5 (fixes Bento build, plus a doc formatting tweak)?

Added 0.16.0 milestone - this will be one of the highlights of the release I think, so we have to merge it.

One thing that would be very useful is to document what is now the policy for adding new BLAS/LAPACK wrappers and what is the preferred way to use BLAS/LAPACK functions from within linalg itself or other Scipy modules (from Python, Cython and C code). I admit I'm not 100% sure how to formulate that now.
This can be done in a long comment (not in the docstring) in linalg/__init__.py or in HACKING.rst.txt - not sure which is better.

@rgommers Thanks. I added the doc fix to the code generation script in an ammended commit.
I'm at a conference at the moment, but I'll try to get up a description of how to use these wrappers within scipy tomorrow evening. Internally, I'd suggest using a relative cimport within Cython modules. For C/C++ code it is possible to use these wrappers via a Cython public declaration, but would still be simpler to link with BLAS/LAPACK directly within scipy.
A module outside scipy could access these at the C/C++ level by making a new module with wrapper functions declared as public in Cython for the functions that they need to use. I haven't added that functionality in these wrappers, though I certainly could. That might be good as a separate PR, though it would be a good inclusion for 0.16 as well.

@rgommers I think what's needed is a full description of how to link a module against the same patched BLAS/LAPACK used in scipy in Cython (using cimport) and then within Fortran, C++, or C (using numpy.distutils). HACKING.rst.txt would probably be a good place to do that, though it seems like oddly specific material to put there. I think that would be better taken care of in a separate PR though. For now I've added comments to the pyx and pxd files here explaining how these wrappers should be used within scipy.

Possible future work (not in this PR):

Exploring the possibility of using public declarations in Cython to make these wrappers available to 3rd party modules at the C/C++ level without having to link directly against the original BLAS/LAPACK libraries
Using the feature added in https://github.com/cython/cython/commit/d389edef323a759031b87fd3bca459847548eeb8 to possibly simplify the generated code here (not yet, since that is only in the development version of Cython)
Writing more memory view friendly wrappers (following the pattern used in the tests here)

I've also just added a warning in the module-level docstrings about checking for alignment before using these wrappers.

Also rebased to add myself to THANKS.txt

Public declarations was the very reason to have these wrappers.

Public declararions for C is not critical to this PR. The main ting is to expose BLAS and LAPACK to third-party Cython modules without a link step.
Also this wrappers can be used from C code without any extra link step if we use Cython to build the interface between Python and C. In that case we can just fill in global function pointers in the C code when the Cython module is loaded. Stricly speaking public declarations are not needed for this. We can just document how to do it.

@insertinterestingnamehere thanks, doc additions look good. A separate PR for more extensive docs on how to use link against this from outside Scipy seems fine. Maybe the module docstring or a separate page in the html reference guide is also a good place; in HACKING.txt that can then be linked from the FAQ.

Thanks. I like the idea of a separate page that can be referenced from HACKING.txt. That puts the ways to link against BLAS/LAPACK and the ways to use those libraries within Cython in the same place.

How about we merge this, baring any further comments on Friday?

sounds good to me

Thanks Ian!

@ewmoore Thanks. I'm glad this is finished.

Fantastic :)

authors: ev-br ewmoore insertinterestingnamehere pv rgommers sturlamolden


text:
BUG: integer overflow in correlate_nd

Wronly indexing with int instead of npy_intp in correlate_nd.

LGTM, thanks Sturla.

authors: jaimefrio sturlamolden


text:
TST: upgrade pip and setuptools on TravisCI.

The current version for pip (6.0.7) has broken verbose output, which causes
build timeouts.

All green, so +1.

In it goes!

authors: e-q ev-br rgommers


text:
ENH: Use faster solvers for firls

Q is symmetric, so we should use that to speed up solving. Also, we can try using linalg.solve first, and fall back to a pinvh if necessary.
I'm not 100% sure that it's okay to use solve in cases where it warns about rank deficiency. It seems to produce okay results in my tests, but if someone thinks it's better to catch the emitted warning and use pinvh in that case, too, I can change it to do so.
Closes #9835.

Not very related to the PR (sorry), but does anyone has a suggestion on how to fix the test failure on pypy3?

The pypy issue is gh-9837, should be fixed already (to remove the failure here this PR needs rebasing, but you can simply ignore it)
______ build/testenv/site-packages/scipy/io/matlab/tests/test_streams.py _______
[gw0] linux -- Python 3.6.1 /home/circleci/project/pypy3.6-latest/bin/pypy3
Worker 'gw0' crashed while running 'build/testenv/site-packages/scipy/io/matlab/tests/test_streams.py::TestZlibInputStream::test_seek'

I'm not 100% sure that it's okay to use solve in cases where it warns about rank deficiency. It seems to produce okay results in my tests, but if someone thinks it's better to catch the emitted warning and use pinvh in that case, too, I can change it to do so.

not quite sure either. @ilayn any thoughts on this one?

If singularity(linalgerror)/ill-conditioned(linalgwarning) condition is rare enough, truncating SVD is numerically better at rounding and normalization handling even though they are equal to eigenvalue decomposition theoretically as noted in the comments. If it is common enough you can get the SVD anyways and do it in one shot regardless of the ill-conditionedness.
If you want to keep it as a two branch solution in the second branch I would suggest using linalg.lstsq as it avoids the formation of the pseudoinverse. Here is some super quick experiment
n = 10
k = 100
bools = np.empty(k, dtype=bool)
from scipy.linalg import qr, norm, lstsq, pinvh
from numpy.random import rand

for ind in range(k):
    # Generate a random ill-conditioned sym. matrix by garbling it with an ortho. matrix
    zzz = rand(n)
    # Make it numerically ill-conditioned
    zzz -= zzz[3] - 1e-8
    q, _ = qr(np.random.rand(n, n))
    A = q.T @ (zzz * q)
    b = rand(n)
    bools[ind] = norm(A @ (lstsq(A, b)[0]) - b) <= norm(A @ (pinvh(A) @ b) - b)

# Print how many times pinvh outperformed out of k tries
print(bools[~bools].size)

If it is common enough you can get the SVD anyways and do it in one shot regardless of the ill-conditionedness.

It shouldn't be too common, and computing the SVD is much (as in at least an order of magnitude) slower than using linalg.solve (for the problems solve works for).

However to enter the second branch you need to catch the LinAlgWarning instead since hitting LinAlgError is pretty rare.

In testing these particular use cases it seemed like the error was more common, but regardless I agree it would be better to catch both.

@ilayn you were right, lstsq was faster even though it does not AFAIK make use of the symmetry of the problem. I added some code to catch the ill-conditioned warning, too. So hopefully I've addressed all comments at this point.

Yes +1 for merge from me.

Benchmark added. On master:
[100.00%] ··· Running signal.FIRLS.time_firls                                                                                ok
[100.00%] ····
               ====== ============ ==============
               --                edges
               ------ ---------------------------
                 n     (0.1, 0.9)   (0.01, 0.99)
               ====== ============ ==============
                 21     397.52μs      384.81μs
                101     834.15μs      868.89μs
                1001    93.97ms       102.93ms
                2001    958.26ms      951.05ms
               ====== ============ ==============

On this PR (note big difference in last two rows in particular a factor of ~5):
              ====== ============ ==============
               --                edges
               ------ ---------------------------
                 n     (0.1, 0.9)   (0.01, 0.99)
               ====== ============ ==============
                 21     330.67μs      431.28μs
                101     580.09μs      575.02μs
                1001    20.66ms       20.72ms
                2001    194.04ms      206.35ms
               ====== ============ ==============

I ultimately removed the 2001 case because it adds some seconds to the benchmark and doesn't add too much to the story.

Looks good to go, merged. Thanks @larsoner and reviewers

authors: ilayn larsoner nmayorov rgommers


text:
Set up release pipeline

Once a release is published, it would be helpful to bundle only the files that need to be transferred over into a single release artifact so a user can just copy it over.
Additionally, a step to compile the skyportal directory to MPY is not required but would fun to do anyway, probably as a separate artifact (I think these can then go into /lib?)
See: https://github.com/adafruit/circuitpython/tree/main/mpy-cross#micropython-cross-compiler and https://pypi.org/project/mpy-cross/

Can probably use https://cli.github.com/manual/gh_release_upload since gh is available on all hosted runners. Still need to figure out the rest of the bits

The pipeline is mostly intact, but mpy compilation needs additional work.
This executes correctly:
    - name: Create compiled bundle
      run: |
        cp bundle_base.tar skyportal_bundle_compiled.tar
        tar -rvf skyportal_bundle_compiled.tar -C ./dist/ .
        gzip skyportal_bundle_compiled.tar

But we get:
code.py output:
Traceback (most recent call last):
  File "code.py", line 9, in <module>
ValueError: Incompatible .mpy file. Please update all .mpy files. See http://adafru.it/mpy-update for more info.

Turns out the PyPI package is targeted at MicroPython & isn't compatible with CircuitPython.
The simplest path forward is to vendor the compilers from Adafruit's S3 and call these in CI. Famous last words but it seems to work fine with local testing.


text:
Set subject

... since UTF8-encoded subject was created but not used, package failed to build.

that's right, thank you!

repo: scorredoira/email


text:
Unknown Error

Hello, I am trying to use upload and for some reason after working fine for the past few days. I am now incountering this error. Any suggestions?
Failed to load: InvalidStateError: Failed to execute 'inverse' on 'SVGMatrix': The matrix is not invertible.
Error: Failed to execute 'inverse' on 'SVGMatrix': The matrix is not invertible.
at l.zoomAtPoint (https://railroad.studio/studio.js:1:14786)
at l.zoom (https://railroad.studio/studio.js:1:14963)
at l.publicZoom (https://railroad.studio/studio.js:1:15115)
at Object.zoom (https://railroad.studio/studio.js:1:21120)
at new pr (https://railroad.studio/studio.js:1:113787)
at new Studio (https://railroad.studio/studio.js:1:134874)
at https://railroad.studio/studio.js:1:188622
at https://railroad.studio/studio.js:1:179223

Resolved by adding a guard around the zoom() call.

authors: CornishGuardian scottanderson


text:
There is NO way to re-init cropit in a single page application (angular)

as far as i can tell, cropit ONLY works on first page load. navigating to another page that uses cropit again always fails. it tries to init cropit again and this never works. only the first load.
i've tried everything from destroy to removing all event listeners - nothing works. it simply breaks as soon as you try to initialize it again - even though the app has essentially loaded a fresh DOM and an empty cropit window.
something tells me cropit uses globals that stick around and get in the way of another cropit init attempt. is this true?
i guess the workaround would be to setup a global cropit instance and try to re-use it when i need it again. this is not ideal as i'm doing brand new cropping tasks and it makes the most sense to start from scratch. i also don't know how this is supposed to work when the cropit element gets removed from the DOM.

Weird - I've been able to using Jquery mobile when navigating from and to the page containing cropit.
Can you show us some examples of how you're transitioning between pages or what frameworks you're using?
Thanks.

after hours of chugging i figure it out. basically i was using a css trick to make the preview a responsive width square. this required it to be position:absolute.
for some reason it would load the image on the first time but any subsequent attempts to use it without refreshing rendered cropit completely unusable until i refreshed the page again. very strange indeed.

Hello dear sorry for comment in here, i have same issue like you i guess, already digging on it i can't make multiple preview with diffrent configuration in css, honestly why  i cannot make multiple preview with this class cropit-image-preview cause in jquery.cropit.js line 927 the author define only for one images, so i copy and rename it, so ihave two js  (jquery.cropit.js) and ( jquery.cropit-user.js) with diffrent class also
here is my html
,,,html

      Resize image

Submit

      Resize image

Submit

,,,
and this is my css
.cropit-image-preview {
    background-color: #f8f8f8;
    background-size: cover;
    border: 1px solid #ccc;
    border-radius: 3px;
    margin-top: 7px;
    width: 250px;
    height: 250px;
    cursor: move;
  }

   .cropit-image-preview-test {
    background-color: #f8f8f8;
    background-size: cover;
    border: 1px solid #ccc;
    border-radius: 3px;
    margin-top: 7px;
    width: 300px;
    height: 300px;
    cursor: move;
  }

and this is my js

$(function() {
        $('.image-editor').cropit();

        $('form').submit(function() {
           var imageData = $('.image-editor').cropit('export');
           $('.hidden-image-data').val(imageData);
		   var formValue = $(this).serialize();
           $('#result-data').text(formValue);
		  return false;
        });

		$('.image-editor-test').cropit();

        $('form#test').submit(function() {
           var imageData = $('.image-editor-test').cropit('export');
           $('.hidden-image-data-test').val(imageData);
		   var formValue = $(this).serialize();
           $('#result-data-test').text(formValue);
		  return false;
        });

      });

Please someone give me little demo about multiple preview with diffrent configuration, thanks
Best Regards
Freddy Sidauruk

authors: asafy3 jpodpro sidaurukfreddy

repo: scottcheng/cropit


text:
next/link not working with passHref

Hey there,
Spent a while trying many different things but ultimately I think there's something up with the behavior of next-router-mock when passHref is used with next/link. We use Stitches CSS to wrap the Link component.
The issue is the mockRouter just never updating.
Let me know if there's something you find here, I very well could be doing something wrong but I've exhausted all different things I could think to try.
const Breadcrumb = ({ history }: Props) => {
  const router = useRouter();
  return (
    <Navigation>
      <Arrow height={16} width={16} onClick={() => router.back()} name="ChevronLeft" />
      <List>
        {history.map((item) => {
          return (
            <Crumb key={item.id}>
              <Link passHref href={{ pathname: item.pathname, query: item.query }}> // next Link
                <CustomLink active={item.active}>{item.name}</CustomLink> // anchor tag
              </Link>
            </Crumb>
          );
        })}
      </List>
    </Navigation>
  );
};

jest.mock('next/router', () => jest.requireActual('next-router-mock'));
const history = [
    {
      pathname: '/admin/plans',
      query: null,
      name: 'Plans',
      id: 'plans',
      active: false,
    },
    {
      pathname: '/admin/plans',
      query: { planId: 1 },
      name: 'Plan 1',
      id: 'plan-1',
      active: true,
    },
  ];
test('clicking previous link should navigate back', () => {
    render(<Breadcrumb history={history} />, { wrapper: MemoryRouterProvider });
    const link = screen.getByRole('link', { name: 'Plans' });

    userEvent.click(link);

    expect(mockRouter).toMatchObject({
      asPath: '/admin/plans',
      pathname: '/admin/plans',
      query: {},
    });
  });

Next v12
next-router-mocker v0.9.8

I don't see anything wrong with this code, but I think it would be nice to see the implementation of <CustomLink>.
I think the way that next/link works is that it'll pass the href to the child, but it also has extra functionality, so it passes things like onClick too.
If you're not passing onClick too, then clicking the link will just have the default Anchor tag behavior, and won't actually call router.push. So that's my guess.

authors: brandonparis scottrippey

repo: scottrippey/next-router-mock


text:
Fixed scheduler process_spider_output() to yield requests

fixes #253

Codecov Report

Merging #254 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master     #254   +/-   ##
=======================================
  Coverage   70.15%   70.15%
=======================================
  Files          68       68
  Lines        4715     4715
  Branches      632      632
=======================================
  Hits         3308     3308
  Misses       1267     1267
  Partials      140      140

Impacted Files
Coverage Δ

frontera/contrib/scrapy/schedulers/frontier.py
96.69% <100%> (ø)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a442055...3df3244. Read the comment docs.

Hey @voith sorry for a long absence. It looks absolutely fine. I hope your complex tests work. Ready for merge?

Hi @sibiryakov, This is ready for merge. You can view that the test works by looking at the builds.
like always I add the regression test in the first commit to show failure and than add the fix in the second commit.

thank you! 🍺

This PR broke Frontera behaviour. Now every yielded request end up in as a call to add_seeds as well. The initial behavior did swallow the requests in the middleware so Scrapy wasn't scheduling them. By yielding the request from the middleware, it now leave them to scrapy who eventually enqueue them with FronteraScheduler::enqueue_request . Either fix enqueue_request or revert this. For the original issue, shouldn't have this been simply fixed by setting a lower priority to the SchedulerSpiderMiddleware?

Ping @sibiryakov

@isra17 I too had noticed that add_seeds was getting called for every request yielded. I wasn't aware that it was my PR that broke this.
You are right about the fact the problem can be solved by setting a lower priority, But I thought that the code added earlier was a mistake. I was expecting @sibiryakov to tell me what could possibly break with this change.
I tested the change I made in this PR and made sure that nothing broke, but did not anticipate that it would affect performance.
Well frontera should have had a test case for this.

I've opened a PR to revert this change https://github.com/scrapinghub/frontera/pull/273

Don't worry about that, this is not the kind of issue that broke vanilla Frontera in obvious manner. I didn't see it until I had a middleware with some logic specific to links_extracted.

this PR fixes the problem probably https://github.com/scrapinghub/frontera/pull/261,
the root of the problem is Scrapy Engine behavior which is calling enqueue_request() every time it gets request instance from one of downloader middlewares (but not always).

There is no need to revert this change, it's a step in the right direction: why should allow other middlewares to operate on objects passed Frontera middlewares.

Unless I'm missing something, won't #261 end up scheduling twice the requests?
process_spider_output is still calling links_extracted for all requests, then passing them to Scrapy which in his turn calls enqueue_request that itself add the request to the _pending_requests. So now the requests live both in the frontier queue and the spider queue.

@isra17 This will happen only if you have some middleware in Scrapy yielding all requests it gets. Normally, this shouldn't happen.

@isra17 I spend more time looking into this, and I think you're right: we will get requests in two places: temp. queue and frontier. I'll release the fix soon.

See https://github.com/scrapinghub/frontera/pull/276

authors: codecov-io isra17 sibiryakov voith

repo: scrapinghub/frontera


text:
Support H3C and Huawei switches

I find that scrapligo already supports the cisco, arista, juniper, and nokia switches. Do you have any plan to extend the scrapligo to support the H3C and Huawei switches?
If not, can you tell me which file to modify and what test cases needed to be added to verify the correctness?

Nope, you just need to make your YAML platform definition(s). You can find the "built in" ones here.

authors: carlmontanari longnehc

repo: scrapli/scrapligo


text:
ready for release

2.36.1 - 2020-03-02

Issue #263

Fixed [SecureString] decryption on Unix machines running PowerShell 7 (found additional bugs)
Migrated private Encrypt and Decrypt to EncryptionHelpers.ps1 in the Private folder to allow a single place to update.

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

See the complete overview on Codacy

repo: scrthq/PSGSuite


text:
introduce MRENCLAVE and shard indentifiers in registry to distinguish worker builds

One substraTEE blockchain can be the registry for many substraTEE-worker projects:

different worker builds with different STF
same worker builds but operating on a different state ( shard). See this issue

The registry should explicitly mention

MRENCLAVE (identifying the worker enclave build) such that different MRENCLAVES can use the same registry
the (possibly multiple) shards a worker supports

only mrenclave is done, shards not yet. reopening

the request format should be changed such that we can ensure the request is for a specific MRENCLAVE and shardid.
The shardid must be specified explicitly and unencrypted (such that the chain event queue can route request to the right workers), but mrenclave could just be included for the signature, but not for the payload (like genesis_hash for substrate xt). Reasoning: you wouldn't expect different mrenclave for the same shard.

resolved in #22

repo: scs/substraTEE-node


text:
Paginated single category pages don't work beyond the index

I have replicated this bug by cloning the sculpin-blog-skeleton and added more than 10 posts that all belong to the same category.
You can see a live version of this bug here http://dannyweeks.com/blog/categories/my-thoughts with the pagination link at the bottom.
I have more than 10 posts in my 'My Thoughts' category so therefore the expected next page url would be something along the lines of http://dannyweeks.com/blog/categories/my-thoughts/page/2 but what is generated is  http://dannyweeks.com/blog/categories/2
Nothing is generated in the category folder, for example /blog/categories/my-thoughts doesn't have  page/2.html within it.
If it helps the source of my site is https://github.com/dannyweeks/dannyweeks.com

While you're right that the link generates as you mentioned, your current configuration seems to download a file when the link is followed.
http://dannyweeks.com/blog/categories/2

Yes strangely that is what is generated. The default sculpin-blog-skeleton also produces this same issue.

authors: dannyweeks tchalvak

repo: sculpin/sculpin


text:
How to split a gene into 2 genes based on the anndata.obs['sample'] values?

Hello,
Can someone help me and tell me how to split a gene into 2 genes based on the anndata.obs['sample'] values?
Because there are 2 transgenic genes and most of the sequences are the same, which can't be separated during alignment.
Those 2 genes are in different samples so I want to split them in the anndata file. Thanks a lot.
Best
Dan

See here: https://stackoverflow.com/a/58645591/247482

repo: scverse/anndata


text:
read_fcs not in requirements

----> 5 from spatialdata_io import visium
      6 import spatialdata as sd
      7 import scanpy as sc

File [~/Projects/spatialdata-io/src/spatialdata_io/__init__.py:3](https://vscode-remote+ssh-002dremote-002btunnel.vscode-resource.vscode-cdn.net/home/t-gpalla/Projects/cl-spatial-datasets/scripts/~/Projects/spatialdata-io/src/spatialdata_io/__init__.py:3)
      1 from importlib.metadata import version
----> 3 from spatialdata_io.readers.codex import codex
      4 from spatialdata_io.readers.cosmx import cosmx
      5 from spatialdata_io.readers.curio import curio

File [~/Projects/spatialdata-io/src/spatialdata_io/readers/codex.py:12](https://vscode-remote+ssh-002dremote-002btunnel.vscode-resource.vscode-cdn.net/home/t-gpalla/Projects/cl-spatial-datasets/scripts/~/Projects/spatialdata-io/src/spatialdata_io/readers/codex.py:12)
     10 import anndata as ad
     11 import pandas as pd
---> 12 import readfcs
     13 from dask_image.imread import imread
     14 from spatialdata import SpatialData

ModuleNotFoundError: No module named 'readfcs'

question is, how did the build/test pass with this. We should add a test about at least importing which we might not have at all.

my bad it is, closing.

repo: scverse/spatialdata-io


text:
node or cluster stop: make timeout configurable

Currently, node.stop(), and cluster.stop() which uses it, are hard-coded
to allow Scylla 127 seconds (=2^7-1), just a bit over two minutes, to
shutdown before failing. In some tests, see for example
https://github.com/scylladb/scylla/issues/4019, we know that we need more
than that time for the shutdown to complete.
So this patch makes the waiting time configurable, via an optional parameter
"wait_seconds" which defaults to the previously hard-coded 127 seconds and
can now be set to any number of seconds.
Signed-off-by: Nadav Har'El nyh@scylladb.com

BTW, this needs backport to branch 3.0 (if we have such a branch for ccm)

ping

Oops, sorry, I missed that it was already committed (didn't refresh automatically...)

repo: scylladb/scylla-ccm


text:
Removed warning in benchmarks

One value was not unwrapped, and the compiler has been warning
about unused result while checking target benchmarks.
Pre-review checklist

[x] I have split my patch into logically separate commits.
[x]  All commit messages clearly explain what they change and why.
[ ] I added relevant tests for new features and bug fixes.
[x] All commits compile, pass static checks and pass test.
[x] PR description sums up the changes and reasons why they should be introduced.
[ ] I added appropriate Fixes: annotations to PR description.

The CI fails because of nonmerged clippy "deprecated use" fix

@wprzytula please rebase, the CI check should pass now but the PR needs to be based on current main.

authors: piodul wprzytula

repo: scylladb/scylla-rust-driver


text:
Add requiresDependencyResolution=compile to JAXRSAnalyzerMojo

This adds all classes needed for compilation to the analysis scope.
Previously not all classes were considered by the JAXRSAnalyzerMojo
which lead to execeptions during build.

Just tried JAXRS-Analyzer Maven Plugin with Keycloak and got exceptions similar to those from #17
As a PoC I tried to analyze the JAX-RS resources from Keycloak.
Before the change:
$ mvnDebug com.sebastian-daschner:jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs -DskipTests
Preparing to Execute Maven in Debug Mode
Listening for transport dt_socket at address: 8000
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Keycloak REST Services 2.4.0.CR1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs (default-cli) @ keycloak-services ---
[INFO] analyzing JAX-RS resources, using Plain text backend
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:12 min
[INFO] Finished at: 2016-11-09T00:59:25+01:00
[INFO] Final Memory: 10M/240M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.sebastian-daschner:jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs (default-cli) on project keycloak-services: Execution default-cli of goal com.sebastian-daschner:jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs failed: A required class was missing while executing com.sebastian-daschner:jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs: org/keycloak/common/ClientConnection
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>com.sebastian-daschner:jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
[ERROR] urls[0] = file:/home/tom/.m2/repository/com/sebastian-daschner/jaxrs-analyzer-maven-plugin/0.11-SNAPSHOT/jaxrs-analyzer-maven-plugin-0.11-SNAPSHOT.jar
[ERROR] urls[1] = file:/home/tom/.m2/repository/com/sebastian-daschner/jaxrs-analyzer/0.11-SNAPSHOT/jaxrs-analyzer-0.11-SNAPSHOT.jar
[ERROR] urls[2] = file:/home/tom/.m2/repository/org/codehaus/plexus/plexus-utils/1.1/plexus-utils-1.1.jar
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import  from realm ClassRealm[maven.api, parent: null]]
[ERROR]
[ERROR] -----------------------------------------------------: org.keycloak.common.ClientConnection
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException

After the change:
$ mvnDebug com.sebastian-daschner:jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs -DskipTests
Preparing to Execute Maven in Debug Mode
Listening for transport dt_socket at address: 8000
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Keycloak REST Services 2.4.0.CR1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- jaxrs-analyzer-maven-plugin:0.11-SNAPSHOT:analyze-jaxrs (default-cli) @ keycloak-services ---
[INFO] analyzing JAX-RS resources, using Plain text backend
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.743 s
[INFO] Finished at: 2016-11-09T01:06:09+01:00
[INFO] Final Memory: 22M/332M
[INFO] ------------------------------------------------------------------------

Oh yes, that should resolve a couple of problems actually. Thanks!

authors: sdaschner thomasdarimont

repo: sdaschner/jaxrs-analyzer-maven-plugin


text:
core: Stop deprecated use of start and stop

Use start_samples, stop_samples instead.

If possible I'd like to tackle this issue.
Any other information I should know considering that this is my first time working on this project?

When you're done, make a PR and the automatic testing should make sure things are working as expected.

Which start/stop are you referring to?
I'm seeing a lot of these exactly, but as arguments only as part of dictionary keys in validation/train kwargs.

I'm not sure off the top of my head.
Basically, find anything in core.py that initializes a Dataset using the start and stop arguments, and have it instead use start_samples and stop_samples instead.
I'd recommend doing it by dropping a breakpoint on like line 300 then looking at the stack to see where it came from.
It'll probably often be via init_dataset.
Does this help?

Do you have any instructions on how to run the code?

Is it possible that these parameters are only explicitly written in the single_pair.json and two_pairs.json files?
Digging around a bit that's where I've been able to find them

authors: sdatkinson yonMaor

repo: sdatkinson/neural-amp-modeler


text:
Fix typo in manifest.go docstring

:)

hah! "Ingored". that's great.
this repo is now basically dead, as the code has moved into github.com/golang/dep (and that line's been changed over there), but sure, why not 😄

Oh! Doh, missed that note on the readme. I think I had followed a link here from an older golang/dep issue discussion I was reading and just happened to see the typo! Haha

Well thanks for merging anyways lol

authors: sdboyer vergenzt

repo: sdboyer/gps


text:
picker 组件 setValue无效的问题解决了

picker组件有一个问题,就是在没有open过的情况下,setValue是不工作的. 经本人调试,在源码sm.js中加入如下代码即可解决.
在2012行(onPickerClose方法之后)加入代码:
//fix setValue
`
p.mySetValue=function(value){
if (p.cols.length!=0) {
p.setValue(value);
return;
}
// Layout
p.layout();
p.params.value=value;
p.value=value;
        // Append
        p.container = $(p.pickerHTML);

        // Store picker instance
        p.container[0].f7Picker = p;

        // Init Events
        p.container.find('.picker-items-col').each(function () {
            var updateItems = true;
            if ((!p.initialized && p.params.value) || (p.initialized && p.value)) updateItems = false;
            p.initPickerCol(this, updateItems);
        });

        // Set value
        if (!p.initialized) {
            if (p.params.value) {
                p.setValue(p.params.value, 0);
            }
        }
        else {
            if (p.value) p.setValue(p.value, 0);
        }
    };

使用方式,代替原来的setValue方法即可:$("#myCtl").picker("mySetValue",[value1,value2]);`

我只想说, 用这玩意儿真的是日了狗了.文档就是狗屎.

这么多bug，官方为啥不更新了。。。

authors: HiJesson virtuecai wggCode

repo: sdc-alibaba/SUI-Mobile