text:
scipy installation hang if pip install at embed 1GiB RAM system

I have an development board with armv7 cpu and 1GiB RAM, when install it with
pip install scipy
the pip download "scipy-1.2.1.tar.gz (23.1MB)" file and try to install it, but the installation will hang as no more memory to finish the scipy source code compilation, because there is 4 cc1plus process exist in top command cosumed all RAM.

Can I disable the scipy parallel(multicore) compile?

The most steps of building is single core, but sometimes there is 4 cc1plus process, I think I need to find out which step launch multiply cc.

After open the verbose mode of pip, there is many compiler processes when it doing:
f951
gfortran:f77:scipy/optimize/minpack/lmpar.f
cc1
arm-linux-gnueabi-gcc: scipy/special/cephes/round.c
cc1plus
arm-linux-gnueabi-g++: scipy/sparse/sparsetools/sparsetools.cxx
It is only out of memory after the output show "compiling C++ sources", may be it is the same problem when I compile other library with "make -f", it only works compile without "f"options.

Is there any one can tell me how to configure the "compiling C++ sources" phase which building launched by pip?

It's invoked by the build_ext command of numpy.distutils. That goes via https://github.com/numpy/numpy/blob/master/numpy/distutils/ccompiler.py#L694, that particular line monkeypatches Python distutils. I don't see anything in numpy.distutils that parallelizes anything, so either it's in Python distutils or it's how your compiler is configured.

I checked the scipy and numpy code and document with the notice of @rgommers , and fixed it with "export NPY_NUM_BUILD_JOBS=1" with lucky. The building phase consume 900MiB RAM at most, the corresponding code is https://github.com/numpy/numpy/blob/e21d2cbdf15a86e16ef4877a42ed98d97ea296d3/numpy/distutils/misc_util.py#L102.

Thanks for the update @Haffon. If that fixed it, I do consider it a bug (likely in numpy.distutils). It seems like the C++ build step calls get_num_build_jobs in the absence of a --parallel flag, which is wrong.

It is fixed with setting NPY_NUM_BUILD_JOBS environment variable to 1.

I investigated a little. There's two forms of parallelism:

building multiple extensions in parallel (controlled by the --parallel (or equivalent -j) flag)
parallel building within a single extension (enabled by default)

You're running into the second form. It happens to be controllable by NPY_NUM_BUILD_JOBS, perhaps not entirely by design.
The default is sensible I think, because it speeds up the build considerably, it's robust, and it's not common anymore that there's not enough memory.
So I think I'll leave it at that for now. Better documentation would be nice, but that's true of everything distutils-related.

Thanks for your explanation, so I can find the correct location to configure the compile options. There is still a long call stack I need to check to make sure I don't miss something.

authors: Haffon rgommers

repo: scipy/scipy


text:
optimize.linprog test failures on macOS

This one I observe locally:
__________________________________________ TestLinprogIPSparsePresolve.test_bug_10466 __________________________________________
../scipy/optimize/tests/test_linprog.py:1595: in test_bug_10466
    assert_allclose(res.fun, -8589934560)
E   AssertionError:
E   Not equal to tolerance rtol=1e-07, atol=0
E
E   x and y nan location mismatch:
E    x: array(nan)
E    y: array(-8589934560)
        A_eq       = [[1.0, 1.0, 0.0, 0.0, 0.0, 0.0, ...], [0.0, 0.0, 1.0, 1.0, 0.0, 0.0, ...], [0.0, 0.0, 0.0, 0.0, 1.0, 1.0, ...], [1.0, 0.0, 1.0, 0.0, 1.0, 0.0, ...], [1.0, 0.0, 1.0, 0.0, 1.0, 0.0, ...], [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...], ...]
        b_eq       = [314572800.0, 419430400.0, 524288000.0, 1006632960.0, 1073741820.0, 1073741820.0, ...]
        c          = [-8.0, -0.0, -8.0, -0.0, -8.0, -0.0, ...]
        o          = {'_sparse_presolve': True, 'autoscale': True, 'sparse': True}
        res        =      con: array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan])
     fun: nan
 message: 'The solution does not sat...float64)
  status: 4
 success: False
       x: array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan])
        self       = <scipy.optimize.tests.test_linprog.TestLinprogIPSparsePresolve object at 0x140ae7130>
        sup        = <numpy.testing._private.utils.suppress_warnings object at 0x140ae7b20>

And this one I see on the scipy-wheels repo (macOS and Linux on Azure):
______________ TestBoundedNelderMead.test_outside_bounds_warning _______________
[gw2] darwin -- Python 3.8.8 /Users/runner/hostedtoolcache/Python/3.8.8/x64/bin/python
../../../../hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/scipy/optimize/tests/test_minimize_constrained.py:738: in test_outside_bounds_warning
    minimize(prob.fun, [-10, 8],
E   Failed: DID NOT RAISE <class 'UserWarning'>
        bounds     = Bounds(array([-inf,   1.]), array([4., 5.]))
        prob       = <scipy.optimize.tests.test_minimize_constrained.Rosenbrock object at 0x123f55670>
        self       = <scipy.optimize.tests.test_minimize_constrained.TestBoundedNelderMead object at 0x123f559d0>

The local one just came in with gh-13515 (Cc @AtsushiSakai). The scipy-wheels probably too, but not sure - CI runs have been infrequent.

Thank you for letting me know the test failure. It does not happen in my environment.. My environment is Docker ubuntu on Mac OS. I will try to reproduce it on Mac OS.

I suspect the issue is old GCC on manylinux1. That may deal with nan/inf differently. If it's not easy to fix, we can just verify after dropping manylinux1 (hopefully before the next release).

Failures silenced in gh-13847.

Weird also the CI was green before I merged (appart from the usual doc_build).

I cannot reproduce the bug locally (1.7.0.dev0+272f7b2 1.20.2 sys.version_info(major=3, minor=8, micro=5, releaselevel='final', serial=0)) I am also on macOS BigSur and running in conda.

nan/inf behavior is very often platform and compiler dependent, so that's not very surprising.

@rgommers are you still seeing these, especially TestLinprogIPSparsePresolve.test_bug_10466?

@rgommers are you still seeing these, especially TestLinprogIPSparsePresolve.test_bug_10466?

That's passing now. Environment has changed in the meantime of course, so not sure if it'll be completely gone for everyone. But maybe worth unskipping the tests and seeing what happens.

Meh I think I'll leave it alone, since it's looking like we'll deprecate the old solvers in gh-15512. I was wondering whether we could close this issue when that merges.

This is now also regularly failing in the Azure CI job prerelease_deps_coverage_64bit_blas (example log:
FAILED scipy/optimize/tests/test_linprog.py::TestLinprogIPSparseCholmod::test_bug_10466
FAILED scipy/optimize/tests/test_linprog.py::TestLinprogIPSparsePresolve::test_bug_10466

thanks!

source: gharchive/issue

authors: AtsushiSakai mdhaber rgommers tupui

license_type: permissive

license_source: github-api


text:
ENH: Add the Irwin-Hall (Uniform Sum) and Bates (Uniform Mean) distributions to scipy.stats

Is your feature request related to a problem? Please describe.
No response
Describe the solution you'd like.
The Irwin-Hall and Bates distributions are closely-related continuous probability distributions which respectively model the sum and mean values of repeated uniform draws. This comes up during repeated discrete processes where each process step has associated uncertainty. For a simple example where this would have been useful, see this stackoverflow question on particle motion and its accepted answer, and for a more complex example see this paper on distributed beamforming.
The Irwin-Hall distribution is implemented in sympy as sympy.stats.UniformSum(). Mathematica implements these as UniformSumDistribution and BatesDistribution. It probably makes sense to follow these naming conventions.
Describe alternatives you've considered.
No response
Additional context (e.g. screenshots, GIFs)
No response

These seem quite reasonable to add. PR welcome, I'd say.

This is also relevant for Rao's Spacing Test and more tests of random divisions of the unit interval.
The derivation of the pdf and cdf were done by Darling in
On a Class of Problems Related to the Random Division of an Interval
I'll look at implementing this

authors: mdhaber nimish scottshambaugh


text:
linalg.svd output is non-deterministic on Windows 32bit

Hi, I'm writing a PR for scikit-learn and from time to time I get a failure on Windows 32bit. The only potential cause of failure is the scipy.linalg.svd calls used in sklearn.decomposition.pca. And indeed I find a very strange behavior coming from scipy.linalg.svd : I wrote a test (see https://github.com/johny-c/scikit-learn/blob/win32/sklearn/neighbors/tests/test_win32.py) that calls svd two times consecutively on the exact same input and compares the resulting decompostion matrices. Most of the time they are equal as they should. But every now and then - I still cannot find any pattern on the frequency - they are not. See https://github.com/johny-c/scikit-learn/blob/win32/sklearn/neighbors/tests/test_win32_digits.out and https://github.com/johny-c/scikit-learn/blob/win32/sklearn/neighbors/tests/test_win32_iris.out. It seems like the output of the LAPACK dgesdd routine is not 100% deterministic in Windows 32 bit.
Reproducing code example:
import numpy as np
from sklearn.utils.validation import check_X_y
from sklearn.utils.testing import assert_allclose
from sklearn.utils.testing import assert_array_equal
from sklearn import datasets
from scipy import linalg
from sklearn.utils.extmath import svd_flip

rng = np.random.RandomState(0)
digits = datasets.load_digits()
perm = rng.permutation(digits.target.size)
digits.data = digits.data[perm]
digits.target = digits.target[perm]

def test_scipy_svd():
    X_init = digits.data
    y_init = digits.target

    # X_init = iris.data
    # y_init = iris.target
    X, y = check_X_y(X_init, y_init, ensure_min_samples=2)
    U1, S1, V1 = linalg.svd(X, full_matrices=False)
    # flip eigenvectors' sign to enforce deterministic output
    U1, V1 = svd_flip(U1, V1)

    U2, S2, V2 = linalg.svd(X, full_matrices=False)
    U2, V2 = svd_flip(U2, V2)

    assert_allclose(S1, S2)
    assert_allclose(U1, U2)
    assert_allclose(V1, V2)

Error message:
Traceback (most recent call last):
File "C:\Users\Joe\Anaconda2\lib\site-packages\nose\case.py", line 197, in runTest
self.test(*self.arg)
File "C:\Users\Joe\projects\python2\scikit-learn\sklearn\neighbors\tests\test_win32.py", line 30, in test_scipy_svd
assert_allclose(U1, U2)
File "C:\Users\Joe\Anaconda2\lib\site-packages\numpy\testing\utils.py", line 1411, in assert_allclose
verbose=verbose, header=header, equal_nan=equal_nan)
File "C:\Users\Joe\Anaconda2\lib\site-packages\numpy\testing\utils.py", line 796, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=1e-07, atol=0
(mismatch 3.12413049527%)
x: array([[  3.209578e-02,   2.183222e-02,  -2.281640e-02, ...,
0.000000e+00,   0.000000e+00,   9.931736e-01],
[  2.288146e-02,   2.978713e-03,  -1.959516e-02, ...,...
y: array([[  3.209578e-02,   2.183222e-02,  -2.281640e-02, ...,
-0.000000e+00,   0.000000e+00,   9.931736e-01],
[  2.288146e-02,   2.978713e-03,  -1.959516e-02, ...,...
Scipy/Numpy/Python version information:
0.19.1, 1.12.1, 2.7.13
and
0.16.0, 1.9.3, 2.7.13
and
0.16.0, 1.9.3, 3.5.3

Floating-point reproducibility is generally not guaranteed (see e.g. intel compiler manuals) on any platform, even for code in the same program can give different results due to e.g. data alignment, unless you take special care with library and compiler options, as it depends on things such as compiler optimizations. This is not generally considered a bug, and correct code should not assume it. It is a speed/precision tradeoff, and official binaries provided by scipy project do not ensure 100% fp reproducibility.
I don't know which binaries you are using, but it's possible e.g. with MKL it's possible to toggle a high-precision mode.

For others who might run into GESDD problems (which could possibly be related to this) -- I have had reproducibility problems with GESDD and MKL on Linux (and confirmed by others on Linux and OSX) depending on CPU architecture, where it can raise LinAlgError on valid (finite) matrices:
https://github.com/ContinuumIO/anaconda-issues/issues/695
https://software.intel.com/en-us/comment/1869912
The problem can at least sometimes be mitigated by changing the number of threads MKL uses. However, unlike this bug -- which seems to fit in the numerical precision category -- solving the SVD vs raising a LinAlgError is beyond numerical precision, so it's an MKL bug.
However, more recently I have begun to suspect that MKL's gesdd might produce incorrect (i.e., sometimes beyond numerical precision incorrect) results without producing LinAlgError:
https://github.com/mne-tools/mne-python/issues/4922#issuecomment-368920709
Still need to isolate a single call, though, to make sure that it's not just numerical inaccuracies being magnified by subsequent processes.

authors: johny-c larsoner pv


text:
stats.ks_2samp() misleading for small data sets.

scipy.stats.ks_2samp(x1, x2) implements the two-sided two-sample Kolmogorov-Smirnov test to compare two data sets.  It uses an asymptotic formula which is best suited to large size data sets.  When given small data sets, the results are quite far from correct
Reproducing code example:
>>> import scipy.stats
>>> scipy.stats.ks_2samp([0],[1])
Ks_2sampResult(statistic=1.0, pvalue=0.28904142837082675)

The statistic=1 is correct. However the actual pvalue is 1.0.
The ECDF of a sample with a single observation is a step function at that observation, and the difference between any 2 such ECDFs will have 2 steps, and the maximum absolute difference D_{1,1}=1.0. I.e. No matter the observations, the value D_{1,1} is always 1.0, hence Pr(D_{1,1} >= 1) = 1, which is not close to 0.289.
Below are some more examples:
>>> scipy.stats.ks_2samp([0],[1,2])
Ks_2sampResult(1.0, 0.2013129706454185)
# Correct is 0.6666666666666667

>>> scipy.stats.ks_2samp([1,2], [3,4,5,6])
Ks_2sampResult(1.0, 0.04686590906054742)
# Correct is 0.1333333333333333

I.e. There plenty of examples where the returned pvalue is small, even less than 5% say which is a common threshold for experimental results, but the true value is much greater.
Scipy/Numpy/Python version information:
1.3.0.dev0+e119b57 1.16.0 sys.version_info(major=3, minor=6, micro=6, releaselevel='final', serial=0)

Should be fixed by #9753, which I've now merged.

authors: pvanmulbregt tylerjereddy


text:
ENH: higher-dimensional input in spatial.SphericalVoronoi

What does this implement/fix?

This adds support for higher-dimensional input in spatial.SphericalVoronoi.  ~It also adds a circumradii attribute, which calculates the radius of each facet.~
Additional information

Builds on the discussion in #10494

Thanks for another great review @tylerjereddy - I am learning a lot about good practices.
I removed the circumradii attribute.  I will think about it some more and perhaps put it in a later PR.

What's the application / use case for the circumradii? I realize you've removed that now. Is that useful in the crystallography domain/ your manuscript?

Ah, you were talking about the Delaunay circumcircles, I think, so yeah maybe awkard for those here.

Just FYI, I need a Voronoi diagram with both the circumcenters and the circumradii for my application.  Of course the circumcenters are the Voronoi vertices, so that part is given.  As long as the vertices are associated with the hull simplices then I can get the circumradii.
BTW, this enhancement is very cool.  I don't really need it in my world, but very cool.

Can you point to an example publication that uses a spherical Voronoi diagram with ndim > 3? Is this the example you gave before from your crystallography work again or is there something else?

My motivation for extending to arbitrary dimensions comes from my crystallography work.  In general though, spherical voronoi diagrams are useful for calculating the covering radius of a point set on a hypersphere.  I will do a literature review and see what else I can find.

I have added tests for the Euler characteristic in arbitrary dimensions.  The generalization is described here: https://en.wikipedia.org/wiki/Euler_characteristic#Generalizations

I rebased on latest master & fixed the deprecated as_dcm() call. I confirmed that spherical_voronoi.py still has 100 % line coverage in this feature branch.
Let me now check the asv benchmarks for 3D input.
Beyond that, I can push fixes for any of the changes you agree with---let me know.

I ran asv continuous master f2f49c6d -b "SphericalVor*" >& output.txt
There was no evidence of a slowdown in performance as a result of adding n-dimensional support, so that's good.

Thanks for the review @tylerjereddy.
I have made the changes you suggested. I pushed the fixes myself because my git skills are poor and this is easier for me to manage.

I updated the appropriate section of the release notes wiki, feel free to expand/correct as you see fit.

source: gharchive/pull-request

authors: pmla rtshort tylerjereddy


text:
MAINT: cleanup of some line_search code

Whilst going back over some code in optimize.line_search I came along some historical cruft in line_search_wolfe1 and line_search_wolfe2. The documented behaviour of those functions is to accept an fprime that is callable. However, they were also accepting an undocumented tuple of the form (fp, eps). Here fp would be a callable of the form of approx_fprime, which would be called as fp(x, func, eps, *args), where func is the original objective function.
This PR removes the undocumented behaviour, which isn't used internally in scipy.

Test failure is a timeout, but with this change if it passes on one matrix entry it should pass on all (change is a flow control change rather than changing numerical things).
The only issue will be if there are external users of these functions who rely on undocumented behaviour.

Core-maintainer approval, so merging.


text:
ENH: Map the rotation quaternion double cover of rotation space to a canonical single cover

Reference issue
Closes https://github.com/scipy/scipy/issues/17024
What does this implement/fix?
Rotation quaternions form a dual cover of the rotation space, such that q = -q describe the same rotation. This introduces ambiguity as to the quaternion representation you will get for a given rotation. This PR reduces the double cover down to a "positive" single cover, so that every rotation will be represented by a unique quaternion. Note that I am not aware of prior art here, so the logic for selecting the polarity when the w component is 0 is new.
Additional information

I'll break it into its own function, but will give that a shot tonight.

Thanks to our discussion over on #17429, I'm a little less inclined flip anything when we get to the case where q[3] == 0. These are always cases which involve a 180deg rotation around an axis. The special thing about 180deg rotations is that a +180deg rotation around an axis ends up doing the same thing as a -180deg rotation around that same axis. Basically, the double-cover representation (the negative angle around the opposite-direction axis) is the same as the negative angle around the same axis. This is different from the double-cover issues with every other rotation angle.
For kind of similar reasons that we have both +0.0 and -0.0 in floating point, I'd suggest that we leave these cases alone. I think that when we get the two distinct quaternions, I think we're mostly getting them because of distinct reasons that would be good to preserve.

Agreed that those edge cases are a bit of a different issue than the double cover. But I don't think the reasoning for signed zeros is quite applicable - my understanding is that the reason for signed zeros is a combination of needing to distinguish between +-∞ when calculating 1/0, and the practical consideration that many functions have a discontinuity at 0 and you want to know from which direction you approached the limit (for calculating derivatives, etc). Neither of those are at play here since unlike the real number line, the space of rotations SO(3) is a smooth, cyclic group.
I think the more apt comparison is the trig functions which are also cyclic. To that end, the only trig function that has a double point is atan2 which returns in the range [-pi, pi] inclusive. But here again this can be explained by atan2 not being smooth and needing to accommodate the discontinuity.
A few more scattered thoughts:

Because again rotations are a smooth cyclic group, uses which rely on a distinction of +-pi are fundamentally fragile and likely not implemented correctly. Breaking rather than accommodating that might be a better user choice.
The big benefit to a fully unique mapping is that it allows you to do equality checks easily, which is something that is not currently implemented for rotations but should be. I'm thinking now though that since this is all a bunch of floating point math, equality checks should really be done with an approx function (something like the below, I'll open an issue). Since an approx check is the right way to solve that problem, a fully unique mapping is maybe not so important.

# a method of the Rotation class, or standalone?
def rotation_approx_equal(self, R2, atol=1e-12):
    Rdiff = self * R2.inv()
    return Rdiff.magnitude() < atol

In general I'm still not super keen on having those double points, but I'm less against it after thinking about the equality checks and if the above reasoning doesn't convince you I can take out the flips when q[3] == 0.

Given that we were talking about non-integral powers, I think a reasonable analogy would be how we deal with the branch cuts in non-integer powers of complexes. I mention signed zeros because that's a case that does come up. The unit complexes are also a smooth cyclic group. -1 represents a 180 degree rotation. We can take the square root of that number and should get either +1j or -1j. The sign of the imaginary zero actually does actually determine which branch we take in the floating point implementation.
[~]
|20> zp = np.array([-1.0, +0.0]).view(np.complex128)[0]

[~]
|21> zm = np.array([-1.0, -0.0]).view(np.complex128)[0]

[~]
|22> zp, zm
((-1+0j), (-1-0j))

[~]
|23> np.sqrt(zp), np.sqrt(zm)
(1j, -1j)

With quaternions and their half-angle representation, we don't even have to rely on such fragile things as signed zeros.
To your points:

There is likely some fragility, but the main place we'd encounter this is the conversion from a rot-vec representation, and that seems to preserve okay and we've shown that it behaves the way I'd expect.
There are multiple possible answers for every non-integral power for any nontrivial rotation. The +/-pi case is not special in that regard. The question here is just whether or not there is value in being able to represent +pi and -pi rotations separately. I think there is because it helps narrow which of the multiple possible answers to give in the non-integral power case.
Yeah, if the main case is for (near-)equality comparisons, that's the better implementation. I'd want to see that implementation regardless of what's done with the double-cover.

I pulled out my copy of Visualizing Quaternions (legally online now?) to refamiliarize myself with the pragmatic issues around the double-cover. Chapter 12 makes me think that there might be some pragmatic value to retaining the double-cover always in order to represent intermediate rotations. I haven't reduced that thought to a constructive demonstration, though. If you think the more geometric implementation of rotation comparison actually solves your use cases, I may suggest leaving the representations alone.

Thanks for that info on the unit complexes in numpy. That was behavior I was unaware of, and I think matching that convention is good justification that rotation quaternions should behave the same - I'm convinced.
As to keeping the double cover in general (cool book!), the 720 degree symmetry is interesting mathematically but I think could cause issues. Going back to non-integral powers, keeping that would distinguish between a +240 and -120 degree rotation, and as we were talking over in that MR, we want to force the smallest angle path to remove ambiguities when calculating e.g. a power of 0.5. (This is why +/-pi is a special case - the smallest angle rotation is not unique like it is for any other point.) We could just apply the single cover constraint as needed at __pow__ function call (like as_rotvec currently does), but at that point I'm really struggling to see any use case for the extra bit of info in the double cover.

@alexander-soare we're talking a bit about whether this is really needed. My use case is checking whether rotations are equal to each other, but that would be better handled by https://github.com/scipy/scipy/issues/17458. Since you're the one who opened https://github.com/scipy/scipy/issues/17024 could you please explain a bit what problem you're trying to solve by having the w component always be positive?

Quaternion slerp with uncanonicalized quaternions isn't broken, per se. It doesn't necessarily represent the shortest great-circle arc but rather the opposite direction. Which can be useful in some practical circumstances, apparently.
On the other hand, we aren't that careful elsewhere about preserving the which of the double-cover quaternions is chosen to represent the rotation through particular operations. When I went to make a demonstration of that example using Rotation, I noticed that .inv() just negates the real part of the quaternion (i.e. the negative of the inverse quaternion), presumably for speed reasons.
All in all, I'd rather we move more towards retaining the full capabilities of the double-cover quaternion space (e.g. implementing Rotation.inv() as the exact inversion of the underlying quaternion) and perhaps provide an explicit canonicalization to a single cover when requested.

This is a bit orthogonal to the current discussion, but the implementation of Slerp in this library uses the as_rotvec representation, which does convert to a positive w term internally and as such takes the shortest great circle path. I think there would definitely be value in a flag to take the opposite longest great circle path instead, but that can be another issue.
To bring it back to the current discussion however, slerp direction of rotation is not something that should be inherited from the quaternion polarity. The quaternion is again not the fundamental object we are working with in this context, but a representation of a Rotation. If for example slerp did pick its direction from the quaternion polarity, then it's easy to imagine a scenario where the user composes several rotations without using quaternions, ends up with a -w term under the hood, and is blindsided when slerp goes the opposite direction. How are they supposed to diagnose and fix that if all they are familiar with is rotation matrices? There is nice mathematical symmetry in keeping the full double cover, but we should not be relying on any of the resulting properties in the API.
That's not really an argument one way or another on what we use under the hood, just that we shouldn't try to be fancy and exploit the properties of the double cover. Users should be able to predict what the library will do without knowing the finer differences between SU(2) and SO(3). Agree that we should fix .inv() in any case.

+1 for @scottshambaugh's concept: this is an API for working with rotations therefore the user shouldn't have to worry or even know about quaternion specific things that might happen under the hood.

If we do want to make sure the double coverage works as expected under the hood as a “hidden feature” for those who work purely with quaternions, then we’ll want to add test coverage of that.

@rkern do you think that implementing as a flag in the as_quat function (eg .as_quat(double_cover=False)) would cover all the bases here?

Ok, I've updated the MR and I think this should align with what everyone is saying:

Made it so that the mapping to the single coverage of rotation space only happens through a flag (default False) in the getter rather than in the underlying representation: q.as_quat(self, single_cover=True)
Fixed q.inv() so that it properly inverts in the double coverage space

Would appreciate a look from a maintainer on the failing musllinux_amd64_test, it's erroring out on the git user and email but I have that on both my commits.
Committer identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'root@cirrus-ci-task-6443399327252480.(none)')

> git log
commit b4e417e897b771178629e96e9634aedbc70faaba (HEAD -> unique_quats, origin/unique_quats)
Author: Scott Shambaugh <14363975+scottshambaugh@users.noreply.github.com>
Date:   Thu Nov 3 09:42:38 2022 -0600

    Add a Rotation.as_quat(single_cover) flag to map the double coverage of rotation space down to a single cover

commit 605d089d32bd1b276942cab9a1f1f8066cfdbd6b
Author: Scott Shambaugh <14363975+scottshambaugh@users.noreply.github.com>
Date:   Sat Dec 3 15:22:36 2022 -0700

    Fix quaternion inversion to work on double cover of rotation space

I'd like to revisit the discussion around downselecting to the "positive" qx, qy, qz values when q[3] == 0. In the context of this discussion around the __pow__ operator, that implementation of raising to a power will wipe out the "extra" data at the 180 deg pole about which direction that was approached from. Plus since the underlying representation is still accessible with as_quat(single_cover=False), I think the proposition of having that single cover be a completely unique 2:1 mapping with no double points (really several lines on the 4D sphere rather than points) is very valuable.

@scottshambaugh sorry I'm not sure exactly what the point you're raising is. I get that your implementation of __pow__ "wipes out the extra data". I get that you're a fan of being able to get a single cover representation on demand. Are these just comments or is there a request for someone else's input?

@alexander-soare since we've landed on not mapping to a single-cover under the hood, the purpose of this PR now is simply to have an easy way to represent it to the user. The value there like you talked about above being a) easier interoperability with other tools/eqns that assume a positive w term, and b) having unique values for a particular orientation when you look at data/telemetry streams.
To (b), the PR as currently written still has double points when the w term is 0, and the below logic is necessary to get a fully unique single-cover:
if ((q[3] < 0)
    or (q[3] == 0 and q[0] < 0)
    or (q[3] == 0 and q[0] == 0 and q[1] < 0)
    or (q[3] == 0 and q[0] == 0 and q[1] == 0 and q[2] < 0)):
    q = -q

So basically looking for feedback on if you/others think that complete mapping is a useful thing to add back in, or if I'm being overly pedantic here and it's good as-is.

Yes, to the extent that you are moving to an on-request canonicalization, I think that the full list of rules is a perfectly good idea.
Bikeshed: possibly canonical=True might be a better spelling to convey the intent. There are other ways to represent rotations as unit quaternions that are single covers, but we don't mean those conventions.

@scottshambaugh yeah I don't think you're being overly pedantic with the full set of conditions. (b) wouldn't be satisfied without that because someone might want to plot the other components (in fact my colleague did just that). Nice one.

Thanks to both of you! I've updated that logic and the bikeshed is now painted a canonical color.

I'm not sure if I fully understand this PR and I'm not sure if the Rotation class is meant to be used like this, but I wanted to mention that there might be a different way someone might want to "canonicalize":
What if you have a sequence of Rotation objects and you want to create interpolated values between the given rotations?
If the canonicalization always chooses a non-negative w, there might be some angles between the given rotations that are larger than 180 degrees.
In this case, I would want to use a different form of "canonicalization": I would like to check if neighboring rotations are larger than 180 degrees, and negate the second quaternion if that's the case.
This would mean that some w value might become negative, but it would also mean that none of the involved rotations are more than 180 degrees.
I've implemented that form of "canonicalization" in https://splines.readthedocs.io/en/latest/python-module/splines.quaternion.html#splines.quaternion.canonicalized
I'm not sure what I want to say with this, maybe just that I've used the word "canonicalize" for something different, for whatever that's worth ...

Your function isn't canonicalizing each rotation/quaternion. Rather, it is canonicalizing a sequence of rotations/quaternions with a rule that happens to be good for one particular application of sequences of rotations. That's great in your context where a sequence is quaternions is a real sequence (the object or frame or whatever is going through those states in order). In our case, even a Rotation object that has many elements (i.e. holds an array of quaternions) is a more general purpose object representing a collection of rotations, not necessarily a sequence of rotations.
Given that the intended use case for as_quat(canonical=True) is to provide a global rule to assign a unique quaternion to each rotation, I think that's fine use of the word. If anyone has another global rule that has actual prior art and some reasoning behind it, that'd be great to hear about. If we did want to implement a function that applied a similar rule as your canonicalized() function, we'd name it something else to indicate the extra semantics that it is overlaying on the Rotation object.
Does that make sense?

Yeah, that makes sense, thanks @rkern!

I've done a final pass over the implementation, tests, and docs, and think this is ready for merge.

@rkern, any further concerns here?

I have a vague preference for leaving the default to be False so that as_quat() continues to behave backwards compatibly, but I'm not too fussed about it.

Based on the codeowners file, this should have a look from @tylerjereddy and/or @peterbell10, unless they want to delegate?
@scottshambaugh also has a bunch of other Rotation-related PRs that are basically ready.

@rkern my thought after this gets merged in is to open up another PR that documents and tests the limited subset of operations that preserve the double cover properties. I think backwards-compatible results here are lost anyway since inv() was not previously implemented correctly wrt that double cover.

There are two distinct changes here: the behavior of inv() and the behavior of as_quat(). I'm talking about as_quat(). I have a mild preference that it not change when given no argument; it should continue to return exactly the quaternion that's stored unless if the user asked for canonicalization.

Sure, I don't care too strongly about it. Changed the default to canonical=False

@rkern, are you comfortable with merging this and/or #17460? (seeing that @tylerjereddy and @peterbell10 haven't had time to respond yet)
Scott has several outstanding PRs in this space (also #17452 & #17542), and the release window for 1.11 is closing fast.

Thanks @tylerjereddy! From my perspective the other 3 Rotation PRs I have open are all ready to roll, but they need review and approval. I'm not sure who to poke on that, I've sent two requests for review out to the dev mailing list and don't want to spam it further.

authors: alexander-soare h-vetinari mgeier rkern scottshambaugh


text:
get_all analyses

Returns all analyses given a group/project/session/acquisition
It will return all analyses within the hierarchy

example, /sessions/{SessionId}/analyses will return any analyses that have that session as a parent or any of the acquisitions in that session as a parent

Must have permission to the top level container and (only applies when making calls at the group level) will only return analyses for which the user has permission to its parent

Review Checklist

Tests were added to cover all code changes
Documentation was added / updated
Code and tests follow standards in CONTRIBUTING.md

Codecov Report

Merging #1017 into master will increase coverage by 0.07%.
The diff coverage is 95.23%.

@@            Coverage Diff             @@
##           master    #1017      +/-   ##
==========================================
+ Coverage    90.6%   90.68%   +0.07%
==========================================
  Files          50       50
  Lines        6762     6783      +21
==========================================
+ Hits         6127     6151      +24
+ Misses        635      632       -3

Awaiting final review by @nagem

LGTM, this came together really nicely - thanks @hkethi002 !

authors: codecov-io hkethi002 nagem ryansanford

repo: scitran/core

license: MIT


text:
Running javascript with WebDriver's Execute

The page.RunScript is fine to run inline javascript but the WebDriver's Execute is a must if I want to run heavier javascript like injecting jQuery and other libs.
I already forked agouti to expose page.Execute (niclupien/agouti) but I feel there's a reason you didn't exposed it before so I'm opening this issue to start the discussion.
Another alternative would be a new method InjectScript that could take the name of file to inject in the page (ex: page.injectScript("jquery.js"))

Hi @niclupien, thanks for opening an issue!
Without any arguments, page.RunScript just wraps whatever JavaScript you provide it with in a closure. This is generally a safe thing to do with arbitrary JavaScript. Unless your script uses the var keyword to declare a variable in the global scope and then expects JavaScript outside of the script to be able to access that variable, there should be no difference. For instance, you should be able to include jQuery with:
script, _ := ioutil.ReadFile("jQuery.js")
page.RunScript(string(script), nil, nil)

It's definitely possible that I've missed something. Does this not work like I expect it to?

@niclupien Did this work for you, or am I missing something?

Hi @niclupien, I'd like to close this issue if I answered your question. Are you still having trouble importing scripts?

I've switched on another project for now, your solution looks good, I'll
try it when I get back to it. This issue can be closed.
On Wed, Feb 18, 2015 at 4:00 PM, Stephen Levine notifications@github.com
wrote:

Hi @niclupien https://github.com/niclupien, I'd like to close this
issue if I answered your question. Are you still having trouble importing
scripts?
—
Reply to this email directly or view it on GitHub
https://github.com/sclevine/agouti/issues/37#issuecomment-74946851.

Thanks! Feel free to re-open this if you run into any issues.

authors: niclupien sclevine

repo: sclevine/agouti

license: mit

license_source: bigquery


text:
Unable to get v0.0.8 from maven

Thanks for new release v0.0.8 but still can't synced from maven
@scottyab Could you please check if it's been updated? Thank you!

Thanks for flaggin, I having a few issues with the upload.

Hi @scottyab just following up when can v0.0.8 be placed in maven.

It's been year or so since the last update and struggling with the upload to nexus repo. The uploadArchives seems to have disappeared or rather it's not actually doing the upload. I'm not sure what's changed in this regard please bear with I have limited time to fault find.

Confirmed https://search.maven.org/artifact/com.scottyab/rootbeer-lib/0.0.8/aar

authors: cyberrob mitchryanjusay scottyab

repo: scottyab/rootbeer


text:
Add detected language to return value of search_dates()

E.g.:
In:  search_dates('You can reach me on the phone')
Out: [('me', datetime.datetime(2018, 3, 14, 0, 0), 'fr')]

Referred in: https://github.com/scrapinghub/dateparser/issues/394

I believe this change can be considered backward incompatible. If I am expecting a list of 2-item tuples and suddenly I receive a 3-item tuple some code may break. A code like this will start failing:
for term, date in search_dates('You can reach me on the phone'):
     # do something

What do you think to include an extra parameter saying that we want to receive the language detected in the function return?
In:  search_dates('You can reach me on the phone', add_detected_language=True)
Out: [('me', datetime.datetime(2018, 3, 14, 0, 0), 'fr')]

In:  search_dates('You can reach me on the phone', add_detected_language=False)
Out: [('me', datetime.datetime(2018, 3, 14, 0, 0)]

Question, instead of making an incompatible change here, why not change search_dates to return an object that can potentially be modified in the future? Something like:
class SearchDateResult(object):
    __slots__ = ('text', 'date', 'language')
    def __init__(self, text, date, language=None):
        self.text = text
        self.date = date
        self.language = language
    def __getitem__(self, index_or_slice):
        return (self.text, self.date)[index_or_slice]

Would allow attribute accession on the results and maintain the current behavior.

@cjgibson I believe that your suggestion is incompatible and can break existing code.
Actually search_dates returns a tuple of two items. Your new object doesn't act like a tuple, so every code that relies that the return of this function is a tuple will break. The change made maintain this behavior and the new information is added only when explicitly asked.

for ver in 2.7 3.3 3.4 3.5 3.6 3.7
do
  command="python$ver"
  $command -c """
from datetime import datetime
from sys import version_info

if version_info.major == 2:
    pass
else:
    basestring = str

class SearchDateResult(object):
    __slots__ = ('text', 'date', 'language')
    def __init__(self, text, date, language=None):
        self.text = text
        self.date = date
        self.language = language
    def __getitem__(self, index_or_slice):
        return (self.text, self.date)[index_or_slice]

def dummy_search_dates(text):
    return [SearchDateResult(word, datetime.now(), 'en') for word in text.split()]

for term, date in dummy_search_dates('example of tuple expansion'):
    assert isinstance(term, basestring)
    assert isinstance(date, datetime)

for result in dummy_search_dates('example of tuple indexing'):
    assert isinstance(result.text, basestring)
    assert isinstance(result.date, datetime)
    assert isinstance(result[0], basestring)
    assert isinstance(result[1], datetime)
    assert result[0] == result.text
    assert result[1] == result.date
"""
done

Appears to work in all versions of Python I have locally - what tuple interface is missing @rennerocha ?

authors: asadurski cjgibson rennerocha

repo: scrapinghub/dateparser


text:
Update mikrotik_routeros.py and fix typo in readme

Description

Fix case when username contains "@" sign (i.e user@domain), and prompt looks like
[user@domain@mkt-01] >

Fix typo in readme

Type of change

[x] Bug fix (non-breaking change which fixes an issue)

thanks for repointing this! lgtm!

oh fyi -- I nuked the readme stuff as that no longer exists in develop, but I did fix that typo in the new docs as well. you can see that fix here: https://scrapli.github.io/scrapli_community/user_guide/project_details/#supported-platforms
thanks again!


text:
Lint

https://github.com/scratch-contributors/scratch-contributors.github.io/issues/5#issuecomment-851955411

https://github.com/marketplace/axe-linter?


text:
should we use black for code formatting?

@luiztauffer Should we use "black" for code formatting?

Personally, I dislike it, and never felt advantage using it. Some of the enforced formatting becomes actually less legible, imo, and most of good practices for legibility are not difficult to be followed without it anyway. At the end of the day, it's a matter of taste I guess. Have you tried it?

I haven't tried it. I would like to conform to recommended formatting though. I just installed pylint and it picks up a lot of issues. Some I want to ignore, but others are helpful - like not allowing exceptions that are too general. Trailing whitespace is something I'd like to get rid of on lines. And should we enforce newline at the end of a file? Maybe we could have a script that does that. IDK, what do you think?

there are a couple of strict rules that actually makes the code look quite bad, and one simply can't control that after adopting black.
e.g., something nicely formatted like this:
def myfunc(
    arg1: type = default_value,
    arg2: type = default_value,
    arg3: type = default_value,
):
    ...

could be enforced by black into a monster like this, based only on the number of characters inside the myfunc definition:
def myfunc(
    arg1: type = default_value, arg2: type = default_value, arg3: type = default_value):
    ...

a good linter should be enough imo

authors: luiztauffer magland

repo: scratchrealm/protocaas


text:
Error E95: buffer with this name already exists when restoring PHP debug session

Issue is raised after stopping debugging PHP classes using BDGp or BDGPavim. When closing the debug session, the previous edit session is restored (I assume) and this error is raised. The exact message is:
Error detected while processing /home/...../dbgpavim_saved_session.1340:
line 205:
E95: Buffer with this name already exists

I don't see how this is a NERDTree issue. Closing.

authors: PhilRunninger donnut

repo: scrooloose/nerdtree

license: WTFPL


text:
Remove external IDs

Hi,
I cannot figure out how to remove external IDs from a user. I only have one external ID set, so the goal is to clear them all. I have tried things like:
Update-GSUser someone -ExternalIds $null

Update-GSUser someone -ExternalIds @()

but neither do the trick.

Hey @cooper - What's odd is that passing $null returns the user object without the ExternalId's present, but pulling the user again does indeed show the ExternalId's still there. Filing as a bug and checking out what I can do to get this working as expected!

Alright, Google seemed to want to take the complicated route on this one... Here's how you would need to clear External ID's with the current model (using type 'login_id' for this example); you would need to clear each external ID one by one 😞 :
Update-GSUser -User someone -ExternalIds (Add-GSUserExternalId -Type login_id -Value $null) -Verbose

authors: cooper scrthq

repo: scrthq/PSGSuite


text:
Investigate ws-client possibilities in IoT

A no_std, efficient ws-client seems nearly impossible. But maybe that's not even necessary, as internet connected hardware devices most likely have their own websocket implementations / libraries / necessities. Like intel sgx does.
The following needs to be done in this issue:

[ ] investigate which devices could serve as an example (e.g. often used, commonly known) but feasible time-wise
[ ] Create an examples(s)
[ ] Document the example use cases somewhere visible (Wiki? Readme?)

Connected to https://github.com/scs/substrate-api-client/issues/279

Just checked: Arduino Uno is a no_std embedded device, which seems to be already commonly used in the Rust world. There are several guides for Rust on Arduino.
(E.g. https://dev.to/creativcoder/how-to-run-rust-on-arduino-uno-40c0)
I think that could be a good example for this use-case.
What do you think @masapr ?

Two other embedded devices that support rust / have libraries developed to support rust and do not support std

STM32
Nordic nrf9160

repo: scs/substrate-api-client


text:
Time elapse

Can you add a time elapse option or param between each request. On my work envioroment, it seems not stable.  I think it may be a good enhancement ：)

Got it. :P

Added in new pull requests.
#8

authors: cleardusk imcmy


text:
minor bug? non-productive contigs appearing as productive and order of contigs

Description of the bug
Hi @grst,
Was testing a couple of combinations to see how scirpy deals with certain combinations of IR V/D/J/C pairings that could come up with the new dandelion update and i came across a possible issue and was wondering if this was intended.
The code show show that in the last two cells, although only the IGHM contig was the productive contig, (1) you can still see the IGHA contig being populated in IR_VDJ_2_* and (2) IR_VDJ_2_productive is shown as True. I suppose (1) is as intended but (2) seems like a bug?
secondly, if I make the 'duplicate_count' column for the non-productive to be higher than the productive column, then it switches the entries in IR_VDJ_1_* and IR_VDJ_2_* and now IR_VDJ_1 is the non-productive IGHA contig. Both IR_VDJ_*_productive columns remain to be True.
I'm using read_airr here, so it's going to reflect how it looks like in dandelion if for whatever reason dandelion messes up the row/cell/contig order.
Minimal reproducible example
import scirpy as ir
import pandas as pd

def airr_data():
    """example airr"""
    airr = pd.DataFrame(
        [
            # IGHM+IGHD+IGHL+IGHAnp
            [
                "IGHM+IGHD+IGHL+IGHAnp_contig_1",
                "",
                "T",
                "T",
                "IGHV1",
                "IGHD1-2",
                "IGHJ2-1",
                "TGGGGAGGAGTCAGTCCCAACCAGGACACGGCCTGGACATGAGGGTCCCTGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCTCAGGTGCCAGATGTGACATCCAGATGACCCAGTCTCCATCCTCCCTGTCTGCATCTGTGGGAGACAGAGTCACCATCACTTGCCAGGCGACACAAGACATTAACAATTATGTAAATTGGTATCAGCAGAAACCAGGGAAAGCCCCTAAACTCCTGATCTACGATGCATTGAATTTAGAAATAGGGGTCCCATCAAGATTCAGTGGAAGAGGGTCTGGGACAGTCTTTATTCTCACCATCAGCAGCCTGCAGCCTGAAGATGTTGCAACATACTACTGTCAACAATATGACGAACTTCCCGTCACTTTCGGCGGAGGGACCAATGTGGAAATGAGACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHD+IGHL+IGHAnp",
                "IGHM",
                1,
                1,
            ],
            [
                "IGHM+IGHD+IGHL+IGHAnp_contig_2",
                "",
                "T",
                "T",
                "IGLV1",
                "",
                "IGLJ2-1",
                "AGGAGTCAGACCCTGTCAGGACACAGCATAGACATGAGGGTCCCCGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCCCAGGTGCCAGATGTGCCATCCGGATGACCCAGTCTCCATCCTCATTCTCTGCATCTACAGGAGACAGAGTCACCATCACTTGTCGGGCGAGTCAGGGTATTAGCAGTTATTTAGCCTGGTATCAGCAAAAACCAGGGAAAGCCCCTAAGCTCCTGATCTATGCTGCATCCACTTTGCAAAGTGGGGTCCCATCAAGGTTCAGCGGCAGTGGATCTGGGACAGATTTCACTCTCACCATCAGCTGCCTGCAGTCTGAAGATTTTGCAACTTATTACTGTCAACAGTATTATAGTTACCCTCGGACGTTCGGCCAAGGGACCAAGGTGGAAATCAAACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGL",
                1,
                1,
                1,
                "IGHM+IGHD+IGHL+IGHAnp",
                "IGLC",
                1,
                1,
            ],
            [
                "IGHM+IGHD+IGHL+IGHAnp_contig_3",
                "",
                "T",
                "F",
                "IGHV1",
                "IGHD1-2",
                "IGHJ2-1",
                "",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHD+IGHL+IGHAnp",
                "IGHD",
                1,
                10,
            ],
            [
                "IGHM+IGHD+IGHL+IGHAnp_contig_4",
                "",
                "T",
                "F",
                "IGHV1",
                "IGHD1",
                "IGHJ2-1",
                "TGGGGAGGAGTCAGTCCCAACCAGGACACGGCCTGGACATGAGGGTCCCTGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCTCAGGTGCCAGATGTGACATCCAGATGACCCAGTCTCCATCCTCCCTGTCTGCATCTGTGGGAGACAGAGTCACCATCACTTGCCAGGCGACACAAGACATTAACAATTATGTAAATTGGTATCAGCAGAAACCAGGGAAAGCCCCTAAACTCCTGATCTACGATGCATTGAATTTAGAAATAGGGGTCCCATCAAGATTCAGTGGAAGAGGGTCTGGGACAGTCTTTATTCTCACCATCAGCAGCCTGCAGCCTGAAGATGTTGCAACATACTACTGTCAACAATATGACGAACTTCCCGTCACTTTCGGCGGAGGGACCAATGTGGAAATGAGACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHD+IGHL+IGHAnp",
                "IGHA",
                1,
                1,
            ],
            # IGHM+IGHAnp
            [
                "IGHM+IGHAnp_contig_1",
                "",
                "T",
                "T",
                "IGHV1",
                "IGHD1-2",
                "IGHJ2-1",
                "TGGGGAGGAGTCAGTCCCAACCAGGACACGGCCTGGACATGAGGGTCCCTGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCTCAGGTGCCAGATGTGACATCCAGATGACCCAGTCTCCATCCTCCCTGTCTGCATCTGTGGGAGACAGAGTCACCATCACTTGCCAGGCGACACAAGACATTAACAATTATGTAAATTGGTATCAGCAGAAACCAGGGAAAGCCCCTAAACTCCTGATCTACGATGCATTGAATTTAGAAATAGGGGTCCCATCAAGATTCAGTGGAAGAGGGTCTGGGACAGTCTTTATTCTCACCATCAGCAGCCTGCAGCCTGAAGATGTTGCAACATACTACTGTCAACAATATGACGAACTTCCCGTCACTTTCGGCGGAGGGACCAATGTGGAAATGAGACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHAnp",
                "IGHM",
                1,
                1,
            ],
            [
                "IGHM+IGHAnp_contig_2",
                "",
                "T",
                "F",
                "IGHV1",
                "IGHD1",
                "IGHJ-1",
                "AGGAGTCAGACCCTGTCAGGACACAGCATAGACATGAGGGTCCCCGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCCCAGGTGCCAGATGTGCCATCCGGATGACCCAGTCTCCATCCTCATTCTCTGCATCTACAGGAGACAGAGTCACCATCACTTGTCGGGCGAGTCAGGGTATTAGCAGTTATTTAGCCTGGTATCAGCAAAAACCAGGGAAAGCCCCTAAGCTCCTGATCTATGCTGCATCCACTTTGCAAAGTGGGGTCCCATCAAGGTTCAGCGGCAGTGGATCTGGGACAGATTTCACTCTCACCATCAGCTGCCTGCAGTCTGAAGATTTTGCAACTTATTACTGTCAACAGTATTATAGTTACCCTCGGACGTTCGGCCAAGGGACCAAGGTGGAAATCAAACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHAnp",
                "IGHA1",
                1,
                1,
            ],
            # IGHM+IGHAnp2 np has highest umi
            [
                "IGHM+IGHAnp2_contig_1",
                "",
                "T",
                "T",
                "IGHV1",
                "IGHD1-2",
                "IGHJ2-1",
                "TGGGGAGGAGTCAGTCCCAACCAGGACACGGCCTGGACATGAGGGTCCCTGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCTCAGGTGCCAGATGTGACATCCAGATGACCCAGTCTCCATCCTCCCTGTCTGCATCTGTGGGAGACAGAGTCACCATCACTTGCCAGGCGACACAAGACATTAACAATTATGTAAATTGGTATCAGCAGAAACCAGGGAAAGCCCCTAAACTCCTGATCTACGATGCATTGAATTTAGAAATAGGGGTCCCATCAAGATTCAGTGGAAGAGGGTCTGGGACAGTCTTTATTCTCACCATCAGCAGCCTGCAGCCTGAAGATGTTGCAACATACTACTGTCAACAATATGACGAACTTCCCGTCACTTTCGGCGGAGGGACCAATGTGGAAATGAGACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHAnp2",
                "IGHM",
                1,
                1,
            ],
            [
                "IGHM+IGHAnp2_contig_2",
                "",
                "T",
                "F",
                "IGHV1",
                "IGHD1",
                "IGHJ-1",
                "AGGAGTCAGACCCTGTCAGGACACAGCATAGACATGAGGGTCCCCGCTCAGCTCCTGGGGCTCCTGCTGCTCTGGCTCCCAGGTGCCAGATGTGCCATCCGGATGACCCAGTCTCCATCCTCATTCTCTGCATCTACAGGAGACAGAGTCACCATCACTTGTCGGGCGAGTCAGGGTATTAGCAGTTATTTAGCCTGGTATCAGCAAAAACCAGGGAAAGCCCCTAAGCTCCTGATCTATGCTGCATCCACTTTGCAAAGTGGGGTCCCATCAAGGTTCAGCGGCAGTGGATCTGGGACAGATTTCACTCTCACCATCAGCTGCCTGCAGTCTGAAGATTTTGCAACTTATTACTGTCAACAGTATTATAGTTACCCTCGGACGTTCGGCCAAGGGACCAAGGTGGAAATCAAACGAACTGTGGCTGCACCATCTGTCTTCATCTTCCCGCCATCTGATGAGCAGTTGAAATCTGGAACTGCCTCTGTTGTGTGCCTGCTGAATAACTTCTATCCCAGAGAGGCCAAAGTACAGTGGAAGGTGGATAACGC",
                "",
                "CATTGCTCTCACTTCCTAGACCCATACTCAGCTCTCTGTGTT",
                "HCSHFLDPYSALCV",
                "",
                "",
                "",
                "F",
                "T",
                "IGH",
                1,
                1,
                1,
                "IGHM+IGHAnp2",
                "IGHA1",
                1,
                100,
            ],
        ],
        columns=[
            "sequence_id",
            "sequence",
            "rev_comp",
            "productive",
            "v_call",
            "d_call",
            "j_call",
            "sequence_alignment",
            "germline_alignment",
            "junction",
            "junction_aa",
            "v_cigar",
            "d_cigar",
            "j_cigar",
            "stop_codon",
            "vj_in_frame",
            "locus",
            "junction_length",
            "np1_length",
            "np2_length",
            "cell_id",
            "c_call",
            "consensus_count",
            "duplicate_count",
        ],
    )
    return airr

data = airr_data()

irdata = ir.io.read_airr(data)
ir.tl.chain_qc(irdata)
irdata.obs

The error message produced by the code above
No response
Version information
scirpy==0.10.2.dev28+g2c5b99e
scanpy==1.9.1
anndata==0.8.0
umap==0.5.3
numpy==1.21.6
scipy==1.8.1
pandas==1.4.2
scikit-learn==1.1.1
statsmodels==0.13.2
python-igraph==0.9.11
pynndescent==0.5.7

hmm but the airr schema says that boolean columns should be encoded as str T/F according to https://github.com/airr-community/airr-standards/blob/master/docs/datarep/rearrangements.rst

Boolean values
Boolean values must be encoded as T for true and F for false.

yes amazing! the fix works!

repo: scverse/scirpy


text:
Test with Go 1.12 and 1.13

Support last two versions of Go, similar to gocql. We also need to
remove the flag.Parse() call because of the change in Go 1.13:

Testing flags are now registered in the new Init function, which is
invoked by the generated main function for the test. As a result,
testing flags are now only registered when running a test binary, and
packages that call flag.Parse during package initialization may cause
tests to fail.

@martin-sucha I think that Megacheck needs a bump too?

Fixed with #122

authors: dahankzter martin-sucha mmatczuk

repo: scylladb/gocqlx


text:
2019.1 scylla-per-server-disk-i-o dashboard: All Memtable Flush I/O and Streaming Reads/Writes graphs are empty

2019.1 scylla-per-server-disk-i-o dashboard.
All Memtable Flush I/O and Streaming Reads/Writes graphs are empty.
For example, Streaming reads class name should be changed from "streaming" to "streaming_read"

Seems like a duplication of #578

Seems like a duplication of #578

Does the 578  change class label name?

#578 was a result of missing backport, the metric name and label were changed on seastar-master and were updated accordingly on 2019.1 dashboards.
I know that the changes in seastar were backported, so it should be resloved

It’s not a duplicate!
In #578 the source was missing backport in seastar.
Here the query is wrong, the class name and the metric has mistakes.

In #578 the missing backport in seastar, made it look like there is a problem with the metric name and labels

Right, that’s why it’s not a duplicate.
This bug is with a later Scylla release that contains the backport.
And now the metric is wrong.

This is the example of correct query:
sum(irate(scylla_io_queue_total_bytes{class="streaming_read", instance=~"[[node]]",cluster=~"$cluster|$^", dc=~"$dc", shard=~"[[shard]]"}[30s])) by ([[by]])

instead of:
sum(irate(scylla_io_queue_read_total_bytes{class="streaming", instance=~"[[node]]",cluster=~"$cluster|$^", dc=~"$dc", shard=~"[[shard]]"}[30s])) by ([[by]])

ok, I see that there are some instances of the old metric format
On Mon, Apr 22, 2019 at 11:15 AM juliayakovlev notifications@github.com
wrote:

This is the example of correct query:
sum(irate(scylla_io_queue_total_bytes{class="streaming_read", instance=~"[[node]]",cluster=~"$cluster|$^", dc=~"$dc", shard=~"[[shard]]"}[30s])) by ([[by]])
instead of:
sum(irate(scylla_io_queue_read_total_bytes{class="streaming", instance=~"[[node]]",cluster=~"$cluster|$^", dc=~"$dc", shard=~"[[shard]]"}[30s])) by ([[by]])
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla-grafana-monitoring/issues/580#issuecomment-485359736,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAQFDP5QXVCNOX7NASQFJULPRVXZTANCNFSM4HHLVGLA
.

authors: amnonh juliayakovlev roydahan

repo: scylladb/scylla-grafana-monitoring


text:
Issue 126 turn key training demo

This PR implements a turn-key training demo-script using hydra config management. The goal is to reduce the entry barrier for the developer. Hydra enables config composition and convenient overwrite from command line which are demonstrated below. It is a widely used config management tool in ML community. The proposed workflow for the developer looks as below.
The developer

clones the repo
creates the environment
downloads the input.wav and output.wav from provided links. (This step can be automated as well)
run demo by calling below from the repo root

python  nam/train/demo.py --config-path=bin/train/inputs/data --config-name=demo.yaml common.x_path=path/to/input.wav common.y_path=path/to/output.wav common.train_path=path/to/training_output"

Note that, hydra allows conveniently overwriting config parameters.

Hi! Thanks for the PR.
What's the difference between this and using bin/train/main.py?

demo.yaml seems to copy bin/train/inputs/data/single_pair.json
This doesn't seem to expose for the demo how the model is configured (cf e.g. bin/train/inputs/models/wavenet.json or other model configs in there) nor the learning algorithm.

Do you think that we might achieve effectively the same result as this PR by sprucing up the README? (I also wouldn't mind not picking up another dependency with Hydra. What's the sell there?)
I think that, at a high level, I'm not convinced that this repo should be going all-in on the wrappers in nam.train, though that's obviously what easy_colab.ipynb and the GUI trainer do...basically if a developer wants to get into this, having something that doesn't conflate the UX with the moving parts underneath may be more helpful?
But obviously I have a PoV bias, so I'm interested to hear if you've got any feedback on these thoughts.

@sdatkinson You are right this PR achieves the same functionality as bin/train/main.py. I have missed that. There are different entry points to the training, from colab, from command-line ...etc. I was pursuing the colab's call path. In that sense, yes, we can achieve similar affect if we improve the README, plus improve config management (more details below). I would also suggest populating all training functionality into single python file and calling relevant functions from the same colab, command line or gui. The idea is reducing the call pathways as much as possible to improve maintainability.
About config management and hydra. In this example I only illustrate the command-line override convenience, where one can override a config parameter without need to change the config file. In this particular example, user can run the training without touching the config file. More powerful part of Hydra is composing configs. It allows you define default behavior and change details as needed. For example you can define all training parameters like optimizer, learning rate, batch size ...etc. in a config file but may prefer to change only the batch size for a particular model. Using hydra you can create a config file and override only the parts of the default config, the batch size in this case. It allows to switch between configs conveniently as well. The idea here is again reducing config duplication, or sources of truth as much as possible to improve maintainability. See here for a composition example. Hydra works with yaml files that is why I converted single_pair.json to demo.yaml.
I hope this clarifies some of the questions above.

Closing this in the light of our discussion and follow up PR.

authors: cdicle sdatkinson

repo: sdatkinson/neural-amp-modeler


text:
RSS feed not returning result using jquery.rss.min.js

We are using the js file from https://github.com/sdepold/jquery-rss/blob/master/dist/jquery.rss.min.js to get the result from https://www.smartbrief.com/servlet/rss?b=ACC . For last 1 week, it is not returning any result, earlier it worked fine. Putting the RSS feed url directly in browser shows result.
Can it be due to any issue in the jquery file? Please help

How can I debug using FeedrApp?

works for me. can you try again? the feedrapp server was down for some days last week due to some misconfiguration.

Yes, it's working now, thank you for your prompt action.

authors: sdepold suparna-b

repo: sdepold/jquery-rss


text:
When I run poetry build I receive this error:   [AttributeError]   '_Printer' object has no attribute 'id'

Hi :)
I am trying to build a package using poetry and I am receiving and error, the steps to reproduce it are these:
poetry new example
cd example
poetry build
and it returns [AttributeError]   '_Printer' object has no attribute 'id'
Do you know how to fix this error?
Is there any documentarion about the pyproject.toml and its options?
Could be great if the default pyproject.toml includes more default options

I found the issue, seems like the license field is required I just added it to the pyproject.toml file and it worked

Could you please either add this field as optional or add it by default to the pyproject.toml file?

@oca159 That's strange. If it is indeed the case then it's a bug since the license field is not mandatory and you should be able to build a project without it.
I will investigate.

1

For me this had to do with my environment.  For some reason my environment + pycharm doesn't put git.exe in my path.
>poetry build -vvv
Building qwatch (0.2.0)
 - Building sdist

  [FileNotFoundError]
  [WinError 2] The system cannot find the file specified

Exception trace:
 c:\users\rgilmore\python36\lib\site-packages\poetry\_vendor\cleo\application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 c:\users\rgilmore\python36\lib\site-packages\poetry\console\application.py in do_run() at line 71
   return super().do_run(i, o)
 c:\users\rgilmore\python36\lib\site-packages\poetry\_vendor\cleo\application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 c:\users\rgilmore\python36\lib\site-packages\poetry\console\commands\command.py in run() at line 22
   return super(BaseCommand, self).run(i, o)
 c:\users\rgilmore\python36\lib\site-packages\poetry\_vendor\cleo\commands\base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 c:\users\rgilmore\python36\lib\site-packages\poetry\_vendor\cleo\commands\command.py in execute() at line 107
   return self.handle()
 c:\users\rgilmore\python36\lib\site-packages\poetry\console\commands\build.py in handle() at line 26
   builder.build(fmt)
 c:\users\rgilmore\python36\lib\site-packages\poetry\masonry\builder.py in build() at line 25
   return builder.build()
 c:\users\rgilmore\python36\lib\site-packages\poetry\masonry\builders\complete.py in build() at line 21
   sdist_file = sdist_builder.build()
 c:\users\rgilmore\python36\lib\site-packages\poetry\masonry\builders\sdist.py in build() at line 74
   files_to_add = self.find_files_to_add(exclude_build=False)
 c:\users\rgilmore\python36\lib\site-packages\poetry\masonry\builders\builder.py in find_files_to_add() at line 67
   excluded = self.find_excluded_files()
 c:\users\rgilmore\python36\lib\site-packages\poetry\masonry\builders\builder.py in find_excluded_files() at line 44
   vcs = get_vcs(self._path)
 c:\users\rgilmore\python36\lib\site-packages\poetry\vcs\__init__.py in get_vcs() at line 11
   return Git(p)
 c:\users\rgilmore\python36\lib\site-packages\poetry\vcs\git.py in __init__() at line 30
   self._config = GitConfig()
 c:\users\rgilmore\python36\lib\site-packages\poetry\vcs\git.py in __init__() at line 10
   stderr=subprocess.STDOUT
 c:\users\rgilmore\python36\lib\subprocess.py in check_output() at line 336
   **kwargs).stdout
 c:\users\rgilmore\python36\lib\subprocess.py in run() at line 403
   with Popen(*popenargs, **kwargs) as process:
 c:\users\rgilmore\python36\lib\subprocess.py in __init__() at line 707
   restore_signals, start_new_session)
 c:\users\rgilmore\python36\lib\subprocess.py in _execute_child() at line 990
   startupinfo)

build [-f|--format FORMAT]

After following this guide:
https://stackoverflow.com/a/26180050/7351746
...And issuing the same command, I successfully built my project.

authors: grabear oca159 sdispater

repo: sdispater/poetry


text:
Corrected an issue with double Spring contexts.

When initializing DXA, two Spring context is actually created, one through the context listener (reading all META-INF/spring-context.xml files) and through the SpringConfiguration class (using @Configuration annotation). I've corrected it so only one context is created and shared between the context listener and the MVC dispatcher servlet.

Closing this PR in favour of the pull request #10

repo: sdl/dxa-modules


text:
Bengali-BN (বাংলা) translation

hi @sdras & maintainers, if you could merge this translation, it'd be great. The translation, being a not-so-mainstream script, seems working as expected. It's live here if you wanna have a look. Here's a screenshot as well.

Yay! Thank you!

authors: kmonsoor sdras

repo: sdras/object-explorer


text:
Cannot read SSID password if there's a space between the SSID's name

If I run sudo wifi-password I get

ERROR: Could not find password

This is because my AP's name has a space inside (e.g: John DoeWifi)
If I create a new AP and name it JohnDoeWifi, I successfully get the result

password123

The program doesn't take into consideration that there may be an SSID with space inside the name. Only SSIDs with no space are read.

Hi @engineer22,
Thank you for bringing up this bug.
On line 65 if you replace that line with the code below, does it fix the issue?
        password = run_command(f"cat '/etc/NetworkManager/system-connections/{ssid}.nmconnection' | grep psk=")

Hi @engineer22,
Thank you for bringing up this bug.
On line 65 if you replace that line with the code below, does it fix the issue?
        password = run_command(f"cat '/etc/NetworkManager/system-connections/{ssid}.nmconnection' | grep psk=")

Hello @sdushantha,
I will test in couple hours and get back to you. Thank you.

Hello @sdushantha,
I will test in couple hours and get back to you. Thank you.

I had the same problem and your line 65 patch (adding 's) fixed it.

I had the same problem and your line 65 patch (adding 's) fixed it.

I think it would be a good idea to add this to the main branch

I think it would be a good idea to add this to the main branch

Fixed.

Fixed.

authors: barak engineer22 sdushantha theRealStyro

repo: sdushantha/wifi-password


text:
Pynguin encounters error when running against webpy

We are evaluating our project pyster(also a Python test generator) with Pynguin and found that Pynguin throws the following exception when running against webpy:
pynguin --algorithm WSPY --project_path public_repo/webpy --module_name web.db --output_path ./public_repo/webpy --budget 600
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/networkx/classes/digraph.py", line 805, in successors
    return iter(self._succ[n])
KeyError: ProgramGraphNode(index=1, basic_block=[<FOR_ITER arg=[<LOAD_CONST arg=ExecutionTracer lineno=369>, <LOAD_METHOD arg='executed_bool_predicate' lineno=369>, <LOAD_CONST arg=False lineno=369>, <LOAD_CONST arg=45 lineno=369>, <CALL_METHOD arg=2 lineno=369>, <POP_TOP lineno=369>, <JUMP_ABSOLUTE arg=[<LOAD_GLOBAL arg='SQLQuery' lineno=375>, <LOAD_METHOD arg='join' lineno=375>, <LOAD_FAST arg='result' lineno=375>, <LOAD_CONST arg='' lineno=375>, <CALL_METHOD arg=2 lineno=375>, <RETURN_VALUE lineno=375>] lineno=369>] lineno=369>])

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/pynguin", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/pynguin/cli.py", line 136, in main
    return generator.run().value
  File "/usr/local/lib/python3.8/site-packages/pynguin/generator.py", line 101, in run
    return self._run()
  File "/usr/local/lib/python3.8/site-packages/pynguin/generator.py", line 224, in _run
    if (setup_result := self._setup_and_check()) is None:
  File "/usr/local/lib/python3.8/site-packages/pynguin/generator.py", line 186, in _setup_and_check
    if not self._load_sut():
  File "/usr/local/lib/python3.8/site-packages/pynguin/generator.py", line 141, in _load_sut
    importlib.import_module(config.INSTANCE.module_name)
  File "/usr/local/Cellar/python@3.8/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/machinery.py", line 32, in exec_module
    super().exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 779, in exec_module
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/machinery.py", line 49, in get_code
    return instrumentation.instrument_module(to_instrument)
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 326, in instrument_module
    return self._instrument_code_recursive(module_code)
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 114, in _instrument_code_recursive
    return self._instrument_inner_code_objects(
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 75, in _instrument_inner_code_objects
    self._instrument_code_recursive(
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 113, in _instrument_code_recursive
    self._instrument_cfg(cfg, code_object_id)
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 130, in _instrument_cfg
    predicate_id = self._instrument_node(
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 172, in _instrument_node
    predicate_id = self._instrument_for_loop(
  File "/usr/local/lib/python3.8/site-packages/pynguin/instrumentation/branch_distance.py", line 423, in _instrument_for_loop
    for successor in dominator_tree.get_transitive_successors(node):
  File "/usr/local/lib/python3.8/site-packages/pynguin/analyses/controlflow/programgraph.py", line 188, in get_transitive_successors
    return self._get_transitive_successors(node, set())
  File "/usr/local/lib/python3.8/site-packages/pynguin/analyses/controlflow/programgraph.py", line 192, in _get_transitive_successors
    for successor_node in self.get_successors(node):
  File "/usr/local/lib/python3.8/site-packages/pynguin/analyses/controlflow/programgraph.py", line 129, in get_successors
    for successor in self._graph.successors(node):
  File "/usr/local/lib/python3.8/site-packages/networkx/classes/digraph.py", line 807, in successors
    raise NetworkXError(f"The node {n} is not in the digraph.") from e
networkx.exception.NetworkXError: The node ProgramGraphNode(1) is not in the digraph.

Hi @JamZYu,
thank you for using Pynguin in your evaluation and for reporting this bug.
After looking into to the stacktrace and webpy's source, I think that the [reparam method] (https://github.com/webpy/webpy/blob/eae8945f2e716d648ed2a4ec876625807c007ca4/web/db.py#L353) in the db module causes this error. The method contains dead code (lines 365-375), which in turn causes problems when Pynguin tries to instrument the methods bytecode.
We might make Pynguin more resilient against dead code in future releases.
A simple fix to continue your evaluation is to remove the dead code from the module and everything should work again.
You should also think about opening a ticket with webpy as this dead code seems to be an oversight on their side.
@stephanlukasczyk FYI.

hello, I am popping on this issue because I am having the same type of errors very often. I don't understand how you saw the problem was dead code (lines 365-375).
Below the interesting part of the error, how do you decrypted the error to know more of where is the problem here exactly ?
KeyError: ProgramGraphNode(index=22, basic_block=[<JUMP_FORWARD arg=[<LOAD_FAST arg='next_visit' lineno=206>, <LOAD_FAST arg='C' lineno=206>, <COMPARE_OP arg=<Compare.EQ: 2> lineno=206>, <POP_JUMP_IF_FALSE arg=[<LOAD_FAST arg='C' lineno=214>, <STORE_FAST arg='next_arm' lineno=214>] lineno=206>] lineno=201>])
Thanks

hello, I am popping on this issue because I am having the same type of errors very often. I don't understand how you saw the problem was dead code (lines 365-375).
Below the interesting part of the error, how do you decrypted the error to know more of where is the problem here exactly ?
KeyError: ProgramGraphNode(index=22, basic_block=[<JUMP_FORWARD arg=[<LOAD_FAST arg='next_visit' lineno=206>, <LOAD_FAST arg='C' lineno=206>, <COMPARE_OP arg=<Compare.EQ: 2> lineno=206>, <POP_JUMP_IF_FALSE arg=[<LOAD_FAST arg='C' lineno=214>, <STORE_FAST arg='next_arm' lineno=214>] lineno=206>] lineno=201>])
Thanks

Hi @MarineChap ,
I guess the conclusion of dead code was done by looking at the source code, not only on the byte code/the program-graph nodes generated by Pynguin. When you look at the code fragment, linked by @Wooza in his answer, it reads like:
def reparam(string_, dictionary):
    """
    Takes a string and a dictionary and interpolates the string
    using values from the dictionary. Returns an `SQLQuery` for the result.
        >>> reparam("s = $s", dict(s=True))
        <sql: "s = 't'">
        >>> reparam("s IN $s", dict(s=[1, 2]))
        <sql: 's IN (1, 2)'>
    """
    return SafeEval().safeeval(string_, dictionary)

    dictionary = dictionary.copy()  # eval mucks with it
    # disable builtins to avoid risk for remote code execution.
    dictionary["__builtins__"] = object()
    result = []
    for live, chunk in _interpolate(string_):
        if live:
            v = eval(chunk, dictionary)
            result.append(sqlquote(v))
        else:
            result.append(chunk)
    return SQLQuery.join(result, "")

Obviously, everything after the first return statement cannot be reached by the execution, since return leaves the method; thus, everything following that line is dead code, since it can never be reached.
Pynguin is not robust enough, which is something we will deal with in a future release. Nevertheless, I suggest that this dead code should also be removed from the webpy project.

Hi @MarineChap ,
I guess the conclusion of dead code was done by looking at the source code, not only on the byte code/the program-graph nodes generated by Pynguin. When you look at the code fragment, linked by @Wooza in his answer, it reads like:
def reparam(string_, dictionary):
    """
    Takes a string and a dictionary and interpolates the string
    using values from the dictionary. Returns an `SQLQuery` for the result.
        >>> reparam("s = $s", dict(s=True))
        <sql: "s = 't'">
        >>> reparam("s IN $s", dict(s=[1, 2]))
        <sql: 's IN (1, 2)'>
    """
    return SafeEval().safeeval(string_, dictionary)

    dictionary = dictionary.copy()  # eval mucks with it
    # disable builtins to avoid risk for remote code execution.
    dictionary["__builtins__"] = object()
    result = []
    for live, chunk in _interpolate(string_):
        if live:
            v = eval(chunk, dictionary)
            result.append(sqlquote(v))
        else:
            result.append(chunk)
    return SQLQuery.join(result, "")

Obviously, everything after the first return statement cannot be reached by the execution, since return leaves the method; thus, everything following that line is dead code, since it can never be reached.
Pynguin is not robust enough, which is something we will deal with in a future release. Nevertheless, I suggest that this dead code should also be removed from the webpy project.

Ah okay...  There is nothing as obvious as this in my code, that's why I asked this. I guess it is likely due to the use of 'yield' at the end of the methods concerned by the errors.
Thanks for the answer.

Ah okay...  There is nothing as obvious as this in my code, that's why I asked this. I guess it is likely due to the use of 'yield' at the end of the methods concerned by the errors.
Thanks for the answer.

You're welcome. Would you mind to share the code you're trying to run Pynguin on? From what you wrote it sounds like you want to generate tests for a generator method? Please note that Pynguin is in a quite early stage thus this might be functionality we need to add to it. Hence, it would be great to also have some real-world examples to see whether a future implementation can deal with them.

You're welcome. Would you mind to share the code you're trying to run Pynguin on? From what you wrote it sounds like you want to generate tests for a generator method? Please note that Pynguin is in a quite early stage thus this might be functionality we need to add to it. Hence, it would be great to also have some real-world examples to see whether a future implementation can deal with them.

Sorry I forget to answer this. Yes definitely the test of a method with generator within was maybe too much. More reasons to have a way to specify don't do test on this method (see my other open issue) ;)
Code example:
https://bitbucket.org/kloostermannerflab/fklab-controller-lab/src/master/hive/implementation/w_alter_task.py
Also, while I am here, I have some server communication in this project, is it possible to mock some part of the code with Pynguin ?
Good luck for your next development.

Sorry I forget to answer this. Yes definitely the test of a method with generator within was maybe too much. More reasons to have a way to specify don't do test on this method (see my other open issue) ;)
Code example:
https://bitbucket.org/kloostermannerflab/fklab-controller-lab/src/master/hive/implementation/w_alter_task.py
Also, while I am here, I have some server communication in this project, is it possible to mock some part of the code with Pynguin ?
Good luck for your next development.

authors: JamZYu MarineChap Wooza stephanlukasczyk


text:
Interprocedural Invariants

Hi,
I have a simple C program like the following:
#include <unistd.h>

int add (int a) {
	if (a > 2)
		a += 4;
	else
		a += 2;
	write(1, "Hello", a);
	return a;
}

int main(int argc, char **argv) {
	int x = 4;
	add(x);
	return 0;
}

I generated the IR for this program and ran crabllvm with the following arguments:
./crabllvm -crab-inter -crab-print-invariants -crab-track=arr -crab-heap-analysis=cs-sea-dsa input.bc

I expected crab to say that a = [4, 4] holds at the entry to add but it says that no invariants hold. Am I correct in this expectation? Thanks!
EDIT: It seems if we change the program to do just add(4) instead of add(x) I see the invariant but I'm not sure if that is to be expected.

If I analyze directly your C program with crabllvm.py using this command
 crabllvm.py -O0 test-inter.c --crab-inter --crab-track=arr --crab-heap-analysis=cs-sea-dsa I get the expected result:
crabllvm.py -O0 test-inter.c --crab-inter --crab-track=arr --crab-heap-analysis=cs-sea-dsa

.0:int declare add(:int)
_2:
/**
  INVARIANTS: ({}, { -> [4, 4]})
**/
  .0.v = ite(- <= -3,4,2);
  .0 = .0.v+;
  _4 = &(.str) + 0;
  havoc(_ret);
  return .0;
/**
  INVARIANTS: ({}, {.0.v -> [4, 4],  -> [4, 4], .0 -> [8, 8]})
**/

@V_10:int declare main(:int,:ptr)
_3:
/**
  INVARIANTS: ({}, {})
**/
  @V_8 = 4;
  _ret = call add(@V_8:int);
  @V_10 = 0;
  return @V_10;
/**
  INVARIANTS: ({}, {@V_8 -> [4, 4], @V_10 -> [0, 0], _ret -> [6, 8]})
**/

Maybe your bitcode is optimized? Can you share your bitcode?

Hi!
Here is the bitcode.

The short answer is that your bitcode has the attribute optnone in all functions.
As a result, the pass mem2reg has no effect. This pass is the one that converts the program in SSA form and more importantly for crab, lowers stack variables to LLVM registers.
Having said this, crab can (in principle) handle code even if mem2reg is not run although things will be harder. I explain below (*) the challenges in case you are interested.
Is there a particular reason why you want to analyze this program with optnone? Since clang 5.0 if you just add -O0 then it will add optnone to all functions. You need to add the flags -Xclang -disable-O0-optnone to avoid adding optnone attribute.  If you call directly crabllvm.py then it will add those flags automatically for you.
(*) The analysis is  imprecise because most LLVM operations are translated to array operations in    Crab.  The current array domain in Crab is just array smashing.
Array smashing always performs weak updates for all array_store crab statements. The only way for array smashing to return something different from top is to have a crab statement array_init which sets the initial value for all array values and then each array_store performs a weak update.
The binary crabllvm does not add array_init statements. The preprocessor crabllvm-pp is the one that might add some array_init statements.

Thanks alot for the explanation! Following your instructions I see the expected invariant at the function entry.

authors: caballa shankarapailoor

repo: seahorn/crab-llvm


text:
Resource runtime configuration is null after cloning an environment

Describe the bug

Steps to reproduce

Create a resource and then clone the environment.
Check the deployment result.

Result

Failed to deploy.

Checked the runtime configuration.

Checked the custom configuration.

Expected behavior

Environment

Walrus version: main 8ff16bfe4326036642753b1ba895fa5b0e3b1791
UI version: dev-7590f46

Additional context
Add any other context about the problem here.

Environment

Walrus version: main 3c468543a2cacb6c277c03f7d36bf2839e850cee
UI version: v0.5.0-rc1

Test Result: pass

repo: seal-io/walrus


text:
Transcript Player Modes

Live - event is happening now; position is current time
Historical - we only know relative position i.e. during event, before snippet X, after snippet Y. Maybe for this case we don't have to store anything - the info to piece it together when needed is all there e.g. the snippets and event

Live player first implemented by f7aa33c and renamed by 0abce10

Historical implemented by ee3a864

repo: seandenigris/The-Project-Project


text:
dropped content

When using the tool on this page:
https://wiki.civicrm.org/confluence/display/CRMDOC/Create+a+Module+Extension#CreateaModuleExtension-Addcustomfields(CiviCRMv4.4+)
it dropped all the script blocks ie:

As it was I decided to reduce these and instead extract out the command to run as it was a bit overwelming and distracted from the main instructions

the raw output I generated is here

Verified.
Thanks for reporting! This seems like a pretty major issue. Will get on it.

I think this commit should fix this issue fully for 98% of cases.
The remaining 2% are cases are where the code block contains the string \n. In this case, the content still comes through, but with a newline there in the markdown (instead of the string \n) which messes up the display of the markdown.
Despite this case which is not handled perfectly, I'm going to close this issue because the work-around is so easy (reformatting the markdown to replace the newline with \n). Going forward, this behavior will be a known limitation.

authors: ErichBSchulz seanmadsen


text:
seat怎么解决redis和mongodb事务回滚问题

业务数据出错我想解决redis和mogondb回滚问题怎么解决？

@lixb3911 you can try to use TCC mode and write the reverse interface corresponding to the forward operation.

Does at mode provide exception rollback listening classes for developers?

@lixb3911 FailureHandler.java

i try mongodb.

authors: Garlichead lixb3911 slievrly zjinlei

repo: seata/seata


text:
seata1.7在获取表索引基数时，由于我们表数据量太大，超过了int最大值

[ ] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description
io.seata.sqlparser.struct.IndexMeta中的private int cardinality;需要修改为long类型
Ⅱ. Describe what happened
If there is an exception, please attach the exception trace:
Just paste your stack trace here!

Ⅲ. Describe what you expected to happen
Ⅳ. How to reproduce it (as minimally and precisely as possible)

xxx
xxx
xxx

Minimal yet complete reproducer code (or URL to code):
Ⅴ. Anything else we need to know?
Ⅵ. Environment:

JDK version(e.g. java -version):
Seata client/server version:
Database version:
OS(e.g. uname -a):
Others:

fixed #5762

authors: runtimeEX slievrly


text:
Seata 2.0.0 什么时候release?有计划吗？

如题，需要使用 #3887 的MSSQL AT Mode,当前是merge到 2.0.0版本的，请问有release计划了吗？
https://github.com/seata/seata/pull/3887

Can try 2.0.0 -SNAPSHOT  first , https://oss.sonatype.org/#welcome

2.0.0 has been released
release note https://github.com/seata/seata/releases/tag/v2.0.0
mvn jar https://mvnrepository.com/artifact/io.seata/seata-all/2.0.0

authors: leizhiyuan slievrly zhiwei55


text:
add -disk to filer command

What problem are we solving?
When I start the volume with -disk=nvme,and use filer to upload files.The filer will prompt no free volumes left and upload failed.
E0225 11:45:13.938780 filer_server_handlers_write.go:48 failing to assign a file id: rpc error: code = Unknown desc = no free volumes left for {"replication":{},"ttl":{"Count":0,"Unit":0}}
E0225 11:45:13.939355 filer_server_handlers_write_upload.go:209 upload error: rpc error: code = Unknown desc = no free volumes left for {"replication":{},"ttl":{"Count":0,"Unit":0}}
I0225 11:45:13.939424 common.go:70 response method:POST URL:/ with httpStatus:500 and JSON:{"error":"rpc error: code = Unknown desc = no free volumes left for {\"replication\":{},\"ttl\":{\"Count\":0,\"Unit\":0}}"}

How are we solving the problem?
I added - disk so that filer can write volume with same disk tag.
How is the PR tested?
Start volume server with -disk=nvme.
Start filer server with -disk=nvme.
Upload the file to filer server.
Checks

[ ] I have added unit tests if possible.
[ ] I will add related wiki document changes and link to this PR after merging.

Need the same fix in "weed server" also.

Need the same fix in "weed server" also.

OK, I pushed the new code and added filer.disk to weed server. At the same time, if the filer service is selected start and filer.disk is empty but volume.disk is not empty, filer.disk will also be set to the value of volume.disk.

authors: chrislusf lfhy

repo: seaweedfs/seaweedfs