text:
menuinst Exception: Nothing to do: traceback when installing Spyder 6.0.0a4 on Windows

Problem Description
I was trying to setup a new conda env with Spyder 6.0.0a4 on Windows and got a traceback related with menuinst. Spyder is still able to launch from the anaconda prompt but I think no shortcut was created 🤔
What steps reproduce the problem?

Run the following to create an env:

conda create -n spyder6_test -c conda-forge python=3.11
conda activate spyder6_test
conda install -c conda-forge/label/spyder_dev -c conda-forge/label/spyder_kernels_rc -c conda-forge spyder=6.0.0a4

What is the expected output? What do you see instead?
No traceback error and shortcut creation to launch Spyder from the env created. Instead I got a traceback related with menuinst after conda installed Spyder
Paste Traceback/Error Below (if applicable)

Executing transaction: \ menuinst Exception
Traceback (most recent call last):
  File "C:\Users\dalth\anaconda3\lib\site-packages\conda\gateways\disk\create.py", line 267, in make_menu
    menuinst.install(join(prefix, win_path_ok(file_path)), remove, prefix)
  File "C:\Users\dalth\anaconda3\lib\site-packages\menuinst\__init__.py", line 75, in install
    _install(path, remove, prefix, mode='user', root_prefix=root_prefix)
  File "C:\Users\dalth\anaconda3\lib\site-packages\menuinst\__init__.py", line 48, in _install
    ShortCut(m, sc).create()
  File "C:\Users\dalth\anaconda3\lib\site-packages\menuinst\win32.py", line 318, in create
    raise Exception("Nothing to do: %r" % self.shortcut)
Exception: Nothing to do: {'name': 'Spyder 6 ({{ ENV_NAME }})', 'description': 'Scientific PYthon Development EnviRonment', 'icon': '{{ MENU_DIR }}/spyder.{{ ICON_EXT }}', 'activate': False, 'terminal': False, 'command': [''], 'platforms': {'win': {'desktop': True, 'app_user_model_id': 'spyder.Spyder', 'command': ['{{ PREFIX }}/pythonw.exe', '{{ PREFIX }}/Scripts/spyder-script.py']}, 'linux': {'Categories': ['Development', 'Science'], 'command': ['{{ PREFIX }}/bin/spyder', '$@'], 'StartupWMClass': 'Spyder'}, 'osx': {'precommand': 'pushd "$(dirname "$0")" &>/dev/null', 'command': ['./python', '{{ PREFIX }}/bin/spyder', '$@'], 'link_in_bundle': {'{{ PREFIX }}/bin/python': '{{ MENU_ITEM_LOCATION }}/Contents/MacOS/python'}, 'CFBundleName': 'Spyder 6', 'CFBundleIdentifier': 'org.spyder-ide.Spyder', 'CFBundleVersion': '6.0.0a4'}done

Versions

Spyder version: 6.0.0a4  (conda)
Python version: 3.11.8 64-bit
Qt version: 5.15.2
PyQt5 version: 5.15.10
Operating System: Windows-10-10.0.19045-SP0

Dependencies

# Mandatory:
atomicwrites >=1.2.0              :  1.4.1 (OK)
chardet >=2.0.0                   :  5.2.0 (OK)
cloudpickle >=0.5.0               :  3.0.0 (OK)
cookiecutter >=1.6.0              :  2.6.0 (OK)
diff_match_patch >=20181111       :  20230430 (OK)
intervaltree >=3.0.2              :  3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1  :  8.22.2 (OK)
jedi >=0.17.2,<0.20.0             :  0.19.1 (OK)
jellyfish >=0.7                   :  1.0.3 (OK)
jsonschema >=3.2.0                :  4.21.1 (OK)
keyring >=17.0.0                  :  24.3.1 (OK)
nbconvert >=4.0                   :  7.16.2 (OK)
numpydoc >=0.6.0                  :  1.6.0 (OK)
paramiko >=2.4.0                  :  3.4.0 (OK)
parso >=0.7.0,<0.9.0              :  0.8.3 (OK)
pexpect >=4.4.0                   :  4.9.0 (OK)
pickleshare >=0.4                 :  0.7.5 (OK)
psutil >=5.3                      :  5.9.8 (OK)
pygments >=2.0                    :  2.17.2 (OK)
pylint >=2.5.0,<3.1               :  2.16.2 (OK)
pylint_venv >=3.0.2               :  3.0.3 (OK)
pyls_spyder >=0.4.0               :  0.4.0 (OK)
pylsp >=1.10.0,<1.11.0            :  1.10.0 (OK)
pylsp_black >=2.0.0,<3.0.0        :  2.0.0 (OK)
pyuca >=1.2                       :  1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0         :  3.2.3 (OK)
qstylizer >=0.2.2                 :  0.2.2 (OK)
qtawesome >=1.3.0                 :  1.3.0 (OK)
qtconsole >=5.5.1,<5.6.0          :  5.5.1 (OK)
qtpy >=2.4.0                      :  2.4.1 (OK)
rtree >=0.9.7                     :  1.2.0 (OK)
setuptools >=49.6.0               :  69.1.1 (OK)
sphinx >=0.6.6                    :  7.2.6 (OK)
spyder_kernels >=3.0.0b4,<3.0.0b5 :  3.0.0b4 (OK)
superqt >=0.6.1,<1.0.0            :  0.6.1 (OK)
textdistance >=4.2.0              :  4.5.0 (OK)
three_merge >=0.1.1               :  0.1.1 (OK)
watchdog >=0.10.3                 :  4.0.0 (OK)
zmq >=22.1.0                      :  25.1.2 (OK)

# Optional:
cython >=0.21                     :  None (NOK)
matplotlib >=3.0.0                :  None (NOK)
numpy >=1.7                       :  None (NOK)
pandas >=1.1.1                    :  None (NOK)
scipy >=0.17.0                    :  None (NOK)
sympy >=0.7.3                     :  None (NOK)

@mrclary, please takea look at this one.

@dalthviz, can you confirm the conda and menuinst versions in your base environment?

Seems like I have conda 23.7.4 and menuinst 1.4.19:

Seems like I have conda 23.7.4 and menuinst 1.4.19:

That would explain it, then. Updating to menusint >= 2.0 and conda >= 23.11 in the base environment will resolve the issue.

Oh I see, then is there some way we could alert this to users when installing? Maybe we need to explain that constraints regarding the base conda env in the Spyder 6.0.0a4 GitHub release installation instructions?

Oh I see, then is there some way we could alert this to users when installing? Maybe we need to explain that constraints regarding the base conda env in the Spyder 6.0.0a4 GitHub release installation instructions?

I'm not sure. @jaimergp, do you know if a menu yaml can constrain the menuinst version? How can updates to menu yaml for menuinst >= 2.0 enforce this constraint in the base environment since menuinst is not necessarily in the install environment?

This really serious because users with old menuinst versions won't have menu entries on Windows in Spyder 6 due to this.
Perhaps we could try to detect if they have menuinst < 2.0 in their base env in our recipe to use the old menu.json file? Could that be possible @mrclary?

I think there's a __conda virtual package you might be able to leverage in the package shipping the JSON.

Without __conda available, there is no way to prevent Spyder 6 from being installed conditioned on the conda version. Doing anything in the post-link script can only affect what we do with the shortcut.
Options if menuinst < 2.0 is discovered by the post-link script:

Remove spyder-menu.json. Spyder would still install, but the shortcut would not be created. This is the easiest thing to do. We can try to print a warning message that advises users to update conda or menuinst, but I don't know if conda will relay stdout from the post-link script or suppress it.
Include a copy of the old menu.json file in the feedstock and rename or remove the unnecessary file conditioned on the version of menuinst.

I prefer option 2. because it's hard to expect that people will read a single message among a lot of other (not so informative) output. Besides, they can install Spyder through Navigator, which won't allow them to see a message like that.

According to the docs, post-link scripts should:

Not depend on any installed or to-be-installed conda packages.
Depend only on simple system tools such as rm, cp, mv, and ln.

The scripts should not write to stdout or stderr unless an error occurs, but they may write to $PREFIX/.messages.txt, which is shown after conda completes all actions.

In order to detect menuinst version, I'll have to assume the base environment python executable is available 🤞🏼

Fixed by https://github.com/conda-forge/spyder-feedstock/pull/161.

authors: ccordoba12 dalthviz jaimergp mrclary


text:
Bug : Not able to save a new file, have to work on the same for all different projects

Description
What steps will reproduce the problem?

What is the expected output? What do you see instead?
Please provide any additional information below
Version and main components

Spyder Version: 3.2.6
Python Version: 3.6.4
Qt Versions: 5.6.2, PyQt5 5.6 on Windows

Dependencies
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.22.0 (OK)
numpy >=1.7      :  1.14.0 (OK)
sphinx >=0.6.6   :  1.6.6 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.11.1 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  1.1.1 (OK)
cython >=0.21    :  0.27.3 (OK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.2.1 (OK)
pylint >=0.25    :  1.8.2 (OK)

If you can describe in detail the circumstances in which the error occurred, and post the information listed in relevant subsection of the Spyder Troubleshooting Guide as a comment to this issue, then please do so. Otherwise, this issue will be closed after a week (7 days). Thanks.

Closed due to lack of response. If you can provide more information, please re-open and do so. Thanks.

source: gharchive/issue

authors: CAM-Gerlach Ishankoradia

repo: spyder-ide/spyder

license: MIT

license_type: permissive

license_source: github-api


text:
Automatic error report

<!---   *** BEFORE SUBMITTING: PASTE CLIPBOARD HERE TO COMPLETE YOUR REPORT ***   ---!>
the values of variable exporer can't be opened and viewed

This is fixed in our latest version. Please update by opening the Anaconda Prompt and running there
conda install spyder=3.3.4


text:
import scipy.io broken under Anaconda after gcc updated to 7.4 (Ubuntu 18.04)

Running Spyder 3.3.4 on Ubuntu 18.04.  Running "import scipy.io" fails since updating gcc to 7.4 four days ago (looks like Anaconda was built with 7.3).
Fixed a similar issue with scipy.linalg by updating Anaconda to 3.3.4 but this issue remains.
Interestingly, "import scipy.io" works fine when python3 is run from a command prompt:
~/anaconda3/lib/python3.7/site-packages/scipy/interpolate$ which python3
/home/tom/anaconda3/bin/python3
~/anaconda3/lib/python3.7/site-packages/scipy/interpolate$ python3
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import scipy.io

Any ideas how to fix?
What steps reproduce the problem?

import scipy.io

What is the expected output? What do you see instead?
import scipy.io

File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/io/init.py", line 97, in
from .matlab import loadmat, savemat, whosmat, byteordercodes
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/init.py", line 13, in
from .mio import loadmat, savemat, whosmat
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio.py", line 10, in
from .miobase import get_matfile_version, docfiller
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/miobase.py", line 22, in
from scipy.misc import doccer
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/misc/init.py", line 68, in
from scipy.interpolate._pade import pade as _pade
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/interpolate/init.py", line 175, in
from .interpolate import *
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/interpolate/interpolate.py", line 26, in
from . import fitpack
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/interpolate/fitpack.py", line 12, in
from ._bsplines import BSpline
File "/home/tom/anaconda3/lib/python3.7/site-packages/scipy/interpolate/_bsplines.py", line 10, in
from . import _bspl
File "stringsource", line 105, in init scipy.interpolate._bspl
AttributeError: type object 'scipy.interpolate._bspl.array' has no attribute 'reduce_cython'

## Versions

Linux Zephyr-U 4.15.0-51-generic #55-Ubuntu SMP Wed May 15 14:27:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)

# packages in environment at /home/tom/anaconda3:
#
# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0                    py37_0
alabaster                 0.7.12                   py37_0
anaconda-client           1.7.2                    py37_0
anaconda-navigator        1.9.7                    py37_0
anaconda-project          0.8.2                    py37_0
asn1crypto                0.24.0                   py37_0
astroid                   2.2.5                    py37_0
astropy                   3.1.2            py37h7b6447c_0
atomicwrites              1.3.0                    py37_1
attrs                     19.1.0                   py37_1
babel                     2.6.0                    py37_0
backcall                  0.1.0                    py37_0
backports                 1.0                      py37_1
backports.os              0.1.1                    py37_0
backports.shutil_get_terminal_size 1.0.0                    py37_2
beautifulsoup4            4.7.1                    py37_1
bitarray                  0.8.3            py37h14c3975_0
bkcharts                  0.2                      py37_0
blas                      1.0                         mkl
bleach                    3.1.0                    py37_0
blosc                     1.15.0               hd408876_0
bokeh                     1.0.4                    py37_0
boto                      2.49.0                   py37_0
bottleneck                1.2.1            py37h035aef0_1
bzip2                     1.0.6                h14c3975_5
ca-certificates           2019.5.15                     0
cairo                     1.14.12              h8948797_3
certifi                   2019.3.9                 py37_0
cffi                      1.12.2           py37h2e261b9_1
chardet                   3.0.4                    py37_1
click                     7.0                      py37_0
cloudpickle               0.8.0                    py37_0
clyent                    1.2.2                    py37_1
colorama                  0.4.1                    py37_0
conda                     4.6.14                   py37_0
conda-build               3.17.8                   py37_0
conda-env                 2.6.0                         1
conda-verify              3.1.1                    py37_0
contextlib2               0.5.5                    py37_0
cryptography              2.6.1            py37h1ba5d50_0
curl                      7.64.0               hbc83047_2
cycler                    0.10.0                   py37_0
cython                    0.29.6           py37he6710b0_0
cytoolz                   0.9.0.1          py37h14c3975_1
dask                      1.1.4                    py37_1
dask-core                 1.1.4                    py37_1
dbus                      1.13.6               h746ee38_0
decorator                 4.4.0                    py37_1
defusedxml                0.5.0                    py37_1
distributed               1.26.0                   py37_1
docutils                  0.14                     py37_0
entrypoints               0.3                      py37_0
et_xmlfile                1.0.1                    py37_0
expat                     2.2.6                he6710b0_0
fastcache                 1.0.2            py37h14c3975_2
filelock                  3.0.10                   py37_0
flask                     1.0.2                    py37_1
fontconfig                2.13.0               h9420a91_0
freetype                  2.9.1                h8a8886c_1
fribidi                   1.0.5                h7b6447c_0
future                    0.17.1                   py37_0
get_terminal_size         1.0.0                haa9412d_0
gevent                    1.4.0            py37h7b6447c_0
glib                      2.56.2               hd408876_0
glob2                     0.6                      py37_1
gmp                       6.1.2                h6c8ec71_1
gmpy2                     2.0.8            py37h10f8cd9_2
graphite2                 1.3.13               h23475e2_0
greenlet                  0.4.15           py37h7b6447c_0
gst-plugins-base          1.14.0               hbbd80ab_1
gstreamer                 1.14.0               hb453b48_1
h5py                      2.9.0            py37h7918eee_0
harfbuzz                  1.8.8                hffaf4a1_0
hdf5                      1.10.4               hb1b8bf9_0
heapdict                  1.0.0                    py37_2
html5lib                  1.0.1                    py37_0
icu                       58.2                 h9c2bf20_1
idna                      2.8                      py37_0
imageio                   2.5.0                    py37_0
imagesize                 1.1.0                    py37_0
importlib_metadata        0.8                      py37_0
intel-openmp              2019.3                      199
ipykernel                 5.1.0            py37h39e3cac_0
ipython                   7.4.0            py37h39e3cac_0
ipython_genutils          0.2.0                    py37_0
ipywidgets                7.4.2                    py37_0
isort                     4.3.16                   py37_0
itsdangerous              1.1.0                    py37_0
jbig                      2.1                  hdba287a_0
jdcal                     1.4                      py37_0
jedi                      0.13.3                   py37_0
jeepney                   0.4                      py37_0
jinja2                    2.10                     py37_0
jpeg                      9b                   h024ee3a_2
jsonschema                3.0.1                    py37_0
jupyter                   1.0.0                    py37_7
jupyter_client            5.2.4                    py37_0
jupyter_console           6.0.0                    py37_0
jupyter_core              4.4.0                    py37_0
jupyterlab                0.35.4           py37hf63ae98_0
jupyterlab_server         0.2.0                    py37_0
keyring                   18.0.0                   py37_0
kiwisolver                1.0.1            py37hf484d3e_0
krb5                      1.16.1               h173b8e3_7
lazy-object-proxy         1.3.1            py37h14c3975_2
libarchive                3.3.3                h5d8350f_5
libcurl                   7.64.0               h20c2e04_2
libedit                   3.1.20181209         hc058e9b_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 8.2.0                hdf63c60_1
libgfortran-ng            7.3.0                hdf63c60_0
liblief                   0.9.0                h7725739_2
libpng                    1.6.36               hbc83047_0
libsodium                 1.0.16               h1bed415_0
libssh2                   1.8.0                h1ba5d50_4
libstdcxx-ng              8.2.0                hdf63c60_1
libtiff                   4.0.10               h2733197_2
libtool                   2.4.6                h7b6447c_5
libuuid                   1.0.3                h1bed415_2
libxcb                    1.13                 h1bed415_1
libxml2                   2.9.9                he19cac6_0
libxslt                   1.1.33               h7d1a2b0_0
llvmlite                  0.28.0           py37hd408876_0
locket                    0.2.0                    py37_1
lxml                      4.3.2            py37hefd8a0e_0
lz4-c                     1.8.1.2              h14c3975_0
lzo                       2.10                 h49e0be7_2
markupsafe                1.1.1            py37h7b6447c_0
matplotlib                3.0.3            py37h5429711_0
mccabe                    0.6.1                    py37_1
mistune                   0.8.4            py37h7b6447c_0
mkl                       2019.3                      199
mkl-service               1.1.2            py37he904b0f_5
mkl_fft                   1.0.10           py37ha843d7b_0
mkl_random                1.0.2            py37hd81dba3_0
more-itertools            6.0.0                    py37_0
mpc                       1.1.0                h10f8cd9_1
mpfr                      4.0.1                hdf1c602_3
mpmath                    1.1.0                    py37_0
msgpack-python            0.6.1            py37hfd86e86_1
multipledispatch          0.6.0                    py37_0
navigator-updater         0.2.1                    py37_0
nbconvert                 5.4.1                    py37_3
nbformat                  4.4.0                    py37_0
ncurses                   6.1                  he6710b0_1
networkx                  2.2                      py37_1
nltk                      3.4                      py37_1
nose                      1.3.7                    py37_2
notebook                  5.7.8                    py37_0
numba                     0.43.1           py37h962f231_0
numexpr                   2.6.9            py37h9e4a6bb_0
numpy                     1.16.2           py37h7e9f1db_0
numpy-base                1.16.2           py37hde5b4d6_0
numpydoc                  0.8.0                    py37_0
olefile                   0.46                     py37_0
openpyxl                  2.6.1                    py37_1
openssl                   1.1.1c               h7b6447c_1
packaging                 19.0                     py37_0
pandas                    0.24.2           py37he6710b0_0
pandoc                    2.2.3.2                       0
pandocfilters             1.4.2                    py37_1
pango                     1.42.4               h049681c_0
parso                     0.3.4                    py37_0
partd                     0.3.10                   py37_1
patchelf                  0.9                  he6710b0_3
path.py                   11.5.0                   py37_0
pathlib2                  2.3.3                    py37_0
patsy                     0.5.1                    py37_0
pcre                      8.43                 he6710b0_0
pep8                      1.7.1                    py37_0
pexpect                   4.6.0                    py37_0
pickleshare               0.7.5                    py37_0
pillow                    5.4.1            py37h34e0f95_0
pip                       19.0.3                   py37_0
pixman                    0.38.0               h7b6447c_0
pkginfo                   1.5.0.1                  py37_0
pluggy                    0.9.0                    py37_0
ply                       3.11                     py37_0
prometheus_client         0.6.0                    py37_0
prompt_toolkit            2.0.9                    py37_0
psutil                    5.6.1            py37h7b6447c_0
ptyprocess                0.6.0                    py37_0
py                        1.8.0                    py37_0
py-lief                   0.9.0            py37h7725739_2
pycodestyle               2.5.0                    py37_0
pycosat                   0.6.3            py37h14c3975_0
pycparser                 2.19                     py37_0
pycrypto                  2.6.1            py37h14c3975_9
pycurl                    7.43.0.2         py37h1ba5d50_0
pyflakes                  2.1.1                    py37_0
pygments                  2.3.1                    py37_0
pylint                    2.3.1                    py37_0
pyodbc                    4.0.26           py37he6710b0_0
pyopenssl                 19.0.0                   py37_0
pyparsing                 2.3.1                    py37_0
pyqt                      5.9.2            py37h05f1152_2
pyrsistent                0.14.11          py37h7b6447c_0
pysocks                   1.6.8                    py37_0
pytables                  3.5.1            py37h71ec239_0
pytest                    4.3.1                    py37_0
pytest-arraydiff          0.3              py37h39e3cac_0
pytest-astropy            0.5.0                    py37_0
pytest-doctestplus        0.3.0                    py37_0
pytest-openfiles          0.3.2                    py37_0
pytest-remotedata         0.3.1                    py37_0
python                    3.7.3                h0371630_0
python-dateutil           2.8.0                    py37_0
python-libarchive-c       2.8                      py37_6
pytz                      2018.9                   py37_0
pywavelets                1.0.2            py37hdd07704_0
pyyaml                    5.1              py37h7b6447c_0
pyzmq                     18.0.0           py37he6710b0_0
qt                        5.9.7                h5867ecd_1
qtawesome                 0.5.7                    py37_1
qtconsole                 4.4.3                    py37_0
qtpy                      1.7.0                    py37_1
readline                  7.0                  h7b6447c_5
requests                  2.21.0                   py37_0
rope                      0.12.0                   py37_0
ruamel_yaml               0.15.46          py37h14c3975_0
scikit-image              0.14.2           py37he6710b0_0
scikit-learn              0.20.3           py37hd81dba3_0
scipy                     1.2.1            py37h7c811a0_0
seaborn                   0.9.0                    py37_0
secretstorage             3.1.1                    py37_0
send2trash                1.5.0                    py37_0
setuptools                40.8.0                   py37_0
simplegeneric             0.8.1                    py37_2
singledispatch            3.4.0.3                  py37_0
sip                       4.19.8           py37hf484d3e_0
six                       1.12.0                   py37_0
snappy                    1.1.7                hbae5bb6_3
snowballstemmer           1.2.1                    py37_0
sortedcollections         1.1.2                    py37_0
sortedcontainers          2.1.0                    py37_0
soupsieve                 1.8                      py37_0
sphinx                    1.8.5                    py37_0
sphinxcontrib             1.0                      py37_1
sphinxcontrib-websupport  1.1.0                    py37_1
spyder                    3.3.4                    py37_0
spyder-kernels            0.4.4                    py37_0
sqlalchemy                1.3.1            py37h7b6447c_0
sqlite                    3.27.2               h7b6447c_0
statsmodels               0.9.0            py37h035aef0_0
sympy                     1.3                      py37_0
tblib                     1.3.2                    py37_0
terminado                 0.8.1                    py37_1
testpath                  0.4.2                    py37_0
tk                        8.6.8                hbc83047_0
toolz                     0.9.0                    py37_0
tornado                   6.0.2            py37h7b6447c_0
tqdm                      4.31.1                   py37_1
traitlets                 4.3.2                    py37_0
unicodecsv                0.14.1                   py37_0
unixodbc                  2.3.7                h14c3975_0
urllib3                   1.24.1                   py37_0
wcwidth                   0.1.7                    py37_0
webencodings              0.5.1                    py37_1
werkzeug                  0.14.1                   py37_0
wheel                     0.33.1                   py37_0
widgetsnbextension        3.4.2                    py37_0
wrapt                     1.11.1           py37h7b6447c_0
wurlitzer                 1.0.2                    py37_0
xlrd                      1.2.0                    py37_0
xlsxwriter                1.1.5                    py37_0
xlwt                      1.3.0                    py37_0
xz                        5.2.4                h14c3975_4
yaml                      0.1.7                had09818_2
zeromq                    4.3.1                he6710b0_3
zict                      0.1.4                    py37_0
zipp                      0.3.3                    py37_1
zlib                      1.2.11               h7b6447c_3
zstd                      1.3.7                h0b5b093_0

### Dependencies
IPython >=4.0     :  7.4.0 (OK)
cython >=0.21     :  0.29.6 (OK)
jedi >=0.9.0      :  0.13.3 (OK)
matplotlib >=2.0.0:  3.0.3 (OK)
nbconvert >=4.0   :  5.4.1 (OK)
numpy >=1.7       :  1.16.2 (OK)
pandas >=0.13.1   :  0.24.2 (OK)
psutil >=0.3      :  5.6.1 (OK)
pycodestyle >=2.3 :  2.5.0 (OK)
pyflakes >=0.6.0  :  2.1.1 (OK)
pygments >=2.0    :  2.3.1 (OK)
pylint >=0.25     :  2.3.1 (OK)
qtconsole >=4.2.0 :  4.4.3 (OK)
rope >=0.9.4      :  0.12.0 (OK)
sphinx >=0.6.6    :  1.8.5 (OK)
sympy >=0.7.3     :  1.3 (OK)

This is not a problem with Spyder but with Anaconda. Please report it there:
https://github.com/ContinuumIO/anaconda-issues/issues/


text:
PR: Add break condition to Markdown syntax highlighter loop

Fixes: #4595
I copied this from BaseWebSH highlighter

Did you check that CHANGELOG.md loads correctly?

Did you check that CHANGELOG.md loads correctly?
yes

Thanks @rlaverde!

source: gharchive/pull-request


text:
[FORMATTING] GO Statement Before Procedure Declaration

Input data
SET
NOCOUNT ON;

GO

CREATE OR ALTER PROCEDURE dbo.sp_say_hi (@Name VARCHAR(25) = 'World')
AS BEGIN
    PRINT CONCAT('Hello, ', @Name);
END;

Expected Output
SET
NOCOUNT ON;

GO

CREATE
OR ALTER
PROCEDURE dbo.sp_say_hi (@Name VARCHAR(25) = 'World')
AS BEGIN
    PRINT CONCAT('Hello, ', @Name);
END;

The GO statement should be on it's own line.
Actual Output
SET
NOCOUNT ON;

GO CREATE
OR ALTER
PROCEDURE dbo.sp_say_hi (@Name VARCHAR(25) = 'World') AS BEGIN
    PRINT CONCAT('Hello, ', @Name);
END;

When executed on the database a fatal error occurs (Incorrect syntax was encountered while CREATE was being parsed) because of the GO.
Usage

How are you calling / using the library?
The library is being called from a Git hook with the below config:{
  "language": "tsql",
  "tab width": 4,
  "useTabs": false,
  "keywordCase": "upper",
  "indentStyle": "standard",
  "logicalOperatorNewline": "before",
  "expressionWidth": 80,
  "linesBetweenQueries": 2,
  "denseOperators": false,
  "newlineBeforeSemicolon": false
}

What SQL language(s) does this apply to?
Transact-SQL/T-SQL
Which SQL Formatter version are you using?
Latest (v12.2.2)

Thanks, the fix is now released in 12.2.2

authors: grantwforsythe nene

repo: sql-formatter-org/sql-formatter


text:
postgres functions have empty Row structs

Version
1.18.0
What happened?
Functions created for Postgres are not generating fields for returned row structs. For example:
type TestFuncSelectBlogRow struct {
}

I suspect that I'm doing something wrong, but I can't seem to figure this out. Any help would be greatly appreciated. Thank you in advance.
Relevant log output
const testFuncGetTime = `-- name: TestFuncGetTime :one
select  from test_fuct_get_time($1)
`

type TestFuncGetTimeParams struct {
	TestFuctGetTime interface{} `json:"testFuctGetTime"`
}

type TestFuncGetTimeRow struct {
}

func (q *Queries) TestFuncGetTime(ctx context.Context, arg TestFuncGetTimeParams) (TestFuncGetTimeRow, error) {
	row := q.db.QueryRow(ctx, testFuncGetTime, arg.TestFuctGetTime)
	var i TestFuncGetTimeRow
	err := row.Scan()
	return i, err
}

const testFuncSelectBlog = `-- name: TestFuncSelectBlog :one
select  from test_func_select_blog($1)
`

type TestFuncSelectBlogParams struct {
	TestFuncSelectBlog interface{} `json:"testFuncSelectBlog"`
}

type TestFuncSelectBlogRow struct {
}

func (q *Queries) TestFuncSelectBlog(ctx context.Context, arg TestFuncSelectBlogParams) (TestFuncSelectBlogRow, error) {
	row := q.db.QueryRow(ctx, testFuncSelectBlog, arg.TestFuncSelectBlog)
	var i TestFuncSelectBlogRow
	err := row.Scan()
	return i, err
}

const testFuncSelectString = `-- name: TestFuncSelectString :one
select  from test_func_select_string($1)
`

type TestFuncSelectStringParams struct {
	TestFuncSelectString interface{} `json:"testFuncSelectString"`
}

type TestFuncSelectStringRow struct {
}

func (q *Queries) TestFuncSelectString(ctx context.Context, arg TestFuncSelectStringParams) (TestFuncSelectStringRow, error) {
	row := q.db.QueryRow(ctx, testFuncSelectString, arg.TestFuncSelectString)
	var i TestFuncSelectStringRow
	err := row.Scan()
	return i, err
}

Database schema
create function test_func_get_time ()
    returns timestamp AS $$
Begin
    return now();
End;
$$ language plpgsql;

create function test_select_string (in p_string text)
    returns text AS $$
Begin
    return p_string;
End;
$$ language plpgsql;

create function test_select_blog(in p_id int)
    returns table (id int, name text, created_at timestamp, updated_at timestamp) AS $$
BEGIN RETURN QUERY
    select id, name, created_at, updated_at from blog where id = p_id;
END;
$$ language plpgsql;

SQL queries
-- name: TestFuncGetTime :one
select * from test_fuct_get_time($1);

-- name: TestFuncSelectString :one
select * from test_func_select_string($1);

-- name: TestFuncSelectBlog :one
select * from test_func_select_blog($1);

Configuration
No response
Playground URL
https://play.sqlc.dev/p/f91c263ba2690afce4165f4febce483987995c3ba53e96a8f40bcd7e0af5fbc8
What operating system are you using?
Linux
What database engines are you using?
PostgreSQL
What type of code are you generating?
Go

This is fixed in v1.23.0 by enabling the database-backed query analyzer. We added a test case for this issue so it won’t break in the future.
You can play around with the working example on the playground

authors: jwc-clinnection kyleconroy

repo: sqlc-dev/sqlc


text:
V-79151 & V-79153 & V-79155 - Get-DbsAcl

Base it off of Set but only show ACLs, don't change them. Should be fun 👍
Check the code in https://github.com/sqlcollaborative/dbadisa/issues/60 to see how to get the audit files maybe? I actually think we enumerate that in Get-DbaAudit

Maybe V-79163 too

repo: sqlcollaborative/dbadisa


text:
Support  DocumentEncryptionCert for AlwaysEncrypted

in new-dbacomputercertificate. sholdnt be too hard

We probably just want the two extensions here:
$cert = New-SelfSignedCertificate `
    -DnsName $MasterKeyDNSName `
    -CertStoreLocation Cert:\CurrentUser\My `
    -Provider 'Microsoft Strong Cryptographic Provider' `
    -KeyUsage KeyEncipherment
    -TextExtension @(
        "2.5.29.37={text}1.3.6.1.5.5.8.2.2,1.3.6.1.4.1.311.10.3.11"
    )

More color commentary here: https://justdaveinfo.wordpress.com/2015/10/15/always-on-encrypted-generating-certificates-and-column-encryption-key-encrypted_value/

repo: sqlcollaborative/dbatools


text:
Enable/Disable Trace Flag New Commands

Type of Change

[ ] Bug fix (non-breaking change, fixes #)
[X] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[X] Pester test is included
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose
Create new command for Enabling and Disabling global trace flags

AWESOME JOB!! Thank you gbargz 🌟

authors: gbargsley potatoqualitee


text:
Fixed bug in SQL Server 2000: the column "state" is not available

Type of Change

[X] Bug fix (non-breaking change, fixes # )
[ ] New feature (non-breaking change, adds functionality, fixes # )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

looks fab! 👌 thank you, kind sir

"istilllove2000" 🤣

authors: potatoqualitee spaghettidba


text:
Set-DbaLogin - add a param so that check_expiration can be set

Set-DbaLogin - add a param so that check_expiration can be set

Added a new param so that the check_expiration for the login can be set
Added a new integration test

Type of Change

[ ] Bug fix (non-breaking change, fixes # )
[x] New feature (non-breaking change, adds functionality, fixes #7090 )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[x] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[x] Adding code coverage to existing functionality
[x] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[x] Documentation
[ ] Build system

Purpose

Adding the option to set the check_expiration for the login.
Commands to test

See the integration test for the PasswordExpirationEnabled param.

@wsmelton, looks like you are doing a major restructuring of the repo. This PR can wait or if necessary I can resubmit it once the repo is restructured.

@wsmelton, looks like you are doing a major restructuring of the repo. This PR can wait or if necessary I can resubmit it once the repo is restructured.

@wsmelton, friendly reminder on this PR.

@wsmelton, friendly reminder on this PR.

@lancasteradam - my sincerest apology for the delays. this part of the lockdown is positively crushing and I've been running from larger GitHub projects. I believe I've got things a bit more under control now and will work to merge your 🔥 PRs faster. Thanks again for all of the incredible enhancements.

Thank you @potatoqualitee and @jpomfret.

authors: lancasteradam potatoqualitee


text:
Parsing fails for postgres queries with CTE's that use MATERIALIZED keyword

Hi!  I'm seeing parsing failures for Postgres queries that use the MATERIALIZED keyword.
Minimal repro in the format of a sqlparser-rs test:
#[test]
fn parse_mat_cte() {
    let sql = r#"WITH cte AS MATERIALIZED ( select id from accounts ) select id from cte;"#;
    pg().verified_stmt(sql);

    let sql2 = r#"WITH cte AS NOT MATERIALIZED ( select id from accounts ) select id from cte;"#;
    pg().verified_stmt(sql2);
}

I tested this on two versions, including the latest:
sqlparser = "0.43.1"
# sqlparser = "0.36.1"

I've opened https://github.com/sqlparser-rs/sqlparser-rs/pull/1148 to address this issue.

This has been resolved by https://github.com/sqlparser-rs/sqlparser-rs/pull/1148

repo: sqlparser-rs/sqlparser-rs


text:
Gracefully work around invalid URLs

With the default HttpURLStreamHandler, URLs are automatically encoded correctly.
For example, this URL: http://host/abc|xyz works fine with the default handler.
But OkHttp throws an exception:

java.io.IOException: Illegal character in path at index 15: http://host/abc|xyz
at com.squareup.okhttp.Request.uri(Request.java:65)
at com.squareup.okhttp.internal.http.HttpEngine.networkRequest(HttpEngine.java:579)
at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:212)
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:420)
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:371)
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:466)

I'm punting this to 2.4. The focus of that release will be our own URL API. I think can probably do something that's better-enough from java.net.URL and java.net.URI that it justifies its own existence.
Handling this kind of thing is just one example.

HttpUrl has this now. We still need to offer HttpUrl as a mechanism that you can use with the rest of OkHttp.

I'm going to actually wire this into HttpEngine in 2.5.

authors: hrj swankjesse

repo: square/okhttp


text:
Fix NaN being displayed on table popover

@michaelfinch

👍

authors: SheepGoesBaa michaelfinch

repo: square/shift


text:
Unable to open nwn_base.key

Compiling 93 scripts
WARNING
... Failed to open .key archive '~/.steam/steam/steamapps/common/Neverwinter/data/nwn_base.key'
... exception 'Failed to open KEY file.'.
Success: compiled 93 scripts
Looks like it's mangling the path because of the space.
should be common/Neverwinter Nights/data/nwn_base.key

What is the output of nasher config --global nssFlags && nasher config --local nssFlags?

-n ~/.steam/steam/steamapps/common/Neverwinter\ Nights/data

The output at the top actually did have the backslash in it, but it doesn't render in the comment.

$ nasher pack mymod --verbose
Located script compiler at /home/darren/NWN/bin/nwnsc
Located erf utility at /home/xxx/.nimble/pkgs/neverwinter-1.4.2/nwn_erf
Located gff utility at /home/xxx/.nimble/pkgs/neverwinter-1.4.2/nwn_gff
Located tlk utility at /home/xxx/.nimble/pkgs/neverwinter-1.4.2/nwn_tlk
Packing target mymod
Updating cache for target mymod
Updating area list
Success: area list updated --> 22 areas listed
Compiling 94 scripts
WARNING
... Failed to open .key archive '~/.steam/steam/steamapps/common/Neverwinter/data/nwn_base.key'
... exception 'Failed to open KEY file.'.
Success: compiled 94 scripts
Packing files for target shire into mod/mymod.mod
Prompt: mod/mymod.mod already exists. Overwrite? (Y/n)
Hint: The file to be packed is newer than the existing file
Answer: yes
Success: packed mod/mymod.mod

If you look at this line
WARNING
... Failed to open .key archive '~/.steam/steam/steamapps/common/Neverwinter/data/nwn_base.key'

You'll see it's missing some of the folder name.  That's because there's a space in your folder name. You need to use escapes because nwnsc doesn't handle spaces file and folder names.

Sorry, I should have asked for --debug rather than --verbose. Was trying to figure out if nasher is sending a bad command or if nwnsc just can't read it. Looks like it's nasher's fault. The Nim method I used to handle the arguments only handles whitespace within quotes.
I will see if there's an elegant way to fix this. A workaround for now is to use quotation marks around your path rather than escaping spaces. For example:
nasher config --nssFlags:'-n "/home/youtuserhere/.steam/steam/steamapps/common/Neverwinter Nights"'

Note that since you have to wrap it in single quotes, you have to use the full path. The ~/ will not be expanded.

Ok, thanks.

I'm not planning on making a change here after all. Using quotes to surround paths with spaces is the solution.
Note that using quotes is only necessary in the issue above since the path is itself part of another string. On Posix systems it's the shell that usually handles escaping spaces (as well as expansion of the tilde), so this works fine:
$ nasher config installDir ~/Documents/Neverwinter\ Nights

authors: radiantone squattingmonk tinygiant98

repo: squattingmonk/nasher


text:
[question] why is this a separate repo?

I got here from the rovers repo, looking for an easy way to run it on kubernetes (thanks!)
It seems like this should simply be a directory under the rovers directory though.
So when more charts are added for other projects, say borges, then they can be easily added to their own repositories, where they're easier to find and maintain.
Is there a reason why this is not the case?

There are a number of reasons

Helm charts are downloadable over https, it is normal to set up a charts repo that can be added locally (à la yum repository) and then download them from there. It is actually the usual way to use them
Application code and packaging do not need to belong to the same repository
We maintain charts that from repositories that don't belong to us
We only maintain charts in the application repo whenever we're the only users of it (blog, landing, etc)

I'm tempted to ask you to document these reasons on the README.md
Actually, now that I think about it, this README.md does not follow the documentation guidelines.

Fair enough. I'll fix it ;-)

authors: campoy rporres

repo: src-d/charts


text:
no new desktop aplications are created when installing from --github --from

OS: manjaro
DE: KDE
Version:2.2.1.1.gb8ed330
edit: after restarting the applications seem to work fine and i can open appimages, though the problem with --github --from still exists

seems i was mistaken as it does create a application and a binary in /usr/bin, it just does not add a application icon and has the appimage in prenthese after the name

repo: srevinsaju/zap


text:
Handle filename accents in add folder method

Hello,
it seems that the method net.lingala.zip4j.ZipFile.addFolder(File folderToAdd, ZipParameters zipParameters) does not handle the accents in file/directory names (i.e.: 'àéÔ').
The result zip/archive skip the accents.
By the way I would like to thank you for this usefull library!
Thanks for your help, best regards.

Used version: 2.6.2

Works fine for me. Here is the code I am using
@Test
public void testIssue243() throws IOException {
    ZipFile zipFile = new ZipFile(BASE_DIR + "test.zip");
    zipFile.addFolder(new File(BASE_DIR));
}

And here is the zip file generated. This zip file contains a folder which contains a file by the name "àéÔ.txt"
test.zip
I see that you also use ZipParameters. Do you set any flag/value in  the zipParameters?

Closing this issue as there is no further feedback.

authors: damien-guillermet srikanth-lingala


text:
Fix all of the e2e tests

This is a WIP. I am making the PR early to collaboratively track progress.

@srleecode check out https://github.com/nrwl/nx/issues/7504 👯

After installing with yarn, I get the below error when running yarn affected:e2e --all. I don't get this error when installing with npm. I also get a similar error using mac "spawnSync /bin/sh ENOENT". I get this with the normal yarn version as well as after running "yarn set version berry" then "yarn set version 3.x" and installing with yarn
spawnSync C:\Windows\System32\cmd.exe ENOENT
at runNxNewCommand (../../../../packages/nx-plugin/src/utils/testing-utils/nx-project.ts:15:18)
at Object.newNxProject (../../../../packages/nx-plugin/src/utils/testing-utils/nx-project.ts:72:3)

@srleecode off... I ran successfully with:
yarn run e2e:domain-cypress-domain-test
yarn run e2e:domain-front-end-angular
yarn run e2e:domain-grouping-folder
yarn run e2e:domain-mock-file

I see the issue.... no --packageManager=yarn is set

This must be specific to the affected command. I will need to update the patches I think. Hang tight.

authors: benpsnyder srleecode

repo: srleecode/domain


text:
Add redirect to fix misprinted kit list document

Summary
There was a misprint on the kit list.
Code review
Testing

[ ] applied the configuration locally
[ ] manually validated the new behaviour

Links
https://docs.google.com/document/d/1N7mYdnkpsnXOb0nHRRT2rYhcLqEnq8EndQXVNaE884E/edit?usp=sharing

@RealOrangeOne especially as this was fixing an issue due to a link not being checked before being printed, please could we check that this actually works? Both that it's what was in fact on the sheets (my recollection of which was that it had a trailing slash) and that it does indeed direct users to the right link.

$ curl https://sr-proxy/kit/return/ -ksI | grep location
location: https://sr-proxy/docs/kit/return/

authors: PeterJCLaw RealOrangeOne

repo: srobo/ansible


text:
🛑 NZBGet is down

In 89c0a17, NZBGet (https://nzbget.srueg.ch/ping) was down:

HTTP code: 503
Response time: 4563 ms

Resolved: NZBGet is back up in 66cbfbf.

repo: srueg/upptime


text:
🛑 Keycloak is down

In c378b6c, Keycloak (https://sso.srueg.ch/health) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Keycloak is back up in 66aea61 after 8 minutes.


text:
AI got borg icon

https://discord.com/channels/1272545509562777621/1317851957418786877/1317851957418786877

fixed


text:
Refactor/refactor video machine

把 categories 的邏輯搬出來 讓 videosMachine 沒那麼大包
在想要不要把 videos & categories machine 用 parallel  寫成一個 machine
改用 matches 判斷 machine 的 state

在想要不要把 videos & categories machine 用 parallel 寫成一個 machine
感覺不錯，照目前的功能也許全部一個機器搞定就好了

@shinenic 感謝回覆
不過我在想這樣我還能用 hook 把 send event 的 function 分開嗎 不然都擠在一個 hook 有點難看


text:
Namespace in external DLL causes issues in component code generation.

I have an enum in a namespace in a DLL that is separate from the code that has my components in it. I'm using that enum in a component.

If I use a 'using' directive at the top of the component then the generated code doesn't bring that over or fully qualify the namespace for the type.
If the enum is in the same DLL as the components then the generated code does fully quality the namespace and will compile.
If the enum is left in a separate DLL but I fully qualify the namespace in the component, then the generated code will fully qualify the namespace and compile.

Here's some code
// This is in Foo.dll
namespace Foo
{
    public enum Color
    {
        Blue,
        Red,
        Yellow,
        Green,
        Purple,
    }
}

// This is in Bar.dll
using Foo;
using Entitas;

namespace Bar.Components
{
    [Game]
    public class ColorComponent : IComponent
    {
        public Color color;
    }
}

// Generated code, in Bar.dll
public partial class GameEntity {

public Bar.Components.ColorComponent color { get { return (Bar.Components.ColorComponent)GetComponent(GameComponentsLookup.Color); } }
public bool hasColor { get { return HasComponent(GameComponentsLookup.Color); } }

// *** The new line is what fails to compile. The function parameter, Color, is not found ***
public void AddColor(Color newColor) {
    var index = GameComponentsLookup.Color;
    var component = CreateComponent<Bar.Components.ColorComponent>(index);
    component.color = newColor;
    AddComponent(index, component);
}

...

If I fully quality the type in my component instead of using a using, then the code generator seems to pick that up.
// This is in Bar.dll
using Entitas;

namespace Bar.Components
{
    [Game]
    public class ColorComponent : IComponent
    {
        public Foo.Color color;
    }
}

// Generated code, in Bar.dll
public partial class GameEntity {

public Bar.Components.ColorComponent color { get { return (Bar.Components.ColorComponent)GetComponent(GameComponentsLookup.Color); } }
public bool hasColor { get { return HasComponent(GameComponentsLookup.Color); } }

// *** Fully qualified type is now listed here ***
public void AddColor(Foo.Color newColor) {
    var index = GameComponentsLookup.Color;
    var component = CreateComponent<Bar.Components.ColorComponent>(index);
    component.color = newColor;
    AddComponent(index, component);
}

...

I added both the path to the folder with the source itself and the path to the folder with the .dll, to the Jenny.SearchPaths, and that didn't seem to have any effect.

Cannot reproduce, works for me.
With which .NET version is the dll compiled?

.NET 4.7.1 for both dlls.
I'm able to work around the issue for now by using the full namespace in the component variables. So this isn't a blocking issue. Just seemed like it might be a bug. It's a shame you can't reproduce it though.

I hope your fine with the workaround. Sorry that I couldn't reproduce. When in doubt, you can send me stripped down demo project with you specific setup so I can try. Will close the issue for now

repo: sschmid/Entitas-CSharp

license: mit

license_source: bigquery


text:
Infisical

Do you want to add or update the vendor? Adjust the title accordingly!
How to use? Fill the yaml file below!

name: Use the vendors name
base_pricing: Use the vendors cheapest available price suitable for teams/companies. A plan limited to less than 10 users may not be suited for companies.
sso_pricing: Use the vendors cheapest price that includes SSO.
scim_pricing: Use the vendors cheapest price that includes SCIM.
pricing_scheme: How is the pricing? Is it per user per month? Per month?
pricing_sources: Add a link or a note to where you got the pricing from. One per line.
notes: If you need to clarify pricing, please add a note.
free_sso_providers: Do they support free SSO providers like Google, Apple, Github etc.? One per line.
updated_at: yyyy-mm-dd of the day you filed this.
vendor_url: Use the vendors public website

---
base_pricing: 0
currency: USD
free_sso_providers:
name: Infisical
notes: You can add SSO to a plan for $15 per u/m.
pricing_scheme: per user/month
pricing_sources:
  - https://infisical.com/pricing
  - Quote
scim_pricing: ???
sso_pricing: 15
updated_at: 2024-05-18
vendor_url: https://infisical.com/

Infisical seems to do a great job, no reason to put them on the negative wall. The free plan is not what a normal business would choose. We would like to compare the SSO pricing tier with the base pricing tier that a business would use. In this case there is only one paid plan which includes SSO.
So that's great actually. I added them instead to the good site. Thanks for your contribution!

repo: ssotax/ssotax


text:
sst dev doesn't forward event bus links

The documentation says that we can use sst dev <command> to run the live lambdas, as well as whatever command to pull in the linked resources to that command, which is handy for development of UIs.
After implementing EventBus I noticed that it doesn't seem passing in needed envs to link the event bus.
sst dev env | grep AWS_
# Nothing returned

sst shell env | grep AWS_
# AWS_SECRET_ACCESS_KEY=jkfldsajkfld;sajklfdsaj
# AWS_SESSION_TOKEN=
# AWS_REGION=us-east-1
# AWS_ACCESS_KEY_ID=lkjsdlkjsddlfkj

For me, running a remix app, I get by just fine by running sst dev and sst shell bun dev in separate terminals.
Of course, this only impacts if the UI (or whatever command) utilizes and emits events to a bus.
I get this error on build (on render, really)
accessKeyId is a required option

    at new AwsClient (file:///Users/.../node_modules/aws4fetch/dist/aws4fetch.esm.mjs:31:36)
    at file:///Users/.../node_modules/sst/dist/aws/client.js:2:23
    at ModuleJob.run (node:internal/modules/esm/module_job:262:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:475:24)
    at async nodeImport (file:///Users/.../node_modules/vite/dist/node/chunks/dep-BcXSligG.js:53493:15)
    at async ssrImport (file:///Users/.../node_modules/vite/dist/node/chunks/dep-BcXSligG.js:53349:16)
    at async eval (/Users/.../redacted/function.ts:9:31)
    at async instantiateModule (file:///Users/.../node_modules/vite/dist/node/chunks/dep-BcXSligG.js:53408:5

@thdxr was this intended?

i need a bit more information i thinkif you are running sst dev <command> it will look for any receivers (eg your remix site) that exist in that directory and pull in whatever is linkedare you sure you have linked bus to your remix site?

Yep, bus is linked.
Would it matter if I'm working in a monorepo, and all of my event handlers are in different packages than remix?

authors: jayair mckamyk thdxr

repo: sst/ion


text:
Sometimes Functions get updated on redeploy when nothing has changed

Requests:

Patrick https://serverless-stack.slack.com/archives/C01JG3B20RY/p1645660108808139?thread_ts=1641851553.159900&cid=C01JG3B20RY

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble

authors: fwang thdxr

repo: sst/sst


text:
get 결과에서 순위가 낮은 번역결과를 제시합니다.

기스 스크레치 없음 잔상 없음 모든 기능 이상 없이 작동 깨
를 번역하면
결과1 : No scratch, no afterimage, no malfunction
결과2 : No scratch scratched afterimage no longer work without breaking any functionality
중에서, 결과2만을 반환하도록 함수가 작동합니다.
googletrans.Translator().translate 함수에서 사용하는
googletrans.Translator()._translate 함수에서
params = utils.build_params(query=text, src=src, dest=dest,
token=token)
의 params 딕셔너리에서, client : t 부분을 client : webapp 으로 수정하면
params['client'] = 'webapp'
data에는 두가지 결과가 모두 담겨서 존재하고
결과적으로 첫번째(우선순위가 높은) 결과가 return 으로 반환됩니다.

단순히, client.py 의
googletrans.Translator()._translate 함수 부분에서
params = utils.build_params(query=text, src=src, dest=dest, token=token)
url = urls.TRANSLATE.format(host=self._pick_service_url())
을
params = utils.build_params(query=text, src=src, dest=dest, token=token)
params['client'] = 'webapp'
url = urls.TRANSLATE.format(host=self._pick_service_url())
로 수정하면 정상작동합니다.

repo: ssut/py-googletrans


text:
Support for PDO parameter binding with PDOStatement::execute()

relates to https://github.com/staabm/phpstan-dba/issues/603
Added support for parameter bindings from PDOStatement::execute() with checking for syntax error and query plan analyzer.
@staabm could you check this PR?

Closing due to inactivity of project maintainer.

Sorry, I don't have time for a in detail review. The original issue is still valid.
When time allows I will take your tests and fix it

authors: staabm thbley

repo: staabm/phpstan-dba


text:
randomConfiguration generates inf position with a model loaded from urdf

Hi all!
I'm trying to understand how to load an urdf in Pinocchio. As a small exercise, I tried to load a talos urdf stored in the example-robot-data repo.
I noticed that when I try to generate a random configuration using randomConfiguration I got the following result
q: [        inf         inf         inf  0.38153428  0.84547241  0.28916507
 -0.23662536  0.29451564  0.28089003 -1.31861968  1.45029334 -0.31744632
  0.13495363 -0.87046041  0.01403343  0.56648208  2.39859869  0.01137326
  0.22755334 -0.93828232  0.37381693 -1.53665649  0.69946369 -1.77282095
 -0.46139765 -1.73770328 -0.27661543 -0.51691005 -0.93325324  1.56854384
 -2.2512585   0.0631996  -0.37908287  0.57012142 -0.56958717  0.1920592
 -0.49816529  0.25507951  1.23772212]

If I load the model with example_robot_data.load('talos'), randomConfiguration generate a meaningful configuration.
q: [ 0.68037543 -0.21123415  0.56619845  0.38153428  0.84547241  0.28916507
 -0.23662536  0.29451564  0.28089003 -1.31861968  1.45029334 -0.31744632
  0.13495363 -0.87046041  0.01403343  0.56648208  2.39859869  0.01137326
  0.22755334 -0.93828232  0.37381693 -1.53665649  0.69946369 -1.77282095
 -0.46139765 -1.73770328 -0.27661543 -0.51691005 -0.93325324  1.56854384
 -2.2512585   0.0631996  -0.37908287  0.57012142 -0.56958717  0.1920592
 -0.49816529  0.25507951  1.23772212]

I installed pinocchio and example-robot-data with conda.
For the sake of completeness I copy-paste the python script I created
import pinocchio as pin
import os
import example_robot_data as robex
import argparse

def load_robot(robot_name=None):

    if robot_name is None:
        r = robex.load('talos')
        model, collision_model, visual_model = r.model, r.collision_model, r.visual_model
    else:
        conda_prefix = os.environ["CONDA_PREFIX"]
        robots_folder = os.path.join("share", "example-robot-data", "robots", "talos_data", "robots")
        urdf_model_path = os.path.join(conda_prefix, robots_folder, robot_name + ".urdf")
        mesh_dir = os.path.join(conda_prefix, "share", "example-robot-data", "robots", "talos_data")
        model, collision_model, visual_model = pin.buildModelsFromUrdf(urdf_model_path, mesh_dir, pin.JointModelFreeFlyer())
    return model, collision_model, visual_model

def main():

    parser = argparse.ArgumentParser()
    parser.add_argument('--model', type=str, default=None,
                        required=False,
                        help='Model name. If not passed the robot_example is used to load the model')
    args = parser.parse_args()

    model, collision_model, visual_model = load_robot(args.model_name)
    q = pin.neutral(model)
    print('q: %s' % q.T)
    q = pin.randomConfiguration(model)
    print('q: %s' % q.T)

if __name__ == "__main__":
    main()

You can run the script as follow:
export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH:+${ROS_PACKAGE_PATH}:}${CONDA_PREFIX}/share
python3 load_urdf.py --model talos_reduced  <---------- This will load talos_reduced.urdf directly from the urdf
python3 load_urdf.py

Environment (conda list):

$ conda list
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                      1_llvm    conda-forge
assimp                    5.0.1                hedfc422_6    conda-forge
boost                     1.74.0           py39h5472131_3    conda-forge
boost-cpp                 1.74.0               h312852a_4    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
console_bridge            1.0.1                h4bd325d_0    conda-forge
eigen                     3.4.0                h4bd325d_0    conda-forge
eigenpy                   2.6.8           py39h5aed9d1_10    conda-forge
example-robot-data        3.11.0           py39h1a9c180_0    conda-forge
hpp-fcl                   1.7.7            py39hbcdfc36_2    conda-forge
icu                       68.1                 h58526e2_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libblas                   3.9.0           12_linux64_openblas    conda-forge
libcblas                  3.9.0           12_linux64_openblas    conda-forge
libffi                    3.4.2                h9c3ff4c_4    conda-forge
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgfortran-ng            11.2.0              h69a702a_11    conda-forge
libgfortran5              11.2.0              h5c6108e_11    conda-forge
liblapack                 3.9.0           12_linux64_openblas    conda-forge
libopenblas               0.3.18          pthreads_h8fe5266_0    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_11    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
llvm-openmp               12.0.1               h4bd325d_1    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
numpy                     1.21.2           py39hdbf815f_0    conda-forge
octomap                   1.9.7                h4bd325d_0    conda-forge
openssl                   3.0.0                h7f98852_1    conda-forge
pinocchio                 2.6.3            py39hbcdfc36_2    conda-forge
pip                       21.3               pyhd8ed1ab_0    conda-forge
python                    3.9.7           hf930737_3_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
qhull                     2020.2               h4bd325d_2    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
setuptools                58.2.0           py39hf3d152e_0    conda-forge
sqlite                    3.36.0               h9cd32fc_2    conda-forge
tinyxml                   2.6.2                h4bd325d_2    conda-forge
tk                        8.6.11               h27826a3_1    conda-forge
tzdata                    2021d                he74cb21_0    conda-forge
urdfdom                   2.3.5                h4bd325d_0    conda-forge
urdfdom_headers           1.0.5                h4bd325d_2    conda-forge
wheel                     0.37.0             pyhd8ed1ab_1    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.0                ha95c52a_0    conda-forge

You need to set the lower and upper bounds on the Free Flyer joint manually, because they are not defined by default due to no default value.
It is hard to find a good default value.
Best,
Justin

authors: GiulioRomualdi jcarpent

repo: stack-of-tasks/pinocchio


text:
[CMake] fix eigen include dir for Fedora

ref. https://github.com/stack-of-tasks/eigenpy/pull/205

@nim65s Do you know what is wrong with travis?

I guess that's because robotpkg-py35-hpp-fcl is currently broken on xenial, ref https://github.com/jrl-umi3218/jrl-cmakemodules/issues/422

Thanks for the feedback. I will then merge your PR.


text:
Add Hbase new version 2.4.17 and compression

Description

Add HBase 2.4.17
Bump jmx-exporter to 0.20.0 and use softlink
Add HBase package upload script
Add snappy compression

fixes https://github.com/stackabletech/docker-images/issues/332
Definition of Done Checklist

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
Please make sure all these things are done and tick the boxes

- [ ] Changes are OpenShift compatible
- [ ] All added packages (via microdnf or otherwise) have a comment on why they are added
- [ ] Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
- [ ] All packages should have (if available) signatures/hashes verified
- [ ] Does your change affect an SBOM? Make sure to update all SBOMs

When we have more than one image that requires patching it probably makes sense to move apply_patches.sh outside of the HBase image to prevent duplication but for now I was too lazy to figure out where it should go etc.

I get the following error when running the kuttl smoke test with the operator from https://github.com/stackabletech/hbase-operator/pull/403
2023-11-06 15:54:43,540 WARN  [master/test-hbase-master-default-0:16000:becomeActiveMaster] wal.AbstractProtobufLogWriter: Init output failed, path=hdfs://test-hdfs/hbase/MasterData/WALs/test-hbase-master-default-0.test-hbase-master-default.kuttl-test-learning-gannet.svc.cluster.local,16000,1699286079146/test-hbase-master-default-0.test-hbase-master-default.kuttl-test-learning-gannet.svc.cluster.local%2C16000%2C1699286079146.1699286083449
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2693)
	at org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.getWALCellCodecClass(WALCellCodec.java:82)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.buildWALHeader0(AbstractProtobufLogWriter.java:81)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.buildWALHeader(AbstractProtobufLogWriter.java:88)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:164)
	at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
	at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295)
	at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200)
	at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:263)
	at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:344)
	at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528)
	at java.base/java.lang.Thread.run(Thread.java:829)

I get the following error when running the kuttl smoke test with the operator from stackabletech/hbase-operator#403
2023-11-06 15:54:43,540 WARN  [master/test-hbase-master-default-0:16000:becomeActiveMaster] wal.AbstractProtobufLogWriter: Init output failed, path=hdfs://test-hdfs/hbase/MasterData/WALs/test-hbase-master-default-0.test-hbase-master-default.kuttl-test-learning-gannet.svc.cluster.local,16000,1699286079146/test-hbase-master-default-0.test-hbase-master-default.kuttl-test-learning-gannet.svc.cluster.local%2C16000%2C1699286079146.1699286083449
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2693)
	at org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.getWALCellCodecClass(WALCellCodec.java:82)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.buildWALHeader0(AbstractProtobufLogWriter.java:81)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.buildWALHeader(AbstractProtobufLogWriter.java:88)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:164)
	at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:116)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:719)
	at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:881)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:574)
	at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:515)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160)
	at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
	at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:295)
	at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:200)
	at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:263)
	at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:344)
	at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528)
	at java.base/java.lang.Thread.run(Thread.java:829)

Sorry should have mentioned this. Images are not build yet, please try with:
docker pull stackable-experimental/hbase:2.4.17-stackable23.11.0-compression-fix

I believe the problem is that the environment variable HBASE_CONF_DIR is missing from the final docker stage. I will test.

The env var is not the problem (but should probably be set anyway).
The classpath  only contains the following:
/stackable/hbase/conf
/usr/lib/jvm/jre-11/lib/tools.jar
/stackable/hbase/bin/..
/stackable/hbase/bin/../lib/shaded-clients/hbase-shaded-client-2.4.17.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/audience-annotations-0.13.0.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/reload4j-1.2.22.jar
/stackable/hbase/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar

and is missing the important part ( /stackable/hbase/lib )
➜  ~ k run hbase --image docker.stackable.tech/stackable/hbase:2.4.17-stackable0.0.0-dev -- hbase classpath
pod/hbase created
➜  ~ k logs hbase
/stackable/hbase/conf:/usr/lib/jvm/jre-11/lib/tools.jar:/stackable/hbase/bin/..:/stackable/hbase/bin/../lib/shaded-clients/hbase-shaded-client-2.4.17.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/audience-annotations-0.13.0.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/reload4j-1.2.22.jar:/stackable/hbase/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar

The docker.stackable.tech/stackable-experimental/hbase:2.4.17-stackable23.11.0-compression-fix uses java 8. The HEAD version uses java 11.

After adding Phoenix to the image, the smoke test fails because apparently Phoenix needs tar which is not available
   logger.go:42: 12:40:40 | smoke_hbase-2.4.17_hdfs-3.2.4_zookeeper-3.8.3_listener-class-external-unstable/60-test-phoenix | error: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec "d0ad2265043eb78e4a1a306d497a48a547fff6f1f8b6a25a92bbaf68eced81e5": OCI runtime exec failed: exec failed: unable to start container process: exec: "tar": executable file not found in $PATH: unknown

Thank you! That one should be easy to fix, when you do, please add a comment what it is needed for so we don't forget!

Merge for now, track improvements here https://github.com/stackabletech/docker-images/issues/495

authors: lfrancke maltesander razvan

repo: stackabletech/docker-images


text:
Improve authorizer performance by sending less data to OPA

In Discord it was reported

We are running  hdfs inside Stackable with Kerberos+OPA. The CPU usage looks strange to me and the performance, even for for simple file operations (find, ls, du) is awful. While running spark jobs with data access we see consistent. high CPU load on the Namenode and OPA:
(197.0 % opa, 130.1% Namenode) in top on the host machine.
Even setting "default allow true" in OPA and removing everything else does nothing. (basically disabling OPA)

I can trigger this with a simplehdfs dfs -du hdfs://our-staackable-hdfs/data/year=2024/month=10
This takes 50 seconds (while idle) for this to run. (9200 files in 2300 dirs.)
And every time ~50 seconds, which leads me to belive that there is zero caching anywhere...
For comparison: Our seven year old server in the old cluster does this query in 2 seconds.

This is probably because the JSON requests send between HDFS and OPA are gigantic. We need to somehow reduce this size.
Without having thought too long about this, one solution could be to not send the inodeAttrs, inodes, pathByNameArr and callerContext fields by default, while still having a opt-in config for users, so that users that care about this fields can still get them.

I just checked a json-dump of an OPA-request I had lying around: 4348 lines (pretty-json) and 150kb. 99% is inodeAttrs, inodes and pathByNameArr . Your are right, this is huge.

Jumping in here, I had a similar experience when starting some real workloads which interact with HDFS and need authz. Apart from stripping the JSON which is sent to OPA, I think there is an easy improvement to be made.
In my tests, by turning on logging in the constructor of StackableAccessControlEnforcer.java we can see it is instantiated multiple times, hence the Jackson ObjectMapper from here is recommended to be made static. I cannot find the original post which I stumbled upon when researching this, but this is answer from SO is pretty similar.

Another wired thing which we experienced when enabling OPA and NNs were under load was them crashing out once every 15-20 minutes due to "too many files open". Connecting VisualVM to the NN, saw an ever increasing number of threads.

Those threads seem to be created from the HttpClient under StackableAccessControlEnforcer.java and while they should normally be released after the HttpClient is no longer used, or at least they were getting created much faster than reclaimed.
AFAIK HttpClient should be thread-safe and would be quite ok to reuse across different instances of StackableAccessControlEnforcer. While it's not specified anywhere in the docs, this introduction of the HttpClient implies multi-threading:

Once built, an HttpClient can be used to send multiple requests.

The only caveat would be that, while HttpClient is thread-safe, apparently SSLContext isn't as mentioned in this post. While it isn't a problem for now, it's worth keeping in mind if you were to use https instead of plain http.
Therefore, by creating a static HttpClient along with using an executor with fixed thread pool or cached thread pool, I was able keep the thread number pretty stable:

The change is pretty minor:
    private final static HttpClient httpClient = HttpClient.newBuilder()
            .executor(Executors.newCachedThreadPool())
            .build();

Thank you for this investigation and the detailed report.
I did review the initial code and I did stumble across the ObjectMapper instantiation. You're right: It's thread-safe and should be reused. In this case I didn't complain because I didn't think this class would be instantiated often. Sorry for the lazy question but you say it is "instantiated multiple times". Do you have numbers for that? I honestly would have thought it's instantiated once per process only.
Don't get me wrong: I believe you, I'd just like to understand what's going on and I won't be at a proper computer to check until next week.
And thanks on the HttpClient hint, that one didn't trigger for me. We'll be sure to take a look and will see if we can get a quick fix in for this in time for the next release. We'll discuss this next week.

I adjusted the logging a little bit to include the thread from where it was instantiated:
LOG.debug("Started HdfsOpaAccessControlEnforcer on thread " + Thread.currentThread().getName());

On the active NameNode, it was instantiated from "IPC Service Handler" threads and definitely many times per same threads. I no longer have some logs, but I can reproduce them easily if needed.

Thank you! I'll check it out in the next couple of days. We're very close to our feature freeze for the next release but I'd really like to get some improvements merged for this as it's hurting....

Here are some logs:
2024-11-02 11:30:33,085 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 47 on default port 8020
2024-11-02 11:30:33,086 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 47 on default port 8020
2024-11-02 11:30:33,086 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 47 on default port 8020
2024-11-02 11:30:33,086 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 47 on default port 8020
2024-11-02 11:30:33,086 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 47 on default port 8020
...
2024-11-02 11:30:33,114 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 15 on default port 8020
2024-11-02 11:30:33,114 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 15 on default port 8020
2024-11-02 11:30:33,114 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 15 on default port 8020
2024-11-02 11:30:33,114 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 15 on default port 8020
...
2024-11-02 11:30:33,820 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 1 on default port 8020
2024-11-02 11:30:33,822 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 1 on default port 8020
...
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,887 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020
2024-11-02 11:30:33,888 WARN  hadoop.StackableAccessControlEnforcer (StackableAccessControlEnforcer.java:<init>(79)) - Started HdfsOpaAccessControlEnforcer on thread IPC Server handler 27 on default port 8020

@lfrancke let me know if I can help you out with further testing or ideas.

So, I did look at it some more and found out that this is what the builder of HttpClient does when you do not specify your own executor:
    Executor ex = builder.executor;
    if (ex == null) {
      ex = Executors.newCachedThreadPool(new DefaultThreadFactory(this.id));
      this.isDefaultExecutor = true;
    } else {
      this.isDefaultExecutor = false;
    }

In other words: It is the same code as you provide.
So, the real change you made was to move it to static so there is only one instance between all clients.
I'll put up a PR that should help by only ever creating a single instance of the Enforcer class in the first place.

I updated the description of this issue and the first PR is already merged: https://github.com/stackabletech/hdfs-utils/pull/50

I measured the performance gains of #50, #51 and #49 and combining all PRs reached the following numbers - which seems pretty good to me.

I used for i in $(seq 0 100); do echo "Creating $i" && bin/hdfs dfs -put -f /etc/hosts /bench/$i; done to propagate some files and
time bin/hdfs dfs -du -h /bench as the benchmark (it does at least 100 calls to OPA)

I used for i in $(seq 0 100); do echo "Creating $i" && bin/hdfs dfs -put -f /etc/hosts /bench/$i; done to propagate some files and time bin/hdfs dfs -du -h /bench as the benchmark (it does at least 100 calls to OPA)
What about a "deeper" path? If I remember my first debugging correctly, the message size exploded for a deep path.

(it does at least 100 calls to OPA)
The amount of calls is an interesting question. du does at least two calls: One for the file path and one for the inode. The number of calls in total could be 1+2100 or up to (1+1+2100) = (root+path+file+inode *100), if the path is not cached in any way.

Can you check, if a longer path changes your numbers significantly?
bin/hdfs dfs -mkdir -p /bench/deep/path/with/many/sub/sub/dirs/
for i in $(seq 0 100); do echo "Creating $i" && bin/hdfs dfs -put -f /etc/hosts /bench/deep/path/with/many/sub/sub/dirs/$i; done time bin/hdfs dfs -du -h /bench

Thanks for the feedback, numbers look good! (WARN log level - so no debugging)

These improvements will ship in SDP 24.11 and I'll go ahead and close this issue.
Release Notes
The performance of the https://docs.stackable.tech/home/stable/hdfs/usage-guide/security.html#_authorization[OPA Authorizer] has been greatly improved. This can be a breaking change so please make sure to read the https://github.com/stackabletech/hdfs-utils/releases/tag/v0.4.0[release notes] of the hdfs-utils for details

authors: iAlex97 jradmacher lfrancke sbernauer

repo: stackabletech/hdfs-utils


text:
BUG: Can't change branch of repo from master to main

name: Bug report
about: Create a report to help us improve
title: 'BUG: Can't change branch of repo from master to main'
labels: 'node,github'
assignees: ''

Describe the bug
A clear and concise description of what the bug is.
Link to the blitz that caused the error
https://stackblitz.com/edit/http-server-m8vf6v
To Reproduce
Steps to reproduce the behavior:

Go to 'editor'
Click on 'repo select'
Scroll down to 'main'
See that nothing happens

Expected behavior
The selector should change to 'main', and the files should switch to that branch
Version of webcontainer
Hash: 7286dbe8d47d3473cde7be488b7d048fb59b05f2
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Browser name  = Chrome
Full version  = 90.0.4430.212
Major version = 90
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36
performance.memory = {
totalJSHeapSize: 62182718,
usedJSHeapSize: 57563038,
jsHeapSizeLimit: 4294705152
}
Additional context
Add any other context about the problem here.

Hi!
There are some issues with are current Git implementation. This is def a bug.
We'll be releasing a completely reworked version of this in the next week that should resolve this.

This is also happening to me with a private repo. I even tried to do a workaround my making a new branch called master. This hack allows for the files to be imported but when I try to switch to any branch then it just loads forever.

Hi there, is this still open? Any updates on progress?

authors: KwintenP bdombro jimbuck thetaPC

repo: stackblitz/webcontainer-core


text:
fix: navbar width inconsistency

The top nav bar was sized inconsistently on docs and non-docs pages.

Old
New

@sylwiavargas The search bar only disappears on these local screenshots. It works / is visible on the live site.

repo: stackblitz/webcontainer-docs


text:
parseFile doesn't seem to escape characters as expected

Here is an .env file I used to illustrate the potential issue:
TEST1="a\nb"
TEST2='a\nb'
TEST3=a\nb
TEST4="a\\nb"

Now, when I run parseFile "testenv" in ghci, I get the following output:

Formatted:
[ ("TEST1","anb"),
  ("TEST2","anb"),
  ("TEST3","anb"),
  ("TEST4","a\\nb"),
]

This is not the output I would expect though.
Output I would expect is the following:
[ ("TEST1","a\nb"),
  ("TEST2","a\nb"),
  ("TEST3","a\nb"),
  ("TEST4","a\\nb")  -- OK I am not 100% sure what to expect here, this might be ok.
]

Is my expectation wrong here, or is this potentially a bug in the Dotenv?
I am using dotenv ^>= 0.10.0.
Thanks!

Thanks @CristhianMotoche !
I don't care much about the "\n" case, but "\n" being parsed as "n" is quite tricky, it makes it impossible to enter a string with a newline inside the env (and for example, Google gives you PEM key as a secret and it has newlines!). The only solution we found so far was to encode it with base64, putting it in .env like that, and then decoding it after being red for dotenv.
Let me know if I can help in some way!

Hey @Martinsos I was able to reproduce the issue. Indeed, it seems something wrong in the parser. FWIW, I think you can add new lines instead of the \n character to represent new lines. For example:
Given this .env:
BAR="a
b"

parseFile will return:
ghci> parseFile ".env"
[("BAR","a\nb")]

I hope that works as a workaround until we find a solution for this issue.

authors: CristhianMotoche Martinsos

repo: stackbuilders/dotenv-hs


text:
fix can't upload from iOS camera/mov files

Description
Allows H264 video/quicktime uploads.
This is a follow-up to the #1617 PR that fixed #1568 core problem (not being able to upload from iOS/Safari).
MOV files being containers for MPEG-4 should be supported on the majority of devices, and iOS recordings are encoded in H264 or HEVC.
But Firefox doesn't support HEVC.[^1]
So what do we do
When a video is exported in Safari or PWA, it gets transcoded into H264 automatically by iOS eliminating (at the moment) the need for dedicated transcoding.
Since there's no way to detect a video codec without an external library and Firefox throws an error if it doesn't support the video[^2] we could assume that if an error is generated and the file is video/quicktime then it's a codec error, throwing a custom toaster error that says:

codec might not be supported, check video settings

Screenshots
n/a will provide later
Additional Context
n/a
Checklist
Are your changes backwards compatible? Please answer below: Doesn't affect uploads of other files and content already uploaded.
On a scale of 1-10 how well and how have you QA'd this change and any features it might affect? Please answer below:
6. I tried on a plethora of Android devices and it worked without any hassle IF H264 ENCODED. After extensive tests I found out that HEVC is the sole reason why mov files are not generally supported, thankfully iOS transcodes it to H264 for us so they work great!
Needless to say that Apple devices and browsers are supported and also browsers such as Firefox, Chromium-based browsers.
For frontend changes: Tested on mobile, light and dark mode? Please answer below: n/a
Did you introduce any new environment variables? If so, call them out explicitly here: No
[^1]: Firefox is rolling out HEVC support (on Windows) with the use of Windows Media Foundation, after paying for it on Windows Store.
[^2]: Throws NS_ERROR_DOM_MEDIA_METADATA_ERR

I updated this PR as iOS automatically transcodes mov files to H264 if HEVC, there's no need for renaming or changing mimetype (also was spotty on Firefox)


text:
Amending paths to match Ark

Treasuredata needs to reference v5 not v4 and the Ark path includes lts
RabbitMQ path should reference noarch rather than x86_64

Found during the upgrade of Graphcore CL0 from Antelope to Caracal

Unfortunately it looks like our CI can't handle external contributors, we don't get very many of these. I've pushed to a new branch under my auth here: https://github.com/stackhpc/stackhpc-kayobe-config/pull/1426
Thanks for the fix @jjo93sa! We'll look into fixing external contributions in the future

repo: stackhpc/stackhpc-kayobe-config


text:
Add a method to retrieve an entity by upstream ID with a hint

Summary
When processing a webhook payload, we typically only have the
information from the webhook payload available which is to say
information about the upstream entity. We even don't know exactly which
provider and which project are we handling.
We used to sort of wing it in the github webhook handler and were just
searching by upstream ID as if it was globally unique, but that no
longer works with the introduction of multiple providers where several
entities might have the same upstream ID, just in different providers.
To handle that, we add a new method to the properties service that
searches by upstream ID with a hint. At the moment, the interface
supports providerID, projectID and provider implements as a hint.
The immediate use is that the github provider will search by the
upstream ID with the hint set to ProviderTypeGithub as that's
implemented both by the app and OAuth github providers.
If any attribute of the hint is set, then the entity's attribute must
match the hint, otherwise the entity is filtered out. Additionally, once
the entity is found, the hint might contain a property with a value that
must match. The use-case there would be deleting entities where we want
to ensure that we are deleting an entity with an appropriate upstream
hook ID.
If no entities match the hints or if multiple entities match a hint, an
error is returned.
Related: #4327
Change Type

[ ] Bug fix (resolves an issue without affecting existing features)
[x] Feature (adds new functionality without breaking changes)
[ ] Breaking change (may impact existing functionalities or require documentation updates)
[ ] Documentation (updates or additions to documentation)
[ ] Refactoring or test improvements (no bug fixes or new functionality)

Testing
as part of a larger branch + unit tests
Review Checklist:

[x] Reviewed my own code for quality and clarity.
[x] Added comments to complex or tricky code sections.
[ ] Updated any affected documentation.
[x] Included tests that validate the fix or feature.
[ ] Checked that related changes are merged.

coverage: 52.962% (+0.1%) from 52.857%
when pulling f70e4db7fcd6c0225c0e0733b6f42217f21405cb on jhrozek:entity_by_upstream_with_hint
into 6f89a1de73ec7e42b1730951b645bbfb6c1387a0 on stacklok:main.

authors: coveralls jhrozek

repo: stacklok/minder


text:
Add compliance levels by standard widget

Description
Adds a Compliance levels by standard widget.
TODO - Write tests.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

Testing Performed
TODO(replace-me)

Current dependencies on/for this PR:

master

PR #2138

PR #2166
PR #2190
PR #2204   👈

This comment was auto-generated by Graphite.

@dvail I was seeing an error when I tried going to the new dashboard view

authors: dvail sachaudh

repo: stackrox/stackrox


text:
ROX-11755: SAC tests for ClusterCVEEdge object

Description
With the migration to postgres and the associated re-desing of scoped access control, it is necessary to validate that the Scoped Access Control data filtering works the same before and after the migration.
This change covers the Dackbox ClusterCVEEdge objects.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
- [ ] Evaluated and added CHANGELOG entry if required
- [ ] Determined and documented upgrade steps
- [ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
CI should be sufficient

/test all

/retest

authors: janisz rhybrillou


text:
fix(pods): remove unused datastore functions

Description
Fix unused GetPodIDs function in the pod datastore.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change
N/A
Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

Current dependencies on/for this PR:

master

PR #8576   👈

This stack of pull requests is managed by Graphite.

Can you also remove WalkAll?


text:
cds translate error

this an error message - it would be helpful if this also included a report of what the sequences name is causing the problem (and sequence file) - I assume there is an internal stop codon, it would be better to just skip failed sequences than crash the whole thing.
2023-10-05 10:17:52,338 PHYling INFO Inputs are dna sequences
Traceback (most recent call last):
  File "/bigdata/stajichlab/shared/condaenv/phyling2/bin/phyling", line 8, in <module>
    sys.exit(main())
  File "/bigdata/stajichlab/shared/condaenv/phyling2/lib/python3.10/site-packages/phyling/phyling.py", line 208, in main
    args.func(**vars(args))
  File "/bigdata/stajichlab/shared/condaenv/phyling2/lib/python3.10/site-packages/phyling/libphyling.py", line 431, in main
    msa = msa_generator(inputs)
  File "/bigdata/stajichlab/shared/condaenv/phyling2/lib/python3.10/site-packages/phyling/libphyling.py", line 155, in __init__
    self._pep_seqs = self._sequences.translate()
  File "pyhmmer/easel.pyx", line 5835, in pyhmmer.easel.DigitalSequenceBlock.translate
  File "pyhmmer/easel.pyx", line 5889, in pyhmmer.easel.DigitalSequenceBlock.translate
ValueError: Incomplete sequence of length 1672 at index 2281

Fix by fe7267fb2ba26062cf70f8dd6d35c320b325a585

repo: stajichlab/PHYling


text:
🛑 Proxima/DNS Server is down

In d63af60, Proxima/DNS Server (http://proxima01.its-telekom.eu/livewatch.php) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Proxima/DNS Server is back up in 2235ef6.

repo: stamateas/upptime


text:
🛑 Jenkins Server is down

In e3e0695, Jenkins Server (https://jenkins01.its-telekom.eu/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Jenkins Server is back up in 61b53f5.