text:
[Autodiff upstream] Add build-script flag for differentiable programm…

…ing.
Add --enable-experimental-differentiable-programming build-script flag.
The build-script flag enables/disables standard library additions
related to differentiable programming. This will allow official Swift
releases to disable these additions.
The build-script flag is on by default to ensure testing of
differentiable programming standard library additions. An additional
driver flag must be enabled to use differentiable programming features:
https://github.com/apple/swift/pull/27446

As discussed here, this PR adds the -enable-experimental-differentiable-programming build-script and CMake flag to gate standard library additions.
@jckarter: could you please help review (or loop in reviewers)? That would be a big help for enabling our progress.

Looks okay to me. @gottesmm does the build-script fu look good here?

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 482f3e7514561a8dc370d9ee0930f35961f807f8

Build failed
Swift Test Linux Platform
Git Sha - 482f3e7514561a8dc370d9ee0930f35961f807f8

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 482f3e7514561a8dc370d9ee0930f35961f807f8

Build failed
Swift Test Linux Platform
Git Sha - 482f3e7514561a8dc370d9ee0930f35961f807f8

Pinging @gottesmm for review, as requested by him

There are tests for this, no?

I updated the utils/build_swift/tests/expected_options.py test and did some manual verification.

Can you squash/make sure this passes pylint? Then LGTM

Squashed. (I tend to use "Squash and merge" when merging.)
I ran pylint with the default configuration and newly added Python code passes (or fails with the same errors as surrounding code, e.g. method name too long).

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 777ebcd7f2932cae83f8fd60ecd5a600b9c31e49

Build failed
Swift Test Linux Platform
Git Sha - 777ebcd7f2932cae83f8fd60ecd5a600b9c31e49

Linux CI failed due to a seemingly unrelated SourceKitLSP error:
12:35:59 2019-10-14 19:35:59.512 SourceKitLSPPackageTests.xctest[72164:9f7fe700] manifest parse error(s):
12:35:59
/home/buildnode/jenkins/workspace/swift-PR-Linux/branch-master/tmp/TemporaryDirectory.N7tywe/pkg/Package.swift:3:5: error: type annotation missing in pattern
12:35:59 let pack
12:35:59     ^

I'll try triggering it again.

@swift-ci Please test linux

Thanks to reviewers!

authors: DougGregor dan-zheng swift-ci


text:
[AutoDiff] Move some 'IndexSubset' method impls to their own file.

Move some 'IndexSubset' method implemenetations from AutoDiff.cpp to a new file called IndexSubset.cpp.
Fix some indentation errors caused by the earlier renaming of AutoDiffIndexSubset.

@swift-ci please test tensorflow

source: gharchive/pull-request

repo: apple/swift

license_type: permissive

license_source: github-api


text:
[SE-0177] Add clamped(to:) methods to Comparable and Strideable

https://forums.swift.org/t/add-a-clamp-function-to-algorithm-swift/5405

@swift-ci Please smoke test

@swift-ci Please smoke test Windows platform

https://forums.swift.org/t/revisiting-se-0177-adding-clamped-to/38332/10


text:
Frontend: remove -swift-module-file. NFC

We are now using a JSON file for explicit module inputs, so this argument can be removed.

@swift-ci please smoke test

@swift-ci please smoke test


text:
DNM: testing apple/swift-installer-scripts#95

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

Please test with following PRs:
apple/swift-installer-scripts#95
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/97
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/97
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/103
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/103
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/103
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/103
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/103
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/103
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/104
@swift-ci please build toolchain windows platform

Please test with following PRs:
https://github.com/apple/swift-installer-scripts/pull/105
@swift-ci please build toolchain windows platform


text:
Restore Prior Behavior for NSCopying Lookup in Subscript Type Mapping

Bear with me, this is going to get weird.
Swift has to import -[NSDictionary objectForKeyedSubscript:] with a special NSCopying
bound that otherwise does not appear in source. This involves the clang importer
looking very specifically for this selector, on this type, in Foundation. The old
behavior was to try to run a type lookup for NSCopying then bail if that failed.
With the introduction of explicit existentials, the failure path changed to check
if the existential itself is NULL. That's currently an impossible condition. What instead
happens is that if your SDK is broken we submit a NULL NSCopying primitive to the
ExistentialType constructor and crash in +asserts compilers.
Quite how you wind up in this position in real
world setups is a mystery - if the Foundation module were somehow corrupt, or the module
cache returned an incoherent type entry lacking a definition for NSCopying we could crash
here. In fact, you can replicate this crash by removing the definition for NSCopying in the
mock SDK and leaving behind a forward declaration!
Since this condition is not representative of an expected setup, restoring the old behavior
sans test is probably the right way to go.
rdar://91062370

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci smoke test

@swift-ci smoke test and merge


text:
[ConstraintSystem] Adjust getCalleeLocator to handle implicit `call…

…AsFunction`
A call to .callAsFunction could be injected after initializer if the type is
callable, getCalleeLocator should recognize that situation and return
appropriate locator otherwise if callAsFunction has e.g. a result builder
attached to one of its parameters the solver wouldn't be able to find it.
Resolves: rdar://92914226

@swift-ci please smoke test


text:
[move-only-addresses] Slicing some patches off of the larger patch.

In this PR, I am just slicing off some work from the larger series of address move only patches that I am preparing. Specifically this:

Teaches the SILVerifier that like copy_value, load [copy] and copy_addr [init] can not be used on move only typed values once we are in canonical SIL.
It adds a new SIL instruction called explicit_copy_addr. This is exactly like copy_addr except that one /can/ use it to copy move only values once we are in canonical SIL. It has two uses: to enable us to implement explicit copy for no implicit copy addresses and to enable the move checker to transform copy_addr that it cannot eliminate into explicit_copy_addr so that we still produce valid SIL when we emit an error. NOTE: We don't need this for load [copy], since we can just transform the load [copy] into a load_borrow + explicit_copy_value.
Since I added docs to SIL.rst for explicit_copy_addr, I decided to clean up the SIL.rst docs around explicit_copy_value/copy_value.
With all that together I added a small very preliminary section to SIL.rst about move only types and their restriction around copy_value/copy_addr/load [copy] and how one can use the explicit variants when appropriate [see point 2 above].

@swift-ci test


text:
Re-enable test/Generics/requirement_machine_diagnostics.swift

The underlying issue should be fixed by https://github.com/apple/swift-syntax/pull/1399
rdar://106457182

@swift-ci Please smoke test


text:
[cxx-interop] add a SWIFT_MUTATING customization macro

Fixes https://github.com/apple/swift/issues/66322

@swift-ci please test

@swift-ci please test source compatibility


text:
[5.10][stdlib] Bump runtime version indicator to 5.10

The stdlib built from release/5.10 needs to report itself as such.
rdar://117549628

@swift-ci test


text:
[CodeCompletion] Always suggest call pattern for nested function calls

When completing in cases like bar(arg: foo(|, option: 1), we don’t know if option belongs to the call to foo or bar. Be defensive and also suggest the signature.

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please test Windows


text:
Fix crash checking objcImpl block property

ObjCImplementationChecker::matchTypes() implicitly assumed that if it was comparing function types, it must be working on an AbstractFunctionDecl. This isn’t true for a property of block type, which could cause crashes when checking their parameter types. Fix this oversight.
Fixes rdar://122280735.

@swift-ci smoke test

@swift-ci smoke test


text:
AttributeError: module 'mxnet' has no attribute 'version'

Facing this issue using Google Colab
# Install libraries - you might need to restart the runtime after doing this
!pip install turicreate==5.6
# The wrong version of mxnet will be installed
!pip uninstall -y mxnet
# Install CUDA10-compatible version of mxnet
!pip install mxnet-cu100

import turicreate as tc
tc.config.set_num_gpus(-1)

print('### Labeling images')
images = tc.load_images('2images')
images['labels'] = images['path'].element_slice(8,-4)

print('### one_shot_object_detector creation...')
model = tc.one_shot_object_detector.create(images, 'labels')

print('### one_shot_object_detector evaluation...')
model.evaluate(data)

print('### exporting Model ')
# Export to Core ML
model.export_coreml('grn.mlmodel')

Error:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-15dc8702d986> in <module>()
     10 images['labels'] = images['path'].element_slice(8,-4)
     11
---> 12 model = tc.one_shot_object_detector.create(images, 'labels')
     13
     14 model.evaluate(data)

3 frames
/usr/local/lib/python3.6/dist-packages/turicreate/toolkits/_mxnet/__init__.py in __getattribute__(self, attr)
     18             import mxnet
     19
---> 20             version_tuple = tuple(int(x) for x in mxnet.__version__.split('.') if x.isdigit())
     21             lowest_version = (0, 11, 0)
     22             not_yet_supported_version = (1, 2, 0)

AttributeError: module 'mxnet' has no attribute '__version__'

pip freeze:
absl-py==0.7.1
alabaster==0.7.12
albumentations==0.1.12
altair==3.1.0
astor==0.8.0
astropy==3.0.5
atari-py==0.1.15
atomicwrites==1.3.0
attrs==19.1.0
audioread==2.1.8
autograd==1.2
Babel==2.7.0
backcall==0.1.0
backports.tempfile==1.0
backports.weakref==1.0.post1
beautifulsoup4==4.6.3
bleach==3.1.0
blis==0.2.4
bokeh==1.0.4
boto==2.49.0
boto3==1.9.189
botocore==1.12.189
Bottleneck==1.2.1
branca==0.3.1
bs4==0.0.1
bz2file==0.98
cachetools==3.1.1
certifi==2019.6.16
cffi==1.12.3
chainer==5.4.0
chardet==3.0.4
Click==7.0
cloudpickle==0.6.1
cmake==3.12.0
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.1.3
coremltools==2.1.0
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.14.6
cupy-cuda100==5.4.0
cvxopt==1.2.3
cvxpy==1.0.15
cycler==0.10.0
cymem==2.0.2
Cython==0.29.12
daft==0.0.4
dask==1.1.5
dataclasses==0.6
datascience==0.10.6
decorator==4.4.0
defusedxml==0.6.0
descartes==1.1.0
dill==0.3.0
distributed==1.25.3
Django==2.2.3
dlib==19.16.0
dm-sonnet==1.34
docopt==0.6.2
docutils==0.14
dopamine-rl==1.0.5
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
en-core-web-sm==2.1.0
entrypoints==0.3
ephem==3.7.6.0
et-xmlfile==1.0.1
fa2==0.3.5
fancyimpute==0.4.3
fastai==1.0.55
fastcache==1.1.0
fastdtw==0.3.2
fastprogress==0.1.21
fastrlock==0.4
fbprophet==0.5
feather-format==0.4.0
featuretools==0.4.1
filelock==3.0.12
fix-yahoo-finance==0.0.22
Flask==1.1.1
folium==0.8.3
fsspec==0.3.2
future==0.16.0
gast==0.2.2
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.49
geopy==1.17.0
gevent==1.4.0
gin-config==0.2.0
glob2==0.7
google==2.0.2
google-api-core==1.13.0
google-api-python-client==1.7.9
google-auth==1.4.2
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.4.0
google-cloud-bigquery==1.14.0
google-cloud-core==1.0.2
google-cloud-datastore==1.8.0
google-cloud-language==1.2.0
google-cloud-storage==1.16.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.1.7
google-resumable-media==0.3.2
googleapis-common-protos==1.6.0
googledrivedownloader==0.4
graph-nets==1.0.4
graphviz==0.8.1
greenlet==0.4.15
grpcio==1.15.0
gspread==3.0.1
gspread-dataframe==3.0.2
gunicorn==19.9.0
gym==0.10.11
h5py==2.8.0
HeapDict==1.0.0
holidays==0.9.10
html5lib==1.0.1
httpimport==0.5.16
httplib2==0.11.3
humanize==0.5.1
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.6
image==1.5.27
imageio==2.4.1
imagesize==1.1.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.9
importlib-metadata==0.19
imutils==0.5.2
inflect==2.1.0
intel-openmp==2019.0
intervaltree==2.1.0
ipykernel==4.6.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.5.0
itsdangerous==1.1.0
jdcal==1.4.1
jedi==0.14.1
jieba==0.39
Jinja2==2.10.1
jmespath==0.9.4
joblib==0.13.2
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.1
jupyter-console==6.0.0
jupyter-core==4.5.0
kaggle==1.5.4
kapre==0.1.3.1
Keras==2.2.4
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
keras-vis==0.4.1
kiwisolver==1.1.0
knnimpute==0.1.0
librosa==0.6.3
lightgbm==2.2.3
llvmlite==0.29.0
lmdb==0.96
lucid==0.3.8
lunardate==0.2.0
lxml==4.2.6
magenta==0.3.19
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.0.3
matplotlib-venn==0.11.5
mesh-tensorflow==0.0.5
mido==1.2.6
mir-eval==0.5
missingno==0.4.2
mistune==0.8.4
mizani==0.5.4
mkl==2019.0
mlxtend==0.14.0
more-itertools==7.1.0
moviepy==0.2.3.5
mpi4py==3.0.2
mpmath==1.1.0
msgpack==0.5.6
multiprocess==0.70.8
multitasking==0.0.9
murmurhash==1.0.2
music21==5.5.0
mxnet-cu100==1.4.0.post0
mxnet-cu80==1.1.0
mxnet-cu90==1.1.0
natsort==5.5.0
nbconvert==5.5.0
nbformat==4.4.0
networkx==2.3
nibabel==2.3.3
nltk==3.2.5
nose==1.3.7
notebook==5.2.2
np-utils==0.5.10.0
numba==0.40.1
numexpr==2.6.9
numpy==1.13.3
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.0.2
okgrade==0.4.3
olefile==0.46
opencv-contrib-python==3.4.3.18
opencv-python==3.4.5.20
openpyxl==2.5.9
osqp==0.5.0
packaging==19.0
palettable==3.2.0
pandas==0.24.2
pandas-datareader==0.7.0
pandas-gbq==0.4.1
pandas-profiling==1.4.1
pandocfilters==1.4.2
parso==0.5.1
pathlib==1.0.1
patsy==0.5.1
pexpect==4.7.0
pickleshare==0.7.5
Pillow==4.3.0
pip-tools==3.9.0
plac==0.9.6
plotly==3.6.1
plotnine==0.5.1
pluggy==0.7.1
portpicker==1.2.0
prefetch-generator==1.0.1
preshed==2.0.1
pretty-midi==0.2.8
prettytable==0.7.2
progressbar2==3.38.0
prometheus-client==0.7.1
promise==2.2.1
prompt-toolkit==1.0.16
protobuf==3.7.1
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.6.0
py==1.8.0
pyarrow==0.14.0
pyasn1==0.4.5
pyasn1-modules==0.2.5
pycocotools==2.0.0
pycparser==2.19
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
pyemd==0.5.1
pyglet==1.4.1
Pygments==2.1.3
pygobject==3.26.1
pymc3==3.7
pymongo==3.8.0
pymystem3==0.2.0
PyOpenGL==3.1.0
pyparsing==2.4.0
pyrsistent==0.15.4
pysndfile==1.3.3
PySocks==1.7.0
pystan==2.19.0.0
pytest==3.6.4
python-apt==1.6.4
python-chess==0.23.11
python-dateutil==2.5.3
python-louvain==0.13
python-rtmidi==1.3.0
python-slugify==3.0.2
python-utils==2.3.0
pytz==2018.9
PyWavelets==1.0.3
PyYAML==3.13
pyzmq==17.0.0
qtconsole==4.5.2
requests==2.21.0
requests-oauthlib==1.2.0
resampy==0.2.1
retrying==1.3.3
rpy2==2.9.5
rsa==4.0
s3fs==0.3.0
s3transfer==0.2.1
scikit-image==0.15.0
scikit-learn==0.21.2
scipy==1.3.0
screen-resolution-extra==0.0.0
scs==2.1.1.post2
seaborn==0.9.0
semantic-version==2.6.0
Send2Trash==1.5.0
setuptools-git==1.2
Shapely==1.6.4.post2
simplegeneric==0.8.1
six==1.12.0
skafos==0.2.1
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==1.8.4
snowballstemmer==1.9.0
sortedcontainers==2.1.0
spacy==2.1.6
Sphinx==1.8.5
sphinxcontrib-websupport==1.1.2
SQLAlchemy==1.3.5
sqlparse==0.3.0
srsly==0.0.7
stable-baselines==2.2.1
statsmodels==0.10.0
sympy==1.1.1
tables==3.4.4
tabulate==0.8.3
tblib==1.4.0
tensor2tensor==1.11.0
tensorboard==1.14.0
tensorboardcolab==0.0.22
tensorflow==1.14.0
tensorflow-estimator==1.14.0
tensorflow-hub==0.5.0
tensorflow-metadata==0.14.0
tensorflow-probability==0.7.0
termcolor==1.1.0
terminado==0.8.2
testpath==0.4.2
text-unidecode==1.2
textblob==0.15.3
textgenrnn==1.4.1
tfds-nightly==1.0.2.dev201907170105
tflearn==0.3.2
Theano==1.0.4
thinc==7.0.8
toolz==0.10.0
torch==1.1.0
torchsummary==1.5.1
torchtext==0.3.1
torchvision==0.3.0
tornado==4.5.3
tqdm==4.28.1
traitlets==4.3.2
turicreate==5.6
tweepy==3.6.0
typing==3.7.4
tzlocal==1.5.1
umap-learn==0.3.9
uritemplate==3.0.0
urllib3==1.22
vega-datasets==0.7.0
wasabi==0.2.2
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.15.5
widgetsnbextension==3.5.0
wordcloud==1.5.0
wrapt==1.11.2
xarray==0.11.3
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yellowbrick==0.9.1
zict==1.0.0
zipp==0.5.2
zmq==0.0.0

TuriCreate does not support CUDA 10. TuriCreate currently uses MXNet for its deep learning. The latest version of MXNet which works with TuriCreate is 1.1.0. The first version of MXNet to support CUDA 10 is 1.3.1.
Do you need CUDA 10? Or would CUDA 9 work? We support CUDA 9.
It looks like you have several different CUDA version of MXNet installed:
mxnet-cu100==1.4.0.post0
mxnet-cu80==1.1.0
mxnet-cu90==1.1.0

You should only have one of these packages installed.
I would try CUDA 9 and uninstall the other version. Run pip uninstall mxnet-cu100 mxnet-cu90 and see if that works.

@TobyRoseman thanks for the hints!
I basically want to make it work on GPU,  doesn't really matter what  CUDA would be used along the way.
https://colab.research.google.com/drive/1GaJLOhdrN-OG8zsL4vgo-wBcqJh6CPYv
heres. it the whole notebook

@MaximBazarov - you may only be able to use a specific version of CUDA with Google Colab. Did you try what I suggested (uninstalling mxnet-cu100 and mxnet-cu90)? Did it work?

@TobyRoseman yes it worked till some point, there's a workaround to install CUDA 8 into the Google Colab:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61

but at some point it crashes with out of memory
Installing collected packages: mxnet-cu80
Successfully installed mxnet-cu80-1.1.0
Augmenting input images using 951 background images.
+------------------+--------------+------------------+
| Images Augmented | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 0                | 5.84s        | 0%               |
| 1                | 5.84s        | 0%               |
| 2                | 5.85s        | 0%               |
| 3                | 5.86s        | 0%               |
| 4                | 5.87s        | 0%               |
| 5                | 5.89s        | 0.25%            |
| 10               | 5.98s        | 0.5%             |
| 50               | 6.98s        | 2.5%             |
| 100              | 7.86s        | 5.25%            |
| 500              | 16.99s       | 26.25%           |
| 1000             | 32.47s       | 52.5%            |
| 1500             | 43.40s       | 78.75%           |
| 1901             | 51.44s       | 99.75%           |
+------------------+--------------+------------------+
Using 'image' as feature column
Using 'label' as annotations column
--> crash *

We're tracking the out of memory error here: #2038.
Since the original issues is resolved, I'm going to close this issue.

source: gharchive/issue

authors: MaximBazarov TobyRoseman

repo: apple/turicreate


text:
median in int column in SFrame summary not correct

Import turicreate as tc
sf=tc.SFrame({'A':[1,2,3,4],'B':['a','a','b','b']})
sf.show()
Expected:
Median for “A” is 2.5.
Actual:
Median for “A” is 3.

@znation This seems by design, can't tell if this is better or worse
import turicreate as tc
sf=tc.SFrame({'A':[1,2,3,4],'B':['a','a','b','b']})
sf.show()

shows median of 3 because its float
import turicreate as tc
sf=tc.SFrame({'A':[1.0,2.0,3.0,4.0],'B':['a','a','b','b']})
sf.show()

shows median of 2.5 because its float
I don't find that unreasonable.

Agreed - according to this StackOverflow post, there isn't really consensus on what integer median should mean when there are an even number of integers in a set (and in the case where there is an odd number of integers, clearly one of the integers is exactly the median). Closing this issue as By Design.

authors: afranklin srikris znation


text:
Telegram connection reset by peer

Hey,
Thanks for this great integration. We've been using this for the past 6 months now and it has been working great for us. For the past two weeks, however, we keep getting this message on our build logs:
Post https://api.telegram.org/bot[secret:telegram_token]/getMe: read tcp 192.168.128.3:36920->149.154.167.220:443: read: connection reset by peer

This causes our build to fail and the message doesn't get delivered to our telegram group. Could you please help me identify the issue and what I can do to fix this?

@rakshith-ravi Hi. Is url https://api.telegram.org/bot[secret:telegram_token]/getMe accessible via manual curl call?

From my system? Yes. From the server, however, it's not. For some reason, the connection keeps getting cut. Not sure why.

I think, that you have trouble on your server. I have same problem, when use plugin, because telegram blocked in Russia. For the testing puproses, try specify socks5 proxy in plugin and trigger CI. Interesting to know is it helped you.

Try the http, https or socks5 proxy:
  - name: send telegram notification
    image: appleboy/drone-telegram
    settings:
      token: xxxxxxxxxx
      to: telegram_user_id
      message: send message using custom socks5 URL
+     socks5: socks5://67.204.21.1:64312

authors: appleboy icecream78 rakshith-ravi

repo: appleboy/drone-telegram

license: MIT


text:
bezier curve intersection test scene

Hello!
I have created test scene for bezier curve intersection issue #768
I have used blender with blenderseed plugin to create scene as described in #767 (comment). Finally I have
exported scene in applessed format which is file in this pull request.
Please, let mi know if this is OK!

Hm, as far as I can tell your scene using a *.binarymesh file which is a triangle mesh and not a Bezier curve.

Oh, yes! Sorry, I have used curve to mesh conversion before exporting.
I will set this right.

authors: dictoon lorentzo

repo: appleseedhq/appleseed

license: mit

license_source: bigquery


text:
Update k8s.gcr.io to registry.k8s.io

What this PR does / why we need it?:

Need to update k8s.gcr.io to registry.k8s.io

Does this PR introduce a user-facing change?

[ ] User guide
[ ] CHANGELOG.md

Which issue(s) this PR fixes:

Related: https://github.com/application-stacks/runtime-component-operator/issues/502

Tests all passed in 373 - merging


text:
archive?

This repo has not had an update in more than 2 years.
Please either archive this repo or provide a brief description of how it is still used and help me designate an owner who can be responsible for ensuring that the repo is properly maintained for security updates.
Archiving is safe and reversible. We will not lose any data in the repo and artifacts such as releases will still be available; it will just prevent future updates or PRs without first explicitly un-archiving the repo.
This issue is assigned to you because you are the last developer to commit code to it. If you believe someone else in a better position to evaluate this decision, re-assign this issue to them.
If you don't have admin permissions on the repo and can't archive it. Re-assign this issue to @thraxil with a note to that effect.
If there has been no response to this issue by the end of sprint 89 (June 3rd), this repo will automatically be archived.

Archiving. I don't think we use t anymore.

authors: OmarIthawi thraxil

repo: appsembler/oppia-xblock


text:
Add front-end documentation for handling CORS errors:

Cross-domain or eval script error detected, error ignored

Thanks @matsimitsu, I tweaked this slightly and moved the content to a new troubleshooting page. If we can create a short URL for this, then I can add the link to the log message in the client itself. 👍

authors: matsimitsu xadamy

repo: appsignal/appsignal-docs


text:
[Bug]: False execute onPageload failure error

Is there an existing issue for this?

[X] I have searched the existing issues

Description
https://user-images.githubusercontent.com/23132741/171658531-41104916-422e-4bf8-8d27-38044e608162.mp4
Here, for the user, the onPageLoad API worked fine but also we noticed a false failure error.
Steps To Reproduce
The steps to reproduce aren't yet clear. The user's exported application is collected for reproduction.
Ping @Rishabh-Rathod  for the application access.
Public Sample App
No response
Version
1.7.0

duplicate of #14141

As this is the intermittent failure of onPageLoad which still will happen after https://github.com/appsmithorg/appsmith/pull/14210.
Reopening to track it separately.

#14210 may have fixed this issue since it hasn't been reported, hence closing this for now. We can reopen if this comes up again

authors: Rishabh-Rathod ramsaptami

repo: appsmithorg/appsmith


text:
[Bug]: Async functions with TypeErrors are not marked as Async in JS Editor

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When a JS Function has a TypeError, even if it's marked as async, it is not treated as one (doesn't show up in the settings tab).
Steps To Reproduce

Go to the JS Editor page
Create an async function with a typeError (cannot read x of undefined).

   myFun1: async () => {
		//write code here
		const user = {
		}
		console.log(user.name.test)
	}

or run an Api
   myFun1: () => {
		//write code here
		const user = {
		}
		console.log(user.name.test)
                Api1.run()
	}

Observe the settings tab, and see that these functions are not treated as async.

Public Sample App
No response
Version
Cloud

As we have removed the concept of async function tagging for JSObject. This issue is no more valid.


text:
[Bug]: In git connected app when user clicks on deploy button the the export event gets added in logs

Is there an existing issue for this?

[X] I have searched the existing issues

Description
In the git connected app when the user clicks on the deploy button the export event gets added in the logs
Steps To Reproduce

open a git-connected app
click on deploy button
now go to the audit log page
validate the latest event displayed

Public Sample App
No response
Version
https://release-audit-logs-test.appsmith.com/app/git-app/page3-633ea2eeedadae4525c9bd66

@RakshaKShetty This will be addressed in the issue - https://github.com/appsmithorg/appsmith/issues/15698

authors: AnaghHegde RakshaKShetty


text:
[Task]: Update secondary and tertiary states of button

Is there an existing issue for this?

[X] I have searched the existing issues

SubTasks
We currently have three states of button:
https://design-system.appsmith.com/?path=/story/design-system-button--primary
https://design-system.appsmith.com/?path=/story/design-system-button--secondary
https://design-system.appsmith.com/?path=/story/design-system-button--tertiary
We use our current secondary design in very few places.
We want to upgrade our current tertiary design to secondary and we will be introducing a new design for the tertiary button.
This will involve

[ ] renaming the existing design system components
[ ] changing the usages of tertiary in core to secondary
[ ]  making a list of pages that use secondary button and take a call on what to do with them
[ ] carrying out the same exercise in ee
[ ] implementing the new tertiary design
[ ] keeping the two branches in sync and merging successfully.

@tanvibhakta Here is the updated design for this. Figma

hey @bhuvanaindukuri, the changes are applicable for the default components as present in the design-system repository. Since this is something that requires coordination across multiple repositories, I will be picking this up. Let me know if there's something else you'd like to pick up?

authors: tanvibhakta vasanth-appsmith


text:
[Bug]: RBAC UI issues

Is there an existing issue for this?

[X] I have searched the existing issues

Description

The search component has a shadow.

Irrelevant tooltip observed for the Groups column in Users screen. Also this tooltip does not follow the design system

When we rename a new group and press Enter, the page refreshes but the name is not saved. It only save after we press Enter the second time.
https://www.loom.com/share/cbda20fcd75849eb841f32cba8eab70a

Dropdown for adding Users does not behave like it does everywhere else in the platform.
https://www.loom.com/share/39591cfd5bb043d798f36543a596920e

On hover cursor turns into a CTA even when the icon is not clickable.
https://www.loom.com/share/48d77545c90e4e338a654b7bf58951eb

Steps To Reproduce

Go to the Access Control page

Public Sample App
No response
Issue video log
No response
Version
Cloud

@shadabbuchh  For point 4, the dropdown showing 3 selected instead of tag names was part of the design. Plus it won't make sense for a user to scroll and read the content, instead, it's better we tell them how many options are selected and they can open the dropdown and check which ones are selected.
Will look into other things pointed out by your here.

Point 2 needs to be handled in design-system repo. Will do the needful.

@shadabbuchh For point 4, the positioning works fine on GSheets because the container is the full height of that page. While this doesn't work on the invite modal which cannot be of full height. This will require some help from the design-system team and this positioning of the dropdown on the invite modal is a known issue to the design-system team.
cc: @albinAppsmith

Verified the above issues in release. All seem to be working fine.
Will raise a separate issue for #4.

Closing this issue as only 1 issue is pending from the list this will be filed seperatly

authors: RakshaKShetty ankitakinger shadabbuchh


text:
[Bug]: User gets incorrect error messages while creating JS Objects

Is there an existing issue for this?

[X] I have searched the existing issues

Description
On trying to add JS Objects quickly, we see incorrect error messages indicating that the JS Object is already present when in reality it is not.
Please have a look at the video for more details
Steps To Reproduce

Login with the below credentials in release:
-username: ide-test@appsmith.com
-password: ide-test@appsmith.com
In an app after adding the 1st JS Object, add new JS Objects by clicking on + button.

Expected: The new JS Objects should be created successfully.
Actual: We see incorrect error messages.
Public Sample App
No response
Environment
Release
Severity
Medium (Frustrating UX)
Issue video log
https://www.loom.com/share/9fc8a612960c44cdaebcae60a7fbb9d5
Version
Cloud

Resolved by adding a loading state between JS creations

authors: hetunandu shadabbuchh


text:
[Bug] UX : On resizing the Multi select the data does not get truncated appropriately

Steps to reproduce:

Add a multi-select widget
Resize the widget (smaller or larger)
and observe the widget

Observation:
It is observed that the truncation for the option happens from the right which disturbs the truncation
Expectation:
It is expected that the data must be visible to the user

We need to give a min width to the multi-select popover

@somangshu It actually has a min width,
Couldn't reproduce this

authors: Tooluloope prapullac somangshu


text:
fix: switch datasource option list contains similar ds

Description
Switching Datasource on the query pane or api pane editor was showing non-similar datasources in dropdown. The list is now filtered using the pluginId of the current datasource with the other active datasources.
Fixes #13326
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Create an application
Add multiple datasources (some similar, some different)
Create a query or api for one of the above datasources
When the editor opens, go to switch datasource dropdown and click on it
You will now see a list of only similar datasources (earlier all the active datasources were listed)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

/ok-to-test sha=3cee6f3

/ok-to-test sha=3ede568

/ok-to-test sha=2b1ba41

/ok-to-test sha=3fb4bdb


text:
chore: update browser list

Description
While building the application, there was a warning about outdated browserlist. This PR updates that and removes that warning.
Media
Warning

No warning

Type of change

Chore (housekeeping or task changes that don't impact user perception)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions, so we can reproduce.
Please also list any relevant details for your test configuration.
Delete anything that is not important

Manual
Jest
Cypress

Test Plan

Add Testsmith test cases links that relate to this PR

Issues raised during DP testing

Link issues raised during DP testing for better visiblity and tracking (copy link from comments dropped on this PR)

Checklist:
Dev activity

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Test plan has been approved by relevant developers
[ ] Test plan has been peer reviewed by QA
[ ] Cypress test cases have been added and approved by either SDET or manual QA
[ ] Organized project review call with relevant stakeholders after Round 1/2 of QA
[ ] Added Test Plan Approved label after reveiwing all Cypress test

/ok-to-test sha=d8214606c6ffea3802bdff0ad83148efc1ec85f1

@mohanarpit please merge this


text:
Added testcase for tablefilter

This has steps to reproduce to add filter to table.

duplicate PR hence closing

authors: NandanAnantharamu prapullac


text:
chore: ME design audit CE

Pull Request Template
Use this template to quickly create a well written pull request. Delete all quotes before creating the pull request.

Description

Add a TL;DR when description is extra long (helps content team)
Please include a summary of the changes and which issue has been fixed. Please also include relevant motivation
and context. List any dependencies that are required for this change
Links to Notion, Figma or any other documents that might be relevant to the PR

PR fixes following issue(s)
Fixes # (issue number)

if no issue exists, please create an issue and ask the maintainers about this first

Media

A video or a GIF is preferred. when using Loom, don’t embed because it looks like it’s a GIF. instead, just link to the video

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Chore (housekeeping or task changes that don't impact user perception)
This change requires a documentation update

Testing

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Also list any relevant details for your test configuration.
Delete anything that is not relevant

[ ] Manual
[ ] Jest
[ ] Cypress

Test Plan

Add Testsmith test cases links that relate to this PR

Issues raised during DP testing

Link issues raised during DP testing for better visiblity and tracking (copy link from comments dropped on this PR)

Checklist:
Dev activity

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Speedbreak features have been covered
[ ] Test plan covers all impacted features and areas of interest
[ ] Test plan has been peer reviewed by project stakeholders and other QA members
[ ] Manually tested functionality on DP
[ ] We had an implementation alignment call with stakeholders post QA Round 2
[ ] Cypress test cases have been added and approved by SDET/manual QA
[ ] Added Test Plan Approved label after Cypress tests were reviewed
[ ] Added Test Plan Approved label after JUnit tests were reviewed

/build-deploy-preview skip-tests=true

/ok-to-test

/build-deploy-preview skip-tests=true

/build-deploy-preview skip-tests=true

/ok-to-test

/build-deploy-preview skip-tests=true

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

/ok-to-test

./ok-to-test

/ok-to-test

This PR was tested in EE PR and here is the comment for approval. Hence adding test plan label myself.


text:
chore: remove unused APPSMITH_TNC_PP env variable

Summary by CodeRabbit

Refactor

Removed configuration and references associated with enableTNCPP across various files.

Chores

Cleaned up legacy configuration elements from the application setup.

I'm approving this PR right now. Not sure how the functionality got removed. In the past, this variable would determine if we showed the Appsmith t&c and privacy policy for self-hosted branded instances. But right now, I don't see this variable being used anywhere and the lack of this functionality is clearly not causing any problems.
Hence, let's remove it from the code base. But I'm adding this comment as a note to our future selves about why the variable was added in the first place.


text:
Add java-openliberty stack to screenshots

Updated the "Initializing project" article to include the java-openliberty stack in appsody list output.

Good point. I'll update those now.

repo: appsody/website


text:
Firebase version removed

When the project has firebase versions defined, then when you import it to any project you must will have to use the exact firebase vversion..
So now by removing those versions, you can set up any of the firebase versions  you want to add.

Thank you!


text:
Problem with static server

run gulp
include some styl file into the build process
create a mistake in the file
try to continue the work :)

Current result: static server falls down and stops watching the files, if you break gulp work and run it again it won't help you, because static server process is detached. You need to stop the node process manually
Expected:

static server continues working if an error happens.
static server stops at the same time the gulp does.

It's a problem with your gulpfile. See solution here.

authors: apsavin jifeon

repo: apsavin/bnsf


text:
[Feature Request]: Add carousel feature in tourist guide section

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Currently in the tourist guide section single cards are there, I want to add a carousel feature in it as there might be more travel guides later so we can scroll through them.
Use Case
Creating the carousel will enhance the UI of the tourist guide section.
Benefits
We can add more tourist guides later. Also now just a single card in a row is taking too much space, so converting it into a carousel will make it look compact and better.
Add ScreenShots
Now
https://github.com/user-attachments/assets/cc2bcf16-2609-44e2-b1c1-90ae97227b7e
after changes
https://github.com/user-attachments/assets/4ea9ecf3-d513-4198-a25c-479a0769ff54
This kind of sliding  feature of carousel will be same but he cards will be the tourist guide section.
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[ ] I'm a VSOC'24 contributor
[X] I have starred the repository

@apu52 sir please assign me this issue.


text:
Issues with podman

aqua info
$ aqua info
{
  "version": "2.23.1",
  "commit_hash": "65fb59915fced5e16413e59cd17b3f0a1b42b972",
  "os": "linux",
  "arch": "amd64",
  "pwd": "/home/(USER)/Projects/github.com/aquaproj/aqua-registry",
  "root_dir": "/home/(USER)/.local/share/aquaproj-aqua",
  "env": {
    "AQUA_ROOT_DIR": "/home/(USER)/.local/share/aquaproj-aqua"
  },
  "config_files": [
    {
      "path": "/home/(USER)/Projects/github.com/aquaproj/aqua-registry/aqua.yaml"
    },
    {
      "path": "/home/(USER)/aqua.yaml"
    }
  ]
}

Overview
Trying to add a new package to aqua-registry with podman on Fedora 39:
$ rpm -qa | grep podman
podman-compose-1.0.6-3.fc39.noarch
podman-tui-0.17.0-1.fc39.x86_64
podman-4.9.3-1.fc39.x86_64
podman-docker-4.9.3-1.fc39.noarch
podman-plugins-4.9.3-1.fc39.x86_64

How to reproduce
Executed command and output
bhundven@mill:~/Projects/github.com/aquaproj/aqua-registry$ cmdx scaffold iamhsa/pkenv
+ set -eu
if [ "false" = true ]; then
  cmdx rm
fi
bash scripts/start.sh
bash scripts/scaffold.sh "iamhsa/pkenv" "" ""
bash scripts/test.sh "iamhsa/pkenv"
bash scripts/start.sh aqua-registry-windows
bash scripts/test-windows.sh "iamhsa/pkenv"

Error: no such object: "aquaproj/aqua-registry"
[INFO] Building the docker image aquaproj/aqua-registry
STEP 1/15: FROM golang:1.22.0-bookworm
Resolved "golang" as an alias (/home/bhundven/.cache/containers/short-name-aliases.conf)
Trying to pull docker.io/library/golang:1.22.0-bookworm...
Getting image source signatures
Copying blob 9ef5d6b83255 done   |
Copying blob 7bb465c29149 done   |
Copying blob 49b40be4436e done   |
Copying blob 2b9b41aaa3c5 done   |
Copying blob d85d001093ea done   |
Copying blob 07ba5748a600 done   |
Copying blob 4f4fb700ef54 done   |
Copying config 0c4ed86491 done   |
Writing manifest to image destination
STEP 2/15: WORKDIR /workspace
--> 018c7015ed79
STEP 3/15: ENV AQUA_ROOT_DIR=/root/aquaproj-aqua
--> 2dff6d7dd83c
STEP 4/15: ENV AQUA_LOG_COLOR=always
--> 5c603fa0e3af
STEP 5/15: ENV AQUA_POLICY_CONFIG=/workspace/aqua-policy.yaml
--> dbe90d979600
STEP 6/15: ENV PATH=$AQUA_ROOT_DIR/bin:/root/.cargo/bin:$PATH
--> a8e9f02dad4b
STEP 7/15: SHELL ["/bin/bash", "-o", "pipefail", "-c"]
WARN[0013] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 1acd37b17b28
STEP 8/15: RUN   apt-get update &&   apt-get install --no-install-recommends -y tree &&   apt-get clean &&   rm -rf /var/lib/apt/lists/*
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8786 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [12.7 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [143 kB]
Fetched 9196 kB in 2s (4648 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  tree
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 52.5 kB of archives.
After this operation, 116 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main amd64 tree amd64 2.1.0-1 [52.5 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 52.5 kB in 0s (482 kB/s)
Selecting previously unselected package tree.
(Reading database ... 15610 files and directories currently installed.)
Preparing to unpack .../tree_2.1.0-1_amd64.deb ...
Unpacking tree (2.1.0-1) ...
Setting up tree (2.1.0-1) ...
WARN[0016] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> eca620644cf5
STEP 9/15: RUN curl -sSfL -O https://raw.githubusercontent.com/aquaproj/aqua-installer/v2.3.0/aqua-installer
WARN[0017] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 882db4483464
STEP 10/15: RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal
info: downloading installer
info: profile set to 'minimal'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2024-02-08, rust version 1.76.0 (07dca489a 2024-02-04)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
info: installing component 'rustc'

info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'
  stable-x86_64-unknown-linux-gnu installed - rustc 1.76.0 (07dca489a 2024-02-04)

Rust is installed now. Great!

To get started you may need to restart your current shell.
This would reload your PATH environment variable to include
Cargo's bin directory ($HOME/.cargo/bin).

To configure your current shell, run:
source "$HOME/.cargo/env"
WARN[0026] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 76738c834599
STEP 11/15: RUN echo "1577b99b74751a5ddeea757198cee3b600fce3ef18990540e4d0e667edcf1b5f  aqua-installer" | sha256sum -c
aqua-installer: OK
WARN[0028] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 47dec0a6c277
STEP 12/15: RUN chmod +x aqua-installer
WARN[0029] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 9bc4742ee9ff
STEP 13/15: RUN ./aqua-installer -v v2.23.1
[INFO] Installing aqua v2.22.0 for bootstrapping...
[INFO] Downloading https://github.com/aquaproj/aqua/releases/download/v2.22.0/aqua_linux_amd64.tar.gz ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 6578k  100 6578k    0     0  5663k      0  0:00:01  0:00:01 --:--:-- 9344k
[INFO] Verifying checksum of aqua v2.22.0 ...
aqua_linux_amd64.tar.gz: OK
[INFO] /tmp/tmp.QWbmczWeXD/aqua update-aqua v2.23.1
INFO[0000] download and unarchive the package            aqua_version=2.22.0 env=linux/amd64 new_version=v2.23.1 package_name=aquaproj/aqua package_version=v2.23.1 program=aqua registry=
INFO[0000] verify a package with slsa-verifier           aqua_version=2.22.0 env=linux/amd64 new_version=v2.23.1 package_name=aquaproj/aqua package_version=v2.23.1 program=aqua registry=
INFO[0000] download and unarchive the package            aqua_version=2.22.0 env=linux/amd64 new_version=v2.23.1 package_name=slsa-framework/slsa-verifier package_version=v2.4.1 program=aqua registry=
Verified signature against tlog entry index 68699067 at URL: https://rekor.sigstore.dev/api/v1/log/entries/24296fb24b8ad77a769569930edd1e1cf2ea9486eb1da26a46df11bbb030527a972db52d4c34fc7d
Verified build using builder "https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v1.9.0" at commit 65fb59915fced5e16413e59cd17b3f0a1b42b972
Verifying artifact /tmp/088656240: PASSED

PASSED: Verified SLSA provenance
INFO[0004] create a symbolic link                        aqua_version=2.22.0 command=aqua env=linux/amd64 new_version=v2.23.1 package_name=aquaproj/aqua package_version=v2.23.1 program=aqua

===============================================================
[INFO] aqua is installed into /root/aquaproj-aqua/bin/aqua
[INFO] Please add the path to the environment variable "PATH"
[INFO] export PATH=${AQUA_ROOT_DIR:-${XDG_DATA_HOME:-$HOME/.local/share}/aquaproj-aqua}/bin:$PATH
===============================================================

aqua version 2.23.1 (65fb59915fced5e16413e59cd17b3f0a1b42b972)
WARN[0035] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> c18e135eaef4
STEP 14/15: COPY aqua-test.yaml aqua.yaml
WARN[0035] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 109c07ba539c
STEP 15/15: COPY aqua-policy.yaml aqua-policy.yaml
COMMIT aquaproj/aqua-registry
WARN[0036] SHELL is not supported for OCI image format, [/bin/bash -o pipefail -c] will be ignored. Must use `docker` format
--> 47c026dd5078
Successfully tagged localhost/aquaproj/aqua-registry:latest
47c026dd50781e2f310d61bece5bbbccdc06d153c865e79ac3e9149f89d3dc45
[INFO] Checking if the container aqua-registry exists
[INFO] Creaing a container aqua-registry
[INFO] Get a GitHub Access token by gh auth token
e9c7723e4cbba947612f16f6326bdcf0e239ddfa4f060f1ce7c20c1a75b76952
+ pkg=iamhsa/pkenv
+ cmd=
+ limit=
+ '[' -d pkgs/iamhsa/pkenv ']'
+ docker exec -ti -w /aqua-registry aqua-registry aqua policy allow
INFO[0000] no policy file is found                       aqua_version=2.23.1 env=linux/amd64 program=aqua
+ docker exec -ti -w /aqua-registry aqua-registry aqua i -l
INFO[0000] download and unarchive the package            aqua_version=2.23.1 env=linux/amd64 package_name=aqua-proxy package_version=v1.2.5 program=aqua registry=
INFO[0000] create a symbolic link                        aqua_version=2.23.1 command=aqua-proxy env=linux/amd64 package_name=aqua-proxy package_version=v1.2.5 program=aqua registry=
+ opts=
+ '[' -n '' ']'
+ '[' -n '' ']'
+ docker exec -ti -w /aqua-registry aqua-registry aqua-registry scaffold iamhsa/pkenv
Error: crun: executable file `aqua-registry` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
exit status 127

Debug output
Output isn't different with AQUA_LOG_LEVEL=debug set.
Expected behaviour
The new package is created in the aqua-registry
Actual behaviour
+ docker exec -ti -w /aqua-registry aqua-registry aqua-registry scaffold iamhsa/pkenv
Error: crun: executable file `aqua-registry` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found

Note
No response

I guess the issue might be related to the way podman names an image without hostname (like docker.io). After I run cmdx s <package> I get:
$ podman images
REPOSITORY                                             TAG              IMAGE ID      CREATED         SIZE
localhost/aquaproj/aqua-registry                       latest           55d991505793  41 minutes ago  1.45 GB

So probably cmdx instead expects aquaproj/aqua-registry where podman creates localhost/aquaproj/aqua-registry by the default naming settings?

So, for me:
$ cmdx con
+ bash scripts/connect.sh
[INFO] Connecting to the container aqua-registry (linux/amd64)
ERRO[0000] install the registry                          aqua_version=2.23.1 env=linux/amd64 error="local registry isn't found" local_registry_file_path=/workspace/registry.yaml program=aqua registry_name=standard
FATA[0000] aqua failed                                   aqua_version=2.23.1 env=linux/amd64 error="it failed to install some registries" program=aqua
exit status 1

Turned out it's not because the name is localhost/aquaproj/aqua-registry, but for Podman and Fedora it just needs --privilege to run the image. See my pull req #20473.

@tadayosi Thank you for looking into this issue!
📝
podman run --help

      --privileged                               Give extended privileges to container

https://docs.podman.io/en/latest/markdown/podman-run.1.html#privileged

Give extended privileges to this container. The default is false.
By default, Podman containers are unprivileged (=false) and cannot, for example, modify parts of the operating system. This is because by default a container is only allowed limited access to devices. A “privileged” container is given the same access to devices as the user launching the container, with the exception of virtual consoles (/dev/tty\d+) when running in systemd mode (--systemd=always).
A privileged container turns off the security features that isolate the container from the host. Dropped Capabilities, limited devices, read-only mount points, Apparmor/SELinux separation, and Seccomp filters are all disabled. Due to the disabled security features, the privileged field should almost never be set as containers can easily break out of confinement.
Containers running in a user namespace (e.g., rootless containers) cannot have more privileges than the user that launched them.

According to your error message, /workspace/registry.yaml wasn't found. https://github.com/aquaproj/aqua-registry/issues/20289#issuecomment-1973140916
But this file should be copied by docker cp command.
https://github.com/aquaproj/aqua-registry/blob/096332d6104bf59282397d1142821623b4a4d0cb/scripts/test.sh#L9
Did the command cmdx s output any error or warning before the above error?
--privilege option may resolve the issue, but I'd like to understand the cause correctly.
I can't reproduce the issue even if privileged is false on M3 Pro.
$ podman inspect aqua-registry | grep rivilege
               "Privileged": false,

@suzuki-shunsuke

I can't reproduce the issue even if privileged is false on M3 Pro.

Because it's a SELinux/Apparmor issue. I'm not sure how the podman machine (CoreOS) on Mac works in terms of SELinux but I guess it's disabled or permissive already?  This article is a good source for learning how podman with the privileged option is supposed to work [1].
[1] https://www.redhat.com/sysadmin/container-permission-denied-errors
When I run cmdx s without that --privileged, indeed the aqua-registry container is created and running. And as you said those files are copied under /workspace.
podman exec aqua-registry ls -l /workspace
total 16
-rwxr-xr-x. 1 root root 3469 Mar  1 11:56 aqua-installer
-rw-r--r--. 1 root root  242 Mar  2 06:36 aqua-policy.yaml
-rw-r--r--. 1 root root  229 Feb 29 10:00 aqua.yaml
-rw-r--r--. 1 root root   43 Mar  2 06:40 pkg.yaml

The issue is not that those files are not copied but aqua fails to read them without --privileged. Actually, when I run cmdx s after sudo setenforce 0 (disabling SELinux) it also works even without --privileged. So I think it's a SELinux issue and SELinux just prohibits accessing those files by default (probably because it's root owned?).

Containers running in a user namespace (e.g., rootless containers) cannot have more privileges than the user that launched them.

So even with privileged it shouldn't get more privileges than the host user so it would be still safer than Docker which runs with sudo.

Oh, I see.
Thank you for your kind explanation.

So even with privileged it shouldn't get more privileges than the host user so it would be still safer than Docker which runs with sudo.

Looks good.

authors: bhundven suzuki-shunsuke tadayosi

repo: aquaproj/aqua-registry


text:
S3 credentials for default user via existing secret

Configuration options: useExistingSecret and existingSecret.
These fields allow the user to specify an existing secret containing
the S3 credentials for the default user.
useExistingSecret is a boolean field defaulted to false.
existingSecret is a string field denoting a secret in the s3gw
namespace. It must contain 2 keys:

accessKey that is the S3 Access Key for the default user.
secretKey that is the S3 Secret Key for the default user.

When useExistingSecret is set to false (default), the chart will use
the values from the preexisting fields accessKey and secretKey.

Tip: you can create a preexisting secret with a command like this:
kubectl create secret generic -n s3gw s3gw-creds --from-literal=accessKey=myaccesskey --from-literal=secretKey=supersecret

Fixes: https://github.com/aquarist-labs/s3gw/issues/303
Signed-off-by: Giuseppe Baccini giuseppe.baccini@suse.com
Describe your changes
Issue ticket number and link
Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] If it is a core feature, I have added thorough tests.
[x] CHANGELOG.md has been updated should there be relevant changes in this PR.

There is an unexpected behavior when trying to use s3gw as subchart with this patch.
Please, do not merge.

@giubacc is this something that needs to be documented? if so, I suggest that we update the documentation too.

@giubacc is this something that needs to be documented? if so, I suggest that we update the documentation too.

sure, doc needs to be updated

authors: giubacc jhmarina


text:
Add the installed package version in the email/jira ticket

Currently in the JIRA ticket we show:
Resource name: libxml2

#
Name
Version
Fix version

1
CVE-2019-20388
none
2.9.10-r3

I would like to add the "Version" information under the resource name. This is the installed resource version.
It is available in the Webhook payload:
"resource": {
"format": "apk",
"name": "apk-tools",
"version": "2.10.0-r0",
"arch": "x86_64",
"cpe": "pkg:/alpine:3.8.0:apk-tools:2.10.0-r0",
"license": "GPL2",
"layer": "/bin/sh -c #(nop) ADD file:25f61d70254b9807a40cd3e8d820f6a5ec0e1e596de04e325f6a33810393e95a in / ",
"layer_digest": "sha256:8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6"
},

Opened by mistake

repo: aquasecurity/alm-integration


text:
Reuse postgres db conncetion instance

This commit includes one change and one addition:

Reuse the Postgres DB connection instead of closing and opening for each operation.
Connecting to a Postgres server with retry functionality.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Elad Zada seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant elad-da

repo: aquasecurity/postee


text:
Fix author argument for Permission Set resource

v0.8.12 produces the following error when running terraform validate if author argument is set for aquasec_permissions_sets:
│ Error: Computed attributes cannot be set
│
│   with module.common.aquasec_permissions_sets.team_contributor,
│   on ../../modules/common/main.tf line 64, in resource "aquasec_permissions_sets" "team_contributor":
│   64:   author = "terraform"
│
│ Computed attributes cannot be set, but a value was set for "author".

Looks like this argument was mistakenly changed from Optional: true to Computed: true in https://github.com/aquasecurity/terraform-provider-aquasec/commit/55dcadc2208bbdbd8c5b0738327125e91ae9cbfb (PR: https://github.com/aquasecurity/terraform-provider-aquasec/pull/144), as the documentation still says that this is an optional argument.
This PR changes the argument back to Optional: true and also reverts the relevant changes in resource_permission_set_test.go

Hi, @sivapalan please run the following command to generate doc on you forked branch:
go run github.com/hashicorp/terraform-plugin-docs/cmd/tfplugindocs
after that please push it.
because you updated the resource schema we can't merge the pr without it.

@yossig-aquasec Updated ✅

authors: sivapalan yossig-aquasec

repo: aquasecurity/terraform-provider-aquasec


text:
Jobs Complete But Pods are Left Behind

What steps did you take and what happened:
Often our vulnerability scanning pods will complete successfully and just stay in the cluster, not removing themselves like they are supposed to. When we have these scanners laying around it sometimes prevents new scans from happening because we have it set to run 3 at a time so as not to put too much pressure on the cluster itself. With the pods left over it will prevent further scanning because the config sees that they exist already.
What did you expect to happen:
We expected the pods to run and then terminate
Anything else you would like to add:
Here are some logs:
> kubectl describe po scan-vulnerabilityreport-75b7686948-jhbsl -n trivy-system
Name:         scan-vulnerabilityreport-75b7686948-jhbsl
Namespace:    trivy-system
Priority:     0
Node:         <node>/10.132.0.11
Start Time:   Fri, 15 Sep 2023 09:40:25 -0400
Labels:       app.kubernetes.io/managed-by=trivy-operator
              controller-uid=dc0aa08a-56b7-4a25-b691-ee783d258229
              job-name=scan-vulnerabilityreport-75b7686948
              resource-spec-hash=59f6d457db
              trivy-operator.resource.kind=ReplicaSet
              trivy-operator.resource.name=warehouses-av-pbpi4bgul00p-ingressgateway-7575b4cc64
              trivy-operator.resource.namespace=istio-system
              vulnerabilityReport.scanner=Trivy
Annotations:  cni.projectcalico.org/containerID: 5af2302ca2e01ba365953261bf4509a1b027d7e0b1eac57c2d21412c1eae4e83
              cni.projectcalico.org/podIP:
              cni.projectcalico.org/podIPs:
Status:       Succeeded
IP:           10.97.15.186
IPs:
  IP:           10.97.15.186
Controlled By:  Job/scan-vulnerabilityreport-75b7686948
Init Containers:
  f0d8315b-2717-40f2-9b50-742e0e833269:
    Container ID:  containerd://4a5051cb5dba683881595573cd01e0fe52c5dfa70c1a37ebc0b84053fbbd26b0
    Image:         ghcr.io/aquasecurity/trivy:0.42.0
    Image ID:      ghcr.io/aquasecurity/trivy@sha256:b75725d2c11ff54a5fe23f6e8b9a8c6177b8bf5221f08697cf0eed43442b1bfa
    Port:          <none>
    Host Port:     <none>
    Command:
      trivy
    Args:
      --cache-dir
      /tmp/trivy/.cache
      image
      --download-db-only
      --db-repository
      ghcr.io/aquasecurity/trivy-db
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 15 Sep 2023 09:40:26 -0400
      Finished:     Fri, 15 Sep 2023 09:40:31 -0400
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     750m
      memory:  750M
    Requests:
      cpu:     256m
      memory:  256M
    Environment:
      HTTP_PROXY:    <set to the key 'trivy.httpProxy' of config map 'trivy-operator-trivy-config'>   Optional: true
      HTTPS_PROXY:   <set to the key 'trivy.httpsProxy' of config map 'trivy-operator-trivy-config'>  Optional: true
      NO_PROXY:      <set to the key 'trivy.noProxy' of config map 'trivy-operator-trivy-config'>     Optional: true
      GITHUB_TOKEN:  <set to the key 'trivy.githubToken' in secret 'trivy-operator-trivy-config'>     Optional: true
    Mounts:
      /tmp from tmp (rw)
Containers:
  istio-proxy:
    Container ID:  containerd://cc59454fe9f3c4b56a6b7692a7ca148498179331e2563efeca9e27b39eee6563
    Image:         ghcr.io/aquasecurity/trivy:0.42.0
    Image ID:      ghcr.io/aquasecurity/trivy@sha256:b75725d2c11ff54a5fe23f6e8b9a8c6177b8bf5221f08697cf0eed43442b1bfa
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
    Args:
      -c
      trivy image --slow 'docker.io/istio/proxyv2:1.16.2' --scanners vuln   --skip-db-update --cache-dir /tmp/trivy/.cache --quiet --list-all-pkgs --format json > /tmp/scan/result_istio-proxy.json &&  bzip2 -c /tmp/scan/result_istio-proxy.json | base64
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 15 Sep 2023 09:40:34 -0400
      Finished:     Fri, 15 Sep 2023 09:40:46 -0400
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     750m
      memory:  750M
    Requests:
      cpu:     256m
      memory:  256M
    Environment:
      TRIVY_SEVERITY:            <set to the key 'trivy.severity' of config map 'trivy-operator-trivy-config'>          Optional: true
      TRIVY_IGNORE_UNFIXED:      <set to the key 'trivy.ignoreUnfixed' of config map 'trivy-operator-trivy-config'>     Optional: true
      TRIVY_OFFLINE_SCAN:        <set to the key 'trivy.offlineScan' of config map 'trivy-operator-trivy-config'>       Optional: true
      TRIVY_JAVA_DB_REPOSITORY:  <set to the key 'trivy.javaDbRepository' of config map 'trivy-operator-trivy-config'>  Optional: true
      TRIVY_TIMEOUT:             <set to the key 'trivy.timeout' of config map 'trivy-operator-trivy-config'>           Optional: true
      TRIVY_SKIP_FILES:          <set to the key 'trivy.skipFiles' of config map 'trivy-operator-trivy-config'>         Optional: true
      TRIVY_SKIP_DIRS:           <set to the key 'trivy.skipDirs' of config map 'trivy-operator-trivy-config'>          Optional: true
      HTTP_PROXY:                <set to the key 'trivy.httpProxy' of config map 'trivy-operator-trivy-config'>         Optional: true
      HTTPS_PROXY:               <set to the key 'trivy.httpsProxy' of config map 'trivy-operator-trivy-config'>        Optional: true
      NO_PROXY:                  <set to the key 'trivy.noProxy' of config map 'trivy-operator-trivy-config'>           Optional: true
    Mounts:
      /tmp from tmp (rw)
      /tmp/scan from scanresult (rw)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  scanresult:
    Type:        EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:   <unset>
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:          <none>

Environment:

Trivy-Operator version (use trivy-operator version): 0.15.1
Kubernetes version (use kubectl version):
OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc):

@jicunningham can you please share trivy-operator logs ?
in addition you can add TTL for jobs to clear it

@chen-keinan Here is what I am seeing from the trivy container that did the scanning:
2023-09-30T02:56:46.151Z	INFO	Need to update DB
2023-09-30T02:56:46.151Z	INFO	DB Repository: ghcr.io/aquasecurity/trivy-db
2023-09-30T02:56:46.151Z	INFO	Downloading DB...
20.28 MiB / 40.07 MiB [------------------------------>______________________________] 50.61% ? p/s ?37.78 MiB / 40.07 MiB [--------------------------------------------------------->___] 94.29% ? p/s ?40.07 MiB / 40.07 MiB [----------------------------------------------------------->] 100.00% ? p/s ?40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 32.95 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 32.95 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 32.95 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 30.83 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 30.83 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 30.83 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 28.84 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 28.84 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 28.84 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 26.98 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 26.98 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 26.98 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [-------------------------------------------------] 100.00% 14.25 MiB p/s 3.0s

As for the logs of the other containers (the ones scanned) all there exists is what looks like a large public/private/cert text block.
Currently, the TTL is set to 24 hours. Here is the status of some of them:

@chen-keinan Here is what I am seeing from the trivy container that did the scanning:
2023-09-30T02:56:46.151Z	INFO	Need to update DB
2023-09-30T02:56:46.151Z	INFO	DB Repository: ghcr.io/aquasecurity/trivy-db
2023-09-30T02:56:46.151Z	INFO	Downloading DB...
20.28 MiB / 40.07 MiB [------------------------------>______________________________] 50.61% ? p/s ?37.78 MiB / 40.07 MiB [--------------------------------------------------------->___] 94.29% ? p/s ?40.07 MiB / 40.07 MiB [----------------------------------------------------------->] 100.00% ? p/s ?40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 32.95 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 32.95 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 32.95 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 30.83 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 30.83 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 30.83 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 28.84 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 28.84 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 28.84 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 26.98 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 26.98 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [---------------------------------------------->] 100.00% 26.98 MiB p/s ETA 0s40.07 MiB / 40.07 MiB [-------------------------------------------------] 100.00% 14.25 MiB p/s 3.0s

As for the logs of the other containers (the ones scanned) all there exists is what looks like a large public/private/cert text block.
Currently, the TTL is set to 24 hours. Here is the status of some of them:

just to clarify the TTL flag I mention above is for jobs not report

@chen-keinan can this be reopened? Was there a solution?

@chen-keinan can this be reopened? Was there a solution?

try setting scanJobTTL param

Exists something similar for node-collector job? The node collector job finish succesfully, but the pod and the job remains in the cluster in complete state.
I'm using the helm chart, version 0.23.3

@FranAguiar do you see any errors in trivy-operator log ?

Hello @chen-keinan, yes, there is an error in the operator, I did not check it because the job was complete and I though everything was ok. This is the error:
{"level":"error","ts":"2024-06-13T09:08:41Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-6f7db594d8","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-6f7db594d8","reconcileID":"1ae4b06b-6299-461c-adfc-1e4c44ccbb85","error":"unexpected end of JSON input","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:222"}

@FranAguiar  can you please delete the node-collector job  and restart trivy-operator

Done, there is another error, specific for the node collector:
{"level":"error","ts":"2024-06-13T09:26:41Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"node-collector-765bcb57b","namespace":"trivy-system"},"namespace":"trivy-system","name":"node-collector-765bcb57b","reconcileID":"0f9aec59-4e0b-45bf-bef3-ca37e608a961","error":"failed to evaluate policies on Node : failed to run policy checks on resources","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:222"}

@FranAguiar can you please share your configmaps ?

Sure:

trivy-operator:

apiVersion: v1
data:
  compliance.failEntriesLimit: "10"
  configAuditReports.scanner: Trivy
  node.collector.imageRef: ghcr.io/aquasecurity/node-collector:0.2.1
  node.collector.nodeSelector: "true"
  nodeCollector.volumeMounts: '[{"mountPath":"/var/lib/etcd","name":"var-lib-etcd","readOnly":true},{"mountPath":"/var/lib/kubelet","name":"var-lib-kubelet","readOnly":true},{"mountPath":"/var/lib/kube-scheduler","name":"var-lib-kube-scheduler","readOnly":true},{"mountPath":"/var/lib/kube-controller-manager","name":"var-lib-kube-controller-manager","readOnly":true},{"mountPath":"/etc/systemd","name":"etc-systemd","readOnly":true},{"mountPath":"/lib/systemd/","name":"lib-systemd","readOnly":true},{"mountPath":"/etc/kubernetes","name":"etc-kubernetes","readOnly":true},{"mountPath":"/etc/cni/net.d/","name":"etc-cni-netd","readOnly":true}]'
  nodeCollector.volumes: '[{"hostPath":{"path":"/var/lib/etcd"},"name":"var-lib-etcd"},{"hostPath":{"path":"/var/lib/kubelet"},"name":"var-lib-kubelet"},{"hostPath":{"path":"/var/lib/kube-scheduler"},"name":"var-lib-kube-scheduler"},{"hostPath":{"path":"/var/lib/kube-controller-manager"},"name":"var-lib-kube-controller-manager"},{"hostPath":{"path":"/etc/systemd"},"name":"etc-systemd"},{"hostPath":{"path":"/lib/systemd"},"name":"lib-systemd"},{"hostPath":{"path":"/etc/kubernetes"},"name":"etc-kubernetes"},{"hostPath":{"path":"/etc/cni/net.d/"},"name":"etc-cni-netd"}]'
  policies.bundle.insecure: "false"
  policies.bundle.oci.ref: ghcr.io/aquasecurity/trivy-checks:0
  report.recordFailedChecksOnly: "true"
  scanJob.compressLogs: "true"
  scanJob.podTemplateContainerSecurityContext: '{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true}'
  vulnerabilityReports.scanner: Trivy
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: trivy-operator
    meta.helm.sh/release-namespace: trivy-system
  creationTimestamp: "2024-06-11T12:57:02Z"
  labels:
    app.kubernetes.io/instance: trivy-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: trivy-operator
    app.kubernetes.io/version: 0.21.3
    helm.sh/chart: trivy-operator-0.23.3
  name: trivy-operator
  namespace: trivy-system
  resourceVersion: "315642750"
  uid: b4e35399-80ba-4c1a-ac57-bd33c2e526a7
apiVersion: v1
data:
  compliance.failEntriesLimit: "10"
  configAuditReports.scanner: Trivy
  node.collector.imageRef: ghcr.io/aquasecurity/node-collector:0.2.1
  node.collector.nodeSelector: "true"
  nodeCollector.volumeMounts: '[{"mountPath":"/var/lib/etcd","name":"var-lib-etcd","readOnly":true},{"mountPath":"/var/lib/kubelet","name":"var-lib-kubelet","readOnly":true},{"mountPath":"/var/lib/kube-scheduler","name":"var-lib-kube-scheduler","readOnly":true},{"mountPath":"/var/lib/kube-controller-manager","name":"var-lib-kube-controller-manager","readOnly":true},{"mountPath":"/etc/systemd","name":"etc-systemd","readOnly":true},{"mountPath":"/lib/systemd/","name":"lib-systemd","readOnly":true},{"mountPath":"/etc/kubernetes","name":"etc-kubernetes","readOnly":true},{"mountPath":"/etc/cni/net.d/","name":"etc-cni-netd","readOnly":true}]'
  nodeCollector.volumes: '[{"hostPath":{"path":"/var/lib/etcd"},"name":"var-lib-etcd"},{"hostPath":{"path":"/var/lib/kubelet"},"name":"var-lib-kubelet"},{"hostPath":{"path":"/var/lib/kube-scheduler"},"name":"var-lib-kube-scheduler"},{"hostPath":{"path":"/var/lib/kube-controller-manager"},"name":"var-lib-kube-controller-manager"},{"hostPath":{"path":"/etc/systemd"},"name":"etc-systemd"},{"hostPath":{"path":"/lib/systemd"},"name":"lib-systemd"},{"hostPath":{"path":"/etc/kubernetes"},"name":"etc-kubernetes"},{"hostPath":{"path":"/etc/cni/net.d/"},"name":"etc-cni-netd"}]'
  policies.bundle.insecure: "false"
  policies.bundle.oci.ref: ghcr.io/aquasecurity/trivy-checks:0
  report.recordFailedChecksOnly: "true"
  scanJob.compressLogs: "true"
  scanJob.podTemplateContainerSecurityContext: '{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true}'
  vulnerabilityReports.scanner: Trivy
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: trivy-operator
    meta.helm.sh/release-namespace: trivy-system
  creationTimestamp: "2024-06-11T12:57:02Z"
  labels:
    app.kubernetes.io/instance: trivy-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: trivy-operator
    app.kubernetes.io/version: 0.21.3
    helm.sh/chart: trivy-operator-0.23.3
  name: trivy-operator
  namespace: trivy-system
  resourceVersion: "315642750"
  uid: b4e35399-80ba-4c1a-ac57-bd33c2e526a7

trivy-operator-policies-config

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: trivy-operator
    meta.helm.sh/release-namespace: trivy-system
  creationTimestamp: "2024-06-11T12:57:02Z"
  labels:
    app.kubernetes.io/instance: trivy-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: trivy-operator
    app.kubernetes.io/version: 0.21.3
    helm.sh/chart: trivy-operator-0.23.3
  name: trivy-operator-policies-config
  namespace: trivy-system
  resourceVersion: "315642749"
  uid: 9823e829-6931-4b1e-9dfb-16379711bac2

trivy-operator-trivy-config

apiVersion: v1
data:
  trivy.additionalVulnerabilityReportFields: ""
  trivy.command: image
  trivy.dbRepository: ghcr.io/aquasecurity/trivy-db
  trivy.dbRepositoryInsecure: "false"
  trivy.filesystemScanCacheDir: /var/trivyoperator/trivy-db
  trivy.ignoreUnfixed: "true"
  trivy.imagePullPolicy: IfNotPresent
  trivy.imageScanCacheDir: /tmp/trivy/.cache
  trivy.includeDevDeps: "false"
  trivy.javaDbRepository: ghcr.io/aquasecurity/trivy-java-db
  trivy.mode: Standalone
  trivy.repository: ghcr.io/aquasecurity/trivy
  trivy.resources.limits.cpu: 500m
  trivy.resources.limits.memory: 500M
  trivy.resources.requests.cpu: 100m
  trivy.resources.requests.memory: 100M
  trivy.sbomSources: ""
  trivy.severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
  trivy.skipJavaDBUpdate: "false"
  trivy.slow: "true"
  trivy.supportedConfigAuditKinds: Workload,Service,Role,ClusterRole,NetworkPolicy,Ingress,LimitRange,ResourceQuota
  trivy.tag: 0.52.0
  trivy.timeout: 5m0s
  trivy.useBuiltinRegoPolicies: "true"
  trivy.useEmbeddedRegoPolicies: "false"
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: trivy-operator
    meta.helm.sh/release-namespace: trivy-system
  creationTimestamp: "2024-06-11T12:57:02Z"
  labels:
    app.kubernetes.io/instance: trivy-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: trivy-operator
    app.kubernetes.io/version: 0.21.3
    helm.sh/chart: trivy-operator-0.23.3
  name: trivy-operator-trivy-config
  namespace: trivy-system
  resourceVersion: "315642748"
  uid: 314204f1-7780-4f33-8501-e35d1ecaf1b7

trivy-operator-config

apiVersion: v1
data:
  CONTROLLER_CACHE_SYNC_TIMEOUT: 5m
  OPERATOR_ACCESS_GLOBAL_SECRETS_SERVICE_ACCOUNTS: "true"
  OPERATOR_BATCH_DELETE_DELAY: 10s
  OPERATOR_BATCH_DELETE_LIMIT: "10"
  OPERATOR_BUILT_IN_TRIVY_SERVER: "false"
  OPERATOR_CACHE_REPORT_TTL: 120h
  OPERATOR_CLUSTER_COMPLIANCE_ENABLED: "true"
  OPERATOR_CLUSTER_SBOM_CACHE_ENABLED: "false"
  OPERATOR_CONCURRENT_NODE_COLLECTOR_LIMIT: "1"
  OPERATOR_CONCURRENT_SCAN_JOBS_LIMIT: "10"
  OPERATOR_CONFIG_AUDIT_SCANNER_ENABLED: "true"
  OPERATOR_CONFIG_AUDIT_SCANNER_SCAN_ONLY_CURRENT_REVISIONS: "true"
  OPERATOR_EXPOSED_SECRET_SCANNER_ENABLED: "true"
  OPERATOR_HEALTH_PROBE_BIND_ADDRESS: :9090
  OPERATOR_INFRA_ASSESSMENT_SCANNER_ENABLED: "true"
  OPERATOR_LOG_DEV_MODE: "false"
  OPERATOR_MERGE_RBAC_FINDING_WITH_CONFIG_AUDIT: "false"
  OPERATOR_METRICS_BIND_ADDRESS: :8080
  OPERATOR_METRICS_CLUSTER_COMPLIANCE_INFO_ENABLED: "false"
  OPERATOR_METRICS_CONFIG_AUDIT_INFO_ENABLED: "true"
  OPERATOR_METRICS_EXPOSED_SECRET_INFO_ENABLED: "true"
  OPERATOR_METRICS_FINDINGS_ENABLED: "true"
  OPERATOR_METRICS_IMAGE_INFO_ENABLED: "false"
  OPERATOR_METRICS_INFRA_ASSESSMENT_INFO_ENABLED: "false"
  OPERATOR_METRICS_RBAC_ASSESSMENT_INFO_ENABLED: "true"
  OPERATOR_METRICS_VULN_ID_ENABLED: "true"
  OPERATOR_PRIVATE_REGISTRY_SCAN_SECRETS_NAMES: '{}'
  OPERATOR_RBAC_ASSESSMENT_SCANNER_ENABLED: "true"
  OPERATOR_SBOM_GENERATION_ENABLED: "true"
  OPERATOR_SCAN_JOB_RETRY_AFTER: 30s
  OPERATOR_SCAN_JOB_TIMEOUT: 5m
  OPERATOR_SCAN_JOB_TTL: 1m
  OPERATOR_SCANNER_REPORT_TTL: 24h
  OPERATOR_SEND_DELETED_REPORTS: "false"
  OPERATOR_VULNERABILITY_SCANNER_ENABLED: "true"
  OPERATOR_VULNERABILITY_SCANNER_SCAN_ONLY_CURRENT_REVISIONS: "true"
  OPERATOR_WEBHOOK_BROADCAST_CUSTOM_HEADERS: ""
  OPERATOR_WEBHOOK_BROADCAST_TIMEOUT: 30s
  OPERATOR_WEBHOOK_BROADCAST_URL: ""
  TRIVY_SERVER_HEALTH_CHECK_CACHE_EXPIRATION: 10h
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: trivy-operator
    meta.helm.sh/release-namespace: trivy-system
  creationTimestamp: "2024-06-11T12:57:02Z"
  labels:
    app.kubernetes.io/instance: trivy-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: trivy-operator
    app.kubernetes.io/version: 0.21.3
    helm.sh/chart: trivy-operator-0.23.3
  name: trivy-operator-config
  namespace: trivy-system
  resourceVersion: "315751696"
  uid: 68bda532-8194-44ac-b552-9d6bcf7da757

can you do a quick test and switch the following params value to:
  trivy.useBuiltinRegoPolicies: "false"
  trivy.useEmbeddedRegoPolicies: "true"

and let me know if you get an error ?

Same error
{"level":"error","ts":"2024-06-13T10:06:18Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"node-collector-6496488658","namespace":"trivy-system"},"namespace":"trivy-system","name":"node-collector-6496488658","reconcileID":"65cb6c3c-3df5-419f-a5c7-b9444bdf5b3c","error":"failed to evaluate policies on Node : failed to run policy checks on resources","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:222"}

strange, config looks ok and should work with both config ways

I use the chart, version 0.23.3. Installed with terraform. My settings below
namespace     = "trivy-system"
chart_name    = "trivy-operator"
chart_version = "0.23.3"
repository    = "https://aquasecurity.github.io/helm-charts"
chart_values = [
  {
    name  = "serviceMonitor.enabled",
    value = true
  },
  {
    name  = "trivy.ignoreUnfixed",
    value = true
  },
  {
    name  = "operator.scanJobTTL",
    value = "1m"
  },
  {
    name = "operator.metricsVulnIdEnabled"
    value = true
  },
  {
    name = "operator.metricsExposedSecretInfo"
    value = true
  },
  {
    name = "operator.metricsConfigAuditInfo"
    value = true
  },
  {
    name = "operator.metricsRbacAssessmentInfo"
    value = true
  },
]

Some how it is working, I have the reports in prometheus/grafana, the only issue is with the node-collector pod

sure, it is not depend,  you are missing clusterInfraAssessment reports and compliance reports because of it

I tried both, but I will start with those for now. Maybe the pod job is an issue in this version, Do you think it worth try an older version?

I do not think so, as we have tests who check it and release has passed.
one think I could suggest is to completely delete trivy-operator include crds and re-install is again
helm uninstall trivy-operator -n  trivy-system

    kubectl delete crd vulnerabilityreports.aquasecurity.github.io
    kubectl delete crd exposedsecretreports.aquasecurity.github.io
    kubectl delete crd configauditreports.aquasecurity.github.io
    kubectl delete crd clusterconfigauditreports.aquasecurity.github.io
    kubectl delete crd rbacassessmentreports.aquasecurity.github.io
    kubectl delete crd infraassessmentreports.aquasecurity.github.io
    kubectl delete crd clusterrbacassessmentreports.aquasecurity.github.io
    kubectl delete crd clustercompliancereports.aquasecurity.github.io
    kubectl delete crd clusterinfraassessmentreports.aquasecurity.github.io
    kubectl delete crd sbomreports.aquasecurity.github.io
    kubectl delete crd clustersbomreports.aquasecurity.github.io
    kubectl delete crd clustervulnerabilityreports.aquasecurity.github.io

Tried that, same behaviour.
I saw that the node collector can be disabled. What does it do? It's optional?

its just a way  to assign it to specific node.
if you want to disable node-collector , you can configure this

I configured that, but node collector still running and keep forever after complete

@FranAguiar you'll have to delete the all node-collector job set the flag and restart trivy-operator

Already tried that, the job reappears, run and complete. This what I see in the trivy-operator logs
{"level":"error","ts":"2024-06-19T11:33:22Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-7c49c8f64f","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-7c49c8f64f","reconcileID":"e2ee6e6e-4640-4cff-a810-2dc4ec56911a","error":"unexpected end of JSON input","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:222"}

Helm chart 0.23.3 and GKE Version: v1.30.1-gke.1261000

@FranAguiar stange it should not reconcile nodes if  InfraAssessmentScannerEnabled is disabled

InfraAssessmentScannerEnabled is enabled by default, I disabled it and now the pod and the job do not stay.

InfraAssessmentScannerEnabled is enabled by default, I disabled it and now the pod and the job do not stay.

thanks for the update, this is workaround, still need to investigate the root cause in your env.

I enabled InfraAssessmentScanner again because all metrics where gone. With the same settings as above I have the below errors in the operator pod
{"level":"error","ts":"2024-06-24T08:26:36Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"node-collector-98d7c6d45","namespace":"trivy-system"},"namespace":"trivy-system","name":"node-collector-98d7c6d45","reconcileID":"78ba1f93-229f-4846-8eb0-f978a7b13d14","error":"failed to evaluate policies on Node : failed to run policy checks on resources","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-06-24T07:42:44Z","msg":"Reconciler error","controller":"job","controllerGroup":"batch","controllerKind":"Job","Job":{"name":"scan-vulnerabilityreport-cf59d7fff","namespace":"trivy-system"},"namespace":"trivy-system","name":"scan-vulnerabilityreport-cf59d7fff","reconcileID":"143546e1-c0cc-433b-b949-12a490dc09c5","error":"unexpected end of JSON input","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.3/pkg/internal/controller/controller.go:222"}

Anything else I can provide?

which trivy-operator version are you using?

Latest helm chart
NAME                             	CHART VERSION	APP VERSION
aquasecurity/trivy-operator      	0.23.3       	0.21.3

authors: FranAguiar chen-keinan jicunningham

repo: aquasecurity/trivy-operator


text:
OperatorHub.io installation documentation is invalid

When I follow the install instructions on operatorhub.io, it says to execute kubectl create -f https://operatorhub.io/install/trivy-operator/trivy-operator.yaml. But this fails because Requested channel does not exists or there is no default channel in operator package. Please update the install instructions.

@Jeansen have you followed trivy-operator docs on

@chen-keinan I know about the trivy docs. And I already have trivy operator running in my cluster. But this issue is about invalid information provided on operatorhub.io ;-)

@Jeansen I see, I'll have a look

authors: Jeansen chen-keinan


text:
build: upgrade yarn and node engines

Summary

This PR upgrades Node.js and Yarn engines
Added

Changed

node.js engine: 20.11.0
yarn engine: 4.0.2

Deprecated

Removed

Fixed

How to test

Odd - does Vercel build this?

Looks good to me!
On a short note, I use fnm so perhaps in the future, we could add an option for fnm in installing node

@acald-creator very nice, please feel free to make a PR for it!

authors: jsosulska sbolel

repo: aquia-inc/template-vite-react


text:
indexer: Implement package indexer for maven

Implements https://github.com/quay/claircore/issues/236.

This is still WIP and based on https://github.com/quay/claircore/pull/203. This will be opened up for upstream review once https://github.com/quay/claircore/pull/203 lands on the upstream claircore.

repo: arajkumar/claircore


text:
How to disable http interface?

When i run arangodb in my server, I need disable the http interface,I don't want to use 'server.disable-authentication' ....and ....how to do ?

In general, its not easy possible to disable the Interface. However, the recommended way is to restrict the access to it:
https://docs.arangodb.com/GeneralHttp/index.html#Authentication
You could also remove the route to it from the _routing system collection:
db._query("FOR route IN _routing FILTER route.url == '/_admin/html' REMOVE route in _routing");
db._query("FOR route IN _routing FILTER route.url == '/_admin/html/index.html' REMOVE route in _routing");
db._query("FOR route IN _routing FILTER route.url == '/' REMOVE route in _routing");

and delete the app reference:
db._query("FOR app IN _apps FILTER app.mount == '/_admin/aardvark' REMOVE route in _routing");

These may however be recreated on update.

@dothebart     I really appreciate your help.

Did this work out for you so we can close it?

authors: dothebart realdog

repo: arangodb/arangodb


text:
Unable to configue rocksdb in

my environment running ArangoDB
I'm using the latest ArangoDB of the respective release series:

[ ] 2.8
[ ] 3.0
[ ] 3.1
[ X] 3.2
[ ] self-compiled devel branch

Mode:

[ ] Cluster
[ X] Single-Server

Storage-Engine:

[ ] mmfiles
[X ] rocksdb

On this operating system:

[ ] DCOS on

[ ] AWS
[ ] Azure
[ X] own infrastructure

[X ] Linux

[ ] Debian .deb
[ ] Ubuntu .deb
[ ] SUSE   .rpm
[ ] RedHat .rpm
[ X] Fedora .rpm
[ ] Gentoo
[ ] docker - official docker library
[ ] other:

[ ] Windows, version:
[ ] MacOS, version:

I installed ArangoDB 3.2 on FC26 using the provided RPM.
I set
storage-engine = rocksdb

in arangod.conf (instead of auto).
ArangoDB won't start with this particular change
● arangodb3.service - ArangoDB database server
   Loaded: loaded (/usr/lib/systemd/system/arangodb3.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2017-07-31 17:23:07 CEST; 3min 35s ago
  Process: 28571 ExecStart=/usr/sbin/arangod --uid arangodb --gid arangodb --pid-file /var/run/arangodb3/arangod.pid --temp.path /var/tmp/arangodb3 --
  Process: 28710 ExecStartPre=/usr/sbin/arangod --uid arangodb --gid arangodb --pid-file /var/run/arangodb3/arangod.pid --server.rest-server false --d
  Process: 28709 ExecStartPre=/usr/bin/chmod 700 /var/lib/arangodb3-apps (code=exited, status=0/SUCCESS)
  Process: 28708 ExecStartPre=/usr/bin/chown -R arangodb:arangodb /var/lib/arangodb3-apps (code=exited, status=0/SUCCESS)
  Process: 28707 ExecStartPre=/usr/bin/chmod 700 /var/lib/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 28706 ExecStartPre=/usr/bin/chown -R arangodb:arangodb /var/lib/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 28705 ExecStartPre=/usr/bin/chmod 700 /var/log/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 28704 ExecStartPre=/usr/bin/chown -R arangodb:arangodb /var/log/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 28703 ExecStartPre=/usr/bin/install -g arangodb -o arangodb -d /var/run/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 28702 ExecStartPre=/usr/bin/install -g arangodb -o arangodb -d /var/tmp/arangodb3 (code=exited, status=0/SUCCESS)
 Main PID: 28572 (code=exited, status=0/SUCCESS)

Jul 31 17:23:07 dev2.zopyx.com systemd[1]: arangodb3.service: Unit entered failed state.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: arangodb3.service: Failed with result 'exit-code'.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: arangodb3.service: Service hold-off time over, scheduling restart.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: Stopped ArangoDB database server.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: arangodb3.service: Start request repeated too quickly.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: Failed to start ArangoDB database server.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: arangodb3.service: Unit entered failed state.
Jul 31 17:23:07 dev2.zopyx.com systemd[1]: arangodb3.service: Failed with result 'exit-code'.

If you had installed ArangoDB with the mmfiles engine (or picked auto), it will already have created a database directory suitable for the mmfiles engine.
When you restart it, it will insist on the selected storage engine being the mmfiles engine then. Changing the storage engine then will make arangod refuse to start.
A quote from the manual:

One storage engine type is supported per server per installation. Live switching of storage engines on already installed systems isn't supported. Configuring the wrong engine (not matching the previously used one) will result in the server refusing to start. You may however use auto to let ArangoDB choose the previously used one.

There is still a way to switch the storage engine later on:
you can either pick a different database directory for rocksdb, so when ArangoDB starts up, it will not find an mmfiles directory. So it will happily create a new directory for the rocksdb engine.
If you have already stored some data in the mmfiles version, you should use arangodump to export it from the mmfiles version of ArangoDB so you can later restore it when the rocksdb version is up and running.
If there is no data in the mmfiles instance that you want to keep, the other option is to stop arangod, remove all contents of the database directory, switch the storage engine to rocksdb in the configuration and restart it. But please be warned that this option will REMOVE ALL DATA in ArangoDB, so use it with care!

Did the above suggestions help?

Can I ask what value you also have written in var/lib/arangodb3/ENGINE ?

In his case the ENGINE file would contain mmfiles.
It was irretation about the choice of the storage engine - so following up a slack conversation we can close this now.

authors: dothebart jsteemann juliusMoreno zopyx