text:
Missing ClusterComplianceReport

What steps did you take and what happened:
Install via helm
[A clear and concise description of what the bug is, and what commands you ran.]
After I noticed the trivy pod had restarted several times, I checked the logs and found these lines:
{"level":"error","ts":1666794056.989362,"logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"ClusterComplianceReport.aquasecurity.github.io","error":"no matches for kind \"ClusterComplianceReport\" in version \"aquasecurity.github.io/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1.1\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/source/source.go:139\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:235\nk8s.io/apimachinery/pkg/util/wait.WaitForWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:662\nk8s.io/apimachinery/pkg/util/wait.poll\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:596\nk8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:547\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/source/source.go:132"} {"level":"error","ts":1666794065.0388808,"logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"ClusterComplianceDetailReport.aquasecurity.github.io","error":"no matches for kind \"ClusterComplianceDetailReport\" in version \"aquasecurity.github.io/v1alpha1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1.1\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/source/source.go:139\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:235\nk8s.io/apimachinery/pkg/util/wait.WaitForWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:662\nk8s.io/apimachinery/pkg/util/wait.poll\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:596\nk8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext\n\t/home/runner/go/pkg/mod/k8s.io/apimachinery@v0.25.2/pkg/util/wait/wait.go:547\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/source/source.go:132"}
What did you expect to happen:
The operator pod to keep running and not restarting
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Environment:

Trivy-Operator version (use trivy-operator version): 0.4.0
Kubernetes version (use kubectl version):
Client Version: v1.25.3
Kustomize Version: v4.5.7
Server Version: v1.22.12-gke.2300
OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc):

@jicunningham thanks for reporting , is this fresh install or upgrade (from starboard)?  its really weird  as we removed ClusterComplianceReport from trivy-operator from the beginning

@jicunningham thanks will remove it for now
btw : we are woking these days to introduce it back , so keep watching our repo.

authors: chen-keinan jicunningham

repo: aquasecurity/trivy-operator


text:
Update Plugin Artifacts Links for v0.185.6

Automated changes by create-pull-request GitHub action

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

source: gharchive/pull-request

repo: aquasecurity/trivy-plugin-aqua

license_type: permissive

license_source: github-api


text:
Names and addresses are parsed without an error

Some name formats are successfully parsed as a valid date
Some address formats are successfully parsed as a valid date

An working example can be found here. I also copied it here for reference.
package main

import (
	"fmt"
	"time"

	"github.com/araddon/dateparse"
)

func main() {
	inputs := []string{
		// names
		"Wayne, Bruce",

		// addresses
		"Miami, Florida",

		// for some bizarre reason this fails to parse (that's good)
		"Doe, John",
	}

	for _, i := range inputs {
		dt, err := dateparse.ParseIn(i, time.FixedZone("", 0))
		fmt.Printf("input: %v\n", i)
		fmt.Printf("date:  %v\n", dt)
		fmt.Printf("error: %v\n\n", err)
	}
}

input: Wayne, Bruce
date:  0000-01-01 00:00:00 +0000 +0000
error: <nil>

input: Miami, Florida
date:  0000-01-01 00:00:00 +0000 +0000
error: <nil>

input: Doe, John
date:  0001-01-01 00:00:00 +0000 UTC
error: parsing time "Doe, John" as "Mon, John": cannot parse "Doe, John" as "Mon"

I have a somewhat similar issue which I think is related to this, in that if a string is only partially parsed, no error is emitted.
For example, I tried to parse 2019-09-01_01:00:00 (using ParseStrict), which ended up getting parsed as 2019-01-01 00:00:00 +0000 (only the year got parsed).
If there were unknown data in the string, I would have expected some sort of error to be thrown. As it is there is no way to tell if the date was properly recognized, which is important when handling human input.

Pretty much any string that satisfies the regex ^(\w+)(,\w+)+$gets parsed as a valid date. We are having to exempt out that regex to use this library.

source: gharchive/issue

authors: phemmer uditrana yonivy

repo: araddon/dateparse

license: mit

license_source: bigquery


text:
arangoimp issue and a general question

my environment running ArangoDB
I'm using the latest ArangoDB of the respective release series: 3.1
Mode: Single-Server
On this operating system: Windows, version: 10
My problem is:
When I open arangoimp.exe file it asks me to enter a password, when it is entered and I press (Enter) it will automatically be closed... So what is the solution ?
My another question is:
Can I have the result of the AQL query by graph ?

arangoimp is meant to be invoked in an interactive shell (like cmd, bash or powershell)
You can use arangoexport to dump parts of a graph:
https://docs.arangodb.com/3.3/Manual/Administration/Arangoexport.html

Hi @mohamadHassanHijab
Did you manage to invoke arangoimp ?
Please send us a feedback
Best

Hi @OmarAyo ,
Yes, It works and I manage to invoke arangoimp.

Perfect !
I'm going to close this ticket, note that tickets can be reopened at any time.
Best

authors: OmarAyo dothebart mohamadHassanHijab

repo: arangodb/arangodb


text:
don't run a full-fledged AQL query to remove an AQL UDF by key

Scope & Purpose
Simplify the removal code for AQL UDFs from running an AQL query to a single-key remove operation

[ ] :hankey: Bugfix
[ ] :pizza: New feature
[x] :hammer: Refactoring
[ ] :book: CHANGELOG entry made
[ ] :muscle: The behavior in this PR was manually tested
[x] :computer: The behavior change can be verified via automatic tests

Backports:

[x] No backports required

Testing & Verification

[x] This change is already covered by existing tests, such as shell_server, shell_client.

http://172.16.10.101:8080/view/PR/job/arangodb-matrix-pr/11953/

Tests blue


text:
Agency hasAsXXX using std::optional

Scope & Purpose
Old versions of hasAsXXX returned a std::pair<XXX, bool> where the first member is only safe to access if the second member is true. Most placed did not check that. I replaced all of this with std::optional<XXX> and accessed the value using value(). value() throws an exception if the optional does not contain a value. Furthermore all the hasAsXXX methods used some operator() to obtain a node given a path. This method always threw an exception when the node did not exist. Since there are many keys that the supervision loop regularly accesses that intentionally do not exist, there are a lot of exceptions. Debugging with catch throw becomes impossible. This PR removed all those exceptions by returning a std::nullopt instead.

[ ] :hankey: Bugfix (requires CHANGELOG entry)
[ ] :pizza: New feature (requires CHANGELOG entry, feature documentation and release notes)
[x] :fire: Performance improvement
[x] :hammer: Refactoring/simplification
[ ] :book: CHANGELOG entry made

Backports:

[x] No backports required
[ ] Backports required for: (Please specify versions and link PRs)

Testing & Verification
(Please pick either of the following options)

[x] This change is a trivial rework / code cleanup without any test coverage.
[ ] The behavior in this PR was manually tested
[x] This change is already covered by existing tests, such as all Supervision and Cluster Tests.
[ ] This PR adds tests that were used to verify all changes:

[ ] Added new C++ Unit tests
[ ] Added new integration tests (e.g. in shell_server / shell_server_aql)
[ ] Added new resilience tests (only if the feature is impacted by failovers)

[ ] There are tests in an external testing repository:
[ ] I ensured this code runs with ASan / TSan or other static verification tools

https://jenkins.arangodb.biz/job/arangodb-matrix-pr/15840/

https://jenkins.arangodb.biz/job/arangodb-matrix-pr/15991/

https://jenkins.arangodb.biz/job/arangodb-matrix-pr/16000/

https://jenkins.arangodb.biz/job/arangodb-matrix-pr/16020/

https://jenkins.arangodb.biz/job/arangodb-matrix-pr/16041/

https://jenkins.arangodb.biz/job/arangodb-matrix-pr/16379/


text:
[CINFRA] Adjust transaction tests for replication2 default

Scope & Purpose
This is a sequel of https://github.com/arangodb/arangodb/pull/20232
More transaction tests have been adapted to support replication2. The main reasons are:

In replication2 the arangodb_dropped_followers_total metrics is not used
Intermediate commits works slightly different

We'll have to make a decision regarding intermediate commits on followers. In replication1, these are enabled by default, whereas in replication2 followers never perform them unless they are performed on the leader. See CINFRA-866 for more details.

[ ] :hankey: Bugfix
[ ] :pizza: New feature
[ ] :fire: Performance improvement
[x] :hammer: Refactoring/simplification

The red test on windows cannot be caused by this PR.

authors: apetenchea mchacki


text:
[3.4] Upgrade procedure to persist LocalDocumentIds in MMFiles

Should form the second half of a solution to https://github.com/arangodb/release-3.4/issues/74 (along with https://github.com/arangodb/arangodb/pull/6870). There are no automated upgrade tests yet, but I will be working to update https://github.com/arangodb/arangodb/pull/3767 in the next few days so that we can add them.

http://jenkins01.arangodb.biz:8080/job/arangodb-matrix-pr/1810/

Superseded by #7133

authors: danielhlarkin jsteemann


text:
fix buffer overrun, remove unused variable

Fix buffer overrun

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/2006/


text:
Add possible mode values to PUT /_admin/cluster/maintenance/<server-id>

Description

The Web UI description for this endpoint doesn't specify the possible mode values:

Whereas a similar endpoint (PUT /_admin/cluster/maintenance) does indeed specify possible values for mode (on & off)
Added a sentence to save users an extra click/search for its documentation page
Upstream PRs

3.10:
3.11:
3.12:

Hmm now I'm wondering why the requestBody snippet is not present in the Rest API section for this endpoint:
https://github.com/arangodb/docs-hugo/blob/b74d7ef33e470580f3be825395c20d8d77fe603d/site/content/3.10/develop/http-api/cluster.md?plain=1#L359-L378
We can close this PR if rendering the requestBody is the preferred solution, imo that is probably cleaner

I'm aware that these values are also present under the Schema section in the Request Body header so feel free to close this PR if that is deemed sufficient

makes senses! closing 👍

repo: arangodb/docs-hugo


text:
Updating docs

Small updates to the contributing page and copyright

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.01%. Comparing base (ed35c9d) to head (00ee103).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #330   +/-   ##
=======================================
  Coverage   98.01%   98.01%
=======================================
  Files          26       26
  Lines        4231     4231
=======================================
  Hits         4147     4147
  Misses         84       84

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: apetenchea codecov-commenter

repo: arangodb/python-arango

license: MIT


text:
PLY vertex size mismatch, expected 248 but file had 15

Error message when trying to convert a PLY created from metascan app.

I was trying to open a normal PLY.

same error!

@yujia-mt this would mean the PLY splat file is of different format than expected. But hard to say how/what, without having the file to look at.

authors: aras-p luis-paolino yujia-mt

repo: aras-p/UnityGaussianSplatting


text:
🛑 video pruebas is down

In 554fe1d, video pruebas (https://ipfs.io/ipfs/QmXbgJEuzsKSKuF8gUCjKd5J3WCRNSP7fXfuMh6C3WMF64) was down:

HTTP code: 502
Response time: 3 ms

Resolved: video pruebas is back up in a3ffa35.

repo: aratan/Monitorizaci-n


text:
openai.error.APIConnectionError when asking a question to a document

In the development environment, as set up using the instructions here, when asking a question to a document an openai.error.APIConnectionError occurs. To be complete, it's a POST request to http://localhost:7091/api/answer in which the error occurs. See Traceback below.
The cause seems to be the clashing of the variable name OPENAI_API_BASE used in DocsGPT and the variable name used by the openai Python package. Another project had the same issue, see here.
This can be solved simply by renaming OPENAI_API_BASE as used in the DocsGPT code to another name, like OPENAI_AZURE_API_BASE.
I already made the changes locally and will make a PR, as other people seem to run into the same issue, see:
https://discord.com/channels/1070046503302877216/1070605825112035399/1118266133305376869
https://discord.com/channels/1070046503302877216/1070605825112035399/1118296369883512864
Error Traceback:
Traceback (most recent call last):
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/openai/api_requestor.py", line 596, in request_raw
    result = _thread_context.session.request(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/requests/sessions.py", line 486, in prepare_request
    p.prepare(
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '/engines/text-embedding-ada-002/embeddings': No scheme supplied. Perhaps you meant https:///engines/text-embedding-ada-002/embeddings?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/rik/projects/DocsGPT2/application/app.py", line 326, in api_answer
    result = run_async_chain(chain, question, chat_history)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/projects/DocsGPT2/application/app.py", line 109, in run_async_chain
    answer = loop.run_until_complete(async_generate(chain, question, chat_history))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.pyenv/versions/3.11.4/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/rik/projects/DocsGPT2/application/app.py", line 100, in async_generate
    result = await chain.arun({"question": question, "chat_history": chat_history})
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/chains/base.py", line 300, in arun
    return (await self.acall(args[0], callbacks=callbacks, tags=tags))[
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/chains/base.py", line 198, in acall
    raise e
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/chains/base.py", line 192, in acall
    await self._acall(inputs, run_manager=run_manager)
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/chains/conversational_retrieval/base.py", line 144, in _acall
    docs = await self._aget_docs(new_question, inputs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/chains/conversational_retrieval/base.py", line 195, in _aget_docs
    docs = await self.retriever.aget_relevant_documents(question)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/vectorstores/base.py", line 410, in aget_relevant_documents
    docs = await self.vectorstore.asimilarity_search(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/vectorstores/base.py", line 199, in asimilarity_search
    return await asyncio.get_event_loop().run_in_executor(None, func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.pyenv/versions/3.11.4/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/vectorstores/faiss.py", line 303, in similarity_search
    docs_and_scores = self.similarity_search_with_score(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/vectorstores/faiss.py", line 244, in similarity_search_with_score
    embedding = self.embedding_function(query)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/embeddings/openai.py", line 316, in embed_query
    embedding = self._embedding_func(text, engine=self.deployment)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/embeddings/openai.py", line 282, in _embedding_func
    return embed_with_retry(
           ^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/embeddings/openai.py", line 64, in embed_with_retry
    return _embed_with_retry(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.pyenv/versions/3.11.4/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.pyenv/versions/3.11.4/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/langchain/embeddings/openai.py", line 62, in _embed_with_retry
    return embeddings.client.create(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/openai/api_resources/embedding.py", line 33, in create
    response = super().create(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
                           ^^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/openai/api_requestor.py", line 288, in request
    result = self.request_raw(
             ^^^^^^^^^^^^^^^^^
  File "/home/rik/.virtualenvs/DocsGPT2/lib/python3.11/site-packages/openai/api_requestor.py", line 609, in request_raw
    raise error.APIConnectionError(
openai.error.APIConnectionError: Error communicating with OpenAI: Invalid URL '/engines/text-embedding-ada-002/embeddings': No scheme supplied. Perhaps you meant https:///engines/text-embedding-ada-002/embeddings?
127.0.0.1 - - [08/Jul/2023 17:37:59] "POST /api/answer HTTP/1.1" 500 -
Error communicating with OpenAI: Invalid URL '/engines/text-embedding-ada-002/embeddings': No scheme supplied. Perhaps you meant https:///engines/text-embedding-ada-002/embeddings?

Solved in #278
Thank you!

authors: RikSchoonbeek dartpain


text:
🛑 ARCaptcha widget is down

In 75fb2d1, ARCaptcha widget (https://widget.arcaptcha.co/1/api.js) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ARCaptcha widget is back up in 9d81a7c after 9 minutes.

repo: arcaptcha/ARCaptcha-monitoring


text:
Add new methods and options to ArchethicRPCClient and TransactionBuilder

This pull request adds new functionality to the ArchethicRPCClient and TransactionBuilder classes.
ArchethicRPCClient

Added a new method refreshCurrentAccount() to refresh the current account.

TransactionBuilder

Added a new option generateEncryptedSeedSC to allow the generation of an encrypted seed for smart contracts.

Added a new method setGenerateEncryptedSeedSC() to set the generateEncryptedSeedSC option.

Fixes #160

Thank you, I took the liberty to forward it to the TransactionRPC struct used by the send_transaction function.

authors: Rudy-Perrin bchamagne

repo: archethic-foundation/libjs


text:
Find function on "Find/Replace" form don't use "Case Sensitive" option with cirillic.

Version of Archi

4.9.2
Archi Plug-ins

Archi Colloboration 0.8.0.2021110121448
Archi Scripting 1.2.0.2021110121451
Lightbox for Archi 1.1.1.20210807
Operating System

Win 10
Expected Behaviour

Find function on "Find/Replace" form don't use "Case Sensitive" option with cirillic.
Actual Behaviour

Positive scen.
I have two Bussiness Actor elements: "Customer" and "customer".
If "Case Sensitive" options hasn't activate on "Find/Replace" form and I searching "customer" or "Customer" then both element will be find.
If "Case Sensitive" options has activate on "Find/Replace" form and I searching "customer" or "Customer"  then one element element will be find.
Negative scen.
I have two Bussiness Actor elements: "Клиент" and "клиент".
If "Case Sensitive" options hasn't activate on "Find/Replace" form and I searching "клиент" or "Клиент" then one element element will be find.
If "Case Sensitive" options has activate on "Find/Replace" form and I searching "клиент" or "Клиент"  then one element element will be find.

Notes for investigation:
com.archimatetool.editor.ui.findreplace.AbstractFindReplaceProvider
Possible change to regex pattern in method getSearchStringPattern:
if(!isCaseSensitive()) {
    searchString = "(?i:" + searchString + ")";
}

Will be fixed for next version.

Fixed in next version

authors: Phillipus margolyesh

repo: archimatetool/archi


text:
Added complete new filebase, creatig pull request to control

finna bouta M ERGE

Merging now

repo: archishou/Control


text:
Adding support for macros to create new Lambdas

This PR is my first step in solving https://github.com/architect/architect/issues/1051, and I think is a good starting point for a discussion and review of supporting macros (or whatever we want to call this supercharged-macros construct) in other core arc packages like create, hydrate, logs, and sandbox.
My prototype is purely additive and backwards compatible with the existing macro construct. As discussed in https://github.com/architect/architect/issues/1051, macros can hook in to arc functionality outside of augmenting the CloudFormation JSON by defining specific functions on its module.exports.
The changes to inventory involve:

Adding a new inventory "pragma" called macromodules

inventory now encapsulates finding and requireing the macro (which I was planning on re-using across all core arc packages, including the one place this logic has thus far existed in deploy) - see the src/config/pragmas/macromodules.js file in this PR for that.

Modifying lambdaSrcDirs and lambdaBySrcDir such that macros that "register" new lambdas (via the newly supported module.exports.create handler macros can now export) get picked up by inventory. This is necessary to get support for macro-created lambdas in hydrate and logs.

It doesn't feel like it 'fits' very well in this way. One alternative I considered: modifying the inventory output so that inventory.inv.macros no longer returns a simple array of macro names, but rather is an object that maps macro names to the macro module required in. However, that may break stuff? So I settled on adding a new property: inventory.inv.macromodules.
Let me know your thoughts @brianleroux and @ryanblock !

One more thought here: looking at how my lambda-creating macro for AWS IoT Rule Topics is shaping up (take a look here: https://github.com/copperinc/macro-iot-rules/blob/plugins/index.js), you kind of have to know the internals of inventory in order to be able to set up the CloudFormation JSON appropriately. Stuff you need to know as a macro author includes:

How to read and compile arc's config.arc for lambdas to pull in any runtime, memory, concurrency, etc. customizations appropriately - which, in my macro, I slapped together into a helper method
Using inventory's `createLambda' helper to put together the CloudFormation JSON object representing the lambda resource.
Naming lambda resources a specific way in order to have them be picked up by @architect/logs (at least, with my upcoming change to logs, this will be the case, following the conventions used in logs where the name must end in ${type}Lambda and I will be adding Macro as a type).

Given all of this, I would also suggest wrapping all of this up into a new helper function in inventory so that macro authors don't have to worry about the specifics (and ensuring we use this same method for built-in arc pragmas that create lambda resources).

One more thought here: looking at how my lambda-creating macro for AWS IoT Rule Topics is shaping up (take a look here: https://github.com/copperinc/macro-iot-rules/blob/plugins/index.js), you kind of have to know the internals of inventory in order to be able to set up the CloudFormation JSON appropriately. Stuff you need to know as a macro author includes:

How to read and compile arc's config.arc for lambdas to pull in any runtime, memory, concurrency, etc. customizations appropriately - which, in my macro, I slapped together into a helper method
Using inventory's `createLambda' helper to put together the CloudFormation JSON object representing the lambda resource.
Naming lambda resources a specific way in order to have them be picked up by @architect/logs (at least, with my upcoming change to logs, this will be the case, following the conventions used in logs where the name must end in ${type}Lambda and I will be adding Macro as a type).

Given all of this, I would also suggest wrapping all of this up into a new helper function in inventory so that macro authors don't have to worry about the specifics (and ensuring we use this same method for built-in arc pragmas that create lambda resources).

Closing this PR as a new RFC is up over at architect/architect#1062. Will probably end up re-using a bunch of this stuff in a new PR down the road.

Closing this PR as a new RFC is up over at architect/architect#1062. Will probably end up re-using a bunch of this stuff in a new PR down the road.

repo: architect/inventory


text:
Typography cssEllipsis 若初始化未触发超出省略则始终不能触发，无法自适应

[ ] I'm sure this does not appear in the issue list of the repository

Basic Info

Package Name And Version: @arco-design/web-react@2.44.0
Browser: chrome109.0.0.0
Reproduction Link: https://codesandbox.io/s/angry-flower-sfi8gw

What is expected?
初始化未触发溢出时，自适应窗口也能正常展示溢出省略号

https://codesandbox.io/s/sharp-bose-f97ip5?file=/src/App.js

@PengJiyuan

authors: ITrokie liumengyuangit yuhan0709

repo: arco-design/arco-design


text:
Issue with x.* versions

Getting NU1100 errors for all package references that use *:
Unable to resolve 'FastEndpoints (>= 5.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'mediatr (>= 12.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'MudBlazor (>= 7.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'tngtech.archunitnet.xunit (>= 0.10.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'xunit (>= 2.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'coverlet.collector (>= 6.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'FastEndpoints (>= 5.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'mediatr (>= 12.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'MudBlazor (>= 7.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'MudBlazor (>= 7.0.0)' for 'net8.0/browser-wasm'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'FastEndpoints.Security (>= 5.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'FastEndpoints (>= 5.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'FastEndpoints.Swagger (>= 5.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'MudBlazor (>= 7.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.
Unable to resolve 'mediatr (>= 12.0.0)' for 'net8.0'. PackageSourceMapping is enabled, the following source(s) were not considered: C:\Program Files\dotnet\library-packs, LocalNuget, Microsoft Visual Studio Offline Packages, nuget.org.

Hey @ardalis I'm looking into this. What version of the template and .Net SDK are you using? Also, could you share the cli commands you used to instantiate the template that produced those errors?

Looking at the error message the issue is caused by the PackageSourceMapping configuration which discards nuget.org as a source in that environment. If the package is not present in any of the mapped configured packages the restore will fail.

A workaround is to specify the specific versions that are present in the nuget sources of that particular environment.
Or I could reverse the change specifying a wildcard for the minor versions. The downside to this is that newer minor versions and patches will need to be updated from time to time manually to allow new template instantiations with  updated updated.

dotnet --version
9.0.100-preview.7.24407.12

I'd just freshly installed the templates.

It's probably better for the templates to work consistently (and maybe require a manual nuget update afterward) than for them to not work at all (but be "up to date").
Also if there's a way to support Directory.Packages.props out of the box with the templates that would be ideal. Probably worth a separate issue for that if there isn't already one.

I figured out the issue here - it's a sourcemapping problem not something with the project (helps if I read the error message)...
If fixed my issues with SourceMapping and now everything here:
https://github.com/ardalis/modulith/blob/main/Test-UiWithDotnetCli.ps1
works fine on my machine. :)

authors: ardalis david-acm

repo: ardalis/modulith


text:
can't delete/update doctors and rooms

Can't delete/update doctors and rooms. Once i added a private default constructor to the entities, it started to work.

Good catch. Want to make a pull request or shall I just fix it?

authors: ardalis florinbadea

repo: ardalis/pluralsight-ddd-fundamentals


text:
Library Manager: No log update for wolfSSL?

Describe the problem
As described in the FAQ:

For example, the Servo library is hosted at https://github.com/arduino-libraries/Servo, so its logs are at:
http://downloads.arduino.cc/libraries/logs/github.com/arduino-libraries/Servo/

The servo library was last updated on April 1. I'm creating this issue on April 2.

However, even yesterday on April 1, the wolfSSL library did not indicate an update before I published a new release:

It was 3 days ago that I published the latest wolfSSL release for Arduino
Today, there's still no update to the wolfssl log, however the registration has been completed as the latest 5.7.0 wolfssl is found in the JSON file and available in the application:

I'm fairly new to the Arduino publishing, so I'm not sure if I missed a step, or if there's genuinely a problem with the logging.
To reproduce
View wolfSSL log and observe there's been no recent update.
Expected behavior
I would expect the log to be refreshed on a regular basis to indicate success or failure.
Additional context
No response
Issue checklist

[X] I searched for previous reports in the issue tracker
[X] My report contains all necessary details

@per1234 I'm going to close this issue.
The problem persisted for some time, then quietly went away.
I didn't make any changes, not sure if you saw anything odd with the process.
I've kept an eye on it, and the log seems to always be fresh.

ok, so no sooner than me closing the issue, the problem has resurfaced.
I've not made any changes since publishing wolfSSL a few weeks ago.
See the servo example vs wolfssl and the current date.

This seems to still be a problem today. There's a wolfSSL 5.7.2 update in https://github.com/wolfSSL/Arduino-wolfSSL/pull/8 that does not seem to be processed by the indexer.
Today I created a new tag, but noticed it has been quite some time since wolfSSL was processed, as compared for example to the servo library:

repo: arduino/library-registry


text:
ADBDEV 6061 Add test for   doRestoreAgent in gpbackup

This PR proposes to add mocking interfaces and unit test for the gprestore's
doRestoreAgent() function.
When we look at the gprestore code, and into doResoreAgent() in particular,
there is not so trivial logic which might be difficult to cover with the
end-to-end and integration tests. From the other side this logic relies on
numerous external dependencies which makes unit testing nearly impossible.
This commit proposes using of well-known mocking technique for unit testing of
gpbackup code with an example based on grprestore and doResoreAgent(). With
interface mocking it becomes possible to:

extend test coverage with more specific use cases
make testing closed to the golang code, resulting in fast tests with no
external dependencies, tests will be more stable.
make test logic cleaner compared to using opaque backup archives and using of
many programs

However, this approach only complements existing end-to-end and integration
tests, it's not intended to replace them. Also it's hardly depend on mock
interfaces clarity quality, it's easy to achieve opposite effects.
The idea for mocking is to replace complex objects with their test imitation.
Unfortunately, gpbackup is written in solid manner, and it's not possible to
just replace built-in functions. To mitigate it, internal dependencies are split
with introducing of interfaces for dependencies external related to code under
test. Production code is using production interface implementation, while in
testing code we use test implementation, which does do only things related to
the test and return values exactly for the test scenario.
This commit adds the following interfaces:

Helper  - an abstraction for functions located in helper/helper.go
RestoreHelper - an abstraction for functions located in
helper/restore_helper.go

This is a minimum change for the supplied tests. It's expected that when more
methods or interfaces will be needed they will be created of necessity. The
changes for the interface introduction are minimal and straightforward.
Test logic is located in the helper/helper_test.go.

I think there are too many changes for one test.

I think there are too many changes for one test.

it is still relevant

Please fix description according to requirements (I see that at least 80 char wrapping requirement is not met).

The same should be done for doBackupAgent()

Please fix description according to requirements (I see that at least 80 char wrapping requirement is not met).

the description should also contain summary line in the beginning (refer to checklist item 2.2)

in the description:
'doResoreAgent()' -> 'doRestoreAgent()'

in the description:
'grprestore' -> 'gprestore'

in the description:
'Also it's hardly depend on mock'
->
'Also it's hardly depending on mock' or
'Also it hardly depends on mock'
And overall sentence 'Also it's hardly depend on mock interfaces clarity quality, it's easy to achieve opposite effects.' is a bit bulky, better to rephrase it.

in the description:
'make testing closed' -> 'make testing closer'

in the description:
This commit adds the following interfaces:

* Helper  - an abstraction for functions located in helper/helper.go
* RestoreHelper - an abstraction for functions located in
  helper/restore_helper.go

if we list all changes, then need to mention RestoreReader.

The same should be done for doBackupAgent()

I agree but I believe it shall be done as a separate task.

The patch for all helper tests is mocked taking into account the steps for skip files. But what if I want to write a test not for skip files?

This section of description should be aligned with the code:

This commit adds the following interfaces:
Helper - an abstraction for functions located in helper/helper.go
RestoreHelper - an abstraction for functions located in
helper/restore_helper.go
RestoreReader - an abstraction for file reader used for restoring data

Fix new conflicts, please

bender build

authors: RekGRpth dkovalev1 whitehawk

repo: arenadata/gpbackup


text:
[WIP] ADBDEV-1938

When we ask GPDB to explain a query, it uses flow to count rows number, and it can be either a single segment command, acommand can be executed on all segments or a limited number of segments in general cases. But there are cases when the flow is not generated, so GPDB uses segments count by default, what can be incorrect for some nodes, which are executed on a single segment only, for example.
DROP TABLE IF EXISTS lab1;
CREATE TABLE lab1 (id int, descr text) WITH (appendonly=true) DISTRIBUTED BY (id);
INSERT INTO lab1 SELECT generate_series(1,5000) AS id, md5(random()::text) AS descr;
explain analyze select t1.id, row_number() over (order by t1.id desc) from lab1 t1 join lab1 t2 using(descr);

If we explain this query, we will get an output like this:
   Order By: lab1.id
   ->  Gather Motion 2:1  (slice2; segments: 2)  (cost=0.00..8179.15 rows=25000000 width=4) (actual time=7877.092..12694.123 rows=25000000 loops=1)
         Merge Key: lab1.id
         ->  Sort  (cost=0.00..7730.15 rows=12500000 width=4) (actual time=7874.529..9583.758 rows=12730000 loops=1)
               Sort Key: lab1.id
               Sort Method:  external merge  Disk: 195520kB
               ->  Hash Join  (cost=0.00..1046.52 rows=12500000 width=4) (actual time=6.390..1543.301 rows=12730000 loops=1)
                     Hash Cond: (lab1.descr = lab1_1.descr)
                     Extra Text: (seg1)   Hash chain length 5000.0 avg, 5000 max, using 1 of 131072 buckets.
                     ->  Seq Scan on lab1  (cost=0.00..431.07 rows=2500 width=37) (actual time=0.180..2.948 rows=2546 loops=1)
                     ->  Hash  (cost=435.54..435.54 rows=5000 width=33) (actual time=4.338..4.338 rows=5000 loops=1)
                           ->  Broadcast Motion 2:2  (slice1; segments: 2)  (cost=0.00..435.54 rows=5000 width=33) (actual time=0.084..2.978 rows=5000 loops=1)
                                 ->  Seq Scan on lab1 lab1_1  (cost=0.00..431.07 rows=2500 width=33) (actual time=0.167..0.803 rows=2546 loops=1)
 Planning time: 24.088 ms
   (slice0)    Executor memory: 232K bytes.
   (slice1)    Executor memory: 204K bytes avg x 2 workers, 204K bytes max (seg0).
 * (slice2)    Executor memory: 67558K bytes avg x 2 workers, 67558K bytes max (seg0).  Work_mem: 65758K bytes max, 612237K bytes wanted.
 Memory used:  128000kB
 Memory wanted:  1837210kB
 Optimizer: Pivotal Optimizer (GPORCA)
 Execution time: 20194.755 ms
(22 rows)

As we see, WindowAgg has 12500000 rows, when its nested plan contains 25000000 rows, and WindowAgg has a number of rows less than expected exaclty the number of segments, what is caused by missing flow.
This patch doesn't rely on flow, it uses gangSize of a current slice which is calculated correctly and always presents. gangSize is a simple and reliable way to get a number of segments which processed some part of a plan.
However, orca has another bug related with how it calculates rows number. The specialty of orca is that it can return fractional number of rows, and explain should ceil them later. It can lead to a situation when orca returns a wrong rows number, explain ceils it and we get an incorrect answer. For example, such query
create table tcorr1(a int, b int);
create table tcorr2(a int, b int);

insert into tcorr1 values (1,99);
insert into tcorr2 values (1,1);

explain
select *
from tcorr1 out
where out.b in (select coalesce(tcorr2.a, 99)
                from tcorr1 left outer join tcorr2 on tcorr1.a=tcorr2.a+out.a);

will get a plan like this
                                               QUERY PLAN
--------------------------------------------------------------------------------------------------------
 Result  (cost=0.00..1356692026.74 rows=1 width=8)
   Filter: (SubPlan 1)
   ->  Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..431.00 rows=1 width=8)
         ->  Seq Scan on tcorr1  (cost=0.00..431.00 rows=1 width=8)
   SubPlan 1  (slice0)
     ->  Result  (cost=0.00..1324032.56 rows=2 width=4)
           ->  Nested Loop Left Join  (cost=0.00..1324032.56 rows=2 width=4)
                 Join Filter: (tcorr1_1.a = (tcorr2.a + tcorr1.a))
                 ->  Materialize  (cost=0.00..431.00 rows=1 width=4)
                       ->  Gather Motion 3:1  (slice3; segments: 3)  (cost=0.00..431.00 rows=1 width=4)
                             ->  Seq Scan on tcorr1 tcorr1_1  (cost=0.00..431.00 rows=1 width=4)
                 ->  Materialize  (cost=0.00..431.00 rows=1 width=4)
                       ->  Gather Motion 3:1  (slice2; segments: 3)  (cost=0.00..431.00 rows=1 width=4)
                             ->  Seq Scan on tcorr2  (cost=0.00..431.00 rows=1 width=4)
 Optimizer: Pivotal Optimizer (GPORCA)
(15 rows)

As we see, nested loop join has two rows when it can have only one. This happens because orca return 1.2 rows for this node and explain ceils this number to 2.

@Stolb27 I suppose, as far as the idea itself belongs to you, it is you, who should finish up the work :)

@Stolb27 I suppose, as far as the idea itself belongs to you, it is you, who should finish up the work :)

@dnskvlnk, please check failed test and update it, if it's needed.

@dnskvlnk, please fix one more failure connected with @red1452's patch merged earlier.

repo: arenadata/gpdb


text:
DO NOT MERGE / Ci test 2826

Here are some reminders before you submit the pull request

[ ] Add tests for the change
[ ] Document changes
[ ] Communicate in the mailing list if needed
[ ] Pass make installcheck
[ ] Review a PR in return to support the community

Failed job Build for x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1304147

Failed job Deploy multiarch Dockerimages: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1304149

Allure report https://allure.adsw.io/launch/69432


text:
UI Issues: unfixed chat icon, footer/navbar design and Responsiveness.

What's wrong with the existing documentation

The chat icon appears to be unfixed in its position.
The design of footer and navbar could be improved. they might lack of visual appeal or clarity.
The website seems to be encountering responsiveness issues, particularly with the footer.
Some images on the website appear to be very large, which can significantly slow down loading time and negatively impact.

Add ScreenShots
No response
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

Sir please assign this task to me.


text:
add code snippets to the UI to prepare_for_training with basic frameworks spacy,transformers

Is your feature request related to a problem? Please describe.
For users, it might sometimes be difficult to assess the way training can be integrated with Argilla.
Describe the solution you'd like
I would like something that I can copy-paste and use directly in a simple setting for all mayor libraries.
from argilla.training import transformers_trainer

transformers_trainer(
    name="my-dataset",
    model_name = "prajjwal1/bert-tiny"
).train()

import argilla as rg
from transformers import  AutoTokenizer

model_name = "prajjwal1/bert-tiny"
ds = rg.load("my-dataset", query="annotated_as: *")
ds_train = ds.prepare_for_training()

# tokenize our datasets
tokenizer = AutoTokenizer.from_pretrained(
   model_name
)

def tokenize_function(examples):
    return tokenizer(examples["text"], padding="max_length", truncation=True, max_length=256)

tokenized_train_ds = ds_train.map(tokenize_function, batched=True)

# split the data into a training and evaluation set
train_dataset, eval_dataset = tokenized_train_ds.train_test_split(
    test_size=0.2, seed=42
).values()

Describe alternatives you've considered
N.A.
Additional context
N.A.

https://github.com/argilla-io/argilla/issues/2438

repo: argilla-io/argilla


text:
ClusterDecisionResource support for labels.

Current functionality
In the ClusterDecisionResource generator, we find the decision resource using the resources name.
Problem
There will be cases where the name for this resource may come from metadata.generateName, where we don't want to always be updating the ApplicationSet spec (such as a gitops flow).  There are also cases where one might desire to select multiple decision resources to combine the results.
Proposed solution
Add metav1.labelSelector, as a field in the clusterDecisionResource generator, so one can use name or labelSelector.matchLabels or labelSelector.matchExpressions

I believe this is now completed, since #272 has merged.

authors: jgwest jnpacker

repo: argoproj-labs/applicationset


text:
chore: Upgrade Go version to 1.18 and update dependencies

Upgrade Go to 1.18
Update few dependencies

Signed-off-by: jannfis jann@mistrust.net

Codecov Report

Merging #523 (9f2b690) into master (f96b134) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 9f2b690 differs from pull request most recent head 9485ab2. Consider uploading reports for the commit 9485ab2 to get more accurate results

@@           Coverage Diff           @@
##           master     #523   +/-   ##
=======================================
  Coverage   65.70%   65.70%
=======================================
  Files          21       21
  Lines        2041     2041
=======================================
  Hits         1341     1341
  Misses        571      571
  Partials      129      129

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

authors: codecov-commenter jannfis

repo: argoproj-labs/argocd-image-updater


text:
Bad Request: template 'clone' type is unknown

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issues exists when I tested with :latest
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?
Hi, I am new to Argo workflow and created a workflow template, but submitting a workflow gives me an error.
can anyone help here?
Version
latest
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: cicd-workflow-template
  namespace: argo
spec:
  arguments:
    parameters:
      - name: branch
        value: master
      - name: dockerRepo
        value: ***

  entrypoint: clone

  podGC:
    strategy: OnPodSuccess

  volumeClaimTemplates:
    - metadata:
        name: work
      spec:
        accessModes: [ ReadWriteOnce ]
        resources:
          requests:
            storage: 64Mi

  templates:
    - name: clone
      inputs:
        artifacts:
        - name: argo-source
          path: /src
          git:
            repo: ********
            usernameSecret:
              name: bitbucket-creds
              key: username
            passwordSecret:
              name: bitbucket-creds
              key: password
            singleBranch: true
            branch: "master"

    - name: docker-build
      volumes:
        - name: docker-config
          secret:
            secretName: registrykey
      container:
        image: moby/buildkit:v0.9.3-rootless
        volumeMounts:
          - mountPath: /go/src/github.com/go-api
            name: work
            subPath: src
          - name: docker-config
            mountPath: /.docker
        workingDir: /go/src/github.com/go-api
        env:
          - name: BUILDKITD_FLAGS
            value: --oci-worker-no-process-sandbox
          - name: DOCKER_CONFIG
            value: /.docker
        command:
          - buildctl-daemonless.sh
        args:
          - build
          - --frontend
          - dockerfile.v0
          - --local
          - context=.
          - --local
          - dockerfile=docker/
          - --output
          - type=image,name={{workflow.parameters.dockerRepo}},push=true
        securityContext:
          privileged: true

Logs from the workflow controller
me="2023-02-15T20:28:54.728Z" level=info msg="Update leases 200"
time="2023-02-15T20:28:59.734Z" level=info msg="Get leases 200"
time="2023-02-15T20:28:59.740Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:00.461Z" level=info msg="List workflows 200"
time="2023-02-15T20:29:00.461Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2023-02-15T20:29:04.745Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:04.757Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:09.762Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:09.778Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:14.785Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:14.791Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:19.800Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:19.807Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:24.813Z" level=info msg="Get leases 200"

Logs from in your workflow's wait container
me="2023-02-15T20:28:54.728Z" level=info msg="Update leases 200"
time="2023-02-15T20:28:59.734Z" level=info msg="Get leases 200"
time="2023-02-15T20:28:59.740Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:00.461Z" level=info msg="List workflows 200"
time="2023-02-15T20:29:00.461Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2023-02-15T20:29:04.745Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:04.757Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:09.762Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:09.778Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:14.785Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:14.791Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:19.800Z" level=info msg="Get leases 200"
time="2023-02-15T20:29:19.807Z" level=info msg="Update leases 200"
time="2023-02-15T20:29:24.813Z" level=info msg="Get leases 200"

Hey @shuhaib3. Thank you for filing the issue. Where are you seeing template clone type is unknown? Is it when you submit via the CLI or the UI? Additionally, are you able to successfully lint the WorkflowTemplate that you are submitting using argo lint?

I submit the workflow via the CLI then return "rpc error: code = InvalidArgument desc = template 'clone' type is unknown",  It turns out that there is a problem with the workflow

@sunze I think you have a syntax error in your workflow, your clone template is incomplete, you have specified the name and inputs but there's no template type.
ref: https://argoproj.github.io/argo-workflows/workflow-templates/

closing this as there's an issue in the submitted workflow template and this is not a bug

repo: argoproj/argo-workflows


text:
Argo incorrectly buffers logging from containerized Python

Is this a BUG REPORT or FEATURE REQUEST?:
BUG REPORT
What happened:

Deployed a simple containerized Python subpar with argo submit workflow.yaml (subpar aspect is probably irrelevant)
Ran argo logs -f <OUTPUT NAME>
Ran kubectl logs -f <OUTPUT NAME> main

What you expected to happen:
Argo would stream the log output of my workflow while it was running, as kubectl logs -f does.
How to reproduce it (as minimally and precisely as possible):
Containerize this script:
from loguru import logger

for _ in range(10):
    logger.info('A log line')
    sleep(1)

exit(0)

and invoke it in a workflow.yaml like:
container:
        command: [python3, "script.py"]
        image: blah.dkr.ecr.us-east-1.amazonaws.com/my-image:release-xxxxx

Anything else we need to know?:
Environment:

Argo version: argo-2.2.1
Kubernetes version :

clientVersion:
  buildDate: "1970-01-01T00:00:01Z"
  compiler: gc
  gitCommit: 721bfa751924da8d1680787490c54b9179b1fed0
  gitTreeState: archive
  gitVersion: v1.13.3
  goVersion: go1.11.5
  major: "1"
  minor: "13"
  platform: darwin/amd64
serverVersion:
  buildDate: "2019-03-01T22:49:39Z"
  compiler: gc
  gitCommit: 7c34c0d2f2d0f11f397d55a46945193a0e22d8f3
  gitTreeState: clean
  gitVersion: v1.11.8-eks-7c34c0
  goVersion: go1.10.8
  major: "1"
  minor: 11+
  platform: linux/amd64

Note: I'll try come back and add better reproducibility details. It's quite late here and I need to head off.

This is a pretty common problem with Docker images running Python. See: https://stackoverflow.com/a/29745541/375530
To summarize, I suggest using ENV PYTHONUNBUFFERED 0 (any value should do) or running python3 -u script.py to avoid the buffering issues you see. I don't think this is unique to argo, as almost all other tools would show the same behavior.

I'm aware of the PYTHONUNBUFFERED problem, but not aware of why it would affect argo logs but not kubectl logs. Able to tell me why that is? I'm imaging that both still rely on the logs exiting the Docker container unbuffered, but they behave differently.

repo: argoproj/argo


text:
add a command line argument to the artifactory unit test

artifiactory unit test is currently skipped since it has external dependency. @shrinandj made a good suggestion of accepting a command line argument to flag if the test should run or skip.

don't see this getting done. closing

authors: jessesuen sandeepbhojwani


text:
How do we monitor our acquisition database?

In addition to previous issue ticket, how do we ensure that all of our SLC acq metadata ES database is up-to-date?
Here are some rough examples - maybe the facets are wrong, but it seems like we can have significantly less acquisition documents in our database for certain time periods.
(Aria)
https://mamba-grq.aria.hysds.io/search/?base64=eyJxdWVyeSI6eyJib29sIjp7Im11c3QiOlt7InRlcm0iOnsiZGF0YXNldC5yYXciOiJhY3F1aXNpdGlvbi1TMS1JV19TTEMifX0seyJxdWVyeV9zdHJpbmciOnsicXVlcnkiOiJtZXRhZGF0YS5zZW5zaW5nU3RhcnQ6ezIwMjAtMDEtMDFUMDA6MDA6MDBaIFRPIDIwMjAtMDEtMDNUMDA6MDA6MDB9IiwiZGVmYXVsdF9vcGVyYXRvciI6Ik9SIn19XX19LCJzb3J0IjpbeyJfdGltZXN0YW1wIjp7Im9yZGVyIjoiZGVzYyJ9fV0sImZpZWxkcyI6WyJfdGltZXN0YW1wIiwiX3NvdXJjZSJdfQ== ( about 1900)
(ASF)
https://search.asf.alaska.edu/#/?start=2020-01-01T00:00:00Z&end=2020-01-02T23:59:59Z&resultsLoaded=true&granule=S1B_IW_SLC__1SDV_20200102T235532_20200102T235602_019645_025205_8E4D-SLC&productTypes=SLC,METADATA_SLC (about 2k)
The change here seems more dramatic:
(aria)
https://mamba-grq.aria.hysds.io/search/?base64=eyJxdWVyeSI6eyJib29sIjp7Im11c3QiOlt7InRlcm0iOnsiZGF0YXNldC5yYXciOiJhY3F1aXNpdGlvbi1TMS1JV19TTEMifX0seyJxdWVyeV9zdHJpbmciOnsicXVlcnkiOiJtZXRhZGF0YS5zZW5zaW5nU3RhcnQ6ezIwMjAtMDUtMjlUMDA6MDA6MDBaIFRPIDIwMjAtMDYtMDFUMDA6MDA6MDB9IiwiZGVmYXVsdF9vcGVyYXRvciI6Ik9SIn19XX19LCJzb3J0IjpbeyJfdGltZXN0YW1wIjp7Im9yZGVyIjoiZGVzYyJ9fV0sImZpZWxkcyI6WyJfdGltZXN0YW1wIiwiX3NvdXJjZSJdfQ== (<250)
(asf)
https://search.asf.alaska.edu/#/?end=2020-05-30T23:59:59Z&resultsLoaded=true&granule=S1A_WV_SLC__1SSV_20200530T234444_20200531T000010_032801_03CCA7_D8B3-METADATA_SLC&productTypes=METADATA_SLC&start=2020-05-29T00:00:00Z (about 2k)

Ok, this is definitely a start. What are your thoughts creating an AOI that spans the globe and then loop through tracks? Is this feasible in time?
There is no documentation for this repo. How was this meant to be run?

Why create a global AOI? You could just set this up as a cron to run however often you'd like and have it iterate through all aoi-track datasets.
I'm not familiar with the code, but reading it, you'll find this snippet that tells you what inputs it takes:
https://github.com/aria-jpl/ops_scripts2/blob/f6a067ee6308641a4fb0ae1269399dd77cccee30/check_acquisitions/check_acquisition_completeness.py#L262

So for the standard product, this could be run for each AOI and track and things could, in theory, be tracked accordingly after learning about said script/PGE.
For coseismic,  when we have to let the pipeline producing AOITracks in along tracks (that could be previously unmonitored), how would we monitor the SLC metadata in this setting to ensure the enumerator works as expected which relies on this? @riverma

It's only recommended that we run the scrapers over the the AOI's because we do not have a reliable, independent way of confirming our collection of SLC metadata is up-to-date. Ideally, we should be up-to-date with SLC metadata world-wide, so I think it would be best to not think in terms of AOI's or AOI tracks and instead focus on developing an independent checker that works in conjunction with the cron scrapers.

@marjo-luc - please go ahead and create a new dev ticket to track the proposed solution you're recommending.

authors: cmarshak marjo-luc riverma


text:
Issue with DVARS calculation for AROMA, high resolution data

I've run Denoiser successfully on a number of datasets, and while the actual denoising step always works, the calculation of DVARS for the QA plot crashes when I denoise fMRIPrep data that was AROMA cleaned (I'm using the "MNI152NLin6Asym_desc-smoothAROMAnonaggr_bold" data).
If I resample the AROMA data from 2x2x2 mm (the required output) to 3x3x3 mm, DVARS calculation works fine.
Here is the error message
_Computing dvars...
Traceback (most recent call last):
File "/onrc/home/pipeline/data/pipelineb/home/Chyatt/onrc/data/Apps/denoiser-master/run_denoise.py", line 469, in
denoise(img_file, tsv_file, out_path, col_names, hp_filter, lp_filter, out_figure_path)
File "/onrc/home/pipeline/data/pipelineb/home/Chyatt/onrc/data/Apps/denoiser-master/run_denoise.py", line 317, in denoise
temp = nac.compute_dvars(in_file=in_file, in_mask=mask_file)[1]
File "/home/Chyatt/onrc/data/Apps/miniconda3/lib/python3.9/site-packages/nipype/algorithms/confounds.py", line 1057, in compute_dvars
ar1 = np.apply_along_axis(
File "<array_function internals>", line 5, in apply_along_axis
File "/home/Chyatt/onrc/data/Apps/miniconda3/lib/python3.9/site-packages/numpy/lib/shape_base.py", line 402, in apply_along_axis
buff[ind] = asanyarray(func1d(inarr_view[ind], *args, **kwargs))
File "/home/Chyatt/onrc/data/Apps/miniconda3/lib/python3.9/site-packages/nitime/algorithms/autoregressive.py", line 96, in AR_est_YW
ak = linalg.solve(Tm, y)
File "/home/Chyatt/onrc/data/Apps/miniconda3/lib/python3.9/site-packages/scipy/linalg/basic.py", line 219, in solve
_solve_check(n, info)
File "/home/Chyatt/onrc/data/Apps/miniconda3/lib/python3.9/site-packages/scipy/linalg/basic.py", line 29, in _solve_check
raise LinAlgError('Matrix is singular.')
numpy.linalg.LinAlgError: Matrix is singular.
_
The Matrix is singular issue only occurs with 2x2x2 data, not 3x3x3 resampled data (so far). I tried inputting a single fairly short fMRI data (only 310 images) that was 2x2x2 mm, to see if it was a memory issue (performed on a computer with 768 GB ram), but this error remained.
Great application! Thank you!

Hi Christopher,
I got the same issue. Did you find any solution?

No. I tried to debug using the pdb python debugger, but I think the
issue (probably a memory issue) is in an area of code that the debugger
can't reach.
I did get denoiser to work though by commenting out all code dealing
with DVARS calculation and display. After removing this code, Denoiser
works, but just doesn't show the DVARS plot in the QA html file.
I attached the revised python code (run_denoise.py) if you would like to
try it.
Chris
On 5/5/2022 8:06 AM, mikojeske wrote:

Hi Christopher,
I got the same issue. Did you find any solution?
—
Reply to this email directly, view it on GitHub
https://github.com/arielletambini/denoiser/issues/41#issuecomment-1118470988,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHT2CHN5Z2ME5MEKS25NSTVIO2T3ANCNFSM5R4JO26A.
You are receiving this because you authored the thread.Message ID:
@.***>

#!/usr/bin/env python3
from nilearn.glm import regression
from nilearn.glm.first_level.design_matrix import make_first_level_design_matrix
import nibabel as nb
import numpy as np
import os, pandas, sys, pdb, argparse, copy, scipy, jinja2
from os.path import join as pjoin
from nilearn import plotting
from nilearn.signal import butterworth
from nilearn.input_data import NiftiMasker
import matplotlib
import pylab as plt
import seaborn as sns
from nilearn._utils.niimg import load_niimg
from nipype.algorithms import confounds as nac
parser = argparse.ArgumentParser(description='Function for performing nuisance regression. Saves resulting output '
'nifti file, information about nuisance regressors and motion (html '
'report), and outputs nibabel object containing clean data')
parser.add_argument('img_file', help='4d nifti img: file path or nibabel object loaded into memory')
parser.add_argument('tsv_file', help='tsv file containing nuisance regressors to be removed')
parser.add_argument('out_path', help='output directory for saving new data file')
parser.add_argument('--col_names',
help='which columns of TSV file to include as nuisance regressors. defaults to ALL columns.',
nargs="+")
parser.add_argument('--hp_filter', help='frequency cut-off for high pass filter (removing low frequencies). Recommend '
'.009 Hz')
parser.add_argument('--lp_filter', help='frequency cut-off for low pass filter (removing high frequencies). Recommend '
'.1 Hz for non-task data')
parser.add_argument('--out_figure_path',
help='output directory for saving figures. Defaults to location of out_path + _figures')
args = parser.parse_args()
img_file = args.img_file
tsv_file = args.tsv_file
out_path = args.out_path
col_names = args.col_names
hp_filter = args.hp_filter
lp_filter = args.lp_filter
out_figure_path = args.out_figure_path
def denoise(img_file, tsv_file, out_path, col_names=False, hp_filter=False, lp_filter=False, out_figure_path=False):
nii_ext = '.nii.gz'
FD_thr = [.5] # CJH
# FD_thr = [.5, 1.0] # CJH
# range from -1 to +2 (3 = up to, but not including 3)
# -1, 0, 1, 2 : so one before vol, include vol, and then two after vol
sc_range = np.arange(-1, 3)
constant = 'constant'
# read in files
img = load_niimg(img_file)
# get file info
img_name = os.path.basename(img.get_filename())
file_base = img_name[0:img_name.find('.')]
save_img_file = pjoin(out_path, file_base + \
                      '_NR' + nii_ext)
data = img.get_fdata()
df_orig = pandas.read_csv(tsv_file, '\t', na_values='n/a')
df = copy.deepcopy(df_orig)
Ntrs = df.values.shape[0]
print('# of TRs: ' + str(Ntrs))
assert (Ntrs == data.shape[len(data.shape) - 1])

# select columns to use as nuisance regressors
if col_names:
    df = df[col_names]
    str_append = '  [SELECTED regressors in CSV]'
else:
    col_names = df.columns.tolist()
    str_append = '  [ALL regressors in CSV]'

# fill in missing nuisance values with mean for that variable
for col in df.columns:
    if sum(df[col].isnull()) > 0:
        print('Filling in ' + str(sum(df[col].isnull())) + ' NaN value for ' + col)
        df[col] = df[col].fillna(np.mean(df[col]))
print('# of Confound Regressors: ' + str(len(df.columns)) + str_append)

# implement HP filter in regression
TR = img.header.get_zooms()[-1]
frame_times = np.arange(Ntrs) * TR
if hp_filter:
    hp_filter = float(hp_filter)
    assert (hp_filter > 0)
    df = make_first_level_design_matrix(frame_times, high_pass=hp_filter, add_regs=df.values,
                            add_reg_names=df.columns.tolist())
    # fn adds intercept into dm

    hp_cols = [col for col in df.columns if 'drift' in col]
    print('# of High-pass Filter Regressors: ' + str(len(hp_cols)))
else:
    # add in intercept column into data frame
    df[constant] = 1
    print('No High-pass Filter Applied')

dm = df.values

# prep data
data = np.reshape(data, (-1, Ntrs))
data_mean = np.mean(data, axis=1)
Nvox = len(data_mean)

# setup and run regression
model = regression.OLSModel(dm)
results = model.fit(data.T)
if not hp_filter:
    results_orig_resid = copy.deepcopy(results.residuals)  # save for rsquared computation

# apply low-pass filter
if lp_filter:
    # input to butterworth fn is time x voxels
    low_pass = float(lp_filter)
    Fs = 1. / TR
    if low_pass >= Fs / 2:
        raise ValueError('Low pass filter cutoff if too close to the Nyquist frequency (%s)' % (Fs / 2))

    temp_img_file = pjoin(out_path, file_base + \
                          '_temp' + nii_ext)
    temp_img = nb.Nifti1Image(np.reshape(results.residuals.T + np.reshape(data_mean, (Nvox, 1)), img.shape).astype('float32'),
                              img.affine, header=img.header)
    temp_img.to_filename(temp_img_file)
    results.residuals = butterworth(results.residuals, sampling_rate=Fs, low_pass=low_pass, high_pass=None)
    print('Low-pass Filter Applied: < ' + str(low_pass) + ' Hz')

# add mean back into data
clean_data = results.residuals.T + np.reshape(data_mean, (Nvox, 1))  # add mean back into residuals

# save out new data file
print('Saving output file...')
clean_data = np.reshape(clean_data, img.shape).astype('float32')
new_img = nb.Nifti1Image(clean_data, img.affine, header=img.header)
new_img.to_filename(save_img_file)

######### generate Rsquared map for confounds only
if hp_filter:
    # first remove low-frequency information from data
    hp_cols.append(constant)
    model_first = regression.OLSModel(df[hp_cols].values)
    results_first = model_first.fit(data.T)
    results_first_resid = copy.deepcopy(results_first.residuals)
    del results_first, model_first

    # compute sst - borrowed from matlab
    sst = np.square(np.linalg.norm(results_first_resid -
                                   np.mean(results_first_resid, axis=0), axis=0))

    # now regress out 'true' confounds to estimate their Rsquared
    nr_cols = [col for col in df.columns if 'drift' not in col]
    model_second = regression.OLSModel(df[nr_cols].values)
    results_second = model_second.fit(results_first_resid)

    # compute sse - borrowed from matlab
    sse = np.square(np.linalg.norm(results_second.residuals, axis=0))

    del results_second, model_second, results_first_resid

elif not hp_filter:
    # compute sst - borrowed from matlab
    sst = np.square(np.linalg.norm(data.T -
                                   np.mean(data.T, axis=0), axis=0))

    # compute sse - borrowed from matlab
    sse = np.square(np.linalg.norm(results_orig_resid, axis=0))

    del results_orig_resid

# compute rsquared of nuisance regressors
zero_idx = np.logical_and(sst == 0, sse == 0)
sse[zero_idx] = 1
sst[zero_idx] = 1  # would be NaNs - become rsquared = 0
rsquare = 1 - np.true_divide(sse, sst)
rsquare[np.isnan(rsquare)] = 0

######### Visualizing DM & outputs
fontsize = 12
fontsize_title = 14
def_img_size = 8

if not out_figure_path:
    out_figure_path = save_img_file[0:save_img_file.find('.')] + '_figures'

if not os.path.isdir(out_figure_path):
    os.mkdir(out_figure_path)
png_append = '_' + img_name[0:img_name.find('.')] + '.png'
print('Output directory: ' + out_figure_path)

# DM corr matrix
cm = df[df.columns[0:-1]].corr()
curr_sz = copy.deepcopy(def_img_size)
if cm.shape[0] > def_img_size:
    curr_sz = curr_sz + ((cm.shape[0] - curr_sz) * .3)
mtx_scale = curr_sz * 100

mask = np.zeros_like(cm, dtype=np.bool)
mask[np.triu_indices_from(mask)] = True

fig, ax = plt.subplots(figsize=(curr_sz, curr_sz))
cmap = sns.diverging_palette(220, 10, as_cmap=True)
sns.heatmap(cm, mask=mask, cmap=cmap, center=0, vmax=cm[cm < 1].max().max(), vmin=cm[cm < 1].min().min(),
            square=True, linewidths=.5, cbar_kws={"shrink": .6})
ax.set_xticklabels(ax.get_xticklabels(), rotation=60, ha='right', fontsize=fontsize)
ax.set_yticklabels(cm.columns.tolist(), rotation=-30, va='bottom', fontsize=fontsize)
ax.set_title('Nuisance Corr. Matrix', fontsize=fontsize_title)
plt.tight_layout()
file_corr_matrix = 'Corr_matrix_regressors' + png_append
fig.savefig(pjoin(out_figure_path, file_corr_matrix))
plt.close(fig)
del fig, ax

# DM of Nuisance Regressors (all)
tr_label = 'TR (Volume #)'
fig, ax = plt.subplots(figsize=(curr_sz - 4.1, def_img_size))
x_scale_html = ((curr_sz - 4.1) / def_img_size) * 890
plotting.plot_design_matrix(df, ax=ax)
ax.set_title('Nuisance Design Matrix', fontsize=fontsize_title)
ax.set_xticklabels(ax.get_xticklabels(), rotation=60, ha='right', fontsize=fontsize)
ax.set_yticklabels(ax.get_yticklabels(), fontsize=fontsize)
ax.set_ylabel(tr_label, fontsize=fontsize)
plt.tight_layout()
file_design_matrix = 'Design_matrix' + png_append
fig.savefig(pjoin(out_figure_path, file_design_matrix))
plt.close(fig)
del fig, ax

# FD timeseries plot
FD = 'FD'
poss_names = ['FramewiseDisplacement', FD, 'framewisedisplacement', 'fd', 'framewise_displacement']
fd_idx = [df_orig.columns.__contains__(i) for i in poss_names]
if np.sum(fd_idx) > 0:
    FD_name = np.array(poss_names)[fd_idx][0] #poss_names[fd_idx == True]
    if sum(df_orig[FD_name].isnull()) > 0:
        df_orig[FD_name] = df_orig[FD_name].fillna(np.mean(df_orig[FD_name]))
    y = df_orig[FD_name].values
    Nremove = []
    sc_idx = []
    for thr_idx, thr in enumerate(FD_thr):
        idx = y >= thr
        sc_idx.append(copy.deepcopy(idx))
        for iidx in np.where(idx)[0]:
            for buffer in sc_range:
                curr_idx = iidx + buffer

CJH remove 3/17/22                if curr_idx >= 0 and curr_idx <= len(idx):
                if curr_idx >= 0 and curr_idx < len(idx):
                    sc_idx[thr_idx][curr_idx] = True
        Nremove.append(np.sum(sc_idx[thr_idx]))

    Nplots = len(FD_thr)
    sns.set(font_scale=1.5)
    sns.set_style('ticks')
    fig, axes = plt.subplots(Nplots, 1, figsize=(def_img_size * 1.5, def_img_size / 2), squeeze=False)
    sns.despine()
    bound = .4
    fd_mean = np.mean(y)
    for curr in np.arange(0, Nplots):
        axes[curr, 0].plot(y)
        axes[curr, 0].plot((-bound, Ntrs + bound), FD_thr[curr] * np.ones((1, 2))[0], '--', color='black')
        axes[curr, 0].scatter(np.arange(0, Ntrs), y, s=20)

        if Nremove[curr] > 0:
            info = scipy.ndimage.measurements.label(sc_idx[curr])
            for cluster in np.arange(1, info[1] + 1):
                temp = np.where(info[0] == cluster)[0]
                axes[curr, 0].axvspan(temp.min() - bound, temp.max() + bound, alpha=.5, color='red')

        axes[curr, 0].set_ylabel('Framewise Disp. (' + FD + ')')
        axes[curr, 0].set_title(FD + ': ' + str(100 * Nremove[curr] / Ntrs)[0:4]
                                + '% of scan (' + str(Nremove[curr]) + ' volumes) would be scrubbed (FD thr.= ' +
                                str(FD_thr[curr]) + ')')
        plt.text(Ntrs + 1, FD_thr[curr] - .01, FD + ' = ' + str(FD_thr[curr]), fontsize=fontsize)
        plt.text(Ntrs, fd_mean - .01, 'avg = ' + str(fd_mean), fontsize=fontsize)
        axes[curr, 0].set_xlim((-bound, Ntrs + 8))

    plt.tight_layout()
    axes[curr, 0].set_xlabel(tr_label)
    file_fd_plot = FD + '_timeseries' + png_append
    fig.savefig(pjoin(out_figure_path, file_fd_plot))
    plt.close(fig)
    del fig, axes
    print(FD + ' timeseries plot saved')

else:
    print(FD + ' not found: ' + FD + ' timeseries not plotted')
    file_fd_plot = None

# Carpet and DVARS plots - before & after nuisance regression

# need to create mask file to input to DVARS function
mask_file = pjoin(out_figure_path, 'mask_temp.nii.gz')
nifti_masker = NiftiMasker(mask_strategy='epi', standardize=False)
nifti_masker.fit(img)
nifti_masker.mask_img_.to_filename(mask_file)

# create 2 or 3 carpet plots, depending on if LP filter is also applied
Ncarpet = 2
total_sz = int(16)
carpet_scale = 840
y_labels = ['Input (voxels)', 'Output \'cleaned\'']
imgs = [img, new_img]
img_files = [img_file, save_img_file]
color = ['red', 'salmon']
labels = ['input', 'cleaned']
if lp_filter:
    Ncarpet = 3
    total_sz = int(20)
    carpet_scale = carpet_scale * (9/8)
    y_labels = ['Input', 'Clean Pre-LP', 'Clean LP']
    imgs.insert(1, temp_img)
    img_files.insert(1, temp_img_file)
    color.insert(1, 'firebrick')
    labels.insert(1, 'clean pre-LP')
    labels[-1] = 'clean LP'

CJH commenting out the DVARS portion of the code
dvars = []
print('Computing dvars...')
for in_file in img_files:
temp = nac.compute_dvars(in_file=in_file, in_mask=mask_file)[1]
dvars.append(np.hstack((temp.mean(), temp)))
del temp
small_sz = 2
fig = plt.figure(figsize=(def_img_size * 1.5, def_img_size + ((Ncarpet - 2) * 1)))
row_used = 0
if np.sum(fd_idx) > 0:  # if FD data is available
    row_used = row_used + small_sz
    ax0 = plt.subplot2grid((total_sz, 1), (0, 0), rowspan=small_sz)
    ax0.plot(y)
    ax0.scatter(np.arange(0, Ntrs), y, s=10)
    curr = 0
    if Nremove[curr] > 0:
        info = scipy.ndimage.measurements.label(sc_idx[curr])
        for cluster in np.arange(1, info[1] + 1):
            temp = np.where(info[0] == cluster)[0]
            ax0.axvspan(temp.min() - bound, temp.max() + bound, alpha=.5, color='red')
    ax0.set_ylabel(FD)

    for side in ["top", "right", "bottom"]:
        ax0.spines[side].set_color('none')
        ax0.spines[side].set_visible(False)

    ax0.set_xticks([])
    ax0.set_xlim((-.5, Ntrs - .5))
    ax0.spines["left"].set_position(('outward', 10))

ax_d = plt.subplot2grid((total_sz, 1), (row_used, 0), rowspan=small_sz)

CJH comment out DVARS plotting
for iplot in np.arange(len(dvars)):
ax_d.plot(dvars[iplot], color=color[iplot], label=labels[iplot])
ax_d.set_ylabel('DVARS')
for side in ["top", "right", "bottom"]:
    ax_d.spines[side].set_color('none')
    ax_d.spines[side].set_visible(False)
ax_d.set_xticks([])
ax_d.set_xlim((-.5, Ntrs - .5))
ax_d.spines["left"].set_position(('outward', 10))
ax_d.legend(fontsize=fontsize - 2)
row_used = row_used + small_sz

st = 0
carpet_each = int((total_sz - row_used) / Ncarpet)
for idx, img_curr in enumerate(imgs):
    ax_curr = plt.subplot2grid((total_sz, 1), (row_used + st, 0), rowspan=carpet_each)
    fig = plotting.plot_carpet(img_curr, figure=fig, axes=ax_curr)
    ax_curr.set_ylabel(y_labels[idx])
    for side in ["bottom", "left"]:
        ax_curr.spines[side].set_position(('outward', 10))

    if idx < len(imgs)-1:
        ax_curr.spines["bottom"].set_visible(False)
        ax_curr.set_xticklabels('')
        ax_curr.set_xlabel('')
        st = st + carpet_each

file_carpet_plot = 'Carpet_plots' + png_append
fig.savefig(pjoin(out_figure_path, file_carpet_plot))
plt.close()

# CJH, del ax0 only if FD available in TSV file
if np.sum(fd_idx) > 0: del ax0
# CJH: removed ax0 and dvars from del items
del fig, ax_curr, ax_d
#del fig, ax_curr, ax_d, dvars
#del fig, ax0, ax_curr, ax_d, dvars

os.remove(mask_file)
#print('Carpet/DVARS plots saved')
print('Carpet plots saved')
if lp_filter:
    os.remove(temp_img_file)
    del temp_img

# Display T-stat maps for nuisance regressors
# create mean img
img_size = (img.shape[0], img.shape[1], img.shape[2])
mean_img = nb.Nifti1Image(np.reshape(data_mean, img_size), img.affine)
mx = []
for idx, col in enumerate(df.columns):
    if not 'drift' in col and not constant in col:
        con_vector = np.zeros((1, df.shape[1]))
        con_vector[0, idx] = 1
        con = results.Tcontrast(con_vector)
        mx.append(np.max(np.absolute([con.t.min(), con.t.max()])))
mx = .8 * np.max(mx)
t_png = 'Tstat_'
file_tstat = []
for idx, col in enumerate(df.columns):
    if not 'drift' in col and not constant in col:
        con_vector = np.zeros((1, df.shape[1]))
        con_vector[0, idx] = 1
        con = results.Tcontrast(con_vector)
        m_img = nb.Nifti1Image(np.reshape(con, img_size), img.affine)

        title_str = col + ' Tstat'
        fig = plotting.plot_stat_map(m_img, mean_img, threshold=3, colorbar=True, display_mode='z', vmax=mx,
                                     title=title_str,
                                     cut_coords=7)
        file_temp = t_png + col + png_append
        fig.savefig(pjoin(out_figure_path, file_temp))
        file_tstat.append({'name': col, 'file': file_temp})
        plt.close()
        del fig, file_temp
        print(title_str + ' map saved')

# Display R-sq map for nuisance regressors
m_img = nb.Nifti1Image(np.reshape(rsquare, img_size), img.affine)
title_str = 'Nuisance Rsq'
mx = .95 * rsquare.max()
fig = plotting.plot_stat_map(m_img, mean_img, threshold=.2, colorbar=True, display_mode='z', vmax=mx,
                             title=title_str,
                             cut_coords=7)
file_rsq_map = 'Rsquared' + png_append
fig.savefig(pjoin(out_figure_path, file_rsq_map))
plt.close()
del fig
print(title_str + ' map saved')

######### html report
templateLoader = jinja2.FileSystemLoader(searchpath="/")
templateEnv = jinja2.Environment(loader=templateLoader)

templateVars = {"img_file": img_file,
                "save_img_file": save_img_file,
                "Ntrs": Ntrs,
                "tsv_file": tsv_file,
                "col_names": col_names,
                "hp_filter": hp_filter,
                "lp_filter": lp_filter,
                "file_design_matrix": file_design_matrix,
                "file_corr_matrix": file_corr_matrix,
                "file_fd_plot": file_fd_plot,
                "file_rsq_map": file_rsq_map,
                "file_tstat": file_tstat,
                "x_scale": x_scale_html,
                "mtx_scale": mtx_scale,
                "file_carpet_plot": file_carpet_plot,
                "carpet_scale": carpet_scale
                }

TEMPLATE_FILE = pjoin(os.getcwd(), "report_template.html")
template = templateEnv.get_template(TEMPLATE_FILE)

outputText = template.render(templateVars)

html_file = pjoin(out_figure_path, img_name[0:img_name.find('.')] + '.html')
with open(html_file, "w") as f:
    f.write(outputText)

print('')
print('HTML report: ' + html_file)
return new_img

denoise(img_file, tsv_file, out_path, col_names, hp_filter, lp_filter, out_figure_path)

authors: eburkevt mikojeske

repo: arielletambini/denoiser


text:
atlas.sum mismatch between windows and linux systems

While I was testing out Atlas Cloud, I noticed I was getting checksum mismatch errors. So I tried running the command atlas migrate validate --dir "file://database/migrations" to validate and was surprised when I didn't get an error code/response.
After discussion in discord, it seems I may be one of the only windows developers 😆. Joking aside, it seems that maybe windows line endings may be impacting the result of the migration checksum.

Thanks @tankbusta, I will take a look.

Hey @tankbusta, the happens due to the UTF-16 encoding of windows in the Terminal/PowerShell. This seems to be a problem a lot of other tools face on Windows as well and the common solution is to change the default encoding to UTF-8, which is what we recommend in this case as well. Is this a suitable solution for you as well?

I'd think the hashing algorithm should disregard new lines (and potentially whitespace) as they don't affect the schema

This is not entirely true, e.g. in:
insert into tbl (col) values ('linux has \n'), ('mac has \r'), ('windows has \r\n');

we cannot simply remove / replace / split on any new line char. This is why we hash on byte level. However, since we hash on byte level there should not be a mismatch between the hashes. Do you by any chance have git configured to handle the line endings based on the client? This would explain the mismatch, since your windows machine would send in CRLF based sum and git would change that to LF on checkout on Linux / CI.

Yeah I totally spaced out on the migration data use case. That makes sense.
I think the default behavior for git on windows is to enable core.autoclrf. I disabled it and it's working now

I think the default behavior for git on windows is to enable core.autoclrf. I disabled it and it's working now

Great to hear that. In that case, I think we can close this, right? Afterall, the current behavior of the sum-file is correct, agree?

Yep!

authors: masseelch tankbusta

repo: ariga/atlas


text:
README.md: add Go Reference (pkg.go.dev) badge

To help package users navigate to the documentation faster.
Generated with https://pkg.go.dev/badge/.

Thanks for adding this @edigaryev. tbh, I'm not sure I want it, because I prefer users to go to atlasgo.io in most cases for documentation (I plan to add there the Go API as well), and there's a link to the godoc in case they want it. I didn't add it to ent README for the reasons.
@rotemtam, your thoughts?

agree @a8m . as atlas is currently mostly used as a CLI, i prefer the first link to be to atlasgo.io.
as you mentioned there's a link to the godoc there.
thanks for the suggestion @edigaryev ! closing for the reason mentioned 🙏

authors: a8m edigaryev rotemtam


text:
Require equals sign after rule name when defining rule

Not as hard to implement as I thought (45 mins!).

done.

repo: arineng/jcrvalidator

license: isc


text:
Undefined index: kirki_test[...]  ===> class-kirki.php ===> line 102 & 108

Hello Kriki
Thanks for this awesome plugin, i installed inside theme folder, and i follow all the steps, but i get the following errors.

Notice: Undefined index: kirki_test[kirki_test[color-alpha_demo_5]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 102

Notice: Undefined index: kirki_test[kirki_test[color-alpha_demo_5]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 108

Notice: Undefined index: kirki_test[kirki_test[color_demo_5]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 102

Notice: Undefined index: kirki_test[kirki_test[color_demo_5]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 108

Notice: Undefined index: kirki_test[kirki_test[color-alpha_demo_2]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 102

Notice: Undefined index: kirki_test[kirki_test[color-alpha_demo_2]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 108

Notice: Undefined index: kirki_test[kirki_test[color_demo_2]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 102

Notice: Undefined index: kirki_test[kirki_test[color_demo_2]] in D:\xampp\htdocs\test1\wp-content\themes\theme_name\theme-admin\options\includes\class-kirki.php on line 108

Thanks!

No worries, these are just from the dummy test data that you probably enabled.
The plugin itself is fine, the problem is with the script containing the test data.

We plan on rebuilding the test data, I just haven't had the time to do that yet.
I'm marking this as a bug and adding it to the tasklist for version 1.1 :+1:

fixed. We completely refactored the way demo data is handled.

Hi Aris.  I would like to test this.  How should i go about testing so i can post my results here?

The new tests work like this:

To enable the demo theme, just add this line to your wp-config.php file:
define( 'KIRKI_CONFIG', true );
Once you add that line, you'll see a new theme in your dashboard called "Kirki Demo".
Activate that theme to test all controls.

(inline docs here)

authors: aristath cmwwebfx hamdan-mahran

repo: aristath/kirki


text:
question on repeater

Can't import in a page my repeater value
Version:       2.2.6
here are the theme-option.php
Kirki::add_field( 'my_config', array(

	'type'        => 'repeater',

	'label'       => esc_attr__( 'Repeater Control', 'my_textdomain' ),

	'section'     => 'header_advertisement',

	'priority'    => 10,

	'settings'    => 'my_repeater',

	'fields' => array(
                'number' => array(

			'type'        => 'text',

			'label'       => esc_attr__( 'post number', 'my_textdomain' ),

			'description' => esc_attr__( 'This will display the post number', 'my_textdomain' ),

			'default'     => '',

		),

                'selector' => array(

                        'type'        => 'select',

    			'settings'    => 'select_demo',

    			'label'       => __( 'This is the label', 'kirki' ),

    			'description' => __( 'This is the control description', 'kirki' ),

    			'priority'    => 10,

    			'choices'     => Kirki_Helper::get_terms( 'category' ),

    			),
)

) );

and here are category-page file
<?php
/**
 * category-page.php
 *
**/
global $wp_customize;
if ( ! $wp_customize ) {
	wp_redirect( trailingslashit( admin_url() ) . 'customize.php' );
	exit;
}

$posts_per_page = get_theme_mod( 'number','6');
$cat_name = get_theme_mod( 'selector','Uncategorized'); // get the category id which will filter the section

	$args = array(
		'posts_per_page' => $posts_per_page,
		'category_name'   =>  $cat_name,
		'post_type'      => 'post',
		'post_status'    => 'publish',
);
query_posts( $args );
?>
<div class="container">
	     <?php if (have_posts()) : ?>
            <?php while (have_posts()) : the_post(); ?>
/*rest of the code*/

the result only takes the default value

Hey there!
First of all, let me just repost your code for the field here with proper indentation so that it's easier to read:
Kirki::add_field( 'my_config', array(
	'type'        => 'repeater',
	'label'       => esc_attr__( 'Repeater Control', 'my_textdomain' ),
	'section'     => 'header_advertisement',
	'priority'    => 10,
	'settings'    => 'my_repeater',
	'fields' => array(
		'number' => array(
			'type'        => 'text',
			'label'       => esc_attr__( 'post number', 'my_textdomain' ),
			'description' => esc_attr__( 'This will display the post number', 'my_textdomain' ),
			'default'     => '',
		),
		'selector' => array(
			'type'        => 'select',
			'settings'    => 'select_demo',
			'label'       => __( 'This is the label', 'kirki' ),
			'description' => __( 'This is the control description', 'kirki' ),
			'priority'    => 10,
			'choices'     => Kirki_Helper::get_terms( 'category' ),
		),
	),
) );

From what I can see on your code, you're trying to directly access the values of number and selector.
That won't work because these are not actual theme_mods.
Your options are saved under my_repeater.
If you do this in your functions.php file, you'll be able to see the saved data:
var_dump( get_theme_mod( 'my_repeater' ) );

The saved structure looks like this:
array(
    0 => array(
        'number' => '5',
        'selector' => '50',
    ),
    1 => array(
        'number' => '10',
        'selector' => '575',
    ),
)

so first you'd have to do this to get the value of the repeater:
$repeater_value = get_theme_mod( 'my_repeater', array() );

Then, to access the values of the first row you'd be able to do it like this:
$first_row = $repeater_value[0];

To access to number subcontrol on the 1st row, you'd do it like this:
$first_row_number = $first_row['number']

or
$first_row_number = $repeater_value[0]['number']

If you want to write a loop to go through all rows in your repeater, you'd do it like this:
$repeater_value = get_theme_mod( 'my_repeater', array() );

// Check that we have something to process
if ( ! empty( $repeater_value ) ) {

	// Loop all repeater rows
	foreach ( $repeater_value as $key => $row_values ) {

		// Get the 'number' value
		$number = $row_values['number'];
		// Get the 'selector' value
		$selector = $row_values['selector'];
		// DO SOMETHING WITH THOSE VALUES HERE
	}
}

I hope that helps!
Cheers.

Thanks a lot :100:

authors: aristath michellomp


text:
Feature: Create hook for view functions

Create a hook around view.ts
https://github.com/arjanjohan/scaffold-move/blob/3e147694e7f798a9088208019ee97646e71ea01b/packages/nextjs/utils/scaffold-move/view.ts#L11

Fixed in https://github.com/arjanjohan/scaffold-move/pull/25

repo: arjanjohan/scaffold-move


text:
OSX: Can't launch server & tests return nothing

Everything I did - is that I changed text "\FILL ME IN!\" to "Hello world" in first example.
Oh, and now I upgraded node from 0.10.32 to 0.12.2_1 and I now get this even with untouched examples:
https://gist.github.com/inem/a037c7616d8044527ab3
Why it happens all the time I try do something with node??!!

Did you try npm rebuild?

Oh, thanks! That helped. Probably you should add required node version to README

I guess the problem wasn't with node version. Wasn't it?


text:
Simpler solution for Grocery Challenge Part 4?

You say:
  // Fill the definition of the following method to allow completing each item
  // Hint 1: Pay attention to the element's index on the list.

This is what I came up with:
  toggleGroceryCompleteness(groceryIndex) {
    var newState = React.addons.update(this.state, {
      groceries: {[groceryIndex]: {completed: {$set: true}}}
    });
    this.setState(newState)
  }

But this seems odd because you never told us about React.addons.update.
Is there a simpler solution that I'm overlooking?  Thanks!

Hello @bronson,
Using React.addons.update is OK. If you want, go for it. However you could just get the mutation done like this:
toggleGroceryCompleteness(groceryIndex) {
  var state = Object.assign({}, this.state}); // It's new ES2015 / ES6 stuff
  state.groceries[groceryIndex].completed = true;
  this.setState(state);
}

Pretty sure this solution violates the "NEVER mutate this.state directly" warning, doesn't it?
https://facebook.github.io/react/docs/component-api.html
(sure, it's probably safe here where updates will never happen very fast, but that seems a bad habit to get into...)

It won't crash if you mutate it to set later, React will check on that and do some stuff around. This note in documentation warns before changing state and not setting it later (with setState method). It may overwrite your changes with next setState.

Still sounds like it violates it though.  :)  Though, I agree, it will work just fine here.
Closing since you've given me the info I need.  Thank you!

@voter101 @bronson I had a similar issue with the violation. Here's what I came up with to avoid using the plugin and the direct data mutation:
toggleGroceryCompleteness(groceryIndex) {
  let groceries = [];
  let currentState;

  // rebuild the groceries array
  for(let index = 0; index < this.state.groceries.length; index++) {
    groceries.push({ name: this.state.groceries[index].name });
    currentState = this.state.groceries[index].completed;

    // if the index matches the groceryIndex, swap the state
    groceries[index].completed = index === groceryIndex ? !currentState : currentState
  }

  this.setState({groceries: groceries});
}

authors: antibland bronson voter101


text:
Support decrypted smtp traffic

Currently arkime doesn't support smtp that has been decrypted in place, so it still has the STARTTLS but nothing is actually encrypted. Will need to change the smtp parser to detect if actually encrypted or not to support.

Hi , any solution in sight for this in the near future ?

Does anyone have a sample pcap they can share or DM me on slack one

authors: TofBaasken awick

repo: arkime/arkime