text:
HDFS-16267. Make hdfs_df tool cross platform

Description of PR
The source files for hdfs_df uses getopt for parsing the command line arguments. getopt is available only on Linux and thus, isn't cross platform. We need to replace getopt with boost::program_options to make this cross platform.
How was this patch tested?
Ran hdfs_df locally and verified that it's giving the right output -

Also, the unit tests ran fine -

For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
44m 22s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

-1 :x:
mvninstall
0m 26s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
0m 23s
/branch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in trunk failed.

-1 :x:
mvnsite
0m 24s
/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in trunk failed.

+1 :green_heart:
shadedclient
2m  5s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

-1 :x:
compile
0m 23s
/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

-1 :x:
cc
0m 23s
/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

-1 :x:
golang
0m 23s
/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

-1 :x:
javac
0m 23s
/patch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-1 :x:
mvnsite
0m 22s
/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

+1 :green_heart:
shadedclient
1m 12s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 23s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
hadoop-hdfs-native-client in the patch failed.

+0 :ok:
asflicense
0m 23s

ASF License check generated no output?

51m 34s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3542/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3542

Optional Tests
dupname asflicense compile cc mvnsite javac unit codespell golang

uname
Linux 332c4f22354c 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 463c7b93cabc9b10286d8dd2bd66b9d378b3bd8c

Default Java
Red Hat, Inc.-1.8.0_302-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3542/1/testReport/

Max. process+thread count
46 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3542/1/console

versions
git=2.9.5 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
patch
0m 19s

https://github.com/apache/hadoop/pull/3542 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/3542

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3542/2/console

versions
git=2.17.1

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

authors: GauthamBanasandra hadoop-yetus


text:
HBASE-26284 Add HBase Thrift API to get all table names along with whether it is enabled or not

(#3693)
Signed-off-by: Norbert Kalmar nkalmar@cloudera.com
Signed-off-by: Wellington Chevreuil wchevreuil@apache.org

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  6s
Docker mode activated.

-0 :warning:
yetus
0m  8s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
5m  1s
branch-2 passed

+1 :green_heart:
compile
1m  6s
branch-2 passed

+1 :green_heart:
shadedjars
8m  2s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m 20s
branch-2 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 20s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 46s
the patch passed

+1 :green_heart:
compile
1m  8s
the patch passed

+1 :green_heart:
javac
1m  8s
the patch passed

+1 :green_heart:
shadedjars
8m  2s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m 18s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
6m 50s
hbase-thrift in the patch passed.

+1 :green_heart:
unit
1m 55s
hbase-examples in the patch passed.

42m 52s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3745/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3745

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux bc5b75d48df1 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / 10584d70d2

Default Java
AdoptOpenJDK-11.0.10+9

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3745/1/testReport/

Max. process+thread count
1595 (vs. ulimit of 12500)

modules
C: hbase-thrift hbase-examples U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3745/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

source: gharchive/pull-request

authors: Apache-HBase horvathdora

repo: apache/hbase

license_type: permissive

license_source: github-api


text:
[BUG]

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
After modifying the monitoring template, the previously customized menu becomes invalid.

Expected Behavior
customized menu not invalid
Steps To Reproduce
No response
Environment
HertzBeat version(s): master code

Debug logs
No response
Anything else?
No response

i can try

source: gharchive/issue

repo: apache/hertzbeat


text:
HIVE-23870: Optimise multiple text conversions in HiveCharWritable/WritableHiveCharObjectInspector

Observed runtime dropping from "7600s --> 4800s" in internal cluster, when running a job which creates materialized view on warehouse/inventory/date_dim (where warehouse had char/varchar).

So, this caches the charLength ?  That is the fix here?

This is an interesting observation.  However, maybe this should be contributed to Hadoop project directly.

https://github.com/apache/hadoop/pull/2157

authors: belugabehr rbalamohan

repo: apache/hive


text:
HIVE-24157: Strict mode to fail on CAST timestamp <-> numeric

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

@jcamachor  could you please take a look?


text:
HIVE-24858: UDFClassLoader leak in Configuration.CACHE_CLASSES

See HIVE-24858.
How was this patch tested?
Manually

not work for me in hive 2.3.7
remove udfclassloader from CACHE_CLASSES when session close

authors: klcopp lklong


text:
[HUDI-8564] Removing WriteStatus references in Hoodie Metadata writer flow

Change Logs
Removing WriteStatus references in Hoodie Metadata writer flow.
This is stacked on top of https://github.com/apache/hudi/pull/12313
Impact
Fixing MDT writes to not rely on RDD
Risk level (write none, low medium or high below)
low
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

The config description must be updated if new configs are added or the default value of the configs are changed
Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the instruction to make
changes to the website.

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

@nsivabalan I ll push some changes and do a final review.. you can take on the testing related gaps, after that.

TestSecondaryIndexPruning#testUpdatesReInsertsDeletes
After insert + index builds:
0 : [trip1,5,city=san_francisco/state=california,-6380283593510729419,-4773949543163463974,0,,1732582749312,0,null,null]
0 : [trip2,5,city=sunnyvale/state=california,1772447978757638219,-5721388430672818475,0,,1732582749312,0,null,null]
0 : [trip3,5,city=austin/state=texas,-5153898977397095021,-5284273978040951062,0,,1732582749312,0,null,null]
0 : [trip4,5,city=houston/state=texas,-7405699635221806163,-9058970397732851524,0,,1732582749312,0,null,null]
0 : [rider-F$trip4,7,null,null,null,null,null,null,null,null,false]
0 : [rider-C$trip2,7,null,null,null,null,null,null,null,null,false]
0 : [rider-A$trip1,7,null,null,null,null,null,null,null,null,false]
0 : [rider-E$trip3,7,null,null,null,null,null,null,null,null,false]
0 : [driver-O$trip3,7,null,null,null,null,null,null,null,null,false]
0 : [driver-P$trip4,7,null,null,null,null,null,null,null,null,false]
0 : [driver-K$trip1,7,null,null,null,null,null,null,null,null,false]
0 : [driver-M$trip2,7,null,null,null,null,null,null,null,null,false]
0 : [__all_partitions__,1,null,null,null,null,null,null,null,null,null]
0 : [city=houston/state=texas,2,null,null,null,null,null,null,null,null,null]
0 : [city=austin/state=texas,2,null,null,null,null,null,null,null,null,null]
0 : [city=sunnyvale/state=california,2,null,null,null,null,null,null,null,null,null]
0 : [city=san_francisco/state=california,2,null,null,null,null,null,null,null,null,null]

After update:
1 : [trip1,5,city=san_francisco/state=california,-6380283593510729419,-4773949543163463974,0,,1732582749312,0,null,null]
1 : [trip2,5,city=sunnyvale/state=california,1772447978757638219,-5721388430672818475,0,,1732582749312,0,null,null]
1 : [trip3,5,city=austin/state=texas,-5153898977397095021,-5284273978040951062,0,,1732582749312,0,null,null]
1 : [trip4,5,city=houston/state=texas,-7405699635221806163,-9058970397732851524,0,,1732582749312,0,null,null]
1 : [rider-E$trip4,7,null,null,null,null,null,null,null,null,false]
1 : [rider-C$trip2,7,null,null,null,null,null,null,null,null,false]
1 : [rider-A$trip1,7,null,null,null,null,null,null,null,null,false]
1 : [rider-E$trip3,7,null,null,null,null,null,null,null,null,false]
1 : [driver-O$trip3,7,null,null,null,null,null,null,null,null,false]
1 : [driver-P$trip4,7,null,null,null,null,null,null,null,null,false]
1 : [driver-K$trip1,7,null,null,null,null,null,null,null,null,false]
1 : [driver-M$trip2,7,null,null,null,null,null,null,null,null,false]
1 : [__all_partitions__,1,null,null,null,null,null,null,null,null,null]
1 : [city=houston/state=texas,2,null,null,null,null,null,null,null,null,null]
1 : [city=austin/state=texas,2,null,null,null,null,null,null,null,null,null]
1 : [city=sunnyvale/state=california,2,null,null,null,null,null,null,null,null,null]
1 : [city=san_francisco/state=california,2,null,null,null,null,null,null,null,null,null]

After delete:
2 : [trip1,5,city=san_francisco/state=california,-6380283593510729419,-4773949543163463974,0,,1732582749312,0,null,null]
2 : [trip2,5,city=sunnyvale/state=california,1772447978757638219,-5721388430672818475,0,,1732582749312,0,null,null]
2 : [trip3,5,city=austin/state=texas,-5153898977397095021,-5284273978040951062,0,,1732582749312,0,null,null]
2 : [rider-C$trip2,7,null,null,null,null,null,null,null,null,false]
2 : [rider-A$trip1,7,null,null,null,null,null,null,null,null,false]
2 : [rider-E$trip3,7,null,null,null,null,null,null,null,null,false]
2 : [driver-O$trip3,7,null,null,null,null,null,null,null,null,false]
2 : [driver-K$trip1,7,null,null,null,null,null,null,null,null,false]
2 : [driver-M$trip2,7,null,null,null,null,null,null,null,null,false]
2 : [__all_partitions__,1,null,null,null,null,null,null,null,null,null]
2 : [city=houston/state=texas,2,null,null,null,null,null,null,null,null,null]
2 : [city=austin/state=texas,2,null,null,null,null,null,null,null,null,null]
2 : [city=sunnyvale/state=california,2,null,null,null,null,null,null,null,null,null]
2 : [city=san_francisco/state=california,2,null,null,null,null,null,null,null,null,null]

After reinsert:
3 : [trip1,5,city=san_francisco/state=california,-6380283593510729419,-4773949543163463974,0,,1732582749312,0,null,null]
3 : [trip2,5,city=sunnyvale/state=california,1772447978757638219,-5721388430672818475,0,,1732582749312,0,null,null]
3 : [trip3,5,city=austin/state=texas,-5153898977397095021,-5284273978040951062,0,,1732582749312,0,null,null]
3 : [trip4,5,city=houston/state=texas,-8294613005162561361,-6714765547006433090,0,,1732582830669,0,null,null]
3 : [rider-C$trip2,7,null,null,null,null,null,null,null,null,false]
3 : [rider-G$trip4,7,null,null,null,null,null,null,null,null,false]
3 : [rider-A$trip1,7,null,null,null,null,null,null,null,null,false]
3 : [rider-E$trip3,7,null,null,null,null,null,null,null,null,false]
3 : [driver-O$trip3,7,null,null,null,null,null,null,null,null,false]
3 : [driver-Q$trip4,7,null,null,null,null,null,null,null,null,false]
3 : [driver-K$trip1,7,null,null,null,null,null,null,null,null,false]
3 : [driver-M$trip2,7,null,null,null,null,null,null,null,null,false]
3 : [__all_partitions__,1,null,null,null,null,null,null,null,null,null]
3 : [city=houston/state=texas,2,null,null,null,null,null,null,null,null,null]
3 : [city=austin/state=texas,2,null,null,null,null,null,null,null,null,null]
3 : [city=sunnyvale/state=california,2,null,null,null,null,null,null,null,null,null]
3 : [city=san_francisco/state=california,2,null,null,null,null,null,null,null,null,null]

After second update:
4 : [trip1,5,city=san_francisco/state=california,-6380283593510729419,-4773949543163463974,0,,1732582749312,0,null,null]
4 : [trip2,5,city=sunnyvale/state=california,1772447978757638219,-5721388430672818475,0,,1732582749312,0,null,null]
4 : [trip3,5,city=austin/state=texas,-5153898977397095021,-5284273978040951062,0,,1732582749312,0,null,null]
4 : [rider-E$trip4,7,null,null,null,null,null,null,null,null,false]
4 : [rider-E$trip2,7,null,null,null,null,null,null,null,null,false]
4 : [rider-A$trip1,7,null,null,null,null,null,null,null,null,false]
4 : [rider-E$trip3,7,null,null,null,null,null,null,null,null,false]
4 : [driver-O$trip3,7,null,null,null,null,null,null,null,null,false]
4 : [driver-Q$trip4,7,null,null,null,null,null,null,null,null,false]
4 : [driver-K$trip1,7,null,null,null,null,null,null,null,null,false]
4 : [driver-M$trip2,7,null,null,null,null,null,null,null,null,false]
4 : [__all_partitions__,1,null,null,null,null,null,null,null,null,null]
4 : [city=houston/state=texas,2,null,null,null,null,null,null,null,null,null]
4 : [city=austin/state=texas,2,null,null,null,null,null,null,null,null,null]
4 : [city=sunnyvale/state=california,2,null,null,null,null,null,null,null,null,null]
4 : [city=san_francisco/state=california,2,null,null,null,null,null,null,null,null,null]

Addressed all feedback. Follow up jira https://issues.apache.org/jira/browse/HUDI-8597

authors: nsivabalan vinothchandar

repo: apache/hudi


text:
[HUDI-2244] Fix database alreadyExists exception while hive sync

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
Fix database alreadyExists exception while hive sync. Now, if the database exists, every hive sync process will throw  database alreadyExists exception, this PR fix this problem and adding test case to cover different situation.
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

hi @jsbali , when I tested the new hive sync code. I found if one database exists and hive sync mode setting as 'hms',  every hive sync process wound throw database alreadyExists exception, so I tried to fix this problem and adding tests. And I also found a bug that $client.getDatabase(databaseName) throw noSuchObjectException instead of returning null while there is no database exists, which will influence the result when autoCreateDatabase setting false. Please have a look, thank you!

Thanks @swuferhong for fixing it. LGTM

authors: jsbali swuferhong


text:
[MINOR][DNM] Validate 0.14.0 RC3 bundles

Change Logs
As above.
Impact
Testing only.  Should not be merged.
Risk level
none
Documentation Update
N/A
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

Closing this testing-only PR.


text:
IGNITE-22278 Sql. Refactor base and planning contexts

https://issues.apache.org/jira/browse/IGNITE-22278
This patch refactors contexts used within sql engine:

BaseQueryContext is removed. Instead lightweight SqlOperationContext is introduced. All planning-related metadata moved to PlanningContext
PrepareService is not aware of SchemaManager. It accepts operation time as input and resolve active schema itself
schemaVersion renamed to catalogVersion in numerous places to better reflect semantic

Thank you for submitting the pull request.
To streamline the review process of the patch and ensure better code quality
we ask both an author and a reviewer to verify the following:
The Review Checklist

[ ] Formal criteria: TC status, codestyle, mandatory documentation. Also make sure to complete the following:
- There is a single JIRA ticket related to the pull request.
- The web-link to the pull request is attached to the JIRA ticket.
- The JIRA ticket has the Patch Available state.
- The description of the JIRA ticket explains WHAT was made, WHY and HOW.
- The pull request title is treated as the final commit message. The following pattern must be used: IGNITE-XXXX Change summary where XXXX - number of JIRA issue.
[ ] Design: new code conforms with the design principles of the components it is added to.
[ ] Patch quality: patch cannot be split into smaller pieces, its size must be reasonable.
[ ] Code quality: code is clean and readable, necessary developer documentation is added if needed.
[ ] Tests code quality: test set covers positive/negative scenarios, happy/edge cases. Tests are effective in terms of execution time and resources.

Notes

Apache Ignite Coding Guidelines

There are few places in sql-engine module where schemaVersion is still being used:

KeyValueModifyPlan has a field schemaVersion
KeyValueGetPlan has a field schemaVersion
SqlSchemaManager defines a method table(schemaVersion, tableId)
ExecutionServiceImpl FragmentCacheKey has a field schemaVersion

Are the names of these fields/variables kept unchanged on purpose?

@lowka my bad, thanks for noticing. I didn't use full text search, just changed names in those places I stumbled upon.
Renamed also mentioned places.


text:
Transaction is being rolled back automatically by some process on Ignite 2.15 and 2.16

Hi community,
Recently when trying to migrate a tool from Ignite 2.14 to 2.15 or 2.16, I started to see that our puts were being rollback automatically by some internal process on Ignite, specifically we are getting the following exception:
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException: Failed to finish transaction because it has been rolled back [timeout=60000, tx=GridDhtTxLocal[xid=2e9746e0091-00000000-13a 6-dc0d-0000-000000000001,......
Please see ignite_trasaction_rollback-log.txt for a complete stack trace.
This is my configuration:

Number of threads: 10 (putting records into caches).
Transaction Concurrency: PESSIMISTIC
Transaction Isolation: REPEATABLE_READ
Transaction timeout: 60 seconds
Cache Atomicity Mode: TRANSACTIONAL

My exact same code works on prior to these new 2 versions, so something new was added that now it is causing a problem about transactions. Could you please help to determine what could be the reason of the problem?
Thanks in advance and regards,

Could it be rolled back due to a timeout?

Hi Pavel,
The problem is that just started to happen on Ignite 2.15 and only when we use more than 1 thread, otherwise we dont have that timeout.
Best regards

only when we use more than 1 thread

Multiple threads cause contention on the same key, leading to transaction timeout and rollback?

authors: adiaz-dev ptupitsyn

repo: apache/ignite


text:
[AIRFLOW-347] Show empty DAG runs in tree view

Dear Airflow Maintainers,
Please accept this PR that addresses the following issues:

https://issues.apache.org/jira/browse/AIRFLOW-347

This issue was reported by @aoen and @criccomini
Use DAG runs to construct the list of dates to display in the tree view. The old logic used execution dates from task instances to construct the list of dates, which meant that empty DAG runs would not show up in the UI.
Testing Done:

Run the scheduler for the example_bash_operator DAG and clear out all task instances for a particular date (11/12 in the screenshot). Before the fix, the 11/12 DAG run doesn't show up in the tree view and after the fix, it does. Interacting with the modal dialog for the empty task instances doesn't throw any exceptions and works as expected.

Before

After

Current coverage is 66.03% (diff: 100%)

Merging #1892 into master will increase coverage by 0.04%

@@             master      #1892   diff @@
==========================================
  Files           129        129
  Lines          9922       9922
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           6548       6552     +4
+ Misses         3374       3370     -4
  Partials          0          0

Powered by Codecov. Last update b93e651...d2e6319

@vijaysbhat Nice! Testing now!

Works! And elegant fix as well! Thx for fixing this. +1
Merging.

+1

@r39132 thanks! I got an email from ASF git services that the PR got merged, but it's showing up as open over here. What gives?

authors: codecov-io r39132 vijaysbhat zodiac

repo: apache/incubator-airflow

license_source: bigquery


text:
[AIRFLOW-685] Add test for MySqlHook.bulk_load()

Dear Airflow Maintainers,
Please accept this PR that addresses the following issues:

https://issues.apache.org/jira/browse/AIRFLOW-685

Testing Done:

All unit tests passed

Current coverage is 66.61% (diff: 100%)

Merging #1929 into master will increase coverage by 0.04%

@@             master      #1929   diff @@
==========================================
  Files           135        135
  Lines         10133      10133
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           6745       6750     +5
+ Misses         3388       3383     -5
  Partials          0          0

Powered by Codecov. Last update b674656...d28c5cc

authors: codecov-io sekikn


text:
[AIRFLOW-1342] enable S3Hook to use host from connection

Dear Airflow maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following AIRFLOW-1342 issue and references them in the PR title.

Description

[x] Change S3Hook to first try to get host from connection, then, if it is None, get it from extra_params

Tests

[x] My PR adds an new test method test_get_host_from_connection in S3HookTest to test host extraction from connection

Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

When possible, can someone help me in explain why did TravisCI marked my branch as failed? I can see the error, but I don't know how can I fix it.

@victorddiniz  Looks like an unrelated test case failed (SchedulerJobTest.test_scheduler_run_duration). It has a time-based assertion that seemed to be causing intermittent failure.
You may want one of the committers to rerun the test, or a quick fix is to do another branch push that triggers the travis CI build

Hi @jgao54! I can rerun a build but the problem is the assertion still fails and it is not related to my PR. My fear is that my PR don't get approved because of that.

@victorddiniz  hmmm... In my past experience rebuild will sometimes fix itself given the assertion is on timing, and it's only failing on 1 of the 6 environments. So it may be worth a shot :)

@jgao54, thank you! I rebuilt just this task and it succeeded! Now I'll just wait for some feedback and, with some luck, get my PR merged! 😄😄😄

@victorddiniz, you can re-trigger build by yourself, for example, with "git push --force-with-lease origin feature/AIRFLOW-1342"

authors: jgao54 victorddiniz xnuinside


text:
The redis cache plug-in cannot access the redis server with password authentication

system introduction
Answer version: 2.2.5
Answer plug-in: the latest version of the plug-in repository
Running method: Build Docker image by yourself, with plug-in
Redis plug-in display version: 0.0.1
Problem Description
Using the latest Redis plug-in, when the Redis address is filled in with password authentication, redis cannot be connected correctly. The Redis address is: redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379,
Test 1
Enable the Redis plug-in, fill in the address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379, log:
2024-02-19T02:40:34.677 app[9185d61a426038] nrt [info] 2024-02-19 10:40:34.675 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:34.831 app[9185d61a426038] nrt [info] 2024-02-19 10:40:34.830 ERROR question/question_repo.go:341 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:34.831 app[9185d61a426038] nrt [info] answer Version: 1.2.5 Revision: 5132cb3

2024-02-19T02:40:49.513 app[9185d61a426038] nrt [info] 2024-02-19 10:40:49.512 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:50.288 app[9185d61a426038] nrt [info] 2024-02-19 10:40:50.288 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.084 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.084 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.130 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.129 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.290 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.289 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.492 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.492 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.656 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.656 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.816 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.816 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:51.998 app[9185d61a426038] nrt [info] 2024-02-19 10:40:51.997 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:52.180 app[9185d61a426038] nrt [info] 2024-02-19 10:40:52.179 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:52.305 app[9185d61a426038] nrt [info] 2024-02-19 10:40:52.305 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:52.506 app[9185d61a426038] nrt [info] 2024-02-19 10:40:52.506 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:53.277 app[9185d61a426038] nrt [info] 2024-02-19 10:40:53.277 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:54.025 app[9185d61a426038] nrt [info] 2024-02-19 10:40:54.024 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:54.051 app[9185d61a426038] nrt [info] 2024-02-19 10:40:54.050 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:54.090 app[9185d61a426038] nrt [info] 2024-02-19 10:40:54.089 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:54.235 app[9185d61a426038] nrt [info] 2024-02-19 10:40:54.235 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:54.967 app[9185d61a426038] nrt [info] 2024-02-19 10:40:54.967 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:58.436 app[9185d61a426038] nrt [info] 2024-02-19 10:40:58.435 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:58.565 app[9185d61a426038] nrt [info] 2024-02-19 10:40:58.564 ERROR action/captcha_strategy.go:36 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:59.318 app[9185d61a426038] nrt [info] 2024-02-19 10:40:59.317 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:59.391 app[9185d61a426038] nrt [info] 2024-02-19 10:40:59.390 ERROR action/captcha_strategy.go:36 dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] 2024-02-19 10:40:59.472 ERROR handler/handler.go:53 code: 500, reason: base.database_error, message: , error: dial tcp: address redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/repo/captcha/captcha.go:94 github.com/apache/incubator-answer/internal/repo/captcha.(*captchaRepo).SetCaptcha

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/service/action/captcha_service.go:175 github.com/apache/incubator-answer/internal/service/action.(*CaptchaService).GenerateCaptcha

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/service/action/captcha_service.go:87 github.com/apache/incubator-answer/internal/service/action.(*CaptchaService).ActionRecord

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/controller/user_controller.go:502 github.com/apache/incubator-answer/internal/controller.(*UserController).ActionRecord

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/context.go:174 github.com/gin-gonic/gin.(*Context).Next

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/base/middleware/auth.go:64 github.com/apache/incubator-answer/internal/router.(*AnswerAPIRouter).RegisterMustUnAuthAnswerAPIRouter.(*AuthUserMiddleware).Auth.func1

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/context.go:174 github.com/gin-gonic/gin.(*Context).Next

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/gin.go:620 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/gin.go:576 github.com/gin-gonic/gin.(*Engine).ServeHTTP

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /usr/lib/go/src/net/http/server.go:2938 net/http.serverHandler.ServeHTTP

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /usr/lib/go/src/net/http/server.go:2009 net/http.(*conn).serve

2024-02-19T02:40:59.473 app[9185d61a426038] nrt [info] /usr/lib/go/src/runtime/asm_amd64.s:1650 runtime.goexit

At the same time, when you visit the project again and log in, you will be prompted to enter a verification code, but the verification code has not been set yet.
Test 2
Test the connection default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379, reset the Answer, and fill in the Redis connection default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379 again. The log is as follows:
2024-02-19T02:49:48.570 app[9185d61a426038] nrt [info] start cron

2024-02-19T02:49:48.731 app[9185d61a426038] nrt [info] 2024-02-19 10:49:48.730 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:49:48.887 app[9185d61a426038] nrt [info] 2024-02-19 10:49:48.886 ERROR question/question_repo.go:341 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:49:48.888 app[9185d61a426038] nrt [info] answer Version: 1.2.5 Revision: 5132cb3

2024-02-19T02:50:11.946 app[9185d61a426038] nrt [info] 2024-02-19 10:50:11.944 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:12.670 app[9185d61a426038] nrt [info] 2024-02-19 10:50:12.669 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:12.678 app[9185d61a426038] nrt [info] 2024-02-19 10:50:12.677 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:12.700 app[9185d61a426038] nrt [info] 2024-02-19 10:50:12.699 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:12.912 app[9185d61a426038] nrt [info] 2024-02-19 10:50:12.912 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:12.928 app[9185d61a426038] nrt [info] 2024-02-19 10:50:12.928 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:12.943 app[9185d61a426038] nrt [info] 2024-02-19 10:50:12.942 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:13.134 app[9185d61a426038] nrt [info] 2024-02-19 10:50:13.133 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:13.251 app[9185d61a426038] nrt [info] 2024-02-19 10:50:13.250 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.047 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.046 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.113 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.112 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.138 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.137 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.296 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.295 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.428 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.428 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.718 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.717 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:14.863 app[9185d61a426038] nrt [info] 2024-02-19 10:50:14.863 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:15.169 app[9185d61a426038] nrt [info] 2024-02-19 10:50:15.168 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:15.410 app[9185d61a426038] nrt [info] 2024-02-19 10:50:15.409 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:15.603 app[9185d61a426038] nrt [info] 2024-02-19 10:50:15.602 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:16.375 app[9185d61a426038] nrt [info] 2024-02-19 10:50:16.374 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:17.158 app[9185d61a426038] nrt [info] 2024-02-19 10:50:17.158 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:17.330 app[9185d61a426038] nrt [info] 2024-02-19 10:50:17.330 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:23.782 app[9185d61a426038] nrt [info] 2024-02-19 10:50:23.781 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:23.897 app[9185d61a426038] nrt [info] 2024-02-19 10:50:23.895 ERROR action/captcha_strategy.go:36 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:24.598 app[9185d61a426038] nrt [info] 2024-02-19 10:50:24.597 ERROR site_info/siteinfo_repo.go:101 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:24.674 app[9185d61a426038] nrt [info] 2024-02-19 10:50:24.673 ERROR action/captcha_strategy.go:36 dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] 2024-02-19 10:50:24.819 ERROR handler/handler.go:53 code: 500, reason: base.database_error, message: , error: dial tcp: address default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/repo/captcha/captcha.go:94 github.com/apache/incubator-answer/internal/repo/captcha.(*captchaRepo).SetCaptcha

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/service/action/captcha_service.go:175 github.com/apache/incubator-answer/internal/service/action.(*CaptchaService).GenerateCaptcha

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/service/action/captcha_service.go:87 github.com/apache/incubator-answer/internal/service/action.(*CaptchaService).ActionRecord

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/controller/user_controller.go:502 github.com/apache/incubator-answer/internal/controller.(*UserController).ActionRecord

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/context.go:174 github.com/gin-gonic/gin.(*Context).Next

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/apache/incubator-answer/internal/base/middleware/auth.go:64 github.com/apache/incubator-answer/internal/router.(*AnswerAPIRouter).RegisterMustUnAuthAnswerAPIRouter.(*AuthUserMiddleware).Auth.func1

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/context.go:174 github.com/gin-gonic/gin.(*Context).Next

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/gin.go:620 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /answer_build1258453270/vendor/github.com/gin-gonic/gin/gin.go:576 github.com/gin-gonic/gin.(*Engine).ServeHTTP

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /usr/lib/go/src/net/http/server.go:2938 net/http.serverHandler.ServeHTTP

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /usr/lib/go/src/net/http/server.go:2009 net/http.(*conn).serve

2024-02-19T02:50:24.822 app[9185d61a426038] nrt [info] /usr/lib/go/src/runtime/asm_amd64.s:1650 runtime.goexit

When logging in, the same problem occurs as in test 1.
Test 3
Test the Redis connection ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379, the log is as follows:
 nrt [info] 2024-02-19 10:56:52.641 ERROR site_info/siteinfo_repo.go:101 dial tcp: lookup ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io: no such host

2024-02-19T02:56:52.956 app[9185d61a426038] nrt [info] 2024-02-19 10:56:52.950 ERROR question/question_repo.go:341 dial tcp: lookup ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io: no such host

2024-02-19T02:56:52.963 app[9185d61a426038] nrt [info] answer Version: 1.2.5 Revision: 5132cb3

Test connection redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379, log:
[info] 2024-02-19 11:25:33.298 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:33.456 app[9185d61a426038] nrt [info] 2024-02-19 11:25:33.455 ERROR question/question_repo.go:341 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:33.458 app[9185d61a426038] nrt [info] answer Version: 1.2.5 Revision: 5132cb3

2024-02-19T03:25:34.563 app[9185d61a426038] nrt [info] 2024-02-19 11:25:34.562 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:36.709 app[9185d61a426038] nrt [info] 2024-02-19 11:25:36.708 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:36.751 app[9185d61a426038] nrt [info] 2024-02-19 11:25:36.751 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:39.425 app[9185d61a426038] nrt [info] 2024-02-19 11:25:39.424 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:39.562 app[9185d61a426038] nrt [info] 2024-02-19 11:25:39.561 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:39.764 app[9185d61a426038] nrt [info] 2024-02-19 11:25:39.764 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

2024-02-19T03:25:40.011 app[9185d61a426038] nrt [info] 2024-02-19 11:25:40.010 ERROR site_info/siteinfo_repo.go:101 dial tcp: address redis://ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379: too many colons in address

Build Dockerfile
FROM apache/answer as answer-builder

RUN apk --no-cache add \
    build-base git bash nodejs npm go && \
    npm install -g pnpm

RUN answer build \
    --with github.com/apache/incubator-answer-plugins/search-meilisearch \
    --with github.com/apache/incubator-answer-plugins/cache-redis \
    --with github.com/apache/incubator-answer-plugins/storage-s3 \
    --output /usr/bin/new_answer

FROM apache/answer

COPY --from=answer-builder /usr/bin/new_answer /usr/bin/answer
RUN chmod 755 /entrypoint.sh

VOLUME /data
EXPOSE 80
ENTRYPOINT ["/entrypoint.sh"]

@LinkinStars My Redis is provided by upstash.com. The link provided is in the form of redis://default:ce5a7b1d80e44320a7877ac2aafae5cd@123.upstash.io:6379. The specific explanation is: Protocol name://username:user password@ Host domain name or IP:port, only use 123.upstash.io:6379, unable to provide access password to upstash.com.

@lincolnzhou Just a small follow-up, do you want to continue contributing features about configuring redis usernames? Because you did the feature of the password before. Of course, if there is no time, I can assign it to someone else. Just asking out of courtesy.

Okay, I'll try to fix it

#71 Please review this pr

@lincolnzhou @LinkinStars I have tested the redis plug-in version 1.2.6. After entering the address, username, and password, redis is enabled normally and this problem is closed.

authors: aaro-n lincolnzhou

repo: apache/incubator-answer-plugins


text:
doc: corrected some typo

What this PR does / why we need it:
there is two Nginx typo  in the documents
Pre-submission checklist:

[x] Did you explain what problem does this PR solve? Or what new features have been added?
[x] Have you added corresponding test cases?
[x] Have you modified the corresponding document?
[x] Is this PR backward compatible?

many thx, merged @lilien1010

repo: apache/incubator-apisix


text:
Zooming into detail text stays at same scale

When zooming in on the demo, the text for roads, places etc all stay at the original scale where as the vector features scale as you zoom. This makes trying to zoom to read text impossible.

Thanks a lot for reporting this issue, I also think the font size is too small right now.

authors: aexley bchapuis

repo: apache/incubator-baremaps


text:
[Bug][framework] Download log is not working

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

What do you expect to happen
being able to download log file
How to reproduce
ui-my-svr02
Anything else
No response
Version
main
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Same to #3948

authors: klesh likyh

repo: apache/incubator-devlake


text:
[Feature][Jira] Add Resolution Field to Domain Layer Issues Entity for Jira Source Type

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Use case
As a user of DevLake, I want to have the resolution field available in the domain layer issues entity when the source type is Jira, so that I can analyze and visualize the resolution status of issues in my engineering analytics.
Description
Currently, the domain layer issues entity does not include the resolution field, which is an important piece of information for understanding the status of issues in Jira. This feature request aims to add the resolution field to the domain layer issues entity and populate it when the source type is Jira.
To implement this feature, the following steps should be taken:

Update the domain layer schema to include a new field called "resolution" in the issues entity. This field should be a string type to accommodate the various resolution statuses that can be present in Jira (e.g., Fixed, Won't Fix, Duplicate, etc.).

Modify the data transformation process to map the resolution field from the Jira tool layer schema to the domain layer schema when the source type is Jira. This may involve updating the ETL (Extract, Transform, Load) process or any other data transformation logic used in DevLake.

Ensure that the resolution field is populated correctly in the domain layer issues entity when the source type is Jira. This may require testing with different Jira projects and issue types to verify that the resolution field is accurately captured and transformed.

Update any relevant documentation and user guides to reflect the addition of the resolution field in the domain layer issues entity.

By adding the resolution field to the domain layer issues entity and populating it when the source type is Jira, users of DevLake will be able to gain more insights into their engineering processes and make data-driven decisions for improving their development workflows.
Related issues
No response
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

[ ] Add a resolution_reason field to table.issues
[ ] Add a resolution_reason field to _tool_jira_issues
[ ] For jira plugin, the value of resolution_reason should be extracted and transformed from fields.resolution.name in table _raw_jira_api_issues.

[TBD] I'm still investigating if there's resolution_date for Zentao and TAPD

authors: Startrekzky narrowizard


text:
[Refactor][framework] Drop temporal support

What and why to refactor
We once believed it would be beneficial to support collecting data distributedly via Temporal and #1513 had been done for more than a year. However, it is never been used by anyone due to the following limitation:

Most data source has rate limit which makes distributed collection practically useless
It is quite expensive to deploy such a distributed system because workers must be deployed into different machines

To sum up, temporal support is not practical for us except for adding complexity to the codebase which makes refactoring harder.
Describe the solution you'd like
Drop Temporal support, remove related code and simplify the pipeline execution logic

I can take this one @klesh


text:
[Feature] [Vectorized] Support for vectorization in load

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
A lot of things to do in load’s vectorization from the top down.
Stream Load：
1.StreamLoadScanNode vec
2.OlapTableSink vec
3.LoadChannel vec
4.Memtable vec
5.Rowset and Segment writter vec may be implemented by  pr #8438
Spark Load:

Parquet read vec
Rowset Writter vec #8438

Use case
No response
Related issues
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Doris has many Load ways, in addition to Spark Load and Steam Load, also include Routine Load and Broker Load.
I think you need a more complete planning.

Stream load is on the way by some other community guys. The PR may be submitted in next couple weeks.

Stream load is on the way by some other community guys. The PR may be submitted in next couple weeks.

this is mean that someone is dealing with stream Load vec work?

Yes

A small suggestion: as there are more and more developers in the community, we really need a global task management list to show which ones are under investigation, which ones are in progress and which ones have been completed, and update the progress in time to avoid unnecessary waste of manpower.

A small suggestion: as there are more and more developers in the community, we really need a global task management list to show which ones are under investigation, which ones are in progress and which ones have been completed, and update the progress in time to avoid unnecessary waste of manpower.

+1

authors: Userwhite morningman spaces-X wangbo wangshuo128

repo: apache/incubator-doris


text:
Restful

reconstruction dubbo admin, based on Vuetify and spring boot

I would like to double confirm about i18n. with this PR, do we still have i18n support?

@htynkn there's no i18n support yet

authors: htynkn nzomkxia

repo: apache/incubator-dubbo-ops


text:
Dubbo2.7.1 服务shutdown后zk中的注册信息没有自动下线

我打算把项目中的dubbo版本由2.5.3升级至2.7.1，依赖升级后程序能够正常启动，但是发现在关闭服务后，注册中心zk中的 /dubbo/xxxService/providers下仍然存在该关闭服务的提供者信息。在2.5.3版本中没有这个问题（当服务关闭后，zk中会自动下线该提供者信息），以下是我的对应依赖版本：
dubbo2.7.1
curator 2.10.0
zookeeper 3.4.6
项目中只是把dubbo升级2.5.3 ->2.7.1 , 其他的那两个依赖包版本没有变动。

2.7.1 zk的服务节点是永久节点 这个问题已经在2.7.2中修复,或者通过在service中配置    <dubbo:service interface="org.apache.dubbo.demo.DemoService" ref="demoService" dynamic="true"/>

试了，加了dynamic="ture"好像管用。。。啥时候发布2.7.2？

试了，加了dynamic="ture"好像管用。。。啥时候发布2.7.2？
不出意外
应该这个月会发布的

好的，谢谢！~ 希望能早点发布。。

repo: apache/incubator-dubbo


text:
EAGLE-348 Alert engine base on spark streaming

EAGLE-424
Support kafka add/remove topic without requiring restart of the alert engine
Read spec metadata by restful web app
Create UnitSparkTopologyRunner to demo spark streaming usage
Update eagle-alert/README.md to describe how to start spark streaming alert engine
EAGLE-425
Use Spark Accumulator to save and recover state
Use Spark Accumulator to support Hybird Time Window

https://issues.apache.org/jira/browse/EAGLE-348

One of the target is to make sure alert engine could be decoupled to run on both Storm and Spark, while they should share the same metadata, I don't understand why do we need spark specific metadata like: SpecMetadataResource?

For now , I fetch spec config directly from metadata service, Just skip coordinator service. So I create a simple service just return spec config. In the future , I will consider to refactor coordinator service base on Spark platform.

repo: apache/incubator-eagle


text:
npm install echarts v4.2.1 在项目中 import 打印看到的版本号是4.1.0

Version
4.2.1
Steps to reproduce
import echarts from 'echarts/lib/echarts'
console.log(echarts.version)
What is expected?
4.2.1
What is actually happening?
4.1.0

安装的时候是 npm i echarts@4.2.1 的吗？

^4.2.1
dist/echarts 中的 version 是4.2.1，zrender为4.0.6
webpack现在设置了externals，表现正常了

authors: Ovilia houjiazong

repo: apache/incubator-echarts


text:
WIP(axisPointer): snapping logic WIP.

Fix #7651
/**
* If both x and y axes are snapping and one and only one of them is
* of category type, then the category axis is used as primary
* snapping, otherwise the baseAxis is used as primary snapping.
* The snapped value of the other axis is calculated from it.
*
*  4 |                 *
*  3 |       *
*  2 |            *
*  1 |  *     +
*    ---------------------
*       a    b    c    d
*
* In the above example, the x axis is in category type, and y axis
*     value.
* The *'s are data items, + is the mouse position.
*
* If both x and y axes are using snapping, then:
* 1. x axis is used as primary snapping axis because it's the only
*     category axis.
* 2. The nearest x value to the mouse is 'b', so 'b' is used as
*     snapped x value.
* 3. The corresponding y value of 'b' is 3, so 3 is used as snapped y
*     value.
* 4. So the axisPointer should be at ('b', 3).
*
* If only x is snapping, then the axisPointer should be at ('b', 1).
* If only y is snapping, then the axisPointer should be at the
*     position of +.
*/

@Ovilia @pissang @100pah This feature would be so awesome to have and it looks like the tests have passed. Do you anticipate this getting merged in? Thanks for your continued work on the project! Love ECharts.

Too many conflicts when merging next. Close this PR and open a new one to next branch.

authors: Ovilia masiamj


text:
The development version has ClassCastException

Question
The development version has ClassCastException, which needs to be solved.

I know this problem, and I will fix it later.

repo: apache/incubator-eventmesh


text:
[VL] Clean up some legacy code related to USE_AVX512

What changes were proposed in this pull request?
These are some legacy code inherited from Gazelle. And it's not the correct place to do this setting for compiler if we want it for compiling all native code.

@zhouyuan

Let's add -mno-avx512f -mbmi2 to compile flag, align with Velox.

Currently, we're passing -march=native, this doesn't seem to respect -mno-avx512f

Using -march=native ends us up generating ASM code with vmovdqu8 instructions when CPU supports AVX512, and makes it incompatible with older CPUs, forcing the build machine and run machine to be of same type. I explicitly passed -mno-avx512f -mbmi2 but it didn't do anything and AVX512 instructions were still present in the ASM code. Any idea why it doesn't respect -mno-avx512f?

Using -march=native ends up generating ASM code with vmovdqu8 instructions when CPU supports AVX512, and makes it incompatible with older CPUs, forcing the build machine and run machine to be of same type. I explicitly passed -mno-avx512f -mbmi2 but it didn't do anything and AVX512 instructions were still present in the ASM code. Any idea why it doesn't respect -mno-avx512f?

@surnaik, -mno-avx512f only disables the subset of avx512 instructions. I recommend you to remove the use of -march=native in cross compilation and set correct target cpu for compiler. For example, you can set -march=haswell if haswell is used on your side.
If your cluster has diverse cpu architectures, you can just use generic setting for compiler. Alternatively, to make the binary fully optimized for newer cpu architecture, I think you have to customize gluten build for different cpu architectures, instead of compiling once for universal use.

@surnaik, -mno-avx512f only disables the subset of avx512 instructions. I recommend you to remove the use of -march=native in cross compilation and set correct target cpu for compiler. For example, you can set -march=haswell if haswell is used on your side. If your cluster has diverse cpu architectures, you can just use generic setting for compiler. Alternatively, to make the binary fully optimized for newer cpu architecture, I think you have to customize gluten build with different -march set for different cpu architectures, instead of compiling once for universal use.

-mno-avx512f should take effect after -march=native.

authors: FelixYBW PHILO-HE surnaik

repo: apache/incubator-gluten


text:
Java 9+ runtime support

Heron should be able to run with Java 9+. Opening this issue to track all the tasks required to make this happen -

JVM unified logging - #3268

the master branch is on java11 now

repo: apache/incubator-heron


text:
Update Kubernetes proxy URLs

Fix https://github.com/apache/incubator-heron/issues/3404
Follow the Kubernetes proxy URLs documented here (Note that proxy is placed after the service name instead of /api/v1):

http://kubernetes_master_address/api/v1/namespaces/namespace_name/services/service_name[:port_name]/proxy

Note that the helm configuration file at https://storage.googleapis.com/heron-charts needs to be updated as well. Otherwise, users following the helm guide will not be able to get any stats from heron UI.

I believe the file has already been updated in this repository (see below), but the file at https://storage.googleapis.com/heron-charts was not changed for a while (The last modified time is 2018-04-18).
https://github.com/apache/incubator-heron/blob/d8db1c50b499c823e319fd16a4476a02f525a367/deploy/kubernetes/helm/templates/tools.yaml#L109-L115

I'm going to let this sit for a day and see if anyone else has any feedback.

I am not familiar with this part but the change looks reasonable. Thanks.
Please feel free to merge.

authors: ShawnZhong joshfischer1108 nwangtw


text:
Allow KieRuntimeBuilder to also create and provide StatelessKieSession

At the moment the KieRuntimeBuilder only allow to retrieve KieSession s, but not StatelessKieSession s. Even worse when a KieSession configured as stateless is required through the current API it creates a stateful one anyway, making things even more confused.
Finally the impossibility of creating a StatelessKieSession from KieRuntimeBuilder is also the indirect cause of a memory leak with the Kogito integration as reported here.

Fixed by https://github.com/apache/incubator-kie-drools/pull/6103 and https://github.com/apache/incubator-kie-kogito-runtimes/pull/3681

repo: apache/incubator-kie-issues


text:
Workflow instances that are in error should have their end date set

Describe the bug
In some UIs, showing running process instances, the end date of shown or used to calculate duration.  For process instances that are in error state, it seems the process instance end date is not set in the data index.  Since process instances in error state cannot be recovered, my suggestion is that the end date is set for these instances as well (similar to completed and aborted).
Expected behavior
No response
Actual behavior
No response
How to Reproduce?
No response
Output of uname -a or ver
No response
Output of java -version
No response
GraalVM version (if different from Java)
No response
Kogito version or git rev (or at least Quarkus version if you are using Kogito via Quarkus platform BOM)
No response
Build tool (ie. output of mvnw --version or gradlew --version)
No response
Additional information
No response

@krisv The issue is not that the end date is not set.  Thats a side effect of current runtimes behaviour. Currently a process in error state is still kept active (in memory or in db) by runtimes. Should we change that behaviour and complete a process instance that is on error? If so, error state will behave like aborted or completed states, unlike now, which behaves as active state.

Closing together with the PR. In error processes are process that are still active so user might retry changing variables. So "error" state should be understood  "as active with previous attempt failed because of an recoverable error"

authors: fjtirado krisv

repo: apache/incubator-kie-kogito-runtimes


text:
NO-ISSUE: DMN Editor: Overlay modal can't be scrolled

Before:

After:

When it has enough area, no scroll is shown:

@Josephblt I think this is a minor issue since it doesn't compromise usability like the issue that this PR is fixing, it is just ugly. We can add "live resize" for the panel that adapts the screen as its available size changes but in my opinion this is another ticket. What do you think, @tiagobento ?

Hello @danielzhe, I completely agree. The improvement is clear.  This change gives access to all the options, and the problems are merely aesthetical. Plus they are easily solvable with a few clicks. So I don´t see any problens in addressing this in another issue.

authors: Josephblt danielzhe

repo: apache/incubator-kie-tools


text:
NO-ISSUE: Maven plugins update

In this PR:

Updated Apache parent pom to version 33
Removed all plugin declarations already present in the Apache parent pom
Updated most of the maven plugin versions
Removed stale profile reference in serverless-workflow

@yesamer I guess I didn't write this anywhere, but the idea of managing the versions of Maven plugins ourselves is to have more control over patches that may contain security fixes. If we delegate it to the Apache parent we may find ourselves in a position of having to declare them anyway to fix something that the Apache parent hasn't.

@tiagobento Well, at the moment a CVE is detected in any plugin version, we can temporarily override. I guess that is a good compromise on having the same plugin versions used anywhere and keeping them updated.

@tiagobento I checked the CVE history of all declared plugins in Apache parent pom.
An example: maven-assembly-plugin
https://security.snyk.io/package/maven/org.apache.maven.plugins%3Amaven-assembly-plugin
https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-assembly-plugin
I didn't find any reported CVE for those plugins EVER.
Based on that, I guess we are safe to proceed ;)

@yesamer Problems are often in transitive dependencies...

@tiagobento Even in such a case, those transitive dependencies are not imported in the production code, as that is used at compile time.

@tiagobento Of course, I'm available to discuss this topic in private, but if you think for real I did this PR casually after all the evidences I gave to you, trying to explain that the plugin management is NOT THE SAME of the dependency management, I think I failed my intent, and I no more interested to keep this effort in place.

@tiagobento Many apologies for my impulsive rant, that's not my standard and my style. Taking a real break here, we can talk about this in private after the vacation.

authors: tiagobento yesamer


text:
Refactoring Dockerfile

Migrate to latest Ubuntu LTS (22.04)
Build inside docker (buildx support)

@foxdalas Thanks for your contribution. No worry about the failure of CI tests, will find the root cause and fix it later.

@git-hulk Thanks. Maybe do you need ci for docker with many platforms?

@git-hulk I built current head on my laptop. You can pull image foxdalas/kvrocks:latest amd64/arm64. Everything works in AWS EKS.

Thanks @foxdalas, are you using this Dockerfile, I got the below error when compiling.
include/jemalloc/internal/rtree.h:109:33: error: variably modified 'root' at file scope
  109 |         rtree_leaf_elm_t        root[1U << (RTREE_NSB/RTREE_HEIGHT)];

@git-hulk I built current head on my laptop. You can pull image foxdalas/kvrocks:latest amd64/arm64. Everything works in AWS EKS.

Thanks @foxdalas, are you using this Dockerfile, I got the below error when compiling.
include/jemalloc/internal/rtree.h:109:33: error: variably modified 'root' at file scope
  109 |         rtree_leaf_elm_t        root[1U << (RTREE_NSB/RTREE_HEIGHT)];

It's strange. I will check it tomorrow. Thanks.

@git-hulk please approve buildx actions.

Sorry that I also don't have the permission now. Maybe @tisonkun @Hexiaoqiao can help to approve it.

Sorry that I also don't have the permission now. Maybe @tisonkun @Hexiaoqiao can help to approve it.

Thanks for support.

@tisonkun @Hexiaoqiao approve action please

@tisonkun @Hexiaoqiao one more run please

@git-hulk app checks have passed. Please check your environment everything forks on ubuntu 22.04 in docker.

Cool, thanks. Will have a try later.

@foxdalas @git-hulk please note that docker workflow takes about 197 minutes to complete. We may not run such workflow for every PR every activities.

@foxdalas @git-hulk please note that docker workflow takes about 197 minutes to complete. We may not run such workflow for every PR every activities.

It's qemu 😢 I think i will find workaround

@tisonkun I can add cross-compiling to project or we can build docker image on tag push.
What do you think about release action with docker build and push ?

@foxdalas We have built and pushed the docker image when releasing the new tag.
Can see: https://github.com/apache/incubator-kvrocks/blob/unstable/.github/workflows/release.yaml#L80

@foxdalas We have built and pushed the docker image: release.yaml#L80. I think we can add the cross compiling on the daily CI instead of every PR, HDYT?

@foxdalas We have built and pushed the docker image: release.yaml#L80. I think we can add the cross compiling on the daily CI instead of every PR, HDYT?

Do you mean docker buildx cross compiling? It's easy

Yes, as @tisonkun mentioned, the new docker job need 197 minutes to complete, it's too slowly and unnecessary for most PR, so it would be better to add to Daily CI.

@git-hulk @tisonkun review please

did you run daily CR? works well? how long did it take? since currently we only run  kvrocks.yaml CI when submitting a PR.

@foxdalas Can you rollback the cross compile in release.yaml? For Daily CI, 197 minutes was ok to me, but it's too slowly to release.

@foxdalas Can you rollback the cross compile in release.yaml? For Daily CI, 197 minutes was ok to me, but it's too slowly to release. My point was we want to guarantee the code can compile on different platforms first.

Yes,sure.
Do you have another way for arm64 release?

@git-hulk Maybe i didn't describe clearly. I think release.yaml is OK since it runs in the background.  But daily Ci takes much time.
Let's keep current solution. We can optimize if there is a real problem.

OK, I'm ok to go head with the current way. We can improve this before GitHub Actions supporting the arm64 runner.

authors: ShooterIT foxdalas git-hulk tisonkun

repo: apache/incubator-kvrocks


text:
Support to use builtin spark python lib with spark.yarn.isPython

Why are the changes needed?

Support to use builtin spark python lib with spark.yarn.isPython
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[x] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

cc @cfmcgrady  @bowenliang123

LGTM. Well done.

Codecov Report

Merging #3908 (4901e84) into master (a0fc33c) will decrease coverage by 0.01%.
The diff coverage is 90.90%.

@@             Coverage Diff              @@
##             master    #3908      +/-   ##
============================================
- Coverage     51.91%   51.89%   -0.02%
  Complexity       13       13
============================================
  Files           508      508
  Lines         29019    29026       +7
  Branches       3984     3985       +1
============================================
- Hits          15066    15064       -2
- Misses        12525    12531       +6
- Partials       1428     1431       +3

Impacted Files
Coverage Δ

.../kyuubi/engine/spark/operation/ExecutePython.scala
83.51% <90.90%> (+0.08%)
:arrow_up:

.../kyuubi/server/mysql/constant/MySQLErrorCode.scala
13.84% <0.00%> (-6.16%)
:arrow_down:

...ache/kyuubi/server/mysql/MySQLCommandHandler.scala
77.77% <0.00%> (-4.05%)
:arrow_down:

...ache/kyuubi/operation/KyuubiOperationManager.scala
79.45% <0.00%> (-2.74%)
:arrow_down:

...ache/kyuubi/server/mysql/MySQLGenericPackets.scala
76.59% <0.00%> (-2.13%)
:arrow_down:

.../engine/spark/session/SparkSQLSessionManager.scala
78.48% <0.00%> (-1.27%)
:arrow_down:

...org/apache/kyuubi/operation/ExecuteStatement.scala
79.16% <0.00%> (ø)

...pache/kyuubi/engine/YarnApplicationOperation.scala
62.96% <0.00%> (ø)

.../org/apache/kyuubi/operation/KyuubiOperation.scala
77.77% <0.00%> (+4.93%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Thanks, merged to master(1.7.0).

authors: bowenliang123 cfmcgrady codecov-commenter turboFei

repo: apache/incubator-kyuubi


text:
[Clojure] - Add Spec Validations to the Optimizer namespace

It would be nice to add some spec validations to the optimizer namespace https://github.com/apache/incubator-mxnet/blob/master/contrib/clojure-package/src/org/apache/clojure_mxnet/optimizer.clj
So that if a user calls a function with the incorrect arguments, it will guide them to the correct form.
Using the util/validate function as in https://github.com/apache/incubator-mxnet/blob/master/contrib/clojure-package/src/org/apache/clojure_mxnet/module.clj#L186 and the correct specs would be a great addition to the project.
Addition of unit tests to test a failing case of the spec that an exception is thrown would be great too :)

Thanks for fixing this @hellonico :)

repo: apache/incubator-mxnet


text:
Cannot access child blocks' parameter in parent block's hybrid_forward

I have a very simple model:
class MyBlock(HybridBlock):
    def __init__(self):
        self.dense1 = nn.Dense(10)
        self.dense2 = nn.Dense(10)

    def hybrid_forward(self, F, x):
        # I want to concat the weights of `self.dense1` and `self.dense2`.
        # how do I do that?

I suggest we deprecate hybrid_forward, in which parameters are passed as function arguments.
Instead users should access parameters via the self.weight.data(ctx) syntax used in Block.
Then you may access child-block parameters as self.childblock.weight.data(ctx).

Shall we also consider the case where input is a symbol so the ctx is not visible to forward?

What's the use-case to input symbol into forward? It wouldn't work in most cases because forward doesn't have F.

nevermind, in gluon 2.0 the symbol and ndarray will be unified. As long as it still support hybridization, it's fine

authors: eric-haibin-lin leezu


text:
Floating point exception in mxnet.contrib.ndarray.arange_like

Description
Floating point exception in mxnet.contrib.ndarray.arange_like and mxnet.ndarray.contrib.arange_like when repeat is 0.
Error Message
In mxnet 1.6:
Floating point exception (core dumped)

In mxnet nightly version (2.0.0):
terminate called after throwing an instance of 'dmlc::Error'
  what():  [22:27:50] ../src/initialize.cc:379: FloatingPointError: Floating point exception

Aborted (core dumped)

To Reproduce
import mxnet as mx
mx.contrib.ndarray.arange_like(data=mx.nd.ones(1), repeat=0)
mx.ndarray.contrib.arange_like(data=mx.nd.ones(1), repeat=0)

Environment
OS: ubuntu 18.04
Python: 3.7.6
pip: 20.0.2
numpy: 1.18.5
mxnet: 1.6.0

0 shapes are unsupported in the mxnet.nd interface. Please update to MXNet 2 and use mxnet.np (or use the experimental mxnet.np in mxnet 1.x versions).
I'm closing the issue as it's unsupported.

You can refer to https://github.com/apache/incubator-mxnet/issues/14253 for more information

@leezu  This issue doesn't involve 0 in shape.

What do you expect repeat=0 to do?

We can reopen this issue and track improving the error message. It would be great to have a human-readable message instead of a crash.

@leezu Sounds great.

@leezu Sounds great.

authors: DNXie leezu


text:
[WIP]Added new C_Apis to suport Large vectors and added tests for initial set of ops

Description
New C_apis and verification of Large tensor support in a couple of operators
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@mxnet-label-bot add [pr-work-in-progress]

@mxnet-label-bot add [pr-awaiting-review]

@apeforest @ChaiBapchya this is now ready for review

@apeforest @ChaiBapchya pr is ready for review

Made a note of the skipped tests in this issue. For easy tracking.

Please paste your output here after you run through the test locally. Thanks!

authors: ChaiBapchya access2rohit apeforest


text:
Comments formatting fix

Description
This PR fixes the formatting of some comments so that they can be parsed by a documentation generator.

@mxnet-bot run ci [centos-gpu, edge, windows-gpu]

@mxnet-bot run ci [centos-gpu, windows-gpu]

@mxnet-bot run ci [centos-gpu, windows-gpu]

@mxnet-bot run ci [windows-gpu]

authors: DominikaJedynak bgawrych


text:
Modify NDArrayIter constructor to receive tuple (i.e. dict in Python)…

Description
For multiple inputs or multiple labels in NDArrayIter, they are assigned IndexedSeq[NDArray] type, which is not flexible to design neural network like Python API because the naming rule here is dataname_0, dataname_1, etc. In more complex network you may want to give a meaningful names like "user" and "item" in Matrix Factorization example.
We modify the constructor to receive IndexedSeq[(String, NDArray)] type to allow assigning custom names. We can initialize the NDArrayIter like this:
val trainData1 = IndexedSeq(("user", NDArray.array(Array(1,2,3,4,5,6,3,2,7,1,6,9), shape = Shape(6,2))))
val trainData2 = IndexedSeq(("item", NDArray.array(Array(1,2,3,4,5,6,3,2,6,1,6,9), shape = Shape(6,2))))
val trainLabel = IndexedSeq(("label", NDArray.array(Array(5,11,17,7,9,24), shape = Shape(6))))
val trainIter = new NDArrayIter(
  trainData1 ++ trainData2,
  trainLabel,
  1,
  false,
  "pad")

In order to compatible with the old argument version, we overload the constructor.

Please consider adding an additional constructor instead of modifying the existing constructor to provide backwards compatibility

Adding an additional constructor is our first consideration. But after observing the original interface:
class NDArrayIter (data: IndexedSeq[NDArray], label: IndexedSeq[NDArray] = IndexedSeq.empty,
                  private val dataBatchSize: Int = 1, shuffle: Boolean = false,
                  lastBatchHandle: String = "pad",
                  dataName: String = "data", labelName: String = "label") extends DataIter {...

we found there is no position to place the custom names. Therefore, since each overloading constructor has to call primary constructor, keeping the original interface may not a possible way if we want to make using multiple input flexible.
Another reason we modifying the existing constructor is that we observer the fields defined in the class:
val initData: IndexedSeq[(String, NDArray)] = IO.initData(_dataList, false, dataName)
val initLabel: IndexedSeq[(String, NDArray)] = IO.initData(_labelList, true, labelName)

We guess maybe using the type IndexedSeq[(String, NDArray)]  is the original motivation at the beginning.
In Python there is a way to manipulate multiple input. But Scala is static type. Maybe we can try to rethink about the API interface design.

Since we are using semantic versioning, this would require a major version
update. Thus, this change can not be accepted in that form. I'd appreciate
it if you could try to find a backwards compatible solution. If you need
help in doing so, feel free to let us know and we will try to assist.
Am 13.02.2018 3:28 vorm. schrieb "Sheng-Ying" notifications@github.com:
Adding an additional constructor is our first consideration. But after
observing the original interface:
class NDArrayIter (data: IndexedSeq[NDArray], label:
IndexedSeq[NDArray] = IndexedSeq.empty,
private val dataBatchSize: Int = 1, shuffle: Boolean = false,
lastBatchHandle: String = "pad",
dataName: String = "data", labelName: String =
"label") extends DataIter {...
we found there is no position to place the custom names. Therefore, since
each overloading constructor has to call primary constructor, keeping the
original interface may not a possible way if we want to make using multiple
input flexible.
Another reason we modifying the existing constructor is that we observer
the fields defined in the class:
val initData: IndexedSeq[(String, NDArray)] = IO.initData(_dataList,
false, dataName)val initLabel: IndexedSeq[(String, NDArray)] =
IO.initData(_labelList, true, labelName)
We guess maybe using the type IndexedSeq[(String, NDArray)] is the original
motivation at the beginning.
In Python there is a way
https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/io.py#L490
to manipulate multiple input. But Scala is static type. Maybe we can try to
rethink about the API interface design.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-mxnet/pull/9771#issuecomment-365139987,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARxB603XJ_iBESltTinPdHqAt2HlfCxTks5tUQF5gaJpZM4SCBSo
.

Thank you for replying. Would you please explain more about backwards compatibility?
Since in our modified version we use this to overload the constructor, initialize NDArrayIter by
either this way
val trainData1 = IndexedSeq(NDArray.array(Array(1,2,3,4,5,6,3,2,7,1,6,9), shape = Shape(6,2)))
val trainData2 = IndexedSeq(NDArray.array(Array(1,2,3,4,5,6,3,2,6,1,6,9), shape = Shape(6,2)))
val trainLabel = IndexedSeq(NDArray.array(Array(5,11,17,7,9,24), shape = Shape(6)))
val trainIter = new NDArrayIter(
  data = trainData1 ++ trainData2,
  label = trainLabel,
  dataBatchSize = 1,
  shuffle = false,
  lastBatchHandle = "pad",
  dataName = "data",
  labelName = "label"
)

or this way
val trainData1 = IndexedSeq(("user", NDArray.array(Array(1,2,3,4,5,6,3,2,7,1,6,9), shape = Shape(6,2))))
val trainData2 = IndexedSeq(("item", NDArray.array(Array(1,2,3,4,5,6,3,2,6,1,6,9), shape = Shape(6,2))))
val trainLabel = IndexedSeq(("label", NDArray.array(Array(5,11,17,7,9,24), shape = Shape(6))))
val trainIter = new NDArrayIter(
  trainData1 ++ trainData2,
  trainLabel,
  1,
  false,
  "pad")

is acceptable. So what is the detail about backwards compatibility?

Your change has the following issues, not providing backwards compatibility:
Type change: data: IndexedSeq[NDArray] -> data: IndexedSeq[(String, NDArray)]
Default value removed: private val dataBatchSize: Int = 1 -> private val dataBatchSize: Int
Default value removed: shuffle: Boolean = false -> shuffle: Boolean
Default value removed: lastBatchHandle: String = "pad" -> lastBatchHandle: String
Argument removed: dataName: String = "data"
Argument removed: labelName: String = "label"

Backwards compatibility means that users using version 1.0 of MXNet, should not receive any errors when they upgrade to version 1.X. In your PR, you're modifying the constructor. This would throw an error for all users if they upgrade to the latest version.
Instead, you should add an additional overload of the constructor in order to support the old constructor as well as providing a new one using your proposed method. Speak, don't modify existing constructors but rather create another one.

@nswamy do you have an idea?

Hi, the community has passed to vote about associating the code changes with JIRA (https://lists.apache.org/thread.html/ab22cf0e35f1bce2c3bf3bec2bc5b85a9583a3fe7fd56ba1bbade55f@<dev.mxnet.apache.org>)
We have updated the guidelines for contributors in https://cwiki.apache.org/confluence/display/MXNET/Development+Process, please ensure that you have created a JIRA at https://issues.apache.org/jira/projects/MXNET/issues/ to describe your work in this pull request and include the JIRA title in your PR as [MXNET-xxxx] your title where MXNET-xxxx is the JIRA id
Thanks!

Sorry had missed this PR, this is already handled by another PR https://github.com/apache/incubator-mxnet/blob/master/scala-package/core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala#L42.
and
https://github.com/apache/incubator-mxnet/blob/master/scala-package/core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala#L59
Like the author mentioned there is no other way than modifying primary constructor, since all secondary constructors have to call the primary constructor. However API compatibility is still maintained since we have secondary constructors that keep the parameters from the original constructor.

@parallelgithub I am closing this PR since we have a fix in place. we are making some more changes to NDArrayIterator in this PR DataDesc, NDArrayIter, would be great to get your feedback on that.

authors: CodingCat marcoabreu nswamy parallelgithub


text:
Archiving NB 10.0

Updating links of NetBeans 10.0 to point to the Apache archive.

Great, makes sense!

authors: geertjanw vieiro

repo: apache/incubator-netbeans-website


text:
Not copying the JavaHelp library into the binary.

This avoids the runtime dependency on JavaHelp (compile-time dependency remains). Help is not working as a result. Can be partially workarounded by saying:
--cp:a <path-to-jhall.jar>
on command line, but there are still bugs with that (as JavaHelp is loaded by a classloader that does not see NetBeans classes).

Good solution for the moment. Thanks. Merging.

authors: geertjanw jlahoda

repo: apache/incubator-netbeans


text:
Elf loader sample app doesn't build

tested with master
make[3]: Entering directory '/root/nuttx/apps/examples/elf'
make[4]: Entering directory '/root/nuttx/apps/examples/elf/tests'
make[5]: Entering directory '/root/nuttx/apps/examples/elf/tests/errno'
LD: errno.o
arm-none-eabi-ld: warning: cannot find entry symbol _start; defaulting to 00008000
errno.o: In function main': errno.c:(.text+0x2c): undefined reference to fwrite'
errno.c:(.text+0x48): undefined reference to fwrite' errno.c:(.text+0x54): undefined reference to fopen'
errno.c:(.text+0x80): undefined reference to fprintf' errno.c:(.text+0x88): undefined reference to exit'
errno.c:(.text+0x98): undefined reference to __errno' errno.c:(.text+0xb0): undefined reference to fprintf'
errno.c:(.text+0xc8): undefined reference to `_impure_ptr'
make[5]: *** [Makefile:70: errno] Error 1

@kernelgurumeditation which configuration are you trying this with? There are some examples such as spresense:elf that I can confirm build.  You can find that config here boards/arm/cxd56xx/spresense/configs/elf/defconfig

hi @btashton ,
thanks for the reply,
I'm trying with nucleo-h743zi/nsh and enabling the Elf support and the Elf loader sample App... should it not work?
I had a look at the spresense/configs/elf/defconfig file, and there is no option difference that would explain the failure...
Is this an issue with the link scripts?
Thanks

That is helpful, I have that board, I will try to do a functional test this evening.

There are changes needed to the nucleo-h743zi/scripts/Make.defs file as
well.
See diff stm32h7/nucleo-h743zi/scripts/Make.defs
cxd56xx/spresense/scripts/Make.defs
Here are some necessary additions:
72a79,103

Loadable module definitions
CMODULEFLAGS = $(CFLAGS) -mlong-calls # --target1-abs
LDMODULEFLAGS = -r -e module_initialize
ifeq ($(CONFIG_CYGWIN_WINTOOL),y)
   LDMODULEFLAGS += -T "${shell cygpath -w
$(TOPDIR)/libs/libc/modlib/gnu-elf.ld}"
else
   LDMODULEFLAGS += -T $(TOPDIR)/libs/libc/modlib/gnu-elf.ld
endif
ELF module definitions
CELFFLAGS = $(CFLAGS)
CXXELFFLAGS = $(CXXFLAGS)
LDELFFLAGS = -r -e main
ifeq ($(CONFIG_CYGWIN_WINTOOL),y)
   LDELFFLAGS += -T "${shell cygpath -w
$(BOARD_DIR)$(DELIM)scripts$(DELIM)gnu-elf.ld}"
else
   LDELFFLAGS += -T $(BOARD_DIR)$(DELIM)scripts$(DELIM)gnu-elf.ld
endif

The latter ELF definitions are certainly necessary.
On 11/19/2020 12:00 PM, Nuno Felicio wrote:

hi @btashton https://github.com/btashton ,
thanks for the reply,
I'm trying with nucleo-h743zi/nsh and enabling the Elf support and the
Elf loader sample App... should it not work?
I had a look at the spresense/configs/elf/defconfig file, and there is
no option difference that would explain the failure...
Is this an issue with the link scripts?
Thanks
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-nuttx-apps/issues/481#issuecomment-730541897,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABFUG6TU4YE5FVH22Z3FPKLSQVMKXANCNFSM4TXY3IMA.

But, it's the wrong direction to duplicate the common setting to individual Make.defs. I have saw many people complain some advanced features don't work(e.g. elf, c++...), just because Make.defs doesn't have the necessary options. We need fix the problem for all board, otherwise people will complain again and again.

Well,  talking as the new Guy..... my main issue is not knowing what is
expected to work.
This is an RTOS, not windows, typically the ppl who search/work on this is
not expecting all to work... but we would like to know, what is expected to
work...
What is very interesting for me regarding NuttX is being a Posixish system
and having a   shell makes it very comfortable for a Unix guy like me, and
it's really amazing what this little OS can do :)
PS: I tried to replicate the Make.defs settings and it didn't fix it...
On Fri, Nov 20, 2020 at 2:43 AM Xiang Xiao notifications@github.com wrote:

But, it's the wrong direction to duplicate the common setting to
individual Make.defs. I have saw many people complain some advanced
features don't work(e.g. elf, c++...), just because Make.defs doesn't have
the necessary options. We need fix the problem for all board, otherwise
people will complain again and again.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-nuttx-apps/issues/481#issuecomment-730809390,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEXNEXI2BMNFF4K72T2AIGLSQXJVPANCNFSM4TXY3IMA
.

@xiaoxiang781216 I agree that we should move more of this code into a single place, but for now I added a fully functional configuration for his board.  This is also helpful for me because I do not have the other boards that use this configuration so I can better test this in the future.
I also want to work on getting these features added into the new documentation as well.

@btashton, thanks for fixing it.
PS: I fixed the USB HOST OTG port for this board, if you want the fix I can send it.

Sure I'm happy to review it. I really want to pull some of the common board logic out so we don't have to keep copy pasting it for every variant, but that will take some time.

Ok,
it's at    https://github.com/apache/incubator-nuttx/pull/2353
Not pretty but works, and can be useful to someone :)

authors: btashton kernelgurumeditation patacongo xiaoxiang781216

repo: apache/incubator-nuttx-apps


text:
Stm32h7 dma fixes

Summary
This set of patches fix some issues we have encountered with dma driver:

Compiling the DMA code for stm32h7 with CONFIG_DEBUG_ASSERTIONS=y fail
It was not possible to DMA to/from AXI sram if CONFIG_STM32H7_DMACAPABLE=y
When doing TX DMA (e.g. in SPI driver in simplex-tx mode), it should not be necessary to
align the data buffers to the cache line. Doing only clean is perfectly fine to a full cache line, even if it contains some other data that is just managed by CPU. Also, since the cache management
(clean/invalidate) is being done from outside the DMA driver, probably checking whether it is done correctly should be done there as well.

Impact
This gives possibility to send arbitrary number of bytes from memory to peripheral using DMA also when using write-back cache
Testing
This has been tested with sending arbitrary long packets from AXI sram to SPI simplex tx, in write-back cache configuration on stm32h743zi platform.

I don't know if I should do something with the existing "brace alignment" style errors. The existing alignment looks good to me?

@jlaitine - this looks good to me.
@patacongo - What do you want to do about the alighment?

@patacongo - What do you want to do about the alighment?

You can ignore them.  Looks like a tool problem to me.
The coding standard and the tools both expect braces to appear in column 4 * N where N is th indention level.  But this rule is not usually enforced in data, only in code.  I don't know what is triggering the check here.

authors: davids5 jlaitine patacongo

repo: apache/incubator-nuttx


text:
xtensa/esp32: Add AES hardware accelerator driver

Summary
Add AES hardware accelerator driver.
Impact
Testing
boards/xtensa/esp32/esp32-devkitc/configs/aes/defconfig.

@donghengqaz the defconfig supplied needs to be refreshed: ./tools/refresh.sh --silent esp32-devkitc/aes

@donghengqaz the defconfig supplied needs to be refreshed: ./tools/refresh.sh --silent esp32-devkitc/aes


text:
CI: remove the additional pull docker container step

There is chance pull docker container failure in build jobs,
remove the additional pull docker container step may make it
more stable. Meanwhile, it also save some time for build job.
Signed-off-by: liuhaitao liuhaitao@xiaomi.com

are you talking about this symptom?
Run ./testing/.github/actions/ci-container
/usr/bin/docker run --name dockerpkggithubcomapacheincubatornuttxtestingnuttxcilinux_ffbfc5 --label d00b33 --workdir /github/workspace --rm -e DOCKER_BUILDKIT -e DOCKER_CONFIG -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/incubator-nuttx-testing/incubator-nuttx-testing":"/github/workspace" docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux  "-c" ""
Unable to find image 'docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux:latest' locally
/usr/bin/docker: Error response from daemon: Get https://docker.pkg.github.com/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers).
See '/usr/bin/docker run --help'.

are you talking about this symptom?
Run ./testing/.github/actions/ci-container
/usr/bin/docker run --name dockerpkggithubcomapacheincubatornuttxtestingnuttxcilinux_ffbfc5 --label d00b33 --workdir /github/workspace --rm -e DOCKER_BUILDKIT -e DOCKER_CONFIG -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/incubator-nuttx-testing/incubator-nuttx-testing":"/github/workspace" docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux "-c" ""
Unable to find image 'docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux:latest' locally
/usr/bin/docker: Error response from daemon: Get https://docker.pkg.github.com/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers).
See '/usr/bin/docker run --help'.

Yes, saving one pull redue the failure posibility. We can keep apache-nuttx-testing to measure the performance.

i don't think we need to keep it in testing. it's better to keep all repo in sync.

If @btashton don't have concern, I am fine to remove it from testing too.

as the log has timestamps on each lines, you can see how long it took to pull images.

have any of you noticed any negative effect with this?
otherwise i want to apply this to other repos as well.
i haven't noticed any problems.

Let's make it consistent. It is a NOP in the end.