text:
[HUDI-1351] Improvements to the hudi test suite for scalability and repeated testing.

What is the purpose of the pull request
Please see HUDI-1351 for description of the issues that are being fixed here.
Brief change log

Added the --clean-input and --clean-output parameters to clean the input and output directories before starting the job
Added the --delete-old-input parameter to deleted older batches for data already ingested. This helps keep number of redundant files low.
Added the --input-parallelism parameter to restrict the parallelism when generating input data. This helps keeping the number of generated input files low.
Added an option start_offset to Dag Nodes. Without ability to specify start offsets, data is generated into existing partitions. With start offset, DAG can control on which partition, the data is to be written.
Fixed generation of records for correct number of partitions

In the existing implementation, the partition is chosen as a random long. This does not guarantee exact number of requested partitions to be created.

Changed variable blacklistedFields to be a Set as that is faster than List for membership checks.
Fixed integer division for Math.ceil. If two integers are divided, the result is not double unless one of the integer is casted to double.

Verify this pull request
This pull request is already covered by existing tests, such as (please describe tests).
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@n3nash  I have made the requested change.


text:
[HUDI-764] [HUDI-765] ORC reader and writer implementation

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2991 (c3ae51d) into master (aba1ead) will increase coverage by 6.93%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2991      +/-   ##
============================================
+ Coverage     54.86%   61.80%   +6.93%
+ Complexity     3837      339    -3498
============================================
  Files           485       54     -431
  Lines         23435     2016   -21419
  Branches       2494      241    -2253
============================================
- Hits          12858     1246   -11612
+ Misses         9423      646    -8777
+ Partials       1154      124    -1030

Flag
Coverage Δ

hudicli
?

hudiclient
?

hudicommon
?

hudiflink
?

hudihadoopmr
?

hudisparkdatasource
?

hudisync
?

huditimelineservice
?

hudiutilities
61.80% <ø> (-7.79%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ies/exception/HoodieSnapshotExporterException.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../apache/hudi/utilities/HoodieSnapshotExporter.java
5.17% <0.00%> (-83.63%)
:arrow_down:

...hudi/utilities/schema/JdbcbasedSchemaProvider.java
0.00% <0.00%> (-72.23%)
:arrow_down:

...he/hudi/utilities/transform/AWSDmsTransformer.java
0.00% <0.00%> (-66.67%)
:arrow_down:

...in/java/org/apache/hudi/utilities/UtilHelpers.java
40.69% <0.00%> (-23.84%)
:arrow_down:

.../org/apache/hudi/sink/compact/CompactFunction.java

.../apache/hudi/common/bootstrap/FileStatusUtils.java

...apache/hudi/common/model/WriteConcurrencyMode.java

...java/org/apache/hudi/common/engine/EngineType.java

...ava/org/apache/hudi/source/StreamReadOperator.java

... and 423 more

source: gharchive/pull-request

authors: codecov-commenter jintaoguan

repo: apache/hudi

license_type: permissive

license_source: github-api


text:
[HUDI-2677] Add DFS based message queue for flink writer

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Should we give user a configuration for mode selection?

Should we give user a configuration for mode selection?

Personally i don't like add options for internal improvements.

+1 LGTM


text:
[HUDI-6819] Fix logic for throwing exception in getRecordIndexUpdates.

Change Logs
Fix logic for throwing exception in getRecordIndexUpdates.
Impact
None
Risk level (write none, low medium or high below)
Low
Documentation Update
None
Contributor's checklist

[X] Read through contributor's guide
[X] Change Logs and Impact were stated clearly
[X] Adequate tests were added if applicable
[X] CI passed

@hudi-bot run azure


text:
[HUDI-6848] Fix non-unique uid for hudi operators

Change Logs
If multiple source operators in flink task belong to the same table, it will cause duplicate uids.

Impact
bug fix
Risk level
low
Documentation Update
N/A
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

Rebased on the latest master to fix CI failure.


text:
[AIRFLOW-917] correct formating of failing status message

Dear Airflow Maintainers,
Please accept this PR that addresses the following issues: https://issues.apache.org/jira/browse/AIRFLOW-917
Testing Done:

This just changes the formatting of the error message, the existing tests don't check the string of the message and I think that's probably correct.

LGTM +1

@vijaykramesh You might want to rebase against master + add the JIRA name in the commit message

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@695ed93). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #2109   +/-   ##
=========================================
  Coverage          ?   67.13%
=========================================
  Files             ?      142
  Lines             ?    10734
  Branches          ?        0
=========================================
  Hits              ?     7206
  Misses            ?     3528
  Partials          ?        0

Impacted Files
Coverage Δ

airflow/ti_deps/deps/dag_ti_slots_available_dep.py
100% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 695ed93...a87c418. Read the comment docs.

authors: artwr codecov-io vijaykramesh

repo: apache/incubator-airflow

license_source: bigquery


text:
问个小白问题h2:grpc 在哪里设置？

我这是grpc-go客户端
我看说只要客户端设置h2:grpc 就可以了，服务端不用动。
纳闷在哪里设置呢？有代码示例吗？
谢谢

grpc-go客户端默认就是HTTP2，”h2:grpc”是给brpc client设置用的

source: gharchive/issue

repo: apache/incubator-brpc


text:
Enable the sbt-rat plugin to perform Apache Rat checks

One can now run 'sbt ratReport' to generate an Apache Rat report in
target/rat.txt to check for any unapproved licensed files. Running 'sbt
ratCheck' will do the same, but will return an error if any unapproved
licenses are found. Specifically, this patch does the following:

Enable the sbt-rat plugin
Removes the .rat-excludes file and instead uses the sbt ratExcludes
setting, which allows exluding full paths rather than just file names
Modify Travis CI config to perform a ratCheck on every commit
Add matchers for the passera and Scala BSD licenses, and mark them as
approved licenses
Rename LICENSE-bin and NOTICE-bin to bin.LICENSE and bin.NOTICE.
Apache Rat does not see them as NOTICE files otherwise
Fix URL in W3C files so rat can detect the license

DAFFODIL-1937

Release Audit Tool. It's usually run by incubator mentors/voters during releases to make sure there aren't incorrectly licensed files. But with this we can now check it continually for every change.

repo: apache/incubator-daffodil


text:
taskInstance state change error

phenomenon：
taskInstance state change from 7 to 1，and user observes the state change from "SUCCESS" to "RUNNING"。
master still loop for check taskInstance state，wait for taskInstance's state is finish.
reason：
master receive the message of TaskResponseProcessor first，and than receive the TaskAckProcessor。

already fixed in 1.3.3-release

我看了Github上的代码，这部分的代码和1.3.2的事一样的，所以麻烦您修复的方案是什么样的啊，代码在什么地方啊，想学习一下

already fixed in 1.3.3-release
我看了Github上的代码，这部分的代码和1.3.2的事一样的，所以麻烦您修复的方案是什么样的啊，代码在什么地方啊，想学习一下

sorry, may be the same problem, please descript

sorry, may be it's not the same problem, please describe the detail of your problem.
eg: the task's content / system info of master and worker ...
master向worker发送task：
1.worker接收到task后，会回复给master一个“ack”，然后去执行task，执行结束之后给master发送result。
2.master接受到ack后将taskInstance的状态更新成1，接收到result后将状态更新成7（假设task的状态是成功）
正常：
ack和result根据worker执行的顺序，ack会先发送到master，task状态从0->1,result 后发送到master ，task状态从1->7。
异常：
在ack的response处理的时候出现阻塞，或者是在netty消息传输或消费的，反正resul更新数据库的时间早于ack，导致状态变化0->7->1.而实际worker已经执行过task了，所以master就会阻塞住，一直循环查看当前节点的状态。

org.apache.dolphinscheduler.server.master.processor.TaskAckProcessor#process
+sleep can reproduce. The task is as simple as possible.

authors: CalvinKirs lenboo wangxj3

repo: apache/incubator-dolphinscheduler


text:
Add StorageRowIterator to uniform read in storage

Currently in storage engine, there is too many interface to read data. Including followings

SegmentReader which is used to read data from Segment
ColumnData which is used to read data from SegmentGroup
RowsetReader which is used to read data from multiple SegmentGroup(now in be_refactor branch)
Reader which is used to read data from table. This class finish Aggregation/Unique operation

And all of these interfaces are different, however their logical is mostly same with each other. This make our system more complex and difficult to understand.
What I want to uniform all these interface to StorageRowIterator. Through this interface we can access any data in storage in row format. And it can reuse some logical, for example Merge/Union. In addition, this can make our system easy to understand.
The interface may look like
struct StorageReadOptions {
    std::shared_ptr<RowCursor> lower_bound;
    bool include_lower_bound = true;
    std::shared_ptr<RowCursor> upper_bound;
    bool include_upper_bound = true;
};

class StorageRowIterator {
public:
    StorageRowIterator() { }
    virtual ~StorageRowIterator() { }

    virtual Status init(const StorageReadOptions& opts);
    virtual Status next_batch(RowBlock* block) = 0;
};

We can assign lower and upper bounds for a scan to eliminate scanning unused data.
this issue relate to #1305

I think StorageRowwiseIterator is a better name because it returns RowBlock, not a row

authors: imay kangpinghuang

repo: apache/incubator-doris


text:
[Bug] When set ‘lower_case_table_names=1’  , the table alias cannot be capitalized

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
Version : 0.15.1-rc09
What's Wrong?
Database changed
mysql> select * from  TB1 a  where A.apply_id='001'  limit 10 \G;
Empty set (0.01 sec)
ERROR:
No query specified
mysql> select * from  TB1 A  where A.apply_id='001'  limit 10 \G;
ERROR 1054 (42S22): errCode = 2, detailMessage = Unknown column 'apply_id' in 'a'
ERROR:
No query specified
mysql> show  variables like  '%low%' ;
+---------------------------------+---------+
| Variable_name                   | Value   |
+---------------------------------+---------+
| allow_partition_column_nullable | true    |
| lower_case_table_names          | 1       |
| max_allowed_packet              | 1048576 |
+---------------------------------+---------+
3 rows in set (0.01 sec)
mysql>
What You Expected?
When using SQL queries, I want table aliases to be capitalized
How to Reproduce?
No response
Anything Else?
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

it must be configured at cluster initialization.

it happened in 0.15.1-rc09 and has been fixed in 1.0.0-preview and following version.

under version 0.15.9-rc09, it looks like:
mysql> select * from part A where A.p_name = "rose moccasin";
ERROR 1054 (42S22): errCode = 2, detailMessage = Unknown column 'p_partkey' in 'a'
mysql> select * from part A where a.p_name = "rose moccasin";
ERROR 1054 (42S22): errCode = 2, detailMessage = Unknown column 'p_partkey' in 'a'
mysql> select * from part a where a.p_name = "rose moccasin";
+-----------+---------------+--------+------------+-----------+------------+--------------------------+--------+-------------+
| p_partkey | p_name        | p_mfgr | p_category | p_brand   | p_color    | p_type                   | p_size | p_container |
+-----------+---------------+--------+------------+-----------+------------+--------------------------+--------+-------------+
|     94251 | rose moccasin | MFGR#1 | MFGR#15    | MFGR#1510 | forest     | LARGE POLISHED COPPER    |     22 | MED CAN     |
|    132258 | rose moccasin | MFGR#4 | MFGR#41    | MFGR#4123 | bisque     | SMALL BURNISHED NICKEL   |     46 | WRAP BAG    |

mysql> select * from part a where A.p_name = "rose moccasin";
+-----------+---------------+--------+------------+-----------+------------+--------------------------+--------+-------------+
| p_partkey | p_name        | p_mfgr | p_category | p_brand   | p_color    | p_type                   | p_size | p_container |
+-----------+---------------+--------+------------+-----------+------------+--------------------------+--------+-------------+
|      3187 | rose moccasin | MFGR#5 | MFGR#55    | MFGR#5529 | honeydew   | SMALL POLISHED NICKEL    |      1 | SM DRUM     |
|     10974 | rose moccasin | MFGR#1 | MFGR#12    | MFGR#1211 | steel      | LARGE POLISHED BRASS     |     29 | WRAP DRUM   |
|     55272 | rose moccasin | MFGR#3 | MFGR#33    | MFGR#3325 | seashell   | MEDIUM PLATED COPPER     |     32 | WRAP PKG    |

And in 1.0.0-preview
mysql> select * from part C where c.p_name = "turquoise sandy";
+-----------+-----------------+--------+------------+-----------+------------+---------------------------+--------+-------------+
| p_partkey | p_name          | p_mfgr | p_category | p_brand   | p_color    | p_type                    | p_size | p_container |
+-----------+-----------------+--------+------------+-----------+------------+---------------------------+--------+-------------+
|    233418 | turquoise sandy | MFGR#1 | MFGR#14    | MFGR#1418 | lace       | LARGE ANODIZED COPPER     |     29 | LG PACK     |

authors: ChPi dataalive longdpt


text:
[Dubbo-#1293][Baiji-9]fix the issue #1293

What is the purpose of the change
fix the issue #1293
Brief changelog
add  the method of getting whether the cache exists
add  the method of putting key and ReferenceConfig
add  a  new method of getting ReferenceConfig through url,name,version
Verifying this change
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[ ] Run mvn clean install -DskipTests & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

Since we have not found a reasonable, easy-to-use way for ReferenceConfigCache, we'll close this PR firstly.

repo: apache/incubator-dubbo


text:
Consul support

What is the purpose of the change
support consul on 2.7 #983
Brief changelog
XXXXX
Verifying this change
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[ ] Run mvn clean install -DskipTests=false & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #3593 into master will decrease coverage by 1.14%.
The diff coverage is 0%.

@@             Coverage Diff              @@
##             master    #3593      +/-   ##
============================================
- Coverage     64.15%   63.01%   -1.15%
  Complexity       71       71
============================================
  Files           664      689      +25
  Lines         28629    30688    +2059
  Branches       4763     5025     +262
============================================
+ Hits          18366    19337     +971
- Misses         8041     9018     +977
- Partials       2222     2333     +111

Impacted Files
Coverage Δ
Complexity Δ

...nter/consul/ConsulDynamicConfigurationFactory.java
0% <0%> (ø)
0 <0> (?)

...onfigcenter/consul/ConsulDynamicConfiguration.java
0% <0%> (ø)
0 <0> (?)

...g/apache/dubbo/registry/consul/ConsulRegistry.java
0% <0%> (ø)
0 <0> (?)

...e/dubbo/registry/consul/ConsulRegistryFactory.java
0% <0%> (ø)
0 <0> (?)

...e/dubbo/remoting/transport/netty/NettyChannel.java
54.11% <0%> (-8.24%)
0% <0%> (ø)

...onfig/spring/extension/SpringExtensionFactory.java
78.04% <0%> (-7.32%)
0% <0%> (ø)

.../apache/dubbo/remoting/transport/AbstractPeer.java
58.69% <0%> (-4.35%)
0% <0%> (ø)

...he/dubbo/registry/multicast/MulticastRegistry.java
67.87% <0%> (-1.81%)
0% <0%> (ø)

...pache/dubbo/registry/support/AbstractRegistry.java
79.52% <0%> (-1.19%)
0% <0%> (ø)

...bbo/rpc/protocol/thrift/io/InputStreamWrapper.java
0% <0%> (ø)
0% <0%> (?)

... and 33 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8d645b...2f82896. Read the comment docs.

Will consul be used as a metadata center?

Will consul be used as a metadata center?

not in the current release's plan, maybe consider it later. This suggestion is worthy of an issue to track it.

[08/03/19 13:34:30:554 CST] NettyServerWorker-5-5  WARN transport.AbstractServer:  [DUBBO] All clients has disconnected from 0.0.0.0/0.0.0.0:20880. You can graceful shutdown now., dubbo version: , current host: 30.5.120.45
[08/03/19 13:34:30:554 CST] DubboServerHandler-30.5.120.45:20880-thread-10  INFO dubbo.DubboProtocol:  [DUBBO] disconnected from /30.5.120.45:64272,url:dubbo://30.5.120.45:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&bean.name=org.apache.dubbo.demo.DemoService&bind.ip=30.5.120.45&bind.port=20880&channel.readonly.sent=true&codec=dubbo&default.deprecated=false&default.dynamic=false&default.export=true&default.register=true&deprecated=false&dubbo=2.0.2&dynamic=false&export=true&generic=false&heartbeat=60000&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=19523&qos.port=22222&register=true&release=&side=provider&timestamp=1552023216919, dubbo version: , current host: 30.5.120.45
[08/03/19 13:34:40:555 CST] NettyServerWorker-5-6  WARN transport.AbstractServer:  [DUBBO] All clients has disconnected from 0.0.0.0/0.0.0.0:20880. You can graceful shutdown now., dubbo version: , current host: 30.5.120.45
[08/03/19 13:34:40:556 CST] DubboServerHandler-30.5.120.45:20880-thread-12  INFO dubbo.DubboProtocol:  [DUBBO] disconnected from /30.5.120.45:64275,url:dubbo://30.5.120.45:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&bean.name=org.apache.dubbo.demo.DemoService&bind.ip=30.5.120.45&bind.port=20880&channel.readonly.sent=true&codec=dubbo&default.deprecated=false&default.dynamic=false&default.export=true&default.register=true&deprecated=false&dubbo=2.0.2&dynamic=false&export=true&generic=false&heartbeat=60000&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=19523&qos.port=22222&register=true&release=&side=provider&timestamp=1552023216919, dubbo version: , current host: 30.5.120.45
[08/03/19 13:34:50:556 CST] NettyServerWorker-5-7  WARN transport.AbstractServer:  [DUBBO] All clients has disconnected from 0.0.0.0/0.0.0.0:20880. You can graceful shutdown now., dubbo version: , current host: 30.5.120.45
[08/03/19 13:34:50:557 CST] DubboServerHandler-30.5.120.45:20880-thread-14  INFO dubbo.DubboProtocol:  [DUBBO] disconnected from /30.5.120.45:64276,url:dubbo://30.5.120.45:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&bean.name=org.apache.dubbo.demo.DemoService&bind.ip=30.5.120.45&bind.port=20880&channel.readonly.sent=true&codec=dubbo&default.deprecated=false&default.dynamic=false&default.export=true&default.register=true&deprecated=false&dubbo=2.0.2&dynamic=false&export=true&generic=false&heartbeat=60000&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=19523&qos.port=22222&register=true&release=&side=provider&timestamp=1552023216919, dubbo version: , current host: 30.5.120.45

    2019/03/08 13:34:10 [DEBUG] agent: Check "service:d258d650" is passing
    2019/03/08 13:34:20 [DEBUG] agent: Check "service:d258d650" is passing
    2019/03/08 13:34:30 [DEBUG] agent: Check "service:d258d650" is passing
    2019/03/08 13:34:40 [DEBUG] agent: Check "service:d258d650" is passing
    2019/03/08 13:34:50 [DEBUG] agent: Check "service:d258d650" is passing

The server side keeps outputing logs showing clients create and then break connection. Is there any problem with consul's health check?

The server side keeps outputing logs showing clients create and then break connection. Is there any problem with consul's health check?

It is TCP check provided by consul, see more from https://www.consul.io/docs/agent/checks.html
It looks like consul uses short TCP connection to check node's liveness, which in fact is reasonable. I think we should bear with the disconnection logging messages since we allow user to adjust check interval like this:
    <dubbo:registry address="consul://localhost:8500">
        <dubbo:parameter key="consul-check-interval" value="60s"/>
        <dubbo:parameter key="consul-check-timeout" value="20s"/>
    </dubbo:registry>

What do you think?

The server side keeps outputing logs showing clients create and then break connection. Is there any problem with consul's health check?

It is TCP check provided by consul, see more from https://www.consul.io/docs/agent/checks.html
It looks like consul uses short TCP connection to check node's liveness, which in fact is reasonable. I think we should bear with the disconnection logging messages since we allow user to adjust check interval like this:
    <dubbo:registry address="consul://localhost:8500">
        <dubbo:parameter key="consul-check-interval" value="60s"/>
        <dubbo:parameter key="consul-check-timeout" value="20s"/>
    </dubbo:registry>

What do you think?

Agree. How about increase the default check interval, say 60s, current interval is to some extent too frequent.


text:
热力图（heatmap）如何自定义参数？How to use custom parameters in heatmap?

What problem does this feature solve?
在热力图中点击某个区域时，希望能传递该区域的自定义参数，比如该区域的区划编号，以进行下一步操作。但是目前好像只能获取到点击的区域的名称（name）。
When clicking on an area in the heatmap, it is hoped that the user-defined parameters of the area, such as the division number of the area, can be passed for the next operation. However, at present, it seems that only the name of the clicked area can be obtained.
What does the proposed API look like?
charts.on("click", function (params) {
        console.log(params);
        console.log(params.CUSTOMPARAMETER);    // 获取自定义参数 Gets the custom parameter named CUSTOMPARAMETER
}

You can try indexing the custom parameters by yourself with the name in params

authors: pissang realchoi

repo: apache/incubator-echarts


text:
树图上的的文字添加点击事件

1、文字添加点击事件，
2、文字鼠标滑过有对应的一个关于信息展示的pop

请参见相关 issue

authors: Ovilia ycshill


text:
10台车，每350ms每台车就会进来一个轨迹点，当轨迹点达到一定数量时，页面很卡

One-line summary [问题简述]
10台车，每350ms每台车就会进来一个轨迹点，当轨迹点达到一定数量时，页面很卡。
数据来的比较快，过不了几分钟就会卡死，目前的做法是shift 一些旧的点。但是希望能保留比较多的轨迹数据。 加入数据 用的setOption 。 是否有别的方法可以解决这个性能问题，我看到api 里有一个appendData的方法，不知道是否适合，能否提供一些思路。非常感谢！
Version & Environment [版本及环境]

ECharts version [ECharts 版本]:

Browser version [浏览器类型和版本]:

OS Version [操作系统类型和版本]:

Expected behaviour [期望结果]
ECharts option [ECharts配置项]

option = {

}

Other comments [其他信息]

我也在做类似的功能，也是清除旧的数据:joy:

我单独.setOption,一个指定的series,echart居然把整个series里所有的系列全都重绘一遍，这不是浪费性能吗？就不能只单独的绘制我指定的series，其他的保持不变吗？  还是有什么设置API 是我没有找到的。
setOption({
series: {
name: "实时"
data:.......
}）

@dandeXiong 那个setOption就是全量更新的，绘制指定的series可以用appendData吧，有seriesIndex参数来确定更新哪个，但这个方法是追加数据

@Shadowzeng  采用appendData 确实能大大提高效率，但是很有局限性啊，像我的散点图，要配合visualMap来做颜色变更，就无法使用appendData。他无法对单独的数据项进行设置。large, largeThreshold ,的开启后也是无法配合使用visualmap的

可以设置单独的 http://echarts.baidu.com/option.html#series-line.zlevel zlevel 放在多个 canvas 中

老铁，这个问题你解决了吗

老铁，这个问题你解决了吗

时间隔得有点久了，我当时的做法，1.轨迹点的设置：showSymbol:false （刚开始是需求需要，后来阉割），这个设置就提高了不少的性能。 2. 每隔两个小时彻底清理一次数据点（这个时间点是根据自己项目监控服务内存峰值来设定的），清空完一般不到半分钟内存就降下来了。   期间也尝试过分层，比如过去多长时间累积的点，就分出一个图表绘制好静置不动放哪里，与用来动态绘点的图表进行联动

在绘制行为轨迹点的时候 数据量 图层过多，可以采用 降采样策略，开启后可以有效的优化图表的绘制效率，默认关闭 ，或者清理历史数据点，定时清空这样内存压力会小很多 希望有用 查看帮助   💪💪
例如：
option = {
    dataset: {
        source: [
            // 有了上面 dimensions 定义后，下面这五个维度的名称分别为：
            // 'date', 'open', 'close', 'highest', 'lowest'
            [12, 44, 55, 66, 2],
            [23, 6, 16, 23, 1],
            ...
        ]
    },
    series: {
        type: 'xxx',
        // 定义了每个维度的名称。这个名称会被显示到默认的 tooltip 中。
        dimensions: ['date', 'open', 'close', 'highest', 'lowest']
    }
}
series: {
    type: 'xxx',
    dimensions: [
        null,                // 如果此维度不想给出定义，则使用 null 即可
        {type: 'ordinal'},   // 只定义此维度的类型。
                             // 'ordinal' 表示离散型，一般文本使用这种类型。
                             // 如果类型没有被定义，会自动猜测类型。
        {name: 'good', type: 'number'},
        'bad'                // 等同于 {name: 'bad'}
    ]
}

authors: LIHAO520 Ovilia Shadowzeng dandeXiong qierkang


text:
[Docs] Add http request doc instruction

Add http request doc instruction for using postman.

@xwm1992 I suggest you create an issue with it, and use curl instead of the postman in the doc.

@xwm1992 I suggest you create an issue with it, and use curl instead of the postman in the doc.

done

repo: apache/incubator-eventmesh


text:
[CH] Fix GlutenLiteralExpressionSuite and GlutenMathExpressionsSuite

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

cc @baibaichen

repo: apache/incubator-gluten


text:
[Bug] DataImport CSV File and mapping data Field Select option only has one,in fact there are four attributes.

Bug Type (问题类型)
others (please edit later)
Before submit

[X] 我已经确认现有的 Issues 与 FAQ 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)

Environment (环境信息)

Server Version: 1.0.0 (Apache Release Version)
Backend: RocksDB x nodes, HDD or SSD
OS: xx CPUs, xx G RAM, Ubuntu 2x.x / CentOS 7.x
Data Size:  xx vertices, xx edges

Expected & Actual behavior (期望与实际表现)

CSV 文件导入后，在配置节点的属性映射时，属性下拉框选项只有一个，但是我这个节点类型有4个属性：

数据字段和属性不能精确配置映射，是不是 Bug呢？
Vertex/Edge example (问题点 / 边数据举例)
No response
Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
No response

I have found the bug place ,it is in VertexMap.tsx :

And remove this  cod 「getPopupContainer={(e: any) => e.parentNode} 」 then the page is OK:

@wanganjuan could u help to address this problem~
And contribution to improving the problem are also welcome @woodvine

fixed https://github.com/apache/incubator-hugegraph-toolchain/pull/535

authors: imbajin wanganjuan woodvine

repo: apache/incubator-hugegraph-toolchain


text:
[Bug] a number of CVEs exist for NPMs exist in dashboard

What happened
I did a dependabot analysis of InLong and have submitted maven fixes but I'm not a UI specialist so haven't attempted to update the NPM modules.
Some of the highest severity issues include (there are many more):

https://github.com/advisories/GHSA-33f9-j839-rf8h
https://github.com/advisories/GHSA-hxcc-f52p-wc94
https://github.com/advisories/GHSA-ww39-953v-wcq6

npm audit and npm audit fix can be used
What you expected to happen
n/a
How to reproduce
n/a
Environment
No response
InLong version
master
InLong Component
InLong Dashboard
Are you willing to submit PR?

[ ] Yes, I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@leezng PTAL, thanks.

@pjfanning, thanks, you are professional!

@pjfanning
I try to use npm audit fix to resolve some vulnerabilities. But there are still some that involve breaking changes. Because they are lower-level dependencies of development dependencies, theoretically they will not affect the UI layer.

authors: dockerzhang gosonzhang leezng pjfanning

repo: apache/incubator-inlong


text:
[Fix_#1632] Update postgres to 15.9 version

Many thanks for submitting your Pull Request :heart:!

Closes/Fixes/Resolves kie-issues#1632

Update postgres to 15.9 version

PR job #1 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer
build-chain build full_downstream  -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u https://github.com/apache/incubator-kie-kogito-runtimes/pull/3791 --skipParallelCheckout
NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3791/1/display/redirect
Test results:

PASSED: 3454
FAILED: 27

Those are the test failures:

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testMigrateSingle
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testMigrateAll
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testBasicFlow
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testMultipleProcesses
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testProcessWithDifferentVersion
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testRemove
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesIT.testUpdate
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testMigrateSingle
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testMigrateAll
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testBasicFlow
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testMultipleProcesses
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testProcessWithDifferentVersion
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testRemove
Error finding all process instances, for processId BPMN2_UserTask

org.kie.persistence.postgresql.PostgresqlProcessInstancesWithLockIT.testUpdate
Error finding all process instances, for processId BPMN2_UserTask

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testPersistence
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testMultipleEmbeddedInstance
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testAdHocProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testEmbeddedProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testAsyncWIH
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testMultipleInstance
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testParallelPersistence
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testPersistence
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testMultipleEmbeddedInstance
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testAdHocProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testEmbeddedProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testAsyncWIH
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testMultipleInstance
1 expectation failed.Expected status code <201> but was <500>.

hi @gmunozfe, I'm ok with the version upgrade, but which is the goal of this upgrade. The issue isn't clear to me.
Not trying to bomb th PR, it's just about understanding the change.

hi @gmunozfe, I'm ok with the version upgrade, but which is the goal of this upgrade. The issue isn't clear to me.
Not trying to bomb th PR, it's just about understanding the change.

Trying to upgrade postgresql to one of the latest stable releases as a good practice, in order to take advantages of enhancements, security improvements, etc. This would also help developers to stay current and make use of the latest developments in the ecosystem.

@gmunozfe agree. I just wanted to know if there was another reason that was requiring the change.

PR job #2 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer
build-chain build full_downstream  -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u https://github.com/apache/incubator-kie-kogito-runtimes/pull/3791 --skipParallelCheckout
NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3791/2/display/redirect
Test results:

PASSED: 3468
FAILED: 13

Those are the test failures:

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testPersistence
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testMultipleEmbeddedInstance
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testAdHocProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testEmbeddedProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testAsyncWIH
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testMultipleInstance
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLOptimisticLockingIT.testParallelPersistence
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testPersistence
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testMultipleEmbeddedInstance
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testAdHocProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testEmbeddedProcess
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testAsyncWIH
1 expectation failed.Expected status code <201> but was <500>.

PR check / Build projects / org.kie.kogito.it.PostgreSQLPersistenceIT.testMultipleInstance
1 expectation failed.Expected status code <201> but was <500>.

authors: gmunozfe kie-ci3 pefernan

repo: apache/incubator-kie-kogito-runtimes


text:
COMPILED State needs considering eagerly executed commands

1. Describe the feature
2. Motivation
3. Describe the solution
4. Additional context

Any explain?

authors: shenbinglife yaooqinn

repo: apache/incubator-kyuubi


text:
[Test] Add Kyuubi on k8s With Spark on k8s client deploy-mode unit test

Why are the changes needed?

Add Kyuubi on k8s With Spark on k8s client deploy-mode unit test
How was this patch tested?

[x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Codecov Report

Merging #2381 (623b356) into master (65a272f) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2381      +/-   ##
============================================
- Coverage     62.64%   62.64%   -0.01%
  Complexity       69       69
============================================
  Files           358      358
  Lines         16968    16968
  Branches       2298     2298
============================================
- Hits          10630    10629       -1
- Misses         5343     5345       +2
+ Partials        995      994       -1

Impacted Files
Coverage Δ

...rg/apache/kyuubi/engine/trino/TrinoStatement.scala
68.96% <0.00%> (-2.30%)
:arrow_down:

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
96.34% <0.00%> (+0.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65a272f...623b356. Read the comment docs.

CI test pass. If you have time, please help review this. @ulysses-you

thanks, merging to master


text:
METRON-60 Created service management functionality for YAF

Added 'service management' functionality for the YAF sensor.

Created a SysV script for the YAF probe to provide start/stop/restart functionality.

The airdeamon tool is used as a process watchdog to automatically restart the probe it if it crashes.

Additional options can be passed to YAF during 'start' to aid debugging.

service yaf stop
service yaf start "--idle-timeout 0"

Looks like deployment/roles/snort/files/snortrules-snapshot-2962.tar.gz is giving an apache-rat license violation.  Can you fix and update?

That is already a problem with 'master' currently, which is addressed in METRON-57.  If you can merge that one, it will address the problem.

+1 looks great!

authors: cestella dlyle65535 nickwallen

repo: apache/incubator-metron


text:
METRON-196: Deployment Fails Without Ansible 2.0.0.2

Contributor Comments
This changeset allows deployment with Ansible 2.0.0.2 and 2.2.2.0.
I also added a version check as a pre-task to the top of the two entry playbooks, so deployment will now fail with a clearer error message if it is run on untested Ansible versions.
@JonZeolla - this is the branch that (I hope) takes care of your issue. Please update Ansible to 2.2.2.0 and give it a whirl when you can. I setup a virtualenv to test it out, I'd recommend you do the same.
Testing
For those wishing to remain on 2.0.0.2, nothing should change, simply run the deployment of your choice.
For those wishing to test on 2.2.2.0, do the following:

Setup virtual environment for testing.

pip install virtualenv
cd ~
virtualenv -p /usr/bin/python2.7 test_ansible
source ~/test_ansible/bin/activate
pip install ansible
pip install boto

After this, your prompt will begin with (test_ansible) indicating you're in the virtual Python environment.

Run deployment of your choosing.

Type deactivate to return to your normal Python environment.

Pull Request Checklist
Thank you for submitting a contribution to Apache Metron (Incubating).
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[X] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[X] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[X] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[N/A] Have you included steps to reproduce the behavior or problem that is being changed or addressed.

[X] Have you included steps or a guide to how the change may be verified and tested manually?

[X] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via:
mvn -q clean integration-test install && build_utils/verify_licenses.sh

[N/A] Have you written or updated unit tests and or integration tests to verify your changes?

[N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[X] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? (Full Dev and EC2)

For documentation related changes:

[X] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
bin/generate-md.sh
mvn site:site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

This is great work @dlyle65535 I'll reserve my +1 for immediately after Jon tries it out.

Taking a look at this today.

I'm getting {"dns": {"ts":1490980024.000000.58808,"uid":"CmLZ8q1IzktXz75Bq" from sensor-stubs, where it should be {"dns": {"ts":1484169493.58808,"uid":"CmLZ8q1IzktXz75Bq" according to this.  Not sure why that extra .000000 is being inserted.

@JonZeolla - That looks like data was sent before the templates were loaded. Is that possible? I had clean data on all my test runs.

It looks like what I found earlier was an unrelated issue in start-bro-stub, which I  was able to fix under METRON-815.  I'm finishing that some testing on that and then taking another look at this.

Ok I've spun this up a few different ways now and I'm consistently seeing no indexes being created in ES, but I am successfully getting what appears to be valid snort and bro logs to the indexing topic.
[vagrant@node1 /]$ curl -XGET 'localhost:9200/_cat/indices?v&pretty'
health status index   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana   1   0         52            0     57.6kb         57.6kb

Storm UI is showing no activity in the indexing bolts/spout.

The worker.log for indexing shows
2017-04-03 02:15:29.420 STDERR [INFO] Error: Could not find or load main class ${topology.worker.childopts}

I'm not 100% sure why this is happening but my initial thought is to take a closer look at aef84636a3427da20b1d54acfe1b8de23e2aaf97.

Looks like this is fixed in #506

Now that master is cleaned up I gave this a shot under 2.2.2.0 and 2.0.0.2.  Looks good to me - spun up full dev on both, checked kibana, etc.
+1

authors: JonZeolla cestella dlyle65535


text:
Incorrect docs link for NDArray

All links for NDArray in the Python documentation link to mxnet.optimizer.NDArray instead of mxnet.ndarray.NDArray. Could add confusion for users, since this is the wrong reference to NDArray. And it's only technically possible because NDArray is imported in optimizer.py.

See https://mxnet.incubator.apache.org/api/python/ndarray/ndarray.html#mxnet.ndarray.save for an example.
Under 'Parameters', 'data', click on 'NDArray'
Goes (incorrectly) to https://mxnet.incubator.apache.org/api/python/optimization/optimization.html#mxnet.optimizer.NDArray

@mxnet-label-bot [Website, Doc]

@aaronmarkham can you take over from here? :)

This seems related to this other issue (why can't Sphinx access the shorthand function calls):
https://github.com/apache/incubator-mxnet/issues/12318
Looking at the code section, it's clear that the erroneous links come from Sphinx's autodoc plugin. It is going through, importing things and checking the functions.
https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/ndarray/utils.py#L235-L238
If it's linking to optimizer then maybe that's because the context that it is in at that point.
I get the feeling that init / relative import stuff is messed up.  Help?

@lanking520 noted that everything is fine in v1.3.0.
I compared the docs build process and note that the optimizer folder is new in master where as in 1.3.0 it was just a optimizer.py file.
We're seeing a bunch of errors in Sphinx:
/home/ubuntu/incubator-mxnet/python/mxnet/autograd.py:docstring of mxnet.autograd.mark_variables:None: WARNING: more than one target found for cross-reference u'NDArray': mxnet.optimizer.NDArray, mxnet.ndarray.NDArray
/home/ubuntu/incubator-mxnet/python/mxnet/autograd.py:docstring of mxnet.autograd.mark_variables:None: WARNING: more than one target found for cross-reference u'NDArray': mxnet.optimizer.NDArray, mxnet.ndarray.NDArray
/home/ubuntu/incubator-mxnet/python/mxnet/autograd.py:docstring of mxnet.autograd.backward:None: WARNING: more than one target found for cross-reference u'NDArray': mxnet.optimizer.NDArray, mxnet.ndarray.NDArray
/home/ubuntu/incubator-mxnet/python/mxnet/autograd.py:docstring of mxnet.autograd.backward:None: WARNING: more than one target found for cross-reference u'NDArray': mxnet.optimizer.NDArray, mxnet.ndarray.NDArray

After checking with @aaronmarkham The issue is introduced in this PR. There are a bunch of more issues that are cross-referenced...

This may be due to the python modules not correctly specifying __all__. The members of __all__ are the ones exported by a Python module. If __all__ is not defined
#12365 splits mx.optimizer into mx.optimizer and mx.optimizer.contrib. It introduced the __all__ statement. See https://github.com/apache/incubator-mxnet/pull/12365/files#diff-ee303d7550561edd63bea9a90d2f7106R24 https://github.com/apache/incubator-mxnet/pull/12365/files#diff-4b133b8d5dae8c5b7f6c8a872a244b3aR27 and https://github.com/apache/incubator-mxnet/pull/12365/files#diff-0c893416e9e93fbd94dfaa9fa6c13d67R34
Looking at the last of the 3 links, NDArray is wrongly part of __all__. I'll open a PR to remove it. This may also fix this issue.

Fixed by #12886

Fixed by #12886.

Fixed by #12886

@leezu thanks for your diagnosis and fix on the potential issues. @aaronmarkham could you please try again to build based on Master to see if the issue is reproducible?

@lanking520 @aaronmarkham No need to rebuild based on master. CI did that already. The docs of CI's build are at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-12886/2/api/python/ndarray/ndarray.html#mxnet.ndarray.save and as far as I can tell don't contain the issue described by @thomelane

thanks everyone for the quick fix! it's looking good from my side, so case closed.

authors: aaronmarkham lanking520 leezu thomelane

repo: apache/incubator-mxnet


text:
Unable to install MXNet with GPU support for python using pip

I am trying to install MXNet for Python using pip having GPU support. I am following the instructions mentioned in this documentation. My OS is Ubuntu 16.04
After I install MXNet, I run the validation code mentioned in documentation. I get the error as stated below.
>>> import mxnet as mx
>>> a = mx.nd.ones((2,3), mx.gpu())
[17:26:19] src/c_api/c_api_ndarray.cc:147: GPU support is disabled. Compile MXNet with USE_CUDA=1 to enable GPU support.
[17:26:19] /home/travis/build/dmlc/mxnet-distro/mxnet-build/dmlc-core/include/dmlc/logging.h:308: [17:26:19] src/c_api/c_api_ndarray.cc:416: Operator _ones is not implemented for GPU.

Stack trace returned 10 entries:
[bt] (0) /home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/libmxnet.so(+0xec23c) [0x7f298299523c]
[bt] (1) /home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/libmxnet.so(+0xde7a52) [0x7f2983690a52]
[bt] (2) /home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/libmxnet.so(MXImperativeInvoke+0x254) [0x7f2983691544]
[bt] (3) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(ffi_call_unix64+0x4c) [0x7f299be918cc]
[bt] (4) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(ffi_call+0x165) [0x7f299be90a15]
[bt] (5) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(_ctypes_callproc+0x283) [0x7f299be88e43]
[bt] (6) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(+0x96af) [0x7f299be806af]
[bt] (7) /home/prasad/miniconda3/envs/myenv/bin/../lib/libpython3.5m.so.1.0(PyObject_Call+0x6a) [0x7f299de4aeca]
[bt] (8) /home/prasad/miniconda3/envs/myenv/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x36b6) [0x7f299df340d6]
[bt] (9) /home/prasad/miniconda3/envs/myenv/bin/../lib/libpython3.5m.so.1.0(+0x15dbd6) [0x7f299df3abd6]

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/ndarray.py", line 1216, in ones
    return _internal._ones(shape=shape, ctx=ctx, dtype=dtype, **kwargs)
  File "<string>", line 15, in _ones
  File "/home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/_ctypes/ndarray.py", line 89, in _imperative_invoke
    c_array(ctypes.c_char_p, [c_str(str(val)) for val in vals])))
  File "/home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/base.py", line 129, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [17:26:19] src/c_api/c_api_ndarray.cc:416: Operator _ones is not implemented for GPU.

Stack trace returned 10 entries:
[bt] (0) /home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/libmxnet.so(+0xec23c) [0x7f298299523c]
[bt] (1) /home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/libmxnet.so(+0xde7a52) [0x7f2983690a52]
[bt] (2) /home/prasad/miniconda3/envs/myenv/lib/python3.5/site-packages/mxnet/libmxnet.so(MXImperativeInvoke+0x254) [0x7f2983691544]
[bt] (3) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(ffi_call_unix64+0x4c) [0x7f299be918cc]
[bt] (4) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(ffi_call+0x165) [0x7f299be90a15]
[bt] (5) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(_ctypes_callproc+0x283) [0x7f299be88e43]
[bt] (6) /home/prasad/miniconda3/envs/myenv/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so(+0x96af) [0x7f299be806af]
[bt] (7) /home/prasad/miniconda3/envs/myenv/bin/../lib/libpython3.5m.so.1.0(PyObject_Call+0x6a) [0x7f299de4aeca]
[bt] (8) /home/prasad/miniconda3/envs/myenv/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x36b6) [0x7f299df340d6]
[bt] (9) /home/prasad/miniconda3/envs/myenv/bin/../lib/libpython3.5m.so.1.0(+0x15dbd6) [0x7f299df3abd6]

However, if I change the context to CPU, the code runs properly. Please can somebody tell me how to install MXNet having GPU support.
Thanks in advance.

Could you paste the command you used when installing from pip and the output of that command?

@szha Thanks for the reply.
I looked into it. The main problem was I had installed previously MXNet using pip install mxnet command. On top of that I had installed the CUDA version MXNet which was the reason why context of mx.gpu() was not working. I uninstalled both the versions of MXNet and then installed the CUDA version and then it has started working.
Thanks for the help again.

Thanks for sharing the findings.

I got the same issue. I use CUDA=8.0. you should install mxnet cuda version by pip with the specific mxnet version number. You may have to uninstall current mxnet before installing below mxnet version.
pip install mxnet-cu80==0.11.0
https://mxnet.incubator.apache.org/get_started/install.html

authors: Prasad9 ToruHironaka szha


text:
3d convolution gives zero output

When using mxnet 3d Convolution layer, the output is all zeros.
import mxnet as mx
data = mx.nd.arange(24).reshape((2,1,2,2,3))

x = mx.sym.Variable('data', shape=(2,1,2,2,3))
conv3D = mx.sym.Convolution(data=x, kernel=(2, 2, 3), pad=(1, 1, 1), num_filter=2)

conv3D_ex = conv3D.simple_bind(mx.gpu(), arg_dicts={'data':data})
args = conv3D_ex.arg_dict

init = mx.init.Uniform(scale=0.01)
for name, arr in args.items():
    if name not in ['data']:
        init(name, arr)

args['data']=data

print args
print conv3D_ex.forward()

{'data':
[[[[[  0.   1.   2.]
    [  3.   4.   5.]]

   [[  6.   7.   8.]
    [  9.  10.  11.]]]]

 [[[[ 12.  13.  14.]
    [ 15.  16.  17.]]

   [[ 18.  19.  20.]
    [ 21.  22.  23.]]]]]
<NDArray 2x1x2x2x3 @cpu(0)>, 'convolution0_weight':
[[[[[ 0.00480439  0.00841988 -0.00921959]
    [ 0.00937926  0.00850281 -0.001073  ]]

   [[ 0.00334638 -0.00780139 -0.00059563]
    [ 0.00026387  0.0055235  -0.00410469]]]]

 [[[[ 0.00428019 -0.00282996  0.00362837]
    [-0.0041597  -0.00361182  0.00621827]]

   [[-0.00691776 -0.00109668 -0.00584008]
    [ 0.00221961 -0.00385449 -0.00168849]]]]]
<NDArray 2x1x2x2x3 @gpu(0)>, 'convolution0_bias':
[ 0.  0.]
<NDArray 2 @gpu(0)>}
[
[[[[[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]]

  [[[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]]]

 [[[[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]]

  [[[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]

   [[ 0.  0.  0.]
    [ 0.  0.  0.]
    [ 0.  0.  0.]]]]]
<NDArray 2x2x3x3x3 @gpu(0)>]

You need to change args['data']=data to args['data'][:] = data so that the content of the data can be copied from cpu to args['data'] on gpu.

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!
Also, do please check out our forum (and Chinese version) for general "how-to" questions.

authors: mafhha reminisce szha


text:
[MXNET-288] Fix ssd example

Description
MXNET-288
Fix issue reported here: https://github.com/apache/incubator-mxnet/issues/10037
Wrong command line arguments for the windows example + missing call to download demo images

@zhreshold

authors: ThomasDelteil piiswrong


text:
[OpPerf] PDF Random ops fix

Description
Newly add PDF Random ops - https://github.com/apache/incubator-mxnet/pull/14617
Caused the opperf.py to error out due to parameter issues.
Hence added new param sample to handle that
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] Changes are complete (i.e. I finished coding on this PR)
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[x] added param

Comments
Ran python benchmark/opperf/opperf.py successfully without error on CPU & GPU
Error resolved
If you checkout master and run python benchmark/opperf/opperf.py
Will give you following error
Traceback (most recent call last):
  File "benchmark/opperf/opperf.py", line 157, in <module>
    sys.exit(main())
  File "benchmark/opperf/opperf.py", line 142, in main
    final_benchmark_results = run_all_mxnet_operator_benchmarks(ctx=ctx, dtype=dtype)
  File "benchmark/opperf/opperf.py", line 73, in run_all_mxnet_operator_benchmarks
    mxnet_operator_benchmark_results.append(run_mx_random_sampling_operators_benchmarks(ctx=ctx, dtype=dtype))
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/nd_operations/random_sampling_operators.py", line 60, in ru
n_mx_random_sampling_operators_benchmarks
    mx_random_sample_op_results = run_op_benchmarks(mx_random_sample_ops, dtype, ctx, warmup, runs)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/benchmark_utils.py", line 127, in run_op_benchmarks
  warmup=warmup, runs=runs)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/benchmark_utils.py", line 109, in run_performance_tes
t
    benchmark_result = _run_nd_operator_performance_test(op, inputs, run_backward, warmup, runs, kwargs_list)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/benchmark_utils.py", line 54, in _run_nd_operator_performance_test
    _, _ = benchmark_helper_func(op, warmup, **kwargs_list[0])
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/profiler_utils.py", line 169, in profile_it
    res = func(*args, **kwargs)
  File "/home/ubuntu/workspace/incubator-mxnet/benchmark/opperf/utils/ndarray_utils.py", line 79, in nd_forward_and_profile
    res = op(*args, **kwargs)
  File "<string>", line 59, in random_pdf_dirichlet
  File "/home/ubuntu/workspace/incubator-mxnet/python/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/home/ubuntu/workspace/incubator-mxnet/python/mxnet/base.py", line 253, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [22:17:53] src/c_api/../imperative/imperative_utils.h:369: Check failed: num_inputs == infered_num_i
nputs (1 vs. 2) : Operator _random_pdf_dirichlet expects 2 inputs, but got 1 instead.

Stack trace:                                                                                [0/1922]
  [bt] (0) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFata
l()+0x32) [0x7fdf8a36f1b2]
  [bt] (1) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(mxnet::imperative::SetNumOutputs(nnvm:
:Op const*, nnvm::NodeAttrs const&, int const&, int*, int*)+0x437) [0x7fdf8d1a4f07]
  [bt] (2) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(MXImperativeInvokeImpl(void*, int, voi
d**, int*, void***, int, char const**, char const**)+0xa50) [0x7fdf8d1a25a0]
  [bt] (3) /home/ubuntu/workspace/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(MXImperativeInvokeEx+0x534) [0x7fdf8d1
a4234]
  [bt] (4) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fdfa3a9aec0]
  [bt] (5) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call+0x22d) [0x7fdfa3a9a87d]
  [bt] (6) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)
 [0x7fdfa3cafe2e]
  [bt] (7) /home/ubuntu/anaconda3/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x12865) [0x7fdfa3cb08
65]
  [bt] (8) python(_PyObject_FastCallDict+0x8b) [0x563b009b2d7b]

@mxnet-label-bot add [Perfornmance, pr-awaiting-review]

@sandeep-krishnamurthy @apeforest


text:
Sparse int64 Large tensor support

Description
Adds support to Sparse tensor creation
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
[x] Code is well-documented:
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[x] src/c_api/c_api.cc
[x]  python/mxnet/ndarray/sparse.py
[x] tests/nightly/test_large_array.py

Test
python -m nose tests/nightly/test_large_array.py:test_sparse_dot
.
----------------------------------------------------------------------
Ran 1 test in 37.702s

OK

Is there a check list of what to change for int64 support in general?

@eric-haibin-lin while I don't think there is uniform guideline for int64 support across all ops. But in most cases, switching the datatype of all the variables holding "indices" from 32bit dtype to 64bit dtype solves the issue. One still has to test that to ensure it doesn't break the functionality and supports int64.
@apeforest @access2rohit any inputs?

@eric-haibin-lin while I don't think there is uniform guideline for int64 support across all ops. But in most cases, switching the datatype of all the variables holding "indices" from 32bit dtype to 64bit dtype solves the issue. One still has to test that to ensure it doesn't break the functionality and supports int64.
@apeforest @access2rohit any inputs?

Plus ensure that the C API being used supports int64_t for indices as well. On language binding side int64_t values need to be passed to C++ backend. For e.g. ctype.int64 should be used for passing large values else they will get truncated at the language binding interface itself.

Adding @reminisce to review the C API change for numpy operator.

One last comment. Otherwise looks good to me. Thanks!

Does https://cwiki.apache.org/confluence/display/MXNET/Large+Tensor+Support correctly document the current practice/changes?

Does https://cwiki.apache.org/confluence/display/MXNET/Large+Tensor+Support correctly document the current practice/changes?

Yes. It enlists the 3 steps needed to extend LTS for operators (includes changes on Front-end language binding side and Back-end C++ api side). I updated the wiki. Please verify. @eric-haibin-lin

authors: ChaiBapchya access2rohit apeforest eric-haibin-lin


text:
Don't typedef wchar_t for C++

Summary
wchar_t is a builtin type for C++.
Retire CONFIG_WCHAR_BUILTIN hack.
Impact
C++
Testing
build tested sim on macOS

see also https://github.com/apache/incubator-nuttx/pull/2064

repo: apache/incubator-nuttx


text:
"runtime=python" fails with "developer error"

package:
  name: hello_world_package
  version: 1.0
  license: Apache-2.0
  actions:
    hello_world_runtime:
      function: src/hello.js
      runtime: python

 "response": {
        "result": {
            "error": "The action failed to generate or locate a binary. See logs for details."
        },

if you select "python@2" it works...

@mrutkows
function: src/hello.js defines js and runtime points to python????
They are not consistent with each other.

@houshengbo yes, i am working on testing variations. Please do not attempt to fix yet.  It is indeed a "mismatch", but trying to see what error messages we emit and when we allow them to pass to OpenWhisk (and do not detect).

However... this worked:
package:
  name: hello_world_package
  version: 1.0
  license: Apache-2.0
  actions:
    hello_world_runtime:
      function: src/hello.py
      runtime: nodejs

        "result": {
            "greeting": "Hello, stranger from unknown"
        },

It seems we SHOULD be doing some client-side mismatch detection and issue some WARNING (and perhaps allow user to still deploy)...  However, is there a bug or inconsistent behavior on the actual OpenWhisk platform?

Now when I try
package:
  name: hello_world_package
  version: 1.0
  license: Apache-2.0
  actions:
    hello_world_runtime:
      function: src/hello.py
      runtime: java

I get an error from wskdeploy:
Error: /Users/Matt/go/src/github.com/apache/incubator-openwhisk-wskdeploy/deployers/servicedeployer.go [623]: OpenWhisk Client Error =====> The request content was malformed:
'main' must be a string defined in 'exec' for 'java' actions (code 31062040) Error code: 144.

This was fixed with PR #669.

authors: houshengbo mrutkows pritidesai

repo: apache/incubator-openwhisk-wskdeploy


text:
Add requiredProperties method to ArtifactStore

Currently required properties related to ArtifactStore is mentioned at various places like.

WhiskAuthStore
WhiskEntityStore
WhiskActivationStore

The properties required are
  def requiredProperties =
    Map(
      dbProvider -> null,
      dbProtocol -> null,
      dbUsername -> null,
      dbPassword -> null,
      dbHost -> null,
      dbPort -> null,
      dbAuths -> null)

These properties are specific to CouchDB implementation and may not be required for a different ArtifactStore implementation
We should add  a requiredProperties to ArtifactStore (like done in LoadBalancerProvider) such that the implementation can govern which all properties are required

Alternatively, we should move the ArtifactStore to pureconfig which completely localizes the concern to the implementation.

we should move the ArtifactStore to pureconfig

Eventually yes but I believe such a change would touch multiple places (ansible scripts) as I am not sure it can be done in backward compatible way. Or can it be done in backward compatible way?

In theory it can but in the past we did all the pureconfig moves by "biting the bullet" and fix every deployment method lazily.
Only the direct value->container mapping is impacted (controller's and invoker's deploy.yml in this case)

Okie would then work on PR which modifies the CouchDBStore to use pureconfig and see if it can be done in backward compatible way say by using substitution as all the current properties are passed via environment variables

Closed via apache/incubator-openwhisk#3369

authors: chetanmeh markusthoemmes

repo: apache/incubator-openwhisk


text:
Change the JDK to use openjdk8-openj9

Apache project will have issue using JDK licensed under oracle BCL or GNU GPL.
For docker image with JDK, we have to switch to use
https://hub.docker.com/r/adoptopenjdk/openjdk8-openj9/.

@markusthoemmes That is good question I am currently seeking the answer as well.
So far, my answer tends to be fine with Apache, because the docker image uses OpenJDK binaries that are built with Eclipse OpenJ9. License can be found here:
Licenses for the products installed within the images:

    Eclipse OpenJ9: The project license is Eclipse Public License 2.0 and Apache License, Version 2.0 (you can choose), but the EPLv2 license has a secondary compatibility for the GPL v2 that is used by OpenJDK. You can combine the two projects and use them together (as OpenJDK's GPL v2 of course). The full license is documented in the LICENSE file.

For detailed information, please refer to https://hub.docker.com/r/adoptopenjdk/openjdk8-openj9/.
However, I do really need to confirm it with legal expert.

@markusthoemmes @houshengbo  Some things that seem clear to me:

The intent of the Oracle Binary Code License Agreement (BCLA) is that "Oracle Java Standard Edition version 8 is still free as long as it is used for general purpose computing." (which includes JDK and JavaFx SDK).
OpenJ9/J9 (equivalent by name now) does not get us the JDK; it is the byte code interpreter only (JVM) and does not (according to my source) include any libs (i.e., JDK) so it does not give us what we need. Given that, If we use Eclipse OpenJ9 we would likely be able to ref. their license just fine.
The "Java community" sponsors distributions that carefully construct JDK+OpenJ9 distributions that have carefully compiled all parts of Java (various licenses) to assure that consumers that they have no onerous licensing problems here: https://adoptopenjdk.net/ (including Docker images. Quoting the site "AdoptOpenJDK provides prebuilt OpenJDK binaries from a fully open source set of build scripts and infrastructure. Looking for docker images? Pull them from our repository on dockerhub". BUT it should still be investigated IMO.

regardless, we should seek to validate our binary packing/distribution decisions with Apache legal.

@mrutkows Apache policy seems to raise caution of BCL:
BCL
    Oracle's (originally Sun's) Binary Code License falls far short of the Open Source Definition, thereby violating the first criterion for license approval.

The openwhisk-runtime-java docker image will not be useful to our users if it doesn't include a J2SE class library.  So I don't see that switching the JVM runtime for licensing reasons is useful.  No matter what JVM runtime we bundle, we will need the class libraries.

@markusthoemmes AFAIK, this discussion was started because the Java runtime's Docker image would be part of our project's (convenience binary) release that would have to consider distribution  issues as that image would not be usable without them.

@rabbah for the source code release, we would not be producing a binary (obviously) which would include no Java-related source code which is fine for that release phase (and all that is required for Apache graduation).   IMO, this issue only arises if we intend to produce our own Docker images as part of a (convenience) binary release.  We have been trying to prepare for both source and binary releases when preparing the LICENSE file.
Options exist; we can proceed with source release (and should) and assure that our LICENSE files clearly indicate which dependencies are source vs. binary (which has been discussed several times, but should look at more closely to assure this distinction is quite clear to any reader).
In the meantime, we should seek Apache guidance IMO if we still intend to produce a java runtime in the form of a Docker image.  @houshengbo your thoughts?

curl --silent --location --retry 3 --header "Cookie: oraclelicense=accept-securebackup-cookie" \
    http://download.oracle.com/otn-pub/java/jdk/"${VERSION}"u"${UPDATE}"-b"${BUILD}"/"${SIG}"/jdk-"${VERSION}"u"${UPDATE}"-linux-x64.tar.gz \
| tar xz -C /tmp && \

@markusthoemmes @dgrove-oss where do we/what are the Java dependencies we have in code? Are there alternatives (non-Java)?

Given Scala's reliance on Java as a pre-req (i.e., it points to Oracle directly):

https://www.scala-lang.org/download/
we may not be able to have binary releases.

@mrutkows  -- Correct.  The controller/invoker are in Scala, so any binary build will have multiple docker images that contain a JVM.  I'd suggest looking to see  how the Kafka and Zookeeper projects handle this.  They are both apache projects with strong Java dependencies.

@dgrove-oss apparently, Kafka's LICENSE and NOTICE files are bolierplate (i.e., no listing of ANY license dependencies for source or binary at all).  Although they clearly produce binaries as part of their tars and Vagrantfiles, etc.)  and clearly up-front in their README simply state
You need to have [Gradle](http://www.gradle.org/installation) and [Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html) installed.

I am very confused as Vincent said Cassandra similarly ignored their Java dependency (even though they may only support source code release, need to confirm).

@dgrove-oss @houshengbo (see above license research on kafka and zookeeper projects).
It looks like for both these cases their "main" repos. appear to have all documentation focused on source code releases.
However, we see somehow that Kafka does provide binaries which, so far, I do not see how they are produced and of course they do not list any dependencies at all!
Of course, the significant difference to consider, when producing binaries, is "do we re-package/re-dist." (i.e., by providing a Docker image pre-installed) where they can simply tell others to "install this yourself".

Does the tool with which binaries are created matter when it comes to the license of the binary (given that the tool's license doesn't prohibit anything like that).
From a dependency PoV, even a binary release like Kafka's is only dependent on some JRE + other binary deps. That shouldn't affect the product's license at all.
The only "binary" where we'd bundle a specific JRE/JDK is our docker images. Are those officially part of the release @mrutkows?

As for docker images: There's no official image for Kafka and the "official" image for Zookeeper is not part of the Apache project. https://github.com/31z4/zookeeper-docker

@markusthoemmes as per the Docker images published, I found the same... It seems that packaging a "jar" is fine, we just cannot redistribute images (or anything that has the actual JRE/JDK.  At least, that is my reading and is consistent with what we are seeing with other Apache projects.

Apache projects, like Kafka, Zookeeper, Casandra, etc, do not ship anything standard in Java, though they depend on Java. They do not ship docker image depending on JDK either. Their binaries depend on JVM to compile, but that's fine.
Our OpenWhisk release, I mean the release of Docker image(which is one openwhisk's strengths of our platform), pull down the oracle JDK in Dockerfile, build it inside our image, etc. This may put us in peril, since Oracle BCL license of oracle JDK is not welcome in Apache.
For source code release, we do not need to bother the Oracle license. This concern is raised for the release of docker images.

Just had conversation with IBM's expert on Apache Harmony and AdoptOpenJSDK project and he said he believed we should have no problems if we either use the adopt image  if we choose to build our own image using the OpenJ9-OpenJDK from https://adoptopenjdk.net/ (with or without the HotSpot JIT).  He is also connecting me with Tim Ellison at Apache (also worked on Harmony and now on Adopt) to confirm his opinion and perhaps get an Apache point-of-view.

Either way, we should plan to use the the OpenJDK/J9...

Quoting from the adoptopenjdk license file (https://hub.docker.com/r/adoptopenjdk/openjdk8-openj9/).

The Dockerfiles and associated scripts are licensed under the Apache License, Version 2.0.
Licenses for the products installed within the images:
Eclipse OpenJ9 + OpenJDK: The combined works license is GNU GPL v2 with Classpath Exception.

As with all Docker images, other software is likely to be included, which might be under other licenses (such as Bash, etc from the base distribution, along with any direct or indirect dependencies of the primary software being contained).
As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.

In other words, this Dockerfile is just like almost any other non-trivial Docker file.  The produced images of course contains software under a variety of licenses because the real world is messy.  I'm missing why this is a particular problem (vs for example an image that contains bash, which is GPL3 https://www.gnu.org/software/bash/).

Good one @dgrove-oss! Didn't think about all the other stuff in a docker image. I think its illusionary to have a docker image with no GPL software in it, so why bother about the JDK specifically?

Superseded by #4217.

authors: dgrove-oss houshengbo markusthoemmes mrutkows rabbah


text:
Restore default runtimes after YARN tests finish

Description

Restores ExecMainfest's runtimes after YARN tests complete. Since ExecMainfest is a global object, modifications to its runtimes may result in unexpected behavior in other tests utilizing the object.
Related issue and scope

[ ] I opened an issue to propose and discuss this change (#????)

My changes affect the following components

[ ] API
[ ] Controller
[ ] Message Bus (e.g., Kafka)
[ ] Loadbalancer
[ ] Invoker
[ ] Intrinsic actions (e.g., sequences, conductors)
[ ] Data stores (e.g., CouchDB)
[x] Tests
[ ] Deployment
[ ] CLI
[ ] General tooling
[ ] Documentation

Types of changes

[x] Bug fix (generally a non-breaking change which closes an issue).
[ ] Enhancement or new feature (adds new functionality).
[ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

[x] I signed an Apache CLA.
[x] I reviewed the style guides and followed the recommendations (Travis CI will check :).
[ ] I added tests to cover my changes.
[ ] My changes require further changes to the documentation.
[ ] I updated the documentation where necessary.

Codecov Report

Merging #4313 into master will decrease coverage by 4.54%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4313      +/-   ##
==========================================
- Coverage   85.57%   81.03%   -4.55%
==========================================
  Files         168      168
  Lines        7839     7839
  Branches      519      519
==========================================
- Hits         6708     6352     -356
- Misses       1131     1487     +356

Impacted Files
Coverage Δ

...core/database/cosmosdb/RxObservableImplicits.scala
0% <0%> (-100%)
:arrow_down:

...core/database/cosmosdb/CosmosDBArtifactStore.scala
0% <0%> (-95.18%)
:arrow_down:

...sk/core/database/cosmosdb/CosmosDBViewMapper.scala
0% <0%> (-92.6%)
:arrow_down:

...whisk/core/database/cosmosdb/CosmosDBSupport.scala
0% <0%> (-84.62%)
:arrow_down:

...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala
4% <0%> (-52%)
:arrow_down:

...in/scala/org/apache/openwhisk/common/Counter.scala
40% <0%> (-20%)
:arrow_down:

...nwhisk/core/database/cosmosdb/CosmosDBConfig.scala
94.11% <0%> (-5.89%)
:arrow_down:

...rg/apache/openwhisk/common/ForcibleSemaphore.scala
88.46% <0%> (-3.85%)
:arrow_down:

...whisk/connector/kafka/KafkaProducerConnector.scala
87.5% <0%> (+15%)
:arrow_up:

.../scala/org/apache/openwhisk/utils/Exceptions.scala
40% <0%> (+20%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2fad9a3...badcde1. Read the comment docs.

Thanks James and Chetan.

authors: codecov-io dubee rabbah


text:
[Improve] Unified command line parameters

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation
close #949
I found that there are a lot of repeated codes in the command line parsing through action, and some parameters need to be parsed separately. I recommend using jcommander for this part of the job.
Solution

Use jcommander to unify console parameters
Remove duplicate code
Optimized help tips

Anything else?
No response
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

@JingsongLi  Do we need this part of the optimization? For example, partition configuration partition=key1=value1,key2=value2. Now we can use partition=key1=value1 key2=value2, there is also no need to parse it separately in the code, just need this      @DynamicParameter(             names = {"--partition"},             description = "partition key value pair, for example key1=value1 key2=value2")     private Map<String, String> partition;

At present, there will be no friendly prompts for parsing parameter exceptions.

At present, the command line parameters are scattered in the method

This is deceptive. We can define parameters as parameters of action, like

this is more maintenance-friendly.

At present, the definition of usage is manually written, which is more detailed but also brings maintenance complexity. We can automatically generate usage through jcommander. Of course, jcommander also provides the ability to customize usage.

@JingsongLi @SteNicholas Can you give me some advice. I'd be happy to parameterize this function

I think this is a good improvement, but I don't think our Actions are so complicated.
I think we can use this later, in the event of a problem occurring, there is no apparent problem at this time

I think this is a good improvement, but I don't think our Actions are so complicated. I think we can use this later, in the event of a problem occurring, there is no apparent problem at this time

I agree with you.

authors: JingsongLi liugddx

repo: apache/incubator-paimon


text:
input kudu  cannot  work  with seatunnel1.5.7

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
I tested the function of seatunnel and found that when the input is kudu, an error will be reported when executing the task. Always says kudu table not found. But the kudu table exists and can be used normally.
Below is the error message. Really appreciate it if you can help
-----------------------------------------------------------------------------------
2022-03-10 05:44:57 INFO  SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://cdh1.f-qa.igen:13000
find and register UDFs & UDAFs
found and registered UDFs count[2], UDAFs count[0]
Exception in thread "main" java.lang.Exception: org.apache.kudu.client.NonRecoverableException: The table does not exist: table_name: "device.device_analysis_data_day"
at io.github.interestinglab.waterdrop.Waterdrop$.main(Waterdrop.scala:62)
at io.github.interestinglab.waterdrop.Waterdrop.main(Waterdrop.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.kudu.client.NonRecoverableException: The table does not exist: table_name: "device.device_analysis_data_day"
SeaTunnel Version
1.5.7
SeaTunnel Config
spark {
  # seatunnel defined streaming batch duration in seconds
  spark.streaming.batchDuration = 5

  spark.app.name = "seatunnel"
  spark.ui.port = 13000
}

 input {
   kudu{
    kudu_master="cdh1.f-qa.igen:7051,cdh2.f-qa.igen:7051,cdh3.f-qa.igen:7051"
    kudu_table="device.device_analysis_data_day"
    result_table_name="current"
  }
 }
 filter {
   sql {
    sql = "select device_sn  from current limit 100 ",
  }
 }
output {
  Stdout {
  }
}

Running Command
./bin/start-seatunnel.sh	--master local[4]  --deploy-mode client  --config ./config/application_kudu2s.conf

Error Exception
2022-03-10 05:44:57 INFO  SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://cdh1.f-qa.igen:13000
find and register UDFs & UDAFs
found and registered UDFs count[2], UDAFs count[0]
Exception in thread "main" java.lang.Exception: org.apache.kudu.client.NonRecoverableException: The table does not exist: table_name: "device.device_analysis_data_day"
	at io.github.interestinglab.waterdrop.Waterdrop$.main(Waterdrop.scala:62)
	at io.github.interestinglab.waterdrop.Waterdrop.main(Waterdrop.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.kudu.client.NonRecoverableException: The table does not exist: table_name: "device.device_analysis_data_day"

Flink or Spark Version
spark version 2.3.0.cloudera4
Java or Scala Version
java 1.8
scala 2.11
Screenshots
11
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

ok

you can try to write
kudu_table="impala::device.device_analysis_data_day"

repo: apache/incubator-seatunnel


text:
[Hotfix][Connector] Fix ConcurrentModificationException when snapshotState based on SourceReaderBase

Purpose of this pull request

Check list

[ ] Code changed are covered with tests, or it does not need tests for reason:
[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

[ ] Update the release-note.

2023-01-31 10:23:30,740 ERROR org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointBarrierTriggerOperation - [ws4]:5801 [seatunnel_default_cluster] [5.1] java.util.ConcurrentModificationException
java.util.concurrent.CompletionException: java.util.ConcurrentModificationException
	at com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:545) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at java.util.ArrayList.forEach(ArrayList.java:1255) ~[?:1.8.0_151]
	at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.sendToAllReader(SourceSplitEnumeratorTask.java:277) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.triggerBarrier(SourceSplitEnumeratorTask.java:139) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointBarrierTriggerOperation.lambda$run$0(CheckpointBarrierTriggerOperation.java:77) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:47) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointBarrierTriggerOperation.run(CheckpointBarrierTriggerOperation.java:72) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:411) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:438) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:601) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:580) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:541) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:241) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.utils.NodeEngineUtil.sendOperationToMemberNode(NodeEngineUtil.java:40) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.sendOperationToMemberNode(CheckpointManager.java:228) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_151]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_151]
	at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1553) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:545) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:1.8.0_151]
	at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:438) ~[?:1.8.0_151]
	at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.triggerCheckpoint(CheckpointCoordinator.java:425) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) ~[?:1.8.0_151]
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577) ~[?:1.8.0_151]
	at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:443) ~[?:1.8.0_151]
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_151]
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) ~[?:1.8.0_151]
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_151]
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[?:1.8.0_151]
Caused by: java.util.ConcurrentModificationException
	at java.util.HashMap.forEach(HashMap.java:1292) ~[?:1.8.0_151]
	at org.apache.seatunnel.connectors.seatunnel.common.source.reader.SourceReaderBase.snapshotState(SourceReaderBase.java:118) ~[?:?]
	at org.apache.seatunnel.connectors.cdc.base.source.reader.IncrementalSourceReader.snapshotState(IncrementalSourceReader.java:172) ~[?:?]
	at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.triggerBarrier(SourceFlowLifeCycle.java:177) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.triggerBarrier(SourceSeaTunnelTask.java:85) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.task.operation.checkpoint.BarrierFlowOperation.lambda$run$0(BarrierFlowOperation.java:77) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:47) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.task.operation.checkpoint.BarrierFlowOperation.run(BarrierFlowOperation.java:72) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:411) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:438) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:601) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:580) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:541) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:241) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.utils.NodeEngineUtil.sendOperationToMemberNode(NodeEngineUtil.java:40) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.execution.TaskExecutionContext.sendToMember(TaskExecutionContext.java:43) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.lambda$sendToAllReader$3(SourceSplitEnumeratorTask.java:276) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597) ~[?:1.8.0_151]
	at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.sendToAllReader(SourceSplitEnumeratorTask.java:275) ~[seatunnel-starter.jar:2.3.1-SNAPSHOT]
	... 33 more

PTAL