text:
Enhance the support for BYTES data type

Add BytesUtils to handle encode/decode for Hex string
Add FieldSpec.DataType.convert() to convert string value to data type
Support Hex string for index(), indexOf(), inRange() in all BYTES dictionaries
Add MutableDictionary.compare(dictId1, dictId2) to help the sort process and support sorting byte[]
Support returning Hex string in group-by for byte[] column
In MutableSegmentImpl, uniform the sort for all types, reduce the memory usage especially for STRING and BYTES columns

Codecov Report

Merging #4198 into master will increase coverage by 0.02%.
The diff coverage is 64.16%.

@@             Coverage Diff              @@
##             master    #4198      +/-   ##
============================================
+ Coverage     67.16%   67.19%   +0.02%
  Complexity       20       20
============================================
  Files          1034     1035       +1
  Lines         51491    51395      -96
  Branches       7214     7194      -20
============================================
- Hits          34585    34535      -50
+ Misses        14520    14498      -22
+ Partials       2386     2362      -24

Impacted Files
Coverage Δ
Complexity Δ

...re/realtime/impl/dictionary/MutableDictionary.java
100% <ø> (ø)
0 <0> (ø)
:arrow_down:

...e/impl/dictionary/BaseOnHeapMutableDictionary.java
90.9% <ø> (ø)
0 <0> (ø)
:arrow_down:

.../impl/dictionary/BaseOffHeapMutableDictionary.java
83.92% <ø> (+2.89%)
0 <0> (ø)
:arrow_down:

...not/core/segment/index/readers/BaseDictionary.java
53.12% <ø> (ø)
0 <0> (ø)
:arrow_down:

...ection/iterator/StringSelectionColumnIterator.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...che/pinot/core/util/FixedIntArrayOffHeapIdMap.java
79.48% <0%> (-2.1%)
0 <0> (ø)

...or/SelectionSingleValueColumnWithDictIterator.java
77.77% <0%> (ø)
0 <0> (ø)
:arrow_down:

...e/impl/dictionary/LongOnHeapMutableDictionary.java
91.3% <100%> (+2.41%)
0 <0> (ø)
:arrow_down:

...apache/pinot/common/utils/primitive/ByteArray.java
76.92% <100%> (ø)
0 <0> (ø)
:arrow_down:

...tion/groupby/DictionaryBasedGroupKeyGenerator.java
96.84% <100%> (ø)
0 <0> (ø)
:arrow_down:

... and 55 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 18d628b...791f804. Read the comment docs.

source: gharchive/pull-request

authors: Jackie-Jiang codecov-io

license_type: permissive

license_source: github-api


text:
Update quota manager to reduce zk access

This PR updates quota manager to reduce zk access.
Currently when a broker resource gets changed, all the tables which contains qps quota will be re-calculated. This PR reduces the zk reads.

Codecov Report

Merging #4979 into master will increase coverage by 21%.
The diff coverage is 32.43%.

@@             Coverage Diff              @@
##             master    #4979      +/-   ##
============================================
+ Coverage     36.35%   57.35%     +21%
- Complexity        0       12      +12
============================================
  Files          1174     1174
  Lines         62244    62268      +24
  Branches       9148     9150       +2
============================================
+ Hits          22627    35713   +13086
+ Misses        37596    23877   -13719
- Partials       2021     2678     +657

Impacted Files
Coverage Δ
Complexity Δ

...quota/HelixExternalViewBasedQueryQuotaManager.java
61.98% <32.14%> (+24.56%)
0 <0> (ø)
:arrow_down:

...ache/pinot/broker/queryquota/QueryQuotaConfig.java
57.14% <33.33%> (+57.14%)
0 <0> (ø)
:arrow_down:

...e/impl/dictionary/LongOnHeapMutableDictionary.java
57.31% <0%> (-4.88%)
0% <0%> (ø)

...r/filter/predicate/PredicateEvaluatorProvider.java
52.17% <0%> (-4.35%)
0% <0%> (ø)

...ot/controller/helix/core/util/HelixSetupUtils.java
88.75% <0%> (ø)
0% <0%> (ø)
:arrow_down:

...impl/dictionary/DoubleOnHeapMutableDictionary.java
36.58% <0%> (ø)
0% <0%> (ø)
:arrow_down:

...ces/PinotSegmentUploadDownloadRestletResource.java
61.34% <0%> (+0.61%)
0% <0%> (ø)
:arrow_down:

.../impl/dictionary/LongOffHeapMutableDictionary.java
64.51% <0%> (+1.07%)
0% <0%> (ø)
:arrow_down:

...e/io/writer/impl/MutableOffHeapByteArrayStore.java
85.71% <0%> (+1.09%)
0% <0%> (ø)
:arrow_down:

...g/apache/pinot/common/request/InstanceRequest.java
28.83% <0%> (+1.14%)
0% <0%> (ø)
:arrow_down:

... and 586 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e81e530...c2733c5. Read the comment docs.

I still dont understand how this reduces ZK access. Correct me if I am wrong with the following description.
We used to fetch the propertystore table config all the time. Now, we do a Stat followed by a fetch if the stat does not match.
So, we have effectively increased the number of read calls to zk.
Yes, we have reduced the overall volume of reads from ZK, but I am not sure that is a bottleneck we are facing. Our problem with ZK is on the write side.

OK, the resolution is that most of the time we don't need to fetch the table config. On clusters with 1000s of tables this change will reduce the volume of data pulled from zk each time we restart a broker.

authors: codecov-io jackjlli mcvsubbu

repo: apache/incubator-pinot


text:
[REEF-456]:Make job submission folder prefix configurable

This addressed the issue by adding a JobSubmissionDirectoryPrefix named parameter and propogating it to driver.
JIRA: REEF-456
This closes #

I'll do a pass. But we also need to test this on a YARN / HDI cluster.

@beysims I've done a pass.

@markusweimer, I have adressed your feedback

I'll do another pass.

@beysims Done with my pass, one minor comment. Once that is addressed, I'll test on HDI.

@markusweimer undone changes to YarnContainerManager. Done with this pass

@beysims Thanks, I'll test

tests pass on Windows local and HDInsight. I'll merge.

authors: beysims markusweimer

repo: apache/incubator-reef


text:
[Bug] [flink-quick-start] Run seatunnel example with flink 1.13.6 cause exception

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
can not run.
SeaTunnel Version
2.3.0
SeaTunnel Config
env {
  execution.parallelism = 1
  job.mode = "BATCH"
}

source {
    FakeSource {
      result_table_name = "fake"
      row.num = 16
      schema = {
        fields {
          name = "string"
          age = "int"
        }
      }
    }
}

transform {

}

sink {
  Console {}
}

Running Command
./bin/start-seatunnel-flink-connector-v2.sh --config test.config

Error Exception
Execute SeaTunnel Flink Job: ${FLINK_HOME}/bin/flink run -c org.apache.seatunnel.core.starter.flink.SeatunnelFlink /Users/chucheng/software/SeaTunnel/apache-seatunnel-incubating-2.3.0/starter/seatunnel-flink-starter.jar --config test.config -Dpipeline.name=SeaTunnel

------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Flink job executed failed
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
	at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
	at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.seatunnel.core.starter.exception.CommandExecuteException: Flink job executed failed
	at org.apache.seatunnel.core.starter.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:57)
	at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39)
	at org.apache.seatunnel.core.starter.flink.SeatunnelFlink.main(SeatunnelFlink.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
	... 8 more
Caused by: org.apache.seatunnel.core.starter.exception.TaskExecuteException: Execute Flink job error
	at org.apache.seatunnel.core.starter.flink.execution.FlinkExecution.execute(FlinkExecution.java:100)
	at org.apache.seatunnel.core.starter.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:55)
	... 15 more
Caused by: org.apache.flink.util.FlinkException: Failed to execute job 'seatunnel'.
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1969)
	at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:137)
	at org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1833)
	at org.apache.seatunnel.core.starter.flink.execution.FlinkExecution.execute(FlinkExecution.java:98)
	... 16 more
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.
	at org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$9(RestClusterClient.java:405)
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884)
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$9(FutureUtils.java:390)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$1(RestClient.java:430)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:570)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:549)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:321)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:337)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.runtime.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.
	at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$9(FutureUtils.java:386)
	... 21 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8081
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
	at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957)
	at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
	... 19 more
Caused by: org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8081
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
	at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:748)

Flink or Spark Version
No response
Java or Scala Version
1.8
Screenshots
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Check the env of flink

@TyrantLucifer thanks. my mistake.

how do you fix that，i have a same problem like it，i use flink 1.17

source: gharchive/issue

repo: apache/incubator-seatunnel


text:
[Bug] connector-influxdb dependency influxdb-java's version shuld be 2.15 not 2.21

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
It's Happened NoSuchMethod when the dependency of  influxdb-java's version 2.1 .
detail:
23/02/27 10:50:06 INFO AbstractPluginDiscovery: Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='InfluxDB'} from classpath
Exception in thread "main" java.lang.NoSuchMethodError: kotlin.collections.ArraysKt.copyInto([B[BIII)[B
at shaded.okio.Segment.writeTo(Segment.kt:169)
at shaded.okio.Segment.compact(Segment.kt:152)
at shaded.okio.Buffer.write(Buffer.kt:1842)
at shaded.okio.Buffer.read(Buffer.kt:1854)
at shaded.okio.Buffer.writeAll(Buffer.kt:1642)
at shaded.okio.Options$Companion.buildTrieRecursive(Options.kt:187)
at shaded.okio.Options$Companion.buildTrieRecursive(Options.kt:174)
at shaded.okio.Options$Companion.buildTrieRecursive$default(Options.kt:113)
at shaded.okio.Options$Companion.of(Options.kt:72)
at shaded.okhttp3.internal.Util.(Util.kt:71)
at shaded.okhttp3.internal.concurrent.TaskRunner.(TaskRunner.kt:309)
at shaded.okhttp3.ConnectionPool.(ConnectionPool.kt:41)
at shaded.okhttp3.ConnectionPool.(ConnectionPool.kt:47)
at shaded.okhttp3.OkHttpClient$Builder.(OkHttpClient.kt:471)
at org.apache.seatunnel.connectors.seatunnel.influxdb.client.InfluxDBClient.getInfluxDB(InfluxDBClient.java:42)
at org.apache.seatunnel.connectors.seatunnel.influxdb.source.InfluxDBSource.prepare(InfluxDBSource.java:84)
at org.apache.seatunnel.core.starter.spark.execution.SourceExecuteProcessor.initializePlugins(SourceExecuteProcessor.java:88)
at org.apache.seatunnel.core.starter.spark.execution.AbstractPluginExecuteProcessor.(AbstractPluginExecuteProcessor.java:49)
at org.apache.seatunnel.core.starter.spark.execution.SourceExecuteProcessor.(SourceExecuteProcessor.java:51)
at org.apache.seatunnel.core.starter.spark.execution.SparkExecution.(SparkExecution.java:50)
at org.apache.seatunnel.core.starter.spark.command.SparkApiTaskExecuteCommand.execute(SparkApiTaskExecuteCommand.java:54)
at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39)
at org.apache.seatunnel.core.starter.spark.SeatunnelSpark.main(SeatunnelSpark.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:855)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:930)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:939)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
SeaTunnel Version
2.3.0
SeaTunnel Config
env {
    spark.executor.memory="8G"
    spark.driver.memory="4g"
    spark.app.name="SeaTunnel"
    spark.executor.instances="3"
}
 source{

InfluxDB {
    url="http://10.x.x.22:8086"
    database="data_iot"
    password="xxxxx"
    sql="select time,temp3 from lunzi where time >= '2023-01-11 16:18:00' and time < '2023-01-11 16:19:00' tz('Asia/Shanghai')"

    fields {
      time = BIGINT
      temp3 = INT
    }

}

}
 transform{

sql {
    source_table_name="source1"
    sql="select id,time,value from source1"
    result_table_name="transform31"
}

}
 sink{

kafka {
    source_table_name="transform31"
    topic="seatunnel-test"
    bootstrap.servers="10.x.x.21:9192,10.x.x.22:9192,10.x.x.23:9192"
}

}

Running Command
./bin/start-seatunnel-spark-connector-v2.sh --master spark://hadoop02:7077 --deploy-mode client --config ./config/influx.to.kafka.template

Error Exception
23/02/27 10:50:06 INFO AbstractPluginDiscovery: Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='InfluxDB'} from classpath
Exception in thread "main" java.lang.NoSuchMethodError: kotlin.collections.ArraysKt.copyInto([B[BIII)[B
        at shaded.okio.Segment.writeTo(Segment.kt:169)
        at shaded.okio.Segment.compact(Segment.kt:152)
        at shaded.okio.Buffer.write(Buffer.kt:1842)
        at shaded.okio.Buffer.read(Buffer.kt:1854)
        at shaded.okio.Buffer.writeAll(Buffer.kt:1642)
        at shaded.okio.Options$Companion.buildTrieRecursive(Options.kt:187)
        at shaded.okio.Options$Companion.buildTrieRecursive(Options.kt:174)
        at shaded.okio.Options$Companion.buildTrieRecursive$default(Options.kt:113)
        at shaded.okio.Options$Companion.of(Options.kt:72)
        at shaded.okhttp3.internal.Util.<clinit>(Util.kt:71)
        at shaded.okhttp3.internal.concurrent.TaskRunner.<clinit>(TaskRunner.kt:309)
        at shaded.okhttp3.ConnectionPool.<init>(ConnectionPool.kt:41)
        at shaded.okhttp3.ConnectionPool.<init>(ConnectionPool.kt:47)
        at shaded.okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:471)
        at org.apache.seatunnel.connectors.seatunnel.influxdb.client.InfluxDBClient.getInfluxDB(InfluxDBClient.java:42)
        at org.apache.seatunnel.connectors.seatunnel.influxdb.source.InfluxDBSource.prepare(InfluxDBSource.java:84)
        at org.apache.seatunnel.core.starter.spark.execution.SourceExecuteProcessor.initializePlugins(SourceExecuteProcessor.java:88)
        at org.apache.seatunnel.core.starter.spark.execution.AbstractPluginExecuteProcessor.<init>(AbstractPluginExecuteProcessor.java:49)
        at org.apache.seatunnel.core.starter.spark.execution.SourceExecuteProcessor.<init>(SourceExecuteProcessor.java:51)
        at org.apache.seatunnel.core.starter.spark.execution.SparkExecution.<init>(SparkExecution.java:50)
        at org.apache.seatunnel.core.starter.spark.command.SparkApiTaskExecuteCommand.execute(SparkApiTaskExecuteCommand.java:54)
        at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39)
        at org.apache.seatunnel.core.starter.spark.SeatunnelSpark.main(SeatunnelSpark.java:34)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:855)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:930)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:939)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Flink or Spark Version
spark 2.4.8
Java or Scala Version
jdk 1.8
Screenshots
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

not allow dowgrade version


text:
[ST-Engine] Improve task and physical plan to support checkpoint

Purpose of this pull request
Each time the user changes the action of parallelism, the vertex will generate a fixed vertex-ID;

Check list

[ ] Code changed are covered with tests, or it does not need tests for reason:
[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

This PR will be split into multiple PR to solve different problems;


text:
Update README.md of ShardingScaling

For #4322.
Changes proposed in this pull request:

Update README.md

Pull Request Test Coverage Report for Build 9870

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 59.961%

Totals

Change from base Build 9865:
0.0%

Covered Lines:
12542

Relevant Lines:
20917

💛  - Coveralls

authors: coveralls ssxlulu

repo: apache/incubator-shardingsphere

license_source: bigquery


text:
upgrade commons-io due to cve

Fixes #3266

Make sure that:

[X] You have read the contribution guidelines.
[ ] You submit test cases (unit or integration tests) that back your changes.
[ ] Your local test passed mvn clean install -Dmaven.javadoc.skip=true.

Codecov Report

Merging #3269 (0c819a3) into master (993f86a) will decrease coverage by 0.12%.
The diff coverage is n/a.

:exclamation: Current head 0c819a3 differs from pull request most recent head e6bd9b0. Consider uploading reports for the commit e6bd9b0 to get more accurate results

@@             Coverage Diff              @@
##             master    #3269      +/-   ##
============================================
- Coverage     58.43%   58.30%   -0.13%
+ Complexity     4646     4641       -5
============================================
  Files           744      744
  Lines         21047    21047
  Branches       1925     1925
============================================
- Hits          12298    12272      -26
- Misses         7662     7682      +20
- Partials       1087     1093       +6

Impacted Files
Coverage Δ

...ache/shenyu/plugin/grpc/cache/GrpcClientCache.java
87.50% <0.00%> (-12.50%)
:arrow_down:

...yu/plugin/logging/utils/LogCollectConfigUtils.java
66.66% <0.00%> (-8.89%)
:arrow_down:

...henyu/plugin/grpc/resolver/ShenyuNameResolver.java
54.25% <0.00%> (-8.52%)
:arrow_down:

...gin/logging/rocketmq/RocketMQLogCollectClient.java
68.00% <0.00%> (-8.00%)
:arrow_down:

...he/shenyu/plugin/grpc/client/ShenyuGrpcClient.java
8.00% <0.00%> (-4.00%)
:arrow_down:

...che/shenyu/sync/data/http/HttpSyncDataService.java
80.48% <0.00%> (-3.26%)
:arrow_down:

...henyu/admin/service/impl/UpstreamCheckService.java
70.14% <0.00%> (-1.50%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 993f86a...e6bd9b0. Read the comment docs.

authors: codecov-commenter pjfanning

repo: apache/incubator-shenyu


text:
5.0-alpha , Can't find Jorg.apache.skywalking.apm.plugin.jedis.v2.edisClusterConstructorWithListHostAndPortArgInterceptor

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ ] Question or discussion
[√ ] Bug
[ ] Requirement
[ ] Feature or performance improvement

Question

What do you want to know?
about this "Can't find Jorg.apache.skywalking.apm.plugin.jedis.v2.edisClusterConstructorWithListHostAndPortArgInterceptor"，how to solve?

Bug

Which version of SkyWalking, OS and JRE?
5.0-alpha，win7，jdk8
Which company or project?
self project
What happen?
If possible, provide a way for reproducing the error. e.g. demo application, component version.

ERROR 2018-04-04 13:28:29 SkyWalkingAgent :  Enhance class redis.clients.jedis.JedisCluster error.
org.apache.skywalking.apm.agent.core.plugin.PluginException: Can't create InstanceConstructorInterceptor.
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ConstructorInter.(ConstructorInter.java:53)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassEnhancePluginDefine.enhanceInstance(ClassEnhancePluginDefine.java:131)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassEnhancePluginDefine.enhance(ClassEnhancePluginDefine.java:73)
at org.apache.skywalking.apm.agent.core.plugin.AbstractClassEnhancePluginDefine.define(AbstractClassEnhancePluginDefine.java:75)
at org.apache.skywalking.apm.agent.SkyWalkingAgent$3.transform(SkyWalkingAgent.java:83)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Transformer$Compound.transform(AgentBuilder.java:2335)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:8902)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:9306)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:9269)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1300(AgentBuilder.java:9047)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:9625)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:9575)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.skywalking.apm.dependencies.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:9194)
at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2277)
at org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:811)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1254)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
at com.cm4j.common.redis.RedisCluster.init(RedisCluster.java:48)
at com.cm4j.common.redis.RedisCluster.init(RedisCluster.java:32)
at com.cm4j.common.redis.RedisCluster.(RedisCluster.java:51)
at com.cm4j.common.redis.RedisCluster.(RedisCluster.java:53)
at com.woniu.sncp.business.imprest.ImprestFacadeServiceImpl.createOrder(ImprestFacadeServiceImpl.java:353)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController.commonImprest(ImprestApiController.java:223)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController.standardImprest(ImprestApiController.java:111)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController.officialImprestCreateOrder$original$ToX8GqGQ(ImprestApiController.java:270)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController.officialImprestCreateOrder$original$ToX8GqGQ$accessor$ujHj16Dc(ImprestApiController.java)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController$auxiliary$h02xV4nX.call(Unknown Source)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:93)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController.officialImprestCreateOrder(ImprestApiController.java)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController$$FastClassByCGLIB$$a1a15f58.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
at com.woniu.sncp.aop.LogRecordAspectJ.logAround(LogRecordAspectJ.java:80)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:55)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:50)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
at com.woniu.sncp.web.imprest.official.api.ImprestApiController$$EnhancerByCGLIB$$b0f839a2.officialImprestCreateOrder()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest$original$9jsvSxxD(InvocableHandlerMethod.java:126)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest$original$9jsvSxxD$accessor$EsOW2ppt(InvocableHandlerMethod.java)
at org.springframework.web.method.support.InvocableHandlerMethod$auxiliary$QHrS7D2P.call(Unknown Source)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:93)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.woniu.sncp.web.filter.DisabledCacheFilter.doFilter(DisabledCacheFilter.java:30)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at com.woniu.sncp.web.imprest.app.LogMonitorFilter.doFilterInternal(LogMonitorFilter.java:70)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at com.woniu.sncp.web.imprest.official.api.filter.ApiJsonpFilter.doFilterInternal(ApiJsonpFilter.java:73)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke$original$0HamuSSj(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardWrapperValve.invoke$original$0HamuSSj$accessor$d5d2Xk4n(StandardWrapperValve.java)
at org.apache.catalina.core.StandardWrapperValve$auxiliary$mz7FOEUf.call(Unknown Source)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:93)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Can't find Jorg.apache.skywalking.apm.plugin.jedis.v2.edisClusterConstructorWithListHostAndPortArgInterceptor
at org.apache.skywalking.apm.agent.core.plugin.loader.AgentClassLoader.findClass(AgentClassLoader.java:119)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.skywalking.apm.agent.core.plugin.loader.InterceptorInstanceLoader.load(InterceptorInstanceLoader.java:77)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ConstructorInter.(ConstructorInter.java:51)
... 129 more

Requirement or improvement

Please describe about your requirements or improvement suggestions.

@50168383 @wu-sheng this bug has been fixed in pr #1021

@candyleer Thanks a lot. I think I miss that :P

authors: 50168383 candyleer wu-sheng

repo: apache/incubator-skywalking


text:
[Bug] 2.0.0 k8s deployment and k8s pod are not generated

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
streampark was deployed
The docker setting is set
Set up the flink home
Authorization k8s is as follows:
cp -rf  /etc/kubernetes/admin.conf  /root/.kube/config
创建Flink作业使用的k8s namespace
kubectl create ns znyw-base-flink-cluster
对default用户进行RBAC资源绑定
kubectl create clusterrolebinding znyw-base-flink-cluster-role-binding-default --clusterrole=edit --serviceaccount=znyw-base-flink-cluster:default
sumbit flinksql to k8s but k8s deployment and k8s pod are not generated

StreamPark Version
test 2.0.0

Java Version
No response
Flink Version
jdk 8
Scala Version of Flink
2.12
Error Exception
2022-11-21 12:04:35.572 StreamPark [ForkJoinPool-1-worker-29] ERROR o.a.s.f.k.KubernetesRetriever:71 - [StreamPark] Get flinkClient error, the error is: java.lang.RuntimeException: org.apache.flink.client.deployment.ClusterRetrieveException: Could not get the rest endpoint of data-2-kafka
2022-11-21 12:06:20.559 StreamPark [ForkJoinPool-1-worker-22] ERROR o.a.s.f.k.KubernetesRetriever:71 - [StreamPark] Get flinkClient error, the error is: java.lang.RuntimeException: org.apache.flink.client.deployment.ClusterRetrieveException: Could not get the rest endpoint of data-2-kafka

Screenshots

Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

if more log infos offered , it would be more better
PS: you can have a try the latest dev version.

repo: apache/incubator-streampark


text:
[feature] Introduce flink jdbc catalog store plugin.

Contribution Checklist

Make sure that the pull request corresponds to a 3913.

What changes were proposed in this pull request
Issue Number: close #3913
Brief change log
Verifying this change

This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
jdbc factory load
*jdbc catalog crud test*
This change added tests and can be verified as follows:

*Added Test to verify the change.
Manually verified the change by testing locally. -->

Does this pull request potentially affect one of the following parts

Dependencies (does it add or upgrade a dependency): (yes)

hi Mrart:
HI, thanks for your contribution. There are a few questions to discuss:

Currently, this PR only implements catalog storage. What we need to do is catalog management. Of course, catalog management includes storage. Do you have a complete plan for the catalog management feature? Can it be divided into subtasks like this PR? In this way, we can better assess risks and know the development progress
The current PR only implements MySQL storage and has JSON type. How do we support databases, such as MySQL, PGSQL, and h2 on the streampark platform? Do you have any relevant plans?

hi Mrart:
HI, thanks for your contribution. There are a few questions to discuss:

Currently, this PR only implements catalog storage. What we need to do is catalog management. Of course, catalog management includes storage. Do you have a complete plan for the catalog management feature? Can it be divided into subtasks like this PR? In this way, we can better assess risks and know the development progress
The current PR only implements MySQL storage and has JSON type. How do we support databases, such as MySQL, PGSQL, and h2 on the streampark platform? Do you have any relevant plans?

1.In the next pr, I'll provide a full catalog backend management implementation. And help coordinate the front-end resources.
2. As mysql I insert json value only use java string，It can supported pg and h2， I have compatibility it by design.

hi:
Why delete .idea/icon.png and idea/vcs.xml?

hi: Why delete .idea/icon.png and idea/vcs.xml?

When moving the module of the project, the load fails, delete the idea directory again and pour it in. Restores.idea/icon.png and idea/vcs.xml

authors: Mrart wolfboys


text:
Error: no such option: --app

When i followed the documentation https://superset.incubator.apache.org/installation.html to install the superset (Cloud-native! - Start with Docker)
in the third step docker-compose run --rm superset ./docker-init.sh
i am getting an error
WARNING: Image for service superset was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.

'[' 1 -ne 0 ']'
exec ./docker-init.sh
flask fab create-admin --app superset
Error: no such option: --app

I tried with multiple machines and the result was same. I cannot continue further due to this error.

under the dir incubator-superset/contrib/docker, follow these:

add this line  - ./docker-init.sh:/home/superset/docker-init.sh to docker-compose.yml
change lask fab create-admin --app superset to fabmanager create-admin --app superset in ./docker-init.sh
run docker-compose run --rm superset ./docker-init.sh again

Hi @vibeesh,
My bad, submitted a PR to fix it. Can you please change docker-init.sh by correcting flask fab create-admin --app superset to flask fab create-admin

Hi,
Thanks @dpgaspar for this fix.
Don't forget docker rmi superset_superset before the docker-compose run
But still, it pops an other error, I think easy to fix :
Usage: flask fab create-admin [OPTIONS]
Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.`

Hi,
Thanks @dpgaspar for this fix.
Don't forget docker rmi superset_superset before the docker-compose run
But still, it pops an other error, I think easy to fix :
Usage: flask fab create-admin [OPTIONS]

Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

Here the options for create-admin:
Usage: flask fab create-admin [OPTIONS]

  Creates an admin user

Options:
  --username TEXT
  --firstname TEXT
  --lastname TEXT
  --email TEXT
  --password TEXT
  --help            Show this message and exit.

It's not about the parameters, as the message says, it needs export FLASK_APP=superset before the create-admin to work.
So I did it, but it's still got an other error:
Traceback (most recent call last):
  File "/usr/local/bin/flask", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 894, in main
    cli.main(args=args, prog_name=name)
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 557, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1064, in invoke
    sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 621, in make_context
    self.parse_args(ctx, args)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 880, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1396, in handle_parse_result
    value = self.full_process_value(ctx, value)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1680, in full_process_value
    return self.prompt_for_value(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1655, in prompt_for_value
    value_proc=lambda x: self.process_value(ctx, x))
  File "/usr/local/lib/python3.6/site-packages/click/termui.py", line 89, in prompt
    value = prompt_func(prompt)
  File "/usr/local/lib/python3.6/site-packages/click/termui.py", line 73, in prompt_func
    return f('')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 4: invalid continuation byte

I've retried with ascii safe chars, got this message:
Usage: flask fab create-admin [OPTIONS]

Error: Failed to find Flask application or factory in module "superset". Use "FLASK_APP=superset:name to specify one.

I've tried with fabmanager create-admin --app superset and I got this error:
"There is no appbuilder var on your package, you can use appbuilder parameter to config"
(I got also this same error if I try on branch 0.30)

repo: apache/incubator-superset


text:
refactor: replace AvatarIcon instances with UserStack

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

unit tested
👀

ADDITIONAL INFORMATION

[x] Has associated issue: Fixes #11263
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #11279 into master will decrease coverage by 9.90%.
The diff coverage is 86.95%.

@@            Coverage Diff             @@
##           master   #11279      +/-   ##
==========================================
- Coverage   65.43%   55.52%   -9.91%
==========================================
  Files         831      404     -427
  Lines       39528    13402   -26126
  Branches     3598     3391     -207
==========================================
- Hits        25866     7442   -18424
+ Misses      13553     5771    -7782
- Partials      109      189      +80

Flag
Coverage Δ

#cypress
55.52% <86.95%> (-0.44%)
:arrow_down:

#javascript
?

#python
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...perset-frontend/src/views/CRUD/chart/ChartList.tsx
73.58% <ø> (-9.75%)
:arrow_down:

...ontend/src/views/CRUD/data/dataset/DatasetList.tsx
1.90% <0.00%> (-70.32%)
:arrow_down:

...rontend/src/views/CRUD/dashboard/DashboardList.tsx
68.93% <50.00%> (-7.69%)
:arrow_down:

...perset-frontend/src/components/UserStack/index.tsx
100.00% <100.00%> (ø)

...perset-frontend/src/components/UserStack/utils.tsx
100.00% <100.00%> (ø)

...uperset-frontend/src/dashboard/util/dnd-reorder.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...rset-frontend/src/dashboard/util/getEmptyLayout.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...rc/dashboard/util/getLayoutComponentFromChartId.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...dashboard/components/resizable/ResizableHandle.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

...d/src/views/CRUD/csstemplates/CssTemplatesList.tsx
3.57% <0.00%> (-93.73%)
:arrow_down:

... and 675 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 634676d...0f370a3. Read the comment docs.

NIT: tooltip-in-popover is a bit "yo dawg" :)

NIT: tooltip-in-popover is a bit "yo dawg" :)

Yea it's a little much visually. We could remove it (ie, check to see if the avatar is beyond the fold and not add the tooltip) but then we'd lose some information.

authors: codecov-io mistercrunch nytai


text:
[bugfix] Encode unicode error message response

When response error is unicode object, raise UnicodeEncodeError

Codecov Report

Merging #5687 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #5687      +/-   ##
==========================================
- Coverage   63.49%   63.48%   -0.02%
==========================================
  Files         360      360
  Lines       22914    22926      +12
  Branches     2551     2551
==========================================
+ Hits        14550    14554       +4
- Misses       8349     8357       +8
  Partials       15       15

Impacted Files
Coverage Δ

superset/views/base.py
68% <100%> (ø)
:arrow_up:

superset/views/datasource.py
95% <0%> (-1.43%)
:arrow_down:

superset/db_engine_specs.py
53.55% <0%> (-0.88%)
:arrow_down:

superset/connectors/sqla/models.py
77.9% <0%> (-0.77%)
:arrow_down:

superset/utils.py
89.67% <0%> (+0.54%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0fbda33...ad4c25f. Read the comment docs.

authors: codecov-io zhaoyongjie


text:
Add the exposed password checking function

Description

Add a builtin function called builtin-password-check.
A corresponding example is also added.

The purpose of this function is to demonstrate a usage scenario of Teaclave on threat intelligence.

CI passed: http://ci.mesalock-linux.org/mssun/incubator-mesatee/1103

repo: apache/incubator-teaclave


text:
TINKERPOP3-941: Improve error message for wrong order().by() arguments

https://issues.apache.org/jira/browse/TINKERPOP3-941
Instead of fixing the error message, I simply made it such that if you don't provide a comparator, it assumes you mean Order.incr. Added a test for both OrderLocalStep and OrderGlobalStep that demonstrate things working. Finally, I also realized we didn't have a proper clone() method for OrderLobalStep. Eek! Added it.
Ran mvn clean install and SparkGraphComputerIntegrationTest (for OrderTest only) and all is golden.
VOTE +1.

Tests all ran fine.  I did run into an issue trying the decr behavior.  Query with modern graph was:
gremlin> g.V().hasLabel("person").order().by("age", valueDecr)
java.lang.Integer cannot be cast to java.util.Map$Entry
Display stack trace? [yN] y
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.util.Map$Entry
	at org.apache.tinkerpop.gremlin.process.traversal.Order$6.compare(Order.java:83)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.ElementValueComparator.compare(ElementValueComparator.java:49)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.ElementValueComparator.compare(ElementValueComparator.java:29)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.OrderGlobalStep$ComparatorTraverser.compare(OrderGlobalStep.java:123)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.OrderGlobalStep$ComparatorTraverser.compare(OrderGlobalStep.java:113)
	at org.apache.tinkerpop.gremlin.util.function.ChainedComparator.compare(ChainedComparator.java:42)

It works if I do order(local) but I expected it to behave the same way your test for the incrementing version worked g.V().hasLabel("person").order().by("age")

Manually tested in OLTP and OLAP. I couldn't find any issues.
VOTE: +1

@twilmes  --- its by('age',decr). valueDecr is for Maps.

D'argh, party foul.  Got it, thanks.
VOTE +1

Like that you made it incr by default. VOTE: +1

authors: PommeVerte dkuppitz okram twilmes

repo: apache/incubator-tinkerpop


text:
[Feature] Support zstd compression

Motivation
ZSTD has a good tradeoff between compression ratio and de/compress speed. For reducing the shuffle-data stored size, it's necessary to support this compression algorithm.
How to do

Introduce the abstract interface of de/compress method
Support zstd

Good idea.

You can raise some other issues for other compression codec , such as snappy, as good first issues. Let more people can join our community more easily.

close by #254

authors: jerqi zuston

repo: apache/incubator-uniffle


text:
[WAYANG-1][WAYANG-2] Parent artifact and modification code structure

correction on the code structure and also generation of a parent module

I did the changes on the variables

Test are running locally

authors: berttty ro-pardo

repo: apache/incubator-wayang


text:
[weex playground][iOS][Dev-tool] 刷新页面 导致 调试工具闪退

终端报文
Your application is running here: http://xxxxx:8081.                                                                                NotAcceptableError: Not Acceptable
at C:\Users\zhipeng.tu\Desktop\weex-app\weex-app\node_modules\serve-index\index.js:162:32
at FSReqWrap.oncomplete (fs.js:141:20)
页面切换时 数据重载 也会导致同样的情况 是否是异步数据 导致 组件加载的时候未能取到，而崩溃？

image 动态设置宽高时，在ios  端 出现闪退现象
 <image class="godds-image" v-for="(item,index) in detailImageList"  :ref="'img'+ index" @load="onImageLoad(index,$event)" :src="item" ></image>

onImageLoad (index,event) {
      var that=this;
      if (event.success) {
        // Do something to hanlde success
        const view = that.$refs['img'+index];
        if (event.success){
            if (Utils.env.isWeb()){

                    view[0].style.width= '10rem';
                    view[0].style.height= 10*event.size.naturalHeight/event.size.naturalWidth +'rem';
                }else{
                    //  modal.toast({
                    //     message: '10rem',
                    //     duration: 0.3
                    // })
                animation.transition(view[0],{
                    styles:{
                    width: '750px',
                    height: `${750*event.size.naturalHeight/event.size.naturalWidth}px`
                    },
                    duration: 0,//需要设置为0，否则无效
                    timingFunction:'ease',
                    delay: 100,
                    needLayout:true,//变化后刷新界面
                    }, function(){
                        console.log(1)
                    })
            }
        }
    }
  },

开启 needLayout:true 直接闪退

这个是暂时前端的解决方案， 在onImageLoad（） 时，获取image的宽高，在@appear的时候，使用 animate 动态改变 宽高，刷新闪退的原因是在加载的时候，突然开启多个 animate 动画效果，可能导致内存溢出 ，直接闪退（猜测），所以才去异步改变的方式 来解决，解决闪退的问题
<image  v-for="(item,index) in detailImageList" @appear="ImageAppear(index)" class="xxxxtest"   ref="img" @load="onImageLoad(index,$event)" :src="item.url" ></image>

 ImageAppear(index){
          var _that=this;
           const view = this.$refs['img'];
           console.log(index)
         if (Utils.env.isWeb()){
              view[index].style.width= '10rem';
              view[index].style.height= _that.detailImageList[index].height;

         }else{
           animation.transition(view[index],{
                    styles:{
                    width: '750px',
                    height: _that.detailImageList[index].height
                    },
                    duration: 0,//需要设置为0，否则无效
                    timingFunction:'ease',
                    delay: 0,
                    needLayout:true,//变化后刷新界面
                    }, function(){
                        console.log(1)
            })
         };

        //    }
      },
      onImageLoad (index,event) {
      var that=this;
      if (event.success) {
        // Do something to hanlde success
        const view = that.$refs['img'];
        console.log(event)
        if (event.success){
            if (Utils.env.isWeb()){
                    that.detailImageList[index].width='10rem';
                    that.detailImageList[index].height= `${10*event.size.naturalHeight/event.size.naturalWidth}rem`;
                    // view[index].style.width= '10rem';
                    // view[index].style.height= 10*event.size.naturalHeight/event.size.naturalWidth +'rem';
                }else{
                that.detailImageList[index].width='750px';
                that.detailImageList[index].height= `${750*event.size.naturalHeight/event.size.naturalWidth}px`;
                // view[index].style.width= '750px';
                // view[index].style.height= 750*event.size.naturalHeight/event.size.naturalWidth +'px';
                // animation.transition(event.target,{
                //     styles:{
                //     width: '750px',
                //     height: `${750*event.size.naturalHeight/event.size.naturalWidth}px`
                //     },
                //     duration: 0,//需要设置为0，否则无效
                //     timingFunction:'ease',
                //     delay: 1,
                //     needLayout:true,//变化后刷新界面
                //     }, function(){
                //         console.log(1)
                //     })
            }
        }
    }
  },

@ZhiPengTu 我看您给这个 issue close了 请问您是怎么解决的

authors: ZhiPengTu yinxinchunios

repo: apache/incubator-weex


text:
weex何时才能完善好一统前端

Describe the bug

To Reproduce

Environment

Expected behavior

Screenshots

Additional context

Thanks for your advice, maybe you could join the mailing list and share your idea in the mailing list.


text:
Show line number overide the paragraph title

Show line number was calling commitParagraph with the wrong parameter as title which result in losing the title when hide\show numbers

Showing the numbers delete the title

LGTM!

LGTM!

authors: Leemoonsoo eranwitkon felixcheung

repo: apache/incubator-zeppelin


text:
[INLONG-8504][Manager] Modify nodeIp in dataproxy message to clientIp

Prepare a Pull Request

Fixes #8504

Motivation
Modify nodeIp in dataproxy message to clientIp.
Modifications
Modify nodeIp in dataproxy message to clientIp.

related to https://github.com/apache/inlong/pull/8508

repo: apache/inlong


text:
[Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB

Search before asking

[X] I searched in the issues and found nothing similar.

Version
1.3.1
Describe the bug and provide the minimal reproduce step
当我使用iotdb-session的Session范例的代码，连接上IoTDB之后，
使用IoTDB的stop-all.sh，start-all.sh的命令进行重启IoTDB，
那么，Session范例的代码，将永远无法连接上远程的IoTDB。
相关异常信息
Exception in thread "main" org.apache.iotdb.rpc.IoTDBConnectionException: Fail to reconnect to server. Please check server status.192.168.1.23:6667
at org.apache.iotdb.session.SessionConnection.(SessionConnection.java:132)
at org.apache.iotdb.session.Session.constructSessionConnection(Session.java:592)
at org.apache.iotdb.session.Session.open(Session.java:501)
at org.apache.iotdb.session.Session.open(Session.java:459)
一直到我使用start-cli.sh命令，连接过一次IoTDB后，Session范例的代码才能重新连接上IoTDB
What did you expect to see?
当IoTDB重启后，使用iotdb-session的JAVA客户端应用，能够重新连接上IoTDB
What did you see instead?
当IoTDB重启后，使用iotdb-session的JAVA客户端应用，就无法连接上IoTDB
Anything else?
No response
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

使用IoTDB的stop-all.sh，start-all.sh的命令进行重启IoTDB，
那么，Session范例的代码，将永远无法连接上远程的IoTDB。
重启之后看下对应的 log_confignode_all.log 和 log_datanode.all.log

清理日志文件，重新复现问题后，
在这两个日志文件中，没有看到异常和出错信息
附件中，是日志文件
------------------ 原始邮件 ------------------
发件人:                                                                                "apache/iotdb"                                                                                @.>;
发送时间: 2024年5月10日(星期五) 中午12:14
@.>;
@.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
使用IoTDB的stop-all.sh，start-all.sh的命令进行重启IoTDB，
那么，Session范例的代码，将永远无法连接上远程的IoTDB。
重启之后看下对应的 log_confignode_all.log 和 log_datanode.all.log
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

直接 运行  ./sbin/start-confignode.sh 看下

这是问题出现，到启动start-cli.sh后，客户端java代码重新连接正常的日志
------------------ 原始邮件 ------------------
发件人:                                                                                "江南"                                                                                @.>;
发送时间: 2024年5月10日(星期五) 中午12:24
@.@.>;
@.>;
主题: 回复： [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
清理日志文件，重新复现问题后，
在这两个日志文件中，没有看到异常和出错信息
附件中，是日志文件
------------------ 原始邮件 ------------------
发件人:                                                                                "apache/iotdb"                                                                                @.>;
发送时间: 2024年5月10日(星期五) 中午12:14
@.>;
@.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
使用IoTDB的stop-all.sh，start-all.sh的命令进行重启IoTDB，
那么，Session范例的代码，将永远无法连接上远程的IoTDB。
重启之后看下对应的 log_confignode_all.log 和 log_datanode.all.log
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

启动不是立刻成功的，可以观察 datanode 的日志启动成功后再使用 session 去连接

运行./sbin/start-confignode.sh后，故障现象依然存在
------------------ 原始邮件 ------------------
发件人:                                                                                "apache/iotdb"                                                                                @.>;
发送时间: 2024年5月10日(星期五) 中午12:26
@.>;
@.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
直接 运行  ./sbin/start-confignode.sh 看下
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

真正能够连接上去，必须要运行过一次start-cli.sh后，才能连接成功
---原始邮件---
发件人: @.>
发送时间: 2024年5月10日(周五) 中午12:26
收件人: @.>;
抄送: @.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
直接 运行  ./sbin/start-confignode.sh 看下
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

这个前面怀疑过，实际上等待比较长的时间后，依然连接不上
------------------ 原始邮件 ------------------
发件人:                                                                                "apache/iotdb"                                                                                @.>;
发送时间: 2024年5月10日(星期五) 中午12:29
@.>;
@.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
启动不是立刻成功的，可以观察 datanode 的日志启动成功后再使用 session 去连接
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

这个现象从来没见过。
启动后，session 连接不上时 不先用 start-cli.sh 先看看日志，看看 6667 的端口占用情况。

我的运行环境
UBUNTU20.0
openjdk 11
友善的r5c设备 CPU是RK3568的ARM嵌入式设备
---原始邮件---
发件人: @.>
发送时间: 2024年5月10日(周五) 中午12:37
收件人: @.>;
抄送: @.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
这个现象从来没见过。
启动后，session 连接不上时 不先用 start-cli.sh 先看看日志，看看 6667 的端口占用情况。
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

我下午复现问题后，再看一下端口占用状况
---原始邮件---
发件人: @.>
发送时间: 2024年5月10日(周五) 中午12:37
收件人: @.>;
抄送: @.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
这个现象从来没见过。
启动后，session 连接不上时 不先用 start-cli.sh 先看看日志，看看 6667 的端口占用情况。
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

我这个设备环境，同时具备IPV6和IPV4
比如，nginx，同时占用ipv6和ipv4的80端口
但是，iotdb，只绑定了ipv6的6667端口，并不是ipv4
------------------ 原始邮件 ------------------
发件人:                                                                                "apache/iotdb"                                                                                @.>;
发送时间: 2024年5月10日(星期五) 中午12:37
@.>;
@.@.>;
主题: Re: [apache/iotdb] [Bug] 当IoTDB重启的时候，Session范例的连接IoTDB的代码，将无法再远程连接上IoTDB (Issue #12498)
这个现象从来没见过。
启动后，session 连接不上时 不先用 start-cli.sh 先看看日志，看看 6667 的端口占用情况。
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

@hupengwu 这个有新的进展吗

authors: CritasWang hupengwu

repo: apache/iotdb


text:
Fix session ci bug(#3343)

Description
Content1 ...
Content2 ...
Content3 ...

This PR has:

[ ] been self-reviewed.

[ ] concurrent read
[ ] concurrent write
[ ] concurrent read and write

[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods.
[ ] added or updated version, license, or notice information
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious
for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold
for code coverage.
[ ] added integration tests.
[ ] been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

duplicate with #3300


text:
MINOR: update doc for default assignor change

Update the doc and upgrade doc for default assignor change.
REF: https://github.com/apache/kafka/pull/10903
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@ableegoldman , I found I forgot to update the doc to reflect the default assignor change. Please take a look. Thanks.

Previous build has one test group timeout. Re-trigger it again.

Seeing as these are just docs changes and at least one build passed completely while others just had a handful of flaky tests fail, I think we can go ahead and merge this

Merged to trunk and cherrypicked to 3.0

authors: ableegoldman showuon

repo: apache/kafka


text:
KAFKA-17613: Remove ZK migration code

Remove the controller machinery for doing ZK migration in Kafka 4.0.
As a note, I've taken the approach of leaving KafkaConfig.migrationEnabled but hard-coding it to
false here. That was because I didn't want to mess around with KafkaServer.scala and related classes.
Since they're getting removed soon anyway, it would be a waste.

I've taken the approach of leaving KafkaConfig.migrationEnabled but hard-coding it to false here. That was because I didn't want to mess around with KafkaServer.scala and related classes. Since they're getting removed soon anyway, it would be a waste.

There are some test failures. I think they'll be fixed after we merge #17284

@cmccabe the other removal PR is merged now, can you merge in trunk?

I've updated this to remove the lingering reference to dual write offset in the metrics class.

I believe those failing tests can be removed if they are specifically designed to run with ZooKeeper migration

@cmccabe Could you please fix the conflicts? the failed tests were removed by #17318

authors: chia7712 cmccabe mumrah


text:
Rename streams tutorial and quickstart

Changed these topic titles:

Write your own Streams Applications -> Tutorial: Write a Streams Application
Play with a Streams Application -> Run the Streams Demo Application

ping @miguno

ping @guozhangwang

LGTM. cc @miguno

Merged to trunk, and cherry-picked to 1.0

authors: guozhangwang joel-hamill


text:
KAFKA-4651: improve test coverage of stores

Working on increasing the coverage of stores in unit tests.
Started with InMemoryKeyValueLoggedStore

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@dguy @guozhangwang @mjsax
Note I did not add tests for deprecated classes.

coverage improvements coming.

updated this

updated per comments, Note that most of the added tests already existed in the parent class, so redundant tests removed.

LGTM!

authors: bbejeck guozhangwang


text:
[KYUUBI #4745] Support Flink's LocalZonedTimestamp DataType

Why are the changes needed?

How was this patch tested?

[X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[X] Run test locally before make a pull request

Codecov Report

Merging #4751 (5f9976d) into master (f6331a2) will decrease coverage by 0.03%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #4751      +/-   ##
============================================
- Coverage     58.07%   58.04%   -0.03%
  Complexity       13       13
============================================
  Files           581      581
  Lines         32297    32325      +28
  Branches       4313     4308       -5
============================================
+ Hits          18755    18762       +7
- Misses        11745    11758      +13
- Partials       1797     1805       +8

see 22 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pan3793 I started a thread to discuss the timestamp format issue https://www.mail-archive.com/dev@flink.apache.org/msg66257.html. But I don't think it's a blocker for this PR.

Thanks, merged to master

repo: apache/kyuubi


text:
[MS25 & MS26] Unified date & time formatting

In #3121, date & time formatting of both Pattern Layout and JSON Template Layout has been rewritten to be served from a unified utility using Java's DateTimeFormatter under the hood with some caching for extra efficiency.
Historical account
MS25, MS26, MS27 had the following sequence of goals:

[MS25] Move the InstantFormatter (used for formatting log event instants) of log4j-layout-template-json to log4j-core
[MS26] Use the InstantFormatter moved to log4j-core in both JSON Template Layout and Pattern Layout
[MS27] Use the InstantFormatter moved to log4j-core in other places where date & time formatting is needed

We encountered several blockers while executing this plan. Some highlights from the thinking process that led us to the current plan are shared below – see this dev@ discussion for details.

Log4j contains two custom date & time formatting utilities for performance reasons: FixedDateFormat and FastDateFormat.
InstantFormatter leverages these two for date & time formatters for patterns they support, otherwise it falls back to DateTimeFormatter.
Once InstantFormatter is moved to log4j-core and wired to Pattern Layout, several tests started failing, because previously employed FixedDateFormat is

incorrectly handling n and x directives
incorrectly calculating DST (#2943)

FastDateFormat was copied from Commons Lang in 2015
No other logging frameworks (Logback, Tinylog, etc.) had similar date & time formatting optimizations

In short, FixedDateFormat and FastDateFormat are liabilities (containing incorrect behaviour and bugs!) that once made sense for performance reasons. Though as shown with performance figures in #3121, the efficiency offered by this liability burden is negligible in the big picture of an end-to-end logging scenario. Eventually, we decided to

Use Java's DateTimeFormatter for date & time formatting
Introduce caching for extra efficiency (e.g., if user needs to format log events in 2024-10-29T14:49:53.997Z form, we only need to generate 53.997 each time and the rest can be precomputed per minute, cached, and reused)
Deprecate all legacy classes, i.e., FixedDateFormat and FastDateFormat et al.
Thoroughly test the new instant formatters to avoid issues like #2943

Log4j 3
#3150 ports #3121 from 2.x to main, the Log4j 3 branch.

Thanks, looks good to me!

authors: grobmeier vy


text:
org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom fails reproducibly

Description
Please see https://lists.apache.org/thread/t5hgzv0q177nf8qqyz6h9qhhvwp366gp or https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/8706 for full details.
Error Message:
java.lang.IndexOutOfBoundsException: Range [0, 0 + 813) out of bounds for length 80

Stack Trace:
java.lang.IndexOutOfBoundsException: Range [0, 0 + 813) out of bounds for length 80
	at __randomizedtesting.SeedInfo.seed([173F7C2A08C39662:65735925B9A32011]:0)
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromIndexSize(Preconditions.java:82)
	at java.base/jdk.internal.util.Preconditions.checkFromIndexSize(Preconditions.java:361)
	at java.base/java.util.Objects.checkFromIndexSize(Objects.java:411)
	at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:180)
	at org.apache.lucene.core@9.6.0-SNAPSHOT/org.apache.lucene.store.ByteBuffersDataInput.readBytes(ByteBuffersDataInput.java:155)
	at org.apache.lucene.core@9.6.0-SNAPSHOT/org.apache.lucene.store.ByteBuffersIndexInput.readBytes(ByteBuffersIndexInput.java:85)
	at org.apache.lucene.test_framework@9.6.0-SNAPSHOT/org.apache.lucene.tests.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:148)
	at org.apache.lucene.core@9.6.0-SNAPSHOT/org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.decompressBlock(Lucene90DocValuesProducer.java:1235)
	at org.apache.lucene.core@9.6.0-SNAPSHOT/org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.next(Lucene90DocValuesProducer.java:1093)
	at org.apache.lucene.search.grouping.TermGroupFacetCollector$MV$SegmentResult.nextTerm(TermGroupFacetCollector.java:437)
	at org.apache.lucene.search.grouping.GroupFacetCollector.mergeSegmentResults(GroupFacetCollector.java:97)
	at org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom(TestGroupFacetCollector.java:429)
...

Gradle command to reproduce
./gradlew test --tests TestGroupFacetCollector.testRandom -Dtests.seed=173F7C2A08C39662 -Dtests.multiplier=3 -Dtests.locale=ru-KZ -Dtests.timezone=America/Mazatlan -Dtests.asserts=true -Dtests.file.encoding=UTF-8

Well, this has been one hell of a treasure hunt.
Based on the stack trace, the addressable problem starts here: https://github.com/apache/lucene/blob/caeabf39309a91997d361b4104bda105d16ae720/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java#L1235
After adding a bunch of log statements and blowing up the GC..
What are learned from the logs and is evident in the error message, that we are trying to read term bytes longer than the array.length. At first glance this does not seem possible. Could there be some data corruption issue in input.readBytes(term.bytes, prefixLength, suffixLength); from
https://github.com/apache/lucene/blob/caeabf39309a91997d361b4104bda105d16ae720/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java#L1105
I'm not sure what could be going on but it seems potentially like a serious issue. Have we previously observed data corruption issues on input streams? I'm really stumped as to where the 813 (or any number > 80 is coming from). If it's important or helpful, I could look into it more.

Looks like this is tracking the same issue I opened back in #11631. I'm going to close out my older issue in favor of this, since it has more detail and is being worked. Hooray!

Fixed by #12555, thanks @epotyom !

authors: MarcusSorealheis cpoerschke gsmiller zhaih

repo: apache/lucene


text:
Speed up PostingsEnum when reading positions.

This PR changes the following:

As much work as possible is moved from nextDoc()/advance() to nextPosition(). This helps only pay the overhead of reading positions when all query terms agree on a candidate.
Frequencies are read lazily. Again, this helps in case a document is needed in a block, but clauses do not agree on a common candidate match, so frequencies are never decoded.
A few other minor optimizations.

                        TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                      IntNRQ      110.78     (11.4%)      108.66     (13.0%)   -1.9% ( -23% -   25%) 0.739
                 CountPhrase        4.37      (1.6%)        4.30      (1.2%)   -1.7% (  -4% -    1%) 0.009
                      OrMany       20.20      (3.6%)       19.88      (4.2%)   -1.6% (  -9% -    6%) 0.381
             DismaxOrHighMed       85.25      (1.7%)       84.24      (3.9%)   -1.2% (  -6% -    4%) 0.403
            DismaxOrHighHigh       69.09      (2.0%)       68.36      (3.6%)   -1.1% (  -6% -    4%) 0.435
                    SpanNear        2.17     (12.4%)        2.14     (12.0%)   -1.0% ( -22% -   26%) 0.855
                   CountTerm     8796.37      (5.0%)     8709.80      (2.8%)   -1.0% (  -8% -    7%) 0.605
                 OrStopWords       34.15      (8.2%)       33.89      (5.8%)   -0.8% ( -13% -   14%) 0.820
           FilteredAnd3Terms      196.11      (2.3%)      194.82      (1.9%)   -0.7% (  -4% -    3%) 0.503
                    Or3Terms      175.20      (5.2%)      174.24      (3.8%)   -0.6% (  -9% -    8%) 0.797
              FilteredOrMany       16.97      (5.4%)       16.88      (5.0%)   -0.5% ( -10% -   10%) 0.834
          Or2Terms2StopWords      165.63      (4.6%)      164.83      (3.5%)   -0.5% (  -8% -    7%) 0.801
                      Fuzzy1       81.21      (1.6%)       80.83      (1.5%)   -0.5% (  -3% -    2%) 0.529
                SloppyPhrase        1.78     (10.5%)        1.77      (3.6%)   -0.4% ( -13% -   15%) 0.907
 FilteredAnd2Terms2StopWords      201.36      (1.9%)      200.55      (1.3%)   -0.4% (  -3% -    2%) 0.599
          FilteredAndHighMed      130.30      (2.7%)      129.81      (2.6%)   -0.4% (  -5% -    5%) 0.764
                      Fuzzy2       76.58      (1.4%)       76.29      (1.2%)   -0.4% (  -2% -    2%) 0.536
                FilteredTerm      156.92      (2.5%)      156.39      (2.1%)   -0.3% (  -4% -    4%) 0.757
                   And3Terms      179.52      (3.2%)      179.00      (3.5%)   -0.3% (  -6% -    6%) 0.855
         FilteredAndHighHigh       63.65      (2.9%)       63.47      (2.0%)   -0.3% (  -5% -    4%) 0.813
        FilteredAndStopWords       48.99      (2.9%)       48.86      (1.9%)   -0.3% (  -4% -    4%) 0.828
         And2Terms2StopWords      167.29      (3.1%)      166.94      (3.3%)   -0.2% (  -6% -    6%) 0.887
           TermDayOfYearSort      631.10      (1.3%)      630.31      (1.2%)   -0.1% (  -2% -    2%) 0.835
            FilteredOr3Terms      168.75      (1.2%)      168.59      (1.4%)   -0.1% (  -2% -    2%) 0.879
             CountAndHighMed      169.49      (1.2%)      169.42      (1.3%)   -0.0% (  -2% -    2%) 0.948
  FilteredOr2Terms2StopWords      151.07      (1.0%)      151.02      (1.2%)   -0.0% (  -2% -    2%) 0.948
                AndStopWords       32.08      (5.1%)       32.07      (5.0%)   -0.0% (  -9% -   10%) 0.993
           FilteredOrHighMed      157.50      (1.1%)      157.49      (0.6%)   -0.0% (  -1% -    1%) 0.992
            CountAndHighHigh       57.71      (1.1%)       57.71      (0.8%)    0.0% (  -1% -    1%) 1.000
                    PKLookup      282.75      (1.9%)      282.86      (1.4%)    0.0% (  -3% -    3%) 0.958
                     Respell       54.63      (1.1%)       54.70      (1.5%)    0.1% (  -2% -    2%) 0.843
                TermGroup100       24.50      (4.5%)       24.54      (3.8%)    0.2% (  -7% -    8%) 0.929
              CountOrHighMed      141.72      (1.9%)      142.10      (1.1%)    0.3% (  -2% -    3%) 0.717
         FilteredOrStopWords       44.42      (1.9%)       44.56      (2.3%)    0.3% (  -3% -    4%) 0.759
             AndHighOrMedMed       43.17      (2.5%)       43.31      (0.7%)    0.3% (  -2% -    3%) 0.720
             CountOrHighHigh       74.88      (1.6%)       75.12      (1.3%)    0.3% (  -2% -    3%) 0.653
           CombinedOrHighMed       70.87      (5.8%)       71.13      (3.7%)    0.4% (  -8% -   10%) 0.873
               TermMonthSort     3404.71      (2.0%)     3418.15      (2.6%)    0.4% (  -4% -    5%) 0.718
          CombinedOrHighHigh       18.54      (5.9%)       18.64      (3.7%)    0.5% (  -8% -   10%) 0.821
               TermTitleSort      159.32      (3.5%)      160.30      (2.9%)    0.6% (  -5% -    7%) 0.686
                   OrHighMed      196.12      (5.6%)      197.40      (4.9%)    0.7% (  -9% -   11%) 0.794
            IntervalsOrdered        2.37      (2.7%)        2.38      (3.2%)    0.7% (  -5% -    6%) 0.627
          FilteredOrHighHigh       65.75      (1.7%)       66.20      (1.5%)    0.7% (  -2% -    3%) 0.365
                TermGroup10K       20.11      (3.6%)       20.25      (3.5%)    0.7% (  -6% -    8%) 0.682
                  DismaxTerm      605.77      (2.9%)      610.10      (2.8%)    0.7% (  -4% -    6%) 0.598
                 TermGroup1M       19.82      (3.1%)       19.97      (3.5%)    0.8% (  -5% -    7%) 0.616
            AndMedOrHighHigh       59.49      (1.9%)       59.96      (1.6%)    0.8% (  -2% -    4%) 0.351
                  TermDTSort      274.15      (1.8%)      276.92      (2.3%)    1.0% (  -3% -    5%) 0.309
                    Wildcard       73.73      (4.2%)       74.49      (3.0%)    1.0% (  -5% -    8%) 0.545
          CombinedAndHighMed       54.61      (4.7%)       55.18      (2.5%)    1.0% (  -5% -    8%) 0.558
                  AndHighMed      131.04      (2.8%)      132.48      (1.7%)    1.1% (  -3% -    5%) 0.308
                  OrHighHigh       52.99      (7.0%)       53.63      (6.2%)    1.2% ( -11% -   15%) 0.697
                        Term      470.85      (4.2%)      476.71      (4.3%)    1.2% (  -7% -   10%) 0.537
         CombinedAndHighHigh       14.94      (5.0%)       15.13      (2.5%)    1.2% (  -5% -    9%) 0.505
                  OrHighRare      271.47      (8.0%)      275.00      (7.1%)    1.3% ( -12% -   17%) 0.715
                CombinedTerm       29.64      (5.1%)       30.08      (3.8%)    1.5% (  -7% -   10%) 0.480
                TermBGroup1M       24.87      (3.9%)       25.26      (3.8%)    1.6% (  -5% -    9%) 0.386
                 AndHighHigh       44.95      (3.0%)       45.68      (1.7%)    1.6% (  -3% -    6%) 0.158
                     Prefix3      127.48      (6.7%)      129.68      (7.8%)    1.7% ( -11% -   17%) 0.615
              TermBGroup1M1P       36.78      (4.9%)       37.59      (5.4%)    2.2% (  -7% -   13%) 0.364
                      Phrase       13.78      (2.7%)       14.90      (2.8%)    8.1% (   2% -   13%) 0.000
              FilteredPhrase       26.01      (3.4%)       30.92      (1.6%)   18.9% (  13% -   24%) 0.000

FilteredPhrase had a good 25% speedup, Phrase had a good 11% speedup, but other phrase queries had a slowdown, e.g. SloppyPhrase (-8%) and CountPhrase (-7%).


text:
[MENFORCER-390] "requireFilesExist" no longer handles non-canonical paths

This reverts the change from MENFORCER-364 as this led to a regressions with symbolic links.
The fundamental issue is that there is no clean way to deal with case-sensitivity as OSs can have multuple filesystems mounted that follow different rules. Thus the simple file.exists() is, despite the limitations, probably best. Those requiring more stringent checks writing their own handling.
Following this checklist to help us incorporate your
contribution quickly and easily:

[x] Make sure there is a JIRA issue filed
for the change (usually before you start working on it).  Trivial changes like typos do not
require a JIRA issue.  Your pull request should address just this issue, without
pulling in other changes.
[x] Each commit in the pull request should have a meaningful subject line and body.
[x] Format the pull request title like [MENFORCER-XXX] - Fixes bug in ApproximateQuantiles,
where you replace MENFORCER-XXX with the appropriate JIRA issue. Best practice
is to use the JIRA issue title in the pull request title and in the first line of the
commit message.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Run mvn clean verify to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
[x] You have run the integration tests successfully (mvn -Prun-its clean verify).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

[x] I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

[ ] In any other case, please file an Apache Individual Contributor License Agreement.

Ok, the simple file.exists() should be enough in normal usage ...
I only think - if we don't break and special cases ... but agree more complicated check should be done optionally or by next rule
By the way we can mention in documentation a way how file existence is checked to be clear.

No, they are not intended. That will be my IDE doing its own thing, I'll
use a different editor and update.
On Mon, 27 Nov 2023 at 15:20, Torbjorn-Svensson @.***>
wrote:

Are all the white space changes in the .md files intended?
As far as I know, some of the trailing white spaces are needed in order to
not join the lines.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/maven-enforcer/pull/297#issuecomment-1828041274,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJHMQU65AH2DOVYZVDQKTUTYGSVUVAVCNFSM6AAAAAA7ZJWO7OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRYGA2DCMRXGQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

That should be the whitepsace issues sorted. I also generated the site locally and it appeared OK to me.

authors: roadSurfer slawekjaranowski

repo: apache/maven-enforcer


text:
Added Python 3.6 and pip to Docker ARM image.

Following the update of the CLI to Python 3, we embed Python 3.6 (the
minimum required Python version) into the Docker images used during
continuous integration.
Refs: cbdb6a5e1c0bb2e61dd0887cfae424ac2422b94d
Fixes: MESOS-9229

@bbannier @bmahler Can you take a look?

@bbannier Are you sure?

Yes, pretty sure, see https://github.com/apache/mesos/blob/0fa42f82770a4afca1908449f0cfd41e5383988e/support/mesos-build/ubuntu-16.04-arm.dockerfile#L42-L55

Merged: https://github.com/apache/mesos/commit/0fa42f82770a4afca1908449f0cfd41e5383988e

authors: bbannier janisz

repo: apache/mesos


text:
NIFI-9600 Remove Elasticsearch 2 Processors

Description of PR
NIFI-9600 Removes the FetchElasticsearch and PutElasticsearch processors based on the Elasticsearch 2 library and the deprecated Transport Client.  The last Elasticsearch 2 version reached end of life in 2018.  Removing these processors eliminates the legacy Elasticsearch 2 dependency and related libraries in nifi-elasticsearch-processors. Additional changes include removing integration test classes and ignored test methods for the other deprecated processors that remain in the nifi-elasticsearch-processors module.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[X] Has your PR been rebased against the latest commit within the target branch (typically main)?

[X] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[ ] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

All makes sense to me @gresockj, good to continue tidying up the Elasticsearch processors, thanks @exceptionfactory 👍

Sounds good, I'll go ahead and merge!

authors: ChrisSamo632 exceptionfactory gresockj

repo: apache/nifi


text:
Proposed improvements to the sensor framwork

I've spent some time playing with the new sensor framework, and I'm much less skeptical about this approach (previous discussion here https://github.com/apache/nuttx/pull/10077). It seems to me that with some changes, it can be a good universal solution. Unfortunately, the lack of documentation causes some misunderstanding. Most information about framework can be found in the discussion under https://github.com/apache/nuttx/pull/2039 and https://github.com/apache/nuttx/pull/2215. Somehow I missed this before :)
Some ideas for improvement from my side:

We should stop referring to this sensor framework as uorb. uorb is optional and is part of the apps. Referring to uorb on the kernel side gives the false impression that uorb is required.

Make sensor register path configurable. Now it is /dev/uorb (changed in  c4bed9eae9037297b3f8de6a3547e8ca6361b933) but as uorb is application-specific property, this should be configurable.

Add options to disable some framework functionality. This can save memory on small systems. For example, disabling polling logic and relying only on the fetch interface saves some space and gives the user full control over sensor sampling. Another thing is timestamp; in many cases, timestamp doesn't matter for user.

The biggest problem I see now is forcing the user to use float. This is not the best solution for applications without FPU (e.g. CM0 which are often used in sensor nodes). What if we made the sensor data type configurable? It would be nice if we had an interface that return data in RAW format, but this may unnecessarily complicate the framework. Support for fixedmath type seems to be a good compromise (b16_t ?).

If there is consent to these changes, I can take care of them.

I've spent some time playing with the new sensor framework, and I'm much less skeptical about this approach (previous discussion here #10077). It seems to me that with some changes, it can be a good universal solution. Unfortunately, the lack of documentation causes some misunderstanding. Most information about framework can be found in the discussion under #2039 and #2215. Somehow I missed this before :)

@Donny9 has a representation about this new driver framework on NuttX workshop 2022:
https://www.youtube.com/watch?v=ESpAE6wqy9o

Some ideas for improvement from my side:

We should stop referring to this sensor framework as uorb. uorb is optional and is part of the apps. Referring to uorb on the kernel side gives the false impression that uorb is required.

Yes, uorb is a simple and optional wrapper on top of sensor ioctl, but we need find a good name since sensor isn't a good name too:

The advanced algorithm(eg. step counter, heart rate) can be built with this framework.
Other notification can publish with this framework(eg. BT/WiFi connection state and signal strength)

Make sensor register path configurable. Now it is /dev/uorb (changed in  c4bed9e) but as uorb is application-specific property, this should be configurable.
Add options to disable some framework functionality. This can save memory on small systems. For example, disabling polling logic and relying only on the fetch interface saves some space and gives the user full control over sensor sampling.

Each request is reasonable.

Another thing is timestamp; in many cases, timestamp doesn't matter for user.

It depends on the sensor type and application. If the timestamp isn't really used, driver could skip fill the timestamp value.

The biggest problem I see now is forcing the user to use float. This is not the best solution for applications without FPU (e.g. CM0 which are often used in sensor nodes). What if we made the sensor data type configurable? It would be nice if we had an interface that return data in RAW format, but this may unnecessarily complicate the framework.

Since hardwire normally report the measurement with the different unit, raw format makes us can't write the general application.

Support for fixedmath type seems to be a good compromise (b16_t ?).

Yes, fixedmath may a good option if the hardware doesn't have FPU.

@Donny9 could you review the suggestion?

”uorb“ is borrow from PX4，which is really a event-driven middle layer for inter-thread communication. And "uorb" is inspired by ROS (but with limited functionality and flexibility). If nuttx has it own middle layer say "nuttx-ROS" that would be great.
ROS2 is claimed to support bare metal, but progress seems slow

the built in uorb is compatible with PX4, but the performance and size will be much better.

repo: apache/nuttx


text:
Fix Kconfig style

Summary
Remove spaces from Kconfig files
Add TABs
Replace help => ---help---
Add comments
Impact
none
Testing

@xiaoxiang781216 any idea why these tests are failing:
=========================== short test summary info ============================
FAILED test_os/test_os.py::test_ostest - assert -2 == 0
FAILED test_os/test_os.py::test_mm - AttributeError: type object 'EOF' has no...
FAILED test_os/test_os.py::test_cxxtest - AttributeError: type object 'EOF' h...
FAILED test_os/test_os.py::test_scanftest - AttributeError: type object 'EOF'...
FAILED test_os/test_os.py::test_getprime - AttributeError: type object 'EOF' ...
FAILED test_os/test_os.py::test_stdio - AttributeError: type object 'EOF' has...
FAILED test_os/test_os.py::test_fs_test - AttributeError: type object 'EOF' h...
FAILED test_os/test_os.py::test_psram_test - AttributeError: type object 'EOF...
ERROR test_os/test_os.py::test_ostest - AttributeError: type object 'EOF' has...
ERROR test_os/test_os.py::test_mm - AttributeError: type object 'EOF' has no ...
ERROR test_os/test_os.py::test_cxxtest - AttributeError: type object 'EOF' ha...
ERROR test_os/test_os.py::test_scanftest - AttributeError: type object 'EOF' ...
ERROR test_os/test_os.py::test_getprime - AttributeError: type object 'EOF' h...
ERROR test_os/test_os.py::test_stdio - AttributeError: type object 'EOF' has ...
ERROR test_os/test_os.py::test_fs_test - AttributeError: type object 'EOF' ha...
ERROR test_os/test_os.py::test_psram_test - AttributeError: type object 'EOF'...
====== 8 failed, 1050 passed, 10 skipped, 8 errors in 2791.38s (0:46:31) =======

@ttnie is investigating the root cause.