text:
[Security] Fix downloading SNAPSHOTs from ASF via https

There was just an incoming security mail ....
I'm not sure if we need the other repository as it's http only as well.
Fixes:
CWE-829: Inclusion of Functionality from Untrusted Control Sphere
CWE-494: Download of Code Without Integrity Check

@chrismattmann can you merge / this PR relates to the mail to private mailing list. Thanks

thanks @ottlinger !

@chrismattmann unfortunately the merge was overwritten by a different merge and the securit yproblem still remains in HEAD. I'll recreate a new pull request for that.

authors: chrismattmann ottlinger

repo: apache/drat


text:
DRILL-5293: Change the seed of the hash function for distribution

The fix is to use a different seed for the hash function in the cases the hash is computed for distribution (i.e., when calling the method getHashExpression(List<DistributionField> fields, RelDataType rowType) ).
The implementation -- added a third parameter seed to createCall() (on line 102 in HashPrelUtil.java), which makes it use the "with seed" versions of the generated code functions (before that it was using the versions that had a built-in seed of zero).
Then made all the calling places pass in a seed -- zero in the case of a Hash Table , or a large fixed prime ( 1301011 ) for distribution.

@kkhatua Actually any number (other than zero) would work here and completely change the hash result; I just went a little extra by picking a large prime.

+1.   Pls squash commits before it is merged.

source: gharchive/pull-request

repo: apache/drill

license_type: permissive

license_source: github-api


text:
SQL: Support "subtotalsSpec"

Description
subtotalsSpec which is closest to GROUPING SET makes it far more efficient to create multiple groups that without it, would need to be done in multiple queries.

I think this is still relevant.

Fixed in https://github.com/apache/druid/pull/9122.

source: gharchive/issue

authors: jihoonson nrwiersma vogievetsky

repo: apache/druid


text:
fix bug: failed to return status

Fixes #16641  .

Description

Fixed the bug ...
Renamed the class ...
Added a forbidden-apis entry ...

Release note

Key changed/added classes in this PR

org.apache.druid.segment.realtime.firehose.ChatHandlerResource
org.apache.druid.segment.realtime.firehose.ChatHandler

This PR has:

[x] been self-reviewed.

[x] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[x] added integration tests.
[x] been tested in a test Druid cluster.

even

Looks the issue is already fixed. Thanks.


text:
[DUBBO-ADMIN-198]: Support multiple registry centers

resolves #198

Hi @nzomkxia , according what you said in comment, I've just updated this PR, I've tested it like this:

modify the dubbo-provider.xml in dubbo-demo as follows to register services in multiple registry centers:

    <!-- provider's application name, used for tracing dependency relationship -->
    <dubbo:application name="meetup-demo-provider"/>

    <dubbo:registry address="zookeeper://127.0.0.1:2181" id="reg1"/>
    <dubbo:registry address="zookeeper://127.0.0.1:2182" id="reg2"/>

    <!-- use dubbo protocol to export service on port 20880 -->
    <dubbo:protocol name="dubbo" port="20880"/>

    <!-- service implementation, as same as regular local bean -->
    <!--<bean id="demoService" class="org.apache.dubbo.demo.impl.DemoServiceImpl"/>-->
    <!--<bean id="userService" class="org.apache.dubbo.demo.impl.UserServiceImpl"/>-->
    <!--<bean id="timeoutTestService" class="org.apache.dubbo.demo.impl.TimeoutTestServiceImpl"/>-->
    <!--<bean id="noResponseService" class="org.apache.dubbo.demo.impl.NoResponseServiceImpl"/>-->

    <!-- declare the service interface to be exported -->
    <dubbo:service registry="reg1" interface="org.apache.dubbo.demo.api.DemoService" ref="demoService"/>
    <dubbo:service registry="reg1" interface="org.apache.dubbo.demo.api.UserService" ref="userService"/>
    <dubbo:service registry="reg1" interface="org.apache.dubbo.demo.api.TimeoutTestService" ref="timeoutTestService"/>
    <dubbo:service registry="reg2" interface="org.apache.dubbo.demo.api.NoResponseService" ref="noResponseService"/>
    <dubbo:service registry="reg2" interface="org.apache.dubbo.demo.api.NoResponse2Service" ref="noResponse2Service"/>

and start the providers

open the OPS and search the services, all the services in different registry center are shown, here is the screenshot:

I'm wondering if I misunderstood your saying, please let me know if I missed anything, thanks

Codecov Report

Merging #314 into develop will increase coverage by 0.64%.
The diff coverage is 54.67%.

@@              Coverage Diff              @@
##             develop     #314      +/-   ##
=============================================
+ Coverage      19.53%   20.17%   +0.64%
- Complexity       289      306      +17
=============================================
  Files             95       96       +1
  Lines           3589     3663      +74
  Branches         555      575      +20
=============================================
+ Hits             701      739      +38
- Misses          2813     2848      +35
- Partials          75       76       +1

Impacted Files
Coverage Δ
Complexity Δ

...ache/dubbo/admin/service/impl/AbstractService.java
66.66% <ø> (ø)
2 <0> (ø)
:arrow_down:

.../dubbo/admin/service/impl/OverrideServiceImpl.java
0.54% <0%> (-0.04%)
1 <0> (ø)

.../dubbo/admin/service/impl/ProviderServiceImpl.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...e/dubbo/admin/service/impl/GenericServiceImpl.java
57.14% <100%> (+3.29%)
2 <0> (ø)
:arrow_down:

...apache/dubbo/admin/service/RegistryServerSync.java
85.91% <100%> (+0.84%)
22 <2> (+2)
:arrow_up:

...org/apache/dubbo/admin/config/AdminProperties.java
62.06% <62.06%> (ø)
5 <5> (?)

...che/dubbo/admin/service/impl/RouteServiceImpl.java
42.1% <72.72%> (+1.3%)
18 <5> (+5)
:arrow_up:

...va/org/apache/dubbo/admin/config/ConfigCenter.java
91.93% <89.74%> (-0.38%)
22 <9> (+5)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc1f920...667bb34. Read the comment docs.

I think if we support multiple registry center, it's necessary to make it clear which registry cneter that the service belongs to.

I think if we support multiple registry center, it's necessary to make it clear which registry cneter that the service belongs to.

@kexianjun Thanks for your advice, it makes sense!! And I've updated according to your suggestion, could you please have a look when you have time :)

是否支持同一服务注册到多个注册中心的情况

why this pr  have not be merged?

why this pr have not be merged?

no response

after a year, this pr still no merge.is something wrong?

authors: CodingSinger codecov-io heyudev johnsonin kexianjun kezhenxu94

repo: apache/dubbo-admin


text:
dubbo_3.1.10因refer MetadataService导致本服务线程池被覆盖，线程数设置失败

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Environment

Dubbo version: 3.1.10
Operating System version: ALL
Java version: 1.8.0

Steps to reproduce this issue

我们2个微服务A,B使用了Dubbo 3.1.10，metadata-type都为默认的local，metadataServicePort未设置，dubbo端口一样
服务B调用服务A，同时服务B对外提供服务
服务B设置服务线程threads为200

Expected Behavior
预期服务B的服务线程池为FixedThreadPool线程数为200
Actual Behavior
实际服务B的服务线程池为CachedThreadPool
bug调试
org.apache.dubbo.registry.client.metadata.StandardMetadataServiceURLBuilder#getMetadataServiceURLsParams
取到的

存在key:"side":"provider"
org.apache.dubbo.registry.client.metadata.StandardMetadataServiceURLBuilder#generateWithMetadata

这里consumer被覆盖为provider
org.apache.dubbo.common.threadpool.manager.DefaultExecutorRepository#getExecutorKey
导致这里的executorKey错误
org.apache.dubbo.common.threadpool.manager.DefaultExecutorRepository#createExecutorIfAbsent
导致这里覆盖了服务端的线程池

遇到了相同问题，可解决。

@xm101 generateUrlWithoutMetadata 也需要修改一下。

@xm101 generateUrlWithoutMetadata 也需要修改一下。

@xm101 generateUrlWithoutMetadata 也需要修改一下。

generateUrlWithoutMetadata方法在注册中心获取不到meta相关信息时调用，就我遇到的这个问题而言，因为这里没有来自注册中心的属性，就不存在覆盖问题

MetadataService 的线程池问题在 3.2 版本已经修复了

The thread pool problem of MetadataService has been fixed in version 3.2

与https://github.com/apache/dubbo/pull/11936 应该不是同个问题，11936是本服务的MetadataService线程池覆盖了业务接口线程池，而这个issues的问题是，业务接口线程池被调用的其他服务暴露出来的MetadataService生成的客户端线程池所覆盖
3.2可能线程池隔离解决了这问题，但是不少像我们一样仍使用JDK8的公司公司需要继续使用3.1.X，希望这个版本也能解决。

Already fixed in #11936

与https://github.com/apache/dubbo/pull/11936 应该不是同个问题，11936是本服务的MetadataService线程池覆盖了业务接口线程池，而这个issues的问题是，业务接口线程池被调用的其他服务暴露出来的MetadataService生成的客户端线程池所覆盖
3.2可能线程池隔离解决了这问题，但是不少像我们一样仍使用JDK8的公司公司需要继续使用3.1.X，希望这个版本也能解决。

Already fixed in #11936

与#11936 应该不是同个问题，11936是本服务的MetadataService线程池覆盖了业务接口线程池，而这个issues的问题是，业务接口线程池被调用的其他服务暴露出来的MetadataService生成的客户端线程池所覆盖 3.2可能线程池隔离解决了这问题，但是不少像我们一样仍使用JDK8的公司公司需要继续使用3.1.X，希望这个版本也能解决。

https://github.com/apache/dubbo/pull/11936 删除了 MetadataService 主动配置线程池的代码
3.2.x 不强制依赖高版本 JDK，目前 Dubbo 3.2.x 包括未来的 3.3.x 都是支持 jdk8 的

Already fixed in #11936

与#11936 应该不是同个问题，11936是本服务的MetadataService线程池覆盖了业务接口线程池，而这个issues的问题是，业务接口线程池被调用的其他服务暴露出来的MetadataService生成的客户端线程池所覆盖 3.2可能线程池隔离解决了这问题，但是不少像我们一样仍使用JDK8的公司公司需要继续使用3.1.X，希望这个版本也能解决。

[3.1] Fix #11524 (#11524) #11936 删除了 MetadataService 主动配置线程池的代码
3.2.x 不强制依赖高版本 JDK，目前 Dubbo 3.2.x 包括未来的 3.3.x 都是支持 jdk8 的

或者这么说吧，我们使用的是3.1.10，已经包含了11524 ConfigurableMetadataServiceExporter这个最新代码，那个问题我也遇到了，所以从3.1.9升级到了3.1.10，但是仍然遇到了这个新问题，上面是一步步调试和排查的结果，PR不小心点了close

Already fixed in #11936

与#11936 应该不是同个问题，11936是本服务的MetadataService线程池覆盖了业务接口线程池，而这个issues的问题是，业务接口线程池被调用的其他服务暴露出来的MetadataService生成的客户端线程池所覆盖 3.2可能线程池隔离解决了这问题，但是不少像我们一样仍使用JDK8的公司公司需要继续使用3.1.X，希望这个版本也能解决。

[3.1] Fix #11524 (#11524) #11936 删除了 MetadataService 主动配置线程池的代码
3.2.x 不强制依赖高版本 JDK，目前 Dubbo 3.2.x 包括未来的 3.3.x 都是支持 jdk8 的

或者这么说吧，我们使用的是3.1.10，已经包含了11524 ConfigurableMetadataServiceExporter这个最新代码，那个问题我也遇到了，所以从3.1.9升级到了3.1.10，但是仍然遇到了这个新问题，上面是一步步调试和排查的结果，PR不小心点了close

ok 我大概知道什么问题了，PR 我会重新打开，等测试跑过后合并。不过现在下一个 3.1.x 的版本发布还没有具体的规划时间，只有 3.2.x 会定期（每月）发版

authors: AlbumenJ stonelion xm101

repo: apache/dubbo


text:
Springboot方式如何配置多协议?

比如
dubbo.protocol.name=hessian
我想同时使用hessian和dubbo,这里无法配置了.
dubbo.protocol.port=20881
这个问题怎么解?

dubbo.protocols.dubbo.name=dubbo
dubbo.protocols.dubbo.port=20880

dubbo.protocols.hessian.name=hessian
dubbo.protocols.hessian.port=20881

具体配置信息在org.apache.dubbo.config.spring.context.annotation.DubboConfigConfiguration这个类

dubbo.protocols.dubbo.name=dubbo
dubbo.protocols.dubbo.port=20880

dubbo.protocols.hessian.name=hessian
dubbo.protocols.hessian.port=20881

具体配置信息在_org.apache.dubbo.config.spring.context.annotation.DubboConfigConfiguration_这个类

感谢指导,protocols多了s的方式几乎没人用过,没有例子可寻,感谢感谢.

你好。能否请教一下配置好了以后怎么在Java中动态选择吗

你好。能否请教一下配置好了以后怎么在Java中动态选择吗

dubbo会默认选择一个,不知道算不算动态选择,其他的代码方式的动态就不试过了.

service 中指定使用其中的hessian的时候报错了。
2020-09-16 15:40:45.283 WARN  [main][{}][o.s.c.a.AnnotationConfigApplicationContext:558] >>> Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dubboServiceMethodConfigManager': Unsatisfied dependency expressed through field 'serviceConfigList'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ServiceBean:apiServiceImpl:com.oppo.finance.external.api.identity.facexx.service.ApiService': Cannot resolve reference to bean 'hessian' while setting bean property 'protocols' with key [0]; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'hessian' available

@fibbery
2020-09-16 15:55:31.975 WARN  [main][{}][o.s.c.a.AnnotationConfigApplicationContext:558] >>> Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ServiceBean:apiServiceImpl:com.external.api.identity.facexx.service.ApiService': Cannot resolve reference to bean 'hessian' while setting bean property 'protocols' with key [0]; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'hessian' available

authors: Jaskey cookiejoo fibbery lsjlongyuan


text:
[Feature] 如何复制tooltip里面的内容

What problem does this feature solve?
tooltip展示的时候, 鼠标可以放进去, 长按拖动选中文字, ctrl + c可复制
What does the proposed API look like?
tooltip在加上enterable之后, 希望可以选中文字复制

@a15218378694 图表容器设置了 user-select: none 的样式，你可以覆盖掉这个样式

感谢

repo: apache/echarts


text:
[Bug] candlestick chart encode option doesn't work for multi-dimension dataset on 'category xAxis'

Version
"echarts": "^5.4.2"
Link to Minimal Reproduction
https://echarts.apache.org/examples/zh/editor.html?c=candlestick-simple&code=PYBwLglsB2AEC8sDeAoWsAmBDMWDOApmAFzJrqYQC2B0eUdpA2gOTZgEsA0sLot3XgGMANsEKCWACwgBzKQTxhJYgO6LlAXXIBfLuQAeAQQMQ8pVBVhgAniAKkWQnAVnAATjZa796GybMLcnRbe0cANywRAFdOH3JCdwhFZmCyKxC7B2EsaAwRDQghAGtuNPRaIWAMbMsM9ANHdk5fetgbZj57aElRcRbeGXkNFWB1JRZter1yzBwsVLbYJgBGAAYeACYN2ABmABYedZ5dgA4ptoB6S-WVrn2d3YBWE52ntc1W-uvlza5du57U4nXY8fb7T6zdA_Jigs5HF6wRH7TYXKxo2A6cjaHQAbiAA
Steps to Reproduce
The link as above.
Current Behavior
dimensions: ['date', 'open', 'close', 'highest', 'lowest']
'date' dimension of the data maps on 'open' dimension of candlestick series.
Expected Behavior
'date' dimension of the data should map as planned.
It works fine on value xAxis,but consumes too many computational resources and the bar width also looks weird.
Actually,I want to update next candlestick bar by clicking the chart,but it seems like it's only availible on bar series.Following link is bar series demo:
https://echarts.apache.org/examples/zh/editor.html?c=bar-simple&code=PYBwLglsB2AEC8sDeAoWszGAG0iAXMgL4A0as2ApgOaXQAmhSp59AhmGwM6VhPnp6EALZ0uUaF0IBtAOQQGlAB6ySsWQDc22AK6VZAXXIt0SgIJKIU5AIwBPEJUKyAxhxrAATndnGy6OwsrJhNYHk8ISmtpW1R0eNhoNlFnAGVtfX8E-0dnACM2T1VbQQ42GRL46QAGNQAmAyzs2Gk6tQBmI2bGyroXYHonG2bTZwVBlSbsuxlNbT1DSqJbZfQjIgBuFBQtT1gALz3EYTsAYQALQrAAOlowAC1PAAoASi3d2HHlBFgAdi3DtcYE9XNgIC4ANaqWAAMx00BckBgsCeIEKyS4L2GsA-IGACjAAEloAAFCBKSjYH7SNGeDFAmEwnhgAAaalp9OAjOZAE0DFt0Lj8dAidAAOIReg_E4XK7XfrQDSUTxgABinmAwjJFOwIOokuheIJxO1lLe5AgMJRMsuKvlME4ClNutk-og9ENwtFzpeWLi8QA9AHYOxOAB9ACM1xAOi45ye0i-SgA1Mm1EmDOaEgquDhKNdsMBqE8k1n4ja5cyAPLgCRPf0JcKRaKVdANkbuwhJqbNMAOIayApFHvZUPlFqJxQptOfKcGHojeJsaAiDgSQgw7Q8VuwVbZLrxIhl5ZHjZAA
Environment
- OS:
- Browser:
- Framework:

Any additional comments?

This is not-a-bug.  Use one of:

series.data with series.dimensions
dataset (eventually with series.encode)

but not both and not mixed. Series.data takes precedence over dataset.
Demo Code

This is not-a-bug. Use one of:

series.data with series.dimensions
dataset (eventually with series.encode)

but not both and not mixed. Series.data takes precedence over dataset. Demo Code

Thanks for the reply, but I need the xAxis type as category. Because when the data size grows, the bar width doesn't change, which looks very weird.

On value type, when the data size grows, the bar width doesn't change, which looks very weird.

Let see that first (Minimal Reproduction) and we'll go from there.

On value type, when the data size grows, the bar width doesn't change, which looks very weird.

Let see that first (Minimal Reproduction) and we'll go from there.
demo
Click the chart, it will update next candlestick. Like I said the width doesn't change when zooming.

Add data to the series instead of adding a new serie with a single candle - Demo Code

Add data to the series instead of adding a new serie with a single candle - Demo Code

Thanks for your constant support, it may be my last resort to the issue. I thought about this way at the beginning, but it requires unnecessarily additional computaitional resources. However,the rendering speed is fast than I expected. Do you think it's a good idea to pull a request to add category sort feature on candlestick series or other cunstom series like the bar series have?

If you mean "sorting" like  bar.realtimeSort, it could be done in the click processing, all data is available there.
Adding a request for a "new feature" that can be easily written by the user himself will get no traction with developers here.
disclaimer
opinions expressed in this post are strictly personal, as ECharts user, not a developer


text:
[Bug] Docs's EventMesh Java SDK in Maven can not be find.

Version tag should be 1.10.0-release.

repo: apache/eventmesh-site


text:
[FLINK-35890][cdc] Select only the increment but still enter the SNAPSHOT lead to ckp failed

ISSUE
Hi, when I was using Flink CDC Postgre, I chose 'debezium.snapshot.mode' = 'never' to skip the snapshot phase, but the code still entered the snapshot branch, and this log was printed, 'Database snapshot phase can 't perform checkpoint, acquired Checkpoint lock.'
(Since our business creates the task first and then writes the data), the task cannot perform ckp, and the status is always stuck in in progress. After the timeout, the task restarts.
Normally, if you only choose incremental data synchronization, you should not enter this branch.
The failure of ckp does not mean that the first one fails. After multiple ckp are successfully performed, it will then get stuck, causing a timeout and restarting the task. This is a problem that must occur. After it appeared in production, it also recurred many times in local testing.
Then we test, if there is data flowing in, the task status will be normal.
sql and conf like this

Why do this？
I think this miss one case，when I  set "snapshot = nerver",  I don't want to enter DB snapshot phase, and it can resolve my problem.

Thanks @leosanqing for the contribution, could you also check other connectors? I think some start up mode of debezium should not pass to underlying debeziunm engine once the Flink CDC incremental snapshot mode enabled.

authors: leonardBang leosanqing

repo: apache/flink-cdc


text:
[FLINK-30423] Introduce generated codes for CastExecutor

This PR aims to introduce codegen for CastExecutor, the main codes are copied from flink CastRule and related class

Hi @JingsongLi I added generated codes in CastRule for CastExecutor in table store, please help to review it when you're free, thanks

Hi @JingsongLi  I have updated this PR, thanks

Hi @zjureel , thanks for the update.

I still think we can introduce a CastExecutors class, each cast is a separate internal class or anonymous internal class inside, and we don't need CastExecutorProvider anymore, static methods are provided directly inside CastExecutors.
I don't need to bring a null check to every cast, we can let the caller take care of the null check himself.

Nice, I think it will be much more concise 👍

Hi @JingsongLi , please help to review it again, THX :)

repo: apache/flink-table-store


text:
[FLINK-17569][FileSystemsems]support ViewFileSystem when wait lease revoke of hadoop filesystem

What is the purpose of the change
This pull request makes HadoopRecoverableFsDataOutputStream#revokeLeaseByFileSystem support ViewFileSystem  when using viewfs the revokeLeaseByFileSystem method will not wait the file Lease to revoke because of  not instance of  DistributedFileSystem
Brief change log

resolve the real path and FileSystem of ViewFileSystem then wait to revoke the file lease

Verifying this change
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot run azure

Thank you for diagnosing this and suggesting this patch!
The patch looks good.
Any chance you could add a simple test that can verify this? Maybe extend the test that rund on the HDFS minicluster and wrap it in a viewFs?

@StephanEwen Ok, I will try to add a test

@StephanEwen added some viewfilesystem test by HDFS minicluster, after test I found last commit was not work because of #11815 has mentioned and the truncate method also need resolved path

Thank you for looking into the test.
Unfortunately, this PR now contains a very large amount of unrelated code format changes and we cannot review or merge it like that [1]. These re-formattings hide the real important changes, hide root causes for line changes, in the git history, etc. Please make sure your IDE does not auto-format classes on save or so. The PR must really include only the relevant changes, not unrelated additional changes.
[1] https://flink.apache.org/contributing/contribute-code.html

updated

authors: StephanEwen wxplovecc

repo: apache/flink


text:
[FLINK-19736] Add the SinkTransformation

What is the purpose of the change
Translate the new sink to the physical operator topology.
Brief change log

Introduce 'SinkTransformation' to represent the new sink api.
Introduce 'SinkTransformationTranslator' that translates the 'SinkTransformation' to the corresponding runtime operators.
Make DataStream Sdk support new sink api.

Verifying this change
This change added tests and can be verified as follows:

Introduce StreamSinkITCase to test end to end result
Introduce SinkTransformationTranslatorTest to test build sink topology

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (JavaDocs )

Given that now we have a transformation that is mapped to multiple operators, I would suggest to not reuse the DataStreamSink but create a new wrapper for the new SinkTransformation that will account for this difference.
In more detail, the new wrapper (I do not have a proposal for a name, for now let's call it NewDataStreamSink) could have different methods for setting the uid of the writer, the committer and the globalCommitter (NewDataStreamSink withWriterUid(...), NewDataStreamSink withCommiterUid(...) and NewDataStreamSink withGlobalCommitterUid(...)), instead of trying to prepend or append things to the user-provided uid. I also propose to follow a builder pattern as you can see from the signatures above (the methods will return this), so that the user can write sth like:
env.addSource()
      ...
      .addSink().withXXXUid(...)

This will also allow us to have a validation step (if we want) that, for example, will check that if the Sink has a committer, then it has to also have a committerUid.
Then this NewDataStreamSink can forward all the details to the new SinkTransformation which will also have such methods, and from there, the translator will be able to set the StreamNode properties accordingly.
The same may make sense also for the resource settings but we can see later about that.
In general, I think that such a design that acknowledges that the new Sink does not fit in the old Transformation paradigm is more flexible and more future-proof.
What do you think @guoweiM and @aljoscha ?

From an offline discussion with @guoweiM I think he is right to not go with my proposal above as it exposes more information about internal implementation than needed and this may tie our hands for the future.

I merged this! 😃

authors: aljoscha guoweiM kl0u


text:
[FLINK-22881] Tasks are blocked while broadcasting stream status

What is the purpose of the change
We revert the toggling between IDLE and ACTIVE status if a record is emitted in an IDLE state. If records are emitted in an IDLE state the toggling incur a lot of performance penalty.
Moreover I revert the fact that we do not mark output as idle if there are no PartialWatermarks.
Lastly, I optimize the WatermarkAssignerOperator slightly so that it emits StreamStatus only if the announcedStatus changed.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Hey @dawidwys , thanks for taking care of this. The documentation about ExecutionConfigOptions::TABLE_EXEC_SOURCE_IDLE_TIMEOUT says by default the idleness detect is not enabled. So I wonder why we need to set it for the hive test?

@lirui-apache The idea was that if we enable TABLE_EXEC_SOURCE_IDLE_TIMEOUT the WatermarkAssignerOperator takes control over its StreamStatus and it does not respect the idleness of the sources.
However, I spotted another difference from the 1.13 in how idleness is handled in sources which I reverted in this PR.

Thanks @dawidwys for the clarifications. I'm not really familiar with the watermark and idleness detection code. So I'll leave this for other folks to have a look.

@pnowojski @AHeise One question I have is if I shall put the logic that the multiplexer can go idle after splits are removed (watermark has progressed from its initial value) or should it be active in that case?

I'll wait for green CI and merge it. Thx @AHeise for the review.

authors: dawidwys lirui-apache


text:
[FLINK-24738][runtime] Ignoring buffer size announcement if the channel is released already

What is the purpose of the change
This PR fixes IllegalStateException during  new buffer size announcement when the channel is released already
Brief change log

Ignore announcement new buffer size if the channel is released

Verifying this change
This change added tests and can be verified as follows:

Added new test in LocalInputChannelTest

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot run azure


text:
[hotfix] [docs] Insert closing parentheses on "Flink DataStream API Programming Guide".

Insert closing parentheses into sample code on "Flink DataStream API Programming Guide".

Thanks for the fix @keijiyoshida.
+1 to merge

Thanks for your checking and merging!

authors: fhueske keijiyoshida


text:
[FLINK-10059] [table] Add LTRIM supported in Table API and SQL

What is the purpose of the change
This pull request add LTRIM supported in Table API and SQL
Brief change log

Add LTRIM supported in Table API and SQL

Verifying this change
This change is already covered by existing tests, such as ScalarFunctionsTest#testLTrim.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

cc @hequn8128 @suez1224 @xccui

thanks @hequn8128 , updated this PR~

@twalthr Does this PR look good to you?

@xccui Can you help advance this PR? thanks.

Hi @yanghua, thanks for your effort. The docs for built-in functions have been moved to a new page (functions.md). You could rebase your branches to apply this change. In addition, it seems that Calcite has provided the LTRIM/RTRIM methods in org.apache.calcite.util.BuiltInMethod.java. Thus there's no need to duplicate them in org.apache.flink.table.codegen.calls .BuiltInMethods.scala.
Thanks, Xingcan

@xccui refactored this PR and will also refactor RTRIM soon.

@xccui refactored this PR and will also refactor RTRIM soon.

@xccui Does this PR look good to you?

@xccui updated, will also update RTRIM

@xccui the travis failure is not caused by this PR's change. I have updated the RTRIM PR : #6509 , please review again thanks.

Thanks for the revision @yanghua. I'll have a final check and merge this, as well as PR #6509.
Thanks, Xingcan

authors: xccui yanghua


text:
[FLINK-10707][web-dashboard] flink cluster overview dashboard improvements

What is the purpose of the change

Improve Flink Dashboard to provide more information

Brief change log

Task slot usage in cluster shown in percent
show total task manager CPU cores
show each task manager in overview

colorize task manager according to how far away heartbeat is away from akka.lookup.timeout
tooltip with metrics for task manager
percentage bar indicating task slot usage per task manager
resize if there are more than 20 task managers

Preview

Verifying this change
This change is already covered by existing tests, such as the Flink Web Dashboard tests.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

@vthinkxie thx for the extended review! I hope to implement the fixes this week still ...

Great job @drummerwolli
I just add some comments, if I missed something, please let me know

@vthinkxie thx for the in-depth review. i changed all your comments. feel free to check again. maybe we can get this done even until thursday in time for 1.9.0 ? :100:
@everyone: ready 4 review again. sry that it took so long to get the changes done.

@vthinkxie JIRA is not working currently, so i'm writing here. sry for the very slow progress on this. due to me not being familiar with Angular, this is a big barrier for me to get back to. i just tried again to have a look, but somehow the build is broken, npm run proxy is yielding this for me:

npm run build also fails:
ERROR in src/app/share/customize/task-manager-item/task-manager-item.component.html(24,6): : Object is possibly 'undefined'.
src/app/share/customize/task-manager-item/task-manager-item.component.html(24,6): : Object is possibly 'undefined'.
src/app/share/customize/task-manager-item/task-manager-item.component.html(24,6): : Object is possibly 'undefined'.
src/app/share/customize/task-manager-item/task-manager-item.component.html(24,6): : Object is possibly 'undefined'.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! flink-dashboard@2.0.0 build: `ng build --prod --base-href ./`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the flink-dashboard@2.0.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/fwollert/.npm/_logs/2020-03-03T10_34_59_322Z-debug.log

can you have a look what needs to be fixed for this? i tried different npm versions, but no success. i tried with npm ci to pick exactly the lib version from the lock file, but still, no success.

Hi @drummerwolli
Welcome back!
there is some breaking change in the RestAPI, you may need to rebase on the master to make it work again.

@vthinkxie that's what I did already (see the force push earlier here). this happens on the current apache/flink master branch for me as well.

you also need to base on the latest flink version if under proxy mode

you mean the cluster accessed/proxied must be a 1.9 or 1.10 cluster? I'm always currently trying to connect to a 1.8 cluster (that's what we have currently running in production)

Effort seems to be abandoned.

authors: drummerwolli tillrohrmann vthinkxie


text:
[FLINK-12143] Pass configuration parameters to plugins manager (FS components)

What is the purpose of the change
This PR changes Flink distribution's starter scripts: defines new configuration environment parameter for plugins dir and passes the configuration to the plugins manager.
Currently, only file system components support the plugin manager, so related (to file system) end-to-end tests also have been modified to be loaded via the plugin manager.
Brief change log

the scripts in flink-dist have been modified, a new FLINK_HOME environment variable replaces FLINK_ROOT_DIR;
the various Flink startup entry points are extended to configure and pass plugins dir to plugins manager;
the plugin manager is configured by system wide parents first patterns for class loading;
some end-to-end tests involving different file system components are modified to use fs components via plugins mechanism.

Verifying this change
This change is already covered by existing tests, such as:

e2e batch wordcount tests for hadoop, presto, azure and dummy fs are modified to use the corresponding fs component via plugin mechanism;
test_streaming_file_sink.sh is modified to use hadoop fs via plugin mechanism;
test_docker_embedded_job.sh is modified to use dummy fs via plugin mechanism;
test_yarn_kerberos_docker.sh is modified to use dummy fs via plugin mechanism.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no) (the distribution Flink scripts)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@pnowojski I'm not really knowledgeable about the start scripts nor deployment unfortuantely. I still skimmed over them and didn't spot anything that looks wildly suspicious.
I wouldn't know who to ask for as an "expert" though.
I'm not entirely sold on migrating from FLINK_ROOT_DIR to FLINK_HOME; it looks like only the docker/mesos and new pyflink code (bar some super-old scripts) were using FLINK_HOME.
So basically my question is, why did you pick one over the other?

Thanks @zentol. I just hoped that someone with more experience here might catch something that's not covered by our tests (like the Windows bat file bug that @1u0 has fixed).

I'm not entirely sold on migrating from FLINK_ROOT_DIR to FLINK_HOME; it looks like only the docker/mesos and new pyflink code (bar some super-old scripts) were using FLINK_HOME. So basically my question is, why did you pick one over the other?

I just wanted to have them unified. I personally preferred FLINK_ROOT_DIR a bit better, but @tillrohrmann was using FLINK_HOME during our offline discussions about it (even when I was referring to it as FLINK_ROOT_DIR ;) ), so I presume he preferred the FLINK_HOME more. I didn't mind one way or another too much so just I sticked to nomenclature that @tillrohrmann was using.
If you have some stronger preference the other way, we could change it the other direction. If not, I would vote for leaving it unified as it is to avoid an extra work.

For me X_HOME is a bit more standard than other names (looking at my local env variables definitions). But I'm not adamant on the actual naming.

Oh we should definitely unify things.
My gut feeling is that we should stick to FLINK_ROOT_DIR, purely because it was used before the dockerfiles were added (and thus introduced the inconsistency). Had we bothered about this consistency back then they probably would have used FLINK_ROOT_DIR as well; so we're fixing the timeline here.
But this is ultimately a minor thing that I wouldn't mind being decided by a coin-flip.

Yes, you are right @zentol. I will just pick the lazy solution - keep it as it is proposed now.
@1u0 could you rebase and resolve the conflict so that I can merge the PR?

@1u0 could you rebase and resolve the conflict so that I can merge the PR?

Done.

Thanks, merging :)

authors: 1u0 pnowojski tillrohrmann zentol


text:
GEODE-6635: Native Client API docs: Use a 3-part version header

Sets the header on API doc pages to the 3-part version number, based on the PRODUCT_VERSION specified at build time. If PRODUCT_VERSION has less than 3 components, adds "0" elements to fill out the string.
Old behavior was a 2-part version number, or an empty string if only one component was specified.
Updated BUILDING.md to clarify the option of defining PRODUCT_VERSION on the cmake config command line, and to fix a typo.

Verified building on supported platforms, using CI.

repo: apache/geode-native


text:
[GEODE-2408] Refactor CacheableDate with C++11 standards.

Moved from old repo to new repo. Can I get a thumbs up please?

LGTM

authors: PivotalSarge pivotal-jbarrett


text:
GEODE-4626: Indicate the region creation from a template is deprecated

Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[x] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[x] Is your initial contribution a single, squashed commit?

[x] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.

FWIW precheckin is green

repo: apache/geode


text:
[GOBBLIN-1952] Make jobname shortening in GaaS more aggressive

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[ ] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-1952

Description

[ ] Here are some details about my PR, including screenshots (if applicable):
Gobblin-as-a-Service creates jobnames using flowgroups flownames, edges, and jobnames from the template. However, this tends to create a very long string which then causes issues in Gobblin job when creating files that use the jobname to create working directories or state stores.

Although there has been previous code that shortens job name lengths, we want to further increase this by being more aggressive with the maximum length of the jobname to reduce the odds of exceeding 255 chars (max length of HDFS component)
The PR reduces the max job name length to 128 characters (1/2 of previous max) as well as forcing jobnames to then be shortened to flowGroup_<job_hash> which will be a UUID at a per-job level for each dag.
The reason why it uses flowGroup still as a prefix before the jobname hash is to reduce the odds that a shared job working directory will run into hash collisions, which can lead to complications on concurrent jobs
Tests

[ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

[ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #3822 (38b9be7) into master (96092f7) will increase coverage by 5.96%.
Report is 5 commits behind head on master.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master    #3822      +/-   ##
============================================
+ Coverage     47.63%   53.60%   +5.96%
+ Complexity    11048     1399    -9649
============================================
  Files          2155      281    -1874
  Lines         85325    10343   -74982
  Branches       9488     1110    -8378
============================================
- Hits          40642     5544   -35098
+ Misses        40994     4279   -36715
+ Partials       3689      520    -3169

Files
Coverage Δ

...gobblin/service/modules/spec/JobExecutionPlan.java
69.87% <100.00%> (ø)

... and 1878 files with indirect coverage changes
:mega: Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

authors: Will-Lo codecov-commenter

repo: apache/gobblin


text:
HDDS-3884. Fix typo in pom.xml.

What changes were proposed in this pull request?
fix typo
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3884
How was this patch tested?
Exisitng CI

Test failures are unrelated.
It is fixing a typo, no code changes. Going to commit these changes.

repo: apache/hadoop-ozone


text:
HADOOP-17630. [JDK 15] TestPrintableString fails due to Unicode 13.0 support.

Remove U+30000 from the test.
JIRA: HADOOP-17630

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
13m 18s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m  4s

trunk passed

+1 :green_heart:
compile
20m 45s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
18m 13s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  6s

trunk passed

+1 :green_heart:
mvnsite
1m 31s

trunk passed

+1 :green_heart:
javadoc
1m  5s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 38s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
2m 22s

trunk passed

+1 :green_heart:
shadedclient
15m 27s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 51s

the patch passed

+1 :green_heart:
compile
20m  7s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m  7s

the patch passed

+1 :green_heart:
compile
18m  7s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
18m  7s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  3s

the patch passed

+1 :green_heart:
mvnsite
1m 30s

the patch passed

+1 :green_heart:
javadoc
1m  2s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 38s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
2m 28s

the patch passed

+1 :green_heart:
shadedclient
15m 36s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 28s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 56s

The patch does not generate ASF License warnings.

190m 40s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2890/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2890

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 6221b055449c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 6fae4ae00f7ea6199df7208105fc5d6e9a89455e

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2890/1/testReport/

Max. process+thread count
3158 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2890/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Merged. Thanks @jojochuang

authors: aajisaka hadoop-yetus

repo: apache/hadoop


text:
HDFS-15975. Use LongAdder instead of AtomicLong

JIRA: HDFS-15975
When counting some indicators, we can use LongAdder instead of AtomicLong to improve performance. The long value is not an atomic snapshot in LongAdder, but I think we can tolerate that.

Nice one. For metrics and statistics, LongAdder provides much better throughput.

Thanks @virajjasani for the review.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 36s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 59s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
27m 32s

trunk passed

+1 :green_heart:
compile
28m 48s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
24m  6s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
5m  6s

trunk passed

+1 :green_heart:
mvnsite
4m 32s

trunk passed

+1 :green_heart:
javadoc
3m  8s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
4m 13s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
10m 30s

trunk passed

+1 :green_heart:
shadedclient
21m 42s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 21s

the patch passed

+1 :green_heart:
compile
20m 20s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m 20s

the patch passed

+1 :green_heart:
compile
18m  4s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
18m  4s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 45s
/results-checkstyle-root.txt
root: The patch generated 5 new + 242 unchanged - 5 fixed = 247 total (was 247)

+1 :green_heart:
mvnsite
4m 10s

the patch passed

+1 :green_heart:
javadoc
3m  0s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
4m  3s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
8m 42s

the patch passed

+1 :green_heart:
shadedclient
16m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 25s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 36s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
246m 51s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m 12s

The patch does not generate ASF License warnings.

495m 56s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.server.balancer.TestBalancer

hadoop.hdfs.TestReconstructStripedFile

hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys

hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2907

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux d880014b6357 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 9d78a92e7028adeacc66fa9b9aeeca32a87f702a

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/1/testReport/

Max. process+thread count
3362 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Those failed unit tests were unrelated to the change. And they work fine locally.
Hi @goiri @tasanuma , could you please review the code? Thanks very much.

Left few nits and one question

Thanks @virajjasani for your good advice, I fixed it.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 40s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 48s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 24s

trunk passed

+1 :green_heart:
compile
21m 58s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
19m 30s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
3m 46s

trunk passed

+1 :green_heart:
mvnsite
4m 10s

trunk passed

+1 :green_heart:
javadoc
2m 53s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 46s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
8m 32s

trunk passed

+1 :green_heart:
shadedclient
17m 41s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 26s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m  4s

the patch passed

+1 :green_heart:
compile
22m 37s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
22m 37s

the patch passed

+1 :green_heart:
compile
19m 52s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
19m 52s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 39s
/results-checkstyle-root.txt
root: The patch generated 5 new + 242 unchanged - 5 fixed = 247 total (was 247)

+1 :green_heart:
mvnsite
4m  3s

the patch passed

+1 :green_heart:
javadoc
2m 50s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
4m  4s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
9m 18s

the patch passed

+1 :green_heart:
shadedclient
17m 46s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
18m 21s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 37s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
244m 34s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m  9s

The patch does not generate ASF License warnings.

474m 18s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys

hadoop.hdfs.TestDFSShell

hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2907

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 4aef76f54001 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / fbb20ec4f413e59c2b24204ab916a6b341426d44

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/2/testReport/

Max. process+thread count
3238 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Nice one. For metrics and statistics, LongAdder provides much better throughput.

Agreed with @virajjasani. Thanks for working on this, @tomscut.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 28s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 21s

trunk passed

+1 :green_heart:
compile
20m 38s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
17m 58s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
3m 46s

trunk passed

+1 :green_heart:
mvnsite
4m 15s

trunk passed

+1 :green_heart:
javadoc
3m  5s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
4m  9s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
8m 13s

trunk passed

+1 :green_heart:
shadedclient
16m 30s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 55s

the patch passed

+1 :green_heart:
compile
20m  3s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m  3s

the patch passed

+1 :green_heart:
compile
18m  2s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
18m  2s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 42s
/results-checkstyle-root.txt
root: The patch generated 6 new + 242 unchanged - 5 fixed = 248 total (was 247)

+1 :green_heart:
mvnsite
4m 13s

the patch passed

+1 :green_heart:
javadoc
3m  0s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
4m  6s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
8m 43s

the patch passed

+1 :green_heart:
shadedclient
16m 35s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 23s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 35s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
235m 13s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m 19s

The patch does not generate ASF License warnings.

453m 10s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys

hadoop.hdfs.TestDFSShell

hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2907

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 4ad4d1f8986a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c894c8e77e51ff22b7a2a6c645a152558a303095

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/3/testReport/

Max. process+thread count
3608 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 37s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 36s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 16s

trunk passed

+1 :green_heart:
compile
20m 46s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
18m 23s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
3m 45s

trunk passed

+1 :green_heart:
mvnsite
4m 14s

trunk passed

+1 :green_heart:
javadoc
3m  2s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
4m  7s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
8m 15s

trunk passed

+1 :green_heart:
shadedclient
16m 26s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 26s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 51s

the patch passed

+1 :green_heart:
compile
20m 12s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
20m 12s

the patch passed

+1 :green_heart:
compile
18m 14s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
18m 14s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 46s
/results-checkstyle-root.txt
root: The patch generated 5 new + 242 unchanged - 5 fixed = 247 total (was 247)

+1 :green_heart:
mvnsite
4m 14s

the patch passed

+1 :green_heart:
javadoc
2m 58s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 59s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
spotbugs
8m 56s

the patch passed

+1 :green_heart:
shadedclient
17m 39s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 33s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 31s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
242m  3s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 59s

The patch does not generate ASF License warnings.

461m 25s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.qjournal.server.TestJournalNodeSync

hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys

hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots

hadoop.hdfs.TestViewDistributedFileSystem

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2907

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 2f8c2108fda1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 699e8185d1ea421e6f153918f01530fbe71ef490

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/4/testReport/

Max. process+thread count
3089 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Merged it. Thanks for your contribution, @tomscut. Thanks for your reviews, @virajjasani.

@tomscut I think this is a good optimization. Would you mind creating another PR for branch-3.3?

@tomscut I think this is a good optimization. Would you mind creating another PR for branch-3.3?

Thanks @tasanuma for the merge. I will create PR for branch-3.3 later.

authors: hadoop-yetus tasanuma tomscut


text:
HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads

Description of PR
Part of HADOOP-18103.
Introducing fs.s3a.min.seek.vectored.read and fs.s3a.max.readsize.vectored.read
to configure min seek and max read during a vectored IO operation in S3A connector.
These propeties actually define how the ranges will be merged. To completely
disable merging set fs.s3a.max.readsize.vectored.read to 0.
How was this patch tested?
For code changes:

[ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

CC @mehakmeet

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
32m 31s

Maven dependency ordering for branch

-1 :x:
mvninstall
21m 31s
/branch-mvninstall-root.txt
root in feature-vectored-io failed.

-1 :x:
compile
14m 23s
/branch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root in feature-vectored-io failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
compile
12m 29s
/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in feature-vectored-io failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
checkstyle
3m 27s

feature-vectored-io passed

+1 :green_heart:
mvnsite
2m 12s

feature-vectored-io passed

+1 :green_heart:
javadoc
1m 27s

feature-vectored-io passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m  5s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 21s

feature-vectored-io passed

+1 :green_heart:
shadedclient
20m 28s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 34s

the patch passed

-1 :x:
compile
14m  3s
/patch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
javac
14m  3s
/patch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt
root in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.

-1 :x:
compile
12m 22s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
12m 22s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 17s
/results-checkstyle-root.txt
root: The patch generated 2 new + 9 unchanged - 0 fixed = 11 total (was 9)

+1 :green_heart:
mvnsite
2m  4s

the patch passed

+1 :green_heart:
javadoc
1m 20s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

-1 :x:
javadoc
0m 32s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39)

+1 :green_heart:
spotbugs
3m 38s

the patch passed

+1 :green_heart:
shadedclient
20m 10s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
17m 15s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 10s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

198m 30s

Reason
Tests

Failed junit tests
hadoop.ipc.TestIPC

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux 1c0d99a8cb8d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / 173dfbbd160a45e7e76b1da62d604b00faa24192

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/1/testReport/

Max. process+thread count
1296 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  2s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
12m 50s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
22m 47s

feature-vectored-io passed

+1 :green_heart:
compile
22m 30s

feature-vectored-io passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
21m 32s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 12s

feature-vectored-io passed

+1 :green_heart:
mvnsite
2m 49s

feature-vectored-io passed

+1 :green_heart:
javadoc
1m 50s

feature-vectored-io passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m 31s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 54s

feature-vectored-io passed

+1 :green_heart:
shadedclient
21m 10s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 29s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 34s

the patch passed

+1 :green_heart:
compile
21m 48s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
21m 48s

the patch passed

-1 :x:
compile
13m 46s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

-1 :x:
javac
13m 46s
/patch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
root in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
3m 42s

the patch passed

+1 :green_heart:
mvnsite
2m 38s

the patch passed

+1 :green_heart:
javadoc
1m 52s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

-1 :x:
javadoc
0m 47s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39)

+1 :green_heart:
spotbugs
4m  9s

the patch passed

+1 :green_heart:
shadedclient
21m 23s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
25m  7s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 35s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  0s

The patch does not generate ASF License warnings.

223m 43s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux c3891488a747 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / 99c983e561cfcdf9afaeadffe9259b1bdcc13bb1

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/3/testReport/

Max. process+thread count
3158 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
12m 13s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
16m 17s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m 53s

feature-vectored-io passed

+1 :green_heart:
compile
22m 29s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
19m 43s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 52s

feature-vectored-io passed

+1 :green_heart:
mvnsite
2m 41s

feature-vectored-io passed

+1 :green_heart:
javadoc
1m 58s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m 32s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
4m  4s

feature-vectored-io passed

+1 :green_heart:
shadedclient
20m 54s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 34s

the patch passed

+1 :green_heart:
compile
21m 40s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
21m 40s

the patch passed

+1 :green_heart:
compile
19m 39s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
19m 39s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
3m 37s
/results-checkstyle-root.txt
root: The patch generated 2 new + 9 unchanged - 0 fixed = 11 total (was 9)

+1 :green_heart:
mvnsite
2m 37s

the patch passed

+1 :green_heart:
javadoc
1m 50s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m 29s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
4m  7s

the patch passed

+1 :green_heart:
shadedclient
21m 19s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
25m  7s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 35s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  1s

The patch does not generate ASF License warnings.

242m 10s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux f148d98dabd6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / dfe002fe8f3f31532f99cc2aa827d3c0c821f830

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/4/testReport/

Max. process+thread count
1261 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
15m 57s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m 47s

feature-vectored-io passed

+1 :green_heart:
compile
22m 32s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
19m 48s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
3m 47s

feature-vectored-io passed

+1 :green_heart:
mvnsite
2m 38s

feature-vectored-io passed

+1 :green_heart:
javadoc
1m 56s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m 31s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
4m  3s

feature-vectored-io passed

+1 :green_heart:
shadedclient
21m  1s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 34s

the patch passed

+1 :green_heart:
compile
21m 49s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
21m 49s

the patch passed

+1 :green_heart:
compile
19m 45s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
19m 45s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
3m 33s

the patch passed

+1 :green_heart:
mvnsite
2m 39s

the patch passed

+1 :green_heart:
javadoc
1m 52s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
2m 31s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
4m 11s

the patch passed

+1 :green_heart:
shadedclient
21m 22s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
25m  6s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 35s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

230m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux 9aac72e6d636 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / 8d1f9bcd680a1af1f3018046e70537bd213fa5dd

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/5/testReport/

Max. process+thread count
1592 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
16m 17s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m  2s

feature-vectored-io passed

+1 :green_heart:
compile
22m 59s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
20m 24s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 25s

feature-vectored-io passed

+1 :green_heart:
mvnsite
3m 48s

feature-vectored-io passed

+1 :green_heart:
javadoc
3m  4s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 35s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 11s

feature-vectored-io passed

+1 :green_heart:
shadedclient
22m 51s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 51s

the patch passed

+1 :green_heart:
compile
22m 11s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
22m 11s

the patch passed

+1 :green_heart:
compile
20m 28s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 50s

the patch passed

+1 :green_heart:
mvnsite
3m 45s

the patch passed

+1 :green_heart:
javadoc
2m 57s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 37s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 10s

the patch passed

+1 :green_heart:
shadedclient
23m  1s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
25m 47s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  7s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 36s

The patch does not generate ASF License warnings.

251m 14s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux 2cab4efb94a0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / 7da65b6d5d7f2e1e8382bca1aa17338d82804685

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/6/testReport/

Max. process+thread count
1568 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
13m  6s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
16m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m  2s

feature-vectored-io passed

+1 :green_heart:
compile
23m  0s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
20m 32s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 29s

feature-vectored-io passed

+1 :green_heart:
mvnsite
3m 45s

feature-vectored-io passed

+1 :green_heart:
javadoc
2m 57s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 28s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m  9s

feature-vectored-io passed

+1 :green_heart:
shadedclient
22m 54s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 34s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 50s

the patch passed

+1 :green_heart:
compile
21m 59s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
21m 59s

the patch passed

+1 :green_heart:
compile
20m 32s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 32s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 14s

the patch passed

+1 :green_heart:
mvnsite
3m 42s

the patch passed

+1 :green_heart:
javadoc
2m 53s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 37s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 10s

the patch passed

+1 :green_heart:
shadedclient
23m  4s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
25m 46s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m 23s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 35s

The patch does not generate ASF License warnings.

262m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux 5a5a4ae070db 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / 86f760ef0952a576518fed52a1ac6d717177327f

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/7/testReport/

Max. process+thread count
1261 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ feature-vectored-io Compile Tests _

+0 :ok:
mvndep
16m 12s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
24m  1s

feature-vectored-io passed

+1 :green_heart:
compile
22m 49s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
20m 27s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
4m 25s

feature-vectored-io passed

+1 :green_heart:
mvnsite
3m 46s

feature-vectored-io passed

+1 :green_heart:
javadoc
2m 58s

feature-vectored-io passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 39s

feature-vectored-io passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m  9s

feature-vectored-io passed

+1 :green_heart:
shadedclient
22m 56s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 32s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 51s

the patch passed

+1 :green_heart:
compile
22m  4s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
22m  4s

the patch passed

+1 :green_heart:
compile
20m 34s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
20m 34s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 19s
/results-checkstyle-root.txt
root: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9)

+1 :green_heart:
mvnsite
3m 44s

the patch passed

+1 :green_heart:
javadoc
2m 47s

the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
3m 38s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
5m 15s

the patch passed

+1 :green_heart:
shadedclient
22m 51s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
25m 41s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m 16s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 36s

The patch does not generate ASF License warnings.

250m 22s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3964

Optional Tests
dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle

uname
Linux e2a0388503cf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
feature-vectored-io / a64892e65fb5843506c46e8950ad7c93c933b838

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/8/testReport/

Max. process+thread count
1261 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3964/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

authors: hadoop-yetus mukund-thakur


text:
[Feature Request]: Syntax highlighting for scripts

What would you like to happen?
Migration: https://issues.apache.org/jira/browse/HOP-3036
https://issues.apache.org/jira/browse/HOP-3224
I think it'd be great if Hop Gui has highlight syntax for components where the end user input some code.
Like SQL scripts , JSON configurations or Javascript code.
Something like we had in PDI / Kettle.
Issue Priority
Priority: 3
Issue Component
Component: Hop Gui

I'd love that. Even if we'd only get bracket highlighting, that alone would be helpful.

authors: PhotoBiker hansva

repo: apache/hop


text:
[SUPPORT]

Hi
Our organization is migrating from Hudi 0.6.0 to Hudi 0.12.1 and also updating the required spark and EMR versions. Our existing data sets (100s of TBs of data on S3) are written using Hudi 0.6.0.
The latest version of Hudi has come way since 0.6.0, we are not sure about how to use 0.12.1 directly.
Could someone provide the steps for upgrading from 0.6.0 to 0.12.1?
Do we have to rebuild our tables, we are more concerned about this as tables are having billions of records ?
Should we expect following imporvements after the upgrade:
– faster upserts
 – columns add/modify (schema evolution)

 – clustering

 – possible solution for storing history of updates performed on recrods

Thanks,
Zamin Hassnain

I would suggest you use the 0.12.3 or 0.14.1, 0.12.1 still got some stability issues.

repo: apache/hudi