text:
[FLINK-16625][k8s] Extract BootstrapTools#getEnvironmentVariables to a general utility in ConfigurationUtils

What is the purpose of the change
BootstrapTools#getEnvironmentVariables actually is a general utility to extract key-value pairs with specified prefix trimmed from the Flink Configuration object. It can not only be used to extract customized environment variables in the YARN setup but also for customized annotations/labels/node-selectors in the Kubernetes setup.
This ticket proposes to rename it to ConfigurationUtils#getPrefixedKeyValuePairs and move it to the flink-core module as a more general utility to share for the YARN/Kubernetes setup.
Verifying this change
This change added tests.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)

@TisonKun Travis gives a green light, could you help take a look?

@flinkbot run azure

@zhengcanbin please rebase on master. I think this PR itself is good to go.

@zhengcanbin please rebase on master. I think this PR itself is good to go.

Thanks, @TisonKun. I have fetched and rebased again.

cc @TisonKun. Both Travis and Azure give a green light.

authors: TisonKun zhengcanbin


text:
[FLINK-19983][network] Remove wrong state checking in SortMergeSubpartitionReader

What is the purpose of the change
Remove wrong state checking in SortMergeSubpartitionReader.
Brief change log

Remove wrong state checking in SortMergeSubpartitionReader.

Verifying this change
This change is already covered by existing tests, such as ShuffleCompressionITCase#testDataCompressionForSortMergeBlockingShuffle.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@AHeise Thanks for your review and comments.
Your concern is reasonable. I have updated my fix. For current bounded partition, only Netty thread can release or poll the subpartition read view so we should never poll a released view. In this case, two reasons cause the failure: 1) There are redundant data availability notifications; 2) The buffers read is not cleared which means the read view can be still available after released.
For multi-thread scenario, from my understand, the root failure cause need to be propagated to the downstream, if nothing is polled from a read view, we will try to find out if the view is released and if there is a failure cause, the logic is like this:
next = reader.getNextBuffer();
if (next == null) {
    if (!reader.isReleased()) {
        continue;
    }

    Throwable cause = reader.getFailureCause();
    if (cause != null) {
        ErrorResponse msg = new ErrorResponse(new ProducerFailedException(cause), reader.getReceiverId());
	ctx.writeAndFlush(msg);
    }
}

@AHeise Thank for you review. I have resolved all comments and updated the PR.

Failure unlreated, merging.

source: gharchive/pull-request

authors: AHeise wsry

repo: apache/flink

license_type: permissive

license_source: github-api


text:
[FLINK-20309][tests] Make source split assignment in UnalignedCheckpointITCase stable. [1.12]

Unchanged backport of #14655.

No changes and green AZP -> merging.

No changes and green AZP -> merging.


text:
[FLINK-25771][connectors][Cassandra][test] Raise the Cassandra driver timeouts at the Cluster level

R: @zentol

@flinkbot run azure

@zentol the failure seems unrelated (failure on a repeated test CastFunctionITCase.testFunction fails at interation 15) WDYT ?

@flinkbot run azure

@flinkbot run azure

@echauchot the PR seems failed on a fixed issue, could you rebase it to the latest master?

@gaoyunhaii done

Hi @zentol, WDYT ?

@zentol thanks for merging !

authors: echauchot gaoyunhaii


text:
[FLINK-28172][changelog] Scatter dstl files into separate directories…

… by job id
What is the purpose of the change
Scatter dstl files into separate directories by job id, described in FLINK-28172
Brief change log

pass jobID to FsStateChangelogStorageFactory, FsStateChangelogStorage and StateChangeFsUploader
make sub directory ':jobID/dstl' under base-path

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot run azure

@flinkbot run azure

@flinkbot run azure

@flinkbot run azure

Thanks @rkhachatryan . Done.


text:
[FLINK-36248][table] Introduce new Join Operator with Async State API

What is the purpose of the change
Introduce new join operator with async state api.
Brief change log

Introduce new join operator with async state api
ITCases and harness tests are added to verity this pr

Verifying this change
ITCases and harness tests are added to verity this pr.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? Yes
If yes, how is the feature documented? A independent doc pr about this feature will be added later.

Note: some ITCases and harness tests are failed because there are some minor issues with the current ForStDB state backend. I'll try to rebase the latest master until all issues are fixed.

LGTM

authors: xishuaidelin xuyangzhong


text:
[FLINK-4613] [ml] Extend ALS to handle implicit feedback datasets

This extension of the ALS algorithm changes some parts of the code if implicitPrefs flag is set to true. Mainly the local parts parts are changed: the Xt * X computation takes into consideration the confidence, thus computing Xt * (C - I) * X instead (see the paper by Hu et al. for details). The Xt * X matrix is precomputed and broadcasted, and that is the only thing that affects distributed execution.
Note, that we use a temporary directory in the test, because there would not be enough memory segments to perform a hash join for prediction. I assume that memory segments are not freed up after the training if no temporary directory is set, but I did not investigate the issue as using a tempdir is a simple workaround.

@gaborhermann @jfeher Could you share results where you benchmarked the algorithm against other implementations, please?

We did not measure performance against Spark or other implementations yet. Those would reflect the performance of Flink ALS implementation, as there is not much difference between the implicit and explicit implementations.
Instead, we compared the implicit case with the explicit case in the Flink implementation on the same datasets, to make sure the implicit case does not decrease the performance significantly. (Of course, we expected the implicit case to be slower due to the extra precomputation and broadcasting of Xt * X.)
        expl  impl
100     8885   9196
1000    7879  11282
10000   8839   9220
100000  7102  10998
1000000 7543  10680

The numbers in the left column indicate the size of the training set (I'm not sure about the measure, but @jfeher can tell about it). The numbers are the training time in milliseconds in the explicit and implicit case respectively. We did the measurements on a small cluster of 3 nodes.
It seems, there is a large constant overhead, but it's not significantly slower in the implicit case.
We could do further, more thorough measurements if needed, but maybe that would be another issue. Benchmarking more and optimizing both the original ALS algorithm and the specific Xt * X computation in the implicit case could be a separate PR.
What are your thoughts on this?

Hello @gaborhermann thank you for your contribution!
Are the numbers here non-zero entries in a matrix?
If that is the case do you think it would be possible to test this on some larger scale datasets?
This would bring it closer to actual use cases someone using Flink might have.

Hi @thvasilo, thanks for reviewing my code.
You are right, @jfeher told me that the numbers in the left column indicate the number of non-zero elements in the rating matrix. We are actually planning to do experiments on a larger data set, but not right now.
Do you think it's important to do so before merging the PR? If so we'll do it ASAP.
Thanks for the suggestion, I marked the PR with the [ml] tag.

I did some minor changes according to user suggestions, and put it in a separate commit (Minor fixes in IALS), so you can see my fixes. Note that before merging I'd prefer to squash the commit with the previous one in order to have proper commits.

@gaborhermann I think have a larger scale test would boost our confidence in the implementation, and maybe point out some problems that do not manifest with small data, which is very common.
If you plan to do it anyway we might as well do it before merging the code, but it's not a blocker.

@gaborhermann @thvasilo I would definitely like to see a test on a larger dataset, that is actually what I was asking for when I mentioned "benchmark", maybe I was not clear then.

Okay, we're on it.
@thvasilo thanks for the dataset suggestion.

Hi, we have measured the training time of als and ials with the given dataset.
After filtering the data to unique item user pairs we got approximatly 64 million rankings.
We measured on a cluster with four nodes and on yarn. All of the nodes had 16 GB of memory.
The taskmanagers got 12 GB and the jobmanager got 2 GB.
We had four taskmanagers, one four each node.
After some testing it looked like a block number between 100 and 1500 is the most optimal.
And between 100 and 300 the running times were steadily low.
For ials we got the following measurments:
The average time for block numbers between 100 and 1500 and 1 iteration in seconds: 2000.33s
The average time for block numbers between 100 and 300 and 1 iteration in seconds: 1729.44s
More detailed results by block sizes on the diagram: http://imgur.com/LjJavti
For als with the same configurations we got the following measurments:
The average time for block numbers between 100 and 1500 and 1 iteration in seconds: 1694.04s
The average time for block numbers between 100 and 300 and 1 iteration in seconds: 1465.77s
So the ials version was 300 s slower on this data than the als.
When we increased the iteration number for 10 the time difference stayed under 1000 s which is less than ten times 300.
This is because the fix time cost for the whole training is big.

Thank you @jfeher!
Could you clarify what you mean by filtering the data to get unique item-user pairs? Is this because the iALS algorithm only supports binary interactions (i.e. number of interactions does not play a role)?
Any idea how these runtime numbers compare to alternative implementations (Spark, Cython)?

Thanks @jfeher for the measurements! :)
@thvasilo The filtering referred to having distinct (user,artist) pairs. It's only because the input of the iALS is a sparse matrix, and it would not make much sense to have more than one values for the same element of the matrix. E.g. to aggregate multiple listenings for the same (user,artist) pair, one could count them, and use the count as the implicit rating. We simply used the value 1.0 for every user-artist pair, but the algorithm works with any (positive) values, not only binary interactions.
We've only measured Flink against itself, as the main ALS algorithm is already in Flink. It would be interesting to measure against Spark and other solutions, but that might not reflect the performance of our iALS extension, but rather the performance of ALS itself. That seems to be another issue for me. Do I see this right?

Hello @gaborhermann,
Yes I think you are right in that respect, just wanted to note that we should perform some comparative benchmarks in the future.
So the benchmarks look good IMHO, we now need to address the couple of comments I had, namely splitting up the tests and deciphering why a java.Iterable was used in that spot if possible.
I was also wondering: For the expected results in the test, where did you get the reference data?

Hi @thvasilo,
Thanks for your thoughts! I agree we should perform a benchmark in the future. Furthermore, based on the results we could optimize the algorithm.
I split up the test, and rebased to the current master. I checked the java.Iterable again, and commented at your original concern. I am afraid we'll have to use the java.Iterable.
Regarding the expected results, I've only generated the small input data by hand. Before that I checked whether the Spark and Flink implementations converged to approximately same factor matrices (I only checked the value of the objective function, not the whole matrices). Because of the random initialization we cannot guarantee to have the same results, but there were 2-3 points that both Spark and Flink converged to.
There might be better methods for testing, but I considered this sufficient as the original ALSITSuite did nothing more. Of course, this test only checks whether the algorithm works the same way after some modifications (e.g. optimization), and does not check whether the algorithm initially works or not, but it's the same case with the original ALS. Do you know what is the assurance for the explicit ALS working good? (It must be good, as I also checked the results of the explicit ALS against Spark on toy-data.) AFAIK Spark generates random matrices of known rank, factorizes them, and checks whether the error is low (see their ALSSuite). In the future, it might be worth to follow their approach.

@gaborhermann Yup the approach taken by the Spark community for testing is closer to what we would like to have for non-deterministic algorithms, but what you have implemented now should suffice on the assumption that the ALS implementation is correct.
@tillrohrmann Initially implemented ALS so I'm not sure how he arrived at the expected results. It would be a good idea for the future to document how we generate test data so it's easy to replicate and validate the process. That should be enough for deterministic algorithms, and for non-deterministic we should have proxies like measuring the error of reconstruction etc.
I'll take a look at the code again now, and will add comments if I find something. Otherwise I hope @mbalassi can find some time to review and merge if no objections come up.

Okay. Thank you again @thvasilo for reviewing our code!

@thvasilo you're right in the future we should make sure that we properly document how we came up with the testing data.
What I've done to come up with the testing data for the ALSITSuite is to factorize the given recommendation matrix with Matlab with the given parameters (factors, iterations, etc). I've implemented ALS in Matlab for that. Ideally, we use a library for that so that others can reproduce the test results. I can share the ALS implementation if you like.
Ideally we do something similar for iALS.

@tillrohrmann thanks for clarifying!
Beside testing against Spark, I also did a small NumPy implementation, to make sure iALS works well. Although, I did not make efforts to imitate the initial random factor generation of Flink, so the results cannot be exactly the same. I am also happy to share that if needed.

For tests it would be good to fix the random seed at least. Otherwise we might not have stable tests depending on the seed.

I agree. I already fixed the seed at the ImplicitALSITSuite. At the ALSITSuite the seed is unset, but by default it's 0, that's why ALSITSuite is deterministic, so I don't think it's a problem. Although, I would reconsider setting the default seed, because that way two training for the same parameters yields the same result, and that might not be what do user expects. The user might expect truly randomized result.
Do you think we should modify the default optional seed Some(0L) to be None, and use the system random generator by default? Again, this is in the original algorithm. Should we do it in this PR or create another issue for refactoring ALS?

Yes I think you're right. We shouldn't fix the seed per default. I think you can include this change in this PR since it's a minor change.

Okay, I've made the seed optional. I also removed the generateRandomMatrix function as it wasn't used at all.

Thank you @thvasilo for your thorough review :)
@mbalassi I think this PR is ready to merge. Could you do a review when you have some time?

Yes @gaborhermann , I finally got here. 😄

Closing since flink-ml is effectively frozen.

authors: gaborhermann jfeher mbalassi thvasilo tillrohrmann zentol


text:
[FLUME-3468] fix default value of passwordFile property when userName is set for jms source

In jms source, when passwordFile property in agent config is not set, currently null value is getting passed which causes exception due to password = Optional.absent(); in https://github.com/apache/flume/blob/trunk/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/JMSSource.java#L173.
This PR, sets default value to empty string in case passwordFile property is not set.

@rgoers can u please review this

repo: apache/flume


text:
GEODE-7003: Fix flaky tests in GemFireTransactionDataSourceIntegratio…

…nTest
There is a race condition within the test. When the test calls dataSource.getConnection(), that method will throw an (expected) exception and expire the connection from the connection pool. There is a background thread which goes through all the expired connections and calls Connection.close(). If this background thread calls close() while the main thread is in the middle of calling BrokeredConnection.isClosed(), the method isClosed() can throw a SQLNonTransientConnectionException.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[x] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[x] Is your initial contribution a single, squashed commit?

[x] Does gradlew build run cleanly?

[x] Have you written or updated unit tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.

@kirklund @agingade please review

repo: apache/geode


text:
[Subtask] add more table event for Iceberg REST server

Describe the subtask
add more table event for Iceberg REST server
Parent issue
#5204

Fixed by #5156

source: gharchive/issue

repo: apache/gravitino


text:
HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields

Rebased on branch-3.1

i just realized that git PR doesn't trigger precommit build for branch-3.1. It's only for trunk. See HADOOP-16035 for details.
As far as I can tell this commit compiles. So +1

authors: jojochuang smengcl

repo: apache/hadoop


text:
HADOOP-17868. Add more tests for BuiltInGzipCompressor

We added BuiltInGzipCompressor recently. It is better to add more compatibility tests for the compressor.

Addressed the comments. Use AssertJ assertions and add exception message.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 38s

trunk passed

+1 :green_heart:
compile
22m 44s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 58s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  3s

trunk passed

+1 :green_heart:
mvnsite
1m 42s

trunk passed

+1 :green_heart:
javadoc
1m  6s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 41s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 50s

trunk passed

+1 :green_heart:
shadedclient
19m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 58s

the patch passed

+1 :green_heart:
compile
23m 54s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
23m 54s

the patch passed

+1 :green_heart:
compile
20m 44s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
20m 44s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 59s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 9 new + 55 unchanged - 0 fixed = 64 total (was 55)

+1 :green_heart:
mvnsite
1m 36s

the patch passed

+1 :green_heart:
javadoc
1m  3s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 37s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 35s

the patch passed

+1 :green_heart:
shadedclient
18m 32s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 59s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

192m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux d90836d5231e 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 0be10caee318d571e217627d9c4fa81b5682aed5

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/2/testReport/

Max. process+thread count
1273 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@steveloughran @sunchao Please take another look. Thanks!

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  3s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 20s

trunk passed

+1 :green_heart:
compile
22m 51s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 21s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 59s

trunk passed

+1 :green_heart:
mvnsite
1m 31s

trunk passed

+1 :green_heart:
javadoc
1m  2s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 31s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 24s

trunk passed

+1 :green_heart:
shadedclient
18m 23s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m  8s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m  8s

the patch passed

+1 :green_heart:
compile
19m 34s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 34s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 59s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 8 new + 55 unchanged - 0 fixed = 63 total (was 55)

+1 :green_heart:
mvnsite
1m 29s

the patch passed

+1 :green_heart:
javadoc
1m  1s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 34s

the patch passed

+1 :green_heart:
shadedclient
18m 23s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 10s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 49s

The patch does not generate ASF License warnings.

189m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux c6bcae0cd4bd 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ec97e9b4424f1f1ec2e4b0ed670ac0afb8a72160

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/3/testReport/

Max. process+thread count
1275 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 19s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 30s

trunk passed

+1 :green_heart:
compile
22m 57s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 22s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  0s

trunk passed

+1 :green_heart:
mvnsite
1m 30s

trunk passed

+1 :green_heart:
javadoc
1m  1s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 31s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 24s

trunk passed

+1 :green_heart:
shadedclient
18m 18s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m  6s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m  6s

the patch passed

+1 :green_heart:
compile
19m 17s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 17s

the patch passed

+1 :green_heart:
blanks
0m  1s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 59s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 8 new + 55 unchanged - 0 fixed = 63 total (was 55)

+1 :green_heart:
mvnsite
1m 30s

the patch passed

+1 :green_heart:
javadoc
1m  0s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 33s

the patch passed

+1 :green_heart:
shadedclient
18m 36s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m  7s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

189m 14s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux d957c0992b16 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 09b60c6625d23b85717944dde20bf5dbc7ceb7dd

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/4/testReport/

Max. process+thread count
2060 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@steveloughran Is this good for you now? Please take another look. Thanks.

kindly ping @steveloughran

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  1s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 15s

trunk passed

+1 :green_heart:
compile
22m 47s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 23s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 59s

trunk passed

+1 :green_heart:
mvnsite
1m 32s

trunk passed

+1 :green_heart:
javadoc
1m  4s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 34s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 25s

trunk passed

+1 :green_heart:
shadedclient
18m 12s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
18m 32s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m 14s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 14s

the patch passed

+1 :green_heart:
compile
19m 29s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 29s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 59s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 8 new + 56 unchanged - 0 fixed = 64 total (was 56)

+1 :green_heart:
mvnsite
1m 30s

the patch passed

+1 :green_heart:
javadoc
0m 59s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 35s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 33s

the patch passed

+1 :green_heart:
shadedclient
18m 24s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 10s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

189m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 1335e73c921a 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / cf1baeadec52f540ef65a70776e7f3654f29f0d2

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/5/testReport/

Max. process+thread count
1518 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

As this is test-only change, as I think I should address all comments. If @steveloughran is too busy to response, could we just move forward? @sunchao

LGTM. @viirya could you address the check style issues - I'll merge this after

Thanks @sunchao ! Updated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 59s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 12s

trunk passed

+1 :green_heart:
compile
23m  1s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 27s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  2s

trunk passed

+1 :green_heart:
mvnsite
1m 31s

trunk passed

+1 :green_heart:
javadoc
1m  3s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 37s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 25s

trunk passed

+1 :green_heart:
shadedclient
23m 51s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
24m 11s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 55s

the patch passed

+1 :green_heart:
compile
22m 12s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 12s

the patch passed

+1 :green_heart:
compile
19m 28s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  0s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 2 new + 56 unchanged - 0 fixed = 58 total (was 56)

+1 :green_heart:
mvnsite
1m 29s

the patch passed

+1 :green_heart:
javadoc
1m  0s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 31s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 34s

the patch passed

+1 :green_heart:
shadedclient
23m 56s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m  5s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 49s

The patch does not generate ASF License warnings.

201m  1s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 377f318073fb 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ae3e8e0a462cc69323fdae4a70025b750a3da49b

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/6/testReport/

Max. process+thread count
3137 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@viirya there are still two more tiny style issues

Fixed. Weird, I didn't see the two in last report.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  8s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 48s

trunk passed

+1 :green_heart:
compile
22m 58s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 34s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  1s

trunk passed

+1 :green_heart:
mvnsite
1m 31s

trunk passed

+1 :green_heart:
javadoc
1m  2s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 34s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 24s

trunk passed

+1 :green_heart:
shadedclient
23m 42s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
24m  3s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m 11s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 11s

the patch passed

+1 :green_heart:
compile
19m 27s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  0s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 1 new + 56 unchanged - 0 fixed = 57 total (was 56)

+1 :green_heart:
mvnsite
1m 30s

the patch passed

+1 :green_heart:
javadoc
1m  0s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 36s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 35s

the patch passed

+1 :green_heart:
shadedclient
23m 31s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 11s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 49s

The patch does not generate ASF License warnings.

201m 23s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 308dcd4ec670 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 217a21e156c170eb15ce320e8fee3d95e0168238

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/7/testReport/

Max. process+thread count
3137 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 23s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
40m 59s

trunk passed

+1 :green_heart:
compile
29m 49s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
27m  2s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 26s

trunk passed

+1 :green_heart:
mvnsite
1m 56s

trunk passed

+1 :green_heart:
javadoc
1m 13s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 45s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 54s

trunk passed

+1 :green_heart:
shadedclient
26m  4s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
26m 25s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m 16s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 16s

the patch passed

+1 :green_heart:
compile
19m 31s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 31s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 59s

the patch passed

+1 :green_heart:
mvnsite
1m 30s

the patch passed

+1 :green_heart:
javadoc
0m 59s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 33s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 34s

the patch passed

+1 :green_heart:
shadedclient
24m 30s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 11s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 50s

The patch does not generate ASF License warnings.

227m 17s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3336

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 0bc571ef90de 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e91252b293dea4f7ef7993099a9ecf8b14559c00

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/8/testReport/

Max. process+thread count
1374 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3336/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

No style issue anymore.

Merged to trunk. Thanks @viirya and @steveloughran !

Thank you @sunchao @steveloughran !

authors: hadoop-yetus sunchao viirya


text:
YARN-11011. Make YARN Router throw Exception to client clearly

Description of PR
When router submits job to yarn, the job is rejected by yarn for some reason. Yarn router just throw exception to log and return not enough info to client.
The router always throws the following non-obvious error to client:
Exception in thread "main" org.apache.hadoop.yarn.server.federation.policies.exceptions.FederationPolicyException:
No active SubCluster available to submit the request.
To make it easy for the user to locate the cause of a job submission failure, we need to make Router throw the true cause of the error to the client clearly.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 50s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m  6s

trunk passed

+1 :green_heart:
compile
0m 36s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 31s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 27s

trunk passed

+1 :green_heart:
mvnsite
0m 38s

trunk passed

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 30s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
0m 55s

trunk passed

+1 :green_heart:
shadedclient
20m 37s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 27s

the patch passed

+1 :green_heart:
compile
0m 25s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 25s

the patch passed

+1 :green_heart:
compile
0m 23s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 23s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 16s

the patch passed

+1 :green_heart:
mvnsite
0m 26s

the patch passed

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 20s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
0m 55s

the patch passed

+1 :green_heart:
shadedclient
20m 18s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
2m 42s
/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt
hadoop-yarn-server-router in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

88m 41s

Reason
Tests

Failed junit tests
hadoop.yarn.server.router.clientrm.TestFederationClientInterceptorRetry

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3701/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3701

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux f3d150ad3813 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 480cc2c0b711df9619b26335969d976f724c7c2a

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3701/1/testReport/

Max. process+thread count
734 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3701/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.


text:
HDFS-16704. Datanode return empty response instead of NPE for GetVolumeInfo during restarting

Description of PR
During datanode starting, I found some NPE in logs:
Caused by: java.lang.NullPointerException: Storage not yet initialized
    at org.apache.hadoop.thirdparty.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:899)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.getVolumeInfo(DataNode.java:3533)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
    at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
    at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)
    at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)

Because the storage of datanode not yet initialized when we trying to get metrics of datanode, and related code as below:
@Override // DataNodeMXBean
public String getVolumeInfo() {
  Preconditions.checkNotNull(data, "Storage not yet initialized");
  return JSON.toString(data.getVolumeInfoMap());
}

The logic is ok, but I feel that the more reasonable logic should be return an empty response instead of NPE, because InfoServer will be started before initBlockPool.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 35s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
38m  3s

trunk passed

+1 :green_heart:
compile
1m 44s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
compile
1m 37s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
1m 25s

trunk passed

+1 :green_heart:
mvnsite
1m 45s

trunk passed

+1 :green_heart:
javadoc
1m 26s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 45s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 38s

trunk passed

+1 :green_heart:
shadedclient
23m  9s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 23s

the patch passed

+1 :green_heart:
compile
1m 27s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javac
1m 27s

the patch passed

+1 :green_heart:
compile
1m 21s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
1m 21s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  0s

the patch passed

+1 :green_heart:
mvnsite
1m 25s

the patch passed

+1 :green_heart:
javadoc
0m 59s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 24s

the patch passed

+1 :green_heart:
shadedclient
22m 51s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
237m 31s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m 15s

The patch does not generate ASF License warnings.

347m 38s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4661

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 776d4764cb81 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7ce6d940f9e0112b441a66bc6948e8895db0330e

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/1/testReport/

Max. process+thread count
3331 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@ayushtkn Thank you very much for helping me review it.
Yes, you are right, we should return an empty value for this case. How about change it same with getDiskBalancerStatus, such as:
@Override // DataNodeMXBean
  public String getVolumeInfo() {
    if (data == null) {
      LOG.debug("Storage not yet initialized.");
      return ""; // it's different with JSON.toString(new HashMap<String, Object>())
    }
    return JSON.toString(data.getVolumeInfoMap());
  }

Developers or SREs are very sensitive to NPE, so I feel we shouldn't out put it in this expected situation.

Extra Stuff: This log line just below is wrong, incorrect placeholder, can fix in a separate jira if interested. :-)
Copy, sir. I will fix it in a separate Jira.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 38s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
39m 45s

trunk passed

+1 :green_heart:
compile
1m 39s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
compile
1m 27s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
1m 21s

trunk passed

+1 :green_heart:
mvnsite
1m 33s

trunk passed

+1 :green_heart:
javadoc
1m 13s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 40s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 47s

trunk passed

+1 :green_heart:
shadedclient
23m 53s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 19s

the patch passed

+1 :green_heart:
compile
1m 22s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javac
1m 22s

the patch passed

+1 :green_heart:
compile
1m 15s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
1m 15s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  0s

the patch passed

+1 :green_heart:
mvnsite
1m 28s

the patch passed

+1 :green_heart:
javadoc
0m 58s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 24s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 38s

the patch passed

+1 :green_heart:
shadedclient
25m 31s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
236m 38s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m 15s

The patch does not generate ASF License warnings.

350m 10s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4661

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 23bc297674f0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 69287015aa98557ef950041a8a4defe7613417dc

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/2/testReport/

Max. process+thread count
3058 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@ZanderXu Thanks for your contribution. IIRC, there is also some other precondition checks before initialized, Would you mind to give another check? such as org.apache.hadoop.hdfs.server.datanode.DataNode#initReplicaRecovery or  replicaHandler = datanode.data.createRbw(storageType, storageId, block, allowLazyPersist); (which datanode.data is null before DataNode initialized) at here.

@Hexiaoqiao Thanks for your review.

Would you mind to give another check?

Sure, I will check them. BTW, If it's ok, can I fix them in a new PR? I hope this PR is just used to fix NPE in getVolumeInfo().

Are we good here? or Waiting for some more modifications?

Yes, we are here, and I have create a new PR-4699 to fix other cases in DataNode.java.
@ayushtkn @Hexiaoqiao Can you help me review them and give me a suggestion that we fix them in two PR or one?

@Hexiaoqiao @ayushtkn Master, ping, please help me merge this PR and review PR-4699? Thanks

Committed to trunk. Thanks @ZanderXu for your contribution, Thanks @ayushtkn for your reviews.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 36s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
38m  1s

trunk passed

+1 :green_heart:
compile
1m 43s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
compile
1m 38s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
1m 27s

trunk passed

+1 :green_heart:
mvnsite
1m 47s

trunk passed

+1 :green_heart:
javadoc
1m 24s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 50s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 43s

trunk passed

+1 :green_heart:
shadedclient
23m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 23s

the patch passed

+1 :green_heart:
compile
1m 27s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javac
1m 27s

the patch passed

+1 :green_heart:
compile
1m 20s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
1m 20s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  2s

the patch passed

+1 :green_heart:
mvnsite
1m 26s

the patch passed

+1 :green_heart:
javadoc
0m 59s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 29s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 25s

the patch passed

+1 :green_heart:
shadedclient
22m 43s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
237m  9s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m 15s

The patch does not generate ASF License warnings.

347m 10s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4661

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux f2e8f1a1b377 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2af6c2bf0f3163ade7457a10f3b29e6afc34906e

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/3/testReport/

Max. process+thread count
3385 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4661/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

authors: Hexiaoqiao ZanderXu ayushtkn hadoop-yetus


text:
HADOOP-18404: Update broken link in Shell.java

Fix link location

Description of PR
How was this patch tested?
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 42s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
38m 41s

trunk passed

+1 :green_heart:
compile
23m  9s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
compile
20m 49s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
checkstyle
1m 43s

trunk passed

+1 :green_heart:
mvnsite
2m  7s

trunk passed

+1 :green_heart:
javadoc
1m 49s

trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m 22s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
3m 12s

trunk passed

+1 :green_heart:
shadedclient
23m 54s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  4s

the patch passed

+1 :green_heart:
compile
22m 21s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javac
22m 21s

the patch passed

+1 :green_heart:
compile
21m 54s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
javac
21m 54s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 33s

the patch passed

+1 :green_heart:
mvnsite
2m  2s

the patch passed

+1 :green_heart:
javadoc
1m 29s

the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

+1 :green_heart:
javadoc
1m  9s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
spotbugs
2m 58s

the patch passed

+1 :green_heart:
shadedclient
23m 49s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
19m 41s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
1m 32s

The patch does not generate ASF License warnings.

218m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4718/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4718

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux ddaad878cecd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 04dbe8c2565ebe68d235cdb8f4ce488dd3d4bd24

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4718/1/testReport/

Max. process+thread count
1263 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4718/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Amended.

authors: hadoop-yetus paulk-asert


text:
HADOOP-18470. 3.3.5 Release wrap-up: jdiff files

This is a commit with the final changes for the 3.3.5 release
-updating later branches with the jdiff and release info

add jdiff xml files from 3.3.5 release
declare 3.3.5 as the latest stable release
copy release notes.

For code changes:

[X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 34s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
28m 51s

trunk passed

+1 :green_heart:
compile
25m 24s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
21m 41s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
mvnsite
11m 22s

trunk passed

+1 :green_heart:
javadoc
6m 42s

trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
6m  5s

trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
shadedclient
138m 38s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 23s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
9m  2s

the patch passed

+1 :green_heart:
compile
24m 48s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
24m 48s

the patch passed

+1 :green_heart:
compile
21m 40s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
javac
21m 40s

the patch passed

-1 :x:
blanks
0m  1s
/blanks-eol.txt
The patch has 3000 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-1 :x:
blanks
0m  1s
/blanks-tabs.txt
The patch 5 line(s) with tabs.

+1 :green_heart:
mvnsite
10m 55s

the patch passed

+1 :green_heart:
javadoc
6m 35s

the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
6m  6s

the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

+1 :green_heart:
shadedclient
49m  2s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 17s

hadoop-project-dist in the patch passed.

+1 :green_heart:
unit
18m 28s

hadoop-common in the patch passed.

+1 :green_heart:
unit
224m 13s

hadoop-hdfs in the patch passed.

+1 :green_heart:
unit
239m  3s

hadoop-yarn in the patch passed.

-1 :x:
unit
172m 35s
/patch-unit-hadoop-mapreduce-project.txt
hadoop-mapreduce-project in the patch passed.

+1 :green_heart:
asflicense
1m 20s

The patch does not generate ASF License warnings.

903m 23s

Reason
Tests

Failed junit tests
hadoop.mapreduce.v2.TestMRJobsWithProfiler

hadoop.mapreduce.v2.TestMRJobs

hadoop.mapreduce.v2.TestUberAM

Subsystem
Report/Notes

Docker
ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5558/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5558

Optional Tests
dupname asflicense codespell detsecrets xmllint mvnsite markdownlint compile javac javadoc mvninstall unit shadedclient

uname
Linux 725520ed497d 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e2d2dee1e66c87fc0b6a816d56a35868ac40ae9d

Default Java
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5558/1/testReport/

Max. process+thread count
2459 (vs. ulimit of 5500)

modules
C: hadoop-project-dist hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5558/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

authors: hadoop-yetus steveloughran


text:
HAWQ-1690. Add common utility used by ORC format

Add common utility used by ORC format

+1

+1

authors: huor stanlyxiang wcl14

repo: apache/hawq


text:
HBASE-24007 Get -PrunLargeTests passing on JDK11

Minor tweaks required to get passing runs of -PrunLargeTests.

Minimum Hadoop version is 3.2.0 due to HADOOP-12760.
JDK11 looks like it consumes more memory than JDK8, so failures due to OOME see more common here. Bumping heap allocated to surefire forks allows better pass rate.

Anyone know of a better way of managing the minimum hadoop version dependency? Some enforcer-plugin foo?

We've seen performance issue with JDK11 too. They took extra time to run that breaks our internal functional tests. Later we realized it's because JDK11 defaults to G1GC and thus the memory consumption is different/more

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 17s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

_ Patch Compile Tests _

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  2s
The patch has no ill-formed XML file.

+1 :green_heart:
hadoopcheck
12m 39s
Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.

_ Other Tests _

+1 :green_heart:
asflicense
0m 16s
The patch does not generate ASF License warnings.

21m 33s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1351

Optional Tests
dupname asflicense hadoopcheck xml

uname
Linux be55078c58c4 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 5e63509781

Max. process+thread count
63 (vs. ulimit of 10000)

modules
C: . U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/console

versions
git=2.17.1 maven=2018-06-17T18:33:14Z)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 40s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
8m 58s
master passed

+1 :green_heart:
compile
5m  8s
master passed

+1 :green_heart:
shadedjars
7m 57s
branch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 24s
root in master failed.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
8m 27s
the patch passed

+1 :green_heart:
compile
5m 17s
the patch passed

+1 :green_heart:
javac
5m 17s
the patch passed

+1 :green_heart:
shadedjars
7m 49s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 21s
root in the patch failed.

_ Other Tests _

-0 :warning:
unit
2m  8s
root in the patch failed.

48m 27s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1351

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 458afc272f14 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 5e63509781

Default Java
2020-01-14

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/testReport/

Max. process+thread count
487 (vs. ulimit of 10000)

modules
C: . U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/console

versions
git=2.17.1 maven=2018-06-17T18:33:14Z)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  9s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m 29s
master passed

+1 :green_heart:
compile
3m 33s
master passed

+1 :green_heart:
shadedjars
5m 33s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
2m 47s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 53s
the patch passed

+1 :green_heart:
compile
3m 29s
the patch passed

+1 :green_heart:
javac
3m 29s
the patch passed

+1 :green_heart:
shadedjars
5m 32s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
2m 47s
the patch passed

_ Other Tests _

-1 :x:
unit
11m 55s
root in the patch failed.

50m 32s

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1351

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 225d91121d52 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 5e63509781

Default Java
1.8.0_232

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/testReport/

Max. process+thread count
870 (vs. ulimit of 10000)

modules
C: . U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/console

versions
git=2.17.1 maven=2018-06-17T18:33:14Z)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

TestVerifyBucketCacheFile strikes again :'(

I was asked offline: the dependency on HADOOP-12760 comes from TestSaslFanOutOneBlockAsyncDFSOutput, which has been on our flakey exclude list for a while.

Bump review request please.

Thanks @HorizonNet !

authors: Apache-HBase jojochuang ndimiduk

repo: apache/hbase


text:
HBASE-24401 Cell size limit check on append should consider 0

or less value to disable the check

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
3m 38s
branch-2 passed

+1 :green_heart:
checkstyle
1m 14s
branch-2 passed

+1 :green_heart:
spotbugs
1m 58s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 10s
the patch passed

+1 :green_heart:
checkstyle
1m 12s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
11m 50s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

+1 :green_heart:
spotbugs
2m  4s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 15s
The patch does not generate ASF License warnings.

33m 14s

Subsystem
Report/Notes

Docker
Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1744/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1744

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 23f01e298e85 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / fbe0da2672

Max. process+thread count
94 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1744/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 20s
Docker mode activated.

-0 :warning:
yetus
0m  6s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2 Compile Tests _

+1 :green_heart:
mvninstall
4m  4s
branch-2 passed

+1 :green_heart:
compile
0m 59s
branch-2 passed

+1 :green_heart:
shadedjars
5m 36s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 38s
branch-2 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 38s
the patch passed

+1 :green_heart:
compile
0m 58s
the patch passed

+1 :green_heart:
javac
0m 58s
the patch passed

+1 :green_heart:
shadedjars
5m 32s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 34s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
204m  0s
hbase-server in the patch passed.

230m  5s

Subsystem
Report/Notes

Docker
Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1744/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1744

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux e11429ecae93 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2 / fbe0da2672

Default Java
1.8.0_232

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1744/1/testReport/

Max. process+thread count
2611 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1744/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Sorry ,i will close this pr.

authors: Apache-HBase wenbang


text:
HBASE-23594 Procedure stuck due to region happen to recorded on two s…

…ervers.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 32s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
5m 26s
master passed

+1 :green_heart:
compile
0m 58s
master passed

+1 :green_heart:
checkstyle
1m 19s
master passed

+1 :green_heart:
shadedjars
4m 36s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 39s
master passed

+0 :ok:
spotbugs
4m 16s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 13s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 57s
the patch passed

+1 :green_heart:
compile
0m 57s
the patch passed

+1 :green_heart:
javac
0m 57s
the patch passed

+1 :green_heart:
checkstyle
1m 17s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
4m 41s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
15m 34s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 34s
the patch passed

+1 :green_heart:
findbugs
4m 25s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
164m 36s
hbase-server in the patch passed.

+1 :green_heart:
asflicense
0m 35s
The patch does not generate ASF License warnings.

222m  6s

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-953/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/953

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux a0c9ee3cd95e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-953/out/precommit/personality/provided.sh

git revision
master / 270eb9886e

Default Java
1.8.0_181

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-953/1/testReport/

Max. process+thread count
4425 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-953/1/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 31s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
5m 32s
master passed

+1 :green_heart:
compile
0m 57s
master passed

+1 :green_heart:
checkstyle
1m 18s
master passed

+1 :green_heart:
shadedjars
4m 37s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 38s
master passed

+0 :ok:
spotbugs
4m 19s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 17s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 55s
the patch passed

+1 :green_heart:
compile
0m 55s
the patch passed

+1 :green_heart:
javac
0m 55s
the patch passed

+1 :green_heart:
checkstyle
1m 18s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
4m 40s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
15m 40s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 37s
the patch passed

+1 :green_heart:
findbugs
4m 22s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
172m 58s
hbase-server in the patch passed.

+1 :green_heart:
asflicense
0m 32s
The patch does not generate ASF License warnings.

230m 17s

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-953/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/953

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 27d459764f32 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-953/out/precommit/personality/provided.sh

git revision
master / 270eb9886e

Default Java
1.8.0_181

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-953/2/testReport/

Max. process+thread count
4468 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-953/2/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


text:
HIVE-26865: Fix TestSQL11ReservedKeyWordsNegative test in branch-3

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

@amanraj2520: when I comment the initializer out, I get the same issue:
  @BeforeClass
  public static void initialize() {
    //SessionState.start(conf);
  }

I'll figure out how is this possible in precommit testing

according to latest run, the problem disappears if I define BeforeClass on the static inner classes in the unit test:
http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3869/2
https://github.com/apache/hive/pull/3869/commits/1f7692d4a5faf70d2f40312a465aca81051af81d
I'm trying to understand why it passes locally even without this hack, but if I cannot make it, we can simply go with this workaround to make further progress towards the next hive 3.x

This is nice @abstractdog Let me try this out and then if no other fix we can go ahead with this

can I get approval on this @zabetak , @ayushtkn ? simple UT fix on branch-3

@zabetak Can you please check and approve this.

Thanks for merging this.

repo: apache/hive


text:
HIVE-26921: Add failover_type and failover_endpoint as new member to metadata column in replication_metrics table

Add failover_type and failover_endpoint as new member to metadata column in replication_metrics table. This would tell us the type of failover(PLANNED/UNPLANNED) and endpoint(SOURCE/TARGET) of failover.

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

+1. LGTM. I'll just leave a note to remember that it's better to remove any magic values.

authors: atsaonerk pudidic


text:
HIVE-27521: Bump bouncycastle (bcprov-jdk15on) to 1.70

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

Is the change a dependency upgrade?

How was this patch tested?

many precommit failures, I guess this remains broken until the next hadoop bump, but haven't checked in detail, keeping this PR open


text:
fix #3242

fix #3242 Parquet File Output Base File Name Field Select

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
[x] Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

[ ] I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004
[ ] In any other case, please file an Apache Individual Contributor License Agreement.

I would like some tests.
What happens when we first send 10 rows to file 1, then 10 to file 2 and then 10 back to file one.

I'll provide some integration tests next week

authors: hansva sramazzina

repo: apache/hop


text:
build(deps): Upgrade datafusion from 0.41 to 0.42

Description
Upgrade datafusion from 0.41 to 0.42 and its dependencies (arrow, pyo3).
Drop support for Python3.8 as it won't receive security updates starting next month.
Bump required python to min. 3.9.
resolves #143
How are the changes test-covered

[ ] N/A
[ ] Automated tests (unit and/or integration tests)
[ ] Manual tests

[ ] Details are described below

Datafusion plan changed a bit:

repo: apache/hudi-rs