text:
HDFS-17374. EC: StripedBlockReader#newConnectedPeer should set SO_TIMEOUT and SO_KEEPALIVE

Description of PR
Refer to HDFS-17374.
more details need time to finish.
How was this patch tested?
For code changes:

[ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 34s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
41m 38s

trunk passed

+1 :green_heart:
compile
1m 19s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
1m 11s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
1m  9s

trunk passed

+1 :green_heart:
mvnsite
1m 18s

trunk passed

+1 :green_heart:
javadoc
1m  5s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
1m 35s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 14s

trunk passed

+1 :green_heart:
shadedclient
34m 20s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  7s

the patch passed

+1 :green_heart:
compile
1m 11s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
1m 11s

the patch passed

+1 :green_heart:
compile
1m  5s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
1m  5s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 57s

the patch passed

+1 :green_heart:
mvnsite
1m  9s

the patch passed

+1 :green_heart:
javadoc
0m 52s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
1m 28s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 16s

the patch passed

+1 :green_heart:
shadedclient
34m 40s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
261m  3s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 42s

The patch does not generate ASF License warnings.

395m 31s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestWriteRead

hadoop.hdfs.TestStripedFileAppend

hadoop.hdfs.TestReadStripedFileWithDecodingCorruptData

hadoop.hdfs.TestDFSStripedOutputStream

hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy

hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy

hadoop.hdfs.TestDFSStripedInputStream

hadoop.hdfs.server.datanode.TestDirectoryScanner

hadoop.hdfs.TestFileAppend2

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6536

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 6495c596dca5 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 8d2e864a1b26d60883428234882ccd9224a8cbdd

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/1/testReport/

Max. process+thread count
3435 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Hexiaoqiao @zhangshuyan0 @tomscut @tasanuma Sir, PTAL when you have free time, thanks a lot~

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 27s

trunk passed

+1 :green_heart:
compile
1m 22s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
1m 18s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
1m 11s

trunk passed

+1 :green_heart:
mvnsite
1m 23s

trunk passed

+1 :green_heart:
javadoc
1m  5s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
1m 31s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 13s

trunk passed

+1 :green_heart:
shadedclient
35m 45s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 15s

the patch passed

+1 :green_heart:
compile
1m 13s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
1m 13s

the patch passed

+1 :green_heart:
compile
1m 13s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
1m 13s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  2s

the patch passed

+1 :green_heart:
mvnsite
1m 13s

the patch passed

+1 :green_heart:
javadoc
0m 51s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
1m 28s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 16s

the patch passed

+1 :green_heart:
shadedclient
36m 37s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
242m 52s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 45s

The patch does not generate ASF License warnings.

384m 23s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy

hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor

hadoop.hdfs.server.namenode.top.window.TestRollingWindow

hadoop.hdfs.TestDFSStripedOutputStream

hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy

hadoop.hdfs.TestReadStripedFileWithDNFailure

hadoop.hdfs.server.namenode.TestAddStripedBlocks

hadoop.hdfs.TestErasureCodingPolicies

hadoop.hdfs.TestDecommissionWithBackoffMonitor

hadoop.hdfs.TestFileChecksum

hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS

hadoop.hdfs.server.namenode.TestFSEditLogLoader

hadoop.hdfs.TestReconstructStripedFile

hadoop.hdfs.server.namenode.TestHostsFiles

hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6536

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux b1487e605ff0 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f775e5a301f6777a42056cbc8fd42dc34ef178e4

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/3/testReport/

Max. process+thread count
3292 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 15s

trunk passed

+1 :green_heart:
compile
1m 18s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
1m 13s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
1m  6s

trunk passed

+1 :green_heart:
mvnsite
1m 20s

trunk passed

+1 :green_heart:
javadoc
1m  2s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
1m 31s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 19s

trunk passed

+1 :green_heart:
shadedclient
35m 37s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  9s

the patch passed

+1 :green_heart:
compile
1m  9s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
1m  9s

the patch passed

+1 :green_heart:
compile
1m  6s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
1m  6s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  3s

the patch passed

+1 :green_heart:
mvnsite
1m 15s

the patch passed

+1 :green_heart:
javadoc
0m 51s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
1m 28s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
3m 30s

the patch passed

+1 :green_heart:
shadedclient
36m 20s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
268m  5s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 48s

The patch does not generate ASF License warnings.

407m 16s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestDecommissionWithStriped

hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocksWithRandomECPolicy

hadoop.hdfs.TestDistributedFileSystemWithECFile

hadoop.hdfs.server.namenode.TestRefreshBlockPlacementPolicy

hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer

hadoop.hdfs.TestDFSStripedInputStream

hadoop.hdfs.tools.TestDFSZKFailoverController

hadoop.hdfs.TestReconstructStripedFile

hadoop.hdfs.server.namenode.TestBlockUnderConstruction

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6536

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 6935c41c5d00 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e89126c724f443cdeac3fdf9873eb25fea9457e0

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/2/testReport/

Max. process+thread count
3211 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6536/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Thanks for the PR, @hfutatzhanghb. It makes sense to me. Did you solve the problem in your cluster with this PR?
Although I don't think that the failed tests are related, just to be sure, I reran CI.

@tasanuma Hi, sir. Thanks for your reviewing.  Now i can not conclude whether it solve this problem on our cluster due to Spring Festival‘s low peak of throughput.  Please allow me to reply you a few days later~ Thanks a lot sir.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

_ Prechecks _

+1 :green_heart:
dupname
0m 01s

No case conflicting files found.

+0 :ok:
spotbugs
0m 00s

spotbugs executables are not available.

+0 :ok:
codespell
0m 00s

codespell was not available.

+0 :ok:
detsecrets
0m 00s

detect-secrets was not available.

+1 :green_heart:
@author
0m 01s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m 00s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
93m 48s

trunk passed

+1 :green_heart:
compile
6m 30s

trunk passed

+1 :green_heart:
checkstyle
5m 09s

trunk passed

+1 :green_heart:
mvnsite
7m 06s

trunk passed

+1 :green_heart:
javadoc
6m 30s

trunk passed

+1 :green_heart:
shadedclient
154m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 59s

the patch passed

+1 :green_heart:
compile
3m 42s

the patch passed

+1 :green_heart:
javac
3m 42s

the patch passed

+1 :green_heart:
blanks
0m 00s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
2m 34s

the patch passed

+1 :green_heart:
mvnsite
4m 43s

the patch passed

+1 :green_heart:
javadoc
3m 47s

the patch passed

+1 :green_heart:
shadedclient
167m 33s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
5m 52s

The patch does not generate ASF License warnings.

444m 11s

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/6536

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
MINGW64_NT-10.0-17763 1b55e27aca49 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys

Build tool
maven

Personality
/c/hadoop/dev-support/bin/hadoop.sh

git revision
trunk / f775e5a301f6777a42056cbc8fd42dc34ef178e4

Default Java
Azul Systems, Inc.-1.8.0_332-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6536/1/testReport/

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6536/1/console

versions
git=2.44.0.windows.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@tasanuma Sir, this problem does not appear in out cluster after we apply this PR.

authors: hadoop-yetus hfutatzhanghb tasanuma


text:
fix misc issues before orc format support

Raise this PR for hour
fix misc issues before orc format support, including typos, bugs, license

+1

source: gharchive/pull-request

repo: apache/hawq

license_type: permissive

license_source: github-api


text:
HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't …

…calculate the distribution for last bucket.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 33s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-2.3 Compile Tests _

+1 :green_heart:
mvninstall
3m 37s
branch-2.3 passed

+1 :green_heart:
checkstyle
0m 15s
branch-2.3 passed

+1 :green_heart:
spotbugs
0m 32s
branch-2.3 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m  7s
the patch passed

+1 :green_heart:
checkstyle
0m 12s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
16m 43s
Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.

+1 :green_heart:
spotbugs
0m 40s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 15s
The patch does not generate ASF License warnings.

32m 48s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2024

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 4e9e65afadce 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.3 / 9af91d4e1f

Max. process+thread count
94 (vs. ulimit of 12500)

modules
C: hbase-hadoop2-compat U: hbase-hadoop2-compat

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 48s
Docker mode activated.

-0 :warning:
yetus
0m  7s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2.3 Compile Tests _

+1 :green_heart:
mvninstall
5m  6s
branch-2.3 passed

+1 :green_heart:
compile
0m 23s
branch-2.3 passed

+1 :green_heart:
shadedjars
7m  3s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 19s
branch-2.3 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 22s
the patch passed

+1 :green_heart:
compile
0m 22s
the patch passed

+1 :green_heart:
javac
0m 22s
the patch passed

+1 :green_heart:
shadedjars
6m 57s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 18s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
0m 31s
hbase-hadoop2-compat in the patch passed.

27m 22s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2024

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux bce949b16934 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.3 / 9af91d4e1f

Default Java
1.8.0_232

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/testReport/

Max. process+thread count
332 (vs. ulimit of 12500)

modules
C: hbase-hadoop2-compat U: hbase-hadoop2-compat

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

-0 :warning:
yetus
0m  7s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2.3 Compile Tests _

+1 :green_heart:
mvninstall
4m 14s
branch-2.3 passed

+1 :green_heart:
compile
0m 20s
branch-2.3 passed

+1 :green_heart:
shadedjars
5m 48s
branch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 21s
hbase-hadoop2-compat in branch-2.3 failed.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 55s
the patch passed

+1 :green_heart:
compile
0m 20s
the patch passed

+1 :green_heart:
javac
0m 20s
the patch passed

+1 :green_heart:
shadedjars
5m 46s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 18s
hbase-hadoop2-compat in the patch failed.

_ Other Tests _

-1 :x:
unit
0m 28s
hbase-hadoop2-compat in the patch failed.

23m  6s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2024

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 3f575e3b6264 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.3 / 9af91d4e1f

Default Java
2020-01-14

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop2-compat.txt

javadoc
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop2-compat.txt

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-hadoop2-compat.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/testReport/

Max. process+thread count
212 (vs. ulimit of 12500)

modules
C: hbase-hadoop2-compat U: hbase-hadoop2-compat

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

authors: Apache-HBase WenFeiYi

repo: apache/hbase


text:
HBASE-25503: HBase code download is failing on windows with invalid path error

renamed HBASE-18070-ROOT_hbase:meta_Region_Replicas.pdf to HBASE-18070-ROOT_hbase_meta_Region_Replicas.pdf

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 28s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

_ Other Tests _

1m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2880

JIRA Issue
HBASE-25503

Optional Tests

uname
Linux 73a5d44cadef 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / dfefff7e59

Max. process+thread count
52 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 28s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

_ Other Tests _

1m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2880

JIRA Issue
HBASE-25503

Optional Tests

uname
Linux 73a5d44cadef 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / dfefff7e59

Max. process+thread count
52 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 26s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  8s
Maven dependency ordering for patch

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

_ Other Tests _

+0 :ok:
asflicense
0m  0s
ASF License check generated no output?

3m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2880

JIRA Issue
HBASE-25503

Optional Tests
dupname asflicense

uname
Linux c55f29c9478d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / dfefff7e59

Max. process+thread count
43 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 26s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  8s
Maven dependency ordering for patch

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

_ Other Tests _

+0 :ok:
asflicense
0m  0s
ASF License check generated no output?

3m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2880

JIRA Issue
HBASE-25503

Optional Tests
dupname asflicense

uname
Linux c55f29c9478d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / dfefff7e59

Max. process+thread count
43 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 53s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 25s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 10s
Maven dependency ordering for patch

_ Other Tests _

3m 25s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2880

JIRA Issue
HBASE-25503

Optional Tests

uname
Linux a05c3d4f3a67 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / dfefff7e59

Max. process+thread count
46 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 53s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 25s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 10s
Maven dependency ordering for patch

_ Other Tests _

3m 25s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2880

JIRA Issue
HBASE-25503

Optional Tests

uname
Linux a05c3d4f3a67 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / dfefff7e59

Max. process+thread count
46 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2880/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

authors: Apache-HBase arshadmohammad


text:
HBASE-22761 Caught ArrayIndexOutOfBoundsException while processing ev…

…ent RS_LOG_REPLAY

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  1s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 52s
master passed

+1 :green_heart:
compile
2m 57s
master passed

+1 :green_heart:
checkstyle
0m 52s
master passed

+1 :green_heart:
spotbugs
1m 59s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 23s
the patch passed

+1 :green_heart:
compile
3m  5s
the patch passed

+1 :green_heart:
javac
3m  5s
the patch passed

+1 :green_heart:
checkstyle
0m 49s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
16m  9s
Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.

+1 :green_heart:
spotbugs
1m 58s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 12s
The patch does not generate ASF License warnings.

43m 13s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4190

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile

uname
Linux d56da9f46ff9 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 10471944bd

Default Java
AdoptOpenJDK-1.8.0_282-b08

Max. process+thread count
60 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  7s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 43s
master passed

+1 :green_heart:
compile
0m 49s
master passed

+1 :green_heart:
shadedjars
4m 19s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 33s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 26s
the patch passed

+1 :green_heart:
compile
0m 51s
the patch passed

+1 :green_heart:
javac
0m 51s
the patch passed

+1 :green_heart:
shadedjars
4m 28s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 29s
the patch passed

_ Other Tests _

-1 :x:
unit
313m 10s
hbase-server in the patch failed.

334m 29s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4190

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 81e73202d343 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 10471944bd

Default Java
AdoptOpenJDK-11.0.10+9

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/testReport/

Max. process+thread count
2387 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 46s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m  6s
master passed

+1 :green_heart:
compile
0m 42s
master passed

+1 :green_heart:
shadedjars
4m 15s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 27s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 42s
the patch passed

+1 :green_heart:
compile
0m 40s
the patch passed

+1 :green_heart:
javac
0m 40s
the patch passed

+1 :green_heart:
shadedjars
4m 15s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 24s
the patch passed

_ Other Tests _

-1 :x:
unit
344m 14s
hbase-server in the patch failed.

364m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4190

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 756dfcf8b9b2 5.4.0-1047-aws #49~18.04.1-Ubuntu SMP Wed Apr 28 23:08:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 10471944bd

Default Java
AdoptOpenJDK-1.8.0_282-b08

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/testReport/

Max. process+thread count
3042 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

The PR is for another issue?

The PR is for another issue?

Thanks, @Apache9 , I have changed the PR issue.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m  0s
Docker mode activated.

-1 :x:
docker
0m  0s
Docker command '/usr/bin/docker' is too old (1.41 < API v 1.27.0).

Subsystem
Report/Notes

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/console

versions
git=2.17.1

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --quick-hadoopcheck --exclude-tests-url --asf-nightlies-general-check-base --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m 11s
master passed

+1 :green_heart:
compile
0m 35s
master passed

+1 :green_heart:
checkstyle
0m 35s
master passed

+1 :green_heart:
spotbugs
1m 18s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 37s
the patch passed

+1 :green_heart:
compile
0m 34s
the patch passed

+1 :green_heart:
javac
0m 34s
the patch passed

+1 :green_heart:
checkstyle
0m 33s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
spotbugs
1m 18s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m  8s
The patch does not generate ASF License warnings.

13m 40s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4190

Optional Tests
dupname asflicense javac spotbugs checkstyle compile

uname
Linux 47d91c3964be 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / a49d147d49

Default Java
AdoptOpenJDK-1.8.0_282-b08

Max. process+thread count
57 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 47s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck --skip-errorprone --exclude-tests-url --build-thread --surefire-first-part-fork-count --surefire-second-part-fork-count

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
5m 48s
master passed

+1 :green_heart:
compile
0m 55s
master passed

+1 :green_heart:
javadoc
0m 32s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 57s
the patch passed

+1 :green_heart:
compile
0m 48s
the patch passed

+1 :green_heart:
javac
0m 48s
the patch passed

+1 :green_heart:
javadoc
0m 30s
the patch passed

_ Other Tests _

-1 :x:
unit
143m  4s
hbase-server in the patch failed.

154m 42s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4190

Optional Tests
javac javadoc unit compile

uname
Linux 70012f3d7c1a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / a49d147d49

Default Java
AdoptOpenJDK-11.0.10+9

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/testReport/

Max. process+thread count
2345 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4190/2/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Use consumeLock is a bit expensive in normal write path.
Let me see if we could fix it by checking whether we have the same writer instance, as described on the jira issue.

Use consumeLock is a bit expensive in normal write path.
Let me see if we could fix it by checking whether we have the same writer instance, as described on the jira issue.

OK, thanks, @Apache9 , looking forward to it.

authors: Apache-HBase Apache9 sunhelly


text:
HBASE-28656 Optimize the verifyCopyResult logic in ExportSnapshot

https://issues.apache.org/jira/browse/HBASE-28656

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 32s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m  0s
master passed

+1 :green_heart:
compile
0m 34s
master passed

+1 :green_heart:
checkstyle
0m 14s
master passed

+1 :green_heart:
spotless
0m 45s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 33s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 45s
the patch passed

+1 :green_heart:
compile
0m 32s
the patch passed

+1 :green_heart:
javac
0m 32s
the patch passed

+1 :green_heart:
checkstyle
0m 13s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
4m 57s
Patch does not cause any errors with Hadoop 3.3.6.

+1 :green_heart:
spotless
0m 43s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 39s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 11s
The patch does not generate ASF License warnings.

21m 46s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux b547fcfc558e 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 60c46118e2

Default Java
Eclipse Adoptium-11.0.17+8

Max. process+thread count
80 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/console

versions
git=2.34.1 maven=3.8.6 spotbugs=4.7.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 11s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
2m 37s
master passed

+1 :green_heart:
compile
0m 19s
master passed

+1 :green_heart:
shadedjars
5m 12s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 15s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 27s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 18s
the patch passed

+1 :green_heart:
shadedjars
5m  8s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
the patch passed

_ Other Tests _

-1 :x:
unit
18m 31s
hbase-mapreduce in the patch failed.

36m 19s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 151870acb877 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 60c46118e2

Default Java
Temurin-1.8.0_352-b08

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-mapreduce.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/testReport/

Max. process+thread count
3200 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 25s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m  7s
master passed

+1 :green_heart:
compile
0m 18s
master passed

+1 :green_heart:
shadedjars
5m 37s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 49s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 18s
the patch passed

+1 :green_heart:
shadedjars
5m 37s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
the patch passed

_ Other Tests _

-1 :x:
unit
18m 49s
hbase-mapreduce in the patch failed.

38m 16s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 42b4e486e922 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 60c46118e2

Default Java
Eclipse Adoptium-11.0.17+8

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-mapreduce.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/testReport/

Max. process+thread count
3256 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
2m 28s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 13s
master passed

+1 :green_heart:
compile
0m 19s
master passed

+1 :green_heart:
shadedjars
5m 41s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 57s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 19s
the patch passed

+1 :green_heart:
shadedjars
5m 43s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 12s
the patch passed

_ Other Tests _

-1 :x:
unit
17m 50s
hbase-mapreduce in the patch failed.

40m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux df267b850378 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 60c46118e2

Default Java
Eclipse Adoptium-17.0.10+7

unit
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-mapreduce.txt

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/testReport/

Max. process+thread count
3292 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/1/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Please check the failed UTs?

@Apache9 Ok. Additionally, I will add a UT to cover the code.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 32s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m  1s
master passed

+1 :green_heart:
compile
0m 35s
master passed

+1 :green_heart:
checkstyle
0m 14s
master passed

+1 :green_heart:
spotless
0m 44s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 34s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 41s
the patch passed

+1 :green_heart:
compile
0m 32s
the patch passed

+1 :green_heart:
javac
0m 32s
the patch passed

+1 :green_heart:
checkstyle
0m 13s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
4m 54s
Patch does not cause any errors with Hadoop 3.3.6.

+1 :green_heart:
spotless
0m 43s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 39s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 11s
The patch does not generate ASF License warnings.

21m 32s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux f934e85c4c01 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Eclipse Adoptium-11.0.17+8

Max. process+thread count
79 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/console

versions
git=2.34.1 maven=3.8.6 spotbugs=4.7.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 14s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
2m 23s
master passed

+1 :green_heart:
compile
0m 19s
master passed

+1 :green_heart:
shadedjars
5m  9s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 29s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 18s
the patch passed

+1 :green_heart:
shadedjars
5m 13s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m 29s
hbase-mapreduce in the patch passed.

34m 56s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux cc9a6ab984cc 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Temurin-1.8.0_352-b08

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/testReport/

Max. process+thread count
3232 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 29s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 13s
master passed

+1 :green_heart:
compile
0m 18s
master passed

+1 :green_heart:
shadedjars
5m 40s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 53s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 18s
the patch passed

+1 :green_heart:
shadedjars
5m 37s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
16m 30s
hbase-mapreduce in the patch passed.

36m 25s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux bb9cfe544d41 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Eclipse Adoptium-17.0.10+7

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/testReport/

Max. process+thread count
3235 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 27s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m  3s
master passed

+1 :green_heart:
compile
0m 17s
master passed

+1 :green_heart:
shadedjars
5m 37s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 45s
the patch passed

+1 :green_heart:
compile
0m 19s
the patch passed

+1 :green_heart:
javac
0m 19s
the patch passed

+1 :green_heart:
shadedjars
5m 36s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 12s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m 40s
hbase-mapreduce in the patch passed.

37m 14s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 74c7c083efb1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Eclipse Adoptium-11.0.17+8

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/testReport/

Max. process+thread count
3215 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/2/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 35s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 47s
master passed

+1 :green_heart:
compile
0m 50s
master passed

+1 :green_heart:
checkstyle
0m 17s
master passed

+1 :green_heart:
spotless
1m  2s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 39s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 46s
the patch passed

+1 :green_heart:
compile
0m 33s
the patch passed

+1 :green_heart:
javac
0m 33s
the patch passed

+1 :green_heart:
checkstyle
0m 13s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
4m 48s
Patch does not cause any errors with Hadoop 3.3.6.

+1 :green_heart:
spotless
0m 41s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 40s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 10s
The patch does not generate ASF License warnings.

23m 12s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux 57d484e87e32 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Eclipse Adoptium-11.0.17+8

Max. process+thread count
80 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/console

versions
git=2.34.1 maven=3.8.6 spotbugs=4.7.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 13s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
2m 27s
master passed

+1 :green_heart:
compile
0m 19s
master passed

+1 :green_heart:
shadedjars
5m 10s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 26s
the patch passed

+1 :green_heart:
compile
0m 19s
the patch passed

+1 :green_heart:
javac
0m 19s
the patch passed

+1 :green_heart:
shadedjars
5m  9s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m 46s
hbase-mapreduce in the patch passed.

35m 13s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux e2f36d8d50aa 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Temurin-1.8.0_352-b08

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/testReport/

Max. process+thread count
3140 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 30s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
2m 55s
master passed

+1 :green_heart:
compile
0m 19s
master passed

+1 :green_heart:
shadedjars
5m 40s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 56s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 18s
the patch passed

+1 :green_heart:
shadedjars
5m 39s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m  6s
hbase-mapreduce in the patch passed.

36m 48s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 773696ea744d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Eclipse Adoptium-17.0.10+7

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/testReport/

Max. process+thread count
3173 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 24s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
2m 44s
master passed

+1 :green_heart:
compile
0m 17s
master passed

+1 :green_heart:
shadedjars
5m 36s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 45s
the patch passed

+1 :green_heart:
compile
0m 18s
the patch passed

+1 :green_heart:
javac
0m 18s
the patch passed

+1 :green_heart:
shadedjars
5m 40s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 13s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
18m 13s
hbase-mapreduce in the patch passed.

37m 40s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux b055d0f7cc8e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / cd4c5c30b7

Default Java
Eclipse Adoptium-11.0.17+8

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/testReport/

Max. process+thread count
3205 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/3/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@jojochuang @Apache9  Could you all take a look? Thanks.

Ping @jojochuang

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 36s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 55s
master passed

+1 :green_heart:
compile
0m 49s
master passed

+1 :green_heart:
checkstyle
0m 16s
master passed

+1 :green_heart:
spotless
1m  0s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 50s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 58s
the patch passed

+1 :green_heart:
compile
0m 46s
the patch passed

+1 :green_heart:
javac
0m 46s
the patch passed

+1 :green_heart:
checkstyle
0m 16s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
5m 26s
Patch does not cause any errors with Hadoop 3.3.6.

+1 :green_heart:
spotless
0m 40s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
0m 39s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 10s
The patch does not generate ASF License warnings.

25m 48s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux a1063d740219 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f8aa3e2312

Default Java
Eclipse Adoptium-11.0.17+8

Max. process+thread count
78 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/console

versions
git=2.34.1 maven=3.8.6 spotbugs=4.7.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 26s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 45s
master passed

+1 :green_heart:
compile
0m 23s
master passed

+1 :green_heart:
shadedjars
6m 23s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 16s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 53s
the patch passed

+1 :green_heart:
compile
0m 16s
the patch passed

+1 :green_heart:
javac
0m 16s
the patch passed

+1 :green_heart:
shadedjars
5m 27s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 12s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
17m 53s
hbase-mapreduce in the patch passed.

39m  4s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux d1bf42324c1c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f8aa3e2312

Default Java
Eclipse Adoptium-11.0.17+8

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/testReport/

Max. process+thread count
3261 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 26s
Docker mode activated.

-0 :warning:
yetus
0m  2s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 37s
master passed

+1 :green_heart:
compile
0m 21s
master passed

+1 :green_heart:
shadedjars
6m 49s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 14s
the patch passed

+1 :green_heart:
compile
0m 22s
the patch passed

+1 :green_heart:
javac
0m 22s
the patch passed

+1 :green_heart:
shadedjars
5m 53s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
18m 44s
hbase-mapreduce in the patch passed.

40m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux c5452820e461 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f8aa3e2312

Default Java
Eclipse Adoptium-17.0.10+7

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/testReport/

Max. process+thread count
3347 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 15s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 16s
master passed

+1 :green_heart:
compile
0m 23s
master passed

+1 :green_heart:
shadedjars
5m 47s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 17s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 16s
the patch passed

+1 :green_heart:
compile
0m 22s
the patch passed

+1 :green_heart:
javac
0m 22s
the patch passed

+1 :green_heart:
shadedjars
6m 22s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 14s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
22m 24s
hbase-mapreduce in the patch passed.

43m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5996

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 31a26a2c3f5d 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / f8aa3e2312

Default Java
Temurin-1.8.0_352-b08

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/testReport/

Max. process+thread count
3217 (vs. ulimit of 30000)

modules
C: hbase-mapreduce U: hbase-mapreduce

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5996/4/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


text:
HIVE-25032: Optimise PartitionManagementTask

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?
No

How was this patch tested?

the added unit test
benchmark

  java -Xmx8g -jar hmsbench-jar-with-dependencies.jar --confdir hiveconf --runMode MSCK -d bm -N 1000 -L 10 -W 2 -H localhost -P 9083

The above command will load the configuration in hiveconf directory, run the benchmark 10 times, each time there are 200 in 1000 tables needed to recover partitions, and the 200 tables each have 500 partitions to add and 500 partitions to drop.
In hiveconf directory, only core-site.xml is here for specifying local FileSystem.

After optimization:
Operation                      Mean     Med      Min      Max      Err%
PartitionManagementTask        1469     1437     1394     1587     4.744
PartitionManagementTask.1000   1.431e+05 1.406e+05 1.382e+05 1.555e+05 3.959
Before optimization:
Operation                      Mean     Med      Min      Max      Err%
PartitionManagementTask        7443     7438     7371     7500     0.5184
PartitionManagementTask.1000   7.266e+05 7.271e+05 7.168e+05 7.359e+05 0.7314

I think the test failures are genuine. Can you please address them?

ack

LGTM +1

Thanks @saihemanth-clouder. Please hold on the merge, I'm thinking how not to introduce the new dropPartitions in IMetaStoreClient`.

repo: apache/hive


text:
count(*) should be using Iceberg basic stats if present

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

depends on HIVE-27327


text:
HIVE-27938: Iceberg: Fix java.lang.ClassCastException during vectoriz…

…ed reads on partition columns

What changes were proposed in this pull request?

Add HiveIdentityPartitionConverters.java for converting partitions datatypes .
Why are the changes needed?

In this pull request https://github.com/apache/iceberg/pull/1130  -   Identity Partitioning: Introduced a iceberg specific constants map  to convert data from java internal representations (e.g., long for timestamps) to Iceberg generics (LocalDateTime or OffsetDateTime for timestamps) .
A similar constants map is required for Hive as quoted in the following discussion.

When we add support for Pig or Hive objects, we would need to convert differently for those.
https://github.com/apache/iceberg/pull/1130#discussion_r443701477

Without this change, vectorised read queries on partition columns fail with java.lang.ClassCastException for DATE, TIMESTAMP and DECIMAL datatypes as described in the jira https://issues.apache.org/jira/browse/HIVE-27938 .
Does this PR introduce any user-facing change?

Is the change a dependency upgrade?

How was this patch tested?

Qfile :
iceberg_partition_vectorized_read.q

Thanks @SourabhBadhya !


text:
Fix #2049 : Add resources configuration Helm Chart

Add a support for resource configuration in helm chart (only for the server part)

@hansva  Note to self: 2.3 requires a Helm chart release too

authors: hansva scali

repo: apache/hop


text:
[SUPPORT] Support for Schema evolution. Facing an error

Hi Hudi team,
In https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-What'sHudi'sschemaevolutionstory you describe, that "As long as the schema passed to Hudi [...] is backwards compatible [...] Hudi will seamlessly handle read/write of old and new data and also keep the Hive schema up-to date."
We need and try to use this mechanism but are failing.
Steps to reproduce:

We have some old schema and events.
We update the schema with a new, optional union field and restart the DeltaStreamer
We ingest new events
We ingest old events again (there are some upserts). In production we would have both event versions until all producers have changed to the new version. At this step the Deltastream crashes with

210545 [pool-28-thread-1] ERROR org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer  - Shutting down delta-sync due to exception
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 22.0 failed 4 times, most recent failure: Lost task 0.3 in stage 22.0 (TID 1162, 100.65.150.166, executor 1): org.apache.hudi.exception.HoodieUpsertException: Error upserting bucketType UPDATE for partition :0
        at org.apache.hudi.table.action.commit.BaseCommitActionExecutor.handleUpsertPartition(BaseCommitActionExecutor.java:253)
        at org.apache.hudi.table.action.commit.BaseCommitActionExecutor.lambda$execute$caffe4c4$1(BaseCommitActionExecutor.java:102)
        at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsWithIndex$1(JavaRDDLike.scala:102)
        at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsWithIndex$1$adapted(JavaRDDLike.scala:102)
        at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsWithIndex$2(RDD.scala:889)
        at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsWithIndex$2$adapted(RDD.scala:889)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:349)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:313)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:349)
        at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:362)
        at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1388)
        at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1298)
        at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1362)
        at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1186)
        at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:360)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:311)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:349)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:313)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:127)
        at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:444)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:447)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hudi.exception.HoodieUpsertException: Failed to close UpdateHandle
        at org.apache.hudi.io.HoodieMergeHandle.close(HoodieMergeHandle.java:302)
        at org.apache.hudi.table.action.commit.CommitActionExecutor.handleUpdateInternal(CommitActionExecutor.java:101)
        at org.apache.hudi.table.action.commit.CommitActionExecutor.handleUpdate(CommitActionExecutor.java:74)
        at org.apache.hudi.table.action.deltacommit.DeltaCommitActionExecutor.handleUpdate(DeltaCommitActionExecutor.java:73)
        at org.apache.hudi.table.action.commit.BaseCommitActionExecutor.handleUpsertPartition(BaseCommitActionExecutor.java:246)
        ... 28 more
Caused by: java.io.EOFException
        at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
        at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
        at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423)
        at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
        at org.apache.hudi.avro.HoodieAvroUtils.bytesToAvro(HoodieAvroUtils.java:108)
        at org.apache.hudi.common.model.OverwriteWithLatestAvroPayload.getInsertValue(OverwriteWithLatestAvroPayload.java:80)
        at org.apache.hudi.io.HoodieMergeHandle.close(HoodieMergeHandle.java:271)
        ... 32 more

I've created a test to reproduce this behaviour here: https://github.com/apache/hudi/pull/1844. I hope the test is helpful.
The test cause the following exception, but i think they are quite related.
Job aborted due to stage failure: Task 0 in stage 16.0 failed 1 times, most recent failure: Lost task 0.0 in stage 16.0 (TID 23, localhost, executor driver): java.lang.ArrayIndexOutOfBoundsException: 19
        at org.apache.avro.generic.GenericData$Record.get(GenericData.java:212)
        at org.apache.hudi.AvroConversionHelper$.$anonfun$createConverterToRow$9(AvroConversionHelper.scala:170)
        at org.apache.hudi.AvroConversionUtils$.$anonfun$createDataFrame$2(AvroConversionUtils.scala:66)
        at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
        at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
        at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
        at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
        at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$2.hasNext(WholeStageCodegenExec.scala:636)
        at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
        at scala.collection.Iterator.isEmpty(Iterator.scala:385)
        at scala.collection.Iterator.isEmpty$(Iterator.scala:385)
        at scala.collection.AbstractIterator.isEmpty(Iterator.scala:1429)
        at org.apache.hudi.AvroConversionUtils$.$anonfun$createRdd$2(AvroConversionUtils.scala:44)
        at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:801)
        at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:801)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:411)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
        at org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer.testSchemaEvolution(TestHoodieDeltaStreamer.java:491)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 19

I think the mailing list entry from avro could help a lot: http://apache-avro.679487.n3.nabble.com/AVRO-schema-evolution-adding-optional-column-with-default-fails-deserialization-td4043025.html
The corresponding place in code is here: https://github.com/apache/hudi/blob/bf1d36fa639cae558aa388d8d547e58ad2e67aba/hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java#L107
Thanks a lot in advance!
Cheers,
Sebastian
Expected behavior
As stated in FAQ schema evolution should not crash the DeltaStreamer
Environment Description

Hudi version : current master branch

Spark version : 3.0.0

Hive version : 3.1.2

Hadoop version : 3.2.1

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) : yes, running on kubernetes

Thanks Sebastian for the detailed information. From looking at the code related to your originally seen exception (not the test), it has to do with plain avro schema compatibility rules. Would you mind providing both versions of schema. From the test code in PR, there are 4 different fields (various combination of nullable vs default types). I would like to see which field specifically is contributing to the issue.

Thanks for your fast reply!
The PR adds a new Test and improves 2 existing tests. The mentioned 4 new cols in TestHoodieAvroUtils increase the number of tested cases, the tests are still working.
The new test is in TestHoodieDeltaStreamer and starts the DeltaStreamer with different schemas and transformers to reproduce the actual problem.
Regarding the both versions of my schema, i can only provide a diff but it should be sufficient. The new field has the same type (union of null and string, default of null) as reproduced in the test here https://github.com/apache/hudi/pull/1844/files#diff-07dd5ed6077721a382c35b2700da0883R130.
diff Event.json_schema Event-aged.json_schema

},
{
  "name": "agedOptionalField",
  "type": [
    "null",
    {
      "type": "string",
      "avro.java.string": "String"
    }
  ],
  "doc": "New aged optional field",
  "default": null

@sbernauer : Are you appending this field to the end of the schema ? Otherwise looks ok. Although honestly,  I have not seen the usage of "avro.java.string": "String" before.
@prashantwason  @n3nash : Any ideas  what is wrong here ?

@bvaradar, yes i am appending the field to end of the schema (as reproduced in the test). The definition of the event is outside my scope, we just consume this events ;)

@sbernauer I briefly looked at the test-case you added and I see what you are trying to reproduce. The issue seems to be as follows :

Generate data with schema A, pass schema A
Generate data with schema B, pass schema B
Generate data with schema A, pass schema B <fails in AvroConversionHelper>
Where schema A is smaller schema, schema B is a larger schema with 1 added field.
@bvaradar Have we tested such a scenario before with the AvroConversionHelper class where we convert the RDD to DF with schema A, but pass schema B as the writer schema ?

Thanks @n3nash
@sbernauer : I think the exception you are seeing in the production could be because of different reasons  than the tests. I would like to decouple them and focus on the exception you are seeing in your prod setup.  To localize the problem (due to pure avro issue vs hudi-avro-usage), can you write an avro only test (without any hudi/spark), writing a record with your old schema and reading with new schema and ensure if it works fine ?
Also, I see that you are using spark and Hadoop 3.x. Please make sure you are using the right version of avro that we support - 1.8.2 at runtime. That could also be an issue.

Hi @bvaradar,
I've created a test using pure avro DataFileWriter to reproduce my DeltaStreamer-test and it kind of works.
You can find it here https://github.com/sbernauer/avro-schema-evolution-test/blob/master/src/test/java/main/AvroSchemaEvolutionTest.java
There is no exception, but the evoluted optional field is missing (Assertion https://github.com/sbernauer/avro-schema-evolution-test/blob/42e634bdcca9c996007566557c647c2785191e94/src/test/java/main/AvroSchemaEvolutionTest.java#L75). Do you know why?
I found another post https://stackoverflow.com/questions/34733604/avro-schema-doesnt-honor-backward-compatibilty, it provides a testcase for schema evolution using a BinaryDecoder and reproduces the EOF exception.
The mailing list entry http://apache-avro.679487.n3.nabble.com/AVRO-schema-evolution-adding-optional-column-with-default-fails-deserialization-td4043025.html describes the same problem but causes the ArrayIndexOutOfBoundsException.
My setup is using avro 1.8.2.

The  javadoc for DatumWriter says:
"Stores in a file a sequence of data conforming to a schema. The schema is stored in the file with the data. Each datum in a file is of the same schema."
Your test case has records with different schema being written to the same file.
Please try using the GenericDatumReader construtor which specifies both the reader/writer schema.
GenericDatumReader(Schema writer, Schema reader)

Yes, to be correct you have to specify both reader and writer schema to GenericDatumReader. Hudi currenty only passes one schema here https://github.com/apache/hudi/blob/c3279cd5989805946267b046007ea23ba4b615c2/hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java#L107 and here https://github.com/apache/hudi/blob/c3279cd5989805946267b046007ea23ba4b615c2/hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java#L117
I think that's a problem
I would have used GenericDatumReader with reader and writer schema, but first I wanted to append new evoluted events to an existing file. Looking at "Each datum in a file is of the same schema"  this seems expected behavior to be impossible.
"Each datum in a file is of the same schema" - How is the strategy of Hudi when there are Upserts for an evoluted event? The new events cant simple be written to an existing file.
Every time a file must be updated, read it correctly with the reader and writer schema and than create a new file with the new schema?

I would have used GenericDatumReader with reader and writer schema, but first I wanted to append new evoluted events to an existing file
In HUDI log files, records are added to AvroDataBlock which have a header containing the schema for the writer. Different AvroDataBlock could have different schemas but all records within the same block should have the same schema.

Log blocks are read using both reader/write schema.
https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java#L143

The issue is probably related to using the wrong schema for Upsert. Let me expand your original steps to reproduce and you can correct me:
Steps to reproduce:

We have some old schema (SCHEMA_V1) and events.
We update the schema with a new (SCHEMA_V2), optional union field and restart the DeltaStreamer
We ingest new events  (Events with SCHEMA_V2)
We ingest old events again (there are some upserts). ?????? What schema is being used here?

If in step 4 you are using SCHEMA_V1 then it wont be able to read in records ingested at step 3 as that is a newer schema.

We ingest old events again (there are some upserts). ?????? What schema is being used here?

At this step I used SCHEMA_V2
We use Deltastreamer in continues mode and only restart it in step 2, where we provide the new SCHEMA_V2 to the Deltastreamer.
I tried to reproduce everything as good as possible in my DeltaStreamer-test

Is there anything we can do further to resolve this issue?

@sbernauer : Do you mean using SCHEMA_v1 at step 4. Otherwise, both reader and writer schema is same. right ?
There are couple of issues  here :

Regarding your production setup, From the looks of it, you are not passing any transformers : In this case the source avro record is converted to HoodieRecord. In this process, we serialize the record to bytes using the schema referenced by the generic record. We do this to reduce memory footprint while shuffling records as Spark native serialization of avro records also serializes Schema for each record.  I am a little surprised on how this issue was not seen early enough in the community. This made me wonder if the schema was indeed backwards compatible. But from your snippet, it does not look to be the case. Let me open a jira to track this issue. We are anyways going to tackle schema evolution in general in next release. For your problem though, you can implement a new Payload class which does not serialize to bytes upon construction (Example : https://github.com/apache/hudi/blob/hoodie-0.4.0/hoodie-spark/src/main/java/com/uber/hoodie/OverwriteWithLatestAvroPayload.java). Note that once you do this, you will be able to write the records to parquet files successfully but we would need to test reading from it. Hudi uses parquet-avro library and passes latest schema to. If parquet-avro is unable to read, then you can set ("hoodie.avro.schema.externalTransformation=true") after https://github.com/apache/hudi/pull/1876 gets landed (ETA: sometime this week).

Regarding your unit-test failure, this happens in AvroConversionUtils which is used when transformer is configured. The code itself is based on spark-avro. I will open a jira for it. Would you be interested in looking at the code in AvroConversionUtils to see how this can be fixed since you already wrote the tests for it ?

For issue (1) : Opened https://issues.apache.org/jira/browse/HUDI-1128
For issue (2) : https://issues.apache.org/jira/browse/HUDI-1129

Thanks @bvaradar for opening the tickets!
I will continue commenting in this tickets.

Do you mean using SCHEMA_v1 at step 4. Otherwise, both reader and writer schema is same. right ?

I could have been more precise. At Step 4 we consume events without the new optional field (we could call them events produced with SCHEMA_V1). The Deltastreamer is passed the SCHEMA_v2.

cc @nsivabalan as well. This is related to the schema evolution PR.

@n3nash The concrete thing to do here, is to just take and repro this scenario with master branch and see if its still an issue.

Hi @vinothchandar, @nsivabalan and me are working on Slack trying to get our schema evolution usecase running. Currently only Inserts, no Upserts for CoW. I'm trying out his patch in https://github.com/apache/hudi/pull/2927
I seems to work fine but only if I put in a dummy transformation and specify both - source and target - Schemas. Setting target schema to null the schema evolution part fails, setting no transformer does also cause problems If I recall correctly.

@nsivabalan Can you please reply above ?

yes, I am in sync w/ @sbernauer via slack. He confirmed that the PR we have put up works for him (older records able to be upserted to hudi after schema evolved w/ hudi table). He is doing more testing for now.

@sbernauer : closing this issue out as we have the fixes landed already. Feel free to open up a new issue if you have more requirements. thanks!

source: gharchive/issue

authors: bvaradar n3nash nsivabalan prashantwason sbernauer vinothchandar

repo: apache/hudi