text:
Retry failures in couch_mrview_compactor:recompact

Currently, if there's an error in the spawned couch_index_updater:update
process, the receive block will not match the message, and the calling
process will block forever.
This commit changes the receive pattern to match both successful updates
as well as errors, logging the errors, and retrying the recompact a
configurable number of times. If the configured retry count is exceeded,
it logs an error, and returns the original state.
COUCHDB-3184

@davisp, not sure if this is exactly what you had in mind, but I thought I'd take a stab at it to get the ball rolling

That's the general idea, though there was a bit I wanted to add using the update/3 so that we know if the updater is making progress between failures. I typed up a quick thing but haven't tried compiling it or anything crazy like that.
diff --git a/src/couch_mrview_compactor.erl b/src/couch_mrview_compactor.erl
index 9dba094..3271103 100644
--- a/src/couch_mrview_compactor.erl
+++ b/src/couch_mrview_compactor.erl
@@ -148,14 +148,31 @@ compact(State) ->

 recompact(State) ->
+    recompact(State, recompact_retry_count()).
+
+recompact(State, 0) ->
+    erlang:error(exceeded_recompact_retry_count);
+
+recompact(State, RetryCount) ->
+    Self = self(),
     link(State#mrst.fd),
     {Pid, Ref} = erlang:spawn_monitor(fun() ->
-        couch_index_updater:update(couch_mrview_index, State)
+        couch_index_updater:update(Self, couch_mrview_index, State)
     end),
+    recompact_loop(State, RetryCount).
+
+recompact_loop(State, RetryCount) ->
     receive
+        {'$gen_cast', {new_state, State}} ->
+            % We've made progress so reset RetryCount
+            recompact_loop(State, recompact_retry_count());
         {'DOWN', Ref, _, _, {updated, Pid, State2}} ->
             unlink(State#mrst.fd),
-            {ok, State2}
+            {ok, State2};
+        {'DOWN', Ref, _, _, Reason} ->
+            unlink(State#mrst.fd),
+            couch_log:warning("Error durring recompaction: ~r", [Reason]),
+            recompact(State, RetryCount - 1)
     end.

 compact_log(LogBtree, BufferSize, Acc0) ->
@@ -265,3 +282,11 @@ swap_compacted(OldState, NewState) ->
     erlang:demonitor(OldState#mrst.fd_monitor, [flush]),

     {ok, NewState#mrst{fd_monitor=Ref}}.
+
+
+recompact_retry_count() ->
+    config:get_integer(
+        "view_compaction",
+        "recompact_retry_count",
+        ?DEFAULT_RECOMPACT_RETRY_COUNT
+    ).

Already see a bug, The $gen_cast message needs to use a different variable name which it passes to recompact_loop so that we don't get a bad match.

@davisp, thanks for the diff. I updated the PR, added Pid and Ref params to recompact_loop function, and fixed the State name issue for '$gen_cast' case.
One concern is that while we have a test for couch_mrview:compact/3, we don't actually test recompact in it. Do you think it's sufficient to just unit test recompact and recompact_loop, or do you think it warrants a full on integration style test such as in couch_mrview_compact_tests?

I'd be fine with just adding unit tests. I don't know how we'd test behavior very well on this with an integration test as the bad cases are all when views get huge, which isn't super amenable to CI tests.

@davisp, I added a couple tests which seem to exercise all the new retry/progress logic.
I'm not familiar with the ~r control sequence in the couch_log:warning, nor am I able to find it in http://erlang.org/doc/man/io.html, but it doesn't seem to cause any problems, either.

@jaydoane The ~r is a custom control character I added to couch_log to simplify formatting error terms:
https://github.com/apache/couchdb-couch-log/commit/038586b7ab6cfec1b15f44162eddb5723803be21

@davisp, thanks for the review. Can you merge this for me?

authors: davisp jaydoane

repo: apache/couchdb-couch-mrview


text:
[Draft] SSE implementation using AsyncContext

This is a suggestion for an implementation using AsyncContext instead of Atmosphere, as discussed with Andryi Redko on the dev cxf list.
This implementation seems to be working correctly for Aries JAX-RS Whiteboard implementation, but some tests under systests/rs-sse in CXF tests fail.
There are probably many things wrong so a review would be very much appreciated.
Thx in advance.
Carlos.

any preliminary comments on this anyone?

In general, I really like the idea.  But I'm concerned about the systest failures.  Do you have more information on why they are failing?  If they are failing because they are coded to Atmosphere-specific behaviors, then I'm less concerned (but then we should fix the tests).  I'm more concerned if this change is removing/altering the SSE behavior.

hey @andymc12 @reta,
I was not very acquainted with the test infra. I made some debugging and I found Jackson ObjectWriter to generate this expection:
	at org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:234)
	at org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:189)
	at org.apache.coyote.http11.InternalNioOutputBuffer.commit(InternalNioOutputBuffer.java:177)
	at org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:740)
	at org.apache.coyote.Response.action(Response.java:179)
	at org.apache.coyote.Response.sendHeaders(Response.java:350)
	at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:335)
	at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
	at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:110)
	at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1100)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:915)
	at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625)
	at org.apache.cxf.jaxrs.sse.OutboundSseEventBodyWriter.writePayloadTo(OutboundSseEventBodyWriter.java:139)
	at org.apache.cxf.jaxrs.sse.OutboundSseEventBodyWriter.writeTo(OutboundSseEventBodyWriter.java:112)
	at org.apache.cxf.jaxrs.sse.OutboundSseEventBodyWriter.writeTo(OutboundSseEventBodyWriter.java:40)
	at org.apache.cxf.jaxrs.sse.SseEventSinkImpl.dequeue(SseEventSinkImpl.java:124)
	at org.apache.catalina.core.AsyncContextImpl$RunnableWrapper.run(AsyncContextImpl.java:571)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)```

this seems to be what is preventing tests from passing 😕

nope... jetty tests fail in a similar way

it looks like only one test fails in TomcatEmbeddedTest and it is testBooksStreamIsBroadcasted because the OutputStream is closed when JacksonJsonProvider tries to flush. Is is possible that the test took some assumptions about the threading that were valid for atmosphere only? I am a bit in the dark here.

@csierra Thanks a lot for looking into it. No, the tests do not make any assumptions regarding threading, they are 100% black box in this regards. However, server-side is quite sensitive to threading (and thread-local proxies), we would have to look into that for sure.

hey @reta @andymc12,
tests are green on my machine now! \o/
I needed to remove the transportId from the war file for those tests to work (🤦♂ ) and I found that testBooksStreamIsBroadcasted was taking for granted that broadcaster.broadcast is synchronous, which is not anymore (I guess it was).
what should I do to have the tests run again in the pull?

hi @reta,
thank you guys for looking into this. I am happy to see that the PR is useful.

Closing in favor of https://github.com/apache/cxf/pull/415, as discussed

source: gharchive/pull-request

authors: andymc12 csierra reta

repo: apache/cxf

license_type: permissive

license_source: github-api


text:
suggestions for a tag with attributes on multiple lines wrong sometimes

When a tags attributes are spread across multiple lines in a self closing tag, and a suggestions is triggered between 2 spaces, the suggestion result is always for the element tag.

This issue was caused by a check for multi-line attributes that should have only been done for non-self-closing tags.

source: gharchive/issue

repo: apache/daffodil-vscode


text:
add partitioning scheme for unresolved shuffle and shuffle reader exec

Which issue does this PR close?

Closes https://github.com/apache/datafusion-ballista/issues/16.
Rationale for this change

unresolved shuffle and shuffle reader exec does not have partitioning info. This becomes a problem when a stage has InterleaveExec, which requires all its children to have hash partitioning on the same expressions. Currently even though the partitioning satisfies InterleaveExec's requirements, because children show UnknownPartitioning planning such plans into stages fail.
What changes are included in this PR?

Carry partitioning scheme through shuffles
Are there any user-facing changes?

Changes I think look good. Can we add a test for it?

@Dandandan should be ready for another look

Thanks @onursatici

authors: Dandandan onursatici

repo: apache/datafusion-ballista


text:
Support unicode character for initcap function

Is your feature request related to a problem or challenge?
Make initcap function work with Unicode character.
Describe the solution you'd like
Current behavior of DataFusion:
select initcap('đẹp đẽ êm ả édouard'); --đẹP đẽ êM ả éDouard
select initcap('ñandú árbol éxito'); --ñAndú áRbol éXito
select initcap('олег иванович'); --олег иванович
select initcap('íslensku þjóðarinnar'); --íSlensku þJóðArinnar
select initcap('ελληνική επιστήμη'); --ελληνική επιστήμη

Expected behavior: (Postgres)
select initcap('đẹp đẽ êm ả édouard'); --Đẹp Đẽ Êm Ả Édouard
select initcap('ñandú árbol éxito'); --Ñandú Árbol Éxito
select initcap('олег иванович'); --Олег Иванович
select initcap('íslensku þjóðarinnar'); --Íslensku Þjóðarinnar
select initcap('ελληνική επιστήμη'); --Ελληνική Επιστήμη

Describe alternatives you've considered
No response
Additional context
Found in comment of @Dandandan

take

repo: apache/datafusion


text:
deps update docker build env to add jdk17

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

repo: apache/doris


text:
fix fix nullptr and token num in TabletStream::close

Proposed changes
Check nullptr in TabletStream::close. And fix _num_threads in LoadStreamMgr.
#0  pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at pthread_sigmask.c:48
#1  0x00007fe448ccd71e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
#2  0x00007fe448cce206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
#3  <signal handler called>
#4  doris::ThreadPoolToken::wait (this=0x0) at /mnt/disk1/chenkaijie/doris/be/src/util/threadpool.cpp:161
#5  0x00005606566d9f68 in doris::TabletStream::close()::$_0::operator()() const (this=0x7fdc7cc72a00) at /mnt/disk1/chenkaijie/doris/be/src/runtime/load_stream.cpp:229
#6  std::__invoke_impl<void, doris::TabletStream::close()::$_0&>(std::__invoke_other, doris::TabletStream::close()::$_0&) (__f=...)
    at /mnt/disk1/chenkaijie/env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61
#7  std::__invoke_r<void, doris::TabletStream::close()::$_0&>(doris::TabletStream::close()::$_0&) (__fn=...)
    at /mnt/disk1/chenkaijie/env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111
#8  std::_Function_handler<void (), doris::TabletStream::close()::$_0>::_M_invoke(std::_Any_data const&) (__functor=...)
    at /mnt/disk1/chenkaijie/env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290
#9  0x00005606566c866e in std::function<void ()>::operator()() const (this=0x7fe036130a38)
    at /mnt/disk1/chenkaijie/env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591
#10 doris::WorkThreadPool<false>::work_thread (this=0x7fe4292d0010, thread_id=<optimized out>) at /mnt/disk1/chenkaijie/doris/be/src/util/work_thread_pool.hpp:158
#11 0x0000560662ae29cf in execute_native_thread_routine ()
#12 0x00007fe44d74c609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#13 0x00007fe44d9f9133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall


text:
[nereids] string literal coercion of in predicate

Description:
The sql execute much slow when the literal value with string format in in predicate; and the real data is integral type。
mysql> set enable_nereids_planner = false;
Query OK, 0 rows affected (0.03 sec)

mysql> select id,sum(clicks) from a_table where id in ('787934713', '306960695') group by id limit 10;
+------------+---------------+
| id | sum(`clicks`) |
+------------+---------------+
|  787934713 |          2838 |
|  306960695 |           339 |
+------------+---------------+
2 rows in set (1.81 sec)

mysql> set enable_nereids_planner = true;
Query OK, 0 rows affected (0.02 sec)

mysql> select id,sum(clicks) from a_table where id in ('787934713', '306960695') group by id limit 10;
+------------+-------------+
| id | sum(clicks) |
+------------+-------------+
|  787934713 |        2838 |
|  306960695 |         339 |
+------------+-------------+
2 rows in set (28.14 sec)

Reason:
In legacy planner, the string literal with convert to integral value, but in the nereids planner do not do this convert and with do string matching in BE。
Solved:
do process string literal with numeric in in predicate like in comparison predicate;
test table:
create table a_table(
    k1 BIGINT NOT NULL,
    k2 VARCHAR(100) NOT NULL,
    v1 INT SUM NULL DEFAULT "0"
) ENGINE=OLAP
AGGREGATE KEY(k1,k2)
distributed BY hash(k1) buckets 2
properties("replication_num" = "1");
insert into a_table values (10, 'name1', 10),(20, 'name2', 10);
explain plan select * from a_table where k1 in ('10', '20001');

before optimize:
+--------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                                |
+--------------------------------------------------------------------------------------------------------------------------------------+
| ========== PARSED PLAN (time: 1ms) ==========                                                                                |
| UnboundResultSink[4] (  )                                                                                |
| +--LogicalProject[3] ( distinct=false, projects=[*], excepts=[] )                                                                    |
|    +--LogicalFilter[2] ( predicates='k1 IN ('10001', '20001') )                                                                      |
|       +--LogicalCheckPolicy (  )                                                                                |
|          +--UnboundRelation ( id=RelationId#0, nameParts=a_table )                                                                   |
|                                                                                |
| ========== ANALYZED PLAN (time: 2ms) ==========                                                                                |
| LogicalResultSink[15] ( outputExprs=[k1#0, k2#1, v1#2] )                                                                             |
| +--LogicalProject[13] ( distinct=false, projects=[k1#0, k2#1, v1#2], excepts=[] )                                                    |
|    +--LogicalFilter[11] ( predicates=cast(k1#0 as TEXT) IN ('10001', '20001') )                                                      |
|       +--LogicalOlapScan ( qualified=internal.db.a_table, indexName=<index_not_selected>, selectedIndexId=12003, preAgg=UNSET )      |
|                                                                                |
| ========== REWRITTEN PLAN (time: 6ms) ==========                                                                                |
| LogicalResultSink[45] ( outputExprs=[k1#0, k2#1, v1#2] )                                                                             |
| +--LogicalFilter[43] ( predicates=cast(k1#0 as TEXT) IN ('10001', '20001') )                                                         |
|    +--LogicalOlapScan ( qualified=internal.db.a_table, indexName=a_table, selectedIndexId=12003, preAgg=OFF, No aggregate on scan. ) |
|                                                                                |
| ========== OPTIMIZED PLAN (time: 6ms) ==========                                                                                |
| PhysicalResultSink[90] ( outputExprs=[k1#0, k2#1, v1#2] )                                                                            |
| +--PhysicalDistribute[87]@1 ( stats=0.33, distributionSpec=DistributionSpecGather )                                                  |
|    +--PhysicalFilter[84]@1 ( stats=0.33, predicates=cast(k1#0 as TEXT) IN ('10001', '20001') )                                       |
|       +--PhysicalOlapScan[a_table]@0 ( stats=1 )                                                                                |
+--------------------------------------------------------------------------------------------------------------------------------------+

after optimize:
+--------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                                |
+--------------------------------------------------------------------------------------------------------------------------------------+
| ========== PARSED PLAN (time: 15ms) ==========                                                                                |
| UnboundResultSink[4] (  )                                                                                |
| +--LogicalProject[3] ( distinct=false, projects=[*], excepts=[] )                                                                    |
|    +--LogicalFilter[2] ( predicates='k1 IN ('10001', '20001') )                                                                      |
|       +--LogicalCheckPolicy (  )                                                                                |
|          +--UnboundRelation ( id=RelationId#0, nameParts=a_table )                                                                   |
|                                                                                |
| ========== ANALYZED PLAN (time: 11ms) ==========                                                                                |
| LogicalResultSink[15] ( outputExprs=[k1#0, k2#1, v1#2] )                                                                             |
| +--LogicalProject[13] ( distinct=false, projects=[k1#0, k2#1, v1#2], excepts=[] )                                                    |
|    +--LogicalFilter[11] ( predicates=k1#0 IN (10001, 20001) )                                                                        |
|       +--LogicalOlapScan ( qualified=internal.db.a_table, indexName=<index_not_selected>, selectedIndexId=12003, preAgg=UNSET )      |
|                                                                                |
| ========== REWRITTEN PLAN (time: 12ms) ==========                                                                                |
| LogicalResultSink[45] ( outputExprs=[k1#0, k2#1, v1#2] )                                                                             |
| +--LogicalFilter[43] ( predicates=k1#0 IN (10001, 20001) )                                                                           |
|    +--LogicalOlapScan ( qualified=internal.db.a_table, indexName=a_table, selectedIndexId=12003, preAgg=OFF, No aggregate on scan. ) |
|                                                                                |
| ========== OPTIMIZED PLAN (time: 4ms) ==========                                                                                |
| PhysicalResultSink[90] ( outputExprs=[k1#0, k2#1, v1#2] )                                                                            |
| +--PhysicalDistribute[87]@1 ( stats=0, distributionSpec=DistributionSpecGather )                                                     |
|    +--PhysicalFilter[84]@1 ( stats=0, predicates=k1#0 IN (10001, 20001) )                                                            |
|       +--PhysicalOlapScan[a_table]@0 ( stats=2 )                                                                                |
+--------------------------------------------------------------------------------------------------------------------------------------+

run buildall

run p0


text:
Feat add expression transformation rule simplify_conditional_function

coalesce(null,null,expr,...) => coalesce(expr,...)
coalesce(expr1(not null able ), expr2, ...., expr_n) => expr1
coalesce(null,null) => null
coalesce(expr1) => expr1
nvl(null,R) => R
nvl(L(not-nullable ),R) => L
nullif(null, R) => Null
nullif(L, null) => Null

run buildall

run buildall

run buildall

run buildall

run buildall

run cloud_p0

run external

run cloud_p1

run buildall

run buildall

run buildall

run buildall

run p0

run buildall

run buildall

run cloud_p0

authors: feiniaofeiafei morrySnow


text:
regression temp, disable case made doris core

Proposed changes
Issue Number: close #xxx

skip buildall

skip buildall


text:
featureSupport more string-to-integer formats and ensure consistent behavior between BE and FE.

Proposed changes
Previously, FE supported scientific notation in casts, while BE did not, leading to discrepancies in constant calculations. Now, a unification is being implemented.

run buildall

run feut

run buildall

run cloud_p0

run performance


text:
fix after fe restarting, external permissions are lost (#43275)

pick: https://github.com/apache/doris/pull/43275

run buildall

run external

run buildall

run external

run p0

authors: morrySnow zddr


text:
opt reduce cache hotspot table insert times

batch insert in FE
enlarge polling interval in FE
ignore stable statistics only catch the dynamic in BE

What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[x] Other reason (it's an opt, no behaviour has been changed)

Behavior changed:

[x] No.
[ ] Yes.

Does this need documentation?

[x] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

run buildall

run buildall

authors: Thearas freemandealer


text:
fix support one phase DeferMaterializeTopN #45693 branch-3.0

What problem does this PR solve?
pick #45693
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

run buildall

authors: englefly hello-stephen


text:
DRILL-5702: Jdbc Driver Class not found

Setting "package.namespace.prefix" to "oadd." by default for all profiles. It can be overridden if necessary within any profile.
Removing duplicated and redundant excluding of commons-logging packages.

The issue is enabling "apache-release" profile disabled "default" profile automatically and then "package.namespace.prefix" had invalid value ${package.namespace.prefix}.

+1 LGTM

authors: parthchandra vdiravka

repo: apache/drill


text:
Prometheus-emitter does not push metrics to the pushgateway server

I'm not sure if there is a problem with my configuration.
The following is the configuration of pushgateway:
druid.extensions.loadList=["druid-s3-extensions","druid-histogram","mysql-metadata-storage","druid-multi-stage-query","druid-prometheus-emitter"]

druid.emitter=prometheus
druid.emitter.prometheus.strategy=pushgateway
druid.emitter.prometheus.pushGatewayAddress=localhost:9091

I used Prometheus push gateway to monitor the Druid process, and found that the push gateway server has never had Druid related metrics.
By looking at the source code, we can see that Prometheus-emitter does not call flush periodically.
Referring to incluxdbemitter, I added scheduledexecutorservice, called flush regularly, and fixed it.
Affected Version
It seems that all versions have this problem

This emitter is used by many users, and we receive some issues/PRs about this emitter, but never hear that it does not work.
I don't use this emitter, but according to current doc, exporter strategy should be used.

Default strategy exporter would expose metrics for scraping purpose. Only peon task (short-lived jobs) need to use pushgateway strategy.

Thank you for your reply I used the ps command to check the parameters of Kafka peon. The pushgateway has been configured, but the metrics is still not reported to the pushgatewayserver.

So you mean metrics are not emitted from peon tasks? If so, can you check if this works?
https://github.com/apache/druid/issues/12242#issuecomment-1216185702

So you mean metrics are not emitted from peon tasks? If so, can you check if this works?
#12242 (comment)

I just checked, and the prefix of druid.indexer.fork.property still doesn't work. Normally, pushgateway needs to continuously report metrics. However, no matter whether the prefix is added or not, the metrics is not continuously reported.
Below is the configuration I just checked
druid.indexer.fork.property.druid.extensions.loadList=["druid-s3-extensions","druid-histogram","mysql-metadata-storage","druid-multi-stage-query","druid-prometheus-emitter"]
druid.indexer.fork.property.druid.emitter=prometheus
druid.indexer.fork.property.druid.emitter.prometheus.strategy=pushgateway
druid.indexer.fork.property.druid.emitter.prometheus.pushGatewayAddress=localhost:9091

which druid version are you using?

which druid version are you using?

I tested two versions: 25.0.0-SNAPSHOT and 0.22.0

I took a look at the implementation, metrics are not pushed to the gateway until the emitter is going to closed. So, actually I think your gateway receives metrics from peon tasks, the only problem is that it doesn't receive metrics when task is running. If a task runs for a long time, this could be a problem.
There're many emitters that have a scheduler, such as KafkaEmitter, and GraphiteEmitter.
I'm wondering if we can abstract such scheduler mechanism to Emitter interface so that any implementations won't do it by itself and they won't forget to do it. This might be out of the scope of the fix of current problem.

Your idea is great, the implementation logic of the scheduler should really be extracted, and others who implement other emitters can remind them when compiling.

There're many emitters that have a scheduler, such as KafkaEmitter, and GraphiteEmitter.
I'm wondering if we can abstract such scheduler mechanism to Emitter interface so that any implementations won't do it by itself and they won't forget to do it. This might be out of the scope of the fix of current problem.

Yes, @FrankChen021,  we should extracted out this common requirement. HttpPostEmitter uses a separate EmittingThread too, and seems to have an involved batching logic. I suppose all of this should be consolidated in a common emitter utility.

repo: apache/druid


text:
Allow missing intervals for Parallel task with hash/range partitioning

Description
This PR allows Parallel task to run without explicit intervals in granularitySpec. If intervals are missing, the parallel task executes an extra step for input sampling which collects the intervals to index.
This PR additionally fixes a bug when numShards is missing in hash partitioning. When numShards is missing, the parallel task computes it by scanning the whole input. However, the computed numShards was ignored when it's serialized into JSON. To fix it, this PR adds another field intervalToNumShardsOverride which stores the computed numShards per interval so that we can handle data skew well across intervals.

This PR has:

[x] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[x] added documentation for new or modified features or behaviors.
[x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

No comment on the code itself, but very exciting to remove the extra requirement from the user!

@abhishekagarwal87 do you have more comments?

@abhishekagarwal87 do you have more comments?

@jihoonson LGTM

@clintropolis @himanshug @abhishekagarwal87 thanks for the review :+1:

authors: abhishekagarwal87 jihoonson vogievetsky


text:
Support https mutil certs

What this PR does:
Support HTTPS multiple certificates.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

Codecov Report

Merging #292 (63264c9) into develop (2890fd7) will increase coverage by 0.81%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           develop     #292      +/-   ##
===========================================
+ Coverage    33.14%   33.95%   +0.81%
===========================================
  Files           62       61       -1
  Lines         3322     3163     -159
===========================================
- Hits          1101     1074      -27
+ Misses        2107     1977     -130
+ Partials       114      112       -2

Impacted Files
Coverage Δ

pkg/model/base.go
0.00% <ø> (ø)

pkg/model/http.go
0.00% <ø> (ø)

pkg/server/listener.go
0.00% <0.00%> (ø)

...adapter/dubboregistry/remoting/zookeeper/client.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2890fd7...63264c9. Read the comment docs.

Resolve conflicts then LGTM

repo: apache/dubbo-go-pixiu


text:
Performance tuning

invoker destroy
connection warn log

Codecov Report

Merging #7324 (182315e) into master (039317c) will decrease coverage by 0.03%.
The diff coverage is 44.44%.

@@             Coverage Diff              @@
##             master    #7324      +/-   ##
============================================
- Coverage     59.08%   59.05%   -0.04%
+ Complexity      463      461       -2
============================================
  Files          1044     1044
  Lines         42480    42489       +9
  Branches       6230     6232       +2
============================================
- Hits          25101    25090      -11
- Misses        14582    14595      +13
- Partials       2797     2804       +7

Impacted Files
Coverage Δ
Complexity Δ

...etadata/store/InMemoryWritableMetadataService.java
71.96% <0.00%> (-1.38%)
0.00 <0.00> (ø)

...apache/dubbo/rpc/protocol/dubbo/DubboProtocol.java
65.53% <0.00%> (+1.89%)
0.00 <0.00> (ø)

...src/main/java/org/apache/dubbo/common/Version.java
70.87% <40.00%> (-1.86%)
0.00 <0.00> (ø)

.../dubbo/registry/integration/RegistryDirectory.java
74.66% <50.00%> (-1.85%)
0.00 <0.00> (ø)

.../rpc/protocol/dubbo/LazyConnectExchangeClient.java
53.84% <100.00%> (ø)
0.00 <0.00> (ø)

...c/protocol/dubbo/ReferenceCountExchangeClient.java
64.15% <100.00%> (+0.68%)
0.00 <0.00> (ø)

.../registry/multicast/MulticastServiceDiscovery.java
38.46% <0.00%> (-7.70%)
0.00% <0.00%> (ø%)

...e/dubbo/remoting/transport/netty/NettyChannel.java
55.68% <0.00%> (-4.55%)
20.00% <0.00%> (-1.00%)

.../apache/dubbo/remoting/transport/AbstractPeer.java
63.04% <0.00%> (-4.35%)
0.00% <0.00%> (ø%)

...he/dubbo/remoting/transport/netty/NettyServer.java
70.17% <0.00%> (-3.51%)
8.00% <0.00%> (-1.00%)

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 039317c...e72ca0f. Read the comment docs.

authors: chickenlj codecov-io

repo: apache/dubbo


text:
[3.0] Fix NullPointerException on consumer startup#8078

修复Consumer启动时报空指针异常的问题，相关issue：https://github.com/apache/dubbo/issues/8078
What is the purpose of the change
修复Consumer启动时报空指针异常的问题，相关issue：apache#8078
Brief changelog

方法 ReferenceAnnotationBeanPostProcessor#isAnnotatedReferenceBean 增加判空逻辑。

Verifying this change
按照现有单元测试即可。
Checklist

[x] Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Add some description to dubbo-website project if you are requesting to add a feature.
[ ] GitHub Actions works fine on your own branch.
[ ] If this contribution is large, please follow the Software Donation Guide.

LGTM

authors: CrazyHZM klzami


text:
[3.0] Clear ThreadLocal before injvm invoke

What is the purpose of the change
Brief changelog
Verifying this change

Checklist

[x] Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Add some description to dubbo-website project if you are requesting to add a feature.
[ ] GitHub Actions works fine on your own branch.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #9946 (4598664) into 3.0 (6da4c0c) will decrease coverage by 0.24%.
The diff coverage is 66.66%.

:exclamation: Current head 4598664 differs from pull request most recent head 7cb57c6. Consider uploading reports for the commit 7cb57c6 to get more accurate results

@@             Coverage Diff              @@
##                3.0    #9946      +/-   ##
============================================
- Coverage     65.70%   65.46%   -0.25%
  Complexity      319      319
============================================
  Files          1213     1214       +1
  Lines         52948    52971      +23
  Branches       8020     8022       +2
============================================
- Hits          34792    34678     -114
- Misses        14363    14485     +122
- Partials       3793     3808      +15

Impacted Files
Coverage Δ

...bbo/common/threadlocal/InternalThreadLocalMap.java
80.00% <60.00%> (-2.50%)
:arrow_down:

.../apache/dubbo/rpc/protocol/injvm/InjvmInvoker.java
73.73% <80.00%> (+1.28%)
:arrow_up:

...dubbo/remoting/zookeeper/ZookeeperTransporter.java
55.55% <0.00%> (-33.34%)
:arrow_down:

...ng/zookeeper/curator5/Curator5ZookeeperClient.java
43.47% <0.00%> (-27.06%)
:arrow_down:

.../remoting/api/SingleProtocolConnectionManager.java
61.53% <0.00%> (-23.08%)
:arrow_down:

...nt/metadata/ServiceInstanceHostPortCustomizer.java
65.78% <0.00%> (-21.06%)
:arrow_down:

...dubbo/common/status/support/LoadStatusChecker.java
46.15% <0.00%> (-15.39%)
:arrow_down:

...e/dubbo/rpc/protocol/tri/transport/WriteQueue.java
64.28% <0.00%> (-14.29%)
:arrow_down:

...bo/rpc/protocol/dubbo/DecodeableRpcInvocation.java
69.44% <0.00%> (-10.19%)
:arrow_down:

...ava/org/apache/dubbo/rpc/proxy/InvocationUtil.java
81.81% <0.00%> (-9.10%)
:arrow_down:

... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81ac64d...7cb57c6. Read the comment docs.

authors: AlbumenJ codecov-commenter


text:
实现直方图缩放并与散点图拼接，直方图无法横向摆放 Achieving the datazoom of hist that connecting with scatter diagram

Version
4.2.1
Steps to reproduce
主要的想法是实现seaborn.jointplot 类似的散点图与两个直方图的拼接：

直方图的实现参考：https://echarts.baidu.com/blog/2017/05/08/echarts-histogram-tutorial.html
这篇文章里面将x轴与y轴的 type 设为 value， 后通过ecStat.histogram() 插件获取直方图数据。
通过这种方法， 与x轴平行的直方图勉强实现，但是有bug，出现了缩放到最大时柱子不会适应性调节宽度，反而会柱子重叠在一起， barCategoryGap失效
在制作与y轴平行的直方图时遇到了由于x，y轴type都为value， 导致无法画出横向直方图。通过简单粗暴的旋转90度以后，发现即使这样实现了横向直方图，也无法正确的将散点图与横向直方图连接起来（echarts.connect())，因为此时横向坐标轴的x轴应该与散点图的y轴相对应，connect无法实现x轴与y轴的连接。
表面光鲜亮丽：

将x轴datazoom拖到最大时，出现的bug：

右侧的直方图为暴力旋转而来，y轴的datazoom还并没有与右边的直方图连上。
What is expected?
实现seaborn.jointplot的进阶动态版，两张直方图与一张散点图的拼接，既实现tooltip，x轴和y轴 还分别有一个datazoom，分别控制对应坐标轴上两张图的缩放。
What is actually happening?
目前遇到的问题：

缩放直方图时出现bug，柱子重叠，barCategoryGap失效，柱子越出坐标轴
制作横向直方图时，由于x，y坐标轴type都为value， 无法画出横向直方图
如果第二个问题没有办法解决，echarts是否能够将一张图的x轴与另一张图的y轴在datazoom上建立联系？

代码实现起来不难，按照连接步骤就能实现，主要问题在于是否能实现横向直方图，如果不能，能否实现一张图的x轴与另一张图的y轴相连接，并且解决柱状图柱子的重叠。现在代码有点乱，如果有需要的话我再贴出来吧。
Sorry I was lazy to explain this task again in English (or maybe lack of skill).  if you are English-only and facing the same problem, inform me so I will translate for you.

I am also interested in this feature. Can it be re-opened? This feature is also called scatter with marginal histogram

authors: MengXinAlex humphreylee

repo: apache/echarts


text:
[Bug] Get friendly dataZoom handleIcon shadow

Version
5.3.2
Link to Minimal Reproduction
No response
Steps to Reproduce
dataZoom-slider: {
handleIcon: 'M10.7,11.9H9.3c-4.9,0.3-8.8,4.4-8.8,9.4c0,5,3.9,9.1,8.8,9.4h1.3c4.9-0.3,8.8-4.4,8.8-9.4C19.5,16.3,15.6,12.2,10.7,11.9z M13.3,24.4H6.7V23h6.6V24.4z M13.3,19.6H6.7v-1.4h6.6V19.6z',
handleStyle: {
color: '#f5f5f5',
shadowBlur: 5,
shadowColor: 'rgba(0, 0, 0, 0.6)',
shadowOffsetX: 2,
shadowOffsetY: 2
}
}
Current Behavior

The first picture (version 4) shows normal, the second picture (version 5) shows a problem.They are configured exactly the same.
The shadow effect is displayed normally in version 4, but displayed abnormally in version 5. The shadow color overlaps with the icon, causing the display to be chaotic.
Expected Behavior
Shadow and icon color do not overlap, display friendly.
Environment
- OS:Linux4.19
- Browser:Chrome 89.0
- Framework:Vue@2

Any additional comments?
No response

Homepage.comd
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Version
5.3.2
Link to Minimal Reproduction
No response
Steps to Reproduce
dataZoom-slider: {
handleIcon: 'M10.7,11.9H9.3c-4.9,0.3-8.8,4.4-8.8,9.4c0,5,3.9,9.1,8.8,9.4
h1.3c4.9-0.3,8.8-4.4,8.8-9.4C19.5,16.3,15.6,12.2,10.7,11.9z
M13.3,24.4H6.7V23h6.6V24.4z M13.3,19.6H6.7v-1.4h6.6V19.6z',
handleStyle: {
color: '#f5f5f5',
shadowBlur: 5,
shadowColor: 'rgba(0, 0, 0, 0.6)',
shadowOffsetX: 2,
shadowOffsetY: 2
}
}
Current Behavior
[image: image]
https://user-images.githubusercontent.com/49702830/168069427-9537426a-0880-450e-9f6c-54e041b92708.png
[image: image]
https://user-images.githubusercontent.com/49702830/168069533-838ed830-099e-49f6-aaa6-9d57db1e8cf9.png
The first picture (version 4) shows normal, the second picture (version 5)
shows a problem.They are configured exactly the same.
The shadow effect is displayed normally in version 4, but displayed
abnormally in version 5. The shadow color overlaps with the icon, causing
the display to be chaotic.
Expected Behavior
Shadow and icon color do not overlap, display friendly.
Environment

OS:Linux4.19- Browser:Chrome 89.0- @.***

Any additional comments?
No response
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROMJK67K27KLSGCXLHLVJTXCNANCNFSM5VYAFFGA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Bringggveveone PgarxiA
On Thursday, May 12, 2022, Brian @.***> wrote:

Homepage.comd
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Version
5.3.2
Link to Minimal Reproduction
No response
Steps to Reproduce
dataZoom-slider: {
handleIcon: 'M10.7,11.9H9.3c-4.9,0.3-8.8,4.4-8.8,9.4c0,5,3.9,9.1,8.8,9.4
h1.3c4.9-0.3,8.8-4.4,8.8-9.4C19.5,16.3,15.6,12.2,10.7,11.9z
M13.3,24.4H6.7V23h6.6V24.4z M13.3,19.6H6.7v-1.4h6.6V19.6z',
handleStyle: {
color: '#f5f5f5',
shadowBlur: 5,
shadowColor: 'rgba(0, 0, 0, 0.6)',
shadowOffsetX: 2,
shadowOffsetY: 2
}
}
Current Behavior
[image: image]
https://user-images.githubusercontent.com/49702830/168069427-9537426a-0880-450e-9f6c-54e041b92708.png
[image: image]
https://user-images.githubusercontent.com/49702830/168069533-838ed830-099e-49f6-aaa6-9d57db1e8cf9.png
The first picture (version 4) shows normal, the second picture (version
5) shows a problem.They are configured exactly the same.
The shadow effect is displayed normally in version 4, but displayed
abnormally in version 5. The shadow color overlaps with the icon, causing
the display to be chaotic.
Expected Behavior
Shadow and icon color do not overlap, display friendly.
Environment

OS:Linux4.19- Browser:Chrome 89.0- @.***

Any additional comments?
No response
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROMJK67K27KLSGCXLHLVJTXCNANCNFSM5VYAFFGA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Place and
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Reopened #17022 https://github.com/apache/echarts/issues/17022.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#event-6597754126, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZRONWD4GVUCMENZDU4YTVJTXT7ANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

Just >,,
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Reopened #17022 https://github.com/apache/echarts/issues/17022.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#event-6597754126, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZRONWD4GVUCMENZDU4YTVJTXT7ANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

What does it mean please

Forver
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

What does it mean please
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124909320,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROMLVRBYQ56P6I5VLVDVJTX37ANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

NlNBE347714N
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

What does it mean please
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124909320,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROMLVRBYQ56P6I5VLVDVJTX37ANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

Together
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

What does it mean please
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124909320,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROMLVRBYQ56P6I5VLVDVJTX37ANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

What does that mean

Pissang@
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

What does that mean
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124910703,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROM3OE6FVABHK6UN7YLVJTYBTANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

Please don't post anything unrelated to the question

Come now at ll
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

PB
On Thursday, May 12, 2022, Brian @.***> wrote:

RB
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

guzmab
On Thursday, May 12, 2022, Brian @.***> wrote:

RB
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

Annnf  sso aillb i
On Thursday, May 12, 2022, Brian @.***> wrote:

guzmab
On Thursday, May 12, 2022, Brian @.***> wrote:

RB
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

I'll get another one
On Thursday, May 12, 2022, Brian @.***> wrote:

Annnf  sso aillb i
On Thursday, May 12, 2022, Brian @.***> wrote:

guzmab
On Thursday, May 12, 2022, Brian @.***> wrote:

RB
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

But you probably  don't have a few people that 🤔
On Thursday, May 12, 2022, Brian @.***> wrote:

I'll get another one
On Thursday, May 12, 2022, Brian @.***> wrote:

Annnf  sso aillb is the best way for me and you my subscribed for a few
days to this because I have a few things to and a little bit more people
about 66666666
On Thursday, May 12, 2022, Brian @.***> wrote:

guzmab
On Thursday, May 12, 2022, Brian @.***> wrote:

RB
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>

With meeting
On Thursday, May 12, 2022, Brian @.***> wrote:

I'll get another one
On Thursday, May 12, 2022, Brian @.***> wrote:

Annnf  sso aillb i
On Thursday, May 12, 2022, Brian @.***> wrote:

guzmab
On Thursday, May 12, 2022, Brian @.***> wrote:

RB
On Thursday, May 12, 2022, Mary Choo @.***> wrote:

Please don't post anything unrelated to the question
—
Reply to this email directly, view it on GitHub
https://github.com/apache/echarts/issues/17022#issuecomment-1124911698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQXZROPCYMHFSRYQ75ZGCVTVJTYFVANCNFSM5VYAFFGA
.
You are receiving this because you commented.Message ID:
@.***>


text:
Cherry-pick "Update dependency gradle to v6.6.1" to 1.4.0 also

back-port #1277 because https://github.com/gradle/gradle/releases/v6.6.1 says We recommend that you use Gradle 6.6.1 over the initial release of Gradle 6.6.
@vidakovic if OK for you, do you want to merge this after the build passed?

@vorburger yes, can do

authors: vidakovic vorburger

repo: apache/fineract


text:
add (obvious) rebase step to PR Help in README

#782 follow-up

@awasum @xurror @nikpawar89 makes sense?
Hopefully this will make it clearer e.g. for #769, #749, #737 ...

+1

This itself is failing. :)

#790 raised for that failing test; I'll rebase this one once that one successfully built and is merged.

awaiting #800, then rebase

authors: maektwain nikpawar89 vorburger


text:
[FLINK-36449] Bump apache-rat-plugin

What is the purpose of the change
Bump apache-rat-plugin
Brief change log
Bump apache-rat-plugin from 0.12 to 0.16.1 to remediate the underlying vulnerabilities in the dependencies.
Vulnerabilities from dependencies:
CVE-2022-4245
CVE-2022-4244
CVE-2020-15250
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency):  yes
The public API, i.e., is any changes to the CustomResourceDescriptors: no
Core observer or reconciler logic that is regularly executed: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

@r-sidd you have about 5 PRs open with one-liner changes, please put them together so we don't need to run the e2es again and again.

@r-sidd you have about 5 PRs open with one-liner changes, please put them together so we don't need to run the e2es again and again.

@gyfora I have two PRs open, each addressing a different scope of changes. Given that they are distinct, should I still consider merging them?

Yes , simple dependency bumps can be done in one PR , open one jira with a list of changes and one PR.

Yes , simple dependency bumps can be done in one PR , open one jira with a list of changes and one PR.

Sure, that makes sense. Right now, one PR includes the version bump, and the other has the exclusion in the POM. I’ll attach for your reference.
https://github.com/apache/flink-kubernetes-operator/pull/889

@gyfora thanks for merging 😃, I’ll make sure to include the dependency bumps in the same Jira moving forward.

authors: gyfora r-sidd

repo: apache/flink-kubernetes-operator


text:
[FLINK-18731][table-planner-blink] Fix monotonicity logic of UNIX_TIMESTAMP & UUID functions

What is the purpose of the change
UNIX_TIMESTAMP function has INCREASING monotonicity only if it has empty function arguments, else has NOT_MONOTONIC monotonicity
Brief change log

Correct the monotonicity logic of UNIX_TIMESTAMP function in FlinkSqlOperatorTable

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

cc @wuchong

Maybe we should also fix the isDeterministic. What do you think to have a specific UnixTimestampSqlFunction class? The logic stayed in FlinkSqlOperatorTable is quite verbose.
Maybe we should also fix UUID#getMonotonicity?
Is it possible to have a test to reproduce the problem?

Thanks for the suggestion,

Maybe we should also fix the isDeterministic. What do you think to have a specific UnixTimestampSqlFunction class? The logic stayed in FlinkSqlOperatorTable is quite verbose.

we can't fix isDeterministic, because different from getMonotonicity(SqlOperatorBinding call),  isDeterministic() knows nothing about operand count. One approach is we can split UNIX_TIMESTAMP into two different functions, but I think it's unnecessary.
If we only fix getMonotonicity, I trend to keep the logic in FlinkSqlOperatorTable.

Maybe we should also fix UUID#getMonotonicity?

good catch

Is it possible to have a test to reproduce the problem?

Tests have been added

@flinkbot run azure

@flinkbot run azure

my private azure is green https://dev.azure.com/godfreyhe/godfreyhe-flink/_build/results?buildId=74&view=results

authors: godfreyhe wuchong

repo: apache/flink


text:
[FLINK-25141][connector/elasticsearch][docs] Add sink parallelism option

What is the purpose of the change
add sink parallelism option in Elasticsearch connector.
Brief change log

Inspired by JDBC and Kafka connector, add a 'sink.parallelism' option, and using SinkProvider#of(sink, sinkParallelism) to set parallelism.

Verifying this change
This change added tests and can be verified as follows:

Added test that validates that parallelism option is valid

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (docs)

@fapaul please review it

@flinkbot run azure

@flinkbot run azure


text:
[FLINK-30959][table][docs] Improve the documentation of UNIX_TIMESTAMP function for different argument formats

What is the purpose of the change
This pull request improves the documentation of UNIX_TIMESTAMP function for different argument formats.
Brief change log

Modify the system functions document

Verifying this change
This change is a document change without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

@flinkbot run azure

@flinkbot run azure


text:
[FLINK-36809][table] Add correct call syntax for functions without ()

What is the purpose of the change
For Table API, there are 5 time-related functions that can't be called right now. This maps these the function calls for CURRENT_DATE, LOCALTIMESTAMP, LOCALTIME, CURRENT_TIMESTAMP, CURRENT_TIME currectly. These function do not support parenthesis when being called.
Brief change log

Add NO_PARENTHESIS call syntax
Use it for CURRENT_DATE, LOCALTIMESTAMP, LOCALTIME, CURRENT_TIMESTAMP, CURRENT_TIME
Update serialization tests

Verifying this change
This change added tests and can be verified as follows:

Added serialization tests to make sure the table api call get's mapped to the proper SQL Function

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes
The serializers: don't know
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

hm, one last comment
I noticed that this PR both in name and commit message refers to FLINK-36809 which is in fact a different issue
IIUC the correct issue should be FLINK-36840
can you fix this please?

hm, one last comment I noticed that this PR both in name and commit message refers to FLINK-36809 which is in fact a different issue IIUC the correct issue should be FLINK-36840 can you fix this please?

Oh, good catch @snuyanzin

+1 post factum,
I've been checking if that covers all the functions.
The parentheses-less handling in Flink is messy. Theoretically Calcite does not allow calling functions with SqlSyntax.FUNCTION_ID with parentheses. That should include: PI, CURRENT_DATE, MATCH_ROWTIME, MATCH_PROCTIME, CURRENT_CATALOG, ...
However we override some of the above in FlinkSqlOperatorTable with SqlSyntax.FUNCTION, but some we don't override them and the original Calicite's are used. The PR should make the table API work with the current state of the engine, but we should figure out where do those discrepancies come from and decide what we want to do for those functions in the long run.

authors: dawidwys gustavodemorais snuyanzin


text:
[FLINK-9047] Fix slot recycling in case of failed release

What is the purpose of the change
In case that a slot cannot be released it will only recycled/reused if the owning
TaskExecutor is still registered at the SlotPool. If this is not the case then we
drop the slot from the SlotPool.
cc @GJL
Brief change log

Only recycle slots which could not be released if owner is still registered

Verifying this change

Added SlotPoolTest#testReleasingIdleSlotFailed.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

Thanks for the review @GJL. Merging this PR.


text:
Use summarizers to make compaction decisions

In Accumulo 2.0 summarizes were introduced.   Fluo could use these to count column types and notifications.  These counts could be use to make compaction decisions.

spec/parser-spec.js

It would be better to do this after Accumulo 2.1.0 releases w/ new compaction updates.  Doing it now would require writing code against the Accumulo CompactionStrategy which is deprecated as of Accumulo 2.1.0 because its highly unstable.

It would be better to do this after Accumulo 2.1.0 releases w/ new compaction updates.  Doing it now would require writing code against the Accumulo CompactionStrategy which is deprecated as of Accumulo 2.1.0 because its highly unstable.

repo: apache/fluo


text:
Scala 3: ClassCastException on deserializing an Array of intersection types

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
0.7.0
Linux
Java 22
Component(s)
Java, Other
Minimal reproduce step
Please see: https://github.com/yoohaemin/fury-bugreport/blob/b4800828e98d605ab76155505b70b7f36f0b02d5/src/main/scala/Main2.scala
What did you expect to see?
Deserialization success
What did you see instead?
Exception in thread "main" java.lang.ClassCastException: class [Ljava.io.Serializable; cannot be cast to class [Lscala.deriving.Mirror$Singleton; ([Ljava.io.Serializable; is in module java.base of loader 'bootstrap'; [Lscala.deriving.Mirror$Singleton; is in unnamed module of loader 'app')
  at test2.Main$.main(Main2.scala:43)
  at test2.Main.main(Main2.scala)

Anything Else?
No response
Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

@yoohaemin Seems it didn't go to fury code path

I'm starting to think this is a Scala compiler issue, sorry for the trouble.

authors: chaokunyang yoohaemin

repo: apache/fury


text:
GEODE-7943 add synchronization to Subscriptions class

Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[ ] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[ ] Is your initial contribution a single, squashed commit?

[ ] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.

@ringles @sabbeyPivotal @xtreme-prasath @prettyClouds @jhutchison - for your review

@prettyClouds Good idea! I've switched to using that.

@dschneider-pivotal - Is there any underlying technical difference? I had played around with using a Vector which, I think, is practically the same thing. The 'lookup' methods still need to have explicit synchronization although the add and remove don't. Since there is only a single instance of this class in play, having the synchronization at the method level vs at the level of the backing structure also does not make any difference. Although I could buy the argument that that is just how it is currently being used. I think I just convinced myself... :)

why is subscriptionIntegrationTest running out of memory?

authors: dschneider-pivotal jdeppe-pivotal

repo: apache/geode


text:
GEODE-9074: Added update of messageQueueSize at putting message to qu…

…eue.
Also added new statistic numberThreadsPutInQueue for number of threads currently adding message in queue statistic.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[*] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[*] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[*] Is your initial contribution a single, squashed commit?

[*] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.

Hi @agingade, @boglesby, and @nabarunnag, could you review this PR?
Thanks.

Hi @agingade, @boglesby, and @nabarunnag, could you review this PR?
Thanks in advance.

There are a couple other stats along with this one would be useful (in progress and time).
Its not really messages waiting to be queued. Its messages being queued. I think these names are better (but I'm no expert on naming things):

messagesBeingQueued
messagesBeingQueuedInProgress
messagesBeingQueuedTime

A common pattern for updating several stats like this is:
long startTime = _proxy._statistics.startPut();
_messageQueue.put(message);
_proxy._statistics.endPut(_messageQueue.size(), startTime);

startPut would:

increment messages being queued
increment messages being queued in progress
return the start time (which would be non-0 if enable-time-statistics=true)

endPut would:

decrement messages being queued
decrement messages being queued in progress
increment messages being queued
set queue size

Here is an example in CachePerfStats for CacheWriter calls:
public long startCacheWriterCall() {
  stats.incLong(cacheWriterCallsInProgressId, 1);
  return getTime();
}

public void endCacheWriterCall(long start) {
  if (clock.isEnabled()) {
    stats.incLong(cacheWriterCallTimeId, getTime() - start);
  }
  stats.incLong(cacheWriterCallsInProgressId, -1);
  stats.incLong(cacheWriterCallsCompletedId, 1);
}

Do we know what kind of performance impact this change will have?
HARegionQueue.size gets a readLock:
public int size() {
  acquireReadLock();
  try {
    return this.idsAvailable.size();
  } finally {
    releaseReadLock();
  }
}

Here are a few nit-picky comments. Feel free to ignore them.
Need a period at the end of the description:
Threads currently adding a message to the queue. Consistently high values indicate that the queue is full and adds are being delayed

There are extra spaces at various places in the CacheClientProxyStats class.

As I understand Barry`s comment, he proposes that we replace messagesWaitingToQueue with:
messagesBeingQueued
messagesBeingQueuedInProgress
messagesBeingQueuedTime.
Do  you agree with this proposal?

I agree with the proposal :-).
Regardless whether you rename the stats, I'd like to see an InProgress stat and the pattern I suggested:
long startTime = _proxy._statistics.startPut();
_messageQueue.put(message);
_proxy._statistics.endPut(_messageQueue.size(), startTime);

Hi,

InProgress statistics is the same as messagesWaitingToQueue statistic.
messagesBeingQueued is not needed, since we have existing messageQueueSize statistic.
Only addition could be messagesBeingQueuedTime

BR,
Mario

Hi @agingade, @boglesby, and @nabarunnag, could you review this PR?
Thanks in advance.

authors: boglesby mivanac


text:
HADOOP-19062. [thirdparty] Improve create-release RUN script.

NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
JIRA: HADOOP-19062. Improve create-release RUN script.
This change is similar to HADOOP-19038 of hadoop-trunk.

@Hexiaoqiao Thanks for reviewing the code, merged into trunk.

repo: apache/hadoop-thirdparty


text:
HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags (#6789)

Configuration adds new method getEnumSet() to get a set of enum values from a configuration string. <E extends Enum> EnumSet getEnumSet(String key, Class enumClass, boolean ignoreUnknown)
Whitespace is ignored, case is ignored and the value "*" is mapped to "all values of the enum". If "ignoreUnknown" is true then when parsing, unknown values are ignored. This is recommended for forward compatiblity with later versions.

This support is implemented in org.apache.hadoop.fs.s3a.impl.ConfigurationHelper -it can be used elsewhere in the hadoop codebase.

A new private FlagSet class in hadoop common manages a set of enum flags.
It implements StreamCapabilities and can be probed for a specific option being set
(with a prefix)

S3A adds an option fs.s3a.performance.flags which builds a FlagSet with enum type PerformanceFlagEnum

which initially contains {Create, Delete, Mkdir, Open}
the existing fs.s3a.create.performance option sets the flag "Create".
tests which configure fs.s3a.create.performance MUST clear fs.s3a.performance.flags in test setup.

Future performance flags are planned, with different levels of safety and/or backwards compatibility.
Contributed by Steve Loughran
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

aws tests in progress. will merge when it and yetus are happy

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
6m 41s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 11 new or modified test files.

_ branch-3.4 Compile Tests _

+0 :ok:
mvndep
13m 57s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m 44s

branch-3.4 passed

+1 :green_heart:
compile
8m 48s

branch-3.4 passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
7m 48s

branch-3.4 passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
2m  3s

branch-3.4 passed

+1 :green_heart:
mvnsite
1m 26s

branch-3.4 passed

+1 :green_heart:
javadoc
1m  6s

branch-3.4 passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
0m 57s

branch-3.4 passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
2m 11s

branch-3.4 passed

+1 :green_heart:
shadedclient
20m  7s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 19s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
0m 50s

the patch passed

+1 :green_heart:
compile
8m 24s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
8m 24s

the patch passed

+1 :green_heart:
compile
7m 59s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
7m 59s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 58s

the patch passed

+1 :green_heart:
mvnsite
1m 29s

the patch passed

+1 :green_heart:
javadoc
1m  5s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
1m  6s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
2m 31s

the patch passed

+1 :green_heart:
shadedclient
20m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
16m 11s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m  6s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 42s

The patch does not generate ASF License warnings.

152m 47s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6966/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6966

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux d32583bded41 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.4 / 4e22e8d78fb3bf3d2dd3dfeb871fa77e5dacb763

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6966/1/testReport/

Max. process+thread count
2188 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6966/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

authors: hadoop-yetus steveloughran

repo: apache/hadoop


text:
HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWrit…

…eLock instead of synchronize

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
7m 21s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 10s
master passed

+1 :green_heart:
checkstyle
1m 12s
master passed

+1 :green_heart:
spotbugs
2m 12s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 51s
the patch passed

-0 :warning:
checkstyle
1m 10s
hbase-server: The patch generated 5 new + 0 unchanged - 2 fixed = 5 total (was 2)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
19m 10s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.

+1 :green_heart:
spotbugs
2m 19s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 13s
The patch does not generate ASF License warnings.

49m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 02ec0bf357f8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / b26395f3bb

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

Max. process+thread count
84 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/console

versions
git=2.17.1 maven=3.6.3 spotbugs=3.1.12

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 27s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 45s
master passed

+1 :green_heart:
compile
0m 55s
master passed

+1 :green_heart:
shadedjars
6m 30s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 38s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 30s
the patch passed

+1 :green_heart:
compile
0m 57s
the patch passed

+1 :green_heart:
javac
0m 57s
the patch passed

+1 :green_heart:
shadedjars
6m 30s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 36s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
140m 13s
hbase-server in the patch passed.

166m 14s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 69a215f94985 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / b26395f3bb

Default Java
AdoptOpenJDK-1.8.0_232-b09

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/testReport/

Max. process+thread count
4243 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 40s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 53s
master passed

+1 :green_heart:
compile
1m 12s
master passed

+1 :green_heart:
shadedjars
7m 25s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 45s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 32s
the patch passed

+1 :green_heart:
compile
1m 11s
the patch passed

+1 :green_heart:
javac
1m 11s
the patch passed

+1 :green_heart:
shadedjars
7m 25s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 41s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
197m 36s
hbase-server in the patch passed.

229m  8s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 4cc34011e549 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / b26395f3bb

Default Java
AdoptOpenJDK-11.0.6+10

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/testReport/

Max. process+thread count
3726 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

LGTM

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 17s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 56s
master passed

+1 :green_heart:
checkstyle
1m 12s
master passed

+1 :green_heart:
spotbugs
2m  7s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 47s
the patch passed

-0 :warning:
checkstyle
1m 11s
hbase-server: The patch generated 3 new + 0 unchanged - 2 fixed = 3 total (was 2)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
19m  2s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.

+1 :green_heart:
spotbugs
2m 16s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 12s
The patch does not generate ASF License warnings.

42m 51s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 79ca719898ba 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 979ad0f3fc

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

Max. process+thread count
84 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/console

versions
git=2.17.1 maven=3.6.3 spotbugs=3.1.12

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Please fix the checkstyle issue? Looks like they are all line length too long, the width should 100, not 120.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 27s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 53s
master passed

+1 :green_heart:
compile
0m 56s
master passed

+1 :green_heart:
shadedjars
6m 38s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 40s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 31s
the patch passed

+1 :green_heart:
compile
0m 56s
the patch passed

+1 :green_heart:
javac
0m 56s
the patch passed

+1 :green_heart:
shadedjars
6m 33s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 37s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
141m 15s
hbase-server in the patch passed.

167m 38s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux b388525b3205 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 979ad0f3fc

Default Java
AdoptOpenJDK-1.8.0_232-b09

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/testReport/

Max. process+thread count
4662 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

now here are three commits, It should be only one commit

now here are three commits, It should be only one commit

Ah, this is not a problem. Just choose 'Squash and merge' when merging the PR, and modify the commit message accordingly.

Thanks for reviewing @Apache9 @nyl3532016 . I have fixed the checkstyle issue and merged the commits to one.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
2m 34s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 41s
master passed

+1 :green_heart:
compile
1m 10s
master passed

+1 :green_heart:
shadedjars
7m 19s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 40s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m 32s
the patch passed

+1 :green_heart:
compile
1m 12s
the patch passed

+1 :green_heart:
javac
1m 12s
the patch passed

+1 :green_heart:
shadedjars
7m 21s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 41s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
207m 29s
hbase-server in the patch passed.

239m 30s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 68a3e172123c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 979ad0f3fc

Default Java
AdoptOpenJDK-11.0.6+10

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/testReport/

Max. process+thread count
3276 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/2/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 43s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 56s
master passed

+1 :green_heart:
checkstyle
1m 20s
master passed

+1 :green_heart:
spotbugs
2m 13s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 29s
the patch passed

+1 :green_heart:
checkstyle
1m  2s
hbase-server: The patch generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
17m 16s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.

+1 :green_heart:
spotbugs
2m  8s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 16s
The patch does not generate ASF License warnings.

40m 50s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 5fff0cafb5ae 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 979ad0f3fc

Max. process+thread count
94 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/console

versions
git=2.17.1 maven=3.6.3 spotbugs=3.1.12

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  7s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m 14s
master passed

+1 :green_heart:
compile
1m  5s
master passed

+1 :green_heart:
shadedjars
6m 36s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 41s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
4m  1s
the patch passed

+1 :green_heart:
compile
1m  6s
the patch passed

+1 :green_heart:
javac
1m  6s
the patch passed

+1 :green_heart:
shadedjars
6m 34s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 40s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
137m  2s
hbase-server in the patch passed.

165m 17s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 55aca9e8b413 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 979ad0f3fc

Default Java
AdoptOpenJDK-11.0.6+10

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/testReport/

Max. process+thread count
4195 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 26s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m 31s
master passed

+1 :green_heart:
compile
0m 54s
master passed

+1 :green_heart:
shadedjars
6m 34s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 36s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 29s
the patch passed

+1 :green_heart:
compile
0m 57s
the patch passed

+1 :green_heart:
javac
0m 57s
the patch passed

+1 :green_heart:
shadedjars
6m 33s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 37s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
142m 34s
hbase-server in the patch passed.

168m 26s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2738

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 0d608c27d208 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 979ad0f3fc

Default Java
AdoptOpenJDK-1.8.0_232-b09

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/testReport/

Max. process+thread count
3943 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2738/3/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

repo: apache/hbase


text:
HIVE-24082: Expose information whether AcidUtils.ParsedDelta contains statementId

https://issues.apache.org/jira/browse/HIVE-24082

cc @harmandeeps

@kgyrtkirk thanks for the info about tests.
Should I also attach a patch in the JIRA or is the PR enough?

no need; it's fine to have the patch only in the PR.

fyi @pvary

Is there something i should do to have this considered for a merge?

Is there something i should do to have this considered for a merge?

A well placed comment in a good time 😄

authors: findepi kgyrtkirk pvary

repo: apache/hive


text:
[SUPPORT]

Tips before filing an issue

Have you gone through our FAQs?

Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced
when i have a empty partition(partition table),hudi create a partition partition_name=default,the partition field type is date.
that will cause an exception when sync the table to Apache Hive metastore.
Failed in executing SQL ALTER TABLE hudi.f_mid_order_details_ro ADD IF NOT EXISTS   PARTITION (payment_date='2019-10-07') LOCATION '/user/hive/warehouse/hudi/hudi/f_mid_order_details/payment_date=2019-10-07'   PARTITION (payment_date='default') LOCATION '/user/hive/warehouse/hudi/hudi/f_mid_order_details/payment_date=default'

this is the write option
hoodie.datasource.hive_sync.table -> f_mid_order_details hoodie.bloom.index.update.partition.path -> true hoodie.bloom.index.filter.type -> DYNAMIC_V0 hoodie.datasource.write.keygenerator.class -> org.apache.hudi.keygen.SimpleKeyGenerator hoodie.datasource.hive_sync.database -> hudi hoodie.datasource.write.table.type -> MERGE_ON_READ hoodie.datasource.write.partitionpath.field -> payment_date hoodie.datasource.hive_sync.partition_fields -> payment_date hoodie.datasource.hive_sync.partition_extractor_class -> org.apache.hudi.hive.MultiPartKeysValueExtractor hoodie.datasource.write.recordkey.field -> key hoodie.datasource.hive_sync.enable -> true hoodie.upsert.shuffle.parallelism -> 100 hoodie.index.type -> GLOBAL_BLOOM hoodie.datasource.hive_sync.jdbcurl -> jdbc:hive2://172.16.117.73:10000 hoodie.compact.inline -> true hoodie.datasource.write.precombine.field -> ts hoodie.table.name -> f_mid_order_details hoodie.datasource.write.hive_style_partitioning -> true hoodie.datasource.write.operation -> upsert
A clear and concise description of the problem.
To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.
Environment Description

Hudi version :
hudi-0.6
Spark version :
spark-2.4.4
Hive version :
hive-2.3.4
Hadoop version :
hadoop-2.7.3
Storage (HDFS/S3/GCS..) :
hdfs
Running on Docker? (yes/no) :
no

Additional context
Add any other context about the problem here.
Stacktrace
Add the stacktrace of the error.

@peng-xin : Sorry for the delay. Somehow, I missed this GH issue. Looking at how Hive handles Null values, this behavior is consistent with Hive null handling.  what is your partition path type ?One option here is to replace MultiPartKeysValueExtractor with your own implementation which handles the default value in partition-path differently based on your use case.

@peng-xin : Sorry for the delay. Somehow, I missed this GH issue. Looking at how Hive handles Null values, this behavior is consistent with Hive null handling.  what is your partition path type ?One option here is to replace MultiPartKeysValueExtractor with your own implementation which handles the default value in partition-path differently based on your use case.

Please reopen if this is still an issue ?

Please reopen if this is still an issue ?

authors: bvaradar peng-xin

repo: apache/hudi