text:
Performance issue with multiple actions

Environment details:

CentOS 7

Steps to reproduce the issue:

Run benchmark with 100 actions

Provide the actual results and outputs:
TPS is severely dropped.
Additional information you deem important:

Issue always happen with current code base.
I changed container reuse logic.

Actually, I already shared same thing via mailing list, but its not that effective to share image, I opened this issue.

I have tested OpenWhisk in terms of performance for last few months.
I want to share my experience on performance of OpenWhisk and discuss the right direction to go further.
I hope this would be a good starting point to improve performance and helps someone facing similar issue with me.
As per my observation, there are three big parts which have performance issue, akka-http, couchdb, dockerd.
In this thread, I will only focus on dockerd.
Since OpenWhisk can serve many different kinds of actions from different namespaces with different runtimes and container sizes, it would be more precise to assume many heterogenous requests come to OpenWhisk.
In this sense, to use a single action to measure performance of OpenWhisk may not be realistic in the production environment.

With current code base, if we run benchmark with many different number of actions, performance is severely degraded.
When I tested with 100 actions, I got about 30~40 TPS with 3 invoker machines with 40 cores, 128GB memory, 2TB ssd.
(There were 3 invoker containers on each hosts, so total 9 invokers were running)

There could be some differences based on deployment, configuration, the number of components and so on.
But I got about 20K TPS with same setup using 1 action, it seems obvious there is a huge performance degradation along different number of actions.

After deep investigation, I found the main reason is reuse of containers.
When we use only 1 action, logically all containers are reused.
But if we use 100 actions, containers are not fully reused and deletion and creation of containers are occurred. (Contianers are reused only if namespace and action name are same.)
Since performance of docker daemon is not good, it causes TPS dropping.

When one of my colleagues did benchmark against Docker daemon directly, we observed only 30
~ 40 TPS with pause/unpause.
(He also performed similar pause/unpause test with runc and only got about 300 TPS.)
When we included run/rm as well, TPS dropped to 1 ~ 10 TPS.
It means if containers are not reused well, it makes dockerd create/remote/unpause/pause containers and TPS is dropped due to poor performance of docker daemon.
I think this issue exists in all serverless frameworks which are based on docker.
So docker daemon is surely performance bottleneck in OpenWhisk, and it's not that simple to resolve it.
Even though huge performance improvement is made on docker daemon, I doubt it can support more than 1K ~ 2K TPS.
(Because docker is not designed to support this kind of traffic, creating about 1K containers in a second is not that plausible.)

Then, in OpenWhisk, main key factor of performance is reuse of containers, in other word, reducing loads to docker daemon.
With 1 actions, there were not many docker traffics, invoker just called /run requests against running containers again and again.
However, with 100 actions, many containers should be deleted and created, this caused huge performance degradation.
Reuse rate: 95%

In above graph, the red part in a circle is the ratio of reused(warmed) containers.
Reuse rate was more than 95%, but I got only 6K TPS.
5 namespaces + 2 actions each with 100 threads

As reuse rate is decreased, TPS is getting worse.
Reuse rate: 60%

10 namespaces + 10 actions each with 400 threads

I changed ContinerProxy code to reuse container based on runtime and container memory size, and also changed gracePause time to 10 mins.
So once container is created, it is not paused in 10 minutes. No matter requests come from any namespaces, if runtime and memory size are same, containers are reused.
And I got about 20K TPS back with this changes.
100 actions with 740 threads

There could be some side effects and hidden defects of this implementation.
But anyway I got about 20K TPS with 100 actions.
This is because all 100 actions are nodejs6 actions and require same container memory size, accordingly all containers are reused.
In current code, TPS is decreased as the number of actions or number of namespaces are increasing.
Since there is no limit on the number of namespaces and the number of actions in them, more and more namespace and actions are used, performance is getting worse.
With this change, now container reuse rate only depends on runtime and container memory size.
OpenWhisk has 6 runtimes(nodejs6, nodejs8, python, php, swift, java), it can vastly increase reuse rate.
And even if more and more actions and namespaces are created, there is no performance degradation if same runtime is used.
One more good thing of this implementation is, it reduces the traffic against dockerd, I could run benchmark for long-term without any docker daemon problem under heavy loads.
Since I am not that an expert on OpenWhisk, I want to discuss whether there could be any issues or side effects with this change.
With my shallow understanding on OpenWhisk, one security issue could happen when user accesses files in a container.
If user creates any files in the code, files could be exposed to other users as same container can be reused among many users.
(But I am not sure OpenWhisk should also guarantee such kinds of stateful approach because serverless is intrinsically stateless.)
Other than that, I have no idea currently.
Apart from my suggestion, I think one more thing worth to try is this one:
https://github.com/apache/incubator-openwhisk/pull/2795
I want to listen any opinions and feedback.
Thanks in advance

Thanks @style95 for the detailed write up. As I noted on the Apache dev list the main issue in a multitenant deployment is the approach compromises container isolation: while a private deployment of openwhisk may allow you to relax some of the security and isolation, it is a nonstarter to reuse a container for more that one subject otherwise. The container can be entirely tainted where traffic is snooped, secrets stolen, or even the code itself.
The container isolation in place today also serves another purpose: isolating bugs related to concurrency.  If your JavaScript code doesn’t properly handle asynchronous callbacks, you might have two actions still running at once across different functions. So beyond security implications there will be performance isolation that’s also risked.
That said, https://github.com/apache/incubator-openwhisk/pull/2795 from @tysonnorris increases intra-container concurrency to reduce the load on the docker daemon. Further, creating more stem cells containers ahead of time is another way to avoid docker runs.

Closing as stale - the findings are great and are the topic of must exploration.

repo: apache/openwhisk


text:
HDDS-5689. chooseRandom in NetworkTopology should exclude the affinity node too

https://issues.apache.org/jira/browse/HDDS-5689

Thanks @sodonnel for the code review.

source: gharchive/pull-request

repo: apache/ozone

license_type: permissive

license_source: github-api


text:
HDDS-6675. Avoid repeating some S3 tests for different bucket types

What changes were proposed in this pull request?
Some S3 tests are independent of the bucket type parameter.  Currently these are run for regular bucket, link, encrypted bucket and EC bucket.  Save some time by running only once per environment (secure/unsecure).
https://issues.apache.org/jira/browse/HDDS-6675
How was this patch tested?
Verified that tagged tests are run only once, e.g.:
https://github.com/adoroszlai/hadoop-ozone/runs/6236210729#step:5:344
https://github.com/adoroszlai/hadoop-ozone/runs/6236210729#step:5:463

Thanks @sodonnel for the review.


text:
HDDS-6949. Add audit of change bucket ACL S3 API.

What changes were proposed in this pull request?
Add audit of change bucket ACL S3 API.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6949
How was this patch tested?
Only exceptions were caught before
aws s3api --endpoint-url http://localhost:9878 put-bucket-acl --bucket bucket2 --grant-full-control id=testuser2

2022-06-27 06:58:10,980 | ERROR | S3GAudit | user=key | ip=127.0.0.1 | op=PUT_ACL {bucket=[bucket], acl=[]} | ret=FAILURE | javax.ws.rs.WebApplicationException: Can't parse request body to XML.
	at org.apache.hadoop.ozone.s3.endpoint.PutBucketAclRequestUnmarshaller.readFrom(PutBucketAclRequestUnmarshaller.java:82)
	at org.apache.hadoop.ozone.s3.endpoint.BucketEndpoint.putAcl(BucketEndpoint.java:502)
2022-06-27 06:59:33,640 | INFO  | S3GAudit | user=key | ip=127.0.0.1 | op=CREATE_BUCKET {bucket=[bucket2]} | ret=SUCCESS |

post fix
2022-06-27 07:11:07,715 | INFO  | S3GAudit | user=key | ip=127.0.0.1 | op=CREATE_BUCKET {bucket=[bucket2]} | ret=SUCCESS |
2022-06-27 07:11:10,779 | INFO  | S3GAudit | user=key | ip=127.0.0.1 | op=PUT_ACL {bucket=[bucket2], acl=[]} | ret=SUCCESS |

Thanks @kerneltime for updating the patch. I think moving the log statement to putAcl method would be more consistent, since error is also logged there, but this is good enough for fixing the bug.
Also, please try to avoid force-push next time.

There is a separate effort on going to not log customer generated messages into s3g log. The audit log should be at the higher level and this should be the right place.


text:
HDDS-7008. Mark TestOzoneHARatisLogParser as flaky

What changes were proposed in this pull request?
Convert TestOzoneHARatisLogParser to JUnit5 and mark it as @Flaky.
https://issues.apache.org/jira/browse/HDDS-7008
How was this patch tested?
Verified that TestOzoneHARatisLogParser is run as part of integration (flaky):
https://github.com/adoroszlai/hadoop-ozone/actions/runs/3688403684/jobs/6243167414#step:6:3148

authors: ChenSammi adoroszlai


text:
HDDS-9668. Zero-Copy in EC GRPC (write)

What changes were proposed in this pull request?
Support zero-copy in EC (write). Please see details in the attached JIRA.
Performance test results are documented in the JIRA.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-9668
How was this patch tested?
Updated integration test.

TestOzoneConfigurationFields is failing because the new config key ozone.ec.grpc.zerocopy.enabled is missing from ozone-default.xml.

authors: adoroszlai duongkame


text:
HDDS-11168. [hsync] Instantiates audit parameter lazily in DataNode dispatch handler

What changes were proposed in this pull request?
HDDS-11168. [hsync] Instantiates audit parameter lazily in DataNode dispatch handler
Please describe your PR in detail:
The profiler flamegraph shows audit parameters are created for every request even when they are not used. It's a small impact but it would be nice to remove the overhead. It is seen inside writeStateMachine() and applyTransaction()
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11168
How was this patch tested?
Existing unit tests. There's no functional change.

Thanks @duongkame. Merged.


text:
default footer still lists Lightbend copyright

The default footer still seems to be
    Global / pekkoParadoxCopyright :=
      """Copyright © 2011-2022 Lightbend, Inc.
        | Apache Pekko, Pekko, and its feather logo are trademarks of The Apache Software Foundation.""".stripMargin,

Should that still be there like that?

This is covered by https://issues.apache.org/jira/browse/LEGAL-634
My reading is that we need a Lightbend copyright and nobody cares too much about adding an ASF copyright.

source: gharchive/issue

authors: pjfanning raboof

repo: apache/pekko-sbt-paradox


text:
Specify Multi-Release property in manifest

Tag: bugfix
This is a very simple PR that modifies Pinot to include the standard tag Multi-Release in the shaded manifest.
This tag is defined in JEP-238. When specified, Java runtime 9 and newer consider the jar as a multi-release jar. You can read the JEP to have more context, but the TL;DR is that multi release jars may include more than one .class file for each Java class. Each .class can be optionally be associated with a given Java version and the JVM will load the correct class depending on their version.
For example, for a given class X, the jar may include a standard X and a custom version of X to be used in Java 11. In this case when running with Java 8, 9 or 10, the standard X will be used but the runtime is Java 11 or higher, it will use the Java 11 version.
It is important to note that we do not provide custom classes for any runtime, but our dependencies do include them. Given that by default we distribute Pinot in a fat-jar, we need to enable Multi-Release in our uber-jar to inform Java that it should use the newest classes.
For example, Lucene or Roaring Bitmap provide their own classes and even we include them in our uber-jar, Java does not load them right now because the Manifest doesn't say so. In the case of Lucene that is a problem because the standard org.apache.lucene.store.MemorySegmentIndexInput class doesn't work in Java 21.
In order to try this PR I've just did the following:

execute mvn clean
execute mvn install package -DskipTests -Pbin-dist -Pbuild-shaded-jar -Djdk.version=11 -T1C (which is what Dockerbuild does)
unzip build/lib/pinot-all-1.1.0-SNAPSHOT-jar-with-dependencies.jar
verify that META-INF/MANIFES.MF contains Multi-Release: true
verify that META-INF/versions/... is not empty (this is not necessary given these classes were already there, but just to verify that).

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (56ecafc) 61.60% compared to head (febea43) 34.83%.

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #12131       +/-   ##
=============================================
- Coverage     61.60%   34.83%   -26.77%
+ Complexity     1152      945      -207
=============================================
  Files          2406     2330       -76
  Lines        130859   127111     -3748
  Branches      20218    19660      -558
=============================================
- Hits          80611    44283    -36328
- Misses        44373    79686    +35313
+ Partials       5875     3142     -2733

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <ø> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <ø> (ø)

java-11
?

java-21
34.83% <ø> (-26.64%)
:arrow_down:

skip-bytebuffers-false
46.53% <ø> (-15.05%)
:arrow_down:

skip-bytebuffers-true
34.81% <ø> (-26.65%)
:arrow_down:

temurin
34.83% <ø> (-26.77%)
:arrow_down:

unittests
46.54% <ø> (-15.05%)
:arrow_down:

unittests1
46.54% <ø> (-0.16%)
:arrow_down:

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: codecov-commenter gortiz

repo: apache/pinot


text:
[Multi-stage] Support is_enable_group_trim agg option

Introduce is_enable_group_trim as an agg option to enable group trim in the leaf stage.
Group trim can be enabled when there is order by and limit.
E.g.
SELECT /*+ aggOptions(is_partitioned_by_group_by_keys='true', is_enable_group_trim='true') */ {tbl1}.num, COUNT(*), SUM({tbl1}.val), SUM({tbl1}.num), COUNT(DISTINCT {tbl1}.val) FROM {tbl1} WHERE {tbl1}.val >= 0 AND {tbl1}.name != 'a' GROUP BY {tbl1}.num ORDER BY COUNT(*) DESC LIMIT 1

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 0.00%. Comparing base (59551e4) to head (aa3bf27).
Report is 1472 commits behind head on master.

:exclamation:  There is a different number of reports uploaded between BASE (59551e4) and HEAD (aa3bf27). Click for more details.
HEAD has 48 uploads less than BASE

Flag
BASE (59551e4)
HEAD (aa3bf27)

integration
7
2

integration2
3
2

temurin
12
2

java-21
7
2

skip-bytebuffers-true
3
1

skip-bytebuffers-false
7
1

unittests
5
0

unittests1
2
0

java-11
5
0

unittests2
3
0

integration1
2
0

custom-integration1
2
0

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #14664       +/-   ##
=============================================
- Coverage     61.75%    0.00%   -61.76%
=============================================
  Files          2436        3     -2433
  Lines        133233        6   -133227
  Branches      20636        0    -20636
=============================================
- Hits          82274        0    -82274
+ Misses        44911        6    -44905
+ Partials       6048        0     -6048

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <ø> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <ø> (ø)

java-11
?

java-21
0.00% <ø> (-61.63%)
:arrow_down:

skip-bytebuffers-false
0.00% <ø> (-61.75%)
:arrow_down:

skip-bytebuffers-true
0.00% <ø> (-27.73%)
:arrow_down:

temurin
0.00% <ø> (-61.76%)
:arrow_down:

unittests
?

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

cc @bziobrowski

authors: Jackie-Jiang codecov-commenter gortiz


text:
Support  for Deltalake format [FEATURE REQUEST]

Is your feature request related to a problem? Please describe.
Polaris catalog seems to be very good. We are thinking to replace HMS with another open source data catalog. Is there any plan that you ass Delta table format?
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response

I am a proponent of this idea.
From my personal point of view, I think having Polaris support multiple formats, to become a format-agnostic all-purpose catalog makes a lot of sense.
@tpang-cxl You have an amusing but unfortunate typo. :)

authors: chimerasaurus tpang-cxl

repo: apache/polaris


text:
schema created via admin extract api does not contain version

Describe the bug
when schema is created via pulsar-admin schema upload ... command, it creates the schema with version and schema info (JSON serialized version of SchemaAndMetadata). but when the schema is created via pulsar-admin schemas extract ... command, it created the schema with schema info only (JSON serialized version of SchemaInfo).
To Reproduce
Steps to reproduce the behavior:

Create a schema using upload command: pulsar-admin schema upload SampleModel_v0.json
Create another schema version using extract command: pulsar-admin schemas extract persistent://public/default/test-topic-7 -c com.flipkart.pulsarsample2.SampleModel -t json -j pulsarsample2-1.0-SNAPSHOT-jar-with-dependencies.jar
Get the schemas created using get command: pulsar-admin schemas get persistent://public/default/test-topic-7 --version 0 and pulsar-admin schemas get persistent://public/default/test-topic-7 --version 1
Compare the output of both commands to see the difference in data structure

Expected behavior
schema extract command should call the SchemaRegistryService to maintain consistent behavior

this is not a bug, i had made a mistake in my input json. closing this ticket.

repo: apache/pulsar


text:
PIP-155: Drop support for Python2

Motivation
Python 2.x has been deprecated for many years now and it was officially end-of-lifed 2.5 years ago (https://www.python.org/doc/sunset-python-2/).
We have well reached the point by which we need to drop Python 2.7 compatibility for Pulsar client and for Pulsar functions.
Goal
Support only Python 3.5+ for Pulsar client and for Pulsar functions.
API Changes
No changes at this time, though Pulsar Python client library will be now free to use Python3 specific syntaxes and libraries.
Changes

Switch the CI build to run Python client lib tests with Python3
Switch integration tests to use Python3
Stop building and distributing wheel files for Python 2.7

Thoughts on minimum py3 version? 3.6+ and 3.7+ are common choices in the python ecosystem.

Do you know where to find a list of feature/syntax that were change in 3.6 or 3.7?
As in: what could we be free to use if we didn't have to support Py 3.5 and 3.6?

Most details are here: https://docs.python.org/3/whatsnew/index.html

This is actually helpful: https://endoflife.date/python
3.6 has already reached end of life and here will be no more security updates. It does indeed make sense for us to stop supporting it too.

Poke

Closed as resolved. I don't find outstanding dependencies.

authors: dimaqq merlimat tisonkun zbentley


text:
PIP-209: Separate C++/Python clients to own repositories

Motivation
Pulsar C++ code base is in the same main repository for the Pulsar project.
While the decision was the right one at the time, there is a considerable overhead
in keeping the C++ client in its current position.
Issues with the current approach
The Pulsar repository has grown a lot in size and number of active developers.

The frequency of changes in various parts of the codebase has increased to a
point where the amount of resources dedicated to CI is very significant.
Every change in Java code will trigger the CI jobs for the C++ client and every
change in the C++ client will do the same.
During a CI job we are building the C++ client multiple times:

For C++ and Python client tests
To build Python wheels to be included in the pulsar Docker images (for supporting
Pulsar functions)

The release process for Pulsar has become very complex and requires building a
large number of binaries for C++ and Python clients. This has become too much
of a burden during the course of a Pulsar release.

Goal
Decouple the development of C++ and Python client libraries from the development
of the core components of Pulsar in Java.
API Changes
No response
Implementation
Changes
Repositories

Move the C++ client code to a new repository github.com/apache/pulsar-client-c++
Move the Python client code to a new repository github.com/apache/pulsar-client-python

The change will be done without losing any history, extracting a sub-directory into
a new Git repository.
git filter-repo --subdirectory-filter  pulsar-client-cpp

Release process
The release process will be split in multiple parts:

the main Pulsar release will only contain the Java parts (server distribution
and Java client library)
The C++ client will have its own release schedule and versioning
The Python client will have its own release schedule and versioning

Versioning
Both C++ and Python clients will continue with their own individual versioning.
In order to not break anything or cause more confusion, we would need to use
a new version that is bigger than the current version (2.11.x).
The suggestion is to start the new releases for both C++ and Python from 3.0.0.
Existing branches
Existing branches of Pulsar, where the C++ client will still be in the same main
the repository and will be receiving bug fixes in their current location.
The different location of the new C++ code will make the cherry-picking process
slightly more painful in the short term, though it will even out in long term.
Projects dependencies
C++/Python --> Pulsar
Both C++ and Python unit/integration tests are designed to run against a standalone
instance of Pulsar broker. In the current form, they're using the master code
that is built to run the tests.
After the split, the unit tests will use a Docker image of Pulsar. We can use a few
different images to test for compatibility

Latest stable (eg: 2.10.1)
Nightly (Pulsar Docker image published every day from master branch)

Pulsar --> Python
To create a Pulsar image, we are now building the Python client wheel file and then
installing it at build time.
Instead, we are going to include a wheel file for a version of the Python client
that has been already released.
Python --> C++
The Python client library is just a wrapper on top of the C++ client. Today these
are built together, with Python wrapper code residing in a sub-directory of C++ client
code, and compiled using the same CMake build script.
By separating the Python client into a different repository, we are going to
depend on an already released version of the C++ client.
Automated documentation in the website
On the Pulsar website we are auto-generating C++ documentation with the Doxygen
tool and the Python one with Pdoc.
Instead of just fetching the main repo code, the website build job should be
also fetching the new repos to run the tooling.
Alternatives
No response
Anything else?
No response

There're still several works to complete PIP-209. For example, dangling references, release processes, and doc gen processes. I will create issues for all of them and try to pick up some.

@merlimat please add these issues to the description as a checklist so that we can properly track subtasks.

https://github.com/apache/pulsar/issues/17916
https://github.com/apache/pulsar/issues/17917
https://github.com/apache/pulsar/issues/17918

Closed as resolved. Doc updates is tracked by https://github.com/apache/pulsar/issues/17918.
Ping once more @BewareMyPower :)

authors: merlimat tisonkun


text:
Flaky test in DebeziumMySqlSourceTester

The DebeziumMySqlSourceTester integration test is intermittently failing, seen it many times failing.
From a superficial look, the test logic is not accounting for dups.
@zhaijack Can you take a look?
Error Message
expected [9] but found [18]
Stacktrace
java.lang.AssertionError: expected [9] but found [18]
	at org.testng.Assert.fail(Assert.java:96)
	at org.testng.Assert.failNotEquals(Assert.java:776)
	at org.testng.Assert.assertEqualsImpl(Assert.java:137)
	at org.testng.Assert.assertEquals(Assert.java:118)
	at org.testng.Assert.assertEquals(Assert.java:652)
	at org.testng.Assert.assertEquals(Assert.java:662)
	at org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester.validateSourceResult(DebeziumMySqlSourceTester.java:103)
	at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.testDebeziumMySqlConnect(PulsarFunctionsTest.java:1985)
	at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.testDebeziumMySqlSource(PulsarFunctionsTest.java:118)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
	at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Standard Output
------- Starting test [TestClass name=class org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest].pulsar-test-suite([])-------
01:24:05.438 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConnectionPool@155] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xf88fa001, L:/127.0.0.1:37796 - R:localhost/127.0.0.1:37177]] Connected to server
01:24:05.447 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl@102] INFO  org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer perf with config: {
  "topicNames" : [ "public/default/dbserver1.inventory.products" ],
  "topicsPattern" : null,
  "subscriptionName" : "debezium-source-tester",
  "subscriptionType" : "Exclusive",
  "receiverQueueSize" : 1000,
  "acknowledgementsGroupTimeMicros" : 100000,
  "negativeAckRedeliveryDelayMicros" : 60000000,
  "maxTotalReceiverQueueSizeAcrossPartitions" : 50000,
  "consumerName" : null,
  "ackTimeoutMillis" : 0,
  "tickDurationMillis" : 1000,
  "priorityLevel" : 0,
  "cryptoFailureAction" : "FAIL",
  "properties" : { },
  "readCompacted" : false,
  "subscriptionInitialPosition" : "Latest",
  "patternAutoDiscoveryPeriod" : 1,
  "regexSubscriptionMode" : "PersistentOnly",
  "deadLetterPolicy" : null,
  "autoUpdatePartitions" : true,
  "replicateSubscriptionState" : false,
  "resetIncludeHead" : false
}
01:24:05.448 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl@103] INFO  org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {
  "serviceUrl" : "pulsar://localhost:37177",
  "authPluginClassName" : null,
  "authParams" : null,
  "operationTimeoutMs" : 30000,
  "statsIntervalSeconds" : 60,
  "numIoThreads" : 1,
  "numListenerThreads" : 1,
  "connectionsPerBroker" : 1,
  "useTcpNoDelay" : true,
  "useTls" : false,
  "tlsTrustCertsFilePath" : "",
  "tlsAllowInsecureConnection" : false,
  "tlsHostnameVerificationEnable" : false,
  "concurrentLookupRequest" : 5000,
  "maxLookupRequest" : 50000,
  "maxNumberOfRejectedRequestPerConnection" : 50,
  "keepAliveIntervalSeconds" : 30,
  "connectionTimeoutMs" : 10000,
  "requestTimeoutMs" : 60000,
  "defaultBackoffIntervalNanos" : 100000000,
  "maxBackoffIntervalNanos" : 30000000000
}
01:24:05.460 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConnectionPool@155] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x2780c771, L:/127.0.0.1:37802 - R:localhost/127.0.0.1:37177]] Connected to server
01:24:05.460 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ClientCnx@189] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0x2780c771, L:/127.0.0.1:37802 - R:localhost/127.0.0.1:37177] Connected through proxy to target broker at pulsar-broker-1:6650
01:24:05.469 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConsumerImpl@492] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [public/default/dbserver1.inventory.products][debezium-source-tester] Subscribing to topic on cnx [id: 0x2780c771, L:/127.0.0.1:37802 - R:localhost/127.0.0.1:37177]
01:24:05.485 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConsumerImpl@590] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [public/default/dbserver1.inventory.products][debezium-source-tester] Subscribed to topic on localhost/127.0.0.1:37177 -- consumer: 0
01:24:05.486 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@73] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - start debezium mysql server container.
01:24:05.486 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.topologies.PulsarCluster@260] INFO  org.apache.pulsar.tests.integration.topologies.PulsarCluster - Starting external service debezium-mysql-example ...
01:24:48.779 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.topologies.PulsarCluster@264] INFO  org.apache.pulsar.tests.integration.topologies.PulsarCluster - Successfully start external service debezium-mysql-example
01:24:48.779 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@80] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - debezium mysql server already contains preconfigured data.
01:24:48.780 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest@609] INFO  org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest - Run command : /pulsar/bin/pulsar-admin source create --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj --source-type debezium-mysql --sourceConfig {"database.user":"debezium","database.server.id":"184054","database.hostname":"debezium-mysql-example","database.password":"dbz","database.server.name":"dbserver1","database.whitelist":"inventory","database.port":"3306","pulsar.service.url":"pulsar://pulsar-proxy:6650"} --destinationTopicName debe-output-topic-name
01:24:48.783 [tc-okhttp-stream-714857618:org.apache.pulsar.tests.integration.utils.DockerUtils$2@216] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source create --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj --source-type debezium-mysql --sourceConfig {"database.user":"debezium","database.server.id":"184054","database.hostname":"debezium-mysql-example","database.password":"dbz","database.server.name":"dbserver1","database.whitelist":"inventory","database.port":"3306","pulsar.service.url":"pulsar://pulsar-proxy:6650"} --destinationTopicName debe-output-topic-name): Executing...
01:24:53.811 [tc-okhttp-stream-714857618:org.apache.pulsar.tests.integration.utils.DockerUtils$2@221] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source create --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj --source-type debezium-mysql --sourceConfig {"database.user":"debezium","database.server.id":"184054","database.hostname":"debezium-mysql-example","database.password":"dbz","database.server.name":"dbserver1","database.whitelist":"inventory","database.port":"3306","pulsar.service.url":"pulsar://pulsar-proxy:6650"} --destinationTopicName debe-output-topic-name): STDOUT: "Created successfully"
01:24:54.227 [tc-okhttp-stream-714857618:org.apache.pulsar.tests.integration.utils.DockerUtils$2@236] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source create --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj --source-type debezium-mysql --sourceConfig {"database.user":"debezium","database.server.id":"184054","database.hostname":"debezium-mysql-example","database.password":"dbz","database.server.name":"dbserver1","database.whitelist":"inventory","database.port":"3306","pulsar.service.url":"pulsar://pulsar-proxy:6650"} --destinationTopicName debe-output-topic-name): Done
01:24:54.229 [tc-okhttp-stream-714857618:org.apache.pulsar.tests.integration.utils.DockerUtils$2@254] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source create --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj --source-type debezium-mysql --sourceConfig {"database.user":"debezium","database.server.id":"184054","database.hostname":"debezium-mysql-example","database.password":"dbz","database.server.name":"dbserver1","database.whitelist":"inventory","database.port":"3306","pulsar.service.url":"pulsar://pulsar-proxy:6650"} --destinationTopicName debe-output-topic-name): completed with 0
01:24:54.231 [tc-okhttp-stream-111575036:org.apache.pulsar.tests.integration.utils.DockerUtils$2@216] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source get --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): Executing...
01:24:58.663 [tc-okhttp-stream-111575036:org.apache.pulsar.tests.integration.utils.DockerUtils$2@221] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source get --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): STDOUT: {
  "tenant": "public",
  "namespace": "default",
  "name": "test-source-connector-PROCESS-name-rfnjltgj",
  "className": "org.apache.pulsar.io.debezium.mysql.DebeziumMysqlSource",
  "topicName": "debe-output-topic-name",
  "configs": {
    "database.user": "debezium",
    "database.server.id": "184054",
    "database.hostname": "debezium-mysql-example",
    "database.password": "dbz",
    "database.server.name": "dbserver1",
    "database.whitelist": "inventory",
    "database.port": "3306",
    "pulsar.service.url": "pulsar://pulsar-proxy:6650"
  },
  "parallelism": 1,
  "processingGuarantees": "ATLEAST_ONCE",
  "resources": {
    "cpu": 1.0,
    "ram": 1073741824,
    "disk": 10737418240
  },
  "archive": "builtin://debezium-mysql"
}
01:24:59.091 [tc-okhttp-stream-111575036:org.apache.pulsar.tests.integration.utils.DockerUtils$2@236] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source get --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): Done
01:24:59.093 [tc-okhttp-stream-111575036:org.apache.pulsar.tests.integration.utils.DockerUtils$2@254] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source get --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): completed with 0
01:24:59.093 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest@652] INFO  org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest - Get source info : {
  "tenant": "public",
  "namespace": "default",
  "name": "test-source-connector-PROCESS-name-rfnjltgj",
  "className": "org.apache.pulsar.io.debezium.mysql.DebeziumMysqlSource",
  "topicName": "debe-output-topic-name",
  "configs": {
    "database.user": "debezium",
    "database.server.id": "184054",
    "database.hostname": "debezium-mysql-example",
    "database.password": "dbz",
    "database.server.name": "dbserver1",
    "database.whitelist": "inventory",
    "database.port": "3306",
    "pulsar.service.url": "pulsar://pulsar-proxy:6650"
  },
  "parallelism": 1,
  "processingGuarantees": "ATLEAST_ONCE",
  "resources": {
    "cpu": 1.0,
    "ram": 1073741824,
    "disk": 10737418240
  },
  "archive": "builtin://debezium-mysql"
}

01:24:59.096 [tc-okhttp-stream-1260941187:org.apache.pulsar.tests.integration.utils.DockerUtils$2@216] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(7fb8d0ad442dba30755250353dc5105363f799dc3918a8d9647e29e77945c190:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): Executing...
01:25:04.438 [tc-okhttp-stream-1260941187:org.apache.pulsar.tests.integration.utils.DockerUtils$2@221] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(7fb8d0ad442dba30755250353dc5105363f799dc3918a8d9647e29e77945c190:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): STDOUT: {
  "numInstances" : 1,
  "numRunning" : 1,
  "instances" : [ {
    "instanceId" : 0,
    "status" : {
      "running" : true,
      "error" : "",
      "numRestarts" : 0,
      "numReceivedFromSource" : 47,
      "numSystemExceptions" : 0,
      "latestSystemExceptions" : [ ],
      "numSourceExceptions" : 0,
      "latestSourceExceptions" : [ ],
      "numWritten" : 47,
      "lastReceivedTime" : 1569288303519,
      "workerId" : "pulsar-functions-worker-process-lvnez-1"
    }
  } ]
}
01:25:04.854 [tc-okhttp-stream-1260941187:org.apache.pulsar.tests.integration.utils.DockerUtils$2@236] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(7fb8d0ad442dba30755250353dc5105363f799dc3918a8d9647e29e77945c190:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): Done
01:25:04.855 [tc-okhttp-stream-1260941187:org.apache.pulsar.tests.integration.utils.DockerUtils$2@254] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(7fb8d0ad442dba30755250353dc5105363f799dc3918a8d9647e29e77945c190:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): completed with 0
01:25:04.855 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest@671] INFO  org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest - Get source status : {
  "numInstances" : 1,
  "numRunning" : 1,
  "instances" : [ {
    "instanceId" : 0,
    "status" : {
      "running" : true,
      "error" : "",
      "numRestarts" : 0,
      "numReceivedFromSource" : 47,
      "numSystemExceptions" : 0,
      "latestSystemExceptions" : [ ],
      "numSourceExceptions" : 0,
      "latestSourceExceptions" : [ ],
      "numWritten" : 47,
      "lastReceivedTime" : 1569288303519,
      "workerId" : "pulsar-functions-worker-process-lvnez-1"
    }
  } ]
}

01:25:04.861 [tc-okhttp-stream-214872987:org.apache.pulsar.tests.integration.utils.DockerUtils$2@216] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): Executing...
01:25:08.999 [tc-okhttp-stream-214872987:org.apache.pulsar.tests.integration.utils.DockerUtils$2@221] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): STDOUT: {
  "numInstances" : 1,
  "numRunning" : 1,
  "instances" : [ {
    "instanceId" : 0,
    "status" : {
      "running" : true,
      "error" : "",
      "numRestarts" : 0,
      "numReceivedFromSource" : 47,
      "numSystemExceptions" : 0,
      "latestSystemExceptions" : [ ],
      "numSourceExceptions" : 0,
      "latestSourceExceptions" : [ ],
      "numWritten" : 47,
      "lastReceivedTime" : 1569288303519,
      "workerId" : "pulsar-functions-worker-process-lvnez-1"
    }
  } ]
}
01:25:09.413 [tc-okhttp-stream-214872987:org.apache.pulsar.tests.integration.utils.DockerUtils$2@236] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): Done
01:25:09.416 [tc-okhttp-stream-214872987:org.apache.pulsar.tests.integration.utils.DockerUtils$2@254] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - DOCKER.exec(ee72e548c17142b65b4482c3c74a291e0f60533251ffdbc6004e1cf64a69a6f8:/pulsar/bin/pulsar-admin source status --tenant public --namespace default --name test-source-connector-PROCESS-name-rfnjltgj): completed with 0
01:25:09.416 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest@724] INFO  org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest - Get source status : {
  "numInstances" : 1,
  "numRunning" : 1,
  "instances" : [ {
    "instanceId" : 0,
    "status" : {
      "running" : true,
      "error" : "",
      "numRestarts" : 0,
      "numReceivedFromSource" : 47,
      "numSystemExceptions" : 0,
      "latestSystemExceptions" : [ ],
      "numSourceExceptions" : 0,
      "latestSourceExceptions" : [ ],
      "numWritten" : 47,
      "lastReceivedTime" : 1569288303519,
      "workerId" : "pulsar-functions-worker-process-lvnez-1"
    }
  } ]
}

01:25:09.419 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":101}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":101,"name":"scooter","description":"Small 2-wheel scooter","weight":3.140000104904175},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.420 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":102}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":102,"name":"car battery","description":"12V car battery","weight":8.100000381469727},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.421 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":103}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":103,"name":"12-pack drill bits","description":"12-pack of drill bits with sizes ranging from #40 to #3","weight":0.800000011920929},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.421 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":104}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":104,"name":"hammer","description":"12oz carpenter's hammer","weight":0.75},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.421 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":105}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":105,"name":"hammer","description":"14oz carpenter's hammer","weight":0.875},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.422 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":106}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":106,"name":"hammer","description":"16oz carpenter's hammer","weight":1.0},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.422 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":107}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":107,"name":"rocks","description":"box of assorted rocks","weight":5.300000190734863},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.422 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":108}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":108,"name":"jacket","description":"water resistent black wind breaker","weight":0.10000000149011612},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.423 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":109}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":109,"name":"spare tire","description":"24 inch spare tire","weight":22.200000762939453},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288235468}}.
01:25:09.423 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":101}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":101,"name":"scooter","description":"Small 2-wheel scooter","weight":3.140000104904175},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.423 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":102}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":102,"name":"car battery","description":"12V car battery","weight":8.100000381469727},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.424 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":103}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":103,"name":"12-pack drill bits","description":"12-pack of drill bits with sizes ranging from #40 to #3","weight":0.800000011920929},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.424 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":104}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":104,"name":"hammer","description":"12oz carpenter's hammer","weight":0.75},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.424 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":105}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":105,"name":"hammer","description":"14oz carpenter's hammer","weight":0.875},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.424 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":106}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":106,"name":"hammer","description":"16oz carpenter's hammer","weight":1.0},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.425 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":107}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":107,"name":"rocks","description":"box of assorted rocks","weight":5.300000190734863},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.425 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":108}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":108,"name":"jacket","description":"water resistent black wind breaker","weight":0.10000000149011612},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:09.425 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester@96] INFO  org.apache.pulsar.tests.integration.io.DebeziumMySqlSourceTester - Received message: key = {"schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"}],"optional":false,"name":"dbserver1.inventory.products.Key"},"payload":{"id":109}}, value = {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"name"},{"type":"string","optional":true,"field":"description"},{"type":"double","optional":true,"field":"weight"}],"optional":true,"name":"dbserver1.inventory.products.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"version"},{"type":"string","optional":true,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"server_id"},{"type":"int64","optional":false,"field":"ts_sec"},{"type":"string","optional":true,"field":"gtid"},{"type":"string","optional":false,"field":"file"},{"type":"int64","optional":false,"field":"pos"},{"type":"int32","optional":false,"field":"row"},{"type":"boolean","optional":true,"default":false,"field":"snapshot"},{"type":"int64","optional":true,"field":"thread"},{"type":"string","optional":true,"field":"db"},{"type":"string","optional":true,"field":"table"},{"type":"string","optional":true,"field":"query"}],"optional":false,"name":"io.debezium.connector.mysql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"dbserver1.inventory.products.Envelope"},"payload":{"before":null,"after":{"id":109,"name":"spare tire","description":"24 inch spare tire","weight":22.200000762939453},"source":{"version":"0.9.5.Final","connector":"mysql","name":"dbserver1","server_id":0,"ts_sec":0,"gtid":null,"file":"mysql-bin.000003","pos":154,"row":0,"snapshot":true,"thread":null,"db":"inventory","table":"products","query":null},"op":"c","ts_ms":1569288301456}}.
01:25:10.426 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.topologies.PulsarCluster@269] INFO  org.apache.pulsar.tests.integration.topologies.PulsarCluster - Stopping external service debezium-mysql-example ...
01:25:10.429 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.utils.DockerUtils@78] INFO  org.apache.pulsar.tests.integration.utils.DockerUtils - /home/jenkins/jenkins-slave/workspace/pulsar_precommit_integrationtests/tests/integration/target/container-logs/PulsarFunctionsProcessTest-xfuhc/docker.log exists, incrementing
01:25:12.739 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.topologies.PulsarCluster@271] INFO  org.apache.pulsar.tests.integration.topologies.PulsarCluster - Successfully stop external service debezium-mysql-example
01:25:12.744 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConsumerImpl@694] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [public/default/dbserver1.inventory.products] [debezium-source-tester] Closed consumer
01:25:12.764 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.client.impl.PulsarClientImpl@534] INFO  org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://localhost:37177
!!!!!!!!! FAILURE-- 01:25:12.766 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ClientCnx@219] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0x2780c771, L:/127.0.0.1:37802 ! R:localhost/127.0.0.1:37177] Disconnected
[TestClass name=class org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest].pulsar-test-suite([])-------
01:25:12.766 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ClientCnx@219] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xf88fa001, L:/127.0.0.1:37796 ! R:localhost/127.0.0.1:37177] Disconnected

When testing testDebeziumMySqlSource in the PulsarFunctionsProcessTest, the following exception occurred, causing the data to change from the default 9 to 18, during this process, MySQL instance was restarted. The test was not completed and some tests were skipped. Such as field SKIPPED in the log below:
09:57:01.287 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.topologies.PulsarCluster@269] INFO  org.apache.pulsar.tests.integration.topologies.PulsarCluster - Stopping external service debezium-mysql-example ...
09:57:02.663 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.tests.integration.topologies.PulsarCluster@271] INFO  org.apache.pulsar.tests.integration.topologies.PulsarCluster - Successfully stop external service debezium-mysql-example
09:57:02.670 [pulsar-client-io-30-1:org.apache.pulsar.client.impl.ConsumerImpl@676] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [public/default/dbserver1.inventory.products] [debezium-source-tester] Closed consumer
09:57:02.696 [TestNG-method=testDebeziumMySqlSource-1:org.apache.pulsar.client.impl.PulsarClientImpl@533] INFO  org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://localhost:34610
09:57:02.699 [pulsar-client-io-30-1:org.apache.pulsar.client.impl.ClientCnx@219] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xf84594bb, L:/127.0.0.1:51968 ! R:localhost/127.0.0.1:34610] Disconnected
09:57:02.700 [pulsar-client-io-30-1:org.apache.pulsar.client.impl.ClientCnx@219] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0x28051bbf, L:/127.0.0.1:51974 ! R:localhost/127.0.0.1:34610] Disconnected
~~~~~~~~~ SKIPPED -- [TestClass name=class org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest].pulsar-test-suite([])-------------- Starting test [TestClass name=class org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest].pulsar-test-suite([])-------
09:57:02.819 [pulsar-client-io-37-1:org.apache.pulsar.client.impl.ConnectionPool@155] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xb8db778b, L:/127.0.0.1:52158 - R:localhost/127.0.0.1:34610]] Connected to server

But I have not reproduced this problem locally and testDebeziumMySqlSource under PulsarFunctionsThreadTest is also normal.
I think this problem can be solved by deleting the topic or emptying topic data before starting the MySQL instance. what do you think? @sijie @merlimat @jiazhai

@tuteng +1 can you try to do what you suggested?

Assigned to @tuteng for applying his suggestions.

@tuteng Please go ahead to do the deleting of topics.

authors: jiazhai merlimat sijie tuteng


text:
Add test check in branch

Is your feature request related to a problem? Please describe.
Currently, the test check of branch not work, there are two main problems:

the java8 tests missing
the cpp tests not work

Describe the solution you'd like
Add test check in branch.
Additional context
#5733

@wolfstudy @sijie @jiazhai Is this issue still relevant for the Gitlab Actions?

@vzhikserg I think it is still relevant. We should add workflows in GitHub Actions to test branches. It is to ensure any cherry-picks to the branch doesn't break the build. Are you interested in contributing a change to it?

@sijie Do you want to run the same checks as for a normal pull requests, but in "branch-*"?
I think it should be easy to achieve with the following additional block in all CI workflows
on:
  push:
    branches:
      - branch-*

If this is the case, I can work on it

authors: sijie vzhikserg wolfstudy


text:
[Website] pulsar-admin bookies related commands are not shown on pulsar-admin website

pulsar-admin bookies related commands are not shown on pulsar-admin website
https://github.com/apache/pulsar/pull/9692#discussion_r582116556

The issue had no activity for 30 days, mark with Stale label.

I believe there's an incomplete fix for this issue: apache/pulsar-site#80. I would like to complete the fix, could you please assign this issue to me?

@SignorMercurio people can not be assigned until he/she leaves a comment on that issue. Would you like to leave a comment on that issue? Then I can assign it to you

@Anonymitaet Sorry, I mean assigning this issue (#9717) to me. apache/pulsar-site#80 is just a pull request I referenced.

@SignorMercurio ok, assigned


text:
fix failed to generate prometheus stats for transaction

fix #10898
modify regular expression.

thank you @xuesongxs
can you please add a unit test that demonstrate the problem and ensures that we won't see regressions in the future ?

thank you @xuesongxs
can you please add a unit test that demonstrate the problem and ensures that we won't see regressions in the future ?

This feature has been included in ManagedLedgerMetricsTest.testManagedLedgerMetrics().

This feature has been included in ManagedLedgerMetricsTest.testManagedLedgerMetrics().
Sorry, I do not understand your answer.
If you change some line in the code, this should be an effect on tests:

make them fail , and in this case you usually have to modify the tests
in case the code you touch is not covered by a test, then you have to add a test

it looks strange to me that you are changing a constant and there is no change in the tests.

@eolivelli I've added unit test case.

@gaoran10 @congbobo184 @codelipenghui Can you review this PR?

@xuesongxs   in 2.8.0 transaction will not generate wrong managedLedger name, so we shouldn't need to fix this problem.

The pr had no activity for 30 days, mark with Stale label.

authors: codelipenghui congbobo184 eolivelli sijie xuesongxs


text:
Increase the rate of topic creation

Motivation
The count of topic created in 1 second is less than 200, when the topic count of the namespace is about 200,000..
Modifications
The reason is that the time cost of topic exist checking is too long when the topic count is large.
In org.apache.pulsar.broker.admin.AdminResource#checkTopicExistsAsync, all the topics would be checked one by one. As the count of topic grows, the time cost will become longer.
Can we use org.apache.pulsar.broker.namespace.NamespaceService#checkTopicExists to check if the topic is exits?
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: ( no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: ( no )

Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?

[ ] doc-required
(If you need help on updating docs, create a doc issue)

[x] no-need-doc
code optimize

[ ] doc
(If this PR contains doc changes)

Nice catch! But I think this will break the current behavior. Currently, we do not allow the duplicate topic name with the persistent or non-persistent domain, with this change it will only check the persistent topic duplication.
I think we should add the non-persistent topic duplication check back.

Got it. I will update later.

@codelipenghui hi, can you please take a look?

@315157973 @eolivelli thanks，i will Optimize the code and add some test about this function.

@suiyuzeng Hello, please go on

@suiyuzeng Hello, please go on

Sorry, a little busy recently~
Find another problem, such as the test case "testCreatePartitionedTopicHavingNonPartitionTopicWithPartitionSuffix".  Some non partition topic has the partition suffix. In this case, we can not use the cache to check anymore. There is some checking about this issue in the current code, to ensure that we can not create no no-parititon topic whit -partition-xxx suffix.  How about add a config? If a cluster has the no-partition whit the suffix, use the current way to check duplication. If no no-paritition topic whit the suffix, use the cache to check.

@suiyuzeng Hello, please go on

Sorry, a little busy recently~
Find another problem, such as the test case "testCreatePartitionedTopicHavingNonPartitionTopicWithPartitionSuffix". Some non partition topic has the partition suffix. In this case, we can not use the cache to check anymore. There is some checking about this issue in the current code, to ensure that we can not create no no-parititon topic whit -partition-xxx suffix. How about add a config? If a cluster has the no-partition whit the suffix, use the current way to check duplication. If no no-paritition topic whit the suffix, use the cache to check.

@suiyuzeng  Nice catch!

I am cutting 2.8.1, would you have time to address @315157973 's comments?
for the restriction for reserved key-words for topic name, such as -partition-xx suffix for non-partitioned topic name, we have started a discussion for this issue: https://lists.apache.org/list.html?dev@pulsar.apache.org

move to 2.8.2

move to 2.8.3

Removing the release/2.8.3 label as this change will target master and probably won't be cherry picked back to stable branches because it is an optimization.

Closed as stale and conflict. Please rebase and resubmit the patch if it's still relevant.
I believe this patch is worth continuing, but holding a stale patch doesn't work. If @suiyuzeng can rebase and resubmit the patch, I'm glad to follow the PR lifecycle. Otherwise, I may pick it up later. Close.