text:
GH 920 - Topic-based retry support

Please refer to the RetryTopicConfigurer class' JavaDoc for an overview of the functionalities: https://github.com/tomazfernandes/spring-kafka/blob/GH-920/spring-kafka/src/main/java/org/springframework/kafka/retrytopic/RetryTopicConfigurer.java
I've separated the code in 5 commits:

Pausing partitions in the MessageListenerContainer -> so that we don't pause the entire consumer and end up backing off the other partition's messages more than we should
BackOff Manager and ListenerAdapter -> Functionality to read a timestamp header and manage the partition's consumption by listening to events
RetryTopic functionality -> Adds the topics / consumers configuration functionality
A few style checks I've missed
Some improvements to the javadoc

Thanks, @tomazfernandes we prefer rebasing PRs rather than adding merge commits (we'll rebase at the end anyway).

Thanks, @tomazfernandes we prefer rebasing PRs rather than adding merge commits (we'll rebase at the end anyway).

Thanks, @tomazfernandes we prefer rebasing PRs rather than adding merge commits (we'll rebase at the end anyway).

Ok @garyrussell, I actually tried that, but couldn’t push or force push to this branch, it seemed to me because of the open PR (not really experienced with rebasing) Should I try squashing this merge / conflict resolution and rebasing again? Or maybe just leave the conflict there as it was?

Thanks, @tomazfernandes we prefer rebasing PRs rather than adding merge commits (we'll rebase at the end anyway).

Ok @garyrussell, I actually tried that, but couldn’t push or force push to this branch, it seemed to me because of the open PR (not really experienced with rebasing) Should I try squashing this merge / conflict resolution and rebasing again? Or maybe just leave the conflict there as it was?

No don't worry; we'll squash and rebase before merging to master any way; weird that you couldn't force push, though.

No don't worry; we'll squash and rebase before merging to master any way; weird that you couldn't force push, though.

Hi @tomazfernandes I finally found some time to take a quick look at this.
It is very impressive and seems to be a complete solution and will be a valuable addition to the framework. I like the approach.
It's a huge amount of code to review, though so it will take some time.
That said, I am inclined to merge it as-is and, perhaps, document it as an "experimental" feature, at least initially; I am sure we can get it into next month's 2.7.0-M2 milestone, as long as you have time to address a few issues.

First a couple of style issues: we wrap our javadocs at column 90 (when possible) and code at 120 (when possible).

Before the milestone, we will need at least some test cases; but the more coverage, the better; we don't want to trigger our Sonar coverage gate (we're currently a little short of 80%, the gate is currently 70%).

We will need to document the feature in src/reference/asciidoc - it probably deserves a whole new chapter, rather than sprinkling stuff throughout the other sections (aside from documenting the new container properties).

We would not need the docs for M2, but the sooner, the better. Our only strict asciidoctor rule is one-sentence-per-line, but you can see the other docs for examples.
Thanks again for such a significant contribution!!

Hi @tomazfernandes I finally found some time to take a quick look at this.
It is very impressive and seems to be a complete solution and will be a valuable addition to the framework. I like the approach.
It's a huge amount of code to review, though so it will take some time.
That said, I am inclined to merge it as-is and, perhaps, document it as an "experimental" feature, at least initially; I am sure we can get it into next month's 2.7.0-M2 milestone, as long as you have time to address a few issues.

First a couple of style issues: we wrap our javadocs at column 90 (when possible) and code at 120 (when possible).

Before the milestone, we will need at least some test cases; but the more coverage, the better; we don't want to trigger our Sonar coverage gate (we're currently a little short of 80%, the gate is currently 70%).

We will need to document the feature in src/reference/asciidoc - it probably deserves a whole new chapter, rather than sprinkling stuff throughout the other sections (aside from documenting the new container properties).

We would not need the docs for M2, but the sooner, the better. Our only strict asciidoctor rule is one-sentence-per-line, but you can see the other docs for examples.
Thanks again for such a significant contribution!!

Hi @garyrussell, that's awesome news! I'm really glad you liked it, thank you very much, I'm really excited about it. Sure, I'll address these issues and implement the tests, no problem. There are a couple of things I've been meaning to improve as well, so I'll get to it.
Also, if you have any suggestions please let me know.
When would it be a good timeframe for me to finish the changes to get in M2?

Hi @garyrussell, that's awesome news! I'm really glad you liked it, thank you very much, I'm really excited about it. Sure, I'll address these issues and implement the tests, no problem. There are a couple of things I've been meaning to improve as well, so I'll get to it.
Also, if you have any suggestions please let me know.
When would it be a good timeframe for me to finish the changes to get in M2?

M2 is currently scheduled for Feb 17: https://github.com/spring-projects/spring-kafka/milestone/135
So, we have a few weeks.

M2 is currently scheduled for Feb 17: https://github.com/spring-projects/spring-kafka/milestone/135
So, we have a few weeks.

Sounds perfect @garyrussell. Thank you very much for the opportunity.

Sounds perfect @garyrussell. Thank you very much for the opportunity.

Hi @garyrussell, just a quick update. I've implemented some changes and improvements, as well as more than 90% test coverage, both integration and unitary.
What's missing is updating the javadocs and addressing the style changes, as well as the documentation, which I plan on doing this week. Unfortunately I had covid and that set me back a couple of weeks, otherwise I'd probably have everything ready by now.
I'll commit the code as is so you can take a look if you want - it won't build with Gradle due to checkstyle issues, but it should build and run normally on IntelliJ or by disabling checkstyle.
What would be the deadline for committing the javadoc / style adjustments in order to make it into M2?
Thanks!

@tomazfernandes Thanks. The PR needs to be clean and reviews complete by end of day February 16.
The reference manual (which has now moved to spring-kafka-docs/src/main.asciidoc) should be lowest priority and can miss M2 if necessary (although it would be nice to have).
FYI, we are not working this Friday (12th) and next Monday (15th).

Thanks for your concern Gary, it wasn't a fun ride at all, but thankfully not as bad as it could have been. I'm well now.
About the code, I'll clean up the PR, think I should have it done by tomorrow. Then I'll work on the documentation until the 16th, but hopefully I'll have it ready sooner. Of course, if you feel that's too close to the M2 release date we can push it back to the following release, although I'd really like if we can make it for M2.

Hi @garyrussell, I've cleaned the PR and formatted the javadocs and code the way you asked. If you could take a look, I think it should be fine now.
I'll start working on the documentation soon to have it ready for M2, please let me know if there's anything else that needs to be done or anything I might have missed.
Thanks again for the opportunity!

Please rebase to master; thanks.

@garyrussell, seems like this time I got the rebase right. I'll start working on the documentation. Thanks.

Hi @garyrussell, I've created the documentation for the non-blocking retry functionality and updated the relevant parts of the previous documentations. Hope it's ok, please let me know if there's anything else needed.
Thanks!

Thanks a lot @garyrussell! I'll make these adjustments today after my 9-5.
Also, there are a few other improvements / functionalities I'd like do add, such as:

make DLT and DLT processing optional
include the possibility of configuring a global timeout for the retries
include the ability to configure retries from the application.properties files
some other minor things

Do you think it's worth trying to get those into M2, or maybe go with what we have now and create a new PR afterwards for the next release (considering all goes well)? I'd probably have it done by Tuesday (since you won't be working next Friday and Monday).

@garyrussell thanks for your comments, I agree on the DLT part.
For the group id, it gets suffixed with the retry topic's suffix as long as the user provides a groupId for the main topic. You can check it out in RetryTopicConfigurer.java:431, that's one of the functions for the customiser added to the KafkaListenerABPP processListener method. So each retry topic / dlt should get it's own consumer group.
I think there might be some minor adjustments to make in that logic to cover all possibilities for the main endpoint, maybe we can look into that some time after M2. But it should be good for most use cases.
Also, I think I should be able to retrieve the application.properties from the ApplicationContext in order to create a retry configuration, shouldn't I? Or even create a bean for that purpose. Maybe there's a more "Springy" way to do that, which probably is the case for other places of the code as well, feel free to point them out as you see them. I'll take a look into how they're handled internally by the Spring Boot app.
As for the other improvements, I think I'll try to code them until Tuesday and maybe commit it to a separate branch, if this PR hasn't been merged by then. Then you can decide whether or not it's small enough for you to review and merge to M2, or if it goes to the release after that - I'll be good with either. Also maybe I can separate the commits so it doesn't have to be an all or nothing decision.
Thanks a lot again!

I'll add this groupId part to the documentation when I make the changes later today.

Also, I think I should be able to retrieve the application.properties from the ApplicationContext in order to create a retry configuration, shouldn't I? Or even create a bean for that purpose.

Right, the properties are in the environment, but it's separation of concerns; application.properties/.yml is processed by Boot's auto configuration in KafkaAnnotationDrivenConfiguration and KafkaProperties . I think it would be too confusing for automatic configuration via properties to be handled in two different places.
Also Boot has much goodness in its property mapping - e.g. some-timeout: 5s Vs. 5000ms for delays, etc., and completion hints/validation in IDE editors, we don't want to reinvent all that here and, if it's not supported, it would be inconvenient for users.
I am sure the Boot team will be receptive to a PR submitted to enhance the auto configuration for this important enhancement.

Sounds great @garyrussell, I'll look into adding it to Spring Boot's properties then, probably after M2. Thank you very much for your support.

@tomazfernandes you don’t need to comment “done” on each change. It spams our inboxes. Just comment if you disagree about something.
Thanks.

Sorry about that @garyrussell, thanks for letting me know.

Hi @garyrussell, just a quick update.
I have:

fixed a couple of bugs in the original PR, other than that everything is the same there.
created a new PR with a few improvements and new features, and updated the documentation accordingly. If you want to see the diff between the new and the original PR you can check it out here: https://github.com/tomazfernandes/spring-kafka/pull/2

I know you're probably having a busy day ahead of the new version so feel free to choose whether to include the second PR in M2 or to push it back to the next version. I think it should add value, but I know it's too close to the release date, so I'm good with either way.
I also realised I haven't written anything for the 'What's new' part of the documentation, should we write something there? If so, is that something you'd like me to do, or do you prefer doing it yourself?
FYI, today is a holiday here, and I'm under quarantine, so I should be available for anything that comes up.
Thanks once again for the opportunity, it's been a very exciting experience!

Thanks guys for the very precious work that you did/are doing. I am very glad, as this is the exact feature I need right now :). One question though, as I cannot properly configure the back-off for the retries. My config is as follows:
@Bean
public ConcurrentKafkaListenerContainerFactory<String, String> kafkaRetryListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(retryConsumerFactory());
    return factory;
}

private ConsumerFactory<String, String> retryConsumerFactory() {
    Map<String, Object> configProps = new HashMap<>();
    configProps.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    configProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    configProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    return new DefaultKafkaConsumerFactory<>(configProps);
}

And the listener annotated method is:
@KafkaListener(topics = "test-topic",
        containerFactory = "kafkaRetryListenerContainerFactory",
        groupId = "test-group")
@RetryableTopic(
        backoff = @Backoff(100L),
        attempts = 10,
        kafkaTemplate = "kafkaTemplate",
        fixedDelayTopicStrategy = FixedDelayStrategy.SINGLE_TOPIC
)
public void consumeRetry30m(String msg) {
    log.info("Received message: {}", msg);
    throw new RuntimeException();
}

Case that I encounter is that the delay seems to be applied only for the first retry, then it goes after ~10s for each consecutive attempt.

I've tried various configuration, e.g. FixedDelayStrategy.MULTIPLE_TOPICS, higher delay, maxDelay.
Is there some sample code, like a PoC, showing how to use this e2e (or some integration test maybe)?
Thanks again!

Hello @Duncol, thanks for trying out this feature! Which Spring Kafka version are you using? There was an issue with the delay precision that should have been fixed in RC1.
Also note that for a 100ms delay you’d have to set the pollTimeout to at most 50ms, as well as partitionIdleEvent, which might not be ideal. Is this delay amount a requirement? In RC1 it should have more accurate timings OOTB with delays above 1 second.

Hi Tomaz, thanks for the quick response! I am using spring-kafka 2.7.0-M2. This is not a requirement (the requirement is far less frequent TBH), just wanted to check how is this behaving and stumbled across this weird, nearly linear 'magic' 10s delay, which I don' where it comes from. I've tried setting this for 60s and at start it seems quite fine, but then it happened to be quite spontaneous:

(looks like the 2nd and 3rd msg is delayed by configured backoff + 10s)
but when message throughput is less 'quiet' (seems like less than 10s in betweeen), I get almost exact 10s delay for each:

Maybe my configuration is not sufficient somehow? I've provided solely the @Backoff(60000L) for this case. Is there something to be configured for the kafka perhaps, or more in the retry feature itself?

@Duncol Try upgrading to 2.7.0-RC1; with M2, it was tightly coupled to the poll timeout and idle interval; I saw similar issues with M2 and @tomazfernandes made some improvements that are included in RC1.

@garyrussell @tomazfernandes I've upgraded to the RC1, but now it does not seems to retry at all. I still get the KafkaBackoffException (more concise in RC1), logging the approx backoff, but nothing happens after that (needs restart of the service to kick next (single) retry). M2 works well for my configuration (except the aforementioned issues with backof times) and the only thing I've changed was the M2 -> RC1 and config data types (int/long -> String) in @RetryableTopic
More info about my approach: My retry is based on RuntimeExceptions (exception thrown -> should retry, no exception -> no further retries). First retry is initiated in try/catch around the core service; catch block sends wrapped message to first retry topic via KafkaTemplate. When first topic retry is exhausted and the wrapped msg lands in the DLT, I pass that msg to another retry topic (via KafkaTemplate as for the initial retry topic). After second retry topic 'completes', the DLT just logs the msg, with no further passing.

Hi @Duncol, thanks a lot for bringing this up! There's indeed a bug when we use the same factory for the KafkaListener and RetryableTopic annotations. It'll be fixed ASAP, but for now as a workaround if you specify a different factory instance for the RetryableTopic annotation it should work.
This scenario will be added to our integration tests so that it doesn't happen again.
Please let us know how it turns out.
Thanks again!

@tomazfernandes There still seems to be something amiss - when I changed my test app to use a different factory, I see this
2021-04-12 16:49:44.152  INFO 35392 --- [   kgh920-0-C-1] com.example.demo.Kgh920Application       : foo from kgh920
2021-04-12 16:49:44.657  INFO 35392 --- [etry-1000-0-C-1] com.example.demo.Kgh920Application       : foo from kgh920-retry-1000
2021-04-12 16:49:45.167  INFO 35392 --- [etry-2000-0-C-1] o.a.k.clients.consumer.KafkaConsumer     : [Consumer clientId=consumer-kgh920-retry-2000-1, groupId=kgh920-retry-2000] Seeking to offset 3 for partition kgh920-retry-2000-0
2021-04-12 16:49:45.167  WARN 35392 --- [etry-2000-0-C-1] essageListenerContainer$ListenerConsumer : Seek to current after exception; nested exception is org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.listener.KafkaBackoffException: Partition 0 from topic kgh920-retry-2000 is not ready for consumption, backing off for approx. 490 millis.
2021-04-12 16:49:47.175  INFO 35392 --- [etry-2000-0-C-1] com.example.demo.Kgh920Application       : foo from kgh920-retry-2000
2021-04-12 16:49:47.681  INFO 35392 --- [etry-4000-0-C-1] o.a.k.clients.consumer.KafkaConsumer     : [Consumer clientId=consumer-kgh920-retry-4000-4, groupId=kgh920-retry-4000] Seeking to offset 3 for partition kgh920-retry-4000-0
2021-04-12 16:49:47.681  WARN 35392 --- [etry-4000-0-C-1] essageListenerContainer$ListenerConsumer : Seek to current after exception; nested exception is org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.listener.KafkaBackoffException: Partition 0 from topic kgh920-retry-4000 is not ready for consumption, backing off for approx. 1494 millis.
2021-04-12 16:49:49.688  INFO 35392 --- [etry-4000-0-C-1] com.example.demo.Kgh920Application       : foo from kgh920-retry-4000
2021-04-12 16:49:50.200  INFO 35392 --- [etry-8000-0-C-1] o.a.k.clients.consumer.KafkaConsumer     : [Consumer clientId=consumer-kgh920-retry-8000-5, groupId=kgh920-retry-8000] Seeking to offset 3 for partition kgh920-retry-8000-0
2021-04-12 16:49:50.200  WARN 35392 --- [etry-8000-0-C-1] essageListenerContainer$ListenerConsumer : Seek to current after exception; nested exception is org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.listener.KafkaBackoffException: Partition 0 from topic kgh920-retry-8000 is not ready for consumption, backing off for approx. 3488 millis.
2021-04-12 16:49:53.699  INFO 35392 --- [etry-8000-0-C-1] com.example.demo.Kgh920Application       : foo from kgh920-retry-8000
2021-04-12 16:49:54.210  INFO 35392 --- [gh920-dlt-0-C-1] o.a.k.clients.consumer.KafkaConsumer     : [Consumer clientId=consumer-kgh920-dlt-6, groupId=kgh920-dlt] Seeking to offset 3 for partition kgh920-dlt-0
2021-04-12 16:49:54.210  WARN 35392 --- [gh920-dlt-0-C-1] essageListenerContainer$ListenerConsumer : Seek to current after exception; nested exception is org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.listener.KafkaBackoffException: Partition 0 from topic kgh920-dlt is not ready for consumption, backing off for approx. 7489 millis.
2021-04-12 16:50:01.705  INFO 35392 --- [gh920-dlt-0-C-1] com.example.demo.Kgh920Application       : foo from kgh920-dlt

@RetryableTopic(attempts = "5", backoff = @Backoff(delay = 1000, multiplier = 2.0),
			listenerContainerFactory = "retryFactory")
@KafkaListener(id = "kgh920", topics = "kgh920")
public void listen(String in, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
	LOG.info(in + " from " + topic);
	throw new RuntimeException("test");
}

The first retry is +500ms instead of 1s, the next retry is +2s (correct), the next is +2.5s instead of 4, the next retry is +4s instead of 8.
I then see 8 seconds before it goes to the DLT - in earlier versions, I am sure that it went straight to the DLT after the 8 second delivery attempt failed.

@garyrussell, there are two things going on there: the first is a bug I've found now where the current topic's delay is being used instead of the next topic's - so it'd wrongly be 0, 1s, 2s, 4s, 8s.
The 500ms+- differences are related to the poll timeout, which has to be a lot smaller in the retry topics if we have low backoffs such as 1s - not much time to go through all the pause - partition idle event - resume container - resume consumer process considering it takes about 500ms to get there in the first place. I've changed the default configuration to do that in the latest PR regarding this.
I've already fixed this bug and will submit a PR after I run the tests.
If you can test this out again tomorrow with the fixes it'll be great, I think everything should work as expected.
I'll also open the PR for the factories bug.
Thanks!

@tomazfernandes Separate ListenerFactory works, thanks for the quick response, I can move forward now :). Just one side question (maybe more for you @garyrussell ) - whatis the planned release date of spring-kafka 2.7.0?

@Duncol Later today https://github.com/spring-projects/spring-kafka/milestones

Found something strange when writing IT for my retry feature - seems like messages addressed for the -retry topics are doubled?

Found something strange when writing IT for my retry feature - seems like messages addressed for the -retry topics are doubled (except the first one)? I might be raising a false alarm due to some misunderstanding of deeper internals (still learning Kafka), but thought it would be worth mentioning.

Hi @Duncol! Can you share the code where you’re getting this list from? Is it a batch consumer?
Thanks for mentioning, your feedback is very important for us!

@tomazfernandes (just mind it's WIP and I'm looking for a cleaner way to register RecordInterceptor just for tests :) )

The previous screen shows this collection after all retries.

Hmm, that’s strange. What’s the “callbackKafkaListenerContainerFactory” for?
The retry topic’s mechanism relies on the battle tested dead letter publishing recovered to forward messages, and we didn’t see any behavior like this before.
The only possibility I see from the feature side would be if we’re registering two consumers per topic, which again never happened in our tests.
So what comes to mind is this:
Can you check if you have two consumer instances for each topic? You might be able to notice that by putting a breakpoint in the pollAndInvoke method in the KafkaMessageListenerContainer class and checking the instance id for each message consumption.
The other possibility I see would be if you’re for some reason registering two interceptors for the same factory instance, not really sure how to check for that but probably a breakpoint in the interceptor assignment will do.
@garyrussell, any thoughts on this?

According to your screenshot all the interceptors are placing their records into the same consumerRecords collection.
So, it might not be a surprise to see the same data in the tests when you produce a record into a topic.
Rings a bell?

@tomazfernandes 'callbackKafkaListenerContainerFactory' is for our main @KafkaListener. I'm not placing the @RetryableTopic directly over this listener - instead, I have separate handler with two listening methods (each having @KafkaListener + @RetryableTopic over them). I also have one method with @DltHandler.
It goes something like this:

(external service) --msg-->
@KafkaListener(containerFactory = "callbackKafkaListenerContainerFactory") --on-error-->
@KafkaListener(topics = "first-topic", containerFactory = "callbackRetryKafkaListenerContainerFactory")
@RetryableTopic(listenerContainerFactory = callbackRetryAuxKafkaListenerContainerFactory) --retry-exhaustion-->
@DltHandler --send-to-next-retry-topic-->
@KafkaListener(topics = "first-topic", containerFactory = "callbackRetryKafkaListenerContainerFactory")
@RetryableTopic(listenerContainerFactory = callbackRetryAuxKafkaListenerContainerFactory) --retry-exhaustion-->
@DltHandler (do nothing more)

callbackRetryKafkaListenerContainerFactory and callbackRetryAuxKafkaListenerContainerFactory are having the same ConsumerFactory
Each of those methods have own logging which prove, that the retry count is correct (i.e. no doubled messages)
I could perhaps place @RetryableTopic directly over the:

@KafkaListener(containerFactory = "callbackKafkaListenerContainerFactory")
But I wanted to decouple the code this way

@artembilan so in each retry, the message goes through both @KafkaListener's and @RetryableTopic's listenerContainerFactory (and thus - configured consumer)? Such scenario would match my outcome. I would expect, that only initial msg arrival is consumed by consumer configured for callbackRetryKafkaListenerContainerFactory (i.e. @KafkaListener) and the retries are just utilizing callbackRetryAuxKafkaListenerContainerFactory's  (i.e. @RetryableTopic) consumer for single message consumption. Is this the way the retry works?

Hmm, well, then you have two listeners per topic, hence two instances of the same message in you collection... That’s the expected behavior, right?
I didn’t understand what exactly you’re trying to achieve with this pattern: a single @KafkaListener method with @RetryableTopic should suffice.
Also, unless that’s somehow a requirement, you don’t need to handle forwarding to the next topic manually in the DLT method, instead you should let the exception go all the way back to the listener (outside of any try/catch) and the framework will handle message forwarding for you.
Makes sense?

@Duncol Perhaps you misunderstood @tomazfernandes when we had that bug (needing two factories). This is what he meant...
@SpringBootApplication
public class Kgh920Application {

	private static final Logger LOG = LoggerFactory.getLogger(Kgh920Application.class);

	public static void main(String[] args) {
		SpringApplication.run(Kgh920Application.class, args);
	}

	@RetryableTopic(attempts = "5", backoff = @Backoff(delay = 1000, multiplier = 2.0),
			listenerContainerFactory = "retryFactory")
	@KafkaListener(id = "kgh920", topics = "kgh920")
	public void listen(String in, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
		LOG.info(in + " from " + topic);
		throw new RuntimeException("test");
	}

	@DltHandler
	public void dlt(String in, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
		LOG.info(in + " from " + topic);
	}

	@Bean
	ConcurrentKafkaListenerContainerFactory<?, ?> retryFactory(
			ConcurrentKafkaListenerContainerFactoryConfigurer configurer,
			ObjectProvider<ConsumerFactory<Object, Object>> kafkaConsumerFactory,
			KafkaProperties kafkaProps) {

		ConcurrentKafkaListenerContainerFactory<Object, Object> factory = new ConcurrentKafkaListenerContainerFactory<>();
		configurer.configure(factory, kafkaConsumerFactory
				.getIfAvailable(() -> new DefaultKafkaConsumerFactory<>(kafkaProps.buildConsumerProperties())));
		return factory;
	}

}

i.e. specify a different factory on the retry annotation.
Using a different factory is no longer needed now that the bug has been fixed, but it you still want to do it, it needs to go on the retry annotation.

Hmm, well, then you have two listeners per topic, hence two instances of the same message in you collection... That’s the expected behavior, right?
I didn’t understand what exactly you’re trying to achieve with this pattern: a single @KafkaListener method with @RetryableTopic should suffice.
Also, unless that’s somehow a requirement, you don’t need to handle forwarding to the next topic manually in the DLT method, instead you should let the exception go all the way back to the listener (outside of any try/catch) and the framework will handle message forwarding for you.
Makes sense?

Given the fact, that there is a '-retry' topic created (single topic strategy) for the retry I assumed that the listener for the initial topic (without the '-retry' suffix) is somehow consuming messages from the '-retry' topics, thus - duplicate msg. I'll check it with the fixes, thanks a lot again, much appreciate work you are doing!
The manual forward to the next topic is due to different backoff/attempt requirement. Can it be reconfigured on the same retry somehow, as a 'second tier approach' maybe?

Hi @Duncol, sorry, I totally missed this message, was cleaning up the inbox and found it now.
How is the feature working for you, is it behaving as expected?
Can you share more details on your retrial requirements, such as number of attempts, delays, etc? Maybe we can work something out to include this second tier.
Thanks and sorry again for taking this long to reply.

Hi @tomazfernandes, sorry for lack of response, but I was quite occupied with some other tasks. I'll try to implement your suggestion (which looks quite nice and seems a nearly golden bullet for our case - and any further granular adjustments that we might need regarding changing the back-off) and let you know ASAP. One other thing I've stumbled across which might be interesting is that it seems that @RetryableTopic creates additional consumer with the same clientId. This causes the app to throw an exception (logged as WARN) regarding initializing MBean ('Already Exists'):
javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=retry-30m-0 	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:436) ~[?:?] 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) ~[?:?] 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) ~[?:?] 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) ~[?:?] 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) ~[?:?] 	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) ~[?:?] 	at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:64) ~[kafka-clients-2.6.0.jar:?] 	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:814) ~[kafka-clients-2.6.0.jar:?] 	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:632) ~[kafka-clients-2.6.0.jar:?] 	at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createRawConsumer(DefaultKafkaConsumerFactory.java:366) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:334) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumerWithAdjustedProperties(DefaultKafkaConsumerFactory.java:310) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:277) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumer(DefaultKafkaConsumerFactory.java:254) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.<init>(KafkaMessageListenerContainer.java:699) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.listener.KafkaMessageListenerContainer.doStart(KafkaMessageListenerContainer.java:317) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:384) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerContainer.java:206) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:384) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.config.KafkaListenerEndpointRegistry.startIfNecessary(KafkaListenerEndpointRegistry.java:312) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.kafka.config.KafkaListenerEndpointRegistry.start(KafkaListenerEndpointRegistry.java:257) ~[spring-kafka-2.7.0-M2.jar:2.7.0-M2] 	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-5.3.3.jar:5.3.3] 	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) ~[spring-context-5.3.3.jar:5.3.3] 	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) ~[spring-context-5.3.3.jar:5.3.3] 	at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?] 	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) ~[spring-context-5.3.3.jar:5.3.3] 	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) ~[spring-context-5.3.3.jar:5.3.3] 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:940) ~[spring-context-5.3.3.jar:5.3.3] 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591) ~[spring-context-5.3.3.jar:5.3.3] 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767) ~[spring-boot-2.4.2.jar:2.4.2] 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759) ~[spring-boot-2.4.2.jar:2.4.2] 	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426) ~[spring-boot-2.4.2.jar:2.4.2] 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:326) ~[spring-boot-2.4.2.jar:2.4.2] 	at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:123) ~[spring-boot-test-2.4.2.jar:2.4.2] 	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99) ~[spring-test-5.3.3.jar:5.3.3] 	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124) ~[spring-test-5.3.3.jar:5.3.3] 	at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:124) ~[spring-test-5.3.3.jar:5.3.3] 	at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:190) ~[spring-test-5.3.3.jar:5.3.3] 	at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:132) ~[spring-test-5.3.3.jar:5.3.3] 	at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:244) ~[spring-test-5.3.3.jar:5.3.3] 	at org.springframework.test.context.junit.jupiter.SpringExtension.postProcessTestInstance(SpringExtension.java:138) ~[spring-test-5.3.3.jar:5.3.3] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeTestInstancePostProcessors$6(ClassBasedTestDescriptor.java:350) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.executeAndMaskThrowable(ClassBasedTestDescriptor.java:355) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeTestInstancePostProcessors$7(ClassBasedTestDescriptor.java:350) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?] 	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?] 	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?] 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] 	at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[?:?] 	at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735) ~[?:?] 	at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) ~[?:?] 	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeTestInstancePostProcessors(ClassBasedTestDescriptor.java:349) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$instantiateAndPostProcessTestInstance$4(ClassBasedTestDescriptor.java:270) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.instantiateAndPostProcessTestInstance(ClassBasedTestDescriptor.java:269) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$testInstancesProvider$2(ClassBasedTestDescriptor.java:259) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at java.util.Optional.orElseGet(Optional.java:362) ~[?:?] 	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$testInstancesProvider$3(ClassBasedTestDescriptor.java:258) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.execution.TestInstancesProvider.getTestInstances(TestInstancesProvider.java:31) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$prepare$0(TestMethodTestDescriptor.java:101) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.prepare(TestMethodTestDescriptor.java:100) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.prepare(TestMethodTestDescriptor.java:65) ~[junit-jupiter-engine-5.7.0.jar:5.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$prepare$1(NodeTestTask.java:111) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.prepare(NodeTestTask.java:111) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:79) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at java.util.ArrayList.forEach(ArrayList.java:1511) ~[?:?] 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at java.util.ArrayList.forEach(ArrayList.java:1511) ~[?:?] 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) ~[junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) [junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) [junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) [junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) [junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) [junit-platform-engine-1.7.0.jar:1.7.0] 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170) [junit-platform-launcher-1.2.0.jar:1.2.0] 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154) [junit-platform-launcher-1.2.0.jar:1.2.0] 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90) [junit-platform-launcher-1.2.0.jar:1.2.0] 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:142) [surefire-junit-platform-2.22.0.jar:2.22.0] 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:117) [surefire-junit-platform-2.22.0.jar:2.22.0] 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383) [surefire-booter-2.22.0.jar:2.22.0] 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344) [surefire-booter-2.22.0.jar:2.22.0] 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) [surefire-booter-2.22.0.jar:2.22.0] 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417) [surefire-booter-2.22.0.jar:2.22.0]
I've tried separate listenerContainerFactory for the @RetryableTopic with explicit clientId, different from the one I set for particular @KafkaListener, but the one from @KafkaListener takes precedence. When I comment out the @Retryable topic, those additional consumers are not being registered and problem disappears.
Have you encounter something similar maybe? Those WARNs do not break anything, but as I've read, it may cut off some metrics and pollutes out logfile a bit :)

Hi @Duncol, thanks for bringing this up!
I'm having trouble reproducing your issue. When we specify a clientIdPrefix in the @KafkaListener annotation, the prefix gets suffixed by the topic's suffix (e.g. retry-250). And when we don't, Kafka's ConsumerConfig class has a monotonically increasing number that it appends to the consumer's id so that they're unique at least within the same app instance (line 576). In my tests all consumers end up with different client ids.
Which Spring Kafka version are you using? Can you share more details on how we can reproduce the issue?
Maybe @garyrussell has something to add?
Thanks again for your input!

I can't reproduce it either; when I enable info logging for sample-04 I get these client ids when I add clientIdPrefix = "test" to the listener:
client.id = test-retry-10000-0
client.id = test-retry-4000-0
client.id = test-0
client.id = test-retry-2000-0
client.id = test-dlt-0
client.id = test-retry-8000-0

The exception seems to indicate you have multiple listeners with the same clientIdPrefix.
If you can put together a minimal example that exhibits the behavior, we can take a look.

authors: Duncol artembilan garyrussell tomazfernandes


text:
Tomcat error when trying to run my app as Spring Boot in Spring Tool suite

2016-01-15 15:34:08.625  INFO 7604 --- [           main] .s.p.TmoSpaceProductofferingsApplication : No active profile set, falling back to default profiles: default
2016-01-15 15:34:08.644  INFO 7604 --- [           main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@7a55af6b: startup date [Fri Jan 15 15:34:08 EST 2016]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@223d2c72
2016-01-15 15:34:11.031  INFO 7604 --- [           main] o.s.b.f.xml.XmlBeanDefinitionReader      : Loading XML bean definitions from class path resource [cassandra-config.xml]
2016-01-15 15:34:11.341  INFO 7604 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'beanNameViewResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]]
2016-01-15 15:34:11.437  INFO 7604 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'ignoredPathsWebSecurityConfigurerAdapter' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.security.SpringBootWebSecurityConfiguration; factoryMethodName=ignoredPathsWebSecurityConfigurerAdapter; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/security/SpringBootWebSecurityConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.actuate.autoconfigure.ManagementWebSecurityAutoConfiguration; factoryMethodName=ignoredPathsWebSecurityConfigurerAdapter; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/actuate/autoconfigure/ManagementWebSecurityAutoConfiguration.class]]
2016-01-15 15:34:11.450  INFO 7604 --- [           main] a.ConfigurationClassBeanDefinitionReader : Skipping bean definition for [BeanMethod:name=cassandraTemplate,declaringClass=org.springframework.boot.autoconfigure.data.cassandra.CassandraDataAutoConfiguration]: a definition for bean 'cassandraTemplate' already exists. This top-level bean definition is considered as an override.
2016-01-15 15:34:12.132  INFO 7604 --- [           main] c.s.PropertySourcesPlaceholderConfigurer : Loading properties file from class path resource [cassandra-cloud.properties]
2016-01-15 15:34:12.146  INFO 7604 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=972e8ed7-ce81-3d36-b368-7a743778b055
2016-01-15 15:34:12.158  INFO 7604 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2016-01-15 15:34:12.307  INFO 7604 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [class org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$93eaf2bd] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2016-01-15 15:34:12.726  INFO 7604 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (http)
2016-01-15 15:34:12.745  INFO 7604 --- [           main] o.apache.catalina.core.StandardService   : Starting service Tomcat
2016-01-15 15:34:12.747  INFO 7604 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.0.30
2016-01-15 15:34:13.014  INFO 7604 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2016-01-15 15:34:13.014  INFO 7604 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 4370 ms
2016-01-15 15:34:13.724 ERROR 7604 --- [cat-startStop-1] org.apache.catalina.core.ContainerBase   : A child container failed during start
java.util.concurrent.ExecutionException: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Tomcat].StandardHost[localhost].StandardContext[]]
at java.util.concurrent.FutureTask.report(FutureTask.java:122) [na:1.8.0_65]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) [na:1.8.0_65]
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:916) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:871) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398) [tomcat-embed-core-8.0.30.jar:8.0.30]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Tomcat].StandardHost[localhost].StandardContext[]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154) [tomcat-embed-core-8.0.30.jar:8.0.30]
... 6 common frames omitted
Caused by: java.lang.NoSuchMethodError: javax.servlet.ServletContext.addFilter(Ljava/lang/String;Ljavax/servlet/Filter;)Ljavax/servlet/FilterRegistration$Dynamic;
at org.springframework.boot.context.embedded.AbstractFilterRegistrationBean.onStartup(AbstractFilterRegistrationBean.java:225) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.FilterRegistrationBean.onStartup(FilterRegistrationBean.java:41) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.selfInitialize(EmbeddedWebApplicationContext.java:225) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.access$000(EmbeddedWebApplicationContext.java:85) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext$1.onStartup(EmbeddedWebApplicationContext.java:209) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatStarter.onStartup(TomcatStarter.java:55) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5244) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
... 6 common frames omitted
2016-01-15 15:34:13.727 ERROR 7604 --- [           main] org.apache.catalina.core.ContainerBase   : A child container failed during start
java.util.concurrent.ExecutionException: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Tomcat].StandardHost[localhost]]
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_65]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_65]
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:916) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:441) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:769) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.startup.Tomcat.start(Tomcat.java:344) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.initialize(TomcatEmbeddedServletContainer.java:89) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.(TomcatEmbeddedServletContainer.java:76) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory.getTomcatEmbeddedServletContainer(TomcatEmbeddedServletContainerFactory.java:462) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory.getEmbeddedServletContainer(TomcatEmbeddedServletContainerFactory.java:168) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.createEmbeddedServletContainer(EmbeddedWebApplicationContext.java:160) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.onRefresh(EmbeddedWebApplicationContext.java:130) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:532) [spring-context-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:118) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:764) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:357) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1124) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1113) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at com.tmobile.space.productofferings.TmoSpaceProductofferingsApplication.main(TmoSpaceProductofferingsApplication.java:12) [classes/:na]
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Tomcat].StandardHost[localhost]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154) [tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_65]
Caused by: org.apache.catalina.LifecycleException: A child container failed during start
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:924) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:871) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [tomcat-embed-core-8.0.30.jar:8.0.30]
... 6 common frames omitted
2016-01-15 15:34:13.728  WARN 7604 --- [           main] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start embedded container; nested exception is org.springframework.boot.context.embedded.EmbeddedServletContainerException: Unable to start embedded Tomcat
2016-01-15 15:34:13.736 ERROR 7604 --- [           main] o.s.boot.SpringApplication               : Application startup failed
org.springframework.context.ApplicationContextException: Unable to start embedded container; nested exception is org.springframework.boot.context.embedded.EmbeddedServletContainerException: Unable to start embedded Tomcat
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.onRefresh(EmbeddedWebApplicationContext.java:133) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:532) ~[spring-context-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:118) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:764) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:357) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1124) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1113) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at com.tmobile.space.productofferings.TmoSpaceProductofferingsApplication.main(TmoSpaceProductofferingsApplication.java:12) [classes/:na]
Caused by: org.springframework.boot.context.embedded.EmbeddedServletContainerException: Unable to start embedded Tomcat
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.initialize(TomcatEmbeddedServletContainer.java:99) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.(TomcatEmbeddedServletContainer.java:76) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory.getTomcatEmbeddedServletContainer(TomcatEmbeddedServletContainerFactory.java:462) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory.getEmbeddedServletContainer(TomcatEmbeddedServletContainerFactory.java:168) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.createEmbeddedServletContainer(EmbeddedWebApplicationContext.java:160) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.onRefresh(EmbeddedWebApplicationContext.java:130) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
... 8 common frames omitted
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardServer[-1]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.startup.Tomcat.start(Tomcat.java:344) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer.initialize(TomcatEmbeddedServletContainer.java:89) ~[spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
... 13 common frames omitted
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardService[Tomcat]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:769) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
... 15 common frames omitted
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Tomcat]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:441) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
... 17 common frames omitted
Caused by: org.apache.catalina.LifecycleException: A child container failed during start
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:924) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) ~[tomcat-embed-core-8.0.30.jar:8.0.30]
... 19 common frames omitted
2016-01-15 15:34:13.744  INFO 7604 --- [       Thread-1] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@223d2c72: startup date [Fri Jan 15 15:34:07 EST 2016]; root of context hierarchy

use   javax.servlet
servlet-api     3.0 or above

source: gharchive/issue

repo: spring-projects/spring-loaded

license_type: permissive

license_source: bigquery


text:
Surefire failing on Java 13

How to reproduce
git clone https://github.com/spring-projects/spring-petclinic.git
cd spring-petclinic
$ ./mvnw package
[INFO] Scanning for projects...
[INFO]
[INFO] ------------< org.springframework.samples:spring-petclinic >------------
[INFO] Building petclinic 2.1.0.BUILD-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (default) @ spring-petclinic ---
[INFO] argLine set to -javaagent:/Users/pkumari/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/Users/pkumari/java/spring-petclinic/target/jacoco.exec
[INFO]
[INFO] --- git-commit-id-plugin:2.2.6:revision (default) @ spring-petclinic ---
[INFO] dotGitDirectory /Users/pkumari/java/spring-petclinic/.git
[INFO] git.build.user.name Priti Kumari
[INFO] git.build.user.email pkumari@redhat.com
[INFO] git.branch master
[INFO] --always = true
[INFO] --dirty = -dirty
[INFO] --abbrev = 7
[INFO] Tag refs [[Ref[refs/tags/1.5.x=c36452a2c34443ae26b4ecbba4f149906af14717]]]
[INFO] Created map: [{}]
[INFO] evalCommit is [f284b29501946f933ee4473ecf59096d013df31c]
[INFO] git.commit.id.describe f284b29
[INFO] git.commit.id f284b29501946f933ee4473ecf59096d013df31c
[INFO] git.commit.id.abbrev f284b29
[INFO] git.dirty false
[INFO] git.commit.user.name Stephane Nicoll
[INFO] git.commit.user.email snicoll@pivotal.io
[INFO] git.commit.message.full Upgrade to Spring Boot 2.1.9.RELEASE
[INFO] git.commit.message.short Upgrade to Spring Boot 2.1.9.RELEASE
[INFO] git.commit.time 2019-10-03T17:29:27+0530
[INFO] git.remote.origin.url https://github.com/spring-projects/spring-petclinic.git
[INFO] git.tags
[INFO] evalCommit is [f284b29501946f933ee4473ecf59096d013df31c]
[INFO] Tag refs [[Ref[refs/tags/1.5.x=c36452a2c34443ae26b4ecbba4f149906af14717]]]
[INFO] Created map: [{}]
[INFO] git.closest.tag.name
[INFO] evalCommit is [f284b29501946f933ee4473ecf59096d013df31c]
[INFO] Tag refs [[Ref[refs/tags/1.5.x=c36452a2c34443ae26b4ecbba4f149906af14717]]]
[INFO] Created map: [{}]
[INFO] git.closest.tag.commit.count
[INFO] git.total.commit.count 665
[INFO] git.build.time 2019-10-09T15:19:55+0530
[INFO] git.build.version 2.1.0.BUILD-SNAPSHOT
[INFO] git.build.host Pritis-MacBook-Pro
[INFO] git.commit.id.describe-short f284b29
[INFO] found property git.tags
[INFO] found property git.closest.tag.commit.count
[INFO] found property git.build.version
[INFO] found property git.commit.user.name
[INFO] found property git.commit.id.abbrev
[INFO] found property git.branch
[INFO] found property git.build.host
[INFO] found property git.commit.id.describe-short
[INFO] found property git.total.commit.count
[INFO] found property git.commit.id.describe
[INFO] found property git.build.user.email
[INFO] found property git.commit.id
[INFO] found property git.commit.message.short
[INFO] found property git.commit.user.email
[INFO] found property git.closest.tag.name
[INFO] found property git.commit.time
[INFO] found property git.build.time
[INFO] found property git.build.user.name
[INFO] found property git.dirty
[INFO] found property git.commit.message.full
[INFO] found property git.remote.origin.url
[INFO] Reading existing properties file [/Users/pkumari/java/spring-petclinic/target/classes/git.properties] (for module petclinic)...
[INFO] Properties file [/Users/pkumari/java/spring-petclinic/target/classes/git.properties] is up-to-date (for module petclinic)...
[INFO]
[INFO] --- spring-boot-maven-plugin:2.1.9.RELEASE:build-info (default) @ spring-petclinic ---
[INFO]
[INFO] --- wro4j-maven-plugin:1.8.0:run (default) @ spring-petclinic ---
[INFO] /Users/pkumari/java/spring-petclinic/src/main/less
[INFO] Executing the mojo:
[INFO] Wro4j Model path: /Users/pkumari/java/spring-petclinic/src/main/wro/wro.xml
[INFO] targetGroups: null
[INFO] minimize: true
[INFO] ignoreMissingResources: null
[INFO] parallelProcessing: false
[INFO] buildDirectory: /Users/pkumari/java/spring-petclinic/target
[INFO] destinationFolder: /Users/pkumari/java/spring-petclinic/target
[INFO] cssDestinationFolder: /Users/pkumari/java/spring-petclinic/target/classes/static/resources/css
[INFO] The following groups will be processed: [petclinic]
[INFO] folder: /Users/pkumari/java/spring-petclinic/target/classes/static/resources/css
[INFO] processing group: petclinic.css
[WARNING] Less warnings are:
[WARNING] 10:1 Cannot link source map. Css result location is not know and could not be deduced from input less source..
[INFO] file size: petclinic.css -> 152399 bytes
[INFO] /Users/pkumari/java/spring-petclinic/target/classes/static/resources/css/petclinic.css (152399 bytes)
[INFO] folder: /Users/pkumari/java/spring-petclinic/target
[INFO] processing group: petclinic.js
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ spring-petclinic ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 35 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ spring-petclinic ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 25 source files to /Users/pkumari/java/spring-petclinic/target/classes
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ spring-petclinic ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/pkumari/java/spring-petclinic/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ spring-petclinic ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 11 source files to /Users/pkumari/java/spring-petclinic/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ spring-petclinic ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file /Users/pkumari/java/spring-petclinic/target/surefire-reports/2019-10-09T15-20-03_591-jvmRun1.dumpstream
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10.828 s
[INFO] Finished at: 2019-10-09T15:20:04+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project spring-petclinic: There are test failures.
[ERROR]
[ERROR] Please refer to /Users/pkumari/java/spring-petclinic/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /Users/pkumari/java/spring-petclinic && /Library/Java/JavaVirtualMachines/adoptopenjdk-13.jdk/Contents/Home/bin/java -javaagent:/Users/pkumari/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/Users/pkumari/java/spring-petclinic/target/jacoco.exec -jar /Users/pkumari/java/spring-petclinic/target/surefire/surefirebooter2569740085637980547.jar /Users/pkumari/java/spring-petclinic/target/surefire 2019-10-09T15-20-03_591-jvmRun1 surefire12554066826053002032tmp surefire_015926845538724409623tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 134
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /Users/pkumari/java/spring-petclinic && /Library/Java/JavaVirtualMachines/adoptopenjdk-13.jdk/Contents/Home/bin/java -javaagent:/Users/pkumari/.m2/repository/org/jacoco/org.jacoco.agent/0.8.2/org.jacoco.agent-0.8.2-runtime.jar=destfile=/Users/pkumari/java/spring-petclinic/target/jacoco.exec -jar /Users/pkumari/java/spring-petclinic/target/surefire/surefirebooter2569740085637980547.jar /Users/pkumari/java/spring-petclinic/target/surefire 2019-10-09T15-20-03_591-jvmRun1 surefire12554066826053002032tmp surefire_015926845538724409623tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 134
[ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
[ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
[ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
[ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
[ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
[ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
[ERROR] 	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
[ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
[ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
[ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
[ERROR] 	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
[ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR] 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
[ERROR] 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
[ERROR] 	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
[ERROR] 	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
[ERROR] 	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
[ERROR] 	at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
[ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
[ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
[ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
[ERROR] 	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
[ERROR] 	at org.apache.maven.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:39)
[ERROR] 	at org.apache.maven.wrapper.WrapperExecutor.execute(WrapperExecutor.java:122)
[ERROR] 	at org.apache.maven.wrapper.MavenWrapperMain.main(MavenWrapperMain.java:61)
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

macOs mojave version 10.14.6 (18G103) and default java version
$ java -version
openjdk version "13" 2019-09-17
OpenJDK Runtime Environment AdoptOpenJDK (build 13+33)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 13+33, mixed mode, sharing)

I searched the issue in stack overflow [1] but could not be able to find any concrete solution that can fit here. Please have a look.
[1] https://stackoverflow.com/questions/55272870/surefire-maven-plugin-corrupted-stdout-by-directly-writing-to-native-stream-in

and default java version

Java 13 is not supported by Spring Boot 2.1.x that Spring Petclinic currently uses. Switching to Java 11 should fix the problem.
This is a build issue but it would be worth investigating why tests are not invoked with Java 13.

Ran into the same issue, make sure JAVA_HOME is set -- for whatever reason it's not exported by default on macOS. /usr/libexec/java_home will print the directory, if you're using a version manager however you'll have to use whatever mechanism it provides to set the variable (asdf comes with a script, sdkman should just work, IIRC)

The issue seems to be related to the jacoco agent.

Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/petclinic
[petclinic] $ /bin/sh -xe /tmp/jenkins8131710444436746716.sh

git clone https://github.com/spring-projects/spring-petclinic.git
Cloning into 'spring-petclinic'...
cd spring-petclinic
mvn package
[INFO] Scanning for projects...
[ERROR] Internal error: java.lang.ArrayIndexOutOfBoundsException: 8393 -> [Help 1]
org.apache.maven.InternalErrorException: Internal error: java.lang.ArrayIndexOutOfBoundsException: 8393
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:120)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 8393
at org.codehaus.plexus.util.xml.pull.MXParser.parsePI (MXParser.java:2502)
at org.codehaus.plexus.util.xml.pull.MXParser.nextImpl (MXParser.java:1283)
at org.codehaus.plexus.util.xml.pull.MXParser.next (MXParser.java:1131)
at org.codehaus.plexus.util.xml.pull.MXParser.nextTag (MXParser.java:1116)
at org.apache.maven.model.io.xpp3.MavenXpp3ReaderEx.parsePluginExecution (MavenXpp3ReaderEx.java:3585)
at org.apache.maven.model.io.xpp3.MavenXpp3ReaderEx.parsePlugin (MavenXpp3ReaderEx.java:3380)
at org.apache.maven.model.io.xpp3.MavenXpp3ReaderEx.parseBuild (MavenXpp3ReaderEx.java:1302)
at org.apache.maven.model.io.xpp3.MavenXpp3ReaderEx.parseModel (MavenXpp3ReaderEx.java:2833)
at org.apache.maven.model.io.xpp3.MavenXpp3ReaderEx.read (MavenXpp3ReaderEx.java:4690)
at org.apache.maven.model.io.xpp3.MavenXpp3ReaderEx.read (MavenXpp3ReaderEx.java:875)
at org.apache.maven.model.io.DefaultModelReader.read (DefaultModelReader.java:105)
at org.apache.maven.model.io.DefaultModelReader.read (DefaultModelReader.java:82)
at org.apache.maven.model.building.DefaultModelProcessor.read (DefaultModelProcessor.java:84)
at org.apache.maven.model.building.DefaultModelBuilder.readModel (DefaultModelBuilder.java:544)
at org.apache.maven.model.building.DefaultModelBuilder.build (DefaultModelBuilder.java:276)
at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:432)
at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:400)
at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:363)
at org.apache.maven.graph.DefaultGraphBuilder.collectProjects (DefaultGraphBuilder.java:414)
at org.apache.maven.graph.DefaultGraphBuilder.getProjectsForMavenReactor (DefaultGraphBuilder.java:405)
at org.apache.maven.graph.DefaultGraphBuilder.build (DefaultGraphBuilder.java:82)
at org.apache.maven.DefaultMaven.buildGraph (DefaultMaven.java:507)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:219)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/InternalErrorException
Build step 'Execute shell' marked build as failure
Finished: FAILURE

can anyone solve it

authors: ameelmohammad prietyc123 rumbletumjum snicoll

repo: spring-projects/spring-petclinic

license_source: github-api


text:
Support InheritableThreadLocal tenancyContextHolder Strategy

Support an InheritableThreadLocal strategy for tenantyContext just as in Spring Security.

PR available #8

repo: spring-projects/spring-tenancy