Morning folks,
Just wanted to share with you two things that i learned when playing with Kafka MirrorMaker this morning.
First is that if you want to put MirrorMaker or other tool that uses Kafka on debug, all you have to do is to modify the tools-log4j.properties file from the config folder of you Apache Kafka installation. It normally looks like this
log4j.rootLogger=WARN, stderr log4j.appender.stderr=org.apache.log4j.ConsoleAppender log4j.appender.stderr.layout=org.apache.log4j.PatternLayout log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n log4j.appender.stderr.Target=System.err
but you can easily set it to
log4j.rootLogger=DEBUG, stderr log4j.appender.stderr=org.apache.log4j.ConsoleAppender log4j.appender.stderr.layout=org.apache.log4j.PatternLayout log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n log4j.appender.stderr.Target=System.err
The second one is related to property that you should put on your consumer.config file in order to grab the data from the topic start and not from the latest offset.
In order to achieve this you will need to add the following option:
auto.offset.reset=earliest
The possible values for this setting are latest, earliest, none.
Cheers