Tag: mirrormaker

  • Important considerations regarding Kafka MirrorMaker

    Morning folks,

    Just wanted to share with you two things that i learned when playing with Kafka MirrorMaker this morning.

    First is that if you want to put MirrorMaker or other tool that uses Kafka on debug, all you have to do is to modify the tools-log4j.properties file from the config folder of you Apache Kafka installation. It normally looks like this

    log4j.rootLogger=WARN, stderr
    
    log4j.appender.stderr=org.apache.log4j.ConsoleAppender
    log4j.appender.stderr.layout=org.apache.log4j.PatternLayout
    log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n
    log4j.appender.stderr.Target=System.err
    

    but you can easily set it to

    log4j.rootLogger=DEBUG, stderr
    
    log4j.appender.stderr=org.apache.log4j.ConsoleAppender
    log4j.appender.stderr.layout=org.apache.log4j.PatternLayout
    log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n
    log4j.appender.stderr.Target=System.err
    

    The second one is related to property that you should put on your consumer.config file in order to grab the data from the topic start and not from the latest offset.

    In order to achieve this you will need to add the following option:

    auto.offset.reset=earliest

    The possible values for this setting are latest, earliest, none.

    Cheers