Delay in Kafka Log4j appender Code Answer

Hello Developer, Hope you guys are doing great. Today at Tutorial Guruji Official website, we are sharing the answer of Delay in Kafka Log4j appender without wasting too much if your time.

The question is published on by Tutorial Guruji team.

So here is my setup, and this is all being done in Eclipse

  1. I am trying to implement the Kafka Log4j appender to pump messages into Kafka.

  2. I have some java code setup to create the log messages:

    public static void main(String[] args) {
    
    SimpleDateFormat sdf = new SimpleDateFormat();
    
    log.debug("Debug message at "+sdf.format(new Date()));
    
    log.info("Info message at "+sdf.format(new Date()));
    
    log.error("Error Message at "+sdf.format(new Date()));
    
    log.fatal("Fatal Message at "+sdf.format(new Date()));
    
    log.warn("Warn Message at "+sdf.format(new Date()));
    
    log.trace("Trace Message at "+sdf.format(new Date()));
    }
    
  3. here are my log4j.properties

    log4j.rootLogger=TRACE, stdout
    log4j.appender.stdout=org.apache.log4j.ConsoleAppender
    log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
    log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L %% %m%n
    
    log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
    log4j.appender.KAFKA.BrokerList=localhost:9092
    log4j.appender.KAFKA.Topic=kfkLogs
    log4j.appender.KAFKA.producer.type=async
    log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
    log4j.appender.KAFKA.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L %% - %m%n
    
    log4j.logger.logGen.TestLog4j=TRACE, KAFKA
    
  4. In another instance of Eclipse, i have my Kafka consumer code running which basically prints out any messages it receieves

Here is my problem,

When i execute my log producer, i do not always see the messages in the consumer. sometimes i see the messages and sometimes i do not. Sometimes i can execute the producer repeatedly for a few seconds and each time i see the messages, but other times, the consumer only gets them once and not for the next minute or so. then it starts receiving again. so the messages that i pushed in the meantime are lost.

Is there anyway that there can be a delay when Kafka published the messages ?

Can it have something to do with my Kafka or Zookeeper configurations ?

I am executing everything on localhost

Please advise.

Answer

log4j.appender.KAFKA.producer.type=sync

fixed it.

We are here to answer your question about Delay in Kafka Log4j appender - If you find the proper solution, please don't forgot to share this with your team members.

Related Posts

Tutorial Guruji