Flume setup with kerberos – write log data to HDFS

We will setup flume to write data from /var/log/messages to HDFS 1. Create the agent configuration in the log source server as below ==================== ## Flume NG Apache Log Collection ## Refer to https://cwiki.apache.org/confluence/display/FLUME/Getting+Started ## # http://flume.apache.org/FlumeUserGuide.html#exec-source agent.sources = apache agent.sources.apache.type = exec agent.sources.apache.command = tail -F /var/log/messages agent.sources.apache.batchSize = 1 agent.sources.apache.channels = memoryChannel agent.sources.apache.interceptors … Continue reading Flume setup with kerberos – write log data to HDFS

Advertisements