在Hadoop集群中安装flume组件
hadoop学习过程中，用到安装flume组件，过程记录如下：[*] 安装flume服务，yum -y install flume-ng*， service flume-ng-agent start
[*] 查看flume安装位置 whereis flume-ng
[*] 在flume安装位置的conf文件夹下，新建配置文件 testflume.conf
tail1.sources = src1
tail1.channels = ch1
tail1.sinks = sink1
tail1.sources.src1.type = exec
tail1.sources.src1.command = tail -F /var/log/httpd/access_log
tail1.sources.src1.channels = ch1
tail1.channels.ch1.type = memory
tail1.channels.ch1.capacity = 500
tail1.sinks.sink1.type = avro
tail1.sinks.sink1.hostname = agent
tail1.sinks.sink1.port = 6000
tail1.sinks.sink1.batch-size = 1
tail1.sinks.sink1.channel = ch1
##
agent1.sources = r1 #seqGenSrc
agent1.channels = c1 #memoryChannel
agent1.sinks = k1 #loggerSink
agent1.sources.r1.type = avro
agent1.sources.r1.bind = agent
agent1.sources.r1.port = 6000
agent1.sources.r1.channels = c1
agent1.sinks.k1.type = hdfs
agent1.sinks.k1.hdfs.path = flume/testflume
agent1.sinks.k1.hdfs.filePrefix = access_log
agent1.sinks.k1.channel = c1
agent1.channels.c1.type = memory
agent1.channels.c1.capacity = 500
4. 依次执行flume-ng agent -c /etc/flume-ng/conf -f /etc/flume-ng/conf/testflume.conf -n agent1 -Dflume.root.logger=INFO,console
flume-ng agent -c /etc/flume-ng/conf -f /etc/flume-ng/conf/testflume.conf -n tail1 -Dflume.root.logger=INFO,console
5. 查看 hadoop fs -ls /user/guowen/flume/testflume，看到有抓取的文件生成
页:
[1]