标签：jobs nsa producer hsi col log trap asi bootstra
1）配置flume(flume-kafka.conf)
# define a1.sources = r1 a1.sinks = k1 a1.channels = c1 # source a1.sources.r1.type = exec a1.sources.r1.command = tail -F -c +0 /opt/module/datas/flume.log a1.sources.r1.shell = /bin/bash -c # sink a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink a1.sinks.k1.kafka.bootstrap.servers = hadoop202:9092,hadoop203:9092,hadoop204:9092 a1.sinks.k1.kafka.topic = first a1.sinks.k1.kafka.flumeBatchSize = 20 a1.sinks.k1.kafka.producer.acks = 1 a1.sinks.k1.kafka.producer.linger.ms = 1 # channel a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # bind a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
2） 启动kafkaIDEA消费者
bin/kafka-console-consumer.sh --zookeeper hadoop202:2181 --from-beginning --topic first
3） 进入flume根目录下，启动flume
bin/flume-ng agent -c conf/ -n a1 -f jobs/flume-kafka.conf
标签：jobs nsa producer hsi col log trap asi bootstra
原文地址：https://www.cnblogs.com/kpwong/p/14033973.html
