| Both sides previous revision Previous revision | |
| flume [2020/11/11 11:42] – [Management] andonovj | flume [2020/11/12 11:53] (current) – andonovj |
|---|
| =====Management===== | =====Management===== |
| |
| | ====Startup==== |
| <Code:bash|Startup Flume Server - Terminal won't return to bash> | <Code:bash|Startup Flume Server - Terminal won't return to bash> |
| [oracle@edvmr1p0 ~]$ flume-ng agent --conf /opt/flume/conf -f /opt/flume/conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent1 | [oracle@edvmr1p0 ~]$ flume-ng agent --conf /opt/flume/conf -f /opt/flume/conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent1 |
| + exec /usr/java/latest/bin/java -Xms100m -Xmx2000m -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp '/opt/flume/conf:/opt/flume/lib/*:/opt/hadoop/etc/hadoop:/opt/hadoop/share/hadoop/common/lib/*:/opt/hadoop/share/hadoop/common/*:/opt/hadoop/share/hadoop/hdfs:/opt/hadoop/ | + exec /usr/java/latest/bin/java -Xms100m -Xmx2000m -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp '/opt/flume/conf:/opt/flume/lib/*:/opt/hadoop/etc/hadoop:/opt/hadoop/share/hadoop/common/lib/*:/opt/hadoop/share/hadoop/common/*:/opt/hadoop/share/hadoop/hdfs:/opt/hadoop/ |
| </Code> | </Code> |
| This terminal won't exit back to shell: | This terminal won't exit back to shell. Also you can use teh following command if you want more logging: |
| |
| | <Code:bash|Startup Flume v2> |
| | [oracle@edvmr1p0 conf]$ flume-ng agent --conf /opt/flume/conf -f /opt/flume/conf/flume.conf -Dflume.root.logger=DEBUG,LOGFILE -n agent1 -Dorg.apache.flume.log.rawdata=true |
| | Info: Sourcing environment configuration script /opt/flume/conf/flume-env.sh |
| | Info: Including Hadoop libraries found via (/opt/hadoop/bin/hadoop) for HDFS access |
| | Info: Including HBASE libraries found via (/opt/hbase/bin/hbase) for HBASE access |
| | + exec /usr/java/latest/bin/java -Xms100m -Xmx2000m -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,LOGFILE -Dorg.apache.flume.log.rawdata=true -cp '/opt/flume/conf:/opt/flume/lib/*:/opt/hadoop/etc/hadoop:/opt/hadoop/share/hadoop/common/lib/*:/opt/hadoop/share/hadoop/common/*:/opt/hadoop/share/hadoop/hdfs:/opt/hadoop/share/hadoop/hdfs/lib/*:/opt/hadoop/share/hadoop/hdfs/*:/opt/hadoop/share/hadoop/yarn/lib/*:/opt/hadoop/share/hadoop/yarn/*:/opt/hadoop/share/hadoop/mapreduce/lib/*:/opt/ha |
| | </Code> |
| |
| | ====Transmit Data==== |
| <Code:bash|Transmit Data (e.g. /etc/passwd in that case) - Terminal won't return to bash> | <Code:bash|Transmit Data (e.g. /etc/passwd in that case) - Terminal won't return to bash> |
| [oracle@edvmr1p0 ~]$ flume-ng avro-client --conf conf -H localhost -p 41414 -F /etc/passwd | [oracle@edvmr1p0 ~]$ flume-ng avro-client --conf conf -H localhost -p 41414 -F /etc/passwd |