storm在linux系统下安装调试
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了storm在linux系统下安装调试,小编现在分享给大家,供广大互联网技能从业者学习和参考。文章包含6141字,纯文字阅读大概需要9分钟。
内容图文
![storm在linux系统下安装调试](/upload/InfoBanner/zyjiaocheng/1145/1ebc28b68f6d412f8a4b0e7eea2b7fac.jpg)
安装:
安装 zookeeper :
- 下载 zookeeper :http://zookeeper.apache.org/releases.html#download。
- 将 zookeeper-3.4.7.tar.gz 解压到目录,/usr/local/zookeeper-3.4.7。
- 拷贝zookeeper的conf/zoo_sample.cfg并重命名为zoo.cfg,内容修改如下:
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/usr/local/zookeeper-3.4.7/data # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 # log dir dataLogDir=/usr/local/zookeeper-3.4.7/log
- 建立zookeeper集群的myid文件
cd /usr/local/zookeeper-3.4.7/data echo 1 > myid
- 启动zookeeper
cd /usr/local/zookeeper-3.4.7/bin ./zkServer.sh start
安装 storm :
- 下载 storm : http://storm.apache.org/downloads.html 。
- 将 apache-storm-0.10.0.tar.gz 解压到目录,/usr/local/storm-0.10.0/bin/apache-storm-0.10.0 。
- 配置nimbus
修改storm的/usr/local/storm-0.10.0/conf/storm.yaml文件如下:
########### These MUST be filled in for a storm configuration storm.zookeeper.servers: - "192.168.1.63" # - "server2" # nimbus.host: "192.168.1.63" # # # ##### These may optionally be filled in: # ## List of custom serializations # topology.kryo.register: # - org.mycompany.MyType # - org.mycompany.MyType2: org.mycompany.MyType2Serializer # ## List of custom kryo decorators # topology.kryo.decorators: # - org.mycompany.MyDecorator # ## Locations of the drpc servers # drpc.servers: # - "server1" # - "server2" ## Metrics Consumers # topology.metrics.consumer.register: # - class: "backtype.storm.metric.LoggingMetricsConsumer" # parallelism.hint: 1 # - class: "org.mycompany.MyMetricsConsumer" # parallelism.hint: 1 # argument: # - endpoint: "metrics-collector.mycompany.org"
- 启动storm
cd /usr/local/storm-0.10.0/ ./storm nimbus >/dev/null 2>&1 & ./storm supervisor >/dev/null 2>&1 & ./storm ui >/dev/null 2>&1 &
- 通过 http://192.168.1.63:8080/index.html 可以打开Storm Web UI,看看Storm集群的状态。
发布程序
./storm jar /home/jsecode/topology/storm-starter-0.10.0.jar storm.ykd.test.SegmentTopologyMain /home/jsecode/topology/testdata/gpsdata
- 日志配置
修改文件 /usr/local/storm-0.10.0/log4j2/worker.xml
<configuration monitorInterval="60"> <properties> <property name="pattern">%d{yyyy-MM-dd HH:mm:ss.SSS} %c{1.} [%p] %msg%n</property> <property name="patternNoTime">%msg%n</property> </properties> <appenders> <RollingFile name="SegmentLog" fileName="${sys:storm.log.dir}/gpsSegmentw.log" filePattern="${sys:storm.log.dir}/gpsSegmentw.%i.log"> <PatternLayout> <pattern>${pattern}</pattern> </PatternLayout> <Policies> <SizeBasedTriggeringPolicy size="100 MB"/> <!-- Or every 100 MB --> </Policies> <DefaultRolloverStrategy max="9"/> </RollingFile> <RollingFile name="RsLog" fileName="${sys:storm.log.dir}/rs.log" filePattern="${sys:storm.log.dir}/rs.%i.log"> <PatternLayout> <pattern>${pattern}</pattern> </PatternLayout> <Policies> <SizeBasedTriggeringPolicy size="100 MB"/> <!-- Or every 100 MB --> </Policies> <DefaultRolloverStrategy max="9"/> </RollingFile> <RollingFile name="A1" fileName="${sys:storm.log.dir}/${sys:logfile.name}" filePattern="${sys:storm.log.dir}/${sys:logfile.name}.%i.gz"> <PatternLayout> <pattern>${pattern}</pattern> </PatternLayout> <Policies> <SizeBasedTriggeringPolicy size="100 MB"/> <!-- Or every 100 MB --> </Policies> <DefaultRolloverStrategy max="9"/> </RollingFile> <RollingFile name="STDOUT" fileName="${sys:storm.log.dir}/${sys:logfile.name}.out" filePattern="${sys:storm.log.dir}/${sys:logfile.name}.out.%i.gz"> <PatternLayout> <pattern>${patternNoTime}</pattern> </PatternLayout> <Policies> <SizeBasedTriggeringPolicy size="100 MB"/> <!-- Or every 100 MB --> </Policies> <DefaultRolloverStrategy max="4"/> </RollingFile> <RollingFile name="STDERR" fileName="${sys:storm.log.dir}/${sys:logfile.name}.err" filePattern="${sys:storm.log.dir}/${sys:logfile.name}.err.%i.gz"> <PatternLayout> <pattern>${patternNoTime}</pattern> </PatternLayout> <Policies> <SizeBasedTriggeringPolicy size="100 MB"/> <!-- Or every 100 MB --> </Policies> <DefaultRolloverStrategy max="4"/> </RollingFile> <Syslog name="syslog" format="RFC5424" host="localhost" port="514" protocol="UDP" appName="[${sys:storm.id}:${sys:worker.port}]" mdcId="mdc" includeMDC="true" facility="LOCAL5" enterpriseNumber="18060" newLine="true" exceptionPattern="%rEx{full}" messageId="[${sys:user.name}:${sys:logging.sensitivity}]" id="storm"/> </appenders> <loggers> <Logger name="storm.ykd.test.bolt.SavePosdataTrackSegmentBolt" level="debug"> <AppenderRef ref="RsLog"/> </Logger> <Logger name="storm.ykd.test" level="debug"> <AppenderRef ref="SegmentLog"/> </Logger> <root level="info"> <!-- We log everything --> <appender-ref ref="A1"/> <appender-ref ref="syslog"/> </root> <Logger name="STDERR" level="INFO"> <appender-ref ref="STDERR"/> <appender-ref ref="syslog"/> </Logger> <Logger name="STDOUT" level="INFO"> <appender-ref ref="STDOUT"/> <appender-ref ref="syslog"/> </Logger> </loggers> </configuration>
原文:http://www.cnblogs.com/myibm/p/5939560.html
内容总结
以上是互联网集市为您收集整理的storm在linux系统下安装调试全部内容,希望文章能够帮你解决storm在linux系统下安装调试所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错,欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 gblab@vip.qq.com 举报,一经查实,本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。