【甘道夫】Sqoop1.99.3基础操作--导入Oracle的数据到HDFS
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了【甘道夫】Sqoop1.99.3基础操作--导入Oracle的数据到HDFS，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含6727字，纯文字阅读大概需要10分钟。
内容图文
fulong@FBI008:~$ sqoop.sh client
Sqoop home directory: /home/fulong/Sqoop/sqoop-1.99.3-bin-hadoop200
Sqoop Shell: Type ‘help‘ or ‘\h‘ for help.
sqoop:000> set server --host FBI003 --port 12000 --webapp sqoop Server is set successfully sqoop:000> show version --all client version: Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013 server version: Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013 Protocol version: [1]
第二步：创建连接
sqoop:000> create connection --cid 001 Creating connection for connector with id 1 Please fill following values to create new connection object Name: 147patents
Connection configuration
JDBC Driver Class: oracle.jdbc.driver.OracleDriver JDBC Connection String: jdbc:oracle:thin:@192.168.0.147:1521:ORCLGBK Username: zhaobiao Password: ****** JDBC Connection Properties: There are currently 0 values in the map: entry#
Security related configuration options
Max connections: 100 New connection was successfully created with validation status FINE and persistent id 1
第三步：创建Job
我这里顺便尝试了一下update命令。所以第一遍创建job时输入了错误的tablename：
sqoop:000> create job Required argument --xid is missing. sqoop:000> create job --xid 1 --type import Creating job for connection with id 1 Please fill following values to create new job object Name: ImportPatents
Database configuration
Schema name: zhaobiao Table name: zhaobiao Table SQL statement: Table column names: Partition column name: Nulls in partition column: Boundary query:
Output configuration
Storage type: 0 : HDFS Choose: 0 Output format: 0 : TEXT_FILE 1 : SEQUENCE_FILE Choose: 0 Compression format: 0 : NONE 1 : DEFAULT 2 : DEFLATE 3 : GZIP 4 : BZIP2 5 : LZO 6 : LZ4 7 : SNAPPY Choose: 0 Output directory: /Data/zhaobiao
Throttling resources
Extractors: Loaders: New job was successfully created with validation status FINE and persistent id 1
sqoop:000> update job Required argument --jid is missing. sqoop:000> update job --jid 1 Updating job with id 1 Please update job metadata: Name: ImportPatents
Database configuration
Schema name: zhaobiao Table name: CMS_NEWS_0625 Table SQL statement: Table column names: Partition column name: Nulls in partition column: Boundary query:
Output configuration
Storage type: 0 : HDFS Choose: 0 Output format: 0 : TEXT_FILE 1 : SEQUENCE_FILE Choose: 0 Compression format: 0 : NONE 1 : DEFAULT 2 : DEFLATE 3 : GZIP 4 : BZIP2 5 : LZO 6 : LZ4 7 : SNAPPY Choose: 0 Output directory: /Data/zhaobiao
Throttling resources
Extractors: Loaders: Job was successfully updated with status FINE
总结： 1.create job时必须指定一个连接id 2.create job完毕后。自己主动分配了一个job id 3.update job时。须要指定job id 4.创建Job过程中指定的Output directory指定的是HDFS中的路径
第四步：运行job
sqoop:000> start job --jid 1 2014-06-27 12:44:17 CST: FAILURE_ON_SUBMIT Exception: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://fulonghadoop/Data/zhaobiao already exists sqoop:000> start job --jid 1 Submission details Job ID: 1 Server URL: http://FBI003:12000/sqoop/ Created by: fulong Creation date: 2014-06-27 12:44:53 CST Lastly updated by: fulong External ID: job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_0005/ 2014-06-27 12:44:53 CST: BOOTING - Progress is not available sqoop:000> status job --jid 1 Submission details Job ID: 1 Server URL: http://FBI003:12000/sqoop/ Created by: fulong Creation date: 2014-06-27 12:44:53 CST Lastly updated by: fulong External ID: job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_0005/ 2014-06-27 12:50:11 CST: RUNNING - 0.00 %
sqoop:000> status job --jid 1 Submission details Job ID: 1 Server URL: http://FBI003:12000/sqoop/ Created by: fulong Creation date: 2014-06-27 12:44:53 CST Lastly updated by: fulong External ID: job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_0005/ 2014-06-27 13:36:49 CST: RUNNING - 25.00 % sqoop:000> status job --jid 1 Submission details Job ID: 1 Server URL: http://FBI003:12000/sqoop/ Created by: fulong Creation date: 2014-06-27 12:44:53 CST Lastly updated by: fulong External ID: job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_0005/ 2014-06-27 13:39:24 CST: RUNNING - 35.00 % sqoop:000> status job --jid 1 Submission details Job ID: 1 Server URL: http://FBI003:12000/sqoop/ Created by: fulong Creation date: 2014-06-27 12:44:53 CST Lastly updated by: fulong External ID: job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_0005/ 2014-06-27 13:47:19 CST: RUNNING - 40.00 % sqoop:000> status job --jid 1 Submission details Job ID: 1 Server URL: http://FBI003:12000/sqoop/ Created by: fulong Creation date: 2014-06-27 12:44:53 CST Lastly updated by: fulong External ID: job_1403248336137_0005 http://FBI003:8088/proxy/application_1403248336137_0005/ 2014-06-27 14:08:36 CST: SUCCEEDED Counters: org.apache.hadoop.mapreduce.JobCounter SLOTS_MILLIS_MAPS: 16499226 TOTAL_LAUNCHED_MAPS: 10 SLOTS_MILLIS_REDUCES: 0 OTHER_LOCAL_MAPS: 15 org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter BYTES_WRITTEN: 421226094 org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter BYTES_READ: 0 org.apache.hadoop.mapreduce.TaskCounter MAP_INPUT_RECORDS: 0 MERGED_MAP_OUTPUTS: 0 PHYSICAL_MEMORY_BYTES: 1185689600 SPILLED_RECORDS: 0 COMMITTED_HEAP_BYTES: 627179520 CPU_MILLISECONDS: 307570 FAILED_SHUFFLE: 0 VIRTUAL_MEMORY_BYTES: 6542311424 SPLIT_RAW_BYTES: 1437 MAP_OUTPUT_RECORDS: 644083 GC_TIME_MILLIS: 5279 org.apache.hadoop.mapreduce.FileSystemCounter FILE_WRITE_OPS: 0 FILE_READ_OPS: 0 FILE_LARGE_READ_OPS: 0 FILE_BYTES_READ: 0 HDFS_BYTES_READ: 1437 FILE_BYTES_WRITTEN: 931780 HDFS_LARGE_READ_OPS: 0 HDFS_WRITE_OPS: 20 HDFS_READ_OPS: 40 HDFS_BYTES_WRITTEN: 431907904 org.apache.sqoop.submission.counter.SqoopCounters ROWS_READ: 644083 Job executed successfully
接下来的问题：怎样导入数据到Hbase中？
【甘道夫】Sqoop1.99.3基础操作--导入Oracle的数据到HDFS
标签：rtp shuf detail progress prot ott UI extra imp
本文系统来源：http://www.cnblogs.com/brucemengbm/p/6934809.html
内容总结
以上是互联网集市为您收集整理的【甘道夫】Sqoop1.99.3基础操作--导入Oracle的数据到HDFS全部内容，希望文章能够帮你解决【甘道夫】Sqoop1.99.3基础操作--导入Oracle的数据到HDFS所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。
来源：【匿名】