sql server 导出数据到 Azure Hbase / Hive 详细步骤
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了sql server 导出数据到 Azure Hbase / Hive 详细步骤，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含6362字，纯文字阅读大概需要10分钟。
内容图文
Goals
In this tutorial, you see three things:
-
How to set up a SQL database on Windows Azure for use with the tutorial.
-
How to use the Remote Desktop feature in Hadoop on Azure to access the head node of the HDFS cluster.
-
How to import relational data from SQL Server to a Hadoop on Azure HDFS cluster by using Sqoop.
Key technologies
- Windows Azure
- Hadoop on Azure
- Sqoop
Setup and Configuration
You must have an account to access Hadoop on Azure and have created a cluster to work through this tutorial. To obtain an account and create an Hadoop cluster, follow the instructions outlined in the Getting started with Microsoft Hadoop on Azure section of the Introduction to Hadoop on Azure topic.
You will also need your outward facing IP address for your current
location when configuring your firewall on SQL Database. To obtain it,
go to the site
WhatIsMyIP
and make a note of it. Later in the procedure, you also need the
outward facing IP address for the head of the Hadoop cluster. You can
obtain this IP address in the same way.
Tutorial
This tutorial is composed of the following segments:
-
How to set up a SQL database.
-
How to use Sqoop from Hadoop on Azure to import data to the HDFS cluster.
How to set up a SQL database
Log in into your Windows Azure account. To create a database server, click the Database icon in the lower left-hand corner on the page.
On the Getting Started page, click the Create a new SQL Database Server option.
Select the type of subscription (such as Pay-As-You-Go) associated with you account in the Create Server window and press Next.
Select the appropriate Region in the Create Server window and click Next.
Specify the login and password of the server-level principal of your SQL Database server and then press Next.
Press Add to specify a firewall rule that allows your current location access to SQL Database to upload the AdventureWorks database. The firewall grants access based on the originating IP address of each request. Use the IP address found with the configuration preliminaries of this tutorial for the values to add. Specify a Rule name, such as shown, but remember to use your IP address, not the one used for illustration purposes below. (You must also add the outward IP address of the head node in you Hadoop cluster. If you know it already, add it now.) Then press the Finish button.
Download the AdventureWorks2012 database onto your local machine from Recommended Downloads link on the Adventure Works for SQL Database site.
Unzip the file, open an Administrator Command Prompt, and navigate to the AdventureWorks directory inside the AdventureWorks2012ForSQLAzure folder.
Run CreateAdventureWorksForSQLAzure.cmd by typing the following:
CreateAdventureWorksForSQLAzure.cmd servername username password
For example, if the assigned SQL Database server is named b1gl33p, the administrator user name "Fred", and the password "Secret", you would type the following:
CreateAdventureWorksForSQLAzure.cmd b1gl33p.database.windows.net Fred@b1gl33p Secret
The script creates the database, installs the schema, and populates the database with sample data.
Return to the WindowsAzurePlatform portal page, click your subscription on the left-hand side (Pay-As-You-Go in the example below) and select your database (here named wq6xlbyoq0). The AventureWorks2012 should be listed in the Database Name column. Select it and press the Manage icon at the top of the page.
Enter the credentials for the SQL database when prompted and press Log on.
This opens the Web interface for the Adventure Works database on SQL Database. Press the New Query icon at the top to open the query editor.
Since Sqoop currently adds square brackets to the table name, we need to add a synonym to support two-part naming for SQL Server tables. To do so, run the following query:
CREATE SYNONYM [Sales.SalesOrderDetail] FOR Sales.SalesOrderDetail
Run the following query and review its result.
select top 200 * from [Sales.SalesOrderDetail]
How to use Sqoop from Hadoop on Azure to import SQL Database query results to the HDFS cluster in Hadoop On Azure.
From your Account page, scroll down to the Open Ports icon in the Your cluster section and click the icon to open the ODBC Server port on the head node in your cluster.
Return to your Account page, scroll down to the Your cluster section and click the Remote Desktop icon this time to open the head node in your cluster.
Select Open when prompted to open the .rdp file.
Select Connect in the Remote Desktop Connection window.
Enter your credentials for the Hadoop cluster (not your Hadoop on Azure account) into the Windows Security window and select OK.
Open Internet Explorer and go to the site
WhatIsMyIP
to obtain the outward facing IP address for the head node of the
cluster. Return the SQL Database management page and add a firewall rule
that allows your Hadoop cluster access to SQL Database. The firewall
grants access based on the originating
IP address of each request.
Double-click on the Hadoop Command Shell icon in the upper left hand of the Desktop to open it. Navigate to the "c:\Apps\dist\sqoop\bin" directory and run the following command:
sqoop import --connect
"jdbc:sqlserver://[serverName].database.windows.net;username=[userName]@[serverName];password=[password];database=AdventureWorks2012"
--table Sales.SalesOrderDetail --target-dir /data/lineitemData -m 1
So, for example, for the following values:
* server name: wq6xlbyoq0
* username: HadoopOnAzureSqoopAdmin
* password: Pa$$w0rd
The sqoop command is:
sqoop import --connect
"jdbc:sqlserver://wq6xlbyoq0.database.windows.net;username=HadoopOnAzureSqoopAdmin@wq6xlbyoq0;password=Pa$$w0rd;;database=AdventureWorks2012"
--table Sales.SalesOrderDetail --target-dir /data/lineitemData -m 1
Return to the Accounts page of the Hadoop on Azure portal and open the
Interactive Console this time. Run the #lsr command from the JavaScript console to list the files and directories on your HDFS cluster.
Run the #tail command to view selected results from the part-m-0000 file.
tail /user/RAdmin/data/SalesOrderDetail/part-m-00000
Summary
In this tutorial, you have seen how to use Sqoop to import data from a SQL database on Windows Azure to an Hadoop on Azure HDFS cluster.
sql server 导出数据到 Azure Hbase / Hive 详细步骤
标签：
本文系统来源：http://www.cnblogs.com/pangguoming/p/4230440.html
内容总结
以上是互联网集市为您收集整理的sql server 导出数据到 Azure Hbase / Hive 详细步骤全部内容，希望文章能够帮你解决sql server 导出数据到 Azure Hbase / Hive 详细步骤所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。