Week twelve Activities: Data transfer to HDFS Directory Storage System

In this Implementation part, I am going to share how to create a directory inside of HDFS default file system and how to transfer data from HDInsihgt Azure storage to directory location using SSH command line.

I will create two directories: one is uploaded data and another one for output of data

Data directory, I will move the raw datasets of 5 CSV files for hive job

Step 1: Connect to HDInsight head node using ssh connection via PuTTY 1.jpg

Step 2: Run directory create command line

2

Step 3: Run the following command to check directories are created or not!

3.jpg

Step 4: Run the following script to move data from HDInsight Storage to directory location of “Data folder”

4

Step 5: Run the following command to check files were moved or not to the data directory folder.

5.jpg

Step 6: You can check the files from the Azure Storage account dashboard from your directory location

6.jpg

 

Thank you 🙂

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s