2017/03/11

Cloudera CDH5 Tutorial - Sqoop import - retail_db into HDFS

Starting to import "retail_db" mysql databse tables into HDFS.  Hit another error!!😕😕

retail_dba user creation 

I initially created the "retail_dba" user using the below syntax:

The  user is created successfully, as listed below ( entries from user_privileges) table.
But when attempted to login to mysql got the error "Access denied for user' as below:


 Dropped the user and recreated by specifying explicitely '@localhost' as below


After recreating the user I am able to login as "retail_dba" and list the available tables.

Sqoop Import Error

Ok, now the issue with "retail_dba" user connectivity been sorted moving to the original task of importing "retail_db" mysql db tables into HDFS using sqoop.

So first issued Sqoop import command from the CLI:
Importing all available tables  [-- import-all-tables] option
from mysql db [ --connect jdbc:mysql://localhost:3306/retail_db]
using the credentials created earlier [ --username=retail_dba and password]

Nothing seem to work first time, hit another error.
This time complaining about missing JSon jar files.



Checked the Sqoop lib directory - /usr/lib/sqoop/lib - and found no JSon jar file.

Download the file from Java Downloads web site - http://www.java2s.com/Code/Jar/j/Downloadjavajsonjar.htm as below
Download the zip file and unpack it.
Copied the file "java-json.jar" to the Sqoop lib path as below

Fingers Crossed.  Time to test the Sqoop import again.

This time attempting to import data from mysql db to HDFS has completed successfully.  Though, I faced some data issues while importing "customers" table.  I had to import one table at a time into HDFS using the below syntax:

When all the tables are imported, we should see one directory for each table under the Hive warehouse path - /user/hive/warehouse - as below:

1 comment:

File Handling with Python

This little utility is for copying files from source to target directories.  On the way it checks whether a directory exists in the target, ...