0

Sqoop import to HCatalog/Hive - table not visible

HDP-2.4.2.0-258 installed using Ambari 2.2.2.0

I have to import several SQL Server schema which should be accessible via Hive, Pig, MR and any third party(in future). I decided to import in HCatalog.

Sqoop provides ways to import to Hive OR HCatalog, I guess if I import to HCatalog, the same table will be accessible from Hadoop Administration Training, CLI, to MR and to Pig(please evaluate my assumption).

Questions :

If imported to Hive directly, will the table be available to Pig, MR?
If imported to HCatalog, what needs to be done for accessing via Hive?
Is pre-creation of tables in Hive necessary? If yes, what is the advantage of the importing in HCatalog, (I can import in Hive directly)/(import in HDFS and then create external table)?

Issue: I wish to achieve the following in one step:

Import of the data(from SQL Server tables)
Avoid 'pre-creating' or writing create statements for those tables(there are 100s of them)
Store the table in ORC format
Store this data at custom HDFS path say /org/data/schema1, /org/data/schema2 and so on(is this impossible because Sqoop says it (--target-dir/--warehouse-dir)
I executed the following command :

-bash-4.2$ sqoop import --connect 'jdbc:sqlserver://<IP>;database=FleetManagement' --username --password  --table SettingAttribute -- --schema Administration  --hcatalog-home /usr/hdp/current/hive-webhcat --hcatalog-database default --hcatalog-table SettingAttribute --create-hcatalog-table --hcatalog-storage-stanza "stored as orcfile"

1 reply

null
    • Senior Director of Product
    • Ian_Macdonald
    • 4 yrs ago
    • Reported - view

    Hi Priyanka,

    Are you sure you have submitted this question to the right vendor product forum? It doesn't seem to be anything to do with Pyramid Analytics!

    Ian

Content aside

  • Status Answered
  • 4 yrs agoLast active
  • 1Replies
  • 41Views
  • 2 Following