Sqoop import to HCatalog/Hive - table not visible

HDP- installed using Ambari

I have to import several SQL Server schema which should be accessible via Hive, Pig, MR and any third party(in future). I decided to import in HCatalog.

Sqoop provides ways to import to Hive OR HCatalog, I guess if I import to HCatalog, the same table will be accessible from Hadoop Administration Training, CLI, to MR and to Pig(please evaluate my assumption).

Questions :

If imported to Hive directly, will the table be available to Pig, MR?
If imported to HCatalog, what needs to be done for accessing via Hive?
Is pre-creation of tables in Hive necessary? If yes, what is the advantage of the importing in HCatalog, (I can import in Hive directly)/(import in HDFS and then create external table)?

Issue: I wish to achieve the following in one step:

Import of the data(from SQL Server tables)
Avoid 'pre-creating' or writing create statements for those tables(there are 100s of them)
Store the table in ORC format
Store this data at custom HDFS path say /org/data/schema1, /org/data/schema2 and so on(is this impossible because Sqoop says it (--target-dir/--warehouse-dir)
I executed the following command :

-bash-4.2$ sqoop import --connect 'jdbc:sqlserver://<IP>;database=FleetManagement' --username --password  --table SettingAttribute -- --schema Administration  --hcatalog-home /usr/hdp/current/hive-webhcat --hcatalog-database default --hcatalog-table SettingAttribute --create-hcatalog-table --hcatalog-storage-stanza "stored as orcfile"

1 reply

    • Senior Director of Product
    • Ian_Macdonald
    • 4 yrs ago
    • Reported - view

    Hi Priyanka,

    Are you sure you have submitted this question to the right vendor product forum? It doesn't seem to be anything to do with Pyramid Analytics!


Content aside

  • Status Answered
  • 4 yrs agoLast active
  • 1Replies
  • 46Views
  • 2 Following