Sqoop import to HCatalog/Hive - table not visible
HDP-2.4.2.0-258 installed using Ambari 2.2.2.0
I have to import several SQL Server schema which should be accessible via Hive, Pig, MR and any third party(in future). I decided to import in HCatalog.
Sqoop provides ways to import to Hive OR HCatalog, I guess if I import to HCatalog, the same table will be accessible from Hadoop Administration Training, CLI, to MR and to Pig(please evaluate my assumption).
Questions :
If imported to Hive directly, will the table be available to Pig, MR?
If imported to HCatalog, what needs to be done for accessing via Hive?
Is pre-creation of tables in Hive necessary? If yes, what is the advantage of the importing in HCatalog, (I can import in Hive directly)/(import in HDFS and then create external table)?
Issue: I wish to achieve the following in one step:
Import of the data(from SQL Server tables)
Avoid 'pre-creating' or writing create statements for those tables(there are 100s of them)
Store the table in ORC format
Store this data at custom HDFS path say /org/data/schema1, /org/data/schema2 and so on(is this impossible because Sqoop says it (--target-dir/--warehouse-dir)
I executed the following command :
-bash-4.2$ sqoop import --connect 'jdbc:sqlserver://<IP>;database=FleetManagement' --username --password --table SettingAttribute -- --schema Administration --hcatalog-home /usr/hdp/current/hive-webhcat --hcatalog-database default --hcatalog-table SettingAttribute --create-hcatalog-table --hcatalog-storage-stanza "stored as orcfile"
1 reply
-
Hi Priyanka,
Are you sure you have submitted this question to the right vendor product forum? It doesn't seem to be anything to do with Pyramid Analytics!
Ian