Common Sqoop Commands and Parameters
Common Sqoop commands
Command |
Description |
---|---|
import |
Imports data to a cluster. |
export |
Exports data of a cluster. |
codegen |
Obtains data from a table in the database to generate a Java file and compress the file. |
create-hive-table |
Creates a Hive table. |
eval |
Executes a SQL statement and view the result. |
import-all-tables |
Imports all tables in a database to HDFS. |
job |
Generates a Sqoop job. |
list-databases |
Lists database names. |
list-tables |
List table names. |
merge |
Merges data in different HDFS directories and saves the data to a specified directory. |
metastore |
Starts the metadata database to record the metadata of a Sqoop job. |
help |
Prints help information. |
version |
Prints the version information. |
Common Parameters
Category |
Parameter |
Description |
---|---|---|
Parameters for database connection |
--connect |
Specifies the URL for connecting to a relational database. |
--connection-manager |
Specifies the connection manager class. |
|
--driver jdbc |
Specifies the driver package for database connection. |
|
--help |
Prints help information. |
|
--password |
Specifies the password for connecting to a database. |
|
--username |
Specifies the username for connecting to a database. |
|
--verbose |
Prints detailed information on the console. |
|
import parameters |
--fields-terminated-by |
Specifies the field delimiter, which must be the same as that in a Hive table or HDFS file. |
--lines-terminated-by |
Specifies the line delimiter, which must be the same as that in a Hive table or HDFS file. |
|
--mysql-delimiters |
Specifies the default delimiter settings of MySQL. |
|
export parameters |
--input-fields-terminated-by |
Specifies the field delimiter. |
--input-lines-terminated-by |
Specifies the line delimiter. |
|
Hive parameters |
--hive-delims-replacement |
Replaces characters such as \r and \n in data with user-defined characters. |
--hive-drop-import-delims |
Removes characters such as \r and \n when data is imported to Hive. |
|
--map-column-hive |
Specifies the data type of fields during the generation of a Hive table. |
|
--hive-partition-key |
Creates a partition. |
|
--hive-partition-value |
Imports data to a specified partition of a database. |
|
--hive-home |
Specifies the installation directory for Hive. |
|
--hive-import |
Specifies that data is imported from a relational database to Hive. |
|
--hive-overwrite |
Overwrites existing Hive data. |
|
--create-hive-table |
Creates a Hive table. The default value is false. A destination table will be created if it does not exist. |
|
--hive-table |
Specifies a Hive table to which data is to be imported. |
|
--table |
Specifies the relational database table. |
|
--columns |
Specifies the fields of a relational data table to be imported. |
|
--query |
Specifies the query statement for importing the query result. |
|
HCatalog parameters |
--hcatalog-database |
Specifies a Hive database and imports data to it using HCatalog. |
--hcatalog-table |
Specifies a Hive table and imports data to it using HCatalog. |
|
Others |
-m or --num-mappers |
Specifies the number of map tasks used by a Sqoop job. |
--split-by |
Specifies the column based on which Sqoop splits work units. This parameter is used together with -m. |
|
--target-dir |
Specifies the temporary directory of HDFS. |
|
--null-string string |
Specifies the string to be written for a null value for string columns. |
|
--null-non-string |
Specifies the string to be written for a null value for non-string columns. |
|
--check-column |
Specifies the column for determining incremental data import. |
|
--incremental append or lastmodified |
Incrementally imports data. append: appends records, for example, appending records that are greater than the value specified by last-value. lastmodified: appends data that is modified after the date specified by last-value. |
|
--last-value |
Specifies the last value of the check column from the previous import. |
|
--input-null-string |
Specifies the string to be interpreted as NULL for string columns. |
|
--input-null-non-string |
Specifies the string to be interpreted as null for non-string columns. If this parameter is not specified, NULL will be used. |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.