On this page

Show all

Shell Command Introduce

Updated on 2022-11-18 GMT+08:00

HDFS Shell

You can use the Hadoop Distributed File System (HDFS) Shell command to perform operations on the HDFS, such as reading and writing files.

To run the HDFS Shell:

Go to the directory of HDFS client and enter the command. An example is shown as follows:

cd /opt/client/HDFS/hadoop/bin

hdfs dfs -mkdir /tmp/input

You can run the following command to seek help about HDFS commands:

hdfs --help

For details about the shell, see

http://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-common/FileSystemShell.html

Table 1 Transparent encryption-related commands

Scenario

Operation

Command

Description

hadoop shell command management key

Create keys.

hadoop key create <keyname> [-cipher <cipher>] [-size <size>] [-description <description>] [-attr <attribute=value>] [-provider <provider>] [-help]

The create subcommand creates a key for the name specified by the <keyname> argument within the provider specified by the -provider argument. You may specify a cipher with the -cipher argument. The default cipher is "AES/CTR/NoPadding" currently.

The default keysize is 128. You may specify the requested key length using the -size argument. Arbitrary attribute=value style attributes may be specified using the -attr argument. The -attr may be specified for multiple times, once per attribute.

Rollback

hadoop key roll <keyname> [-provider <provider>] [-help]

The roll subcommand creates a new version for the specified key within the provider indicated using the -provider argument.

Delete keys

hadoop key delete <keyname> [-provider <provider>] [-f] [-help]

The delete subcommand deletes all versions of the key specified by the <keyname> argument within the provider specified by the -provider argument. The command asks for user confirmation unless -f is specified.

View keys

hadoop key list [-provider <provider>] [-metadata] [-help]

The list subcommand displays the keynames contained in a particular provider as configured in core-site.xml or specified with the -provider argument. The -metadata argument displays the metadata.

Table 2 Shell commands of Colocation client

Operation

Command

Description

Group creation

hdfs colocationadmin -createGroup -groupId

<groupID> -locatorIds <comma separated locatorIDs> or -file <path of the file contains all of locatorIDs>

Used to create a group. In the command, groupID is the group name and locatorID is the locator name. You can enter comma-separated locator IDs using command lines. You can also write locator IDs into a file so that the system can obtain the locator IDs by reading the file.

Group deletion

hdfs colocationadmin -deleteGroup <groupID>

Used to delete the specified group.

Group query

hdfs colocationadmin -queryGroup <groupID>

Used to query details about a specified group, including locators in the group and information about each locator and its corresponding DataNode.

Viewing all groups

hdfs colocationadmin -listGroups

Used to list all groups and their creation time.

Setting ACL permissions on Colocation directories

hdfs colocationadmin -setAcl

Used to set ACL permissions on Colocation directories in ZooKeeper.

The default root directory of Colocation in ZooKeeper is /hadoop/colocationDetails.

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback