HDFS on Hue
Hue provides the file browser function for users to use HDFS in GUI mode.
The Hue page is used to view and analyze data such as files and tables. Do not perform high-risk management operations such as deleting objects on the page. If an operation is required, you are advised to perform the operation on each component after confirming that the operation has no impact on services. For example, you can use the HDFS client to perform operations on HDFS files and use the Hive client to perform operations on Hive tables.
How to Use File Browser
Access the Hue web UI. For details, see Accessing the Hue Web UI.
Click . The File Browser page is displayed. You can perform the following operations:
- Viewing files or directories
By default, the directory and files in the directory of the login user are displayed. You can view Name, Size, User, Group,Permission, and Date.
Click a file name to view the text information or binary data in the text file. The file content can be edited.
If there are a large number of files and directories, you can enter keywords in the search text box to search for specific files or directories.
- Creating files or directories
Click New in the upper right corner. Choose File to create the file. Choose Directory to create a directory.
- Managing files or directories
Select the check box of a file or director, and click Actions. In the displayed menu, choose Rename, Move, Copy, and Change permissions to rename, move, copy, or change the file or directory permissions.
- Uploading files
Click Upload in the upper right corner and click Select files or drag the file to the window.
How to Use Storage Policies
If the value of Hue parameter fs_defaultFS is set to viewfs://ClusterX, the big data storage policy cannot be enabled.
Storage policies on the Hue web UI are classified into the following two types:
- Static Storage Policies
According to the access frequency and importance of documents in HDFS, specify a storage policy for an HDFS directory, such as ONE_SSD or ALL_SSD. The files in this directory can be migrated to the storage media.
- Dynamic Storage Policies
Set rules for an HDFS directory. The system can automatically change the storage policy, change the number of file copies, or move the file directory. For details, see Configuring HDFS Cold and Hot Data Migration.
Before configuring a dynamic storage policy on the Hue WebUI, you must set the CRON expressions for cold and hot data migration and start automatic cold and hot data migration on Manager.
Operations:
Modify the following NameNode parameters of HDFS. For details, see Modifying Cluster Service Configuration Parameters.
Parameter
Description
Example Value
dfs.auto.data.mover.enable
Whether to enable automatic hot and cold data migration. The default value is false.
true
dfs.auto.data.mover.cron.expression
CRON expression for hot and cold data migration in HDFS, which is used to control the start time of data migration. This parameter is available only when dfs.auto.data.mover.enable is set to true. The default value is 0 * * * *, indicating that the task is executed on the hour.
0 * * * *
Table 1 describes the expression for modifying the dfs.auto.data.mover.cron.expression parameter. * indicates consecutive time segments.
To set storage policies on the web UI, perform the following operations:
- Log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager (MRS 3.x or Later).
- On FusionInsight Manager, choose
.
- Set Role Name.
- In the Configure Resource Permission area, choose Name of the desired cluster > Hue, select Storage Policy Admin, and click OK. Then, grant the permission to the role.
- Choose System > Permission > User Group > Create User Group. Set Group Name, and click Add next to Role. On the displayed page, select the created role, click OK to add the role to the group, and click OK.
- Choose System > Permission > User > Create.
- Username: Enter the name of the user to be added.
- Set User Type to Human-machine.
- Set Password and Confirm Password for logging in to the Hue web UI.
- Click Add next to User Group. On the page that is displayed, select the created user group in 3, supergroup, hadoop, and hive, and click OK.
- Set Primary Group to hive.
- Click Add next to Role. On the page that is displayed, select the created role in 2 and the System_administrator role, and click OK.
- Click OK. The user is added successfully.
- Access the Hue web UI as the created user.
- In the left navigation pane, click . The File Browser page is displayed.
- Select the check box of a directory and choose Action on the top of the page. Choose Storage policies.
Figure 1 Storage policies
- In the dialog box that is displayed, set a new storage policy and click OK.
- On the Static Storage Policy page, you can set a static storage policy and click Save.
- On the Dynamic Storage Policy page, you can create, delete, or modify a dynamic storage policy. Table 2 describes the parameters.
Table 2 Parameters of the dynamic storage policy Category
Parameter
Description
Rule
Last Access to File
Indicates the time when the file is last accessed.
Last File Modification
Indicates the time when the file is last modified.
Operation
Change Number of Copies
Indicates the number of file copies.
Modify Storage Policy
Indicates that you can modify storage policies to the following: HOT, WARM, COLD, ONE_SSD, and ALL_SSD.
Move to Directory
Indicates that you can move the file to another directory.
- You need to consider whether the rules conflict with each other and whether the rules damage the system when setting rules.
- When a directory is configured with multiple rules and operations, the rule that is triggered first is located at the bottom of the rule/operation list, and the rules that are triggered later are placed from bottom to top to prevent repeated operations.
- The system checks whether the files under the directory specified by the dynamic storage policy meet the rules on an hourly basis. If the files meet the rules, the execution is triggered. Execution logs are recorded in the /var/log/Bigdata/hdfs/nn/hadoop.log directory of the active NameNode.
Typical Scenarios
On the Hue page, view and edit HDFS files in text or binary mode as follows:
Viewing a File
- Access the Hue web UI.
- In the left navigation pane, click . The File Browser page is displayed.
- Click the name of the file to be viewed.
- Click View as binary to switch from the text mode to the binary mode. Click View as file to switch from the binary mode to the text mode.
Editing a file
- Click Edit File. The file content can be edited.
- Click Save or Save As to save the file.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.