HDFS on Hue

How to Use File Browser

Access the Hue web UI. For details, see Accessing the Hue Web UI.

Click . The File Browser page is displayed. You can perform the following operations:

How to Use Storage Policies

If the value of Hue parameter fs_defaultFS is set to viewfs://ClusterX, the big data storage policy cannot be enabled.

Storage policies on the Hue web UI are classified into the following two types:

• Static Storage Policies
  1. Current storage policy

     According to the access frequency and importance of documents in HDFS, specify a storage policy for an HDFS directory, such as ONE_SSD or ALL_SSD. The files in this directory can be migrated to the storage media.

  2. Current EC policy

     Specify EC policies for HDFS directories, such as RS-10-4-1024k and RS-3-2-1024k. .

• Dynamic Storage Policies

  Set rules for an HDFS directory. The system can automatically change the storage policy, the number of file copies, migrate the file directory..

  Before configuring a dynamic storage policy on the Hue WebUI, you must set the CRON expressions for cold and hot data migration and start automatic cold and hot data migration on Manager.

  Operations:

  Change the value of dfs.auto.data.mover.cron.expression for NameNode of the HDFS service. For details, see Modifying Cluster Service Configuration Parameters.

  

  • dfs.auto.data.mover.cron.expression indicates the CRON expression for checking whether HDFS data meets the dynamic storage policy rule. It is used to control the start time of data migration. The default value is 0 * * * *, indicating that the detection is performed on the hour. When the dynamic storage policy rule is met, cold and hot data migration tasks are executed on the hour.
  • The default value of dfs.auto.data.mover.enable is false. This parameter value is valid only when dfs.auto.data.mover.enable is set to true.

  Table 1 describes the expression for modifying this parameter. * indicates consecutive time segments.

  Table 1 Parameters in the execution expression

  Column	Description
  1	Minute. The value ranges from 0 to 59.
  2	Hour. The value ranges from 0 to 23.
  3	Date. The value ranges from 1 to 31.
  4	Month. The value ranges from 1 to 12.
  5	Week. The value ranges from 0 to 6. 0 indicates Sunday.

To set storage policies on the web UI, perform the following operations:

1. Log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager.

2. On FusionInsight Manager, choose System > Permission > Role > Create Role.
   1. Set Role Name.
   2. In the Configure Resource Permission area, choose Name of the desired cluster > Hue, select Storage Policy Admin, and click OK. Then, grant the permission to the role.

3. Choose System > Permission > User Group > Create User Group. Set Group Name, and click Add next to Role. On the displayed page, select the created role, click OK to add the role to the group, and click OK.
4. Choose System > Permission > User > Create.
   1. Username: Enter the name of the user to be added.
   2. Set User Type to Human-machine.
   3. Set Password and Confirm Password for logging in to the Hue web UI.
   4. Click Add next to User Group. On the page that is displayed, select the created user group in 3, supergroup, hadoop, and hive, and click OK.
   5. Set Primary Group to hive.
   6. Click Add next to Role. On the page that is displayed, select the created role in 2 and the System_administrator role, and click OK.
   7. Click OK. The user is added successfully.

5. Access the Hue web UI as the created user. For details, see Accessing the Hue Web UI.
6. In the left navigation pane, click . The File Browser page is displayed.
7. Select the check box of a directory and choose Action on the top of the page. Choose Storage policies.
   Figure 1 Storage policies
   

8. In the dialog box that is displayed, set a new storage policy and click OK.
   • On the Static Storage Policy tab page, select a policy from the New EC Policy drop-down list box and click Save.
   • On the Dynamic Storage Policy tab page, you can create, delete, or modify a dynamic storage policy. Table 2 describes the parameters.

     Table 2 Parameters of the dynamic storage policy

     Category	Parameter	Description
     Rule	Last Access to File	Indicates the time when the file is last accessed.
     	Last File Modification	Indicates the time when the file is last modified.
     Operation	Change Number of Copies	Indicates the number of file copies.
     	Modify Storage Policy	Indicates that you can modify storage policies to the following: HOT, WARM, COLD, ONE_SSD, and ALL_SSD.
     	Move to Directory	Indicates that you can move the file to another directory.

     

     • You need to consider whether the rules conflict with each other and whether the rules damage the system when setting rules.
     • When a directory is configured with multiple rules and operations, the rule that is triggered first is located at the bottom of the rule/operation list, and the rules that are triggered later are placed from bottom to top to prevent repeated operations.
     • The system checks whether the files under the directory specified by the dynamic storage policy meet the rules on an hourly basis. If the files meet the rules, the execution is triggered. Execution logs are recorded in the /var/log/Bigdata/hdfs/nn/hadoop.log directory of the active NameNode.

Typical Scenarios

On the Hue page, view and edit HDFS files in text or binary mode as follows:

Viewing a File

1. Access the Hue web UI. For details, see Accessing the Hue Web UI.
2. In the left navigation pane, click . The File Browser page is displayed.
3. Click the name of the file to be viewed.
4. Click View as binary to switch from the text mode to the binary mode. Click View as file to switch from the binary mode to the text mode.

Editing a file

5. Click Edit File. The file content can be edited.
6. Click Save or Save As to save the file.