"well-known file is not secure" Is Displayed When the jstack Command Is Used to Check the Process Stack

Symptom

When a user runs the jstack command to view the process stack information, the following error message is displayed:

omm@hadoop02:~> jstack 62517
62517: well-known file is not secure

Cause Analysis

1. The user running the jstack command is inconsistent with the user submitting the process for viewing the pid information.
2. Storm uses the feature of differentiating users for implementing tasks. When the worker process is started, the process UID and GID are changed to the user submitting the task and ficommon. This way, logviewer can access logs of the worker process and only log file permission 640 is open. After the user is changed, the jstack and jmap commands fail to be executed for the worker process, because the default GID of the user is not ficommon. You need to run the ldap command to change the user GID to 9998 (ficommon).

Solution

You can use either of the following two methods to resolve the problem:

Method 1: View the process stack on the native Storm page.

1. Log in to the native Storm page.

   MRS Manager:

   1. Access MRS Manager.
   2. Choose Services > Storm. In Storm WebUI of Storm Summary, click any UI link to access the Storm WebUI.

   FusionInsight Manager:

   1. Log in to FusionInsight Manager.
   2. On Manager, choose Cluster > Service > Storm. On the Storm WebUI page of Overview, click any UI link to open the Storm WebUI.

2. Select the topology to be viewed.

   

3. Select the spout or bolt to be viewed.

   

4. Select the log file of the node to be viewed, and then click JStack or Heap. JStack corresponds to the stack information, and Heap corresponds to the heap information.

   

Method 2: View the process stack by modifying user-defined parameters.

1. Access the Storm parameter configuration page.

   MRS Manager: Log in to MRS Manager, choose Services > Storm > Service Configuration, and select All from the Type drop-down list.

   Operation on FusionInsight Manager: Log in to FusionInsight Manager and choose Cluster > Services > Yarn > Configurations > All Configurations.

2. In the navigation tree on the left, choose supervisor > Customize and add the variable supervisor.run.worker.as.user=false.

3. Click Save Configuration and select Restart the affected services or instances. Click OK to restart the services.
4. Submit the topology again.
5. Switch to the omm user on the background node and run the jps command to view the PID of the worker process.

   

6. Run the jstack pid command to view the jstack information.