Updated on 2022-09-14 GMT+08:00

Handling an Authentication Failure

Symptom

An authentication failure occurs during the commissioning and running of an example project.

Troubleshooting Process

There are many reasons that will cause an authentication failure. You are advised to perform the following steps for troubleshooting in different scenarios:

  1. Check whether the network connection between the device where this application runs and the cluster is normal, and check whether the TCP and UDP ports required by Kerberos authentication can be accessed.
  2. Check whether each configuration file is correctly read and stored in a correct directory.
  3. Ensure that the username and keytab file are obtained according to the operation guide.
  4. Check whether the configuration information is properly set before initiating an authentication.
  5. Check whether multiple authentication requests are initiated in the same process. That is, check whether the login() method is called repeatedly.
  6. If the problem persists, contact Huawei engineers for further analysis.

Authentication Failure Examples

"clock skew too great" is displayed during authentication.

  1. Check the cluster time.
  2. Check the time of the machine where the development environment is located. The difference between the machine time and the cluster time must be less than 5 minutes.

"(Receive time out) can not connect to kdc server" is displayed during authentication.

  1. Check whether the content of the krb5.conf file is correct. That is, check whether the file content is the same as the service IP address configuration of KerberoServer in the cluster.
  2. Check whether the Kerberos service is running properly.
  3. Check whether the firewall is disabled.

An exception occurs when a client application submits a task to the Hadoop cluster, and the message "Failed to find any Kerberos tgt" or "No valid credentials provided" is displayed.

  1. Check whether the kinit command is executed. If no, perform kinit authentication before submitting the task.
  2. In the multi-thread scenario, when the process starts, call the loginfromkeytab function provided by Hadoop to log in to KDC to obtain TGT. Before submitting tasks, call the reloginFromKeytab function to update the TGT.

    // After the first login succeeds, set user group information.
    UserGroupInformation.loginUserFromKeytab(this.userPrincipal,this.keytabFile);
    // Before the thread submits the task:
    UserGroupInformation.getLoginUser().reloginFromKeytab();

  3. If multiple scripts use the kinit command to authenticate the same user, run the export KRB5CCNAME=keytab_path command before running the kinit command in each script to ensure that the path specified by KRB5CCNAME in each script process is inconsistent.