Quota for Submitting Flink Jobs to ZooKeeper Exceeds the Threshold
Symptom
The quota for submitting Flink jobs to ZooKeeper exceeds the threshold.
Possible Causes
- The number of submitted jobs exceeds the maximum number of jobs that can be processed by the flinkznode node.
- By default, each znode can store a maximum of 20000 files. The high availability (HA) policy of Flink allows files to be stored in the /flink_xxx directory on ZooKeeper. If the number of files exceeds the upper limit, the jobs fail to be submitted.
Solution
- Solution 1: Delete the root directory of flink_base.
By default, Flink submits jobs in flink_base node. You can change the root directory of flink_base to avoid the problem.
- Solution 2: Modify the quota limit of the root directory that reports the error.
- Log in to any ZooKeeper node.
Go to the ${BIGDATA_HOME}/FusionInsight_Current/X_X_quorumpeer/install/zookeeper/conf directory, find the zookeeper.keytab file, and upload the ticket to the client node where the command is executed.
- Run the following command to query the ticket information:
klist -k
- Run the following command to connect to ZooKeeper:
zkCli.sh zkCli.sh -server IP address of the ZooKeeper instance:Port number
The default port number is 24002. You can obtain the value from the ZooKeeper parameter clientPort.
- Run the following command to change the total quota:
setquota -n 250000 /flink_base
If the jobs are not submitted to a znode in flink_base, determine the node on which the quota is to be modified based on the error messages in the log.
- Log in to any ZooKeeper node.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot