Typical Flink Parameters
Parameter |
Description |
Recommended Value |
Description |
---|---|---|---|
-c |
Main class name |
Set this parameter as needed. |
Yes |
-yjm |
Memory of the JobManager process. The default value is 2 GB. |
Set this parameter as needed. |
No |
-ytm |
Memory of the TaskManager process. The default value is 4 GB. |
Set this parameter as needed. |
No |
-ynm |
Name of the Flink YARN job. |
Set this parameter as needed. |
Yes |
-ys |
Number of slots in the TaskManager. |
2 |
No |
execution.checkpointing.interval |
Interval for triggering a checkpoint, which can be added using -yD. The unit is ms. |
60000 |
Yes |
execution.checkpointing.timeout |
Checkpoint timeout interval. You can run the -yD command to add a checkpoint timeout interval. The default value is 30 minutes. |
30min |
Yes |
execution.checkpointing.tolerable-failed-checkpoints |
Total number of tolerable checkpoint failures, which can be added using -yD. |
1000 |
No |
state.checkpoints.num-retained |
Number of reserved checkpoints, which can be added using -yD. |
5 |
No |
state.backend |
RocksDB used as the state backend, which can be added using -yD. |
rocksdb |
Enabled |
state.backend.incremental |
Incremental RocksDB state backends, which can be added using -yD. |
TRUE |
Yes |
state.backend.rocksdb.block.blocksize |
Size of data blocks on the write state backend. The value can be added using -yD. |
512KB |
Yes |
state.backend.rocksdb.block.cache-size |
Size of the block cache on the state backend, which can be added using -yD. |
1024MB |
Yes |
taskmanager.memory.jvm-overhead.max |
Maximum size of the local memory used for other JVM overheads, such as stack space and garbage collection space. The value can be added using -yD. |
10g |
No |
taskmanager.memory.jvm-overhead.fraction |
Ratio of the local memory used for other JVM overheads to TM memory, such as stack space and garbage collection space. The value can be added using -yD. |
0.2 |
No |
parallelism.default |
Job parallelism. For example, to add the job parallelism for the join operator, use -yD. The default value is 1. |
Set this parameter as needed. |
No |
table.exec.state.ttl |
TTL (join ttl) of Flink state, which can be added using -yD. The default value is 0. |
Set this parameter as needed. |
Yes |
heartbeat.timeout |
Heartbeat timeout interval between the JM and TM, which can be added using -yD. |
1800000 |
Yes |
akka.ask.timeout |
Akka communication timeout, which can be added using -yD. |
240s |
Yes |
taskmanager.memory.segment-size |
Size (in bytes) of the memory buffer block used by the memory management and network stack. The default value is 32768 (32 KB). You can add the value using -yD. |
64kb |
No |
taskmanager.network.memory.max-buffers-per-channel |
Maximum number of buffers that each channel can hold. If there are many idle segments, you can set a larger value. Otherwise, the channel will be blocked because it cannot obtain segments. You can run -yD to set this parameter. |
100 |
No |
taskmanager.network.memory.buffers-per-channel |
Number of buffers exclusively used by each channel, which can be added by using -yD. |
10 |
No |
taskmanager.network.memory.floating-buffers-per-gate |
Number of floating buffers used by each channel, which can be added by using -yD. |
2000 |
No |
taskmanager.network.netty.server.numThreads |
Number of Netty server threads in each TaskManager, which can be added using -yD. |
20 |
No |
taskmanager.network.netty.client.numThreads |
Number of Netty client threads in each TaskManager, which can be added using -yD. |
20 |
No |
state.backend.rocksdb.files.open |
Maximum number of files that can be opened. The value -1 indicates that there is no limit. You can run -yD to set this parameter. |
-1 |
No |
state.backend.rocksdb.compaction.level.use-dynamic-size |
Dynamic data volume threshold of data storage at each layer. You can use -yD to set this parameter. |
TRUE |
No |
state.backend.rocksdb.levels.num |
Number of compaction layers allowed by RocksDB, which can be added using -yD. |
10 |
No |
state.backend.rocksdb.compaction.style |
Compaction algorithm, which can be added using -yD. |
FIFO |
No |
state.backend.rocksdb.verify.checksum |
Whether to disable data check during read. You can run -yD to set this parameter. |
FALSE |
No |
state.backend.rocksdb.thread.num |
Maximum number of concurrent threads responsible for flushing and compaction in the background. You can run -yD to set this parameter. |
4 |
No |
state.backend.rocksdb.writebuffer.count |
Maximum number of memtables, which can be added using -yD. |
5 |
No |
state.backend.rocksdb.writebuffer.number-to-merge |
Minimum number of memtables that are merged before flush. The value can be added using -yD. |
3 |
No |
state.backend.rocksdb.background.compaction.max |
Maximum number of compaction threads, which can be added using -yD. |
10 |
No |
state.backend.rocksdb.flush.max |
Number of RocksDB flush threads, which can be added using -yD. |
1 |
No |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot