On this page

Show all

Why Data loading Fails During off heap?

Updated on 2024-12-11 GMT+08:00

Question

Why Data Loading fails during off heap?

Answer

YARN Resource Manager will consider (Java heap memory + spark.yarn.am.memoryOverhead) as memory limit, so during the off heap, the memory can exceed this limit. So you need to increase the memory by increasing the value of the parameter spark.yarn.am.memoryOverhead.

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback