更新时间:2025-11-25 GMT+08:00

配置MRS客户端

  1. 登录Spark客户端节点,进入Spark客户端安装目录“/opt/client/Spark2x/spark/conf”

    cd /opt/client/Spark2x/spark/conf

  2. 在spark-defaults.conf配置文件如下参数:
    spark.pyspark.driver.python=/usr/bin/python3
    spark.yarn.dist.archives=hdfs://hacluster/user/python/python36.zip#Python