更新时间:2024-11-29 GMT+08:00

配置MRS客户端

进入Spark客户端安装目录“/opt/client/Spark2x/spark/conf”,在spark-defaults.conf配置文件如下参数:

spark.pyspark.driver.python=/usr/bin/python3
spark.yarn.dist.archives=hdfs://hacluster/user/python/python36.zip#Python