Installing Python 3

Updated on 2023-05-15 GMT+08:00

View PDF

Log in to the client node outside the cluster as user root and run the following command to check whether Python 3 is installed:

python3 --version
- If yes, go to Configuring the MRS Client.
- If no, go to 2.
Install Python. Python 3.6.6 is used as an example.
1. Run the following commands to install dependencies:
  yum install zlib zlib-devel zip -y
  
  yum install gcc-c++
  
  yum install openssl-devel
  
  yum install sqlite-devel -y
  
  If the pandas library requires the following dependencies:
  
  yum install -y xz-devel
  
  yum install bzip2-devel
2. Run the wget https://www.python.org/ftp/python/3.6.6/Python-3.6.6.tgz command to download the source code of Python.
3. Run the following command to decompress the Python source code package, for example, to the opt directory:
  cd /opt
  
  tar –xvf Python-3.6.6.tgz
4. Create a Python installation directory, for example, /opt/python36:
  mkdir /opt/python36
5. Compile Python.
  cd /opt/python-3.6.6
  
  ./configure --prefix=/opt/python36
  
  The following information is displayed if the commands are executed successfully:
  
  Run the make -j8 command. The following information is displayed if the command is executed successfully:
  
  Run the make install command. The following information is displayed if the command is executed successfully:
6. Run the following commands to configure the Python environment:
  export PYTHON_HOME=/opt/python36
  
  export PATH=$PYTHON_HOME/bin:$PATH
7. Run the python3 --version command. Python has been installed if the following information is displayed:
Verify Python 3.

pip3 install helloword

python3

import helloworld

helloworld.say_hello("test")
Install third-party Python libraries, such as pandas and sklearn.

pip3 install pandas

pip3 install backports.lzma

pip3 install sklearn
Run the python3 -m pip list command to check the installation result.
Pack them into Python.zip.

cd /opt/python36/

zip -r python36.zip ./*
Create an HDFS directory and upload the package to the directory for future use.

hdfs dfs -mkdir /user/python

hdfs dfs -put python36.zip /user/python