Help Center/ MapReduce Service/ User Guide/ Interconnecting Jupyter Notebook with MRS Using Custom Python/ Installing Python 3
Updated on 2023-05-15 GMT+08:00
View PDF

Installing Python 3

  1. Log in to the client node outside the cluster as user root and run the following command to check whether Python 3 is installed:

    python3 --version

  2. Install Python. Python 3.6.6 is used as an example.

    1. Run the following commands to install dependencies:

      yum install zlib zlib-devel zip -y

      yum install gcc-c++

      yum install openssl-devel

      yum install sqlite-devel -y

      If the pandas library requires the following dependencies:

      yum install -y xz-devel

      yum install bzip2-devel

    2. Run the wget https://www.python.org/ftp/python/3.6.6/Python-3.6.6.tgz command to download the source code of Python.
    3. Run the following command to decompress the Python source code package, for example, to the opt directory:

      cd /opt

      tar –xvf Python-3.6.6.tgz

    4. Create a Python installation directory, for example, /opt/python36:

      mkdir /opt/python36

    5. Compile Python.

      cd /opt/python-3.6.6

      ./configure --prefix=/opt/python36

      The following information is displayed if the commands are executed successfully:

      Run the make -j8 command. The following information is displayed if the command is executed successfully:

      Run the make install command. The following information is displayed if the command is executed successfully:

    6. Run the following commands to configure the Python environment:

      export PYTHON_HOME=/opt/python36

      export PATH=$PYTHON_HOME/bin:$PATH

    7. Run the python3 --version command. Python has been installed if the following information is displayed:

  3. Verify Python 3.

    pip3 install helloword

    python3

    import helloworld

    helloworld.say_hello("test")

  4. Install third-party Python libraries, such as pandas and sklearn.

    pip3 install pandas

    pip3 install backports.lzma

    pip3 install sklearn

  5. Run the python3 -m pip list command to check the installation result.

  6. Pack them into Python.zip.

    cd /opt/python36/

    zip -r python36.zip ./*

  7. Create an HDFS directory and upload the package to the directory for future use.

    hdfs dfs -mkdir /user/python

    hdfs dfs -put python36.zip /user/python