Uploading Files from a Local Path to JupyterLab

Methods for Uploading a File

• For a file not exceeding 100 MB, directly upload it to the target notebook instance. Detailed information, such as the file size, upload progress, and upload speed are displayed.
• For a file that exceeds 100 MB but does not exceed 50 GB, upload the file to OBS (an object bucket or a parallel file system), and then download the file from OBS to a notebook instance. After the download is complete, the file is deleted from OBS.
• For a file that exceeds 50 GB, upload it by calling ModelArts SDK or MoXing.
• For a file that shares the same name with an existing file in the current directory of a notebook instance, overwrite the existing file or cancel the upload.
• A maximum of 10 files can be uploaded at a time. The other files are in awaiting upload state. No folders can be uploaded. If a folder is required, compress it into a package, upload the package to notebook, and decompress the package in Terminal.

  unzip xxx.zip  # Directly decompress the package in the path where the package is stored.

  For more details, search for the decompression command in mainstream search engines.

• When multiple files are uploaded in a batch, the total number of files to be uploaded and the number of files that have been uploaded are displayed at the bottom of the JupyterLab window.

  

Uploading File

Method 1: Use JupyterLab to open a running notebook environment.

Figure 1 Dragging the file to JupyterLab

Method 2: Click in the navigation bar on the top of the window. In the displayed dialog box, drag or select a local file and upload it.

Figure 2 Clicking the upload button

Uploading a Local File Smaller Than 100 MB to JupyterLab

For a file not exceeding 100 MB, directly upload it to the target notebook instance. Detailed information, such as the file size, upload progress, and upload speed are displayed.

Figure 3 Uploading a file less than 100 MB

A message is displayed after the file is uploaded.

Figure 4 Uploaded

Uploading a Local File that Is 100 MB to 50 GB to JupyterLab

For a file that exceeds 100 MB but does not exceed 50 GB, upload the file to OBS (an object bucket or a parallel file system), and then download the file from OBS to the target notebook instance. After the download is complete, the file is automatically deleted from OBS.

For example, in the scenario shown in the following figure, upload the file through OBS.

Figure 5 Uploading a large file through OBS

To upload a large file through OBS, set an OBS path.

Figure 6 Uploading a file through OBS

Set an OBS path for uploading local files to JupyterLab. After the setting, this path is used by default in follow-up operations. To change the path, click in the lower left corner of the file upload window, as shown in Figure 10.

• Method 1: Enter a valid OBS path in the text box and click OK.
  Figure 7 Configuring an OBS path
  

• Method 2: Select an OBS path in OBS File Browser and click OK.
  Figure 8 OBS File Browser
  

• Method 3: Use the default path.
  Figure 9 Using the default path to upload a file
  
  Figure 10 Setting an OBS path to upload a local file
  

After the OBS path is set, upload a file.

Figure 11 Uploading a file

Decompressing a package

After a large file is uploaded to Notebook JupyterLab as a compressed package, you can decompress the package in Terminal.

unzip xxx.zip  # Directly decompress the package in the path where the package is stored.

For more details, search for the decompression command in mainstream search engines.

Uploading a Local File Larger Than 50 GB to JupyterLab

A file exceeding 50 GB cannot be directly uploaded to JupyterLab.

To upload files exceeding 50 GB, upload them to OBS. Then, call the ModelArts MoXing or SDK API in the target notebook instance to read and write the files in OBS.

Figure 12 Uploading and downloading large files in a notebook instance

The procedure is as follows:

1. Upload the file from a local path to OBS. For details, see Uploading an Object.
2. Download the file from OBS to the notebook instance by calling the ModelArts SDK or MoXing API.
   • Method 1: Call the ModelArts SDK to download a file from OBS.

     Example code:

     1 2 3

     from modelarts.session import Session session = Session() session.obs.copy("obs://bucket-name/obs_file.txt","/home/ma-user/work/")

   • Method 2: Call the ModelArts MoXing API for reading an OBS file.

     import moxing as mox
     
     # Download the OBS folder sub_dir_0 from OBS to a notebook instance.
     mox.file.copy_parallel('obs://bucket_name/sub_dir_0', '/home/ma-user/work/sub_dir_0')
     # Download the OBS file obs_file.txt from OBS to a notebook instance.
     mox.file.copy('obs://bucket_name/obs_file.txt', '/home/ma-user/work/obs_file.txt')

     If a .zip file is downloaded, run the following command on the terminal to decompress the package:

     unzip xxx.zip  # Directly decompress the package in the path where the package is stored.

     After the code is executed, open the terminal shown in Figure 13 and run the ls /home/ma-user/work command to view the file downloaded to the notebook instance. Alternatively, go to JupyterLab. In the navigation pane on the left, view the downloaded file. If the file is not displayed, refresh the page, as shown in Figure 14.

     Figure 13 Terminal
     
     Figure 14 File downloaded to a notebook instance
     

Error Handling

If you download a file from OBS to your notebook instance and the system displays error message "Permission denied", perform the following operations for troubleshooting:

• Ensure that the target OBS bucket and notebook instance are in the same region. If they are in different regions, the access to OBS will be denied.
• In this case, ensure that the notebook account has the permission to read data in the OBS bucket.

For details, see Incorrect OBS Path on ModelArts.