Help Center> ModelArts> Best Practices> Frequently-used Frameworks> Using Caffe for Handwritten Digit Recognition
View PDF

Using Caffe for Handwritten Digit Recognition

This section describes how to use Caffe to recognize handwritten digits in images from an MNIST dataset on the ModelArts platform.

The following describes the process of using Caffe to recognize handwritten digits in images.

  1. Preparing Data: Obtain the MNIST dataset and upload it to OBS.
  2. Compiling a Model: Compile the training script, compile the .prototxt file to define the network structure, configure various parameters for network training, and upload the parameters to the OBS bucket.
  3. Training the Model: Use the ModelArts service to create a training job for model training.
  4. Deploying the Model: After training is completed, import the model to ModelArts and deploy the model as a real-time service.
  5. Performing Prediction: Initiate a prediction request and obtain the prediction result.

Preparing Data

ModelArts provides a sample MNIST dataset named Mnist-Data-Set. This example uses this dataset to build a model. Perform the following operations to upload the dataset to the OBS directory test-modelarts/dataset-mnist created in preparation.

  1. Download the Mnist-Data-Set dataset to the local PC.
  2. Decompress the Mnist-Data-Set.zip file, for example, to the Mnist-Data-Set directory on the local PC.
  3. Upload all files in the Mnist-Data-Set folder to the test-modelarts/dataset-mnist directory on OBS in batches. For details about how to upload files, see Uploading a File.

    The following provides content of the Mnist-Data-Set dataset. .gz is the compressed package.

    • t10k-images-idx3-ubyte: validation set, which contains 10,000 samples
    • t10k-images-idx3-ubyte.gz: compressed package file of the validation set.
    • t10k-labels-idx1-ubyte: labels of the validation set, which contains the labels of the 10,000 samples
    • t10k-labels-idx1-ubyte.gz: compressed package file of the validation set label.
    • train-images-idx3-ubyte: training set, which contains 60,000 samples
    • train-images-idx3-ubyte.gz: compressed package file of the training set.
    • train-labels-idx1-ubyte: labels of the training set, which contains the labels of the 60,000 samples
    • train-labels-idx1-ubyte.gz: compressed package file of the training set label.

Compiling a Model

After data is prepared, compile a model and related configuration files using the Caffe API. In this sample, ModelArts provides compiled training scripts and related configuration files.

  1. Download the ModelArts-Lab project from Gitee, decompress the file, and obtain the train.py and src folders from the ModelArts-Lab-master\official_examples\Using_Caffe_to_Create_a_MNIST_Dataset_Recognition_Application\codes directory of the project. For details about the files contained in the src folder, see Table 1.
    Table 1 Sample code files

    File

    Description

    train.py

    Training script

    src/lenet_solver.prototxt

    .prototxt file for configuring training parameters

    src/lenet_train_test.prototxt

    Network structure file during training

    src/lenet_deploy.prototxt

    Inference model

    src/customize_service.py

    Inference script

    src/config.json

    Configuration file used for inference
  2. Upload the obtained files to OBS, for example, /test-modelarts/caffe/codes/. The structure is as follows:

    The absolute paths of the configuration files and training script have been compiled. It is recommended that the OBS folders you upload be named codes.

    test-modelarts
    |--caffe
        |--codes
            |--train.py
            |--src
                |--lenet_solver.prototxt
                |--lenet_train_test.prototxt
                |--lenet_deploy.prototxt
                |--customize_service.py
                |--config.json

Training the Model

Use the uploaded dataset and the training program script to train the model. After the training, the model is generated in the specified directory.

  1. On the ModelArts management console, choose Training Management > Training Jobs, and click Create in the upper left corner.
  2. On the Create Training Job page, set required parameters based on Figure 1 and Figure 2, and click Next.

    Algorithm Source: The selected code directory must contain the files listed in Table 1 and must be named codes.

    Data Source: Select Data path, and then select the OBS path for saving the dataset.

    Running Parameter: data_path_suffix specifies the name of the last level of the code directory. In this example, data_path_suffix is codes. If the codes directory is not set, the training job will fail to be executed.

    Resource Pool: The GPU resource pool is recommended because the training takes a long time.
    Figure 1 Basic information for creating a training job
    Figure 2 Parameters for creating a training job
  3. On the Confirm tab page, check the parameters of the training job and click Submit.
  4. On the Training Jobs page, when the training job status changes to Running Success, the model training is completed. If any exception occurs, click the job name to go to the job details page and view the training job logs.
    The system stores the generated model in the model folder under Training Output Path. You can view the generated model file in OBS. In this example, the generated model is stored in /test-modelarts/caffe/mnist-model/model.

    The training job may take more than 10 minutes to complete. If the training time exceeds a certain period (for example, one hour), manually stop it to release resources. Otherwise, the account balance may be insufficient, especially for the training job using GPUs.

Deploying the Model

After the model training is completed, deploy the model as a real-time prediction service. Before deploying a model, ensure that the inference code and configuration file prepared in Preparing Data have been uploaded as required. You can also develop your own inference code and configuration file based on Model Package Specifications.

  1. On the ModelArts management console, choose Model Management > Models in the left navigation pane. On the displayed Models page, click Import.
  2. On the Import Model page, set required parameters as shown in Figure 3 and click Next.
    In the Meta Model Source area, select OBS. Set Meta Model to the path specified by Training Output Path in the training job.
    Figure 3 Import Model

    On the Models page, if the model status changes to Normal, the model has been imported successfully.

  3. Click the triangle next to a model name to expend all versions of the model. In the row of a version, choose Deploy > Real-Time Services in the Operation column to deploy the model as a real-time service.
    On the Deploy page, set parameters shown in Figure 4 and follow the instructions to create a real-time service.
    Figure 4 Deploying the model

Performing Prediction

After the model is deployed, wait until the service deployment is completed. If the service status changes to Running, the service has been deployed successfully.

  1. On the Real-Time Services page, click the name of the real-time service. The real-time service details page is displayed.
  2. Choose Prediction. Click Upload next to Image File to upload an image with a white handwritten digit on a black background and click Predict.

    After the prediction is completed, the prediction result is displayed in the Test Result pane. According to the prediction result, the digit on the image is 1.

    • As specified in the inference code and configuration file, the size of the image used for prediction must be 28 x 28 pixels, and the image must contain white handwritten digits on a black background.
    • You are advised not to use the images provided by the dataset. You can use the drawing tool provided by the Windows operating system to draw an image for prediction.
    Figure 5 Prediction results
  3. If you do not need to use this model and real-time service any more, clear related resources to avoid unnecessary fees.
    • On the Real-Time Services page, choose More > Stop or Delete to stop or delete the created real-time service.
    • On the Model Management page, choose More > Delete to delete the imported model.
    • On the Training Jobs page, choose More > Delete to delete the finished training job.
    • Go to OBS and delete the OBS bucket, folders, and files in this example.