Help Center/ PanguLargeModels/ User Guide/ Using Data Engineering to Create a Dataset/ Publishing a Dataset/ Publishing Video Datasets/ Evaluating Video Datasets
Updated on 2025-07-02 GMT+08:00
View PDF
Share

Evaluating Video Datasets

Before publishing a video dataset, ModelArts Studio allows you to evaluate the dataset to optimize data quality, ensure that the data meets high standards, and improve model performance.

If data evaluation is not required, skip this section and proceed with Publishing Video Datasets.

Creating Evaluation Standards for Video Datasets

ModelArts Studio provides preset evaluation standards for video datasets, covering multiple quality dimensions such as video definition, frame rate, integrity, and label accuracy. You can directly use those standards or create your evaluation standards.

If you want to use the preset evaluation standards, skip this section and proceed with Creating a Video Dataset Evaluation Task.

To create evaluation standards for video datasets, perform the following steps:

  1. Log in to ModelArts Studio Large Model Deveopment Platform. In the My Spaces area, click the required workspace.
    Figure 1 My Spaces
  2. In the navigation pane, choose Data Engineering > Data Management > Data Evaluation. On the Manual Evaluation Standard tab page, the preset video dataset evaluation standard Video Data Quality Standard V1.0 are displayed. You can click the evaluation standard name to view the specific evaluation items.
    Figure 2 Preset evaluation standards for video datasets

  3. On the Manual Evaluation Standard page, click Create custom standards, select Preset Standard, and set the evaluation standard name and description.
  4. Edit evaluation items.

    You can delete evaluation items or create custom evaluation items as required. When creating a custom evaluation item, ensure that the evaluation type, evaluation item, and evaluation item description are clear and unambiguous.

  5. Click Complete Creation.

    After evaluation standards are created, you can view, edit, and delete them on the Manual Evaluation Standard page.

Creating a Video Dataset Evaluation Task

Only processed datasets can be evaluated.

Before creating a video dataset evaluation task, generate a processed dataset by referring to Processing Video Datasets.

To create a video dataset evaluation task, perform the following steps:

  1. Log in to ModelArts Studio Large Model Deveopment Platform. In the My Spaces area, click the required workspace.
    Figure 3 My Spaces
  2. In the navigation pane, choose Data Engineering > Data Management > Data Evaluation. On the displayed page, click Create Evaluation Task in the upper right corner.
  3. Select the dataset to be evaluated and set the number of samples.
  4. Click Next Step and select the evaluation standard. Click Next Step to select the evaluator, and click Next Step to enter the task name.
  5. Click Complete Creation to go to the Data Evaluation page. After the evaluation task is created, the status becomes Created.
  6. Click Evaluate in the Operation column.
  7. On the evaluation page, label the problems of the current data by referring to the evaluation items. If the data meets the requirements, click Passed. If the data does not meet the requirements, click Not passed.
  8. After all data is evaluated, check that the evaluation progress is 100% on the Manual Evaluation page.

    Click Report in the Operation column to view the dataset quality evaluation report.