Manual Calibration

Constraints

• A maximum of 10 manual calibration tasks can be created using an IAM account.
• The number of samples in a dataset cannot exceed 10,000.

Creating a Manual Calibration Task

1. Log in to the ModelArts console. In the navigation pane on the left, choose Data Preparation > Data Refining.
2. In the upper right corner of the Manual Calibration tab page, click Create. On the Create Manual Calibration page, configure related information and click OK.
   Figure 1 Create Manual Calibration
   

   Table 1 Parameters for creating a manual calibration task

   Parameter		Description	Example Value
   Basic Information	Name	Custom task name. The default value is data-calibration-YYYYMMDDHHMMSS. The name must start with a letter and end with a letter or digit. It can contain 2 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.	data-calibration-20260425164425
   	Description	Description of the manual calibration task. Only letters, digits, spaces, hyphens (-), underscores (_), commas (,), periods (.), brackets, colons (:), and commas are allowed. It can contain a maximum of 256 characters.	-
   	Task due date.	The maximum duration is 30 days. Manual calibration tasks cannot be operated after expiration; complete them before the deadline.	2026/05/25 23:59:59
   Calibration Configuration	Type	The text type is supported. You can select single-turn or multi-turn Q&A as needed. Single-turn Q&A: Precise Q&A pairing for static knowledge; efficiently builds single-turn dialogue samples. Multi-turn Q&A: Maintains contextual flow for in-depth consultation; creates logically coherent interaction samples.	Multi-turn Q&A
   Associated Dataset		The system automatically filters the types of My Datasets available for manual calibration based on your selected calibration scenario. Click the card, select a dataset as required, and click OK.	Multi-turn Q&A chain-of-thought ShareGPT-0327-001

   When the status of the manual calibration task changes to Processing, the task is created.

Executing a Calibration Task

Perform calibration on the visualized labeling page. Use the visual interface to fix errors, add missing labels, and use batch verification tools to guarantee high data accuracy. The following uses a multi-turn Q&A task as an example.

1. On the Manual Calibration tab page, click Label on the right of the manual calibration task whose status is Processing.
2. Proofread the input, thinking process, and output in the Q&A pair, and click Submit.
   • Incorrect content: Modify the content as required.
   • Invalid content: Click Invalid Data in the lower right corner of the page. Marking items as invalid will reset the annotations to their original state.

   The task progress is displayed on the right of the manual calibration task name. After all data is labeled, the task status changes to Completed.

Generating Datasets

Summarize and export the calibration results, and integrate the calibration data. You can preview details, verify integrity, generate standard datasets in one-click mode, and synchronize the datasets to My Data for future use.

1. On the Manual Calibration tab page, click Generate on the right of the manual calibration task.
2. In the Generate Dataset pane, configure related information and click OK.

   Table 2 Parameters for generating a dataset

   Parameter	Description	Example Value
   Dataset Name	Name of a custom dataset. The name must start with a letter and end with a letter or digit. It can contain 2 to 63 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.	dataset
   Output Data Content	Select labeled data, unlabeled content, valid data, and invalid data as required.	Labeled data and valid data
   Storage Location	Choose Object Storage Service – Bucket or Object Storage Service – Parallel File System as the storage type. Click to select an OBS storage address or manually enter an OBS storage address. The storage address must start with obs:// or / and end with a slash (/). It cannot contain double slashes (//) except in the prefix. For example, obs://bucketname/path/ or /bucketname/path/.	obs://bucketname/path/
   Dataset Property	Click to configure dataset properties as required, such as the industry and language.	-
   Description	Description of a custom dataset. Only letters, digits, spaces, hyphens (-), underscores (_), commas (,), periods (.), brackets, colons (:), and commas are allowed. The value can contain a maximum of 100 characters.	-
   Dataset Status	Only published datasets can be used by downstream tasks such as model training. If you select Publish Dataset, the generated dataset is in the Online state on the Asset Management > Data > My Data page and can be directly used by downstream model training jobs. If you do not select Publish Dataset, the generated dataset will be in the Offline state on the Asset Management > Data > My Data page and cannot be directly used by downstream model training jobs. You need to manually publish the dataset before using it.	Select Publish Dataset.

Viewing Calibration Task Details

On the Manual Calibration tab page, click the name of a manual calibration task to view the task details, including the number of samples that have been calibrated, total number of samples, total progress, basic information about the calibration task, associated datasets, and generated datasets.

Deleting a Manual Calibration Task

You can delete unnecessary manual calibration tasks. Deleted manual calibration tasks cannot be restored. Proceed with caution.

1. On the Manual Calibration tab page, click Delete on the right of the manual calibration task name.
2. In the Delete Manual Calibration Task dialog box, enter DELETE and click OK.

Follow-Up Operations

The generated dataset can be used for secondary manual calibration, model training, and other operations.