Updated on 2024-04-30 GMT+08:00

Parameter Overview

You can use DatasetImportStep to create a dataset import phase. The following is an example of defining a DatasetImportStep.

Table 1 DatasetImportStep

Parameter

Description

Mandatory

Data Type

name

Name of a dataset import phase. The name contains a maximum of 64 characters, including only letters, digits, underscores (_), and hyphens (-). It must start with a letter and must be unique in a workflow.

Yes

str

inputs

Inputs of the dataset import phase.

Yes

DatasetImportInput or DatasetImportInput list

outputs

Outputs of the dataset import phase.

Yes

DatasetImportOutput or DatasetImportOutput list

properties

Configurations for dataset import.

Yes

ImportDataInfo

title

Title for frontend display.

No

str

description

Description of the dataset import phase.

No

str

policy

Phase execution policy.

No

StepPolicy

depend_steps

Dependency phases.

No

Step or step list

Table 2 DatasetImportInput

Parameter

Description

Mandatory

Data Type

name

Input name of the dataset import phase. The name can contain a maximum of 64 characters, including only letters, digits, underscores (_), and hyphens (-), and must start with a letter. The input name of a step must be unique.

Yes

str

data

Input data object of the dataset import phase.

Yes

Dataset, OBS, or labeling job object. Currently, only Dataset, DatasetConsumption, DatasetPlaceholder, OBSPath, OBSConsumption, OBSPlaceholder, LabelTask, LabelTaskPlaceholder, LabelTaskConsumption, and DataConsumptionSelector are supported.

Table 3 DatasetImportOutput

Parameter

Description

Mandatory

Data Type

name

Output name of the dataset import phase. The name can contain a maximum of 64 characters, including only letters, digits, underscores (_), and hyphens (-), and must start with a letter. The output name of a step must be unique.

Yes

str

Table 4 ImportDataInfo

Parameter

Description

Mandatory

Data Type

annotation_format_config

Configurations of the imported labeling format

No

AnnotationFormatConfig

excluded_labels

Samples with specified labels are not imported.

No

Label list

import_annotated

Whether to import the labeled samples in the original dataset to the To Be Confirmed tab. The default value is false, indicating that the labeled samples in the original dataset are not imported to the To Be Confirmed tab. Options:

  • true: The labeled samples in the original dataset are imported to the To Be Confirmed tab.
  • false: The labeled samples in the original dataset are not imported to the To Be Confirmed tab.

No

bool

import_annotations

Whether to import labels. Options:

  • true: The labels are imported. (Default)
  • false: The labels are not imported.

No

bool

import_samples

Whether to import samples. Options:

  • true: The samples are imported. (Default)
  • false: The samples are not imported.

No

bool

import_type

Import mode. Options:

  • dir: imported from an OBS path
  • manifest: imported from a manifest file

No

ImportTypeEnum

included_labels

Samples with specified labels are imported.

No

Label list

label_format

Label format. This parameter is used only for text datasets.

No

LabelFormat

Table 5 AnnotationFormatConfig

Parameter

Description

Mandatory

Data Type

format_name

Name of a labeling format

No

AnnotationFormatEnum

parameters

Advanced parameters of the labeling format

No

AnnotationFormatParameters

scene

Labeling scenario, which is optional

No

LabelTaskTypeEnum

Table 6 AnnotationFormatParameters

Parameter

Description

Mandatory

Data Type

difficult_only

Whether to import only hard examples. Options:

  • true: Only hard examples are imported.
  • false: All the samples are imported. (Default)

No

bool

included_labels

Samples with specified labels are imported.

No

Label list

label_separator

Separator between labels. By default, the comma (,) is used as the separator. The separator needs to be escaped. The separator can contain only one character, which must be a letter, a digit, or any of the following special characters: !@#$%^&*_=|?/':.;,

No

str

sample_label_separator

Separator between the text and label. By default, the Tab key is used as the separator. The separator needs to be escaped. The separator can contain only one character, which must be a letter, a digit, or any of the following special characters: !@#$%^&*_=|?/':.;,

No

str