Updated on 2024-06-12 GMT+08:00

Parsing a Manifest File

Parse a manifest file in either a local or OBS path. If an OBS path is used, a session is required.

manifest.parse_manifest(manifest_path, encoding='utf-8')

Sample Code

Parse a manifest file.
from modelarts.session import Session 
from modelarts.dataset.format.manifest import Manifest

path = "obs://your-obs-bucket/manifest/V001.manifest"
session = Session() 
manifest_info= Manifest.parse_manifest(path,session=session)

Parameters

Table 1 Request parameters

Parameter

Mandatory

Type

Description

manifest_path

Yes

String

Path for storing a manifest file, which can be a local path or an OBS path. If an OBS path is used, a session is required.

encoding

No

String

File encoding format, which defaults to UTF-8.

Table 2 manifest_info parameters

Parameter

Type

Description

size

Long

Number of samples.

samples

JSON Array

Sample list. For details, see Table 3.

Table 3 sample parameters

Parameter

Type

Description

source

String

URI of the labeled object. Supported schemes are OBS, HTTPS, and Content. Content indicates text, for example, "source": "s3://path-to-jpg" and "source": " content://I love machine learning".

annotations

JSON Array

Sample labels. If this parameter is not specified, the object is not labeled.

The annotations value is an object list. For details, see Table 4.

usage

String

What an object is used for, which can be training (TRAIN), evaluation (EVAL), test (TEST), or inference (INFERENCE). If this parameter is not specified, you can determine how to use the object.

inference_loc

String

Location of an inference result file. This parameter is available if a manifest file is generated in an inference service.

id

String

Sample ID.

source_type

String

Source type, for example, csv.

source_property

String

Attribute of the source.

hard

Boolean

Hard example or not. true for hard examples and false for not.

hard_coefficient

Double

Difficulty coefficient, ranging from 0 to 1.

hard_reasons

String

Label-level hard example reasons. Use a hyphen (-) to separate reason IDs of a hard example.

source_map

String

Source mapping.

Table 4 annotation parameters

Parameter

Type

Description

name

String

Label name

type

String

Label type

id

String

Label ID

annotation_loc

String

Location where a labeled file is stored. This parameter is mandatory only for object detection labeled files.

annotation_property

String

Label properties

confidence

Double

Confidence of machine labeling, which is a numeral ranging from 0 to 1

creation_time

String

Time when a label was created, which is the time when the label was written, not the time when the manifest file was generated

annotated_by

String

Annotator

annotation_format

String

Format of a labeled file, which defaults to PASCAL VOC

hard

Boolean

Hard example

hard_coefficient

Double

Difficulty level

annotation_loc_map

String

Mapping of the path for storing a labeled file