Help Center/ MetaStudio/ API Reference/ Virtual Avatar Livestreaming/ Livestreaming Task Management/ Starting a Virtual Human Intelligent Livestreaming Task
Updated on 2024-07-04 GMT+08:00

Starting a Virtual Human Intelligent Livestreaming Task

Function

Starts a virtual human intelligent livestreaming task.

Calling Method

For details, see Calling APIs.

URI

POST /v1/{project_id}/smart-live-rooms/{room_id}/smart-live-jobs

Table 1 Path Parameters

Parameter

Mandatory

Type

Description

project_id

Yes

String

Project ID. For details about how to obtain the project ID, see Obtaining a Project ID.

room_id

Yes

String

Script ID.

Request Parameters

Table 2 Request header parameters

Parameter

Mandatory

Type

Description

X-Auth-Token

No

String

User token. This parameter is mandatory when token authentication is used.

You can obtain the token by calling the IAM API used to obtain a user token.

Value of X-Subject-Token in the response header.

Authorization

No

String

Authentication information. This parameter is mandatory for AK/SK authentication.

X-Sdk-Date

No

String

Time when the request is sent. This parameter is mandatory for AK/SK authentication.

The format is YYYYMMDD'T'HHMMSS'Z'.

X-Project-Id

No

String

Project ID. This parameter is mandatory for AK/SK authentication.

X-App-UserId

No

String

Third-party user ID, which does not allow Chinese characters.

Table 3 Request body parameters

Parameter

Mandatory

Type

Description

video_config

No

VideoConfig object

Video output configuration.

play_policy

No

PlayPolicy object

Script playback policy.

output_urls

No

Array of strings

Address of the third-party livestreaming platform for RTMP video stream push.

stream_keys

No

Array of strings

Stream key of the third-party livestreaming platform for RTMP video stream push, which corresponds to the ingest URL.

interaction_callback_url

No

String

Interactive callback URL, including authentication information.

live_event_callback_config

No

LiveEventCallBackConfig object

Live event callback notification configuration.

rtc_callback_config

No

RTCLiveEventCallBackConfig object

RTC callback event configuration. Compatible processing. If the configuration is not carried, LIVE_PROGRESS is subscribed by default.

view_mode

No

String

Landscape or portrait mode. Default value: VERTICAL

  • LANDSCAPE: landscape

  • VERTICAL: portrait

co_streamer_config

No

CoStreamerConfig object

Co-streamer configuration

Table 4 VideoConfig

Parameter

Mandatory

Type

Description

clip_mode

No

String

Clipping mode of the output video. Default value: RESIZE.

  • RESIZE: video scaling

  • CROP: video cropping

codec

Yes

String

Video encoding format and video file format.

  • H264: H.264 encoding, MP4 file output

  • VP8: VP8 encoding, WebM file output

bitrate

Yes

Integer

Average output bitrate.

Unit: kbit/s

The value ranges from 40 to 30,000.

NOTE:
  • Quality is prioritized for virtual avatar video production, which may exceed the preset bitrate.

  • Bitrate range for virtual avatar video production: [1000, 8000].

width

Yes

Integer

Video width.

Unit: pixel.

NOTE:
  • When clip_mode is set to RESIZE, the following resolutions are supported: 1920 x 1080, 1080 x 1920, 1280 x 720, 720 x 1280, 3840 x 2160, and 2160 x 3840. 4K videos can be used only when the virtual avatar model supports 4K.

  • When clip_mode is set to CROP, (dx, dy) is the origin, and the width is the actual width of the reserved video.

  • Currently, only 1080 x 1920 is supported for virtual avatar livestreaming.

height

Yes

Integer

Video height.

Unit: pixel.

NOTE:
  • When clip_mode is set to RESIZE, the following resolutions are supported: 1920 x 1080, 1080 x 1920, 1280 x 720, 720 x 1280, 3840 x 2160, and 2160 x 3840.

  • When clip_mode is set to CROP, (dx, dy) is the origin, and the height is the actual height of the reserved video.

  • Currently, only 1080 x 1920 is supported for virtual avatar livestreaming.

frame_rate

No

String

Frame rate.

Unit: FPS

NOTE:
  • The frame rate of virtual avatar videos is fixed at 25 FPS.

is_subtitle_enable

No

Boolean

Whether the output video is subtitled. The default value is false.

  • true: Displays the subtitle.

  • false: Hides the subtitle.

subtitle_config

No

SubtitleConfig object

Subtitle configuration.

dx

No

Integer

Coordinates of the pixel in the upper left corner of the cropped video.

This parameter takes effect when clip_mode is set to crop.

NOTE:
  • The canvas size is based on the model resolution. For example, for a model with the resolution of 1920 x 1080, the value of dx ranges from 0 to 1920.

dy

No

Integer

Coordinates of the pixel in the upper left corner of the cropped video.

This parameter takes effect when clip_mode is set to crop.

NOTE:
  • The canvas size is based on the model resolution. For example, for a model with the resolution of 1920 x 1080, the value of dy ranges from 0 to 1080.

is_enable_super_resolution

No

Boolean

Whether super resolution is enabled for a video.

NOTE:

true: enable

false: do not enable

Table 5 SubtitleConfig

Parameter

Mandatory

Type

Description

dx

No

Integer

Coordinates of the pixel in the lower left corner of the subtitle.

NOTE:
  • The video resolution is 1920 x 1080 in landscape mode (16:9) and 1080 x 1920 in portrait mode (9:16).

dy

No

Integer

Coordinates of the pixel in the lower left corner of the subtitle.

NOTE:
  • The video resolution is 1920 x 1080 in landscape mode (16:9) and 1080 x 1920 in portrait mode (9:16).

font_name

No

String

Font. The following fonts are supported:

  • HarmonyOS_Sans_SC_Black: HarmonyOS bold

  • HarmonyOS_Sans_SC_Regular: HarmonyOS normal

  • HarmonyOS_Sans_SC_Thin: HarmonyOS light

font_size

No

Integer

Font size.

The value ranges from 4 to 120.

h

No

Integer

Subtitle box height

NOTE:
  • The parameter h is used to facilitate the calculation of the coordinates in the upper left corner of the subtitle box by the front end. This parameter is not used in the background.

w

No

Integer

Subtitle box width

NOTE:
  • The subtitle box width is fixed at 80/ %of the screen width.

  • The parameter w is used to facilitate the calculation of the coordinates in the upper left corner of the subtitle box by the front end. This parameter is not used in the background.

Table 6 PlayPolicy

Parameter

Mandatory

Type

Description

repeat_count

No

Integer

Number of times that a script is played repeatedly.

The value –1 indicates that the playback is repeated until it is stopped manually.

The value 0 indicates that it is played only once.

If the value is n, the actual number of running times is n + 1.

auto_play_script

No

Boolean

Indicates whether to automatically play a script.

true: The script is automatically played after the task is initialized.

false: After the task is initialized, the script is not played until signal appears.

play_mode

No

String

Control mode. The default value is TEXT.

TEXT: text control, that is, TTS is used

AUDIO: speech control

NO_PRESET: no preset script, that is, manual control is used

random_play_mode

No

String

Random broadcast mode.

  • NONE: Random broadcast is disabled.

  • SCENE: Random broadcast by scene is enabled. Paragraphs in a scene are broadcast in sequence.

  • SCRIPT_ITEM: Random broadcast by paragraph is enabled. Scenes are broadcast in sequence.

  • SCENE_AND_SCRIPT_ITEM: Scenes and paragraphs are broadcast randomly.

need_independent_capture_client

No

Boolean

Whether an independent data collection device is required

Table 7 LiveEventCallBackConfig

Parameter

Mandatory

Type

Description

live_event_type_callback_url

No

String

Live event callback URL. HTTPS URL, which must contain an authentication string.

auth_type

No

String

Authentication type.

  • None: The URL contains authentication.

  • MSS_A: In HMACSHA256 signature mode, the hwSecret and hwTime parameters are added to the URL. Value format: hwSecret=hmac_sha256(Key, URI (live_event_callback_url) + hwTime)&hwTime=hex(timestamp)

  • MSS_A_HEAD: In HMACSHA256 signature mode, the hwSecret and hwTime parameters are placed in Head.

    Value format: x-hw-mss-secret=hmac_sha256(Key, URI (live_event_callback_url) + hwTime)

    x-hw-mss-time=hex(timestamp)

  • MEITUAN_DEFAULT: This parameter is used only for Meituan's API callback.

key

No

String

Key

callback_event_type

No

Array of strings

Live event types in callbacks.

Currently, only the following options are supported:

  • SHOOT_SCRIPT_SWITCH: script paragraph switching event

  • RTMP_STREAM_STATE_CHANGE: RTMP link change callback event

  • REPLY_COMMAND_FINISH: Playback is complete.

    The callback event structure is defined as follows:

  • event_type: event type

  • message: event description

    The SHOOT_SCRIPT_SWITCH event callback is defined as follows:

{
  "event_type":  "SHOOT_SCRIPT_SWITCH",
  "message":"{\"room_id\":\"26f065244f754b3aa853b649a21aaf66\",\"job_id\":\"e87104f76d7546ce8a46ac6b04c49c3c\",\"scene_script_name\":\"Product 1\",\"shoot_script_sequence_no\":\"2\",\"shoot_script_title\":\"Paragraph 2\"}"
}

The RTMP_STREAM_STATE_CHANGE callback is defined as follows:

{
  "event_type":  "RTMP_STREAM_STATE_CHANGE",
  "message":"{\"room_id\":\"26f065244f754b3aa853b649a21aaf66\",\"job_id\":\"e87104f76d7546ce8a46ac6b04c49c3c\",\"output_url\":\"rtmp://xxx/xx/xx\",\"stream_key\":\"xxxxx\",\"state\":\"CONNECTED\"}"
}

The value of state can be CONNECTING, CONNECTED, DISCONNECTED, RECONNECTING, or END.

The REPLY_COMMAND_FINISH callback is defined as follows:

{
  "event_type":  "REPLY_COMMAND_FINISH",
  "message":"{\"room_id\":\"26f065244f754b3aa853b649a21aaf66\",\"job_id\":\"e87104f76d7546ce8a46ac6b04c49c3c\",\"reply_id\":\"e87104f76d7546ce8a46ac6b04c49c3c"}"
}
Table 8 RTCLiveEventCallBackConfig

Parameter

Mandatory

Type

Description

rtc_callback_event_type

No

Array of strings

Live event types in RTC callbacks.

Currently, only the following options are supported:

LIVE_PROGRESS: live script progress notification

REPLY_COMMAND_FINISH: Playback is complete.

The callback event structure is defined as follows:

  • message_type: message type

  • data: message description

    The LIVE_PROGRESS event callback is defined as follows:

{
    "message_type": "live_progress_notify",
    "data": {
        "script_name": "Scene 1",
        "shoot_script_sequence_no": 2,
        "shoot_script_title": "Greeting",
        "offset": "247",
        "reply_id": "e87104f76d7546ce8a46ac6b04c49c3c"
    }
}

The REPLY_COMMAND_FINISH callback is defined as follows:

{
  "message_type": "reply_command_finish_notify",
  "data":"{
    "reply_id":"e87104f76d7546ce8a46ac6b04c49c3c"
  }"
}
Table 9 CoStreamerConfig

Parameter

Mandatory

Type

Description

voice_config

No

VoiceConfig object

Voice configuration parameter.

streamer_action

No

String

Behavior of the streamer when the co-streamer speaks.

  • SILENCE: silent

  • VOLUME_DOWN: decreased volume

Table 10 VoiceConfig

Parameter

Mandatory

Type

Description

voice_asset_id

Yes

String

Timbre asset ID.

speed

No

Integer

Speech speed.

The value defaults to 100 and ranges from 50 to 200.

NOTE:
  • The value 100 indicates the normal speaking speed of an adult, which is about 250 words per minute.

  • 50 indicates 0.5x speaking speed, 100 indicates normal speaking speed, and 200 indicates 2x speaking speed.

pitch

No

Integer

Pitch.

The value defaults to 100 and ranges from 50 to 200.

volume

No

Integer

Volume.

The value defaults to 140 and ranges from 90 to 240.

Response Parameters

Status code: 202

Table 11 Response header parameters

Parameter

Type

Description

X-Request-Id

String

Request ID.

Table 12 Response body parameters

Parameter

Type

Description

job_id

String

Livestreaming task ID.

rtc_room_info

RTCRoomInfoList object

RTC room information.

live_event_report_url

String

URL for reporting livestreaming events. Users report their own livestreaming events to this URL to trigger intelligent interaction and automatic reply.

live_event_callback_config

LiveEventCallBackConfig object

Live event callback notification configuration.

live_warning_info

Array of LiveWarningItem objects

List of livestreaming risk alarms.

Table 13 RTCRoomInfoList

Parameter

Type

Description

app_id

String

RTC application ID.

room_id

String

RTC room ID.

users

Array of RTCUserInfo objects

Information about users joining an RTC room.

Table 14 RTCUserInfo

Parameter

Type

Description

user_type

String

User type.

  • CAPTURE: Live assistant, which pushes video streams captured by cameras to RTC rooms.

  • ANIMATION: Visual control service, which pulls video streams from RTC rooms to generate motion data.

  • RENDER: Rendering service, which renders motion data into virtual human animations.

  • PLAYER: Common viewer, who can watch the original video stream or virtual human animation video stream.

  • INFERENCE_USER: Virtual human inference user, who receives audio streams from and pushes video streams to an RTC room.

  • END_USER: Device user, who receives video streams from and pushes audio streams to an RTC room.

user_id

String

RTC user ID.

signature

String

RTC authentication token.

ctime

Long

Validity period. Timestamp, in seconds.

Table 15 LiveEventCallBackConfig

Parameter

Type

Description

live_event_type_callback_url

String

Live event callback URL. HTTPS URL, which must contain an authentication string.

auth_type

String

Authentication type.

  • None: The URL contains authentication.

  • MSS_A: In HMACSHA256 signature mode, the hwSecret and hwTime parameters are added to the URL. Value format: hwSecret=hmac_sha256(Key, URI (live_event_callback_url) + hwTime)&hwTime=hex(timestamp)

  • MSS_A_HEAD: In HMACSHA256 signature mode, the hwSecret and hwTime parameters are placed in Head.

    Value format: x-hw-mss-secret=hmac_sha256(Key, URI (live_event_callback_url) + hwTime)

    x-hw-mss-time=hex(timestamp)

  • MEITUAN_DEFAULT: This parameter is used only for Meituan's API callback.

key

String

Key

callback_event_type

Array of strings

Live event types in callbacks.

Currently, only the following options are supported:

  • SHOOT_SCRIPT_SWITCH: script paragraph switching event

  • RTMP_STREAM_STATE_CHANGE: RTMP link change callback event

  • REPLY_COMMAND_FINISH: Playback is complete.

    The callback event structure is defined as follows:

  • event_type: event type

  • message: event description

    The SHOOT_SCRIPT_SWITCH event callback is defined as follows:

{
  "event_type":  "SHOOT_SCRIPT_SWITCH",
  "message":"{\"room_id\":\"26f065244f754b3aa853b649a21aaf66\",\"job_id\":\"e87104f76d7546ce8a46ac6b04c49c3c\",\"scene_script_name\":\"Product 1\",\"shoot_script_sequence_no\":\"2\",\"shoot_script_title\":\"Paragraph 2\"}"
}

The RTMP_STREAM_STATE_CHANGE callback is defined as follows:

{
  "event_type":  "RTMP_STREAM_STATE_CHANGE",
  "message":"{\"room_id\":\"26f065244f754b3aa853b649a21aaf66\",\"job_id\":\"e87104f76d7546ce8a46ac6b04c49c3c\",\"output_url\":\"rtmp://xxx/xx/xx\",\"stream_key\":\"xxxxx\",\"state\":\"CONNECTED\"}"
}

The value of state can be CONNECTING, CONNECTED, DISCONNECTED, RECONNECTING, or END.

The REPLY_COMMAND_FINISH callback is defined as follows:

{
  "event_type":  "REPLY_COMMAND_FINISH",
  "message":"{\"room_id\":\"26f065244f754b3aa853b649a21aaf66\",\"job_id\":\"e87104f76d7546ce8a46ac6b04c49c3c\",\"reply_id\":\"e87104f76d7546ce8a46ac6b04c49c3c"}"
}
Table 16 LiveWarningItem

Parameter

Type

Description

warning_type

String

Alarm type.

  • TOO_LESSS_SCRIPT_ITEMS: There are too few paragraphs (scripts).

  • TOO_SHORT_SCRIPT_TIME: The total duration of paragraphs (scripts) is too short.

  • TOO_LESS_DANMAKU_RULES: There are too few on-screen comment interaction rules.

  • RANDOM_PLAY_CLOSED: The random playback function is disabled.

  • ROTATION_MODEL_CLOSED: Streamer rotation is not configured.

Status code: 400

Table 17 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

error_msg

String

Error description.

Status code: 401

Table 18 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

error_msg

String

Error description.

Status code: 500

Table 19 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

error_msg

String

Error description.

Example Requests

POST https://{endpoint}/v1/70b76xxxxxx34253880af501cdxxxxxx/smart-live-rooms/24bad716-87b1-45e5-8912-6102f7693265/smart-live-jobs

{
  "output_urls" : [ "rtmp://vfxpush.hwcloudvr.cn/live/lysa" ]
}

Example Responses

Status code: 202

Succeeded.

{
  "job_id" : "26f06524-4f75-4b3a-a853-b649a21aaf66"
}

Status code: 400

{
  "error_code" : "MSS.00000003",
  "error_msg" : "Invalid parameter"
}

Status code: 401

{
  "error_code" : "MSS.00000001",
  "error_msg" : "Unauthorized"
}

Status code: 500

{
  "error_code" : "MSS.00000004",
  "error_msg" : "Internal Error"
}

Status Codes

Status Code

Description

202

Succeeded.

400

Parameters error, including the error code and its description.

401

Authentication is not performed or fails.

500

Internal service error.

Error Codes

See Error Codes.