Skip to content

Kling โ€‹

Updated

Endpoint: POST https://vod.intl.tencentcloudapi.comAction: CreateAigcVideoTask

Kling is one of the most full-featured video generation models, supporting text-to-video, first-frame / first-and-last-frame video generation, and multimodal reference generation from images and video. The latest Kling 4 (4.0 / 4.0-Omni / 4.0-Flash) generates up to 30 seconds natively in a single clip โ€” see its dedicated API page. 3.0-Omni / O1 support multi-shot generation (smart storyboarding) and video editing, with extended capabilities such as motion control, lip sync, digital humans, and custom voices, and can generate audio at the same time.

Basic information โ€‹

ItemValue
ModelNameKling
ModelVersion4.0 / 4.0-Omni / 4.0-Flash / 3.0 / 3.0-turbo / 3.0-Omni / O1 / 2.6 / 2.5 / 2.1 / 2.0 / 1.6
DefaultModelVersion=3.0 / Resolution=1080P
Disable guardrails supportSupported

Supported specs by version โ€‹

VersionResolutionRatioLength
Common480P / 720P / 1080P / 2K / 4Kโ€”โ€”
4.0 / 4.0-Omni720P / 1080P / 2K (upscaled) / 4K (upscaled)21:9 / 16:9 / 9:16 / 1:13โ€“30 seconds
4.0-Flash720P / 1080P (upscaled) / 2K (upscaled) / 4K (upscaled)21:9 / 16:9 / 9:16 / 1:13โ€“20 seconds
3.0-turbo720P / 1080P / 2K / 4KText generation 16:9 / 9:16 / 1:13โ€“15 seconds

4.0-Flash is not simply "a cheaper 4.0"

Flash has explicit capability cuts: 20 seconds max (4.0 allows 30) and no last-frame or middle-frame support (image-to-video accepts a single first frame only), and it renders natively only up to 720P (1080P and above are upscaled). It does support omni-reference generation, but subjects (SubjectInfos) and voices remain 4.0-Omni only. Swapping the version string to 4.0-Flash in a 4.0 example will fail โ€” see Choosing a version.

Request parameters โ€‹

ParameterRequiredTypeDescription
ModelNameRequiredStringFixed value Kling
ModelVersionOptionalString4.0 / 4.0-Omni / 4.0-Flash / 3.0 / 3.0-turbo / 3.0-Omni / O1 / 2.6 / 2.5 / 2.1 / 2.0 / 1.6
PromptRequiredStringGeneration prompt
FileInfos.NOptionalArrayReference input. Usage is FirstFrame (first frame) or Reference (reference)
OutputConfig.ResolutionOptionalString480P / 720P / 1080P / 2K / 4K
OutputConfig.DurationOptionalIntegerVideo length (seconds)
OutputConfig.AspectRatioOptionalString16:9 / 9:16 / 1:1, etc.
ExtInfoOptionalStringWithin 2, encoded in JSON. Passes the detailed parameters of the advanced features below (motion control, lip sync, digital human).
FirstFrameFileId / LastFrameUrlOptionalStringFor first / last frame generation. Specify the reference frame as FileInfos's Usage=FirstFrame, and the last frame as LastFrameUrl/LastFrameFileId (2.1 version requires 1080P)
OutputConfig.
InputComplianceCheck
OptionalStringPassing Disabled explicitly disables input moderation. Contact your sales representative before use
OutputConfig.
OutputComplianceCheck
OptionalStringPassing Disabled explicitly disables output moderation. Contact your sales representative before use

Request example โ€‹

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "3.0",
  "Prompt": "a calm sunset over the ocean, cinematic",
  "OutputConfig": {
    "Resolution": "1080P",
    "Duration": 5,
    "AspectRatio": "16:9",
    "StorageMode": "Temporary"
  },
  "SessionContext": "job-001"
}

Response example โ€‹

json
{
  "AigcVideoTask": {
    "Status": "FINISH",
    "ErrCode": 0,
    "Progress": 100,
    "Output": {
      "FileInfos": [
        {
          "FileUrl": "http://<host>.vod2.myqcloud.com/.../aigcVideoGenFile.mp4",
          "ExpireTime": "2026-08-01T10:29:48Z",
          "MetaData": {
            "Width": 1920,
            "Height": 1080,
            "Duration": 5.07,
            "Container": "mov,mp4,m4a",
            "Bitrate": 9494850
          }
        }
      ]
    }
  }
}

Reference count limits (based on Kling) โ€‹

The numbers of reference images, reference subjects, and reference videos affect one another.

ConditionLimit
Reference video presentReference images + reference subjects combined: 4 or fewer
Reference video โ€”Reference images + reference subjects combined: 7 or fewer
Last frame usedUp to 2 reference images

You must also check ErrCode

Check both Status="FINISH" and ErrCode=0 in the completion response.

Special settings โ€‹

Disable guardrails โ€‹

Input and output moderation can be set separately.

To use the guardrail-disable parameters, first contact your sales representative to confirm usage permission. Even on supported engines, guardrails are disabled only when the parameter is passed explicitly. If the parameter is omitted, they are not disabled.

ParameterDisable valueApplies to
OutputConfig.InputComplianceCheckDisabledDisables input moderation
OutputConfig.OutputComplianceCheckDisabledDisables output moderation

Merge the following settings into an existing generation request. To disable both input and output moderation, pass both fields together.

json
{
  "OutputConfig": {
    "InputComplianceCheck": "Disabled",
    "OutputComplianceCheck": "Disabled"
  }
}

First / last frame generation (First / Last Frame) โ€‹

Given a start frame image and an end frame image, this creates a video that joins the two frames naturally. Specify the first frame as FileInfos's Usage=FirstFrame, and the last frame as LastFrameUrl (or LastFrameFileId).

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "2.1",
  "Prompt": "camera slowly pushes in, cinematic",
  "FileInfos": [
    {
      "Type": "Url",
      "Category": "Image",
      "Usage": "FirstFrame",
      "Url": "https://<cdn>/first.jpg"
    }
  ],
  "LastFrameUrl": "https://<cdn>/last.jpg",
  "OutputConfig": {
    "Resolution": "1080P",
    "Duration": 5,
    "AspectRatio": "16:9",
    "StorageMode": "Temporary"
  },
  "SessionContext": "job-001"
}

The 2.1 version requires 1080P

Kling 2.1 first / last frame generation uses OutputConfig.Resolution=1080P.

Motion control (motion_control) โ€‹

Applies the motion of a reference video to a person image to create a new video. Specify it with SceneType=motion_control, and specify the reference video and the person image together via FileInfos. Use version 3.0 (new) or 2.6 (standard entry).

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "2.6",
  "SceneType": "motion_control",
  "Prompt": "Generate a new video using the motion of the reference video",
  "FileInfos": [
    {
      "Type": "Url",
      "Category": "Video",
      "Url": "https://<cdn>/ref_motion.mp4"
    },
    {
      "Type": "Url",
      "Category": "Image",
      "Url": "https://<cdn>/person.webp"
    }
  ],
  "ExtInfo": "{\"AdditionalParameters\":\"{\\\"keep_original_sound\\\":\\\"no\\\",\\\"character_orientation\\\":\\\"video\\\"}\"}",
  "OutputConfig": {
    "StorageMode": "Temporary"
  },
  "SessionContext": "job-001"
}

Main ExtInfo parameters

keep_original_sound: yes (keep original audio)/no. character_orientation: image (image person direction, reference video โ‰ค10 seconds)/video (video person direction, reference video โ‰ค30 seconds).

Lip sync (lip_sync) โ€‹

Synchronizes mouth shapes to audio for a source where a person speaks. First use DescribeAigcFaceInfo to get SessionId and face information, then call SceneType=lip_sync. Pass both audio and video information via ExtInfo, and leave FileInfos empty.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "2.6",
  "SceneType": "lip_sync",
  "Prompt": "lip sync",
  "ExtInfo": "{\"AdditionalParameters\":\"{\\\"session_id\\\":\\\"845736590818832460\\\",\\\"face_choose\\\":[{\\\"face_id\\\":0,\\\"sound_file\\\":\\\"https://<cdn>/audio.mp3\\\",\\\"sound_start_time\\\":0,\\\"sound_end_time\\\":5000,\\\"sound_insert_time\\\":0,\\\"sound_volume\\\":2,\\\"original_audio_volume\\\":0}]}\"}",
  "OutputConfig": {
    "StorageMode": "Temporary"
  },
  "SessionContext": "job-001"
}

Two-stage flow

1 stage: use DescribeAigcFaceInfo to get session_id and face_id. 2 stage: when calling lip_sync, leave FileInfos empty and, in the face_choose array, specify face_id / sound_file, the interval (ms), and volume.

Digital human (avatar_i2v) โ€‹

Creates a video of a person speaking or moving from a person image 1 and audio. Specify it with SceneType=avatar_i2v, the person image with FileInfos, and the audio as ExtInfo's sound_file.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "2.6",
  "SceneType": "avatar_i2v",
  "Prompt": "talking naturally",
  "FileInfos": [
    {
      "Type": "Url",
      "Category": "Image",
      "Url": "https://<cdn>/portrait.png"
    }
  ],
  "ExtInfo": "{\"AdditionalParameters\":\"{\\\"sound_file\\\":\\\"https://<cdn>/audio.mp3\\\"}\"}",
  "OutputConfig": {
    "StorageMode": "Temporary"
  },
  "SessionContext": "job-001"
}

Audio input rules

Provide either sound_file (mp3/wav/m4a/aac, up to 5MB, 2~300 seconds) or audio_id.

Image / video reference (image_list / video_list) โ€‹

Inside the prompt you can point to items in the reference list with the <<<...>>> notation. The image list and video list are passed via FileInfos, and the kind is distinguished by Category. Order follows the order of the FileInfos array.

NotationTarget
<<<image_1>>> / <<<image_2>>> ...Category=Image items
<<<video_1>>> / <<<video_2>>> ...Category=Video items
<<<element_1>>> / <<<element_2>>> ...Reference subjects passed via SubjectInfos or element_list
json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "3.0-Omni",
  "FileInfos": [
    {
      "Type": "Url",
      "Category": "Image",
      "Url": "https://<cos>/f0.jpeg",
      "Usage": "Reference"
    },
    {
      "Type": "Url",
      "Category": "Image",
      "Url": "https://<cos>/f1.jpeg",
      "Usage": "Reference"
    },
    {
      "Type": "Url",
      "Category": "Video",
      "Url": "https://<cos>/1.mp4",
      "Usage": "Reference"
    }
  ],
  "Prompt": "let <<<image_1>>> hold hands with <<<image_2>>> in the environment of <<<video_1>>>",
  "OutputConfig": {
    "Duration": 8,
    "Resolution": "1080P",
    "AspectRatio": "16:9",
    "AudioGeneration": "Disabled",
    "StorageMode": "Temporary"
  },
  "SessionContext": "job-001"
}

ReferenceType: nature of the reference video โ€‹

When Category=Video, specify the purpose with ReferenceType. Applies to GV, Kling, and PixVerse.

ValueMeaning
featureFeature reference. References only the style or motion features of the video
baseEdit target. Modifies the video itself

Reference count limits โ€‹

ConditionLimit
Reference video presentReference images + reference subjects combined: 4 or fewer
Reference video โ€”Reference images + reference subjects combined: 7 or fewer
Last frame usedUp to 2 reference images

Registered reference subjects (SubjectInfos) โ€‹

Use this when reusing the same person or object across multiple tasks. Creating a reference subject in advance issues an ID, and the generation request references that ID. Kling supports only registered reference subjects.

Creating a reference subject API โ€‹

APIDescription
CreateAigcAdvancedCustomElementRecommended. Asynchronous. Supports both images and video. Returns TaskId in the response and queries with DescribeTaskDetail
CreateAigcCustomElementLegacy. Synchronous. Images only. Kept only for backward compatibility
DescribeAigcAdvancedCustomElementsLists created reference subjects
DeleteAigcAdvancedCustomElementDeletes a reference subject

If the overseas reference subject library is already activated, CreateAigcAdvancedCustomElement with DisableModeration=True passed in uses the overseas reference subject library.

Character registration โ€‹

Put the reference subject ID into SubjectInfos.N.Id and point to it with <<<element_N>>> in the prompt. Name is optional; even when attached, the reference notation in the prompt stays the same.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "3.0-Omni",
  "SubjectInfos": [
    {
      "Id": "858477278396170315"
    },
    {
      "Id": "858477602846711835"
    }
  ],
  "Prompt": "let <<<element_1>>> hold hands with <<<element_2>>> and spin around",
  "OutputConfig": {
    "StorageMode": "Temporary",
    "Resolution": "1080P"
  },
  "SessionContext": "job-001"
}

Registered reference subjects (legacy, ExtInfo) โ€‹

You can achieve the same behavior with ExtInfo's element_list, but SubjectInfos is recommended for new implementations.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "3.0-Omni",
  "Prompt": "let <<<element_1>>> hold hands with <<<element_2>>> and spin around",
  "OutputConfig": {
    "StorageMode": "Temporary",
    "Resolution": "1080P"
  },
  "ExtInfo": "{\"AdditionalParameters\": \"{\\\"element_list\\\": [{\\\"element_id\\\": 858477278396170315}, {\\\"element_id\\\": 858477602846711835}]}\"}",
  "SessionContext": "job-001"
}

Smart shots (multi_shot) โ€‹

The Kling 3.0 family can automatically compose multiple shots from a single sentence in the prompt, or you can specify a per-shot script directly. Pass it through ExtInfo's AdditionalParameters.

FieldTypeDefaultDescription
multi_shotBoolfalseWhen true, multi-shot mode. In this case the top-level Prompt is ignored
shot_typeStringEmptycustomize (specify directly) / intelligence (automatic). Required when multi_shot=true
multi_promptArrayEmptyPer-shot information. At most 6, at least 1

multi_prompt item structure:

FieldDescription
indexShot number
promptShot script. Up to 512 characters
durationShot length (seconds). 1 or more, and not more than the total length

The sum of shot lengths must equal the total length

Specify the sum of each shot's duration to be the same as OutputConfig.Duration.

customize: specify shots directly โ€‹

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "3.0",
  "Prompt": "not used when multi_shot is on",
  "OutputConfig": {
    "StorageMode": "Temporary",
    "Resolution": "1080P",
    "Duration": 5,
    "AspectRatio": "16:9",
    "AudioGeneration": "Enabled"
  },
  "ExtInfo": "{\"AdditionalParameters\": \"{\\\"multi_shot\\\": true, \\\"shot_type\\\": \\\"customize\\\", \\\"multi_prompt\\\": [{\\\"index\\\": 1, \\\"prompt\\\": \\\"A person sitting on a park bench, sunlight filtering through trees\\\", \\\"duration\\\": 2}, {\\\"index\\\": 2, \\\"prompt\\\": \\\"A car speeding down a rainy street, headlights glowing. Dynamic angle, focus on motion.\\\", \\\"duration\\\": 3}]}\"}",
  "SessionContext": "job-001"
}

Below is an example of building ExtInfo in Python.

python
import json
kl_ext_info = {
    "multi_shot": True,
    "shot_type": "customize",
    "multi_prompt": [
        {"index": 1, "prompt": "A person sitting on a park bench, sunlight filtering through trees", "duration": 2},
        {"index": 2, "prompt": "A car speeding down a rainy street, headlights glowing. Dynamic angle, focus on motion.", "duration": 3},
    ],
}
inner = json.dumps(kl_ext_info, ensure_ascii=False)
ext_info = json.dumps({"AdditionalParameters": inner}, ensure_ascii=False)

intelligence: automatic splitting โ€‹

When shot_type=intelligence, multi_prompt is ignored and the model composes shots based on Prompt.

Custom voice (voice_id) โ€‹

Specifies the voice when generating content that includes speech. Supported versions are 2.6 / 3.0 / 3.0-Omni; on 2.6 a voice ID can be specified only in 1080P.

Create the voice in advance through a separate API, pass it via ExtInfo's voice_list, then point to it with <<<voice_N>>> in the prompt.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "2.6",
  "FileInfos": [
    {
      "Type": "Url",
      "Category": "Image",
      "Url": "https://<cos>/portrait.png",
      "Usage": "FirstFrame"
    }
  ],
  "Prompt": "the person in the image says loudly with <<<voice_1>>>: I want to be free",
  "OutputConfig": {
    "StorageMode": "Permanent",
    "Duration": 5,
    "Resolution": "1080P",
    "AspectRatio": "9:16",
    "AudioGeneration": "Enabled"
  },
  "ExtInfo": "{\"AdditionalParameters\": \"{\\\"voice_list\\\": [{\\\"voice_id\\\": 869048851066937391}]}\"}",
  "SessionContext": "job-001"
}

Creating a custom voice (CreateAigcCustomVoice) โ€‹

The voice_id in the voice_list above is not something you already have. You first create your own voice from a piece of reference audio (or a video that contains audio) with the CreateAigcCustomVoice API, obtain its VoiceId, and then put it into the generation request.

Endpoint: POST https://vod.intl.tencentcloudapi.comAction: CreateAigcCustomVoice

Notes:

  • Custom voices are currently managed by the caller (create, record the VoiceId, and reuse it).
  • CreateAigcCustomVoice is an asynchronous API that returns a TaskId. Save the TaskId and poll the result with DescribeTaskDetail.
  • Use clean reference audio with a clear human voice and little noise; for value conventions, refer to the Kling platform documentation.

Request parameters

ParameterRequiredTypeDescription
SubAppIdRequiredIntegerVOD subapplication ID
VoiceNameRequiredStringVoice name, used for later identification and management
VoiceUrlOptionalStringURL of the reference audio / video (choose one of VoiceUrl / VideoId)
VideoIdOptionalStringVOD media ID used as the reference audio source (choose one of VoiceUrl / VideoId)
SessionIdOptionalStringSession ID for request deduplication
SessionContextOptionalStringSource context passed through to the callback
TasksPriorityOptionalIntegerTask priority

Request example

json
{
  "SubAppId": 221073,
  "VoiceName": "myVoice",
  "VoiceUrl": "https://cg-sdk-1258344699.cos.ap-nanjing.myqcloud.com/BackendBeauty/testcase/input.mp4",
  "VideoId": null,
  "SessionId": null,
  "SessionContext": null,
  "TasksPriority": null
}

Code example

python
# -*- coding: utf-8 -*-
import os
import json

from tencentcloud.common.common_client import CommonClient
from tencentcloud.common import credential
from tencentcloud.common.exception.tencent_cloud_sdk_exception import TencentCloudSDKException
from tencentcloud.common.profile.client_profile import ClientProfile
from tencentcloud.common.profile.http_profile import HttpProfile

try:
    # Read keys from environment variables to avoid hardcoding: set TENCENTCLOUD_SECRET_ID and TENCENTCLOUD_SECRET_KEY first
    # Keys can be obtained from the console at https://console.tencentcloud.com/cam/capi
    cred = credential.Credential(os.getenv("TENCENTCLOUD_SECRET_ID"), os.getenv("TENCENTCLOUD_SECRET_KEY"))
    httpProfile = HttpProfile()
    httpProfile.endpoint = "vod.intl.tencentcloudapi.com"
    clientProfile = ClientProfile()
    clientProfile.httpProfile = httpProfile

    params = "{\"SubAppId\":221073,\"VoiceName\":\"myVoice\",\"VoiceUrl\":\"https://cg-sdk-1258344699.cos.ap-nanjing.myqcloud.com/BackendBeauty/testcase/input.mp4\"}"
    common_client = CommonClient("vod", "2018-07-17", cred, "", profile=clientProfile)
    print(common_client.call_json("CreateAigcCustomVoice", json.loads(params)))
except TencentCloudSDKException as err:
    print(err)
go
package main

import (
	"os"
	"fmt"

	"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common"
	"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common/errors"
	"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common/profile"
	vod "github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/vod/v20180717"
)

func main() {
	// Read keys from environment variables: set TENCENTCLOUD_SECRET_ID and TENCENTCLOUD_SECRET_KEY first
	credential := common.NewCredential(
		os.Getenv("TENCENTCLOUD_SECRET_ID"),
		os.Getenv("TENCENTCLOUD_SECRET_KEY"),
	)
	cpf := profile.NewClientProfile()
	cpf.HttpProfile.Endpoint = "vod.intl.tencentcloudapi.com"
	client, _ := vod.NewClient(credential, "", cpf)

	request := vod.NewCreateAigcCustomVoiceRequest()
	request.SubAppId = common.Uint64Ptr(221073)
	request.VoiceName = common.StringPtr("myVoice")
	request.VoiceUrl = common.StringPtr("https://cg-sdk-1258344699.cos.ap-nanjing.myqcloud.com/BackendBeauty/testcase/input.mp4")

	response, err := client.CreateAigcCustomVoice(request)
	if _, ok := err.(*errors.TencentCloudSDKError); ok {
		fmt.Printf("An API error has returned: %s", err)
		return
	}
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s", response.ToJsonString())
}

Response example

The API returns a TaskId immediately, used to query the creation progress later:

json
{
  "Response": {
    "RequestId": "424661ad-4e48-4d42-8f31-5066cae7157c",
    "TaskId": "221073-CreateAigcCustomVoice-77226813bc1038c2720a63c6962b16e6t"
  }
}

Query the creation result (DescribeTaskDetail)

Use the TaskId returned above to poll DescribeTaskDetail until Status=FINISH. The voice ID is located at CreateAigcCustomVoiceTask.Output.InfoList[].VoiceId.

json
{
  "Response": {
    "CreateAigcCustomVoiceTask": {
      "ErrCode": 0,
      "Input": {
        "VideoId": "",
        "VoiceName": "myVoice",
        "VoiceUrl": "https://cg-sdk-1258344699.cos.ap-nanjing.myqcloud.com/BackendBeauty/testcase/input.mp4"
      },
      "Output": {
        "InfoList": [
          {
            "VoiceId": "858488987974373383",
            "VoiceInfo": "{\"voice_id\":\"858488987974373383\",\"voice_name\":\"myVoice\",\"trial_url\":\"https://.../xxx.quality.wav\",\"owned_by\":\"826925436873121851\",\"status\":\"succeed\"}"
          }
        ]
      },
      "Progress": 100,
      "Status": "FINISH",
      "TaskId": "221073-CreateAigcCustomVoice-77226813bc1038c2720a63c6962b16e6t"
    },
    "TaskType": "CreateAigcCustomVoice",
    "Status": "FINISH",
    "RequestId": "a6676cfe-865b-4564-b614-ea40ce126f46"
  }
}

Once you have the VoiceId (e.g. 858488987974373383), pass it as the voice_id value under ExtInfo โ†’ voice_list in the generation request described above. The trial_url in VoiceInfo is a temporary preview URL, so download and save it as soon as possible.

Video editing (base video) โ€‹

Takes an already generated video as the edit target and changes its content with a prompt. In Category=Video, specify ReferenceType=base.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "3.0-Omni",
  "FileInfos": [
    {
      "Type": "Url",
      "Category": "Video",
      "Url": "https://<cos>/source.mp4",
      "ReferenceType": "base"
    }
  ],
  "Prompt": "change the color of the main character's dress to white",
  "OutputConfig": {
    "StorageMode": "Permanent",
    "MediaName": "kling-video-edit"
  },
  "SessionContext": "job-001"
}

Difference between feature and base

feature takes only the features of the reference video and creates a new video. base edits that video itself. If the purpose is editing, you must specify base.

Multi-element editing (multi_elements) โ€‹

A Kling editing feature that selects a specific element (subject) in a video to delete or regenerate. With the two session API it prepares SessionId by creating a selection area, then submits the final generation task as CreateAigcVideoTask's SceneType=multi_elements โ€” a 3-stage flow.

1 stage InitializeAigcMultiElementsSelection uploads and parses the input video and issues SessionId 2 stage EditAigcMultiElementsSelection selection area editing (add / delete / clear / preview). Repeat until satisfied 3 stage CreateAigcVideoTask generates the final video with SceneType=multi_elements

Input video constraints

The input video must be 10 seconds or less in length, with width and height each 720px or more. The issued SessionId is valid for 24 hours.

Stage 1: initialize selection session (InitializeAigcMultiElementsSelection) โ€‹

Uploads and parses the video to edit and issues SessionId.

ParameterRequiredTypeDescription
SubAppIdRequiredIntegerVOD application ID
FileInfoOptionalObjectVideo source
FileInfo.TypeRequiredStringFile or Url
FileInfo.FileIdOptionalStringRequired when Type=File. VOD file ID
FileInfo.UrlOptionalStringRequired when Type=Url. An accessible video URL
json
{
  "SubAppId": 123456789,
  "FileInfo": {
    "Type": "Url",
    "Url": "https://<cdn>/input.mp4"
  }
}

Response fields:

FieldDescription
SessionIdSession ID. Valid for 24 hours; used in later calls
StatusRecognition code. 0 means success
FpsVideo frame rate
OriginalDurationVideo length (seconds)
Width / HeightVideo width / height
TotalFrameTotal number of frames
NormalizedVideoNormalized video URL
json
{
  "Response": {
    "SessionId": "<session-id>",
    "Status": 0,
    "Fps": 30.0,
    "OriginalDuration": 5.0,
    "Width": 720,
    "Height": 1280,
    "TotalFrame": 150,
    "NormalizedVideo": "https://<cdn>/normalized.mp4",
    "RequestId": "<request-id>"
  }
}

Stage 2: edit selection area (EditAigcMultiElementsSelection) โ€‹

One API distinguishes four actions by SelectionAction. add (add selection point) / delete (delete selection point) / clear (reset all) / preview (mask preview). Call it repeatedly until the selection is satisfactory.

ParameterRequiredTypeDescription
SubAppIdRequiredIntegerVOD application ID
SelectionActionRequiredStringadd / delete / clear / preview
SessionIdRequiredStringSession ID obtained in the 1 stage
FrameIndexOptionalIntegerRequired for add/delete. Target frame number
PointsOptionalArrayRequired for add/delete. Normalized coordinates. Point.X / Point.Y are in the 0~1 range
json
{
  "SubAppId": 123456789,
  "SelectionAction": "add",
  "SessionId": "<session-id>",
  "FrameIndex": 0,
  "Points": [
    {
      "X": 0.773,
      "Y": 0.297
    }
  ]
}

Response fields:

FieldAppears inDescription
StatusCommonRecognition code. 0 means success
Result.FrameIndexadd / deleteFrame number operated on
Result.RleMaskList.ObjectIdadd / deleteSelected object ID
Result.RleMaskList.RleMaskadd / deleteSelection mask RLE (Counts) and size (Size)
Result.RleMaskList.PngMask.Urladd / deletePNG mask URL
Result.Video / VideoCoverpreviewPreview video / cover URL with the mask shown
Result.TrackingOutputpreviewPer-frame mask tracking result URL

Responses by action

The response structure of delete is the same as add. clear has no Result and returns only Status.

Stage 3: final generation (CreateAigcVideoTask, SceneType=multi_elements) โ€‹

Trigger the multi-element editing scene with SceneType=multi_elements. Into ExtInfo's AdditionalParameters put the session_id and edit_mode created earlier, and in Prompt reference the edit target with <<<video_1>>>.

json
{
  "SubAppId": 123456789,
  "ModelName": "Kling",
  "ModelVersion": "1.6",
  "FileInfos": [
    {
      "Type": "Url",
      "Url": "https://<cdn>/ref.jpeg",
      "Usage": "Reference"
    }
  ],
  "Prompt": "Regenerate a specific subject in <<<video_1>>>",
  "OutputConfig": {
    "Duration": 10,
    "StorageMode": "Temporary"
  },
  "SceneType": "multi_elements",
  "ExtInfo": "{\"AdditionalParameters\":\"{\\\"session_id\\\":\\\"<session-id>\\\",\\\"edit_mode\\\":\\\"addition\\\"}\"}",
  "SessionContext": "job-001"
}
json
{
  "Response": {
    "TaskId": "<task-id>",
    "RequestId": "<request-id>"
  }
}

Poll the returned TaskId with DescribeTaskDetail following the task acceptance response procedure below.

Built with VitePress ยท Deployed on Tencent EdgeOne Pages