Skip to content

Media AI Agent API Reference โ€‹

Updated

Endpoint: https://smartmedia.vod-qcloud.com/aguiAuthentication: VOD ApiToken / Bearer

VOD defines the authentication, messaging, streaming response, and run management specifications of the Media AI Agent. It applies to both video production and media question answering.

Call Specification โ€‹

Basic Information โ€‹

ItemValue
Hostsmartmedia.vod-qcloud.com
AuthenticationAuthorization: Bearer $VOD_API_TOKEN
Request formatapplication/json
Run responseSSE / AG-UI events
Run / ResumePOST /agui/chat
Message historyPOST /agui/history
Cancel runPOST /agui/cancel

Preparing to Call โ€‹

Specify the SubAppId of the VOD application and issue an API key first. To issue a key, call CreateAigcApiToken of vod.intl.tencentcloudapi.com with a TC3 signature.

json
{
  "SubAppId": 123456789
}

Use the ApiToken from the response as the Bearer key for Agent requests. The key is bound to the selected application, and the VOD assets passed via fileId must reside in the same application. External assets can be passed as URLs.

For key issuance and management, see VOD API Key Issuance. Do not put the TC3 signature used for key issuance in the Agent request body.

API List and Request Parameters โ€‹

Request Parameters โ€‹

ParameterRequiredTypeDescription
threadIdRequiredStringConversation ID. Shares context within the same conversation
runIdRequiredStringUnique ID of the current run
messagesRequiredArrayUser messages or external tool results
forwardedPropsRequiredObjectScenario, model, and run options
toolsOptionalArrayExternal tools to be executed on the client
resumeOptionalArrayApproval / rejection results for tools awaiting approval

Run Options โ€‹

ParameterRequiredTypeDescription
forwardedProps.modelOptionalStringVega Agent model ID
forwardedProps.scenario_nameRequiredStringvideo-mixcut / video-qa
forwardedProps.approval_modeOptionalStringnever / level:high. Default never
forwardedProps.databaseOptionalStringKnowledge base name for video-qa. Default default

Messages โ€‹

ParameterRequiredTypeDescription
messages.roleRequiredStringUser input user / external tool result tool
messages.contentRequiredString / ArrayText or a ContentPart array
messages.toolCallIdConditionally requiredStringOriginal tool call ID when role: tool

Asset Input โ€‹

ParameterRequiredTypeDescription
typeRequiredStringtext / mixcut_assets
textConditionally requiredStringProduction instructions for type: text
metadataConditionally requiredObjectAsset information for type: mixcut_assets
metadata.attachmentsConditionally requiredArrayAsset list
metadata.attachments.urlConditionally requiredStringExternal asset URL. Mutually exclusive with fileId in the same item
metadata.attachments.fileIdConditionally requiredStringVOD file ID. Mutually exclusive with url in the same item

Responses and Run Flow โ€‹

Response Events โ€‹

EventDescription
RUN_STARTEDRun started
TEXT_MESSAGE_START / TEXT_MESSAGE_CONTENT / TEXT_MESSAGE_ENDAnswer message started / text appended / ended
REASONING_START / REASONING_ENDReasoning step started / ended
REASONING_MESSAGE_START / REASONING_MESSAGE_CONTENT / REASONING_MESSAGE_ENDReasoning message started / content appended / ended
TOOL_CALL_START / TOOL_CALL_ARGS / TOOL_CALL_ENDTool call started / arguments appended / call information delivery completed
TOOL_CALL_RESULTTool execution result
RUN_FINISHEDCurrent run ended, or awaiting approval / external tool results
RUN_ERRORRun error
MESSAGES_SNAPSHOTFull message snapshot from a history query

Event Data Example โ€‹

Parse the data field of the SSE as JSON. The following is the data of a run started event.

json
{
  "type": "RUN_STARTED",
  "timestamp": 1784102011155,
  "threadId": "thread-001",
  "runId": "run-001"
}

Text accumulates delta per messageId.

json
{
  "type": "TEXT_MESSAGE_CONTENT",
  "timestamp": 1784102019050,
  "messageId": "msg-002",
  "delta": "Preparing the edit structure based on the selected assets."
}

Tool arguments accumulate TOOL_CALL_ARGS.delta per toolCallId and are then parsed as JSON. TOOL_CALL_END marks the end of call information delivery, and the tool execution result is confirmed in TOOL_CALL_RESULT.

Run Status Classification โ€‹

Received StatusHandling
RUN_FINISHED + outcome.type: interruptDisplay the items awaiting approval and deliver the decision via resume
RUN_FINISHED after an external tool callExecute the tool on the client and deliver the result
RUN_FINISHED with no pending approval / external tool callCurrent run ended. Confirm the final output from the messages and tool results
RUN_ERRORRecord the error details and run identifier, and mark the failure status

History Query โ€‹

bash
curl -N -sS https://smartmedia.vod-qcloud.com/agui/history \
  -H "Authorization: Bearer $VOD_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "threadId": "thread-001",
    "runId": "history-001",
    "forwardedProps": {
      "scenario_name": "video-mixcut"
    }
  }'

Pass threadId and the runId of the query request. The response is SSE, and all messages of the conversation are included in the messages of the MESSAGES_SNAPSHOT event. On reconnection, restore the screen from this snapshot. The RUN_FINISHED in a history query response indicates the end of the query.

Cancelling a Run โ€‹

bash
curl -sS https://smartmedia.vod-qcloud.com/agui/cancel \
  -H "Authorization: Bearer $VOD_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "threadId": "thread-001",
    "runId": "cancel-001",
    "forwardedProps": {
      "scenario_name": "video-mixcut"
    }
  }'

Closing the browser or dropping the SSE connection alone does not cancel a run. To stop it, call the cancellation API.

Errors and Call Limits โ€‹

Check the HTTP status, the error code, and the request identifier in the response together. For authentication and permission errors, verify your configuration and retry the request; if a task or resource has already been created, query its current status before resubmitting. For per-API limits and error codes, see the related documents below.

Request Example โ€‹

Request Example โ€‹

bash
curl -N -sS https://smartmedia.vod-qcloud.com/agui/chat \
  -H "Authorization: Bearer $VOD_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @request.json

Save the JSON of a two-video concatenation request as request.json. For the full configuration, see Vega Agent Request Example.

Result and Resource Management โ€‹

Optional Features โ€‹

The video concatenation example runs without the configurations below. Use them when an approval step or integration with an external system is needed.

Approval and Resumption

approval_mode: level:high requests approval before executing a high-risk tool. The following is the data example of an event awaiting approval.

json
{
  "type": "RUN_FINISHED",
  "threadId": "thread-001",
  "runId": "run-001",
  "outcome": {
    "type": "interrupt",
    "interrupts": [
      {
        "id": "interrupt-001",
        "reason": "tool_call",
        "message": "Applying the edit plan and rendering the final video.",
        "toolCallId": "call-001",
        "responseSchema": {
          "type": "object",
          "properties": {
            "feedback": {
              "type": "string"
            }
          }
        }
      }
    ]
  }
}

Resume Parameters

ParameterRequiredTypeDescription
resume.interruptIdRequiredStringThe interrupts.id of the approval request
resume.statusRequiredStringApprove resolved / reject cancelled
resume.payloadOptionalObjectAdditional information about the decision
resume.payload.feedbackOptionalStringModification request or reason for rejection

Resume with a new runId within the same threadId. Keep the approval_mode and scenario_name from the initial request, and specify each pending approval ID only once.

json
{
  "threadId": "thread-001",
  "runId": "run-002",
  "messages": [
    {
      "role": "user",
      "content": ""
    }
  ],
  "forwardedProps": {
    "model": "wand-vega-agent-1.0-pro",
    "scenario_name": "video-mixcut",
    "approval_mode": "level:high"
  },
  "resume": [
    {
      "interruptId": "interrupt-001",
      "status": "resolved"
    }
  ]
}

External Tools

External tools are functions whose invocation the Agent decides and the client executes. Register the name, description, and JSON Schema in tools.

json
{
  "name": "lookup_product",
  "description": "Looks up the approved product description by product code.",
  "parameters": {
    "type": "object",
    "properties": {
      "productCode": {
        "type": "string"
      }
    },
    "required": [
      "productCode"
    ]
  }
}

Put this object into the tools array of the request. When you receive a tool call, validate the input of the registered function and execute it, then deliver the result in the format below. Keep the same threadId, tool definitions, and scenario.

json
{
  "threadId": "thread-001",
  "runId": "run-003",
  "messages": [
    {
      "role": "tool",
      "toolCallId": "call-ext-001",
      "content": "Product code P100: portable wireless speaker. Color: black."
    }
  ],
  "forwardedProps": {
    "model": "wand-vega-agent-1.0-pro",
    "scenario_name": "video-mixcut",
    "approval_mode": "never"
  },
  "tools": [
    {
      "name": "lookup_product",
      "description": "Looks up the approved product description by product code.",
      "parameters": {
        "type": "object",
        "properties": {
          "productCode": {
            "type": "string"
          }
        },
        "required": [
          "productCode"
        ]
      }
    }
  ]
}

Waiting for external tool results is also delivered as RUN_FINISHED without outcome. Check at the same time whether any unhandled external tool calls remain.

Built with VitePress ยท Deployed on Tencent EdgeOne Pages