Mikshi Video Understanding API

Use the Mikshi Video Understanding API to extract information from your videos and make it available to your applications. The API is organized around REST and returns responses in JSON format. It is compatible with most programming languages, and you can use one of the available SDKs, Postman, or other REST clients to interact with the API.

If you are using Python, see the Python SDK reference instead.

Base URL

https://<your-host>

Authentication

Every request must include your developer API key in the X-API-Key header.

X-API-Key: sk_AnFZjh95_...

Requests without a valid key return 401 Unauthorized. Never embed keys in client-side code that you ship to end users — proxy through your own backend, or mint short-lived keys per session.

Content type

Unless noted otherwise (multipart/form-data on direct upload, raw bytes on S3 PUT), request and response bodies are JSON:

Content-Type: application/json
Accept: application/json

Errors

Failures return a non-2xx status with a JSON body. The HTTP status code determines the error class:

Status	Meaning
400	Bad request — invalid input or business-rule violation
401	Missing or invalid API key
403	Authenticated but not allowed to access the resource
404	Resource not found
409	Conflict — e.g. duplicate name, video already in collection, pipeline pre-condition not met
413	Payload too large — direct upload exceeded the size limit
422	Validation error
429	Rate limited — honor the `Retry-After` header
500	Internal server error
502	Bad gateway — upstream service returned an invalid response
503	Service unavailable — auth service or background queue offline

Error body shape — depending on the deployment's exception handler, the code/message lives in either message or detail:

json

{ "detail": "human-readable message or short code" }

json

{ "message": "human-readable message or short code" }

The dev API uses message (custom handler); FastAPI's default uses detail. Clients should read both — e.g. body.message ?? body.detail. For YouTube imports specifically, this field carries a short code (e.g. YOUTUBE_VIDEO_PRIVATE, YOUTUBE_VIDEO_AGE_RESTRICTED) you can branch on.

Uploads

Three ways to ingest a video:

Multipart upload — chunked S3 upload for files of any size. Recommended for anything non-trivial.
Direct upload — single request, files up to 100 MB.
YouTube import — server pulls from a YouTube URL.

Multipart flow

Five calls. The actual chunk bytes are PUT directly to S3, not to this API.

1. POST /api/backend/v1/developer/uploads/multipart/initiate          → upload_id, s3_object_name, video_id
2. POST /api/backend/v1/developer/uploads/multipart/part-url          → pre-signed S3 URL (per chunk)
   PUT  <pre-signed url>                                   → S3 returns ETag header
3. POST /api/backend/v1/developer/uploads/multipart/complete          → finalize on S3
4. POST /api/backend/v1/developer/uploads/preprocess                  → create the video record + start processing
5. GET  /api/backend/v1/developer/uploads/status/{video_id}           → poll until processing finishes

Initiate

POST/api/backend/v1/developer/uploads/multipart/initiate

Body

json

{ "filename": "movie.mp4", "content_type": "video/mp4" }

Response

json

{
  "upload_id": "abc...",
  "s3_object_name": "uploads/...",
  "video_id": "c2a1..."
}

Stash all three — every subsequent call needs them.

Get a part URL

POST/api/backend/v1/developer/uploads/multipart/part-url

Body

json

{
  "s3_object_name": "uploads/...",
  "upload_id": "abc...",
  "part_number": 1
}

part_number must be in the range 1–10000.

Response

json

{ "url": "https://s3.../?...", "part_number": 1 }

PUT the chunk bytes directly to that URL. Capture ETag from the S3 response headers — you need it for complete.

List parts (optional, for resume)

POST/api/backend/v1/developer/uploads/multipart/list-parts

Body

json

{ "s3_object_name": "uploads/...", "upload_id": "abc..." }

Response

json

{ "parts": [{ "part_number": 1, "etag": "\"abc\"" }, "..."] }

Complete

POST/api/backend/v1/developer/uploads/multipart/complete

Body

json

{
  "s3_object_name": "uploads/...",
  "upload_id": "abc...",
  "parts": [
    { "part_number": 1, "etag": "\"abc\"" },
    { "part_number": 2, "etag": "\"def\"" }
  ]
}

Abort

POST/api/backend/v1/developer/uploads/multipart/abort

Body

json

{ "s3_object_name": "uploads/...", "upload_id": "abc..." }

Call this if the upload fails partway through to avoid orphaned S3 parts.

Preprocess (kick off processing)

POST/api/backend/v1/developer/uploads/preprocess

Body

json

{
  "s3_object_name": "uploads/...",
  "video_id": "c2a1...",
  "name": "My Movie",
  "use_dynamic_hls": false
}

Response — 202 Accepted:

json

{ "video_id": "c2a1..." }

Poll GET /api/backend/v1/developer/uploads/status/{video_id} to track per-pipeline progress.

Errors: 401, 409 ({"detail": "Video already exists."}), 503 (Kafka publish failed; the DB row is rolled back).

Direct upload

POST/api/backend/v1/developer/uploads/direct

multipart/form-data body:

Field	Type	Description
`file`	binary	Required. The video file.
`name`	string	Optional. Display name; falls back to the filename.
`use_dynamic_hls`	boolean	Optional. Default `false`.

Cap is configured server-side per env (settings.developer_direct_upload_max_bytes). Returns 413 Payload Too Large if exceeded — switch to multipart.

Response — 202 Accepted:

json

{ "video_id": "c2a1..." }

Errors: 400 ({"detail": "Uploaded file is empty."}), 401, 413, 500, 503.

YouTube import

POST/api/backend/v1/developer/uploads/youtube

Body

Field	Type	Required	Notes
`url`	string	yes	YouTube URL.
`name`	string	no	Display name; falls back to the YouTube title.
`use_dynamic_hls`	boolean	no	Default `false`.

Response — 202 Accepted:

json

{ "video_id": "c2a1..." }

On 400, the body carries one of these short codes. The field is message on the dev API (custom exception handler) and detail on deployments using FastAPI's default — clients should check both:

YOUTUBE_VIDEO_UNAVAILABLE
YOUTUBE_VIDEO_PRIVATE
YOUTUBE_VIDEO_AGE_RESTRICTED
YOUTUBE_VIDEO_GEO_BLOCKED
YOUTUBE_VIDEO_BLOCKED            (copyright)
YOUTUBE_AUTH_REQUIRED
YOUTUBE_AUTH_EXPIRED
YOUTUBE_AUTH_MISSING
YOUTUBE_AUTH_INVALID
YOUTUBE_IMPORT_FAILED            (catch-all)

Example body: {"message": "YOUTUBE_VIDEO_PRIVATE"} or {"detail": "YOUTUBE_VIDEO_PRIVATE"}.

503 means Kafka publish failed (DB row rolled back; the S3 object remains).

Upload status

GET/api/backend/v1/developer/uploads/status/{video_id}

Response — 200 OK:

json

{
  "video_id": "c2a1...",
  "thumbnail_generation_status": { "status": "completed",   "percentage": 100, "error": null },
  "transcoding_status":          { "status": "in_progress", "percentage": 42,  "error": null },
  "hls_generation_status":       { "status": "pending",     "percentage": 0,   "error": null },
  "error_message": null
}

Per-pipeline status is one of pending, in_progress, completed, failed. There is no top-level overall status — derive it from the three pipelines (e.g. all completed ⇒ done, any failed ⇒ failed).

Errors: 401, 403 ({"detail": "Not authorized for this video"}), 404 ({"detail": "Video not found"}).

Videos

Endpoints for managing already-uploaded videos and running processing pipelines on them.

GET/api/backend/v1/developer/videos/list

Query parameters (all optional)

Param	Type	Description
`status`	string	One of `uploaded`, `analysed`, `search_processed`, `processed`. `processed` requires both pipelines complete; the middle two filter on a single pipeline.
`start_time`	ISO-8601	Lower bound on `created_at`.
`end_time`	ISO-8601	Upper bound on `created_at`.
`not_in_collection`	bool	If `true`, only return videos not in any collection.
`skip`	int	Offset.
`limit`	int	Page size.
`sort_by`	string	Field to sort by.
`sort_order`	string	`asc` or `desc`.

Response — array of VideoListItem:

json

[
  {
    "video_id": "c2a1...",
    "name": "Warehouse cam 3 - 2026-04-12",
    "status": "processed",
    "thumbnail_path": "https://...",
    "hls_playlist_path": "https://.../hls-playlist?video_id=...&playlist=master.m3u8",
    "transcoding_status": "completed",
    "hls_generation_status": "completed",
    "thumbnail_generation_status": "completed",
    "analyse_status": "done",
    "search_process_status": "done",
    "created_at": "2026-04-12T10:21:33Z"
  }
]

Get a video

GET/api/backend/v1/developer/videos/get-video-details?video_id=<uuid>

Response — VideoDetails, including video_metadata:

json

{
  "video_id": "c2a1...",
  "name": "...",
  "video_path": "https://...",
  "thumbnail_path": "https://...",
  "hls_playlist_path": "https://...",
  "transcoding_status": "completed",
  "hls_generation_status": "completed",
  "thumbnail_generation_status": "completed",
  "video_metadata": { "duration_seconds": 312.5, "width": 1920, "height": 1080 },
  "created_at": "2026-04-12T10:21:33Z"
}

Delete a video

DELETE/api/backend/v1/developer/videos/delete?video_id=<uuid>

Permanent. Removes the source object and all derived S3 assets (thumbnails, HLS segments, etc.).

Pipelines

Two optional processing pipelines that run on top of an uploaded video. Each has the same shape: initiate to enqueue, status to poll.

Pre-condition: all */initiate endpoints return 409 Conflict if transcoding_status != "completed". Wait for upload processing to finish before kicking these off.

Search-process

POST/api/backend/v1/developer/videos/search-process/initiate

GET/api/backend/v1/developer/videos/search-process/status/{video_id}

Analyse

POST/api/backend/v1/developer/videos/analyse/initiate

GET/api/backend/v1/developer/videos/analyse/status/{video_id}

Common shapes

POST .../initiate body:

json

{ "video_id": "c2a1..." }

Initiate response:

json

{ "video_id": "c2a1...", "status": "enqueued" }

Status response:

json

{
  "video_id": "c2a1...",
  "status": "running",
  "percentage": 42,
  "error": null
}

status is one of pending, running, done, failed. percentage and error are populated on a Redis hit; on the database fallback only status is set (percentage will be null).

Collections

Named groups of fully-processed videos. Always scoped to the authenticated user — no team/group sharing.

List collections

GET/api/backend/v1/developer/collections/list

Cached for 300s per user.

Query parameters

Param	Type	Default	Constraint
`page`	int	`1`	`>= 1`
`page_size`	int	`10`	`1`–`100`

Response — 200 OK — PaginatedCollectionResponse:

json

{
  "data": [
    {
      "collection_id": "uuid",
      "parent_id": "uuid",
      "parent_type": "user",
      "name": "Warehouse incidents",
      "description": "Flagged moments from warehouse cams",
      "created_at": "2026-04-12T10:21:33Z",
      "video_count": 12,
      "thumbnails": ["https://.../thumb1.jpg"]
    }
  ],
  "page": 1,
  "page_size": 10,
  "total": 47
}

thumbnails is up to four signed URLs.

Errors: 401, 422 (invalid page / page_size).

Create a collection

POST/api/backend/v1/developer/collections/create

Body

Field	Type	Required	Notes
`name`	string	yes	1-255 chars.
`description`	string	no

json

{
  "name": "Warehouse incidents",
  "description": "Flagged moments from warehouse cams"
}

Response — 200 / 201 OK — CollectionResponse:

json

{
  "collection_id": "uuid",
  "parent_id": "uuid",
  "parent_type": "user",
  "name": "Warehouse incidents",
  "description": "Flagged moments from warehouse cams",
  "created_at": "2026-05-08T...",
  "video_count": 0,
  "thumbnails": null
}

Errors: 401, 409 ({"detail": "A collection with this name already exists."}), 422.

Update a collection

PATCH/api/backend/v1/developer/collections/update/{collection_id}

Body — all fields optional; pass only what's changing. Server returns 400 if the body has nothing to change.

json

{ "name": "Warehouse - April", "description": "..." }

Response — 200 OK — same shape as CollectionResponse with refreshed video_count and thumbnails.

Errors: 400 ({"detail": "No fields provided to update"}), 401, 403 ({"detail": "Not authorized for this collection"}), 404, 409.

Delete a collection

DELETE/api/backend/v1/developer/collections/delete/{collection_id}

Removes the collection and its video links — the videos themselves stay intact.

Response — 200 OK:

json

{ "message": "Collection deleted successfully." }

Errors: 401, 403, 404, 500 ({"detail": "Failed to delete collection"}).

GET/api/backend/v1/developer/collections/{collection_id}/videos

Paginated and sortable. Collection videos are always processed by add-time constraint, so there's no status_totals here.

Query parameters

Param	Type	Default	Notes
`page`	int	`1`	`>= 1`.
`page_size`	int	`10`	`1`–`100`.
`sort_by`	string	`created_at`	e.g. `created_at`, `name`.
`sort_order`	string	`desc`	`asc` or `desc`.

Response — 200 OK — PaginatedCollectionVideosResponse:

json

{
  "data": [
    {
      "video_id": "uuid",
      "name": "incident-20260423",
      "parent_id": "uuid",
      "parent_type": "user",
      "status": "processed",
      "video_path": "https://...signed...",
      "thumbnail_path": "https://...signed...",
      "hls_playlist_path": "https://...proxy...",
      "created_at": "...",
      "thumbnail_generation_status": "completed",
      "transcoding_status": "completed",
      "hls_generation_status": "completed"
    }
  ],
  "page": 1,
  "page_size": 10,
  "total": 12
}

Errors: 401, 403, 404.

Add a video to a collection

POST/api/backend/v1/developer/collections/{collection_id}/videos

Body

json

{ "video_id": "uuid" }

Constraints (server-enforced):

The video must be fully processed: analyse_status == "DONE" AND search_process_status == "DONE".
The video and the collection must share the same parent (your user).

Response — 201 Created:

json

{ "message": "Video added to collection." }

Errors:

400 — {"detail": "Only fully processed videos can be added to a collection."} or {"detail": "Video and collection must share the same parent (user)."}
401
403 — collection isn't yours
404 — {"detail": "Collection not found"} or {"detail": "Video not found"}
409 — {"detail": "Video is already in this collection."}

Remove a video from a collection

DELETE/api/backend/v1/developer/collections/{collection_id}/videos/{video_id}

Removes the link only — the video record stays intact and can be re-added later.

Response — 200 OK:

json

{ "message": "Video removed from collection." }

Errors: 401, 403, 404 ({"detail": "Collection not found"} or {"detail": "Video is not in this collection."}).

Chat

Natural-language Q&A over a video. Backed by the chat service's retrieval + rerank + LLM pipeline.

Ask

POST/api/chat/v1/developer/chat/ask

Body

Field	Type	Required	Notes
`question`	string	yes	The user's question.
`video_id`	string (UUID)	yes	Target video. Must be owned by the API-key user.
`top_k`	int	no	Retriever top-k. Defaults to the server's `default_k` (5).
`history`	list of `{role, content}`	no	OpenAI-style chat history (`role` is `"user"` or `"assistant"`). The server is stateless — resend prior turns each call.
`stream`	bool	no	Set to `true` for Server-Sent Events streaming. Default `false`.

Minimal:

json

{
  "question": "Describe what happens in this video.",
  "video_id": "a249266f-052d-4148-bc2f-3efa76dc4269"
}

Multi-turn:

json

{
  "question": "What was my first question?",
  "video_id": "a249266f-052d-4148-bc2f-3efa76dc4269",
  "history": [
    { "role": "user",      "content": "Describe what happens in this video." },
    { "role": "assistant", "content": "A man in a striped shirt is approached by two individuals on a motorcycle..." },
    { "role": "user",      "content": "How many people are involved?" },
    { "role": "assistant", "content": "There are 4 people involved: ..." }
  ]
}

Response (non-streaming) — 200 OK:

json

{ "answer": "A pedestrian crosses the road while a car waits at the signal..." }

Response (streaming, stream: true) — 200 OK, Content-Type: text/event-stream. The server emits one token event per LLM chunk, then exactly one terminal event (done on success, error on failure):

event: token
data: {"content": "A "}

event: token
data: {"content": "man in a striped shirt "}

event: token
data: {"content": "is approached..."}

event: done
data: {"answer": "A man in a striped shirt is approached..."}

On mid-stream failure the server emits:

event: error
data: {"detail": "<short message>"}

…and the credit is refunded automatically. Always check for the terminal event — the stream closes after done or error.

Side effects: 1 credit deducted from the API-key owner via the auth-service. If the pipeline crashes (5xx) — or emits a streaming error event — the server refunds the credit before signalling failure.

Errors

Status	When
401	Missing/invalid `X-API-Key`, key revoked, or auth-service payload missing/invalid `user_id`.
402	Insufficient credits (raised by auth-service `deduct_credits`).
403	The video isn't owned by the API-key user (`videos.parent_id != api_key_user_id`). Returned before credit deduction.
422	Pydantic validation — invalid UUID or missing required field. No credit charged.
500	Pipeline failure (retrieval / rerank / LLM crash). The deducted credit is refunded automatically.
502 / 503	Auth-service unreachable when calling `deduct_credits`. Treat as transient and retry.

Search

Two endpoints power vector search and visualization over segment embeddings. Both run against videos that have completed the analyse and search-process pipelines.

Ownership: the caller must own every video referenced (directly via video_ids or transitively through collection_id). Unowned IDs return 403 Forbidden.

Vector search

POST/api/chat/v1/developer/search

Vector-search across the developer's own videos. Resolves segments via Milvus, clusters near-duplicates (top 3 per cluster), then returns top_k ranked hits. Costs 1 credit per call (refunded on failure).

Body — must provide at least one of video_ids or collection_id:

Field	Type	Notes
`query`	string	Required.
`video_ids`	list of UUIDs	Defaults to `[]`.
`collection_id`	UUID \| null	Optional.
`top_k`	int	Default `5`.

json

{
  "query": "person walking near the gate",
  "video_ids": ["b2f1a3c4-...-1111", "b2f1a3c4-...-2222"],
  "collection_id": null,
  "top_k": 5
}

Response — 200 OK:

json

{
  "results": [
    {
      "rank": 1,
      "start": "00:00:12",
      "end": "00:00:18",
      "video_id": "b2f1a3c4-...-1111",
      "thumbnail_url": "https://signed-storage/.../thumb.jpg?sig=...",
      "hls_playlist_url": "https://<host>/hls/.../master.m3u8",
      "video_url": "https://signed-storage/.../video.mp4?sig=..."
    }
  ]
}

Errors

Status	When
400	Malformed `video_id` UUID.
401	Missing/invalid `X-API-Key`, or auth-service payload missing `user_id`.
402 / 400	Insufficient credits.
403	Caller doesn't own one of the videos: `Video(s) not owned by user: <ids>`.
404	No videos resolved (empty collection + no `video_ids`).
422	Pydantic validation (e.g. neither `video_ids` nor `collection_id`).
502 / 503	Auth service returned a non-JSON response / unreachable.

Visualize

POST/api/chat/v1/developer/visualize

Returns a 2-D UMAP projection (cosine metric, MinMax-scaled to [0, 1]) of every segment embedding across the requested videos, plus per-video color/HLS/signed URLs and per-segment thumbnails. Free (no credit deduction).

Body — must provide at least one of video_ids or collection_id:

Field	Type	Notes
`video_ids`	list of UUIDs	Defaults to `[]`.
`collection_id`	UUID \| null	Optional.

json

{
  "video_ids": ["b2f1a3c4-...-1111"],
  "collection_id": null
}

Response — 200 OK:

json

{
  "videos": {
    "b2f1a3c4-...-1111": {
      "video_id": "b2f1a3c4-...-1111",
      "color": "hsl(217, 70%, 55%)",
      "hls_playlist_url": "https://<host>/hls/.../master.m3u8",
      "video_url": "https://signed-storage/.../video.mp4?sig=..."
    }
  },
  "points": [
    {
      "segment_id": "5e7c...-aaaa",
      "video_id": "b2f1a3c4-...-1111",
      "x": 0.4123,
      "y": 0.7841,
      "start": "00:00:00",
      "end": "00:00:06",
      "thumbnail_url": "https://signed-storage/.../thumb.jpg?sig=..."
    }
  ]
}

videos is keyed by stringified video_id. x and y are normalized to [0, 1].

Errors

Status	When
400	Malformed `video_id` UUID, or fewer than 2 segments across the selection (`Need at least 2 segments across the selected videos for projection.`).
401	Same as search.
403	Caller doesn't own one of the videos.
404	No videos resolved: `No videos to visualize.`
422	Pydantic validation.
502 / 503	Auth service issues.

Embeddings

Two endpoints expose the raw embedding vectors used by search and chat: one returns the stored per-segment vectors for a video (read from the vector store, no recompute), the other embeds an arbitrary text string with the active backend.

Video embeddings

GET/api/chat/v1/developer/embeddings/video/{video_id}

Returns every stored segment embedding for a video. Reads directly from the vector store — no embedder invocation, no recompute. Costs 1 credit per call (refunded on 404 / 408 / 502 / 5xx).

Path parameter

Name	Type	Required	Notes
`video_id`	string (UUID)	yes	Must be a valid UUID; caller must own the video.

No request body.

Response — 200 OK:

json

{
  "id": "a249266f-052d-4148-bc2f-3efa76dc4269",
  "status": "ready",
  "data": [
    {
      "embedding": [0.0123, -0.0456, "..."],
      "embedding_option": "visual",
      "embedding_scope": "clip",
      "start": "00:00:00",
      "end": "00:00:05"
    }
  ]
}

start / end are HH:MM:SS strings. embedding is a list of floats (1024-dim with the current backend); embedding_option and embedding_scope describe how the vector was produced.

Errors

Status	When
400	`video_id` is not a valid UUID.
401	Missing/invalid API key.
403 → 404	Video not owned by the caller (returned as 404 to prevent enumeration).
404	Video not found, or no segments stored. Credit refunded.
408	Vector-store timeout. Credit refunded.
409	Video's `search_process_status != "done"` — pipeline still running. Wait and retry.
500	Vector store not initialized server-side.
502	Vector store unreachable. Credit refunded.

Text embeddings

POST/api/chat/v1/developer/embeddings/text

Embeds an arbitrary text string with the active backend (EMBED_BACKEND selects between RunPod and Gemini server-side). Costs 1 credit per call (refunded on 408 / 502 / 5xx).

Body

Field	Type	Required	Notes
`text`	string	yes	1–4096 characters.

json

{ "text": "chain wearing man" }

Response — 200 OK:

json

{
  "status": "ready",
  "data": [
    { "embedding": [0.0123, -0.0456, "..."] }
  ]
}

Errors

Status	When
401	Missing/invalid API key.
408	Embedder timeout. Credit refunded.
422	`text` empty or > 4096 characters.
500	Embedder not initialized server-side.
502	Embedder unreachable. Credit refunded.

HLS playback

Once a video's hls_generation_status is completed, play it back via:

GET /api/backend/v1/videos/hls-playlist?video_id=<uuid>&playlist=master.m3u8
X-API-Key: <your developer key>

The proxy is auth-gated. Every manifest and segment fetch the player makes must carry the X-API-Key header. How you wire that depends on the player:

hls.js (web) — set a custom xhrSetup that adds the header.
Video.js / Shaka Player — both expose request-interceptor hooks for the same purpose.
Native iOS (AVPlayer) / Android (ExoPlayer) — use the platform's HTTP-header injection on the media source.

Don't embed the developer key in client-side code. Either proxy through your own backend, or mint short-lived per-session keys.

Recipes

End-to-end workflows shown with curl. Replace $API_KEY and $BASE_URL with your values before running:

bash

export API_KEY="sk_..."
export BASE_URL="https://api.your-host.com"

1. Direct upload (≤ 100 MB) and poll

bash

# 1. upload
RESPONSE=$(curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/direct" \
  -H "X-API-Key: $API_KEY" \
  -F "file=@clip.mp4" \
  -F "name=My Clip")

VIDEO_ID=$(echo "$RESPONSE" | jq -r .video_id)
echo "video_id=$VIDEO_ID"

# 2. poll per-pipeline status; derive a single overall state
while true; do
  RESPONSE=$(curl -sS "$BASE_URL/api/backend/v1/developer/uploads/status/$VIDEO_ID" \
    -H "X-API-Key: $API_KEY")
  STATUSES=$(echo "$RESPONSE" | jq -r '
    [.thumbnail_generation_status.status,
     .transcoding_status.status,
     .hls_generation_status.status] | @tsv
  ')
  echo "  $STATUSES"
  echo "$STATUSES" | grep -q failed && { echo "[FAIL]"; exit 1; }
  echo "$STATUSES" | grep -qE 'pending|in_progress' || break  # all completed
  sleep 2
done

# 3. fetch the playable record
curl -sS "$BASE_URL/api/backend/v1/developer/videos/get-video-details?video_id=$VIDEO_ID" \
  -H "X-API-Key: $API_KEY" | jq .hls_playlist_path

2. End-to-end multipart upload (any size)

For files larger than 100 MB you must use multipart. The body of each chunk is PUT directly to S3 with the pre-signed URL — those PUTs do not carry your API key.

bash

FILE="big-movie.mp4"
NAME="Big Movie"
CHUNK_SIZE=$((10 * 1024 * 1024))   # 10 MB

# 1. initiate
INIT=$(curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/multipart/initiate" \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"filename\":\"$FILE\",\"content_type\":\"video/mp4\"}")

UPLOAD_ID=$(echo "$INIT" | jq -r .upload_id)
S3_OBJECT=$(echo "$INIT" | jq -r .s3_object_name)
VIDEO_ID=$(echo "$INIT" | jq -r .video_id)

# 2. chunk + PUT each part
PARTS="[]"
PART_NUMBER=1
SPLIT_DIR=$(mktemp -d)
split -b $CHUNK_SIZE "$FILE" "$SPLIT_DIR/part_"

for CHUNK in "$SPLIT_DIR"/part_*; do
  URL=$(curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/multipart/part-url" \
    -H "X-API-Key: $API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"s3_object_name\":\"$S3_OBJECT\",\"upload_id\":\"$UPLOAD_ID\",\"part_number\":$PART_NUMBER}" \
    | jq -r .url)

  ETAG=$(curl -sS -D - -X PUT --data-binary "@$CHUNK" "$URL" \
    | grep -i '^etag:' | awk '{print $2}' | tr -d '\r"')

  PARTS=$(echo "$PARTS" | jq ". + [{\"part_number\":$PART_NUMBER,\"etag\":\"$ETAG\"}]")
  PART_NUMBER=$((PART_NUMBER + 1))
done

# 3. complete + 4. preprocess
curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/multipart/complete" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{\"s3_object_name\":\"$S3_OBJECT\",\"upload_id\":\"$UPLOAD_ID\",\"parts\":$PARTS}"

curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/preprocess" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{\"s3_object_name\":\"$S3_OBJECT\",\"video_id\":\"$VIDEO_ID\",\"name\":\"$NAME\"}"

rm -rf "$SPLIT_DIR"

If anything fails between initiate and complete, call abort to release the partial S3 object:

bash

curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/multipart/abort" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{\"s3_object_name\":\"$S3_OBJECT\",\"upload_id\":\"$UPLOAD_ID\"}"

3. Resume a multipart upload

If your client crashed mid-upload but you persisted upload_id and s3_object_name, ask the server which parts already landed and only PUT the missing ones.

bash

ALREADY=$(curl -sS -X POST "$BASE_URL/api/backend/v1/developer/uploads/multipart/list-parts" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{\"s3_object_name\":\"$S3_OBJECT\",\"upload_id\":\"$UPLOAD_ID\"}")

# Extract part numbers already uploaded
echo "$ALREADY" | jq '.parts[].part_number'

Skip those part_numbers in the upload loop, then call complete with the union of ALREADY.parts and the parts you just uploaded.

4. YouTube import and poll

The server pulls the video from a YouTube URL and processes it like any other upload. On failure the error code lives in message (dev API) or detail (FastAPI default) — read both. See the YouTube import section above for the full list of fatal codes.

bash

RESPONSE=$(curl -sS -w "\n%{http_code}" -X POST "$BASE_URL/api/backend/v1/developer/uploads/youtube" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d '{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ","name":"My Clip"}')

STATUS=$(echo "$RESPONSE" | tail -n1)
BODY=$(echo "$RESPONSE" | sed '$d')

if [ "$STATUS" != "202" ]; then
  CODE=$(echo "$BODY" | jq -r '.message // .detail')
  echo "[FAIL] $STATUS: $CODE"
  exit 1
fi

VIDEO_ID=$(echo "$BODY" | jq -r .video_id)
echo "video_id=$VIDEO_ID"

# Poll until processing finishes — same loop shape as Recipe 1.
while true; do
  RESPONSE=$(curl -sS "$BASE_URL/api/backend/v1/developer/uploads/status/$VIDEO_ID" \
    -H "X-API-Key: $API_KEY")
  STATUSES=$(echo "$RESPONSE" | jq -r '
    [.thumbnail_generation_status.status,
     .transcoding_status.status,
     .hls_generation_status.status] | @tsv
  ')
  echo "  $STATUSES"
  echo "$STATUSES" | grep -q failed && { echo "[FAIL]"; exit 1; }
  echo "$STATUSES" | grep -qE 'pending|in_progress' || break  # all completed
  sleep 2
done

If you also want to react to specific error codes (skip private/age-restricted videos in a batch), see the YouTube error-branching pattern in Recipe 10.

5. Run a processing pipeline

The two pipelines (analyse, search-process) are interchangeable on the wire — substitute the path segment.

bash

# 1. enqueue
curl -sS -X POST "$BASE_URL/api/backend/v1/developer/videos/analyse/initiate" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{\"video_id\":\"$VIDEO_ID\"}"

# 2. poll
while true; do
  RESPONSE=$(curl -sS "$BASE_URL/api/backend/v1/developer/videos/analyse/status/$VIDEO_ID" \
    -H "X-API-Key: $API_KEY")
  STATUS=$(echo "$RESPONSE" | jq -r .status)
  PCT=$(echo "$RESPONSE" | jq -r '.percentage // "?"')
  echo "  $STATUS ($PCT%)"
  [ "$STATUS" = "done" ] && break
  [ "$STATUS" = "failed" ] && exit 1
  sleep 2
done

Initiate returns 409 Conflict if transcoding_status != "completed" — wait for upload processing first.

6. Build a collection

bash

# 1. create the collection
COLLECTION_ID=$(curl -sS -X POST "$BASE_URL/api/backend/v1/developer/collections/create" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d '{"name":"Warehouse incidents","description":"Flagged moments"}' \
  | jq -r .collection_id)

# 2. add fully-processed videos (must have analyse_status and search_process_status both DONE)
for VIDEO_ID in "vid-1" "vid-2" "vid-3"; do
  curl -sS -X POST "$BASE_URL/api/backend/v1/developer/collections/$COLLECTION_ID/videos" \
    -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
    -d "{\"video_id\":\"$VIDEO_ID\"}"
done

# 3. paginate the collection's videos (each carries pre-signed URLs)
curl -sS "$BASE_URL/api/backend/v1/developer/collections/$COLLECTION_ID/videos?page=1&page_size=50" \
  -H "X-API-Key: $API_KEY" \
  | jq '.data[] | {video_id, hls_playlist_path}'

add_video returns 400 if the video isn't fully processed yet, and 409 if it's already in the collection — wait for both analyse_status and search_process_status to be DONE before adding.

7. Ask (single-turn and multi-turn)

bash

# Single-turn
curl -sS -X POST "$BASE_URL/api/chat/v1/developer/chat/ask" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{
    \"question\": \"Describe what happens in this video.\",
    \"video_id\": \"$VIDEO_ID\"
  }" | jq -r .answer

# Multi-turn — caller owns history; resend prior turns each call.
# History is OpenAI-style {role, content} messages.
HISTORY='[
  {"role":"user","content":"Describe what happens in this video."},
  {"role":"assistant","content":"A man in a striped shirt is approached by two individuals on a motorcycle..."}
]'

curl -sS -X POST "$BASE_URL/api/chat/v1/developer/chat/ask" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{
    \"question\": \"How many people are involved?\",
    \"video_id\": \"$VIDEO_ID\",
    \"history\": $HISTORY
  }" | jq -r .answer

# Streaming (SSE) — set "stream": true. Use curl -N to disable buffering.
curl -sSN -X POST "$BASE_URL/api/chat/v1/developer/chat/ask" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{
    \"question\": \"How many people are involved?\",
    \"video_id\": \"$VIDEO_ID\",
    \"stream\": true
  }"
# Emits a sequence of `event: token` lines followed by exactly one
# `event: done` (success, with full answer) or `event: error` (failure).

Each call costs 1 credit (refunded on 5xx, or on streaming error events). The server is stateless — there's no session ID; rebuild history on the client.

8. Search and visualize

bash

# Vector search across two videos
curl -sS -X POST "$BASE_URL/api/chat/v1/developer/search" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{
    \"query\": \"person walking near the gate\",
    \"video_ids\": [\"$VIDEO_ID_1\", \"$VIDEO_ID_2\"],
    \"top_k\": 5
  }" | jq '.results[] | {rank, start, end, hls_playlist_url}'

# UMAP visualization across an entire collection
curl -sS -X POST "$BASE_URL/api/chat/v1/developer/visualize" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d "{
    \"collection_id\": \"$COLLECTION_ID\"
  }" | jq '{
    video_count: (.videos | length),
    point_count: (.points | length)
  }'

search costs 1 credit per call (refunded on failure); visualize is free. Both require the caller to own every referenced video.

9. Pagination

/collections/list is page-based; /videos/list uses skip/limit.

bash

# Walk all collections, page by page. /collections/list now returns
# {data, page, page_size, total} — stop once we've seen `total` items.
PAGE=1
SEEN=0
while true; do
  RESPONSE=$(curl -sS "$BASE_URL/api/backend/v1/developer/collections/list?page=$PAGE&page_size=100" \
    -H "X-API-Key: $API_KEY")
  COUNT=$(echo "$RESPONSE" | jq '.data | length')
  TOTAL=$(echo "$RESPONSE" | jq '.total')
  [ "$COUNT" = "0" ] && break
  echo "$RESPONSE" | jq -r '.data[] | .collection_id'
  SEEN=$((SEEN + COUNT))
  [ "$SEEN" -ge "$TOTAL" ] && break
  PAGE=$((PAGE + 1))
done

# Walk recent videos with filters.
SKIP=0
while true; do
  BATCH=$(curl -sS "$BASE_URL/api/backend/v1/developer/videos/list?status=processed&skip=$SKIP&limit=100&sort_by=created_at&sort_order=desc" \
    -H "X-API-Key: $API_KEY")
  COUNT=$(echo "$BATCH" | jq 'length')
  [ "$COUNT" = "0" ] && break
  echo "$BATCH" | jq -r '.[] | "\(.video_id) \(.name)"'
  [ "$COUNT" -lt 100 ] && break
  SKIP=$((SKIP + 100))
done

10. Handling rate limits and errors

The API returns 429 with a Retry-After header when you exceed your quota. A minimal retry loop:

bash

request() {
  local response status retry
  response=$(curl -sS -w "\n%{http_code}" "$@")
  status=$(echo "$response" | tail -n1)
  body=$(echo "$response" | sed '$d')

  if [ "$status" = "429" ]; then
    retry=$(curl -sSI "$@" | grep -i '^retry-after:' | awk '{print $2}' | tr -d '\r')
    sleep "${retry:-1}"
    request "$@"
    return
  fi
  echo "$body"
}

request -X POST "$BASE_URL/api/backend/v1/developer/collections/create" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d '{"name":"My collection"}'

For YouTube import (POST /api/backend/v1/developer/uploads/youtube), branch on the detail short code:

bash

RESPONSE=$(curl -sS -w "\n%{http_code}" -X POST "$BASE_URL/api/backend/v1/developer/uploads/youtube" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d '{"url":"https://www.youtube.com/watch?v=..."}')

STATUS=$(echo "$RESPONSE" | tail -n1)
BODY=$(echo "$RESPONSE" | sed '$d')

if [ "$STATUS" = "400" ]; then
  CODE=$(echo "$BODY" | jq -r '.message // .detail')
  case "$CODE" in
    YOUTUBE_VIDEO_PRIVATE)        echo "private — skipping" ;;
    YOUTUBE_VIDEO_AGE_RESTRICTED) echo "age-restricted — skipping" ;;
    *)                            echo "unknown error: $CODE"; exit 1 ;;
  esac
fi

Object reference

UploadStatus

json

{
  "video_id": "uuid",
  "thumbnail_generation_status": TaskStatus,
  "transcoding_status": TaskStatus,
  "hls_generation_status": TaskStatus,
  "error_message": "string | null"
}

TaskStatus:

json

{ "status": "pending | in_progress | completed | failed", "percentage": 0, "error": "string | null" }

Video

json

{
  "video_id": "uuid",
  "name": "string",
  "parent_id": "uuid | null",
  "parent_type": "string | null",
  "status": "string | null",
  "video_path": "url | null",
  "thumbnail_path": "url | null",
  "hls_playlist_path": "url | null",
  "transcoding_status": "string | null",
  "thumbnail_generation_status": "string | null",
  "hls_generation_status": "string | null",
  "created_at": "ISO-8601 | null"
}

Collection

json

{
  "collection_id": "uuid",
  "name": "string",
  "description": "string | null",
  "parent_id": "uuid | null",
  "parent_type": "string | null",
  "created_at": "ISO-8601",
  "video_count": 0,
  "thumbnails": ["url", "..."]
}

PaginatedCollections

json

{
  "data": [Collection, "..."],
  "page": 1,
  "page_size": 10,
  "total": 47
}

PaginatedCollectionVideos

json

{
  "data": [Video, "..."],
  "page": 1,
  "page_size": 10,
  "total": 12
}

AskResponse

json

{ "answer": "string" }

SearchHit

start / end are HH:MM:SS timestamp strings.

json

{
  "rank": 1,
  "start": "00:00:12",
  "end": "00:00:18",
  "video_id": "uuid",
  "thumbnail_url": "url | null",
  "hls_playlist_url": "url | null",
  "video_url": "url | null"
}

VideoEmbeddingsResponse

json

{
  "id": "uuid",
  "status": "ready",
  "data": [
    {
      "embedding": [0.0, 0.0, "..."],
      "embedding_option": "visual",
      "embedding_scope": "clip",
      "start": "HH:MM:SS",
      "end": "HH:MM:SS"
    }
  ]
}

TextEmbeddingResponse

json

{
  "status": "ready",
  "data": [
    { "embedding": [0.0, 0.0, "..."] }
  ]
}

Visualization

json

{
  "videos": {
    "<video_id>": {
      "video_id": "uuid",
      "color": "string | null",
      "hls_playlist_url": "url | null",
      "video_url": "url | null"
    }
  },
  "points": [
    {
      "segment_id": "string",
      "video_id": "uuid",
      "x": 0.0,
      "y": 0.0,
      "start": "HH:MM:SS",
      "end": "HH:MM:SS",
      "thumbnail_url": "url | null"
    }
  ]
}

Mikshi Video Understanding API

Base URL

Authentication

Content type

Errors

Uploads

Multipart flow

Initiate

Get a part URL

List parts (optional, for resume)

Complete

Abort

Preprocess (kick off processing)

Direct upload

YouTube import

Upload status

Videos

List videos

Get a video

Delete a video

Pipelines

Search-process

Analyse

Common shapes

Collections

List collections

Create a collection

Update a collection

Delete a collection

List videos in a collection

Add a video to a collection

Remove a video from a collection

Chat

Ask

Search

Vector search

Visualize

Embeddings

Video embeddings

Text embeddings

HLS playback

Recipes

1. Direct upload (≤ 100 MB) and poll

2. End-to-end multipart upload (any size)

3. Resume a multipart upload

4. YouTube import and poll

5. Run a processing pipeline

6. Build a collection

7. Ask (single-turn and multi-turn)

8. Search and visualize

9. Pagination

10. Handling rate limits and errors

Object reference

UploadStatus

Video

Collection

PaginatedCollections

PaginatedCollectionVideos

AskResponse

SearchHit

VideoEmbeddingsResponse

TextEmbeddingResponse

Visualization