Node.js SDK
The official Node.js SDK for the Video Platform Developer API. Wraps the HTTP API in a small ES-module client that returns plain JavaScript objects for every response.
If you are calling the API directly, see the HTTP API reference instead.
Install
From npm (recommended for application code):
npm install mikshii-js-sdk
yarn add mikshii-js-sdk
pnpm add mikshii-js-sdk
Then import it from your app:
import { Client } from 'mikshii-js-sdk';
const client = new Client({ apiKey: 'sk_...', baseUrl: 'https://vision.mikshi.ai/dev/' });
From a local checkout (e.g. while developing against an unreleased branch):
git clone <repo-url>
cd nodejs_sdk
npm install # install runtime dependencies
Then either install it into another project by path:
npm install /absolute/path/to/nodejs_sdk
…or import directly from src/index.js if you're hacking inside the repo:
import { Client } from './src/index.js';
Requires: Node.js 18+ (uses the global fetch-era FormData/Blob, ES modules, top-level await) and axios.
The SDK is shipped as an ES module — import { Client } from 'mikshii-js-sdk'. CommonJS (require) is not supported directly; use dynamic import() from a CJS file:
// my-cjs-file.cjs
(async () => {
const { Client } = await import('mikshii-js-sdk');
const client = new Client({ apiKey: 'sk_...' });
})();
Quick start
End-to-end: upload a video, wait for transcoding + HLS, run the analyse and search-process pipelines so the video is queryable, then run a vector search and ask a question about it.
import { Client, derivedUploadStatus, isPipelineTerminal, isPipelineFailed } from './src/index.js';
const client = new Client({
apiKey: 'sk_AnFZjh95_...',
baseUrl: 'https://vision.mikshi.ai/dev/',
});
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
try {
// 1. Upload — chunks the file, PUTs to S3, completes, and kicks off
// transcoding + HLS generation. Returns immediately with a video_id.
const job = await client.uploads.uploadFile('clip.mp4', { name: 'My Clip' });
console.log(`uploaded ${job.video_id}`);
// 2. Wait for upload processing (transcode + HLS) to finish.
while (true) {
const status = await client.uploads.getStatus(job.video_id);
const derived = derivedUploadStatus(status);
if (derived === 'completed') break;
if (derived === 'failed') throw new Error(status.error_message || 'upload processing failed');
await sleep(2000);
}
// 3. Run analyse + search-process. These two pipelines are what make the
// video searchable and chat-ready — search/visualize/ask all require them.
await client.videos.initiateAnalyse(job.video_id);
await client.videos.initiateSearchProcess(job.video_id);
while (true) {
const a = await client.videos.getAnalyseStatus(job.video_id);
const sp = await client.videos.getSearchProcessStatus(job.video_id);
if (isPipelineTerminal(a) && isPipelineTerminal(sp)) {
if (isPipelineFailed(a) || isPipelineFailed(sp)) {
throw new Error(`pipeline failed: analyse=${a.status}, search=${sp.status}`);
}
break;
}
await sleep(2000);
}
// 4. Vector search — ranked clip-level hits with pre-signed thumbnails and
// the auth-gated HLS URL for each segment.
const hits = await client.search.search('person near the door', {
videoIds: [job.video_id],
topK: 3,
});
for (const hit of hits) {
console.log(` #${hit.rank} [${hit.start} - ${hit.end}] ${hit.hls_playlist_url}`);
}
// 5. Chat — natural-language Q&A over the same video.
const result = await client.chat.ask('Summarize what happens in the video.', {
videoId: job.video_id,
});
console.log(result.answer);
} finally {
client.close();
}
client.close() is currently a no-op (axios has no persistent client to close) but is provided for API parity with the Python SDK — wrap your work in try { ... } finally { client.close(); } so you can swap in a custom HTTP client later without restructuring callers.
Client
import { Client } from './src/index.js';
const client = new Client({
apiKey: 'sk_...', // required
baseUrl: 'https://vision.mikshi.ai/dev/', // optional; defaults to http://localhost:8001
timeout: 30000, // optional; per-request timeout in ms
});
The client exposes six resource groups:
| Attribute | Resource |
|---|---|
client.uploads | Multipart, direct, YouTube, status |
client.videos | List, get, delete, processing pipelines |
client.collections | Collection CRUD + membership |
client.chat | Natural-language Q&A over a video |
client.search | Vector search + UMAP visualization |
client.embeddings | Raw segment / text embedding vectors |
Authentication is handled automatically — your key is sent as X-API-Key on every request.
Response shape
Every method returns the parsed JSON response as a plain JavaScript object — snake_case field names from the wire (e.g. video_id, hls_playlist_path, created_at as an ISO-8601 string). Helper functions in models.js (re-exported from index.js) provide the equivalents of Python's model methods:
import {
estimateUploadCredits, // pure helper, no network
derivedUploadStatus, // roll the three upload pipelines into one state
isPipelineDone,
isPipelineFailed,
isPipelineTerminal,
} from './src/index.js';
Uploads
Four ways to ingest a video, ordered from highest-level to lowest:
uploadFile— recommended. Runs the full multipart flow for you.direct— single-shot upload for files ≤ 100 MB.importYoutube— server pulls from a YouTube URL.- Manual multipart — only if you need retry, parallelism, or resume.
Runnable examples in examples/uploads/:
upload_file.js— high-level multipart helper (recommended)upload_direct.js— single-shot uploadupload_youtube.js— server-side YouTube importupload_multipart_manual.js— manual multipart, end-to-endupload_resume.js—listParts+abortMultipartagainst an in-progress uploadestimate_credits.js— preview the storage credit cost for a file (no network)
| Method | HTTP |
|---|---|
client.uploads.uploadFile(path, { name, ... }) | (helper — runs the full multipart flow) |
client.uploads.direct(file, { name, useDynamicHls }) | POST /api/backend/v1/developer/uploads/direct |
client.uploads.importYoutube(url, name) | POST /api/backend/v1/developer/uploads/youtube |
client.uploads.initiateMultipart(filename, contentType) | POST /api/backend/v1/developer/uploads/multipart/initiate |
client.uploads.getPartUrl(s3ObjectName, uploadId, partNumber) | POST /api/backend/v1/developer/uploads/multipart/part-url |
client.uploads.listParts(s3ObjectName, uploadId) | POST /api/backend/v1/developer/uploads/multipart/list-parts |
client.uploads.completeMultipart(s3ObjectName, uploadId, parts) | POST /api/backend/v1/developer/uploads/multipart/complete |
client.uploads.abortMultipart(s3ObjectName, uploadId) | POST /api/backend/v1/developer/uploads/multipart/abort |
client.uploads.preprocess(s3ObjectName, videoId, name, useDynamicHls) | POST /api/backend/v1/developer/uploads/preprocess |
client.uploads.getStatus(videoId) | GET /api/backend/v1/developer/uploads/status/{video_id} |
estimateUploadCredits(sizeBytes) (pure helper, top-level import) | (no network — mirrors the server's 1 credit per 25 MiB formula) |
Upload a file (recommended)
For files of any size, uploadFile chunks the file, PUTs each chunk to S3, completes the multipart upload, and kicks off processing — all in one call.
const job = await client.uploads.uploadFile('movie.mp4', { name: 'My Movie' });
console.log(job.video_id); // poll getStatus(video_id) for processing progress
With progress reporting:
function onProgress(uploaded, total) {
console.log(` ${uploaded}/${total} bytes (${Math.floor((uploaded * 100) / total)}%)`);
}
const job = await client.uploads.uploadFile('movie.mp4', {
name: 'My Movie',
chunkSize: 25 * 1024 * 1024, // 25 MB chunks (default is 10 MB)
onProgress,
});
What the helper does and doesn't do:
- Sequential chunks (default 10 MB), guesses
contentTypefrom the extension,onProgress(uploaded, total)callback. - If anything fails before
completeMultipartsucceeds, the helper callsabortMultipartso you don't pay for orphaned S3 parts. - No retry on failed chunks, no parallel uploads, no resume. For any of those, fall back to the low-level methods below.
- If
preprocessfails after a successful S3 upload, the S3 object is left in place (it's durable at that point). Use the manual flow if you need to stashs3_object_namefor retry.
Direct upload (≤ 100 MB)
const job = await client.uploads.direct('movie.mp4', { name: 'My Movie' });
console.log(job.video_id);
file accepts a path string. Raises PayloadTooLargeError on 413 — switch to uploadFile for larger files.
YouTube import
const job = await client.uploads.importYoutube('https://www.youtube.com/watch?v=dQw4w9WgXcQ');
On failure the SDK raises YouTubeImportError (a subclass of BadRequestError) with the short code on .code. The full set of known codes is exported as YOUTUBE_FATAL_CODES:
import { YouTubeImportError } from './src/index.js';
try {
await client.uploads.importYoutube(url);
} catch (e) {
if (e instanceof YouTubeImportError) {
console.log(`YouTube import rejected: ${e.code}`);
if (e.code === 'YOUTUBE_VIDEO_PRIVATE') {
// skip
}
} else {
throw e;
}
}
The SDK reads the code from whichever field the server uses (message on the dev API, detail on FastAPI defaults), so you don't need to inspect the response body yourself.
Manual multipart (only if the helper doesn't fit)
Use this when you need retry on failed parts, parallel chunk uploads, resume, or custom chunk handling.
import fs from 'node:fs';
import axios from 'axios';
const init = await client.uploads.initiateMultipart('movie.mp4', 'video/mp4');
const CHUNK = 10 * 1024 * 1024;
const total = fs.statSync('movie.mp4').size;
const parts = [];
const fd = fs.openSync('movie.mp4', 'r');
const buf = Buffer.alloc(CHUNK);
let uploaded = 0;
let partNumber = 1;
while (uploaded < total) {
const bytesRead = fs.readSync(fd, buf, 0, CHUNK, uploaded);
if (bytesRead === 0) break;
const chunk = bytesRead === CHUNK ? buf : buf.subarray(0, bytesRead);
const p = await client.uploads.getPartUrl(init.s3_object_name, init.upload_id, partNumber);
const r = await axios.put(p.url, chunk, { timeout: 300000 });
parts.push({ part_number: partNumber, etag: r.headers.etag.replace(/"/g, '') });
uploaded += bytesRead;
partNumber++;
}
fs.closeSync(fd);
await client.uploads.completeMultipart(init.s3_object_name, init.upload_id, parts);
const job = await client.uploads.preprocess(init.s3_object_name, init.video_id, 'My Movie');
Poll status
getStatus returns per-pipeline state only (no top-level overall_status). Use derivedUploadStatus(status) to roll the three pipelines into a single pending / in_progress / completed / failed state:
import { derivedUploadStatus } from './src/index.js';
while (true) {
const status = await client.uploads.getStatus(job.video_id);
const derived = derivedUploadStatus(status);
if (derived === 'completed' || derived === 'failed') {
if (derived === 'failed') throw new Error(status.error_message || 'processing failed');
break;
}
console.log(
` transcode=${status.transcoding_status?.percentage ?? 0}% ` +
`hls=${status.hls_generation_status?.percentage ?? 0}%`,
);
await new Promise((r) => setTimeout(r, 2000));
}
Per-pipeline status values are pending / in_progress / completed / failed.
Videos
Once a video is uploaded, these endpoints manage its lifecycle and run optional processing pipelines (search-process, analyse).
Runnable examples: examples/videos/ — list_videos.js, get_video.js, run_pipeline.js, delete_video.js.
| Method | HTTP |
|---|---|
client.videos.list({ status, startTime, endTime, notInCollection, skip, limit, sortBy, sortOrder }) | GET /api/backend/v1/developer/videos/list |
client.videos.get(videoId) | GET /api/backend/v1/developer/videos/get-video-details |
client.videos.delete(videoId) | DELETE /api/backend/v1/developer/videos/delete |
client.videos.initiateSearchProcess(videoId) | POST /api/backend/v1/developer/videos/search-process/initiate |
client.videos.getSearchProcessStatus(videoId) | GET /api/backend/v1/developer/videos/search-process/status/{video_id} |
client.videos.initiateAnalyse(videoId) | POST /api/backend/v1/developer/videos/analyse/initiate |
client.videos.getAnalyseStatus(videoId) | GET /api/backend/v1/developer/videos/analyse/status/{video_id} |
// List with filters (only set what you need)
const videos = await client.videos.list({ status: 'processed', limit: 50, sortOrder: 'asc' });
// Single-video details (includes video_metadata)
const details = await client.videos.get(videoId);
console.log(details.name, details.video_metadata);
// Kick off the analyse pipeline, then poll
await client.videos.initiateAnalyse(videoId);
while (true) {
const s = await client.videos.getAnalyseStatus(videoId);
if (isPipelineTerminal(s)) break;
console.log(` analyse ${s.percentage ?? 0}%`);
await sleep(2000);
}
// Permanent delete (also removes derived S3 assets)
await client.videos.delete(videoId);
status on list is one of uploaded, analysed, search_processed, processed. processed requires both pipelines to be done; the middle two filter on a single pipeline.
The pipeline initiate* calls return 409 ConflictError if transcoding_status !== "completed" — wait for upload processing to finish before kicking these off.
Playing back HLS
details.hls_playlist_path (and the same field on search hits, collection videos, etc.) is an auth-gated proxy URL — every manifest and segment fetch the player makes must carry the X-API-Key header. Don't embed the key in client-side code; proxy through your own backend or mint short-lived per-session keys.
See HLS playback in the HTTP reference for player-side wiring (hls.js, Video.js, AVPlayer, ExoPlayer).
Collections
Runnable examples: examples/collections/ — list_collections.js, collections_crud.js.
| Method | HTTP |
|---|---|
client.collections.list({ page, pageSize }) | GET /api/backend/v1/developer/collections/list |
client.collections.create(name, description) | POST /api/backend/v1/developer/collections/create |
client.collections.update(id, { name, description }) | PATCH /api/backend/v1/developer/collections/update/{id} |
client.collections.delete(id) | DELETE /api/backend/v1/developer/collections/delete/{id} |
client.collections.listVideos(id, { page, pageSize, sortBy, sortOrder }) | GET /api/backend/v1/developer/collections/{id}/videos |
client.collections.addVideo(id, videoId) | POST /api/backend/v1/developer/collections/{id}/videos |
client.collections.removeVideo(id, videoId) | DELETE /api/backend/v1/developer/collections/{id}/videos/{video_id} |
Notes
listreturns aPaginatedCollectionsshape withdata,page,page_size,total. Each collection carries up to four signed thumbnail URLs.listVideosreturns aPaginatedCollectionVideosshape with the same pagination metadata. Collection videos are always fully processed by add-time constraint.updateonly sends fields you pass. Calling it with all defaults raisesBadRequestErrorfrom the server.addVideorequires a fully-processed video — bothanalyse_status === "DONE"andsearch_process_status === "DONE". The server returns400 BadRequestErrorif the video isn't ready and409 ConflictErrorif it's already in the collection.deleteremoves the collection and its video links — the underlying video records stay intact.
Chat
Natural-language Q&A over a video. Backed by the chat service's retrieval + rerank + LLM pipeline.
Runnable example: examples/chat/ask.js — single-turn ask, with an optional --stream flag for SSE.
| Method | HTTP |
|---|---|
client.chat.ask(question, { videoId, topK, history }) | POST /api/chat/v1/developer/chat/ask |
client.chat.askStream(question, { videoId, topK, history }) | POST /api/chat/v1/developer/chat/ask (SSE, stream=true) |
Notes
videoIdis the target video. The caller must own it — unowned IDs raisePermissionDeniedError(403) before credit deduction.- Each call costs 1 credit. If the pipeline fails (5xx), the server automatically refunds the credit before re-raising — you are not billed for failed asks.
- The server is stateless. To maintain a multi-turn conversation, the caller owns the history and resends prior turns on every call.
historyis an OpenAI-style array of{ role: "user" | "assistant", content: string }objects.
Single-turn
const result = await client.chat.ask('Describe what happens in this video.', {
videoId: 'a249266f-052d-4148-bc2f-3efa76dc4269',
});
console.log(result.answer);
Multi-turn
const VIDEO_ID = 'a249266f-052d-4148-bc2f-3efa76dc4269';
const history = [];
async function turn(question) {
const result = await client.chat.ask(question, { videoId: VIDEO_ID, history });
history.push({ role: 'user', content: question });
history.push({ role: 'assistant', content: result.answer });
return result.answer;
}
console.log(await turn('Describe what happens in this video.'));
console.log(await turn('How many people are involved?'));
console.log(await turn('What was my first question?'));
topK
topK controls how many segments the retriever pulls before rerank/LLM. Omit it to use the server default (5). Bumping it gives the LLM more context but raises latency:
await client.chat.ask('Walk me through every event in chronological order.', {
videoId: VIDEO_ID,
topK: 20,
});
Streaming (SSE)
askStream returns an async iterator that yields { event, data } objects: one token per LLM chunk, then either a done event (full answer attached) or error event (mid-stream failure; credit auto-refunded).
for await (const ev of client.chat.askStream('How many people are involved?', {
videoId: VIDEO_ID,
})) {
if (ev.event === 'token') {
process.stdout.write(ev.data.content);
} else if (ev.event === 'done') {
console.log(); // full answer is also on ev.data.answer
} else if (ev.event === 'error') {
console.log(`\n[error] ${ev.data.detail}`);
break;
}
}
The stream terminates after the done or error event — the iterator stops on its own.
Search
Two endpoints power semantic search and visualization over segment embeddings. Both run against videos that have completed the analyse and search-process pipelines.
Runnable examples: examples/search/ — search.js, visualize.js.
| Method | HTTP |
|---|---|
client.search.search(query, { videoIds, collectionId, topK }) | POST /api/chat/v1/developer/search |
client.search.visualize({ videoIds, collectionId }) | POST /api/chat/v1/developer/visualize |
Notes
- You must provide at least one of
videoIdsorcollectionId. Passing neither raisesUnprocessableEntityErrorfrom the server. - The caller must own every video in the request. Unowned IDs raise
PermissionDeniedError(403). searchcosts 1 credit per call (refunded on failure).visualizeis free.visualizeneeds at least 2 segment embeddings across the selection (UMAP requires ≥2 points). Fewer raisesBadRequestError.
Vector search
Returns the topK ranked clip-level hits. Each hit carries pre-signed thumbnail_url and video_url, plus the auth-gated hls_playlist_url.
const hits = await client.search.search('person walking near the gate', {
videoIds: ['b2f1...-1111', 'b2f1...-2222'],
topK: 5,
});
for (const hit of hits) {
console.log(
`#${hit.rank} [${hit.start} - ${hit.end}] video=${hit.video_id}\n` +
` hls: ${hit.hls_playlist_url}\n` +
` thumb: ${hit.thumbnail_url}`,
);
}
You can scope to a collection instead:
const hits = await client.search.search('forklift incident', { collectionId: coll.collection_id });
Or both together — videos and collection are unioned server-side.
Visualization
Returns a 2-D UMAP projection (cosine metric, MinMax-scaled to [0, 1]) of every segment across the requested videos, plus per-video color/HLS/signed URLs and per-segment thumbnails. Useful for plotting an embedding map in a frontend.
const viz = await client.search.visualize({ collectionId: '3fa8...' });
console.log(Object.keys(viz.videos).length, 'videos,', viz.points.length, 'points');
for (const [videoId, info] of Object.entries(viz.videos)) {
console.log(videoId, 'color=', info.color);
}
for (const p of viz.points.slice(0, 5)) {
console.log(` (${p.x.toFixed(4)}, ${p.y.toFixed(4)}) segment=${p.segment_id} start=${p.start}`);
}
Response shape:
{
"videos": {
"<video_id>": {
"video_id": "uuid",
"color": "hsl(217, 70%, 55%)",
"hls_playlist_url": "...",
"video_url": "..."
}
},
"points": [
{
"segment_id": "uuid",
"video_id": "uuid",
"x": 0.41, // 0.0 - 1.0
"y": 0.78,
"start": "00:00:00",
"end": "00:00:06",
"thumbnail_url": "..."
}
]
}
Embeddings
Two endpoints expose the raw vectors used by search and chat: the stored per-segment vectors for a video (read directly from the vector store, no recompute), and the embedding of an arbitrary text string with the active backend.
Runnable examples: examples/embeddings/ — video_embeddings.js, text_embedding.js.
| Method | HTTP |
|---|---|
client.embeddings.video(videoId) | GET /api/chat/v1/developer/embeddings/video/{video_id} |
client.embeddings.text(text) | POST /api/chat/v1/developer/embeddings/text |
Notes
- Both calls cost 1 credit (refunded on 404 / 408 / 502 / 5xx).
videoreads from the vector store. If the video'ssearch-processpipeline hasn't finished yet, the server returns409and the SDK raisesConflictError— wait untilgetSearchProcessStatus(videoId)reports done and retry.textmust be 1–4096 characters.
Video embeddings
const resp = await client.embeddings.video('a249266f-052d-4148-bc2f-3efa76dc4269');
console.log(resp.status, resp.id, resp.data.length, 'segments');
for (const seg of resp.data) {
console.log(
` [${seg.start} - ${seg.end}] ` +
`option=${seg.embedding_option} scope=${seg.embedding_scope} ` +
`dims=${seg.embedding.length}`,
);
}
Text embeddings
const resp = await client.embeddings.text('chain wearing man');
const [vector] = resp.data;
console.log(`dims=${vector.embedding.length} status=${resp.status}`);
data is an array to keep the wire shape consistent with video, but the text endpoint always returns exactly one vector.
Partner integrations
The two embedding endpoints make it straightforward to plug the platform's video embeddings into your own vector database. Every integration follows the same two steps:
- Once per video, after
search-processfinishes: callclient.embeddings.video(videoId)and upsert the returned per-segment vectors into your DB. - At query time: call
client.embeddings.text(query)and run a top-K similarity query against your DB with the returned vector.
Notes that apply to every DB:
- Vector dimension: 1024 with the current backend.
- Metric: the platform stores vectors normalized for cosine similarity. Configure your DB to match (cosine, or dot-product on normalized vectors).
- Idempotent upserts: generate each row id deterministically from
(video_id, start, end)so re-ingesting the same video is safe. The snippets below usenode:cryptoUUIDv5 for this.
Pinecone
Install: npm install @pinecone-database/pinecone.
import crypto from 'node:crypto';
import { Pinecone } from '@pinecone-database/pinecone';
const NAMESPACE_URL = '6ba7b811-9dad-11d1-80b4-00c04fd430c8'; // RFC-4122 URL namespace
const uuid5 = (name) => {
const hash = crypto.createHash('sha1').update(
Buffer.concat([Buffer.from(NAMESPACE_URL.replaceAll('-', ''), 'hex'), Buffer.from(name)]),
).digest();
hash[6] = (hash[6] & 0x0f) | 0x50;
hash[8] = (hash[8] & 0x3f) | 0x80;
const h = hash.toString('hex');
return `${h.slice(0,8)}-${h.slice(8,12)}-${h.slice(12,16)}-${h.slice(16,20)}-${h.slice(20,32)}`;
};
const pc = new Pinecone({ apiKey: '...' });
const index = pc.Index('video-segments');
// 1. Ingest — pull segment embeddings from our SDK, upsert into Pinecone.
const resp = await client.embeddings.video(videoId);
await index.upsert(resp.data.map((seg) => ({
id: uuid5(`${resp.id}|${seg.start}|${seg.end}`),
values: seg.embedding,
metadata: { video_id: resp.id, start: seg.start, end: seg.end },
})));
// 2. Query — embed the user's text with our SDK, search Pinecone.
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const hits = await index.query({ vector: qv.embedding, topK: 5, includeMetadata: true });
for (const m of hits.matches) {
console.log(m.score, m.metadata.video_id, m.metadata.start);
}
Qdrant
Install: npm install @qdrant/js-client-rest.
import { QdrantClient } from '@qdrant/js-client-rest';
const qd = new QdrantClient({ url: 'http://localhost:6333' });
await qd.recreateCollection('video-segments', {
vectors: { size: 1024, distance: 'Cosine' },
});
// 1. Ingest.
const resp = await client.embeddings.video(videoId);
await qd.upsert('video-segments', {
points: resp.data.map((seg) => ({
id: uuid5(`${resp.id}|${seg.start}|${seg.end}`),
vector: seg.embedding,
payload: { video_id: resp.id, start: seg.start, end: seg.end },
})),
});
// 2. Query.
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const hits = await qd.search('video-segments', {
vector: qv.embedding,
limit: 5,
with_payload: true,
});
for (const h of hits) {
console.log(h.score, h.payload.video_id, h.payload.start);
}
Weaviate
Install: npm install weaviate-client.
import weaviate from 'weaviate-client';
const wv = await weaviate.connectToLocal();
await wv.collections.create({
name: 'VideoSegment',
vectorizers: weaviate.configure.vectorizer.none(),
properties: [
{ name: 'video_id', dataType: 'text' },
{ name: 'start', dataType: 'text' },
{ name: 'end', dataType: 'text' },
],
});
const coll = wv.collections.get('VideoSegment');
// 1. Ingest — Weaviate accepts our vector directly.
const resp = await client.embeddings.video(videoId);
await coll.data.insertMany(resp.data.map((seg) => ({
properties: { video_id: resp.id, start: seg.start, end: seg.end },
vectors: seg.embedding,
})));
// 2. Query.
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const results = await coll.query.nearVector(qv.embedding, { limit: 5 });
for (const obj of results.objects) {
console.log(obj.metadata?.distance, obj.properties.video_id, obj.properties.start);
}
pgvector (Postgres)
Install: npm install pg pgvector.
import pg from 'pg';
import pgvector from 'pgvector/pg';
const conn = new pg.Client({ connectionString: 'postgres://localhost/video' });
await conn.connect();
await pgvector.registerType(conn);
await conn.query(`
CREATE TABLE IF NOT EXISTS video_segments (
segment_id uuid PRIMARY KEY,
video_id uuid NOT NULL,
start_ts text NOT NULL,
end_ts text NOT NULL,
embedding vector(1024)
);
CREATE INDEX IF NOT EXISTS video_segments_vec_idx
ON video_segments USING hnsw (embedding vector_cosine_ops);
`);
// 1. Ingest.
const resp = await client.embeddings.video(videoId);
for (const seg of resp.data) {
await conn.query(
`INSERT INTO video_segments (segment_id, video_id, start_ts, end_ts, embedding)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (segment_id) DO UPDATE SET embedding = EXCLUDED.embedding`,
[
uuid5(`${resp.id}|${seg.start}|${seg.end}`),
resp.id,
seg.start,
seg.end,
pgvector.toSql(seg.embedding),
],
);
}
// 2. Query — `<=>` is cosine distance (smaller = closer).
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const v = pgvector.toSql(qv.embedding);
const { rows } = await conn.query(
`SELECT segment_id, video_id, start_ts, end_ts, embedding <=> $1 AS distance
FROM video_segments
ORDER BY embedding <=> $1
LIMIT 5`,
[v],
);
for (const row of rows) console.log(row.distance, row.video_id, row.start_ts);
Resolving a hit back to a playable clip
Whichever DB you use, the payload carries video_id and start — enough to call back into the platform for a signed HLS URL:
async function hitToPlayable(videoId, start) {
const details = await client.videos.get(videoId);
return `${details.hls_playlist_path}#t=${start}`;
}
details.hls_playlist_path is the auth-gated proxy URL — see Playing back HLS for the player wiring.
Errors
All errors inherit from VideoSDKError. HTTP status codes map to specific subclasses so you can catch what matters:
| HTTP | Exception |
|---|---|
| 400 | BadRequestError — YouTubeImportError is a subclass raised by uploads.importYoutube with the short code on .code |
| 401 | AuthenticationError |
| 402 | InsufficientCreditsError |
| 403 | PermissionDeniedError |
| 404 | NotFoundError |
| 409 | ConflictError |
| 413 | PayloadTooLargeError |
| 422 | UnprocessableEntityError |
| 429 | RateLimitError (sets .retryAfter from the Retry-After header) |
| 500 | InternalServerError |
| 502 | BadGatewayError |
| 503 | ServiceUnavailableError |
| other | APIError |
Every exception has .statusCode and .responseBody properties.
import { Client, ConflictError, RateLimitError } from './src/index.js';
try {
await client.collections.create('already-taken');
} catch (e) {
if (e instanceof ConflictError) {
console.log('name is taken');
} else if (e instanceof RateLimitError) {
if (e.retryAfter) await new Promise((r) => setTimeout(r, e.retryAfter * 1000));
} else {
throw e;
}
}
Response shapes
Responses are plain JS objects with snake_case fields straight from the wire. Below are the shapes you'll see most often.
UploadJob
Returned by direct, importYoutube, preprocess, and uploadFile. The server's 202 response is just { "video_id": "..." } — poll getStatus(videoId) to track processing.
{ "video_id": "uuid" }
UploadStatus
Returned by client.uploads.getStatus(videoId). Per-pipeline only — there is no top-level overall_status. Call derivedUploadStatus(status) to collapse the three pipelines into a single state.
{
"video_id": "uuid",
"thumbnail_generation_status": { "status": "completed", "percentage": 100, "error": null },
"transcoding_status": { "status": "in_progress", "percentage": 42, "error": null },
"hls_generation_status": { "status": "pending", "percentage": 0, "error": null },
"error_message": null
}
Per-pipeline status is one of pending / in_progress / completed / failed.
Multipart helpers
// initiateMultipart
{ "upload_id": "string", "s3_object_name": "string", "video_id": "uuid" }
// getPartUrl
{ "url": "https://s3...", "part_number": 1 }
// listParts → array of:
{ "part_number": 1, "etag": "string" }
// completeMultipart
{ "s3_object_name": "string", "upload_id": "string", "etag": "string" }
Video / VideoListItem / VideoDetails
videos.list returns an array of VideoListItems. videos.get returns a VideoDetails which additionally carries video_metadata.
{
"video_id": "uuid",
"name": "string",
"status": "PROCESSED",
"video_path": "...signed...",
"thumbnail_path": "...signed...",
"hls_playlist_path": "...auth-gated proxy...",
"transcoding_status": "completed",
"hls_generation_status": "completed",
"thumbnail_generation_status": "completed",
"analyse_status": "done", // list only
"search_process_status": "done", // list only
"video_metadata": { "...": "..." }, // get only
"created_at": "ISO-8601"
}
PipelineEnqueued / PipelineStatus
// initiateSearchProcess / initiateAnalyse
{ "video_id": "uuid", "status": "enqueued" }
// getSearchProcessStatus / getAnalyseStatus
{ "video_id": "uuid", "status": "done", "percentage": 100, "error": null }
The two pipelines use different status vocabularies server-side: analyse returns "done" on success, search-process returns "success". Use isPipelineDone(s) / isPipelineFailed(s) / isPipelineTerminal(s) for polling loops instead of comparing the string directly.
Collection / PaginatedCollections
{
"data": [
{
"collection_id": "uuid",
"name": "string",
"description": "string | null",
"parent_id": "uuid",
"parent_type": "user",
"created_at": "ISO-8601",
"video_count": 0,
"thumbnails": ["url", "..."]
}
],
"page": 1,
"page_size": 10,
"total": 47
}
listVideos returns the same wrapper but data is an array of Video objects with pre-signed paths.
AskResponse
{ "answer": "string" }
SearchHit (array element from search.search)
{
"rank": 1,
"start": "00:00:12",
"end": "00:00:18",
"video_id": "uuid",
"thumbnail_url": "...signed...",
"hls_playlist_url": "...auth-gated...",
"video_url": "...signed..."
}
Visualization
See the Search section above.
VideoEmbeddingsResponse
{
"id": "uuid",
"status": "ready",
"data": [
{
"embedding": [/* 1024 floats */],
"embedding_option": "visual",
"embedding_scope": "clip",
"start": "HH:MM:SS",
"end": "HH:MM:SS"
}
]
}
TextEmbeddingResponse
{
"status": "ready",
"data": [ { "embedding": [/* 1024 floats */] } ]
}
Recipes
End-to-end workflows that combine multiple SDK calls. Every recipe assumes you've already constructed a Client:
import { Client } from './src/index.js';
const client = new Client({ apiKey: 'sk_...', baseUrl: 'https://vision.mikshi.ai/dev/' });
Upload recipes
Upload a file and play it back
The full happy path: ingest a file, wait for transcoding + HLS to finish, hand the playlist URL to a player.
import { derivedUploadStatus } from './src/index.js';
async function uploadAndWait(path, name, pollInterval = 2000) {
const job = await client.uploads.uploadFile(path, { name });
console.log(`queued ${job.video_id}`);
while (true) {
const status = await client.uploads.getStatus(job.video_id);
const derived = derivedUploadStatus(status);
if (derived === 'completed') break;
if (derived === 'failed') throw new Error(`upload failed: ${status.error_message}`);
console.log(
` transcode=${status.transcoding_status?.percentage ?? 0}% ` +
`hls=${status.hls_generation_status?.percentage ?? 0}%`,
);
await new Promise((r) => setTimeout(r, pollInterval));
}
return await client.videos.get(job.video_id);
}
const video = await uploadAndWait('clip.mp4', 'My Clip');
console.log('HLS playlist:', video.hls_playlist_path);
video.hls_playlist_path is the auth-gated proxy URL — see Playing back HLS for the player wiring.
Resumable multipart upload
The uploadFile helper aborts on any failure. If you need to survive crashes (e.g. a long upload over a flaky network), drive multipart yourself and persist the upload_id so you can resume from where listParts says you left off.
import fs from 'node:fs';
import path from 'node:path';
import axios from 'axios';
const CHUNK = 10 * 1024 * 1024;
const STATE_FILE = '.upload-state.json';
async function resumeOrStart(localPath, name) {
let state;
if (fs.existsSync(STATE_FILE)) {
state = JSON.parse(fs.readFileSync(STATE_FILE, 'utf8'));
console.log(`resuming upload ${state.upload_id}`);
} else {
const init = await client.uploads.initiateMultipart(path.basename(localPath), 'video/mp4');
state = {
upload_id: init.upload_id,
s3_object_name: init.s3_object_name,
video_id: init.video_id,
};
fs.writeFileSync(STATE_FILE, JSON.stringify(state));
}
const already = await client.uploads.listParts(state.s3_object_name, state.upload_id);
const done = new Set(already.map((p) => p.part_number));
const parts = [...already];
const total = fs.statSync(localPath).size;
const fd = fs.openSync(localPath, 'r');
const buf = Buffer.alloc(CHUNK);
let uploaded = 0, partNumber = 1;
while (uploaded < total) {
const bytesRead = fs.readSync(fd, buf, 0, CHUNK, uploaded);
if (bytesRead === 0) break;
if (!done.has(partNumber)) {
const chunk = bytesRead === CHUNK ? buf : buf.subarray(0, bytesRead);
const p = await client.uploads.getPartUrl(state.s3_object_name, state.upload_id, partNumber);
const r = await axios.put(p.url, chunk, { timeout: 300000 });
parts.push({ part_number: partNumber, etag: r.headers.etag.replace(/"/g, '') });
}
uploaded += bytesRead;
partNumber++;
}
fs.closeSync(fd);
parts.sort((a, b) => a.part_number - b.part_number);
await client.uploads.completeMultipart(state.s3_object_name, state.upload_id, parts);
const job = await client.uploads.preprocess(state.s3_object_name, state.video_id, name);
fs.unlinkSync(STATE_FILE);
return job;
}
await resumeOrStart('big-movie.mp4', 'Big Movie');
YouTube import with error branching
Branch on YouTubeImportError.code so private/age-restricted videos are skipped instead of crashing a batch import.
import { YouTubeImportError } from './src/index.js';
async function safeYoutubeImport(url) {
try {
return await client.uploads.importYoutube(url);
} catch (e) {
if (e instanceof YouTubeImportError) {
if (e.code === 'YOUTUBE_VIDEO_PRIVATE') {
console.log('video is private, skipping');
return null;
}
if (e.code === 'YOUTUBE_VIDEO_AGE_RESTRICTED') {
console.log('age-restricted, skipping');
return null;
}
}
throw e;
}
}
See YouTube import for the full set of fatal codes (also available as YOUTUBE_FATAL_CODES).
Video recipes
Pipeline orchestration: backfill missing analysis
Find every video that's been transcoded but hasn't run through analyse or search-process, and kick off both pipelines.
async function backfillPipelines() {
const all = await client.videos.list({ limit: 500 });
const pending = all.filter(
(v) =>
v.transcoding_status === 'completed' &&
(v.analyse_status !== 'done' || v.search_process_status !== 'done'),
);
console.log(`backfilling ${pending.length} videos`);
await Promise.all(pending.map(async (v) => {
if (v.analyse_status !== 'done') await client.videos.initiateAnalyse(v.video_id);
if (v.search_process_status !== 'done') await client.videos.initiateSearchProcess(v.video_id);
}));
return pending.map((v) => v.video_id);
}
await backfillPipelines();
Walk videos in a date range
videos.list uses skip/limit (not page). Iterate by incrementing skip until a short page tells you you've reached the end.
async function* recentVideos(limitPerCall = 100) {
const weekAgo = new Date(Date.now() - 7 * 24 * 3600 * 1000);
let skip = 0;
while (true) {
const batch = await client.videos.list({
startTime: weekAgo,
sortBy: 'created_at',
sortOrder: 'desc',
skip,
limit: limitPerCall,
});
if (batch.length === 0) return;
for (const v of batch) yield v;
if (batch.length < limitPerCall) return;
skip += limitPerCall;
}
}
for await (const video of recentVideos()) {
console.log(video.video_id, video.name, video.created_at);
}
Collection recipes
Bulk-upload a folder into a new collection
Create a collection, upload every file, run analyse + search-process (since addVideo requires both pipelines DONE), then add each video to the collection.
import fs from 'node:fs';
import path from 'node:path';
import { isPipelineTerminal, isPipelineFailed } from './src/index.js';
async function waitPipelines(videoId) {
await client.videos.initiateAnalyse(videoId);
await client.videos.initiateSearchProcess(videoId);
while (true) {
const a = await client.videos.getAnalyseStatus(videoId);
const sp = await client.videos.getSearchProcessStatus(videoId);
if (isPipelineTerminal(a) && isPipelineTerminal(sp)) {
if (isPipelineFailed(a) || isPipelineFailed(sp)) {
throw new Error(`pipeline failed: analyse=${a.status}, search=${sp.status}`);
}
return;
}
await new Promise((r) => setTimeout(r, 2000));
}
}
async function bulkUploadToCollection(folder, collectionName) {
const coll = await client.collections.create(collectionName);
console.log(`created collection ${coll.collection_id}`);
const files = fs.readdirSync(folder).filter((f) => f.endsWith('.mp4')).sort();
for (const file of files) {
const video = await uploadAndWait(path.join(folder, file), path.parse(file).name);
await waitPipelines(video.video_id);
await client.collections.addVideo(coll.collection_id, video.video_id);
console.log(` added ${file}`);
}
return coll;
}
await bulkUploadToCollection('./footage', 'Warehouse - April');
Idempotent create with retry on rate limits
Wrap the create call so duplicate-name conflicts return the existing collection instead of raising, and 429 responses respect Retry-After.
import { RateLimitError, ConflictError } from './src/index.js';
async function withRetries(call, { retries = 5 } = {}) {
for (let attempt = 0; attempt < retries; attempt++) {
try {
return await call();
} catch (e) {
if (!(e instanceof RateLimitError)) throw e;
const wait = (e.retryAfter ?? Math.pow(2, attempt)) * 1000;
console.log(`rate limited, sleeping ${wait}ms`);
await new Promise((r) => setTimeout(r, wait));
}
}
throw new Error('retries exhausted');
}
async function getOrCreateCollection(name) {
try {
return await withRetries(() => client.collections.create(name));
} catch (e) {
if (!(e instanceof ConflictError)) throw e;
const page = await client.collections.list({ page: 1, pageSize: 100 });
const found = page.data.find((c) => c.name === name);
if (found) return found;
throw e;
}
}
Paginate every collection
collections.list is page-based — walk every result by incrementing page until you've yielded total.
async function* allCollections(pageSize = 100) {
let page = 1, yielded = 0;
while (true) {
const result = await client.collections.list({ page, pageSize });
if (result.data.length === 0) return;
for (const c of result.data) {
yield c;
yielded++;
}
if (yielded >= result.total) return;
page++;
}
}
Chat recipes
Multi-turn chat helper
The chat endpoint is stateless — wrap it in a small helper that owns the OpenAI-style history array so callers don't have to.
class Conversation {
constructor(client, { videoId, topK = null } = {}) {
this.client = client;
this.videoId = videoId;
this.topK = topK;
this.history = [];
}
async ask(question) {
const result = await this.client.chat.ask(question, {
videoId: this.videoId,
topK: this.topK,
history: this.history,
});
this.history.push({ role: 'user', content: question });
this.history.push({ role: 'assistant', content: result.answer });
return result.answer;
}
}
const chat = new Conversation(client, { videoId: 'a249266f-052d-4148-bc2f-3efa76dc4269' });
console.log(await chat.ask('Describe what happens.'));
console.log(await chat.ask('How many people are involved?'));
console.log(await chat.ask('What was my first question?'));
If the pipeline crashes (5xx), the server refunds the credit automatically — and the helper doesn't append to history on failure since the error propagates.
Search recipes
Search-then-play
Run a query, then jump the player to the first hit's start time.
const hits = await client.search.search('package being dropped', {
collectionId: coll.collection_id,
topK: 10,
});
if (hits.length === 0) {
console.log('no matches');
} else {
const top = hits[0];
console.log(`open ${top.hls_playlist_url}#t=${top.start}`);
}
The HLS URL is auth-gated — your player must inject X-API-Key, just like for client.videos.get(videoId).hls_playlist_path. See Playing back HLS.
Visualize a collection's embedding map
Group points by video so a frontend can render each video's segments in its assigned color.
async function groupedVisualization(collectionId) {
const viz = await client.search.visualize({ collectionId });
const byVideo = new Map();
for (const point of viz.points) {
if (!byVideo.has(point.video_id)) byVideo.set(point.video_id, []);
byVideo.get(point.video_id).push(point);
}
for (const [videoId, points] of byVideo) {
const info = viz.videos[videoId];
console.log(`${videoId} color=${info.color} segments=${points.length}`);
}
return { viz, byVideo };
}
await groupedVisualization(coll.collection_id);
visualize requires at least 2 segments across the selection — for very small collections you'll get BadRequestError. Catch it and skip rendering.
Development
npm install
node --check src/index.js # quick syntax check
Running examples
export VIDEOSDK_API_KEY=sk_...
export VIDEOSDK_BASE_URL=https://vision.mikshi.ai/dev/
node examples/collections/list_collections.js
PowerShell:
$env:VIDEOSDK_API_KEY = "sk_..."
$env:VIDEOSDK_BASE_URL = "https://vision.mikshi.ai/dev/"
node examples/collections/list_collections.js