Build/Node.js SDKv1

Node.js SDK

The official Node.js SDK for the Video Platform Developer API. Wraps the HTTP API in a small ES-module client that returns plain JavaScript objects for every response.

If you are calling the API directly, see the HTTP API reference instead.


Install

From npm (recommended for application code):

bash
npm install mikshii-js-sdk
bash
yarn add mikshii-js-sdk
bash
pnpm add mikshii-js-sdk

Then import it from your app:

javascript
import { Client } from 'mikshii-js-sdk';

const client = new Client({ apiKey: 'sk_...', baseUrl: 'https://vision.mikshi.ai/dev/' });

From a local checkout (e.g. while developing against an unreleased branch):

bash
git clone <repo-url>
cd nodejs_sdk
npm install            # install runtime dependencies

Then either install it into another project by path:

bash
npm install /absolute/path/to/nodejs_sdk

…or import directly from src/index.js if you're hacking inside the repo:

javascript
import { Client } from './src/index.js';

Requires: Node.js 18+ (uses the global fetch-era FormData/Blob, ES modules, top-level await) and axios.

The SDK is shipped as an ES module — import { Client } from 'mikshii-js-sdk'. CommonJS (require) is not supported directly; use dynamic import() from a CJS file:

javascript
// my-cjs-file.cjs
(async () => {
  const { Client } = await import('mikshii-js-sdk');
  const client = new Client({ apiKey: 'sk_...' });
})();

Quick start

End-to-end: upload a video, wait for transcoding + HLS, run the analyse and search-process pipelines so the video is queryable, then run a vector search and ask a question about it.

javascript
import { Client, derivedUploadStatus, isPipelineTerminal, isPipelineFailed } from './src/index.js';

const client = new Client({
  apiKey: 'sk_AnFZjh95_...',
  baseUrl: 'https://vision.mikshi.ai/dev/',
});

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

try {
  // 1. Upload — chunks the file, PUTs to S3, completes, and kicks off
  // transcoding + HLS generation. Returns immediately with a video_id.
  const job = await client.uploads.uploadFile('clip.mp4', { name: 'My Clip' });
  console.log(`uploaded ${job.video_id}`);

  // 2. Wait for upload processing (transcode + HLS) to finish.
  while (true) {
    const status = await client.uploads.getStatus(job.video_id);
    const derived = derivedUploadStatus(status);
    if (derived === 'completed') break;
    if (derived === 'failed') throw new Error(status.error_message || 'upload processing failed');
    await sleep(2000);
  }

  // 3. Run analyse + search-process. These two pipelines are what make the
  // video searchable and chat-ready — search/visualize/ask all require them.
  await client.videos.initiateAnalyse(job.video_id);
  await client.videos.initiateSearchProcess(job.video_id);
  while (true) {
    const a = await client.videos.getAnalyseStatus(job.video_id);
    const sp = await client.videos.getSearchProcessStatus(job.video_id);
    if (isPipelineTerminal(a) && isPipelineTerminal(sp)) {
      if (isPipelineFailed(a) || isPipelineFailed(sp)) {
        throw new Error(`pipeline failed: analyse=${a.status}, search=${sp.status}`);
      }
      break;
    }
    await sleep(2000);
  }

  // 4. Vector search — ranked clip-level hits with pre-signed thumbnails and
  // the auth-gated HLS URL for each segment.
  const hits = await client.search.search('person near the door', {
    videoIds: [job.video_id],
    topK: 3,
  });
  for (const hit of hits) {
    console.log(`  #${hit.rank}  [${hit.start} - ${hit.end}]  ${hit.hls_playlist_url}`);
  }

  // 5. Chat — natural-language Q&A over the same video.
  const result = await client.chat.ask('Summarize what happens in the video.', {
    videoId: job.video_id,
  });
  console.log(result.answer);
} finally {
  client.close();
}

client.close() is currently a no-op (axios has no persistent client to close) but is provided for API parity with the Python SDK — wrap your work in try { ... } finally { client.close(); } so you can swap in a custom HTTP client later without restructuring callers.


Client

javascript
import { Client } from './src/index.js';

const client = new Client({
  apiKey: 'sk_...',                              // required
  baseUrl: 'https://vision.mikshi.ai/dev/',      // optional; defaults to http://localhost:8001
  timeout: 30000,                                // optional; per-request timeout in ms
});

The client exposes six resource groups:

AttributeResource
client.uploadsMultipart, direct, YouTube, status
client.videosList, get, delete, processing pipelines
client.collectionsCollection CRUD + membership
client.chatNatural-language Q&A over a video
client.searchVector search + UMAP visualization
client.embeddingsRaw segment / text embedding vectors

Authentication is handled automatically — your key is sent as X-API-Key on every request.

Response shape

Every method returns the parsed JSON response as a plain JavaScript object — snake_case field names from the wire (e.g. video_id, hls_playlist_path, created_at as an ISO-8601 string). Helper functions in models.js (re-exported from index.js) provide the equivalents of Python's model methods:

javascript
import {
  estimateUploadCredits,        // pure helper, no network
  derivedUploadStatus,          // roll the three upload pipelines into one state
  isPipelineDone,
  isPipelineFailed,
  isPipelineTerminal,
} from './src/index.js';

Uploads

Four ways to ingest a video, ordered from highest-level to lowest:

  1. uploadFile — recommended. Runs the full multipart flow for you.
  2. direct — single-shot upload for files ≤ 100 MB.
  3. importYoutube — server pulls from a YouTube URL.
  4. Manual multipart — only if you need retry, parallelism, or resume.

Runnable examples in examples/uploads/:

  • upload_file.js — high-level multipart helper (recommended)
  • upload_direct.js — single-shot upload
  • upload_youtube.js — server-side YouTube import
  • upload_multipart_manual.js — manual multipart, end-to-end
  • upload_resume.jslistParts + abortMultipart against an in-progress upload
  • estimate_credits.js — preview the storage credit cost for a file (no network)
MethodHTTP
client.uploads.uploadFile(path, { name, ... })(helper — runs the full multipart flow)
client.uploads.direct(file, { name, useDynamicHls })POST /api/backend/v1/developer/uploads/direct
client.uploads.importYoutube(url, name)POST /api/backend/v1/developer/uploads/youtube
client.uploads.initiateMultipart(filename, contentType)POST /api/backend/v1/developer/uploads/multipart/initiate
client.uploads.getPartUrl(s3ObjectName, uploadId, partNumber)POST /api/backend/v1/developer/uploads/multipart/part-url
client.uploads.listParts(s3ObjectName, uploadId)POST /api/backend/v1/developer/uploads/multipart/list-parts
client.uploads.completeMultipart(s3ObjectName, uploadId, parts)POST /api/backend/v1/developer/uploads/multipart/complete
client.uploads.abortMultipart(s3ObjectName, uploadId)POST /api/backend/v1/developer/uploads/multipart/abort
client.uploads.preprocess(s3ObjectName, videoId, name, useDynamicHls)POST /api/backend/v1/developer/uploads/preprocess
client.uploads.getStatus(videoId)GET /api/backend/v1/developer/uploads/status/{video_id}
estimateUploadCredits(sizeBytes) (pure helper, top-level import)(no network — mirrors the server's 1 credit per 25 MiB formula)

For files of any size, uploadFile chunks the file, PUTs each chunk to S3, completes the multipart upload, and kicks off processing — all in one call.

javascript
const job = await client.uploads.uploadFile('movie.mp4', { name: 'My Movie' });
console.log(job.video_id);   // poll getStatus(video_id) for processing progress

With progress reporting:

javascript
function onProgress(uploaded, total) {
  console.log(`  ${uploaded}/${total} bytes (${Math.floor((uploaded * 100) / total)}%)`);
}

const job = await client.uploads.uploadFile('movie.mp4', {
  name: 'My Movie',
  chunkSize: 25 * 1024 * 1024,  // 25 MB chunks (default is 10 MB)
  onProgress,
});

What the helper does and doesn't do:

  • Sequential chunks (default 10 MB), guesses contentType from the extension, onProgress(uploaded, total) callback.
  • If anything fails before completeMultipart succeeds, the helper calls abortMultipart so you don't pay for orphaned S3 parts.
  • No retry on failed chunks, no parallel uploads, no resume. For any of those, fall back to the low-level methods below.
  • If preprocess fails after a successful S3 upload, the S3 object is left in place (it's durable at that point). Use the manual flow if you need to stash s3_object_name for retry.

Direct upload (≤ 100 MB)

javascript
const job = await client.uploads.direct('movie.mp4', { name: 'My Movie' });
console.log(job.video_id);

file accepts a path string. Raises PayloadTooLargeError on 413 — switch to uploadFile for larger files.

YouTube import

javascript
const job = await client.uploads.importYoutube('https://www.youtube.com/watch?v=dQw4w9WgXcQ');

On failure the SDK raises YouTubeImportError (a subclass of BadRequestError) with the short code on .code. The full set of known codes is exported as YOUTUBE_FATAL_CODES:

javascript
import { YouTubeImportError } from './src/index.js';

try {
  await client.uploads.importYoutube(url);
} catch (e) {
  if (e instanceof YouTubeImportError) {
    console.log(`YouTube import rejected: ${e.code}`);
    if (e.code === 'YOUTUBE_VIDEO_PRIVATE') {
      // skip
    }
  } else {
    throw e;
  }
}

The SDK reads the code from whichever field the server uses (message on the dev API, detail on FastAPI defaults), so you don't need to inspect the response body yourself.

Manual multipart (only if the helper doesn't fit)

Use this when you need retry on failed parts, parallel chunk uploads, resume, or custom chunk handling.

javascript
import fs from 'node:fs';
import axios from 'axios';

const init = await client.uploads.initiateMultipart('movie.mp4', 'video/mp4');

const CHUNK = 10 * 1024 * 1024;
const total = fs.statSync('movie.mp4').size;
const parts = [];
const fd = fs.openSync('movie.mp4', 'r');
const buf = Buffer.alloc(CHUNK);

let uploaded = 0;
let partNumber = 1;
while (uploaded < total) {
  const bytesRead = fs.readSync(fd, buf, 0, CHUNK, uploaded);
  if (bytesRead === 0) break;
  const chunk = bytesRead === CHUNK ? buf : buf.subarray(0, bytesRead);
  const p = await client.uploads.getPartUrl(init.s3_object_name, init.upload_id, partNumber);
  const r = await axios.put(p.url, chunk, { timeout: 300000 });
  parts.push({ part_number: partNumber, etag: r.headers.etag.replace(/"/g, '') });
  uploaded += bytesRead;
  partNumber++;
}
fs.closeSync(fd);

await client.uploads.completeMultipart(init.s3_object_name, init.upload_id, parts);
const job = await client.uploads.preprocess(init.s3_object_name, init.video_id, 'My Movie');

Poll status

getStatus returns per-pipeline state only (no top-level overall_status). Use derivedUploadStatus(status) to roll the three pipelines into a single pending / in_progress / completed / failed state:

javascript
import { derivedUploadStatus } from './src/index.js';

while (true) {
  const status = await client.uploads.getStatus(job.video_id);
  const derived = derivedUploadStatus(status);
  if (derived === 'completed' || derived === 'failed') {
    if (derived === 'failed') throw new Error(status.error_message || 'processing failed');
    break;
  }
  console.log(
    `  transcode=${status.transcoding_status?.percentage ?? 0}%  ` +
    `hls=${status.hls_generation_status?.percentage ?? 0}%`,
  );
  await new Promise((r) => setTimeout(r, 2000));
}

Per-pipeline status values are pending / in_progress / completed / failed.


Videos

Once a video is uploaded, these endpoints manage its lifecycle and run optional processing pipelines (search-process, analyse).

Runnable examples: examples/videos/list_videos.js, get_video.js, run_pipeline.js, delete_video.js.

MethodHTTP
client.videos.list({ status, startTime, endTime, notInCollection, skip, limit, sortBy, sortOrder })GET /api/backend/v1/developer/videos/list
client.videos.get(videoId)GET /api/backend/v1/developer/videos/get-video-details
client.videos.delete(videoId)DELETE /api/backend/v1/developer/videos/delete
client.videos.initiateSearchProcess(videoId)POST /api/backend/v1/developer/videos/search-process/initiate
client.videos.getSearchProcessStatus(videoId)GET /api/backend/v1/developer/videos/search-process/status/{video_id}
client.videos.initiateAnalyse(videoId)POST /api/backend/v1/developer/videos/analyse/initiate
client.videos.getAnalyseStatus(videoId)GET /api/backend/v1/developer/videos/analyse/status/{video_id}
javascript
// List with filters (only set what you need)
const videos = await client.videos.list({ status: 'processed', limit: 50, sortOrder: 'asc' });

// Single-video details (includes video_metadata)
const details = await client.videos.get(videoId);
console.log(details.name, details.video_metadata);

// Kick off the analyse pipeline, then poll
await client.videos.initiateAnalyse(videoId);
while (true) {
  const s = await client.videos.getAnalyseStatus(videoId);
  if (isPipelineTerminal(s)) break;
  console.log(`  analyse ${s.percentage ?? 0}%`);
  await sleep(2000);
}

// Permanent delete (also removes derived S3 assets)
await client.videos.delete(videoId);

status on list is one of uploaded, analysed, search_processed, processed. processed requires both pipelines to be done; the middle two filter on a single pipeline.

The pipeline initiate* calls return 409 ConflictError if transcoding_status !== "completed" — wait for upload processing to finish before kicking these off.

Playing back HLS

details.hls_playlist_path (and the same field on search hits, collection videos, etc.) is an auth-gated proxy URL — every manifest and segment fetch the player makes must carry the X-API-Key header. Don't embed the key in client-side code; proxy through your own backend or mint short-lived per-session keys.

See HLS playback in the HTTP reference for player-side wiring (hls.js, Video.js, AVPlayer, ExoPlayer).


Collections

Runnable examples: examples/collections/list_collections.js, collections_crud.js.

MethodHTTP
client.collections.list({ page, pageSize })GET /api/backend/v1/developer/collections/list
client.collections.create(name, description)POST /api/backend/v1/developer/collections/create
client.collections.update(id, { name, description })PATCH /api/backend/v1/developer/collections/update/{id}
client.collections.delete(id)DELETE /api/backend/v1/developer/collections/delete/{id}
client.collections.listVideos(id, { page, pageSize, sortBy, sortOrder })GET /api/backend/v1/developer/collections/{id}/videos
client.collections.addVideo(id, videoId)POST /api/backend/v1/developer/collections/{id}/videos
client.collections.removeVideo(id, videoId)DELETE /api/backend/v1/developer/collections/{id}/videos/{video_id}

Notes

  • list returns a PaginatedCollections shape with data, page, page_size, total. Each collection carries up to four signed thumbnail URLs.
  • listVideos returns a PaginatedCollectionVideos shape with the same pagination metadata. Collection videos are always fully processed by add-time constraint.
  • update only sends fields you pass. Calling it with all defaults raises BadRequestError from the server.
  • addVideo requires a fully-processed video — both analyse_status === "DONE" and search_process_status === "DONE". The server returns 400 BadRequestError if the video isn't ready and 409 ConflictError if it's already in the collection.
  • delete removes the collection and its video links — the underlying video records stay intact.

Chat

Natural-language Q&A over a video. Backed by the chat service's retrieval + rerank + LLM pipeline.

Runnable example: examples/chat/ask.js — single-turn ask, with an optional --stream flag for SSE.

MethodHTTP
client.chat.ask(question, { videoId, topK, history })POST /api/chat/v1/developer/chat/ask
client.chat.askStream(question, { videoId, topK, history })POST /api/chat/v1/developer/chat/ask (SSE, stream=true)

Notes

  • videoId is the target video. The caller must own it — unowned IDs raise PermissionDeniedError (403) before credit deduction.
  • Each call costs 1 credit. If the pipeline fails (5xx), the server automatically refunds the credit before re-raising — you are not billed for failed asks.
  • The server is stateless. To maintain a multi-turn conversation, the caller owns the history and resends prior turns on every call.
  • history is an OpenAI-style array of { role: "user" | "assistant", content: string } objects.

Single-turn

javascript
const result = await client.chat.ask('Describe what happens in this video.', {
  videoId: 'a249266f-052d-4148-bc2f-3efa76dc4269',
});
console.log(result.answer);

Multi-turn

javascript
const VIDEO_ID = 'a249266f-052d-4148-bc2f-3efa76dc4269';
const history = [];

async function turn(question) {
  const result = await client.chat.ask(question, { videoId: VIDEO_ID, history });
  history.push({ role: 'user', content: question });
  history.push({ role: 'assistant', content: result.answer });
  return result.answer;
}

console.log(await turn('Describe what happens in this video.'));
console.log(await turn('How many people are involved?'));
console.log(await turn('What was my first question?'));

topK

topK controls how many segments the retriever pulls before rerank/LLM. Omit it to use the server default (5). Bumping it gives the LLM more context but raises latency:

javascript
await client.chat.ask('Walk me through every event in chronological order.', {
  videoId: VIDEO_ID,
  topK: 20,
});

Streaming (SSE)

askStream returns an async iterator that yields { event, data } objects: one token per LLM chunk, then either a done event (full answer attached) or error event (mid-stream failure; credit auto-refunded).

javascript
for await (const ev of client.chat.askStream('How many people are involved?', {
  videoId: VIDEO_ID,
})) {
  if (ev.event === 'token') {
    process.stdout.write(ev.data.content);
  } else if (ev.event === 'done') {
    console.log();                     // full answer is also on ev.data.answer
  } else if (ev.event === 'error') {
    console.log(`\n[error] ${ev.data.detail}`);
    break;
  }
}

The stream terminates after the done or error event — the iterator stops on its own.


Two endpoints power semantic search and visualization over segment embeddings. Both run against videos that have completed the analyse and search-process pipelines.

Runnable examples: examples/search/search.js, visualize.js.

MethodHTTP
client.search.search(query, { videoIds, collectionId, topK })POST /api/chat/v1/developer/search
client.search.visualize({ videoIds, collectionId })POST /api/chat/v1/developer/visualize

Notes

  • You must provide at least one of videoIds or collectionId. Passing neither raises UnprocessableEntityError from the server.
  • The caller must own every video in the request. Unowned IDs raise PermissionDeniedError (403).
  • search costs 1 credit per call (refunded on failure). visualize is free.
  • visualize needs at least 2 segment embeddings across the selection (UMAP requires ≥2 points). Fewer raises BadRequestError.

Returns the topK ranked clip-level hits. Each hit carries pre-signed thumbnail_url and video_url, plus the auth-gated hls_playlist_url.

javascript
const hits = await client.search.search('person walking near the gate', {
  videoIds: ['b2f1...-1111', 'b2f1...-2222'],
  topK: 5,
});

for (const hit of hits) {
  console.log(
    `#${hit.rank}  [${hit.start} - ${hit.end}]  video=${hit.video_id}\n` +
    `    hls:   ${hit.hls_playlist_url}\n` +
    `    thumb: ${hit.thumbnail_url}`,
  );
}

You can scope to a collection instead:

javascript
const hits = await client.search.search('forklift incident', { collectionId: coll.collection_id });

Or both together — videos and collection are unioned server-side.

Visualization

Returns a 2-D UMAP projection (cosine metric, MinMax-scaled to [0, 1]) of every segment across the requested videos, plus per-video color/HLS/signed URLs and per-segment thumbnails. Useful for plotting an embedding map in a frontend.

javascript
const viz = await client.search.visualize({ collectionId: '3fa8...' });

console.log(Object.keys(viz.videos).length, 'videos,', viz.points.length, 'points');

for (const [videoId, info] of Object.entries(viz.videos)) {
  console.log(videoId, 'color=', info.color);
}

for (const p of viz.points.slice(0, 5)) {
  console.log(`  (${p.x.toFixed(4)}, ${p.y.toFixed(4)})  segment=${p.segment_id}  start=${p.start}`);
}

Response shape:

jsonc
{
  "videos": {
    "<video_id>": {
      "video_id": "uuid",
      "color": "hsl(217, 70%, 55%)",
      "hls_playlist_url": "...",
      "video_url": "..."
    }
  },
  "points": [
    {
      "segment_id": "uuid",
      "video_id": "uuid",
      "x": 0.41,    // 0.0 - 1.0
      "y": 0.78,
      "start": "00:00:00",
      "end":   "00:00:06",
      "thumbnail_url": "..."
    }
  ]
}

Embeddings

Two endpoints expose the raw vectors used by search and chat: the stored per-segment vectors for a video (read directly from the vector store, no recompute), and the embedding of an arbitrary text string with the active backend.

Runnable examples: examples/embeddings/video_embeddings.js, text_embedding.js.

MethodHTTP
client.embeddings.video(videoId)GET /api/chat/v1/developer/embeddings/video/{video_id}
client.embeddings.text(text)POST /api/chat/v1/developer/embeddings/text

Notes

  • Both calls cost 1 credit (refunded on 404 / 408 / 502 / 5xx).
  • video reads from the vector store. If the video's search-process pipeline hasn't finished yet, the server returns 409 and the SDK raises ConflictError — wait until getSearchProcessStatus(videoId) reports done and retry.
  • text must be 1–4096 characters.

Video embeddings

javascript
const resp = await client.embeddings.video('a249266f-052d-4148-bc2f-3efa76dc4269');

console.log(resp.status, resp.id, resp.data.length, 'segments');
for (const seg of resp.data) {
  console.log(
    `  [${seg.start} - ${seg.end}]  ` +
    `option=${seg.embedding_option}  scope=${seg.embedding_scope}  ` +
    `dims=${seg.embedding.length}`,
  );
}

Text embeddings

javascript
const resp = await client.embeddings.text('chain wearing man');
const [vector] = resp.data;
console.log(`dims=${vector.embedding.length}  status=${resp.status}`);

data is an array to keep the wire shape consistent with video, but the text endpoint always returns exactly one vector.


Partner integrations

The two embedding endpoints make it straightforward to plug the platform's video embeddings into your own vector database. Every integration follows the same two steps:

  1. Once per video, after search-process finishes: call client.embeddings.video(videoId) and upsert the returned per-segment vectors into your DB.
  2. At query time: call client.embeddings.text(query) and run a top-K similarity query against your DB with the returned vector.

Notes that apply to every DB:

  • Vector dimension: 1024 with the current backend.
  • Metric: the platform stores vectors normalized for cosine similarity. Configure your DB to match (cosine, or dot-product on normalized vectors).
  • Idempotent upserts: generate each row id deterministically from (video_id, start, end) so re-ingesting the same video is safe. The snippets below use node:crypto UUIDv5 for this.

Pinecone

Install: npm install @pinecone-database/pinecone.

javascript
import crypto from 'node:crypto';
import { Pinecone } from '@pinecone-database/pinecone';

const NAMESPACE_URL = '6ba7b811-9dad-11d1-80b4-00c04fd430c8'; // RFC-4122 URL namespace
const uuid5 = (name) => {
  const hash = crypto.createHash('sha1').update(
    Buffer.concat([Buffer.from(NAMESPACE_URL.replaceAll('-', ''), 'hex'), Buffer.from(name)]),
  ).digest();
  hash[6] = (hash[6] & 0x0f) | 0x50;
  hash[8] = (hash[8] & 0x3f) | 0x80;
  const h = hash.toString('hex');
  return `${h.slice(0,8)}-${h.slice(8,12)}-${h.slice(12,16)}-${h.slice(16,20)}-${h.slice(20,32)}`;
};

const pc = new Pinecone({ apiKey: '...' });
const index = pc.Index('video-segments');

// 1. Ingest — pull segment embeddings from our SDK, upsert into Pinecone.
const resp = await client.embeddings.video(videoId);
await index.upsert(resp.data.map((seg) => ({
  id: uuid5(`${resp.id}|${seg.start}|${seg.end}`),
  values: seg.embedding,
  metadata: { video_id: resp.id, start: seg.start, end: seg.end },
})));

// 2. Query — embed the user's text with our SDK, search Pinecone.
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const hits = await index.query({ vector: qv.embedding, topK: 5, includeMetadata: true });
for (const m of hits.matches) {
  console.log(m.score, m.metadata.video_id, m.metadata.start);
}

Qdrant

Install: npm install @qdrant/js-client-rest.

javascript
import { QdrantClient } from '@qdrant/js-client-rest';

const qd = new QdrantClient({ url: 'http://localhost:6333' });
await qd.recreateCollection('video-segments', {
  vectors: { size: 1024, distance: 'Cosine' },
});

// 1. Ingest.
const resp = await client.embeddings.video(videoId);
await qd.upsert('video-segments', {
  points: resp.data.map((seg) => ({
    id: uuid5(`${resp.id}|${seg.start}|${seg.end}`),
    vector: seg.embedding,
    payload: { video_id: resp.id, start: seg.start, end: seg.end },
  })),
});

// 2. Query.
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const hits = await qd.search('video-segments', {
  vector: qv.embedding,
  limit: 5,
  with_payload: true,
});
for (const h of hits) {
  console.log(h.score, h.payload.video_id, h.payload.start);
}

Weaviate

Install: npm install weaviate-client.

javascript
import weaviate from 'weaviate-client';

const wv = await weaviate.connectToLocal();
await wv.collections.create({
  name: 'VideoSegment',
  vectorizers: weaviate.configure.vectorizer.none(),
  properties: [
    { name: 'video_id', dataType: 'text' },
    { name: 'start',    dataType: 'text' },
    { name: 'end',      dataType: 'text' },
  ],
});
const coll = wv.collections.get('VideoSegment');

// 1. Ingest — Weaviate accepts our vector directly.
const resp = await client.embeddings.video(videoId);
await coll.data.insertMany(resp.data.map((seg) => ({
  properties: { video_id: resp.id, start: seg.start, end: seg.end },
  vectors: seg.embedding,
})));

// 2. Query.
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const results = await coll.query.nearVector(qv.embedding, { limit: 5 });
for (const obj of results.objects) {
  console.log(obj.metadata?.distance, obj.properties.video_id, obj.properties.start);
}

pgvector (Postgres)

Install: npm install pg pgvector.

javascript
import pg from 'pg';
import pgvector from 'pgvector/pg';

const conn = new pg.Client({ connectionString: 'postgres://localhost/video' });
await conn.connect();
await pgvector.registerType(conn);

await conn.query(`
  CREATE TABLE IF NOT EXISTS video_segments (
    segment_id uuid PRIMARY KEY,
    video_id   uuid NOT NULL,
    start_ts   text NOT NULL,
    end_ts     text NOT NULL,
    embedding  vector(1024)
  );
  CREATE INDEX IF NOT EXISTS video_segments_vec_idx
    ON video_segments USING hnsw (embedding vector_cosine_ops);
`);

// 1. Ingest.
const resp = await client.embeddings.video(videoId);
for (const seg of resp.data) {
  await conn.query(
    `INSERT INTO video_segments (segment_id, video_id, start_ts, end_ts, embedding)
     VALUES ($1, $2, $3, $4, $5)
     ON CONFLICT (segment_id) DO UPDATE SET embedding = EXCLUDED.embedding`,
    [
      uuid5(`${resp.id}|${seg.start}|${seg.end}`),
      resp.id,
      seg.start,
      seg.end,
      pgvector.toSql(seg.embedding),
    ],
  );
}

// 2. Query — `<=>` is cosine distance (smaller = closer).
const query = await client.embeddings.text('man wearing striped shirt');
const [qv] = query.data;
const v = pgvector.toSql(qv.embedding);
const { rows } = await conn.query(
  `SELECT segment_id, video_id, start_ts, end_ts, embedding <=> $1 AS distance
   FROM video_segments
   ORDER BY embedding <=> $1
   LIMIT 5`,
  [v],
);
for (const row of rows) console.log(row.distance, row.video_id, row.start_ts);

Resolving a hit back to a playable clip

Whichever DB you use, the payload carries video_id and start — enough to call back into the platform for a signed HLS URL:

javascript
async function hitToPlayable(videoId, start) {
  const details = await client.videos.get(videoId);
  return `${details.hls_playlist_path}#t=${start}`;
}

details.hls_playlist_path is the auth-gated proxy URL — see Playing back HLS for the player wiring.


Errors

All errors inherit from VideoSDKError. HTTP status codes map to specific subclasses so you can catch what matters:

HTTPException
400BadRequestErrorYouTubeImportError is a subclass raised by uploads.importYoutube with the short code on .code
401AuthenticationError
402InsufficientCreditsError
403PermissionDeniedError
404NotFoundError
409ConflictError
413PayloadTooLargeError
422UnprocessableEntityError
429RateLimitError (sets .retryAfter from the Retry-After header)
500InternalServerError
502BadGatewayError
503ServiceUnavailableError
otherAPIError

Every exception has .statusCode and .responseBody properties.

javascript
import { Client, ConflictError, RateLimitError } from './src/index.js';

try {
  await client.collections.create('already-taken');
} catch (e) {
  if (e instanceof ConflictError) {
    console.log('name is taken');
  } else if (e instanceof RateLimitError) {
    if (e.retryAfter) await new Promise((r) => setTimeout(r, e.retryAfter * 1000));
  } else {
    throw e;
  }
}

Response shapes

Responses are plain JS objects with snake_case fields straight from the wire. Below are the shapes you'll see most often.

UploadJob

Returned by direct, importYoutube, preprocess, and uploadFile. The server's 202 response is just { "video_id": "..." } — poll getStatus(videoId) to track processing.

jsonc
{ "video_id": "uuid" }

UploadStatus

Returned by client.uploads.getStatus(videoId). Per-pipeline only — there is no top-level overall_status. Call derivedUploadStatus(status) to collapse the three pipelines into a single state.

jsonc
{
  "video_id": "uuid",
  "thumbnail_generation_status": { "status": "completed", "percentage": 100, "error": null },
  "transcoding_status":          { "status": "in_progress", "percentage": 42, "error": null },
  "hls_generation_status":       { "status": "pending", "percentage": 0, "error": null },
  "error_message": null
}

Per-pipeline status is one of pending / in_progress / completed / failed.

Multipart helpers

jsonc
// initiateMultipart
{ "upload_id": "string", "s3_object_name": "string", "video_id": "uuid" }

// getPartUrl
{ "url": "https://s3...", "part_number": 1 }

// listParts → array of:
{ "part_number": 1, "etag": "string" }

// completeMultipart
{ "s3_object_name": "string", "upload_id": "string", "etag": "string" }

Video / VideoListItem / VideoDetails

videos.list returns an array of VideoListItems. videos.get returns a VideoDetails which additionally carries video_metadata.

jsonc
{
  "video_id": "uuid",
  "name": "string",
  "status": "PROCESSED",
  "video_path": "...signed...",
  "thumbnail_path": "...signed...",
  "hls_playlist_path": "...auth-gated proxy...",
  "transcoding_status": "completed",
  "hls_generation_status": "completed",
  "thumbnail_generation_status": "completed",
  "analyse_status": "done",        // list only
  "search_process_status": "done", // list only
  "video_metadata": { "...": "..." }, // get only
  "created_at": "ISO-8601"
}

PipelineEnqueued / PipelineStatus

jsonc
// initiateSearchProcess / initiateAnalyse
{ "video_id": "uuid", "status": "enqueued" }

// getSearchProcessStatus / getAnalyseStatus
{ "video_id": "uuid", "status": "done", "percentage": 100, "error": null }

The two pipelines use different status vocabularies server-side: analyse returns "done" on success, search-process returns "success". Use isPipelineDone(s) / isPipelineFailed(s) / isPipelineTerminal(s) for polling loops instead of comparing the string directly.

Collection / PaginatedCollections

jsonc
{
  "data": [
    {
      "collection_id": "uuid",
      "name": "string",
      "description": "string | null",
      "parent_id": "uuid",
      "parent_type": "user",
      "created_at": "ISO-8601",
      "video_count": 0,
      "thumbnails": ["url", "..."]
    }
  ],
  "page": 1,
  "page_size": 10,
  "total": 47
}

listVideos returns the same wrapper but data is an array of Video objects with pre-signed paths.

AskResponse

jsonc
{ "answer": "string" }

SearchHit (array element from search.search)

jsonc
{
  "rank": 1,
  "start": "00:00:12",
  "end":   "00:00:18",
  "video_id": "uuid",
  "thumbnail_url": "...signed...",
  "hls_playlist_url": "...auth-gated...",
  "video_url": "...signed..."
}

Visualization

See the Search section above.

VideoEmbeddingsResponse

jsonc
{
  "id": "uuid",
  "status": "ready",
  "data": [
    {
      "embedding": [/* 1024 floats */],
      "embedding_option": "visual",
      "embedding_scope":  "clip",
      "start": "HH:MM:SS",
      "end":   "HH:MM:SS"
    }
  ]
}

TextEmbeddingResponse

jsonc
{
  "status": "ready",
  "data": [ { "embedding": [/* 1024 floats */] } ]
}

Recipes

End-to-end workflows that combine multiple SDK calls. Every recipe assumes you've already constructed a Client:

javascript
import { Client } from './src/index.js';
const client = new Client({ apiKey: 'sk_...', baseUrl: 'https://vision.mikshi.ai/dev/' });

Upload recipes

Upload a file and play it back

The full happy path: ingest a file, wait for transcoding + HLS to finish, hand the playlist URL to a player.

javascript
import { derivedUploadStatus } from './src/index.js';

async function uploadAndWait(path, name, pollInterval = 2000) {
  const job = await client.uploads.uploadFile(path, { name });
  console.log(`queued ${job.video_id}`);

  while (true) {
    const status = await client.uploads.getStatus(job.video_id);
    const derived = derivedUploadStatus(status);
    if (derived === 'completed') break;
    if (derived === 'failed') throw new Error(`upload failed: ${status.error_message}`);
    console.log(
      `  transcode=${status.transcoding_status?.percentage ?? 0}%  ` +
      `hls=${status.hls_generation_status?.percentage ?? 0}%`,
    );
    await new Promise((r) => setTimeout(r, pollInterval));
  }

  return await client.videos.get(job.video_id);
}

const video = await uploadAndWait('clip.mp4', 'My Clip');
console.log('HLS playlist:', video.hls_playlist_path);

video.hls_playlist_path is the auth-gated proxy URL — see Playing back HLS for the player wiring.

Resumable multipart upload

The uploadFile helper aborts on any failure. If you need to survive crashes (e.g. a long upload over a flaky network), drive multipart yourself and persist the upload_id so you can resume from where listParts says you left off.

javascript
import fs from 'node:fs';
import path from 'node:path';
import axios from 'axios';

const CHUNK = 10 * 1024 * 1024;
const STATE_FILE = '.upload-state.json';

async function resumeOrStart(localPath, name) {
  let state;
  if (fs.existsSync(STATE_FILE)) {
    state = JSON.parse(fs.readFileSync(STATE_FILE, 'utf8'));
    console.log(`resuming upload ${state.upload_id}`);
  } else {
    const init = await client.uploads.initiateMultipart(path.basename(localPath), 'video/mp4');
    state = {
      upload_id: init.upload_id,
      s3_object_name: init.s3_object_name,
      video_id: init.video_id,
    };
    fs.writeFileSync(STATE_FILE, JSON.stringify(state));
  }

  const already = await client.uploads.listParts(state.s3_object_name, state.upload_id);
  const done = new Set(already.map((p) => p.part_number));
  const parts = [...already];

  const total = fs.statSync(localPath).size;
  const fd = fs.openSync(localPath, 'r');
  const buf = Buffer.alloc(CHUNK);
  let uploaded = 0, partNumber = 1;
  while (uploaded < total) {
    const bytesRead = fs.readSync(fd, buf, 0, CHUNK, uploaded);
    if (bytesRead === 0) break;
    if (!done.has(partNumber)) {
      const chunk = bytesRead === CHUNK ? buf : buf.subarray(0, bytesRead);
      const p = await client.uploads.getPartUrl(state.s3_object_name, state.upload_id, partNumber);
      const r = await axios.put(p.url, chunk, { timeout: 300000 });
      parts.push({ part_number: partNumber, etag: r.headers.etag.replace(/"/g, '') });
    }
    uploaded += bytesRead;
    partNumber++;
  }
  fs.closeSync(fd);

  parts.sort((a, b) => a.part_number - b.part_number);
  await client.uploads.completeMultipart(state.s3_object_name, state.upload_id, parts);
  const job = await client.uploads.preprocess(state.s3_object_name, state.video_id, name);
  fs.unlinkSync(STATE_FILE);
  return job;
}

await resumeOrStart('big-movie.mp4', 'Big Movie');

YouTube import with error branching

Branch on YouTubeImportError.code so private/age-restricted videos are skipped instead of crashing a batch import.

javascript
import { YouTubeImportError } from './src/index.js';

async function safeYoutubeImport(url) {
  try {
    return await client.uploads.importYoutube(url);
  } catch (e) {
    if (e instanceof YouTubeImportError) {
      if (e.code === 'YOUTUBE_VIDEO_PRIVATE') {
        console.log('video is private, skipping');
        return null;
      }
      if (e.code === 'YOUTUBE_VIDEO_AGE_RESTRICTED') {
        console.log('age-restricted, skipping');
        return null;
      }
    }
    throw e;
  }
}

See YouTube import for the full set of fatal codes (also available as YOUTUBE_FATAL_CODES).

Video recipes

Pipeline orchestration: backfill missing analysis

Find every video that's been transcoded but hasn't run through analyse or search-process, and kick off both pipelines.

javascript
async function backfillPipelines() {
  const all = await client.videos.list({ limit: 500 });
  const pending = all.filter(
    (v) =>
      v.transcoding_status === 'completed' &&
      (v.analyse_status !== 'done' || v.search_process_status !== 'done'),
  );
  console.log(`backfilling ${pending.length} videos`);

  await Promise.all(pending.map(async (v) => {
    if (v.analyse_status !== 'done') await client.videos.initiateAnalyse(v.video_id);
    if (v.search_process_status !== 'done') await client.videos.initiateSearchProcess(v.video_id);
  }));

  return pending.map((v) => v.video_id);
}

await backfillPipelines();

Walk videos in a date range

videos.list uses skip/limit (not page). Iterate by incrementing skip until a short page tells you you've reached the end.

javascript
async function* recentVideos(limitPerCall = 100) {
  const weekAgo = new Date(Date.now() - 7 * 24 * 3600 * 1000);
  let skip = 0;
  while (true) {
    const batch = await client.videos.list({
      startTime: weekAgo,
      sortBy: 'created_at',
      sortOrder: 'desc',
      skip,
      limit: limitPerCall,
    });
    if (batch.length === 0) return;
    for (const v of batch) yield v;
    if (batch.length < limitPerCall) return;
    skip += limitPerCall;
  }
}

for await (const video of recentVideos()) {
  console.log(video.video_id, video.name, video.created_at);
}

Collection recipes

Bulk-upload a folder into a new collection

Create a collection, upload every file, run analyse + search-process (since addVideo requires both pipelines DONE), then add each video to the collection.

javascript
import fs from 'node:fs';
import path from 'node:path';
import { isPipelineTerminal, isPipelineFailed } from './src/index.js';

async function waitPipelines(videoId) {
  await client.videos.initiateAnalyse(videoId);
  await client.videos.initiateSearchProcess(videoId);
  while (true) {
    const a  = await client.videos.getAnalyseStatus(videoId);
    const sp = await client.videos.getSearchProcessStatus(videoId);
    if (isPipelineTerminal(a) && isPipelineTerminal(sp)) {
      if (isPipelineFailed(a) || isPipelineFailed(sp)) {
        throw new Error(`pipeline failed: analyse=${a.status}, search=${sp.status}`);
      }
      return;
    }
    await new Promise((r) => setTimeout(r, 2000));
  }
}

async function bulkUploadToCollection(folder, collectionName) {
  const coll = await client.collections.create(collectionName);
  console.log(`created collection ${coll.collection_id}`);

  const files = fs.readdirSync(folder).filter((f) => f.endsWith('.mp4')).sort();
  for (const file of files) {
    const video = await uploadAndWait(path.join(folder, file), path.parse(file).name);
    await waitPipelines(video.video_id);
    await client.collections.addVideo(coll.collection_id, video.video_id);
    console.log(`  added ${file}`);
  }

  return coll;
}

await bulkUploadToCollection('./footage', 'Warehouse - April');

Idempotent create with retry on rate limits

Wrap the create call so duplicate-name conflicts return the existing collection instead of raising, and 429 responses respect Retry-After.

javascript
import { RateLimitError, ConflictError } from './src/index.js';

async function withRetries(call, { retries = 5 } = {}) {
  for (let attempt = 0; attempt < retries; attempt++) {
    try {
      return await call();
    } catch (e) {
      if (!(e instanceof RateLimitError)) throw e;
      const wait = (e.retryAfter ?? Math.pow(2, attempt)) * 1000;
      console.log(`rate limited, sleeping ${wait}ms`);
      await new Promise((r) => setTimeout(r, wait));
    }
  }
  throw new Error('retries exhausted');
}

async function getOrCreateCollection(name) {
  try {
    return await withRetries(() => client.collections.create(name));
  } catch (e) {
    if (!(e instanceof ConflictError)) throw e;
    const page = await client.collections.list({ page: 1, pageSize: 100 });
    const found = page.data.find((c) => c.name === name);
    if (found) return found;
    throw e;
  }
}

Paginate every collection

collections.list is page-based — walk every result by incrementing page until you've yielded total.

javascript
async function* allCollections(pageSize = 100) {
  let page = 1, yielded = 0;
  while (true) {
    const result = await client.collections.list({ page, pageSize });
    if (result.data.length === 0) return;
    for (const c of result.data) {
      yield c;
      yielded++;
    }
    if (yielded >= result.total) return;
    page++;
  }
}

Chat recipes

Multi-turn chat helper

The chat endpoint is stateless — wrap it in a small helper that owns the OpenAI-style history array so callers don't have to.

javascript
class Conversation {
  constructor(client, { videoId, topK = null } = {}) {
    this.client = client;
    this.videoId = videoId;
    this.topK = topK;
    this.history = [];
  }

  async ask(question) {
    const result = await this.client.chat.ask(question, {
      videoId: this.videoId,
      topK: this.topK,
      history: this.history,
    });
    this.history.push({ role: 'user', content: question });
    this.history.push({ role: 'assistant', content: result.answer });
    return result.answer;
  }
}

const chat = new Conversation(client, { videoId: 'a249266f-052d-4148-bc2f-3efa76dc4269' });
console.log(await chat.ask('Describe what happens.'));
console.log(await chat.ask('How many people are involved?'));
console.log(await chat.ask('What was my first question?'));

If the pipeline crashes (5xx), the server refunds the credit automatically — and the helper doesn't append to history on failure since the error propagates.

Search recipes

Search-then-play

Run a query, then jump the player to the first hit's start time.

javascript
const hits = await client.search.search('package being dropped', {
  collectionId: coll.collection_id,
  topK: 10,
});
if (hits.length === 0) {
  console.log('no matches');
} else {
  const top = hits[0];
  console.log(`open ${top.hls_playlist_url}#t=${top.start}`);
}

The HLS URL is auth-gated — your player must inject X-API-Key, just like for client.videos.get(videoId).hls_playlist_path. See Playing back HLS.

Visualize a collection's embedding map

Group points by video so a frontend can render each video's segments in its assigned color.

javascript
async function groupedVisualization(collectionId) {
  const viz = await client.search.visualize({ collectionId });
  const byVideo = new Map();
  for (const point of viz.points) {
    if (!byVideo.has(point.video_id)) byVideo.set(point.video_id, []);
    byVideo.get(point.video_id).push(point);
  }
  for (const [videoId, points] of byVideo) {
    const info = viz.videos[videoId];
    console.log(`${videoId}  color=${info.color}  segments=${points.length}`);
  }
  return { viz, byVideo };
}

await groupedVisualization(coll.collection_id);

visualize requires at least 2 segments across the selection — for very small collections you'll get BadRequestError. Catch it and skip rendering.


Development

bash
npm install
node --check src/index.js   # quick syntax check

Running examples

bash
export VIDEOSDK_API_KEY=sk_...
export VIDEOSDK_BASE_URL=https://vision.mikshi.ai/dev/
node examples/collections/list_collections.js

PowerShell:

powershell
$env:VIDEOSDK_API_KEY = "sk_..."
$env:VIDEOSDK_BASE_URL = "https://vision.mikshi.ai/dev/"
node examples/collections/list_collections.js