Simple Vector Database

A Standby Actor that gives you a vector database with zero setup. Send text, get semantic search — no API keys, no external services, no vector math. Everything runs inside the Actor.

Why

Adding semantic search to an Apify Actor today means setting up Pinecone, Qdrant, or similar — managing accounts, API keys, billing, and network config. This Actor eliminates all of that. Send text in, get relevant results back.

How it works

• Local embedding — Uses all-MiniLM-L6-v2 (384 dimensions) running in-process. No external calls.
• Automatic chunking — Long documents are split on sentence boundaries (~200 words per chunk) so nothing is lost to the model's token limit. You never manage chunks — it's transparent.
• HNSW index — Approximate nearest neighbor search via hnswlib-node. Fast, in-memory.
• Persistent — Collections survive restarts. Data is stored in named Apify KV stores and rebuilt on startup.

Limitations

• Best for small-to-medium collections (up to ~10k documents). The entire index lives in memory.
• Eventual consistency — Data written between the last checkpoint and a crash may need re-embedding on restart.
• Single instance — No sharding or replication.
• Dense vector search only — No keyword/BM25 fallback.

Quick start

VECTOR_DB="https://your-vector-db.apify.actor"

# Create a collection
curl -X PUT "$VECTOR_DB/collections/products"

# Add documents (ID is optional — auto-generated if omitted)
curl -X POST "$VECTOR_DB/collections/products/upsert" \
  -H "Content-Type: application/json" \
  -d '{
    "documents": [
      {"text": "Lightweight running shoes with foam cushioning", "metadata": {"price": 89}},
      {"text": "Waterproof hiking boots with ankle support", "metadata": {"price": 149}},
      {"text": "Casual canvas sneakers in multiple colors", "metadata": {"price": 45}}
    ]
  }'

# Semantic search
curl -X POST "$VECTOR_DB/collections/products/query" \
  -H "Content-Type: application/json" \
  -d '{"query": "comfortable shoes for running", "topK": 2}'

API

Collections

PUT /collections/:name

Create a new collection. Dimensions (384) and distance metric (cosine) are fixed.

Response (201):

{
  "name": "products",
  "dimensions": 384,
  "distanceMetric": "cosine",
  "documentCount": 0
}

Returns 409 if the collection already exists.

GET /collections

List all collections.

Response (200):

{
  "collections": [
    {"name": "products", "dimensions": 384, "distanceMetric": "cosine", "documentCount": 42}
  ]
}

DELETE /collections/:name

Delete a collection and all its data. Returns 204 on success, 404 if not found.

Documents

POST /collections/:name/upsert

Add or update documents. Text is embedded and chunked automatically.

Request:

{
  "documents": [
    {
      "id": "doc-1",
      "text": "The Eiffel Tower is a wrought-iron lattice tower in Paris.",
      "metadata": {"source": "wikipedia"}
    },
    {
      "text": "Tokyo is the capital of Japan."
    }
  ]
}

• text (required) — The document content. Long texts are automatically chunked.
• id (optional) — Document identifier. Auto-generated UUID if omitted. If an existing ID is provided, the document is overwritten.
• metadata (optional) — Arbitrary JSON stored alongside the document.

Maximum 100 documents per request.

Response (200):

{
  "upserted": 2,
  "ids": ["doc-1", "a1b2c3d4-e5f6-7890-abcd-ef1234567890"]
}

The ids array matches the input order. Use these to fetch or overwrite documents later.

POST /collections/:name/query

Semantic search. The query is embedded and matched against all document chunks.

Request:

{
  "query": "famous landmarks in France",
  "topK": 5
}

• query (required) — Natural language search text.
• topK (optional) — Maximum results to return. Default 10, max 100.

Results are individual chunks, not deduplicated by document. If a long document has multiple relevant passages, they each appear as separate results with the same id. Use GET /documents/:id to fetch the full document.

Response (200):

{
  "results": [
    {
      "id": "doc-1",
      "score": 0.92,
      "matchedChunk": "The Eiffel Tower is a wrought-iron lattice tower in Paris.",
      "metadata": {"source": "wikipedia"}
    },
    {
      "id": "doc-1",
      "score": 0.85,
      "matchedChunk": "The tower stands 330 meters tall and weighs 10,100 tonnes.",
      "metadata": {"source": "wikipedia"}
    }
  ]
}

• id — Parent document ID. Use it to fetch the full document or group results.
• score — Cosine similarity (0 to 1, higher is better).
• matchedChunk — The specific chunk that matched. For short documents this equals the full text.

GET /collections/:name/documents/:id

Fetch a single document by ID.

Response (200):

{
  "id": "doc-1",
  "text": "The Eiffel Tower is a wrought-iron lattice tower in Paris.",
  "metadata": {"source": "wikipedia"}
}

Returns 404 if the collection or document doesn't exist.

Usage from another Actor

const VECTOR_DB = "https://your-vector-db.apify.actor";

// Create collection
await fetch(`${VECTOR_DB}/collections/articles`, { method: "PUT" });

// Upsert scraped data
const { ids } = await fetch(`${VECTOR_DB}/collections/articles/upsert`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    documents: scrapedPages.map((page) => ({
      text: page.content,
      metadata: { url: page.url, title: page.title },
    })),
  }),
}).then((r) => r.json());

// Query
const { results } = await fetch(`${VECTOR_DB}/collections/articles/query`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ query: "climate change policy", topK: 5 }),
}).then((r) => r.json());

Persistence

Each collection uses two named KV stores:

• svdb-{name}-index — Serialized HNSW graph + label mapping.
• svdb-{name}-meta — One entry per document (full text, metadata, chunk texts).

A registry store (svdb-registry) tracks which collections exist.

The meta store is the source of truth. If the index is missing or stale on startup, the Actor re-chunks and re-embeds from meta automatically.

Tech stack

Component	Choice
Runtime	Node.js 22 (TypeScript)
HTTP	Express 4
Vector index	hnswlib-node (HNSW)
Embedding	@xenova/transformers / all-MiniLM-L6-v2 (384 dims, ONNX)
Persistence	Apify Named KV Stores