Automatic embeddings

Automatic embeddings turn one or more text columns of a table into a vector(768) column that stays current on its own. When a row is inserted, or one of its source columns changes, NestCore re-embeds it in the background with your project's own AI provider key. Your app then searches the table by meaning with semantic(), under the same row-level security as any other query.

How it works

When you embed a column, NestCore changes the table in your project database:

• It adds three columns: the vector column (vector(768), named <first source column>_embedding unless you choose a name), <target>_embed_model and <target>_embed_hash.
• It adds a trigger that fires on INSERT and on UPDATE of the source columns. Updating any other column queues nothing.
• It adds an HNSW index for cosine distance on the vector column.
• It queues every existing row with text in a source column (the backfill).

A background worker then embeds queued rows in batches of 64. The embedded text is the non-empty source values joined with newlines, in the order you listed the columns, cut to 8,000 characters. The worker stores an MD5 hash of that text, so a row whose text did not change is not embedded again. If every source column of a row becomes empty, its vector, model and hash are cleared.

Each vector is stored with the model that made it. Searches only use rows embedded with the column's current model, so vectors from different models are never compared.

Requirements

• The table is in the public schema and has a primary key. The worker writes vectors back by primary key.
• ai_* and auth_* tables can't be embedded.
• 1 to 10 source columns, all existing columns of the table.
• The target column name is at most 50 characters (the platform appends _embed_model and _embed_hash) and differs from the source columns. If a column with that name already exists, it must be a vector column.

Quick start

1. Embed a column

In Studio, open Project Settings → AI → Embeddings and click Embed a column. Choose the table, tick the source columns in the order they should be joined, and optionally set the target column, the model and the provider.

From a terminal or an AI agent:

nestcore ai embeddings enable --table articles --columns title,body
nestcore ai embeddings list

The MCP tool nestcore_enable_embeddings does the same (it needs confirmEnable: true). Embedding a column needs the workspace admin role.

2. Search from your app

import { createClient } from '@nestcore/js';

const client = createClient({ apiKey: import.meta.env.VITE_NESTCORE_ANON_KEY });
await client.auth.signIn({ email, password });

const { data, error } = await client.db
  .from('articles')
  .eq('published', true)
  .semantic('title_embedding', 'how do I reset my password?', { k: 5 });

// data.rows: the matching rows (without the vector columns), each with _similarity and _score
// data.model: the embedding model of the column

The Node.js SDK (@nestcore/node, from version 0.14.0) has the same method. The web SDK has it from version 0.8.0.

Flutter:

final result = await nestcore
    .from('articles')
    .eq('published', true)
    .semantic('title_embedding', 'how do I reset my password?', k: 5);

semantic() options

semantic(column, query, options) keeps the builder's filters (eq, neq, gt, gte, lt, lte, like, ilike, in, is) and sends them with the query. It can't be combined with textSearch() or nearest(). A builder .limit() sets k. Without a builder, call client.db.semantic(table, column, query, options).

Each returned row has every column except the three embedding columns, plus:

• _similarity: cosine similarity to the query. It is null for a row found only by the keyword match of hybridColumn.
• _score: the fused rank score. Without hybridColumn, rows are sorted by _similarity; with it, by _score.

Keyword matching uses PostgreSQL's English text search (websearch_to_tsquery), so quotes, OR and -word work in the query. minSimilarity never drops keyword-only rows.

The query text is embedded with the column's model on every call. That is an ordinary AI gateway call: it counts toward your per-minute AI limit and spend caps, and appears in the usage ledger and in traces. If the column has no model yet (nothing has been embedded), semantic() returns no rows.

Change the model or re-embed

nestcore ai embeddings reembed articles title_embedding --model text-embedding-3-small

Re-embedding queues every row again. With a new model, the column switches to it at once: progress drops to zero and semantic() only finds rows already embedded with the new model until the backfill finishes. Without --model, the rows are re-embedded with the current model, which also retries rows that failed. In Studio, use Re-embed on the column.

When you don't set a model, the first batch fixes it: the provider's default embedding model (gemini-embedding-001 for Gemini, text-embedding-3-small for OpenAI), or the default embedding model saved for the provider in Project Settings → AI. OpenAI-compatible providers have no built-in default, so save one or pass a model.

Stop embedding

nestcore ai embeddings disable articles title_embedding
nestcore ai embeddings disable articles title_embedding --drop-columns

Disabling removes the trigger, the queue entries and the configuration. The vector columns stay unless you pass --drop-columns (in Studio, tick Also drop the vector columns).

Progress and failures

nestcore ai embeddings list, the MCP tool nestcore_list_embeddings and the Embeddings tab in Studio show, for each embedded column:

A batch that fails is retried after 30 seconds. If the failure is a client error, such as a missing key, a rejected model or a quota, the worker waits 10 minutes before retrying.

Access rules

The queue and configuration live in the _orbitnest schema of your project database, which app users and the anon role can't read.

Limits and quotas

Rows embedded by the worker count toward the workspace's monthly embedding allowance, shared with collection ingestion:

When the allowance is used up, the worker stops and lastError shows the quota message. Embedding resumes at the start of the next month or after an upgrade. The allowance is in addition to what your provider bills you for the calls. See Plans & billing.

Other limits:

• Embedding text is cut to 8,000 characters per row.
• semantic() queries are limited to 4,000 characters and 20 filters; an in filter takes at most 500 values.

HTTP API

The app endpoint uses project keys. Send the anon key in apikey and the user's access token in Authorization: Bearer, or the service-role key.

POST /api/project/{slug}/database/semantic
Content-Type: application/json

{
  "table": "articles",
  "column": "title_embedding",
  "query": "how do I reset my password?",
  "k": 5,
  "filters": [{ "column": "published", "op": "eq", "value": true }],
  "hybridColumn": "body",
  "minSimilarity": 0.2
}

The response is { "rows": [...], "model": "..." }. filters[].op is one of eq, neq, gt, gte, lt, lte, like, ilike, in (an array value) or is (null, true or false).

Management endpoints authenticate with a Studio access token (the session the CLI and MCP server use). :projectId is the project's UUID.

Errors

The query embedding of semantic() can also fail with the AI gateway errors, such as AI_PROVIDER_NOT_CONFIGURED, AI_RATE_LIMITED or AI_BUDGET_EXCEEDED.

Related

• Collections index documents (PDF, HTML, text) rather than table rows.
• Vector search with pgvector covers nearest() for vectors you compute yourself.