Threads

A thread is a saved conversation between a user and a model. You add the user's message and run the thread; NestCore assembles the context (recent history, the user's memories, passages from your collections), streams the reply, and saves it with its token count and cost as it streams. If the user switches device or reconnects, the conversation and any half-finished reply are already there.

Before you start

• Add a provider key in Project Settings → AI. Runs use your own key through the AI gateway, with the same models, spend caps and errors.
• A signed-in user owns the threads they create and is the only user who can read or run them. The service-role key can work with every thread and create threads for a user.
• Threads live in the ai_threads and ai_messages tables of your project database, created on first use with row-level security.

Quick start

import { createClient } from '@nestcore/js';

const client = createClient({ apiKey: import.meta.env.VITE_NESTCORE_ANON_KEY });
await client.auth.signIn({ email, password });

const { data: thread } = await client.ai.threads.create({ title: 'Trip ideas' });

let reply = '';
for await (const ev of client.ai.threads.from(thread.id).run({
  input: 'Plan a weekend in Oslo',
  memory: true,
  collections: ['handbook'],
})) {
  if (ev.type === 'delta') reply += ev.text;
  if (ev.type === 'citation') console.log(`[${ev.index}]`, ev.title, ev.sourceRef);
  if (ev.type === 'done') console.log(`${ev.tokensIn}+${ev.tokensOut} tokens, $${ev.costUsd}`);
  if (ev.type === 'error') console.error(ev.code, ev.message);
}

run yields events instead of throwing, so one loop handles failures too. To wait for the whole reply instead, use runOnce(options), which resolves to { messageId, text, citations, toolCalls, usage }.

client.ai.threads is in the Node.js SDK from version 0.14.0 and the web SDK (@nestcore/js) from version 0.8.0.

Flutter: run returns a Stream; cancelling the subscription stops the reply.

await for (final ev in nestcore.ai.threads.from(threadId).run(input: 'Plan a weekend in Oslo', memory: true)) {
  // text deltas, citations, tool events, then done or error
}

Run options

Context and citations

Before calling the model, a run builds one system message, then adds recent history:

1. Your system text.
2. With memory, up to 5 of the user's memories that match the last user message (cosine similarity 0.3 or more), under "What you know about the user".
3. With collections, the 6 best passages across those collections, numbered [1], [2]… with an instruction to cite them as [n].
4. As much recent history as still fits maxContextTokens. The newest message is always kept and older ones drop first.

When the budget is tight, sources are dropped before history and memories before sources. Tokens are estimated at about 4 characters each, and an image at 800.

Every source that made it into the context arrives as a citation event before the reply, with index, collection, documentId, chunkId, title, sourceRef and snippet. Match the model's [n] markers to index to link answers to their sources. The citations are also saved in the assistant message's metadata.citations.

Built-in tools

With tools, the model can search during the run instead of relying only on the context given up front:

• memory_search searches the user's memories (offered only when memory is set).
• collections_search searches the run's collections (offered only when collections is set).

NestCore runs these tools itself and streams a tool_call event and then a tool_result event for each call. A run allows up to 4 rounds of tool calls. You can't add your own functions to a thread run; for that, use tool calling with client.ai.chat.

Streaming events

The stream is server-sent events, one JSON object per data: line:

Problems found before the reply starts come back as a normal JSON error response, which the SDK turns into one error event with status: an unknown thread, invalid options, a thread with no user message, no provider key, a spend cap, the rate limit, or a provider that refuses the first call. Nothing is saved for those (a user message passed as input is kept). A failure after the reply has started (the provider breaks off mid-stream) arrives as an error event, and that assistant message is marked error with the text received so far.

Saved replies, cancel and resume

The assistant message is created with status: "streaming" when the run starts, and its text is saved about once a second as it streams. When the run ends, the message is complete with model, provider, tokens_in, tokens_out and cost_usd. Its metadata holds the citations, the tool_calls and what the context used.

To cancel, pass { signal } as the second argument of run and abort it, or leave the for await loop. The server stops generating. The message keeps the partial text and is marked error with "The client disconnected." If the provider fails part-way, the message is marked error the same way.

To resume after a reload or on another device, read the messages:

const { data: messages } = await client.ai.threads.from(threadId).messages();
const last = messages.at(-1);
if (last?.status === 'streaming') {
  // still being generated: listen on the realtime channel below, or poll messages({ after })
}

// Only messages created after a timestamp, oldest first:
await client.ai.threads.from(threadId).messages({ after: last.created_at });

Realtime mirror

Every event of a run is also broadcast on the realtime channel broadcast:ai:thread:<thread id> with the event name ai, so another tab or device can follow a reply it didn't start. Appending a message broadcasts { type: "appended", message }, and a failed run { type: "error", messageId, message }.

client
  .channel(`ai:thread:${threadId}`)
  .on('broadcast', { event: 'ai' }, ({ payload }) => {
    if (payload.type === 'delta') appendText(payload.text);
  })
  .subscribe();

final ch = nestcore.channel('ai:thread:$threadId')
  ..onBroadcast(event: 'ai', callback: (msg) => print(msg.payload));
await ch.subscribe();

Only the thread's owner (with their access token) and the service-role key can join. Anyone else's subscription is refused with a forbidden error and receives nothing. See Realtime.

Messages

Content is a string (up to 200,000 characters) or 1 to 50 parts:

• { type: 'text', text }
• { type: 'image', url } with a data: URL of about 1 MB at most
• { type: 'image', storage: { bucket, path } } for a larger image in project storage (user messages only). It is read when the thread runs, with the caller's storage access: png, jpeg, webp or gif up to 5 MB. An image that can't be read is replaced by a short note to the model.

A thread's title is taken from its first user message (up to 80 characters) unless you set one.

Threads

Tokens and cost

Each assistant message records tokens_in, tokens_out and cost_usd for the whole run: every model call of every tool round, plus the embedding calls made to look up memories and sources. It matches the usage ledger entries tagged with that message, so the thread totals in threads.list are what the thread cost on your key.

Access rules

Deleting one of your app's users also deletes the threads they own.

Limits

• A run counts once against your per-minute AI rate limit, however many model, tool and embedding calls it makes. Spend caps are checked before each call.
• Up to 4 tool rounds, 5 collections and 6 sources per run.
• Each streamed model call can run for up to 120 seconds, as in the AI gateway.
• Memory lookups count as memory operations.

In Studio

Project Settings → AI → Threads lists threads with their owner, message count, last update and cost. Open one to read its messages with the model, tokens and cost of each reply. Deleting a thread needs the workspace admin role.

HTTP API

Under /api/project/{slug}/ai/v1. Send the anon key in apikey and the user's access token in Authorization: Bearer, or the service-role key.

curl -N "https://api.nestcore.io/api/project/my-app/ai/v1/threads/$THREAD_ID/run" \
  -H "apikey: $NESTCORE_ANON_KEY" \
  -H "Authorization: Bearer $USER_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input": "How do refunds work?", "collections": ["handbook"]}'

Errors

Runs can also fail with the AI gateway errors (AI_PROVIDER_NOT_CONFIGURED, AI_BUDGET_EXCEEDED, AI_RATE_LIMITED and the rest) and with QUOTA_EXCEEDED when memory is used past the monthly allowance.