Pradra StudioDocs
Sign in

Conversations & streaming

Two features that are usually wanted together: an assistant that remembers the last question, and an answer that appears as it is written instead of after a silent pause.

Step 1 — Open a session per end user

const chat = await pradra.sessions.create({
  capabilityId,
  externalRef: "shopper_991",                          // YOUR user id
  principal: { tenant_id: "store-a", user_id: "u_913" }, // stamped once, forever
  title: "Order help",                                  // optional
});

Two fields decide whether this is safe:

externalRef is your own identifier for the person talking. Without it you have one session shared across every customer, and one customer's conversation becomes another's context. Your end users never become Pradra accounts.

principal is fixed at creation and cannot be changed afterwards — the API refuses an attempt rather than ignoring it, because silently dropping it would let you believe a conversation had been re-scoped to a different customer when it had not. To converse as someone else, open a new session.

Sessions work with generative capabilities and with agents. A classic ML capability is refused: it has no notion of a previous turn.

Step 2 — Send turns

await pradra.sessions.send(chat.id, { message: "Where is order 12345?" });
await pradra.sessions.send(chat.id, { message: "And can I still cancel it?" });

The second question resolves because the transcript is replayed into the prompt. The keys are the capability's contract fields, exactly as in a plain predict.

A generative capability answers synchronously. An agent capability answers { status: "accepted" } with a run_id in outputs — the same asynchrony agents.run has, not a new one. Poll it with pradra.agents.wait(runId).

You can also pass the session to the ordinary calls, which is the same thing:

await pradra.predict(capabilityId, inputs, { sessionId: chat.id });
await pradra.agents.run(capabilityId, { goal, sessionId: chat.id });

Omit sessionId and the call is stateless — no history is loaded, nothing is written. That is exactly the pre-session behaviour, unchanged.

Do not pass principal alongside sessionId. A session already has one, and supplying a second is refused (422) rather than merged — otherwise one request mid-conversation could re-point the agent, and every tool it calls, at another customer's data.

Step 3 — Read the transcript back

const messages = await pradra.sessions.messages(chat.id);
// [{ role: "user", content: "…" }, { role: "assistant", content: "…", cost_usd: … }]

const mine = await pradra.sessions.list({ externalRef: "shopper_991" });

Use this to rebuild your chat UI after a page reload. Messages come in replay order.

What a turn actually remembers

Replaying everything would eventually be impossible, so the replay is bounded twice — a message ceiling and a character budget. Either alone fails in the opposite direction: a message count lets a few enormous turns blow the prompt, and a character budget alone lets many tiny turns be trimmed to nothing. The newest turns always survive.

Older turns fold into a rolling summary, regenerated off the request path. If the summarizer is unavailable the conversation keeps working and only loses distant memory — an outage should not end conversations.

For agent turns, a digest of what the tools returned is kept with the turn. Without it, "what was the ETA you found?" is unanswerable whenever the number appeared in a tool result but not in the prose answer.

Full detail: Conversations.

Step 4 — Stream the answer

for await (const ev of pradra.predictStream(capabilityId, { message: text }, {
  sessionId: chat.id,
})) {
  switch (ev.type) {
    case "delta":  process.stdout.write(ev.text); break;
    case "done":   console.log("\n", ev.usage, ev.cost_usd); break;
    case "error":  console.error(ev.code, ev.message); break;
  }
}

Agent runs stream the same way, with progress events:

for await (const ev of pradra.agents.stream(capabilityId, { goal, sessionId: chat.id })) {
  if (ev.type === "state")       showStatus(ev.phase);        // retrieving | calling_tool | generating
  if (ev.type === "tool_call")   showStatus(`calling ${ev.tool}`);
  if (ev.type === "tool_result") showStatus(`${ev.tool} ${ev.is_error ? "failed" : "ok"}`);
  if (ev.type === "delta")       append(ev.text);
  if (ev.type === "approval_required") showApproval(ev.tool, ev.args, ev.run_id);
}

Streaming is a transport, not a second path: every gate applies in the same order and from the same code. An exhausted budget arrives as an error event with code RATE_LIMITED and zero text.

Step 5 — Handle the awkward cases

Hanging up. break out of the loop, or pass an AbortSignal. For a generative stream that really stops the generation, so you stop paying for text nobody will read.

const ac = new AbortController();
setTimeout(() => ac.abort(), 30_000);
for await (const ev of pradra.predictStream(id, inputs, { signal: ac.signal })) { … }

An agent run does not stop when you disconnect. Its trace is run state, and a write that already fired cannot be un-fired by closing a socket. The run finishes and is recorded; what a disconnect costs is the answer, which you collect with pradra.agents.get(runId).

A parked approval closes the stream cleanly. That is not a dropped connection — a park can wait a day for a human, and a socket held that long is a leak. Show the approval, then poll or wait for the decision.

tool_result carries no payload, only name, outcome and duration. A tool result can be large and can hold data that must not reach the person watching. Do not build a "show result" affordance: the data is not on the wire. approval_required is the deliberate exception — it carries the proposed arguments, because an approver has to see exactly what will execute.

Errors before the stream opens are ordinary errors. A malformed request throws ApiError from the first await; only after the response has begun do failures arrive as an error event. Handle both.

Step 6 — If you proxy this to a browser

Your server holds the API token, so the stream reaches the browser through you. Two things to get right:

  • Disable response buffering for the proxied route. Most proxies buffer by default, which holds every chunk until the response ends: correct code, and the user still sees the answer appear all at once.
  • Buffer to a blank line if you write your own parser. Frames split anywhere across network chunks, including mid-character; parsing per chunk is the classic way to lose the end of an answer.
  • Hang up by cancelling the reader, not by leaving the loop. A reader already waiting on the network stays parked until the server sends something, so a stop built on the loop alone does nothing on a stream that is thinking.

Next