# AI requests, context and speech

![Akari listens to a continuous voice waveform in a Gynara sound studio.](/images/engineering/gynara-v2/akari-speech.webp)

AI surfaces share provider capabilities but retain different product contracts. A private Gynoid's
character card governs that character; it must not inherit the site-assistant Akari persona.

## Shared inference machines

For the boundary between the website, Pinokio's app environment, ChatterBox and GPU execution,
read [Local AI hosting with Pinokio and ChatterBox](pinokio-and-chatterbox.md). It explains the
verified RTX 5090 CPU-fallback repair, matched PyTorch packages and per-host deployment checks.

AI Provider Settings offers a one-click **Development PC / Home Lab** switch for direct text and speech.
The switch saves immediately; Development PC endpoints have their own collapsible editor.
**Test all nodes** checks every configured Home Lab service from the website host; each machine also
has **Test node**. Results show readiness, response time, a check timestamp and details such as an
unreachable host, missing shared model or missing speech API. These checks generate no member replies
or audio and do not change routing. They show a point-in-time API result, not proof of GPU use or a
successful generation. Slow responses are flagged when they exceed the routing readiness deadline.
Development PC uses its two editable endpoints; registry selection and pooling are bypassed together.
Home Lab uses enabled registered capabilities. Switching preserves node membership and the shared model,
and takes effect for new requests without restarting. All nodes, including disabled machines, remain
visible with enable/disable controls and an editor for replacement hosts, endpoints, capabilities,
priorities and capacity. Enabled is an administrative choice, not a live health result.

`IInferenceRoutingSettingsStore` owns the shared selection; its atomic file implementation stores only
the current record in `App_Data/ai-providers/routing.json`, with a cross-process lock and revision check.
Before the first save, `AiProvider:UseRegisteredNodes` defaults to true; the Development endpoints use
`LmStudio:LocalBaseUrl` and `AiProvider:DevelopmentChatterboxUrl` (default loopback ports 1234/v1 and 7860).
All hosts must share this private application-data root. Loopback means the application host's computer.
Hermes gateways retain their separate connections. Turning off load balancing alone does not bypass
registry selection. This workflow makes no schema changes or recovery snapshots.

Direct LM Studio and ChatterBox requests use separate pools. The shipped priority strategy prefers
each service's lowest-numbered enabled node, then tries ready fallbacks when it is busy or unavailable.
The primary becomes preferred again when its capacity or readiness returns. Busy nodes are skipped, with one active request per currently
registered machine. A slot stays reserved through the complete streamed answer or whole speech
generation. Readiness failures can select another machine before generation starts; an interrupted
generation is not automatically replayed elsewhere. Cached audio does not occupy a speech slot.
All text nodes must offer the site-wide model. Speech keeps the selected character's exact reference,
English language and Turbo settings on either node. A healthy busy speech node is available to queue;
one offline speech node does not make the entire pool unavailable.

Text now separates waiting for capacity from waiting for network progress. By default each local
priority or shared-node capacity wait ends after 60 seconds for private chat, and an unanswered
provider connection or stalled response ends after 120 seconds
without further data. An actively arriving reply has no new total-duration ceiling. A stalled request
releases its node slot and does not become a completed paid reply. A provider-confirmed complete reply
survives a missing trailing usage message. These limits prevent indefinite waits; they cannot repair
an unreachable machine or its network. Speech availability checks eligible nodes concurrently so a
silent primary cannot hide a responsive fallback.

The November 5 configuration prefers Beelink for text inference, with the two Aurora nodes as
fallbacks, and the RTX 5070 Laptop node for primary speech. Three warmed
128-token text samples measured about 35 tokens/second on Beelink
versus about 20 on either other text node. Three identical speech fixtures with the same Akari reference
and Turbo settings completed in a median 1.80 seconds on the 5070 and 2.44 seconds on the 4060 Ti.
These small samples describe this installation, not general hardware rankings or latency guarantees.
Private operational records retain endpoints and deployment checks.

Drakima owns database access. It validates the member, reads authorized history/lore and assembles the
existing prompt, sends that request to a model node, then validates and stores the answer centrally.
Core and Cognition connection strings stay on the application hosts. Moving from the Development PC
to the Web Server changes the application deployment and database connections, not the model's access
rights. No direct PostgreSQL login is needed on an inference machine. Hermes gateways are separate
services with their own internal provider routing; this direct-provider pool does not route their calls.

The application pool contracts live in `InferenceNodePoolContracts`; `AiProviderSettingsService`
reads current membership and `HealthyInferenceNodePool` owns readiness, rotation and capacity leases.
Prompts, private character identities, speech purchases and completed-reply storage keep their existing owners.

## Conversation flow

Private-room admission validates the owned chat, feature availability, selected Lore and price before
inference. The selected conversation sends every completed message, in order and unchanged. The
provider owns context capacity; exhaustion preserves history and refunds the reservation rather than
silently dropping messages. Queueing/generation can be cancelled by the member and renews its reservation.

Private replies stream ordinary prose. An optional terminal `[drakima-control]` footer carries display
and explicit memory intent. The stream parser withholds the footer across chunks; invalid controls fall
back locally without another model call. A completed `stop` or `length` result is retained once. An
interrupted stream is not treated as a completed paid answer.

Community-thread replies on Journal posts, videos and novel chapters let LM Studio determine their
length (`max_tokens: -1`). They have no site output-token, character, paragraph, response-byte or
generation-time ceiling. Model/context capacity and caller cancellation still apply. A generation
heartbeat keeps a long-running queue claim alive; a provider `length` finish is treated as incomplete
instead of publishing a fragment. The full candidate reaches policy review and is retained through
drafting and publication. The member's own comment composer keeps its existing 2,000-character limit.

Core migration `20261028190835_AllowFullCommunityReplies` widens `ContentComments.Body` and
`CommunityThreadParticipationRequests.DraftBody` from `varchar(2000)` to `text`. Apply this reviewed
migration before using the updated worker. It changes no existing message text. Rollback refuses if
longer replies exist, rather than truncating them. The Studio draft editor also accepts the full reply.
On October 28 the owner explicitly waived the backup/restore-drill prerequisite for this two-column
expansion. It was applied to the existing `DrakimaCoreDb` in one transaction; before/after row digests
confirmed unchanged comment and draft contents. No backup or additional database was created.

Other roles retain their own bounded schemas and context budgets. Ordinary Guide/Companion turns use
a compact identity and bounded conversation context. Explicit research routes use their separate
evidence workflow only when admitted. Research defaults off for Beta. Member video generation is retired; TTS and curated Private Videos remain available.
Display emotion/action does not update private-room Mood.

Authored VN Hybrid openings and replies preserve who performed an established action. A bounded,
server-projected scene reference and attribution rule remain present after Admin template rendering,
including when the template omits ordinary role context. Public bridges quote their frozen handoff,
visible stage labels, and a bounded excerpt of the shared Entry prefix before its first authored choice;
later scene bridges can retain that opening's attributed facts without reconstructing a chosen branch.
The reference stops before the current bridge, follows an earlier optional bridge's frozen return,
and excludes future dialogue. Unknown actions and environmental details remain unspecified;
generated dialogue does not become authored canon. This does not reconstruct unvisited branches,
change immutable releases, rewrite saved dialogue, or alter optional-AI pricing. Private-room scene
metadata does not enable this VN-only contract.

An empty provider response to a schema-guided JSON request gets one compatibility retry with the
same prompt, model, limits and overall cancellation deadline, without provider-side grammar. The
usual presentation and expression checks still validate the reply. Repeated empty output keeps the
existing recoverable fallback; ordinary private-room prose does not take this retry path.

Admin Operations cancellation and execution cleanup share one process-level state gate. Completion
publishes its terminal record before removing the active entry. A cancellation racing with cleanup
returns acknowledgement or the terminal disposition (`Completed` for a cancelled execution, `TooLate`
otherwise), including when it already captured the active object. A completed execution never cancels
its disposed source. This coordination belongs to `InMemoryAiInteractionExecutionCoordinator`;
durable cancellation receipts remain owned by the cancellation registry.

## Published page evidence

New chat on the site widget and full Akari page clears the current visit, draft, retry action and
speech playback. Old history/response callbacks cannot restore a cleared visit. Previous conversations
remain stored. Shared visit receipts and connection labels live in `akari-visit-state.js`; each surface
retains its own presentation and request lifetime. Speech generation keeps Stop available while waiting,
and an unavailable provider reports a retryable failure without charging for failed generation.

Admitted Studio page releases contribute public Guide evidence. Akari resolves their exact canonical
path and current authored text, while dynamic blocks link to their canonical content owners. Retrieval
rechecks release, text/owner-link fingerprint, serving switches, image integrity and dependencies.
Private drafts never become page evidence; withdrawal withholds old indexed rows immediately, even
when notification delivery or periodic reconciliation is delayed.

## Personal memory

Memory is independent of Thinking and Research. Explicit saves go through consent, evidence and bounded
validation. Ordinary turns do not automatically load saved personal memories. Recall searches only the
current account and Gynoid's records. A server-issued saved-memory receipt confirms persistence;
generated dialogue saying “I remember” does not.

## Speech is a separate operation

Public chat composers show the shared response-cost legend: text costs 1 credit,
Thinking costs 2 instead of the text price, and optional TTS costs 1 additional
credit per complete response. Available modes differ by surface. The legend also
appears in the live dialogue for A Date with Akari and the Tutor; ready audio
replays free. Prices come from the same Application constants as settlement.
The visible legend contains only the price row, centered inside the Meet the
Gynoids chat widget. A Date with Akari's invitation uses “Talk with AI” without a
price suffix; its separate free-story button retains its label.
Its hybrid invitation and catalog card show only Text and TTS prices because the
game does not offer Thinking or Research modes. The live conversation keeps the
account balance but does not repeat the prices already shown in the invitation.

Identical uncached ChatterBox requests now wait on a process-local recording gate
before reserving an inference node. After the first request writes its WAV, the
waiter reuses it. Different recordings can still run across the pool concurrently.
Cancelling a waiter does not cancel the recording owner; cancelled/failed owners
release the gate so another request can retry. Gate entries are removed when their
last owner/waiter leaves. Each successful render logs only the node key, character
count, queue time and rendering/cache-maintenance time, never the spoken text.
This reduces duplicate work under contention; it does not speed up a unique GPU
inference or coordinate duplicate misses across separate web processes.

### Speech speed review — November 5, 2026

The Development PC's separate provider environment was subsequently repaired and verified with
CUDA speech generation. The [Pinokio/ChatterBox guide](pinokio-and-chatterbox.md) records the
compatibility lesson and the distinction between cold loading and warm generation. That result
does not update or certify the registered Home Lab machines discussed below.

All public speech paths converge on `ChatterboxAkariVoiceService`, including private
rooms, site-wide Akari, Ask Akari, Tutor/Interview and hybrid VN dialogue. The current
two enabled speech nodes both select Turbo and allow one simultaneous job each.
The configured priority strategy now prefers the primary and uses available fallbacks; it does not
continuously rank nodes by measured latency.
Keep voice references, text completeness, cancellation and paid-ready receipts intact.

The prior provider benchmark in [operations](operations.md) measured medians of
3.33 seconds on Node 2 and 8.87 seconds on Node 5 for the same short sample. Uploads
took only 53–66 ms; the optional reference bridge did not establish a consistent
end-to-end improvement. These are historical samples, not a new beta speed claim.
The useful next experiments are:

- Compare current cold/warm and concurrent latency by node using the new timing logs;
  then evaluate faster-node-first speech routing with the slower node as overflow.
- Keep Turbo resident on the GPU and investigate competing GPU workloads on the
  slower host before changing quality/sampling settings. The upstream
  [ChatterBox documentation](https://github.com/resemble-ai/chatterbox) identifies
  Turbo as the lower-compute 350M English model; this installation already uses it.
- For long replies, evaluate early passage playback through the existing progressive
  speech contract. Private-room speech currently waits for the joined complete WAV.
  This would reduce time to first audio, not total inference time, and requires
  dedicated cancellation, continuity and failed-passage settlement acceptance.

The duplicate-work fix does not change node selection, remote provider installations,
voice identity, passage lengths or the existing whole-response payment contract.

Members explicitly generate speech for a completed owned answer or opt into Auto speech. It costs one
credit for the complete response, including every packet. Ready audio replays without another charge.
Authored room greetings, farewells, reactions and story recordings have their separate room-audio setting.
Completing text never implicitly authorizes a speech purchase.

Private-room speech availability first checks the owned reply's paid receipt. Uncached replies also
check the currently routed ChatterBox provider through the existing connection-test contract. The
process-level `ChatterboxSpeechAvailability` coalesces checks and caches their result for ten seconds;
one departing member cannot cancel other readers. This performs no synthesis or credit operation.
An offline provider shows **TTS Offline** beside **Check speech availability**. Text chat and saved
audio remain available. A successful retry only restores **Generate speech**; purchasing still needs an
explicit click or the existing Auto speech opt-in. Cached replay and prerecorded room audio remain
available independently. Delayed checks cannot start Auto speech after another room action cancels it.

Private Gynoid chat speech begins after the completed answer is displayed. Each new generation gives
the entire completed reply to the speech provider and plays one continuous recording. The provider
renders sentence-aware passages of at most 240 characters sequentially, then joins their PCM/float
samples into a single WAV. Turbo's 1,000-token generation ceiling otherwise cuts off long replies at
roughly 40 seconds and long inference can degrade delivery. Every passage retains the same selected
reference, model, language and sampling settings; direct Gradio uploads the reference once per reply.
The reference bridge uses the same passage limits. No browser packet limit truncates the reply's
65,536-character transcript ceiling. Whole-reply generation retains its five-minute deadline and
cancellation. A failed passage, incompatible audio format or oversized combined recording fails the
whole operation without a partial ready result or speech charge.
Existing complete receipts replay their original audio, including older packet-based recordings;
they are not regenerated or charged again. Standard and VN room widgets share this playback owner.

Stop remains available during initial generation and playback. New prompts, new chats, Lead/moment
changes, room exit and the other existing speech interruptions abort the stream, clear waiting audio
and prevent late callbacks from restarting it. Autoplay restrictions pause the current packet for
**Start voice**. Provider cancellation remains cooperative; no further packets are requested after
cancellation. Incomplete or failed generation creates no complete receipt or speech debit. A completed
generation still costs one credit for the whole answer, and its ready audio replays for free.

The Gynoid feature's request-scoped `ProgressiveReplySpeech` port publishes provisional packets while
the existing `PaidReplySpeechService` retains the complete receipt and idempotent debit.
`PaidSpeechStreamResult` sends private NDJSON
audio frames from trusted server-resolved files, without accepting browser text or paths.
`gynoid-speech-queue.js` owns ordered, bounded playback; the room's existing abort owner cancels it.

The shared ChatterBox planner removes HTML presentation tags/comments and decodes entities before
synthesis. It preserves both dialogue and narration. `<center>` must never become the spoken
word “center.” Markdown presentation and control metadata stay out of speech while the saved transcript
remains unchanged. Other speech surfaces retain their existing packet planners and limits.
Existing audio bytes are not rewritten by a text-cleanup update.

Speech input also removes complete emoji graphemes before synthesis and cache-key calculation.
`Services/Akari/SpeechEmojiFilter.cs` recognizes Unicode 17 emoji, including flags, skin tones,
keycaps and joined sequences, without stripping ordinary Unicode words or standalone numbers.
Emojis remain in the saved and displayed chat. Whole replies still use one application-level provider
operation and one continuous recording; internal inference passages never discard text. An emoji-only
answer has no spoken text. Previously purchased audio remains its existing recording.

Turbo voices can receive one supported, bounded vocal event from reply display intent. Other models
receive no automatic event tags. Stop, new chat, new requests and page exit cancel pending playback;
late completions cannot restart audio. Speech-control movement must not recreate the audio owner.

Implementation evidence: `Services/Akari/ChatterboxAkariVoiceService.cs`, `Services/Gynoids`,
`src/DrakimaCore.GynoidConversations.Application`, `src/DrakimaCore.Infrastructure`,
`Tests/DrakimaCore.Tests/SpeechPresentationTests.cs`,
`Tests/DrakimaCore.Tests/ChatterboxLongReplyTests.cs`.

October 29 Akari investigation verified the installed and uploaded reference against the approved
September 5 pack (`dd5326707f37924d...`), English (`en`) and Turbo. The latest 764-character reply's
cache fingerprint reproduced exactly from those inputs, ruling out another reference or language
for that recording. The provider reached its 1,000-token ceiling; a short English control using the
same reference was confirmed correct by the owner. The repaired 681-character, four-passage recording
also passed the owner's listening check through its final sentence. All live synthesis now uses cache
contract 2 so new operations cannot reuse contract-1 inference output. Existing paid receipts retain
their original audio; playback-only callers prefer contract 2 and may still read legacy contract 1.

This is a shared provider repair, covering every Gynoid, Akari Guide/Companion/Support/Concierge and
Lorekeeper roles, Interview Studio, Admin Operations, VN Maker Copilot, authored VN recordings,
Hybrid VN speech and Studio character voice previews. All resolve through `IAkariVoiceService` to
the single registered `IChatterboxSynthesisProvider`; no surface has a separate live Gradio client.
Their existing admission, voice selection, outer playback plans and billing contracts remain owned
by those surfaces. Short passages use one inference; longer inputs are bounded inside the provider.

VN Maker's standalone Scene Editor layout resolves the same trusted `StudioSpeechAccess` decision
as the regular Studio layout. Creator speech controls therefore remain available in Akari Station
and its grounded Copilot answers; ordinary staff still cannot synthesize speech. Scripted Maker
Guide/Presence cues are prerecorded audio, separate from generated Copilot speech.
Opening Assist and sending its first question share one pending session request. Each generated
answer's speech control keeps that answer's original session binding, including after reconnect;
server checks still require the current creator, project access, completed operation and valid ticket.

Interview Studio invalidates unfinished topic captures and responses when a session is reset.
`interview-answer.js` cancels the previous text reveal on reset or replacement, so it cannot repaint
the new greeting or hide it from assistive technology. Completed answers remain fully readable.
Speech generation uses the configured bounded queue wait (15 seconds by default); simultaneous
requests can exhaust a single Development PC's capacity. A failed generation retains the text,
charges no speech credit and permits an explicit retry after the queue clears.

Lorekeeper's pure request and answer decisions have focused owners under `Services/Akari`:
`LorekeeperRequestPolicy` preserves normalization and operation fingerprints;
`LorekeeperTextPatterns` owns generated culture-invariant patterns; `LorekeeperConversationPolicy`
owns ordinary conversation classification and greeting continuity; `LorekeeperLaneInstructions`
owns lane audience and prompt boundaries; `LorekeeperGroundingPolicy` projects approved sources,
citations and deterministic fallbacks; `LorekeeperWorkroomPresentation` maps an existing Workroom
result into a Lorekeeper answer. These owners perform no provider, filesystem or database operations.
`AkariLorekeeperService` owns common admission, research and grounded-reply orchestration. Companion,
Support, Concierge, Workroom and Interview replies each have a focused `Lorekeeper*Replies` owner;
`LorekeeperLaneReplies` preserves the ordered lane-selection rules and dispatches admitted turns.
`LorekeeperDirectContext` composes the compact audience and character-card prompt.
`LorekeeperGuideReplies` owns ordinary/grounded Guide retrieval, response assembly, citation validation
and completion; `LorekeeperShellReader` owns the existing presentation cache and Admin invalidation key.
The main orchestrator retains common audience/operation/research admission and failure disposition.

`ILorekeeperConversationStore` gives these workflows bounded reads and scoped operation commands without
an EF query surface. Its Runtime-linked `ApplicationDbContextLorekeeperConversationStore` adapter owns
lease claim/renewal, staged answer JSON, transcript completion, owner-bound resume/navigation and the
existing Core credit coordinator. Inference remains outside a transaction. Saved answers, stable credit
keys and optimistic concurrency keep their existing formats and behavior. Feature entities remain
compatibility handles until their model/migration boundary can move without changing retained Arcade
pins. Further work is tracked in the [developer workflow](developer-workflow.md#october-27-architecture-audit-remediation).

LM Studio's `ConversationResponsePolicy` owns completed JSON/prose parsing, bounded control-footer
decoding and local display-intent selection under `src/DrakimaCore.Runtime/Ai`. It performs no provider,
prompt or persistence operations. `LmStudioAiConversationService` retains the same payload construction,
timeouts, circuit leases, streaming, grammar retry and presentation-repair decisions. This extraction
preserves prompt bytes, history, memory admission, parsing precedence and invalid-footer fallbacks.