AI requests, context and speech

AI surfaces share provider capabilities but retain different product contracts. A private Gynoid's character card governs that character; it must not inherit the site-assistant Akari persona.

Shared inference machines

For the boundary between the website, Pinokio's app environment, ChatterBox and GPU execution, read Local AI hosting with Pinokio and ChatterBox. It explains the verified RTX 5090 CPU-fallback repair, matched PyTorch packages and per-host deployment checks.

AI Provider Settings offers a one-click Development PC / Home Lab switch for direct text and speech. The switch saves immediately; Development PC endpoints have their own collapsible editor. Test all nodes checks every configured Home Lab service from the website host; each machine also has Test node. Results show readiness, response time, a check timestamp and details such as an unreachable host, missing shared model or missing speech API. These checks generate no member replies or audio and do not change routing. They show a point-in-time API result, not proof of GPU use or a successful generation. Slow responses are flagged when they exceed the routing readiness deadline. Development PC uses its two editable endpoints; registry selection and pooling are bypassed together. Home Lab uses enabled registered capabilities. Switching preserves node membership and the shared model, and takes effect for new requests without restarting. All nodes, including disabled machines, remain visible with enable/disable controls and an editor for replacement hosts, endpoints, capabilities, priorities and capacity. Enabled is an administrative choice, not a live health result.

IInferenceRoutingSettingsStore owns the shared selection; its atomic file implementation stores only the current record in App_Data/ai-providers/routing.json, with a cross-process lock and revision check. Before the first save, AiProvider:UseRegisteredNodes defaults to true; the Development endpoints use LmStudio:LocalBaseUrl and AiProvider:DevelopmentChatterboxUrl (default loopback ports 1234/v1 and 7860). All hosts must share this private application-data root. Loopback means the application host's computer. Hermes gateways retain their separate connections. Turning off load balancing alone does not bypass registry selection. This workflow makes no schema changes or recovery snapshots.

Direct LM Studio and ChatterBox requests use separate pools. The shipped priority strategy prefers each service's lowest-numbered enabled node, then tries ready fallbacks when it is busy or unavailable. The primary becomes preferred again when its capacity or readiness returns. Busy nodes are skipped, with one active request per currently registered machine. A slot stays reserved through the complete streamed answer or whole speech generation. Readiness failures can select another machine before generation starts; an interrupted generation is not automatically replayed elsewhere. Cached audio does not occupy a speech slot. All text nodes must offer the site-wide model. Speech keeps the selected character's exact reference, English language and Turbo settings on either node. A healthy busy speech node is available to queue; one offline speech node does not make the entire pool unavailable.

Text now separates waiting for capacity from waiting for network progress. By default each local priority or shared-node capacity wait ends after 60 seconds for private chat, and an unanswered provider connection or stalled response ends after 120 seconds without further data. An actively arriving reply has no new total-duration ceiling. A stalled request releases its node slot and does not become a completed paid reply. A provider-confirmed complete reply survives a missing trailing usage message. These limits prevent indefinite waits; they cannot repair an unreachable machine or its network. Speech availability checks eligible nodes concurrently so a silent primary cannot hide a responsive fallback.

The November 5 configuration prefers Beelink for text inference, with the two Aurora nodes as fallbacks, and the RTX 5070 Laptop node for primary speech. Three warmed 128-token text samples measured about 35 tokens/second on Beelink versus about 20 on either other text node. Three identical speech fixtures with the same Akari reference and Turbo settings completed in a median 1.80 seconds on the 5070 and 2.44 seconds on the 4060 Ti. These small samples describe this installation, not general hardware rankings or latency guarantees. Private operational records retain endpoints and deployment checks.

Drakima owns database access. It validates the member, reads authorized history/lore and assembles the existing prompt, sends that request to a model node, then validates and stores the answer centrally. Core and Cognition connection strings stay on the application hosts. Moving from the Development PC to the Web Server changes the application deployment and database connections, not the model's access rights. No direct PostgreSQL login is needed on an inference machine. Hermes gateways are separate services with their own internal provider routing; this direct-provider pool does not route their calls.

The application pool contracts live in InferenceNodePoolContracts; AiProviderSettingsService reads current membership and HealthyInferenceNodePool owns readiness, rotation and capacity leases. Prompts, private character identities, speech purchases and completed-reply storage keep their existing owners.

Conversation flow

Private-room admission validates the owned chat, feature availability, selected Lore and price before inference. The selected conversation sends every completed message, in order and unchanged. The provider owns context capacity; exhaustion preserves history and refunds the reservation rather than silently dropping messages. Queueing/generation can be cancelled by the member and renews its reservation.

Private replies stream ordinary prose. An optional terminal [drakima-control] footer carries display and explicit memory intent. The stream parser withholds the footer across chunks; invalid controls fall back locally without another model call. A completed stop or length result is retained once. An interrupted stream is not treated as a completed paid answer.

Community-thread replies on Journal posts, videos and novel chapters let LM Studio determine their length (max_tokens: -1). They have no site output-token, character, paragraph, response-byte or generation-time ceiling. Model/context capacity and caller cancellation still apply. A generation heartbeat keeps a long-running queue claim alive; a provider length finish is treated as incomplete instead of publishing a fragment. The full candidate reaches policy review and is retained through drafting and publication. The member's own comment composer keeps its existing 2,000-character limit.

Core migration 20261028190835_AllowFullCommunityReplies widens ContentComments.Body and CommunityThreadParticipationRequests.DraftBody from varchar(2000) to text. Apply this reviewed migration before using the updated worker. It changes no existing message text. Rollback refuses if longer replies exist, rather than truncating them. The Studio draft editor also accepts the full reply. On October 28 the owner explicitly waived the backup/restore-drill prerequisite for this two-column expansion. It was applied to the existing DrakimaCoreDb in one transaction; before/after row digests confirmed unchanged comment and draft contents. No backup or additional database was created.

Other roles retain their own bounded schemas and context budgets. Ordinary Guide/Companion turns use a compact identity and bounded conversation context. Explicit research routes use their separate evidence workflow only when admitted. Research defaults off for Beta. Member video generation is retired; TTS and curated Private Videos remain available. Display emotion/action does not update private-room Mood.

Authored VN Hybrid openings and replies preserve who performed an established action. A bounded, server-projected scene reference and attribution rule remain present after Admin template rendering, including when the template omits ordinary role context. Public bridges quote their frozen handoff, visible stage labels, and a bounded excerpt of the shared Entry prefix before its first authored choice; later scene bridges can retain that opening's attributed facts without reconstructing a chosen branch. The reference stops before the current bridge, follows an earlier optional bridge's frozen return, and excludes future dialogue. Unknown actions and environmental details remain unspecified; generated dialogue does not become authored canon. This does not reconstruct unvisited branches, change immutable releases, rewrite saved dialogue, or alter optional-AI pricing. Private-room scene metadata does not enable this VN-only contract.

An empty provider response to a schema-guided JSON request gets one compatibility retry with the same prompt, model, limits and overall cancellation deadline, without provider-side grammar. The usual presentation and expression checks still validate the reply. Repeated empty output keeps the existing recoverable fallback; ordinary private-room prose does not take this retry path.

Admin Operations cancellation and execution cleanup share one process-level state gate. Completion publishes its terminal record before removing the active entry. A cancellation racing with cleanup returns acknowledgement or the terminal disposition (Completed for a cancelled execution, TooLate otherwise), including when it already captured the active object. A completed execution never cancels its disposed source. This coordination belongs to InMemoryAiInteractionExecutionCoordinator; durable cancellation receipts remain owned by the cancellation registry.

Published page evidence

New chat on the site widget and full Akari page clears the current visit, draft, retry action and speech playback. Old history/response callbacks cannot restore a cleared visit. Previous conversations remain stored. Shared visit receipts and connection labels live in akari-visit-state.js; each surface retains its own presentation and request lifetime. Speech generation keeps Stop available while waiting, and an unavailable provider reports a retryable failure without charging for failed generation.

Admitted Studio page releases contribute public Guide evidence. Akari resolves their exact canonical path and current authored text, while dynamic blocks link to their canonical content owners. Retrieval rechecks release, text/owner-link fingerprint, serving switches, image integrity and dependencies. Private drafts never become page evidence; withdrawal withholds old indexed rows immediately, even when notification delivery or periodic reconciliation is delayed.

Personal memory

Memory is independent of Thinking and Research. Explicit saves go through consent, evidence and bounded validation. Ordinary turns do not automatically load saved personal memories. Recall searches only the current account and Gynoid's records. A server-issued saved-memory receipt confirms persistence; generated dialogue saying “I remember” does not.

Speech is a separate operation

Public chat composers show the shared response-cost legend: text costs 1 credit, Thinking costs 2 instead of the text price, and optional TTS costs 1 additional credit per complete response. Available modes differ by surface. The legend also appears in the live dialogue for A Date with Akari and the Tutor; ready audio replays free. Prices come from the same Application constants as settlement. The visible legend contains only the price row, centered inside the Meet the Gynoids chat widget. A Date with Akari's invitation uses “Talk with AI” without a price suffix; its separate free-story button retains its label. Its hybrid invitation and catalog card show only Text and TTS prices because the game does not offer Thinking or Research modes. The live conversation keeps the account balance but does not repeat the prices already shown in the invitation.

Identical uncached ChatterBox requests now wait on a process-local recording gate before reserving an inference node. After the first request writes its WAV, the waiter reuses it. Different recordings can still run across the pool concurrently. Cancelling a waiter does not cancel the recording owner; cancelled/failed owners release the gate so another request can retry. Gate entries are removed when their last owner/waiter leaves. Each successful render logs only the node key, character count, queue time and rendering/cache-maintenance time, never the spoken text. This reduces duplicate work under contention; it does not speed up a unique GPU inference or coordinate duplicate misses across separate web processes.

Speech speed review — November 5, 2026

The Development PC's separate provider environment was subsequently repaired and verified with CUDA speech generation. The Pinokio/ChatterBox guide records the compatibility lesson and the distinction between cold loading and warm generation. That result does not update or certify the registered Home Lab machines discussed below.

All public speech paths converge on ChatterboxAkariVoiceService, including private rooms, site-wide Akari, Ask Akari, Tutor/Interview and hybrid VN dialogue. The current two enabled speech nodes both select Turbo and allow one simultaneous job each. The configured priority strategy now prefers the primary and uses available fallbacks; it does not continuously rank nodes by measured latency. Keep voice references, text completeness, cancellation and paid-ready receipts intact.

The prior provider benchmark in operations measured medians of 3.33 seconds on Node 2 and 8.87 seconds on Node 5 for the same short sample. Uploads took only 53–66 ms; the optional reference bridge did not establish a consistent end-to-end improvement. These are historical samples, not a new beta speed claim. The useful next experiments are:

• Compare current cold/warm and concurrent latency by node using the new timing logs; then evaluate faster-node-first speech routing with the slower node as overflow.
• Keep Turbo resident on the GPU and investigate competing GPU workloads on the slower host before changing quality/sampling settings. The upstream ChatterBox documentation identifies Turbo as the lower-compute 350M English model; this installation already uses it.
• For long replies, evaluate early passage playback through the existing progressive speech contract. Private-room speech currently waits for the joined complete WAV. This would reduce time to first audio, not total inference time, and requires dedicated cancellation, continuity and failed-passage settlement acceptance.

The duplicate-work fix does not change node selection, remote provider installations, voice identity, passage lengths or the existing whole-response payment contract.

Members explicitly generate speech for a completed owned answer or opt into Auto speech. It costs one credit for the complete response, including every packet. Ready audio replays without another charge. Authored room greetings, farewells, reactions and story recordings have their separate room-audio setting. Completing text never implicitly authorizes a speech purchase.

Private-room speech availability first checks the owned reply's paid receipt. Uncached replies also check the currently routed ChatterBox provider through the existing connection-test contract. The process-level ChatterboxSpeechAvailability coalesces checks and caches their result for ten seconds; one departing member cannot cancel other readers. This performs no synthesis or credit operation. An offline provider shows TTS Offline beside Check speech availability. Text chat and saved audio remain available. A successful retry only restores Generate speech; purchasing still needs an explicit click or the existing Auto speech opt-in. Cached replay and prerecorded room audio remain available independently. Delayed checks cannot start Auto speech after another room action cancels it.

Private Gynoid chat speech begins after the completed answer is displayed. Each new generation gives the entire completed reply to the speech provider and plays one continuous recording. The provider renders sentence-aware passages of at most 240 characters sequentially, then joins their PCM/float samples into a single WAV. Turbo's 1,000-token generation ceiling otherwise cuts off long replies at roughly 40 seconds and long inference can degrade delivery. Every passage retains the same selected reference, model, language and sampling settings; direct Gradio uploads the reference once per reply. The reference bridge uses the same passage limits. No browser packet limit truncates the reply's 65,536-character transcript ceiling. Whole-reply generation retains its five-minute deadline and cancellation. A failed passage, incompatible audio format or oversized combined recording fails the whole operation without a partial ready result or speech charge. Existing complete receipts replay their original audio, including older packet-based recordings; they are not regenerated or charged again. Standard and VN room widgets share this playback owner.

Stop remains available during initial generation and playback. New prompts, new chats, Lead/moment changes, room exit and the other existing speech interruptions abort the stream, clear waiting audio and prevent late callbacks from restarting it. Autoplay restrictions pause the current packet for Start voice. Provider cancellation remains cooperative; no further packets are requested after cancellation. Incomplete or failed generation creates no complete receipt or speech debit. A completed generation still costs one credit for the whole answer, and its ready audio replays for free.

The Gynoid feature's request-scoped ProgressiveReplySpeech port publishes provisional packets while the existing PaidReplySpeechService retains the complete receipt and idempotent debit. PaidSpeechStreamResult sends private NDJSON audio frames from trusted server-resolved files, without accepting browser text or paths. gynoid-speech-queue.js owns ordered, bounded playback; the room's existing abort owner cancels it.

The shared ChatterBox planner removes HTML presentation tags/comments and decodes entities before synthesis. It preserves both dialogue and narration. <center> must never become the spoken word “center.” Markdown presentation and control metadata stay out of speech while the saved transcript remains unchanged. Other speech surfaces retain their existing packet planners and limits. Existing audio bytes are not rewritten by a text-cleanup update.

Speech input also removes complete emoji graphemes before synthesis and cache-key calculation. Services/Akari/SpeechEmojiFilter.cs recognizes Unicode 17 emoji, including flags, skin tones, keycaps and joined sequences, without stripping ordinary Unicode words or standalone numbers. Emojis remain in the saved and displayed chat. Whole replies still use one application-level provider operation and one continuous recording; internal inference passages never discard text. An emoji-only answer has no spoken text. Previously purchased audio remains its existing recording.

Turbo voices can receive one supported, bounded vocal event from reply display intent. Other models receive no automatic event tags. Stop, new chat, new requests and page exit cancel pending playback; late completions cannot restart audio. Speech-control movement must not recreate the audio owner.

Implementation evidence: Services/Akari/ChatterboxAkariVoiceService.cs, Services/Gynoids, src/DrakimaCore.GynoidConversations.Application, src/DrakimaCore.Infrastructure, Tests/DrakimaCore.Tests/SpeechPresentationTests.cs, Tests/DrakimaCore.Tests/ChatterboxLongReplyTests.cs.

October 29 Akari investigation verified the installed and uploaded reference against the approved September 5 pack (dd5326707f37924d...), English (en) and Turbo. The latest 764-character reply's cache fingerprint reproduced exactly from those inputs, ruling out another reference or language for that recording. The provider reached its 1,000-token ceiling; a short English control using the same reference was confirmed correct by the owner. The repaired 681-character, four-passage recording also passed the owner's listening check through its final sentence. All live synthesis now uses cache contract 2 so new operations cannot reuse contract-1 inference output. Existing paid receipts retain their original audio; playback-only callers prefer contract 2 and may still read legacy contract 1.

This is a shared provider repair, covering every Gynoid, Akari Guide/Companion/Support/Concierge and Lorekeeper roles, Interview Studio, Admin Operations, VN Maker Copilot, authored VN recordings, Hybrid VN speech and Studio character voice previews. All resolve through IAkariVoiceService to the single registered IChatterboxSynthesisProvider; no surface has a separate live Gradio client. Their existing admission, voice selection, outer playback plans and billing contracts remain owned by those surfaces. Short passages use one inference; longer inputs are bounded inside the provider.

VN Maker's standalone Scene Editor layout resolves the same trusted StudioSpeechAccess decision as the regular Studio layout. Creator speech controls therefore remain available in Akari Station and its grounded Copilot answers; ordinary staff still cannot synthesize speech. Scripted Maker Guide/Presence cues are prerecorded audio, separate from generated Copilot speech. Opening Assist and sending its first question share one pending session request. Each generated answer's speech control keeps that answer's original session binding, including after reconnect; server checks still require the current creator, project access, completed operation and valid ticket.

Interview Studio invalidates unfinished topic captures and responses when a session is reset. interview-answer.js cancels the previous text reveal on reset or replacement, so it cannot repaint the new greeting or hide it from assistive technology. Completed answers remain fully readable. Speech generation uses the configured bounded queue wait (15 seconds by default); simultaneous requests can exhaust a single Development PC's capacity. A failed generation retains the text, charges no speech credit and permits an explicit retry after the queue clears.

Lorekeeper's pure request and answer decisions have focused owners under Services/Akari: LorekeeperRequestPolicy preserves normalization and operation fingerprints; LorekeeperTextPatterns owns generated culture-invariant patterns; LorekeeperConversationPolicy owns ordinary conversation classification and greeting continuity; LorekeeperLaneInstructions owns lane audience and prompt boundaries; LorekeeperGroundingPolicy projects approved sources, citations and deterministic fallbacks; LorekeeperWorkroomPresentation maps an existing Workroom result into a Lorekeeper answer. These owners perform no provider, filesystem or database operations. AkariLorekeeperService owns common admission, research and grounded-reply orchestration. Companion, Support, Concierge, Workroom and Interview replies each have a focused Lorekeeper*Replies owner; LorekeeperLaneReplies preserves the ordered lane-selection rules and dispatches admitted turns. LorekeeperDirectContext composes the compact audience and character-card prompt. LorekeeperGuideReplies owns ordinary/grounded Guide retrieval, response assembly, citation validation and completion; LorekeeperShellReader owns the existing presentation cache and Admin invalidation key. The main orchestrator retains common audience/operation/research admission and failure disposition.

ILorekeeperConversationStore gives these workflows bounded reads and scoped operation commands without an EF query surface. Its Runtime-linked ApplicationDbContextLorekeeperConversationStore adapter owns lease claim/renewal, staged answer JSON, transcript completion, owner-bound resume/navigation and the existing Core credit coordinator. Inference remains outside a transaction. Saved answers, stable credit keys and optimistic concurrency keep their existing formats and behavior. Feature entities remain compatibility handles until their model/migration boundary can move without changing retained Arcade pins. Further work is tracked in the developer workflow.

LM Studio's ConversationResponsePolicy owns completed JSON/prose parsing, bounded control-footer decoding and local display-intent selection under src/DrakimaCore.Runtime/Ai. It performs no provider, prompt or persistence operations. LmStudioAiConversationService retains the same payload construction, timeouts, circuit leases, streaming, grammar retry and presentation-repair decisions. This extraction preserves prompt bytes, history, memory admission, parsing precedence and invalid-footer fallbacks.