# Local AI hosting with Pinokio and ChatterBox

Drakima runs the website and its AI providers as separate services. Pinokio manages the local
ChatterBox application; ChatterBox turns approved text into speech. This guide explains those
responsibilities and the GPU compatibility repair verified on November 5, 2026.

## From a conversation to speech

| Component | Responsibility |
| --- | --- |
| Drakima | Member access, conversation context, credit settlement, selected voice, provider routing and audio delivery |
| LM Studio | Language-model inference for configured text requests |
| Pinokio | Installing and launching the provider app with its own Python environment and dependencies |
| ChatterBox TTS | Speech synthesis through the app's Gradio API, using the requested voice reference and model |
| PyTorch and the NVIDIA driver | Running supported model operations on the GPU |

Pinokio is an application manager, not the language model or speech model. Its management page
and a running app's API can have different addresses. Drakima must reach the ChatterBox Gradio
endpoint selected by its routing settings. A visible Pinokio page alone does not prove that speech
generation is ready. See [Pinokio](https://pinokio.co/) and [ChatterBox's upstream project](https://github.com/resemble-ai/chatterbox).

After a member requests speech, Drakima checks the owned answer and speech receipt, prepares text
and the selected voice, and reserves provider capacity when needed. ChatterBox generates audio;
the website stores and returns the completed recording. Ready audio can be replayed without another
generation charge. Meet the Gynoids, site-wide Akari, Engineering Ask Akari, Tutor/Interview and
live visual-novel dialogue use the shared speech services. Authored story recordings have a separate
production and playback lifecycle. See [AI requests, context and speech](ai-and-speech.md).

## Why a working service can still use the CPU

The repaired installation could generate speech and report its model loaded while running on the
CPU. Its PyTorch 2.6.0 CUDA 12.4 wheel did not contain kernels for the RTX 5090's `sm_120`
architecture. The ChatterBox app detected this and deliberately selected `cpu` before loading Turbo.
The warning described a real provider-environment problem; changing website routing or hiding the
warning would not add the missing GPU support.

The RTX 5090 and RTX 5070 both belong to NVIDIA's Blackwell family with compute capability 12.0.
A lower model number does not make the 5070 compatible with that older wheel.
[NVIDIA's capability table](https://developer.nvidia.com/cuda/gpus) identifies both devices.

Keep three versions distinct: the GPU's compute capability, the installed NVIDIA driver, and the
CUDA runtime used by the PyTorch wheel. A system's CUDA Toolkit version is another separate
installation. Installing CUDA 13 by itself does not replace an incompatible Python package.
Newer drivers can run supported older CUDA applications through
[NVIDIA's backward compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/minor-version-compatibility.html).

## The compatibility repair

The following is the verified Windows configuration from the repair, not a claim that these are
the newest packages or a requirement to upgrade every provider to the latest release:

| Component | Verified selection |
| --- | --- |
| App interpreter | Existing Python 3.11.15 virtual environment |
| PyTorch | `2.7.1+cu128` |
| torchaudio | `2.7.1+cu128` |
| torchvision | `0.22.1+cu128` |
| Speech model exercised | ChatterBox Turbo, English |

[PyTorch 2.7 introduced Blackwell support](https://pytorch.org/blog/pytorch-2-7/), and its
[stable wheel instructions](https://pytorch.org/get-started/previous-versions/) include the matched
2.7.1 CUDA 12.8 set. A nightly build was unnecessary for this repair. Matching torch, torchaudio
and torchvision matters because their compiled dependencies must agree.

Pinokio activates its Conda toolchain and then the app's `app/env` virtual environment. Updating
Conda base or creating an unrelated environment would leave the interpreter used by ChatterBox's
launcher unchanged. The repair reused the app environment and updated the launcher's install pins.

There was also an upstream package constraint: ChatterBox 0.1.7 pinned torch and torchaudio to
2.6.0 for Python below 3.14. The reviewed launcher now prepares the same pinned upstream source
with a small, identified package-metadata patch accepting the supported 2.6/2.7 range. It verifies
the downloaded source checksum, retains the model code, and checks the installed dependencies.
This local compatibility adaptation must travel with the provider's launcher when reproducing
the setup; it is not a change shipped by the Drakima website or an upstream ChatterBox release.

The proposed `zlib-wapi` installation was not needed: this ChatterBox environment did not contain
faster-whisper or CTranslate2. A missing DLL in a different application needs its own diagnosis;
it is not a universal Blackwell patch.

## What counts as GPU evidence

Check the selected device in the app's startup log, confirm PyTorch supports the device architecture,
execute a CUDA operation, and then generate real speech through the running API. Correlate that
request with the serving Python process in NVIDIA's GPU monitor. A GPU name, a ready web page,
or “model loaded” on its own is insufficient.

The repaired RTX 5090 installation passed those checks. In one short Turbo test, the first request
took about 32 seconds including initial model loading; the next request took about 1.3 seconds
and produced about 2.6 seconds of audio. These are individual observations, not a latency guarantee
or a general throughput benchmark. Text length, selected model, voice preparation, competing GPU
work and simultaneous requests can change the result.

## Deploying speech separately from the website

Each home-lab provider needs its own environment and real generation check. A successful development
machine repair does not certify another GPU, driver, model or host. Deploy the reviewed provider
launcher and its compatible packages separately from the website, then configure the intended
ChatterBox endpoint and verify it from the web server. `localhost` always means the machine making
the request; on a separate web server it does not refer to a remote speech machine.

Use the **Development PC / Home Lab** switch in Admin AI Provider Settings to save the environment
for both LM Studio and ChatterBox. The choice persists across restarts; new requests use the chosen
environment while running work retains its endpoint. Home Lab prefers the configured primary for
each service and uses ready fallbacks when that primary is busy or unavailable. Keep the existing
node database and shared private `App_Data` routing record with the deployment. The switch selects
services; it does not start Pinokio or LM Studio on remote computers.

In the November 5 home-lab check, the RTX 5070 Laptop speech node completed three warmed fixtures
in a median 1.80 seconds, versus 2.44 seconds on the RTX 4060 Ti. Both used the same Akari reference,
Turbo model, text and settings, and both returned valid audio. This supports this installation's
5070-primary/4060-fallback order; it does not establish a universal hardware ranking.

Keep the chosen model resident when practical and measure cold, warm and queued requests separately.
Drakima can reuse ready recordings and coalesce identical uncached requests within a web process;
those measures avoid duplicate work but do not accelerate a unique model inference. Changing voice
identity, truncating responses or increasing concurrency without measuring GPU capacity is not an
equivalent performance improvement.

## Discuss this topic with Akari

The **Ask Akari** action on this handbook page preselects this chapter. It is also available from the collapsible **Topics** tab on the
left of the Engineering conversation. The selected guide supplies reference material; Akari does
not inspect a member's computer or live provider health from a conversation.

Implementation evidence: `Services/Akari/ChatterboxAkariVoiceService.cs`,
`src/DrakimaCore.Infrastructure/Akari/` and the separately installed provider's `install.js`,
`torch.js`, `app/prepare_chatterbox.py` and `app/app.py`. Provider setup and measured results have
their staff operational record; this public guide contains no machine endpoints or credentials.