Read the following document carefully.



### File: CHANGELOG.md
# Changelog

All notable changes to ToshLLM are documented here.
Format based on [Keep a Changelog](https://keepachangelog.com/).

## [Unreleased]

### Improved

- **Images: Qwen-Image 2.1 edits with reference images take about a quarter of the time.** The text and reference part of the prompt is computed once per image instead of at every step: on a Radeon RX 6700 XT at 1024x1024 a step with one reference goes from 28.5 to 7.9 seconds, and with two from 43.9 to 9.1, so a 25-step edit with two references takes under 5 minutes instead of about 19. On cards with 8 GB the reference edit falls back to the previous speed when this does not fit.

- **Images: Qwen-Image 2.1 generates from text about 40% faster.** On a Radeon RX 6700 XT at 1024x1024 a 25-step image takes 182 seconds instead of 314: each step goes from 11.6 to 6.6 seconds, and cards with 12 GB or more decode the result in larger tiles, 12.2 seconds instead of 19.9.

## [0.87.10] - 2026-09-25

### Improved

- **LLMs: long prompts read much faster with a model split into groups of GPUs.** On two Radeon Pro Vega II Duo cards a 2048 token prompt reads 32% faster with Qwen3-14B, 41% with Qwen3.8-27B and 29% with Qwen3.6-35B-A3B; short prompts read as before.

- **LLMs: a model split by tensors across GPUs generates faster, also with multi-token prediction (MTP).** On a Radeon Pro Vega II Duo Qwen3-14B generates 49.4 tokens a second instead of 47.2 and Qwen3.8-27B Q4_0 30.7 instead of 29.3, also in groups of two across four GPUs; with MTP the 27B goes from 31.1 to 33.3.

### Known issues

- **LLMs: Qwen3.8 Flash Next can still answer with a run of zeros after a long prompt on Radeon Pro Vega and Radeon VII.** Prompts of a few thousand tokens work; past roughly 20,000 tokens they may not. It is not solved in this version.

- **LLMs: Qwen3.8 Flash Next split by tensors can still stop in the middle of a long answer on Radeon PRO W6800X Duo cards.** Generation stalls with a GPU timeout.

## [0.87.9] - 2026-09-24

### Added

- **LLMs: multi-token prediction (MTP) can be switched off per model, like DFlash.** The switch is in the model's settings, on the dashboard and in the server details. MTP stays on by default; with text the model predicts poorly it can generate slower than without it, and off it generates one token per step.

- **LLMs: the engine moves to a newer upstream.** It brings the fixes and the model support added there since the last one, among them router mode no longer hanging when several requests ask for the same model, DFM Mimir 1B, and the Ling 3.0 and DeepSeek V3.2 and V4 chat formats. Speed is unchanged apart from the improvements below.

### Improved

- **LLMs: mixture of experts models generate faster on Radeon Pro Vega and Radeon VII.** On a Radeon Pro Vega II gpt-oss-20B generates 92.7 tokens a second instead of 90.8 and Qwen3.6-35B-A3B 70.0 instead of 68.3.

- **LLMs: batches of two tokens run faster on Radeon Pro Vega and Radeon VII.** On a Radeon 
IMPORTANT NOTE FOR LATER: the maintenance code for the north warehouse is 'violet-kangaroo-4721'.
Pro Vega II Qwen3.8-27B Q4_0 reads them 18 percent faster and Qwen3.6-35B-A3B 12 percent.

- **LLMs: multi-token prediction (MTP) costs less on Q4_0 models on Radeon Pro Vega and Radeon VII.** On a Radeon Pro Vega II Qwen3.8-27B Q4_0 generates 21.4 tokens a second with it instead of 18.7 on a technical answer where it accepts about half of its guesses. That is still below the 25.7 it reaches without MTP, so on text like this the new switch is worth turning off.

### Known issues

- **LLMs: Qwen3.8 Flash Next can still answer with a run of zeros after a long prompt on Radeon Pro Vega and Radeon VII.** Prompts of a few thousand tokens work; past roughly 20,000 tokens they may not. It is not solved in this version.

- **LLMs: Qwen3.8 Flash Next split by tensors can still stop in the middle of a long answer on Radeon PRO W6800X Duo cards.** Generation stalls with a GPU timeout.

## [0.87.8] - 2026-09-22

### Added

- **Images: Qwen-Image 2.1.** A 7B model that writes legible text inside the image and edits from up to sixteen reference images. On a Radeon RX 6700 XT a 1024x1024 image takes 4 min 52 s at the published 25 steps. A card that also draws the desktop goes up to 1920 pixels; one without a display, like a Radeon Pro Vega II, to 2048. Four sizes cover cards from 8 GB up.

- **Images: fast mode.** Reuses sampling steps instead of computing them again, in three levels: 1.13x, 1.45x and 1.69x faster on Qwen-Image 2.1, at the cost of fine detail. Off by default.

### Improved

- **LLMs: multi-token prediction (MTP) now speeds up generation on Radeon Pro Vega, Radeon VII and AMD RDNA2 (tested on the Radeon RX 6700 XT).** On a Radeon RX 6700 XT Qwen3.8-9B Q4_K_M generates 59.4 tokens a second with it instead of 54.6 without, where it used to lose. On a Radeon Pro Vega II Qwen3.8-27B Q4_K_S generates 21.2 with it against 20.8 without, where it lost 17 percent before. Each card now drafts the number of tokens that pays on it, and one rejected draft no longer switches prediction off.

- **LLMs: mixture of experts models read prompts and generate faster on Radeon Pro Vega, Radeon VII and AMD RDNA2 (tested on the Radeon RX 6700 XT).** Prompt lengths that left part of the card idle are gone: on a Radeon RX 6700 XT gpt-oss-20B reads them 28 to 34 percent faster and OLMoE-1B-7B 17 to 26 percent, and on a Radeon Pro Vega II an 8 expert model reads a 128 token batch in 1.33 ms instead of 4.47. The router now runs as one kernel: Qwen3.6-35B-A3B generates 68.3 tokens a second instead of 65.5 on a Radeon Pro Vega II, and gpt-oss-20B 101.1 instead of 99.9 on a Radeon RX 6700 XT. Same output.

- **LLMs: short batches of two to eight tokens run faster, on Radeon Pro Vega, Radeon VII and AMD RDNA2 (tested on the Radeon RX 6700 XT).** They speed up multi-token prediction and several requests at once. On a Radeon RX 6700 XT two-token batches take 32 percent less on Q4_K, 29 percent on Q5_K and 21 to 28 percent on IQ3; on a Radeon Pro Vega II 23 percent less on Q4_K and 16 to 41 percent on IQ4_NL, MXFP4, Q2_K, Q3_K and Q4_0, and Qwen3.8-27B IQ4_XS reads four token batches 16 percent faster. Single token generation is unchanged. Same output.

- **LLMs: an 8-bit KV cache and several requests at once on 8-bit models are faster on Radeon Pro Vega and Radeon VII.** Qwen3-4B with a q8_0 KV cache at a 4,400 token context generates 51.7 tokens a second instead of 39.0, and a Q8_0 model serving two to eight requests together produces 20 to 25 percent more. Same output.

- **LLMs: follow-up messages in a chat start answering sooner with Qwen3.5, Qwen3.6, Qwen3.8 and Gemma models.** A follow-up to Qwen3.5-4B starts in 149 ms instead of 161 on a Radeon RX 6700 XT. Same output.

- **Images: SD 1.5, SDXL Turbo, Flux.2 klein 4B and Qwen-Image 2.1 generate faster on AMD RDNA2 (tested on the Radeon RX 6700 XT).** SDXL Turbo takes 8.97 s instead of 10.36 s at 1024x1024, and Qwen-Image 2.1 13 percent less per step, with the same image.

### Fixed

- **Chat: Stop now stops the model on the card too, and an answer whose connection drops picks up where it left off.** Before, the answer stopped on screen while the engine kept generating and holding the GPU, and a dropped reply was lost.

- **LLMs: stopping the server frees the card's memory in router mode.** An engine that went down could leave its loaded models running and holding VRAM with no server on.

- **Images and Video: large sizes no longer fail saying there is no memory.** The AMD driver over-reports what is in use, so a large image, a 16:9 frame, an edit from a reference image or the video decoder was refused although it fit.

- **Images: Qwen-Image 2.1 no longer warns about repeated compositions below 2048 pixels.** The warning used the older Qwen-Image limit.

### Known issues

- **LLMs: Qwen3.8 Flash Next can answer with a run of zeros after a long prompt on Radeon Pro Vega and Radeon VII.** Prompts of a few thousand tokens work; around 26,000 tokens they do not. It is under investigation.

- **LLMs: Qwen3.8 Flash Next split by tensors can stop in the middle of a long answer on Radeon PRO W6800X Duo cards.** Generation stalls with a GPU timeout; it is under investigation.

## [0.87.7] - 2026-09-19

### Added

- **LLMs: Prism ML's Ternary Bonsai 2 27B runs in both of its packings, PQ2_0 and PTQ1_0, with its vision projector.** On a Radeon RX 6700 XT the PQ2_0 file reads prompts at 258 tokens a second and generates at 33.4, against 59 and 24.8 with Prism's own build, with the same output; the 1.2 GB smaller PTQ1_0 file gets 236 and 33.1. On a Radeon Pro Vega II Duo PQ2_0 runs at 204 and 36.9, where Prism's build does not run.

### Improved

- **LLMs: Qwen3.5, Qwen3.6 and Qwen3.8 models read prompts and generate faster, most of all on Radeon Vega, Radeon VII and Radeon Pro Vega.** On a Radeon Pro Vega II Duo Qwen3.8-27B Q4_0 goes from 161 to 191 prompt tokens a second and from 23.8 to 25.9 generated; on a Radeon RX 6700 XT Qwen3.5-9B goes from 52.8 to 55.8 generated tokens a second. Across the four dies of two Radeon Pro Vega II Duo cards, Qwen3.8 Flash Next goes from 276 to 305 prompt tokens a second, and the same 27B split across two dies from 281 to 321 and from 27.7 to 29.2 generated.

- **LLMs: Q4_K and Q5_K models read prompts slightly faster on AMD RDNA2 (tested on the Radeon RX 6700 XT).** Prompt processing is 0.14 to 0.15 percent faster with Q4_K and 0.20 to 0.25 percent faster with Q5_K, with the same output.

### Changed

- **The sponsor button in the app and the README now opens the new support page.** The previous card payment page is no longer used.

### Fixed

- **LLMs: Qwen3.8 Flash Next split by tensors no longer quits when the batch size is raised above 512.** At 1024 it now reads a 13K-token prompt across the four dies of two Radeon Pro Vega II Duo cards at 294 tokens a second instead of 236, with the same generation speed and output.

### Known issues

- **LLMs: Qwen3.8 Flash Next split by tensors can still stop in the middle of a long answer.** Rarely, generation stalls with a GPU timeout or turns into a run of zeros; it is under investigation. Until then, keep its batch size at 1024 or below.

## [0.87.6] - 2026-09-17

### Improved

- **LLMs: a model split by tensors across the four dies of two Radeon Pro Vega II Duo cards reads prompts faster.** An 8B goes from 845 to 1196 prompt tokens a second and a 27B from 281 to 359, with generation and output unchanged. Thanks to [Chris Hafey](https://github.com/chafey) for the hardware.

### Fixed

- **LLMs: the engine no longer quits on the first request on the FirePro D500 and D700 of a Mac Pro 6,1.** Reading a prompt now falls back to the generic attention kernels when the driver refuses the AMD ones, the way generating already did. Reported in [#12](https://github.com/engeldlgado/toshllm/issues/12).

- **LLMs: a collective on a split model no longer folds in what the previous round left in its buffers.** Generation across the four dies of two Radeon Pro W6800X Duo cards goes from 32.7 to 43.2 tokens a second, and reading a prompt and generation stay where they were on Radeon Pro Vega II Duo. Contributed in [#104](https://github.com/engeldlgado/toshllm/pull/104).

- **Typing into a settings field and leaving for another tab no longer discards what you typed.** The field wrote its value only when it lost focus, and changing tabs takes it away before that happens.

## [0.87.5] - 2026-09-16

### Improved

- **LLMs: long conversations on a model split by tensors across the dies of a Radeon Pro Vega II Duo generate faster.** At 8K of context an 8B goes from 52 to 59 tokens a second, a 14B from 38 to 41 and a 1B from 150 to 160, with the same output and memory.

- **LLMs: Qwen3.8 Flash Next reads prompts and generates faster with TensorMesh.** Against the published 0.87.4 binary, the 177B model goes from 181 to 277 prompt tokens a second and from 23.5 to 25.0 generated tokens a second. The complete A/B, including the 8B and 27B models, is in [Radeon Pro Vega II Duo](docs/performance/0.87.4-radeon-pro-vega-ii-duo.md#0874-against-0875).

### Fixed

- **LLMs: Qwen3.8 Flash Next keeps answering correctly across repeated requests.** Its separate prediction head could work on the first request and then produce repeated zeroes. The engine now refuses that unstable path and continues with the main model instead.

- **LLMs: tensor-parallel work keeps its synchronization objects and collective buffers alive until every GPU has finished with them.** This closes three lifetime and fallback gaps in the fused two-die path without changing its output.

## [0.87.4] - 2026-09-16

### Improved

- **LLMs: generating on a model split by tensors across the dies of a Radeon Pro Vega II Duo.** A 27B goes from 19.1 to 24.1 tokens a second on the two dies of one card, from 13.9 to 21.7 across the four dies of two cards, and from 18.9 to 23.8 with TensorMesh, with identical output. Tables per arrangement in [Radeon Pro Vega II Duo](docs/performance/0.87.4-radeon-pro-vega-ii-duo.md).

- **LLMs: a model split by tensors across two Radeon Pro Vega II Duo cards reads long prompts faster.** A 177B mixture-of-experts goes from 197 to 309 tokens a second with TensorMesh, and generates 25.3 instead of 20.3.

### Fixed

- **LLMs: GPT-OSS-20B downloads from the catalog again.** The catalog asked Hugging Face for a filename that differs in capitalisation from the one it hosts, so the transfer failed the moment it started. Contributed in [#100](https://github.com/engeldlgado/toshllm/pull/100).

- **LLMs: applying a profile brings back the GPU split it was saved with.** The snapshot left out how a model splits across cards, and vision and local-network discovery with it, so two profiles differing only in the split applied the same configuration. Contributed in [#101](https://github.com/engeldlgado/toshllm/pull/101).

## [0.87.3] - 2026-09-12

### Fixed

- **LLMs: a model split across cards answers with text instead of nothing.** About a quarter of the requests came back empty, splitting by tensors and with TensorMesh alike: 9 of 48 runs over three models and four arrangements before, none after, and reading a prompt and generation stay where they were.

- **LLMs: sparse attention returns the right answer on Radeon Vega, Radeon VII and Radeon Pro Vega.** The engine's attention suite goes from 1806 to 1812 of 1812 cases, with every other card unchanged.

- **Chat: typing with a Chinese, Japanese or Korean input method no longer loses the syllable being composed.** Reported in [#99](https://github.com/engeldlgado/toshllm/issues/99).

## [0.87.2] - 2026-09-11

### Fixed

- **Chat: local MCP servers connect straight away instead of staying on Connecting.** Reported in [#98](https://github.com/engeldlgado/toshllm/issues/98).

- **Benchmarks no longer closes the app in a narrow window.** The results table scrolls sideways when it does not fit. Reported in [#97](https://github.com/engeldlgado/toshllm/issues/97).

- **Benchmarks no longer ke
IMPORTANT NOTE FOR LATER: the courier who delivered the spare pump was named Ingrid Solberg.
eps a processor core busy at some window widths.**

## [0.87.1] - 2026-09-11

### Added

- **Chat: MCP servers that run on your own machine.** Until now a server had to be reachable at a URL, which left out nearly every local one, since they are started by a command and talked to over their pipes. Settings also takes the `mcpServers` block other MCP clients use, so a configuration can be pasted in rather than retyped.

### Improved

- **The interface has been redesigned across every screen, with the same controls as before.** Idle memory use drops from about 600 MB to about 120 MB.

- **Models in subfolders are found.** One recursive inventory now serves language, image and video models alike, so a library sorted into folders no longer looks half empty.

- **The estimated speed matches your card.** It was calculated from a single card's bandwidth, and how much of that decode reaches turns out to differ by a factor of nearly three between families: measured 58.8 tokens per second on a Radeon Pro Vega II Duo against 61 on an RX 6700 XT with the same 8B model, despite 2.7 times the bandwidth. On a Radeon RX 570 the figure was out by four times.

- **Text and number fields respond immediately.** A keystroke no longer rebuilds the whole screen, and numeric fields reject anything that is not a number.

- **LLMs: GCN cards verify speculation up to a third faster.** With a 27B model and its DFlash2 draft, generation goes from 15.5 to 20.9 tokens per second; how much it gains depends on the prompt. Reading a prompt and plain generation are unchanged. Contributed by [malzzz](https://github.com/malzzz) in [#94](https://github.com/engeldlgado/toshllm/pull/94).

### Fixed

- **A DFlash/DFlash2 draft works with a model split across cards by tensors instead of taking the engine down.** The output head is now kept whole on each card for that pairing, which costs its size per card and nothing in any other configuration. On four cards it turns out to be the fastest arrangement of the three: a 27B generates 20.5 tokens per second against 14.6 splitting by layers, accepting the same drafts on a Radeon Pro Vega II Duo.

- **Attention no longer crashes on 384- and 640-wide heads.** Outside TurboQuant they had no kernel, and the engine reached for it anyway. They now fall back to the processor. Contributed by [malzzz](https://github.com/malzzz) in [#96](https://github.com/engeldlgado/toshllm/pull/96).

- **The engine builds again with its expert cache compiled out.** Contributed by [malzzz](https://github.com/malzzz) in [#95](https://github.com/engeldlgado/toshllm/pull/95).

- **LLMs: Radeon RX 400/500, Radeon Pro 400/500 and Radeon Pro WX write text again instead of nonsense.** Since 0.87.0 the engine read the weights from addresses those cards need aligned, so half of them came back as neighbouring data. Every other card keeps the code and the speed it had. Reported by [FreQRiDeR](https://www.reddit.com/user/FreQRiDeR/).

- **Chat: a model that writes its tool calls in the wrong shape no longer kills the turn.** The engine answered with an error and the message was lost, on every tool and every attempt. Such a model stops being offered tools and says so once; Settings lists them under Tools and gives each one back with a click.

- **Images: LoRAs apply to f16 models instead of ending the render with an error.** Adding one to a checkpoint whose weights are f16, an SDXL safetensors file among them, stopped generation on cards without unified memory. Reported in #93.

## [0.87.0] - 2026-09-07

### Improved

- **LLMs: GCN cards generate and read prompts faster.** They now have kernels written for them instead of running code laid out for RDNA. Generation gains about 18% on average across twenty quantization types and reading a prompt about 4%. Perplexity and generated text are identical. Per-type tables in [GCN/VEGA](docs/performance/radeon-pro-vega.md).

- **LLMs: RDNA reads Q5_0 and Q5_1 prompts up to 17% faster.** Both were being handed a wide tile that costs them at every prompt length. Reading a prompt gains about 9% on those two and Q5_K generates 3% faster; the other types are unchanged. Perplexity and generated text are identical. Per-type tables in [RDNA](docs/performance/radeon-rx.md).

- **LLMs: the engine moves to a newer upstream.** It brings the fixes and the model architectures added there since the last one. Speed and generated text are unchanged.

### Added

- **Simplified Chinese.** The interface now ships in Spanish, English, Italian, Japanese and Simplified Chinese. Requested in #86.

- **The turns the model archives can be sent to an address of your choosing.** When the model sets a range of turns aside to free context, they can now be posted as JSON to a URL in Settings, so an external index keeps what leaves the conversation. Nothing is lost if the receiver is down or the app quits mid-flight: deliveries wait on disk and resume, and they carry an id so a retry is not stored twice. Empty means off, and a bearer token can be set. Requested in #90.

- **LoRA files apply to image generation.** Put them in an `imagen/lora` folder and a menu next to the prompt inserts the tag for one, with the weight editable in the tag itself. Works with every model in the catalogue. Requested in #93.

- **Extra arguments can be set per model and per server, not only once for everything.** The field in Settings still applies to every server, and now a model's settings popover and every server card carry one of their own, under Advanced options. What a model sets is added after the shared field; what a server sets replaces it, an empty value included. Requested in #89.

### Fixed

- **Downloads work when the models folder is on an exFAT volume.** The free-space check used a figure macOS only reports for APFS, and read it as zero anywhere else, so a download stopped before it started saying no space was left. It now falls back to the plain figure, and lets the download through when neither can be read rather than assuming the worst. Reported in #92.

## [0.86.6] - 2026-09-02

### Fixed

- **LLMs: Qwen3.8 Flash Next keeps its state across a conversation.** Restoring its cache, copying a sequence and keying block positions were wrong for this architecture, so a saved or branched conversation could come back in a different state than it left in. Taken from upstream.

- **LLMs: splitting Qwen3.8 Flash Next across cards by tensor gives corrupted output.** Not a regression: it has done so on every version that could load the model, and upstream now refuses the combination outright. Split it by layer, or set `TOSH_MGPU_TENSOR_GROUP=2` to keep tensor splitting within pairs of cards, which measures correct. Reported in #87.

- **Video: LTX-2.3 no longer offers sizes that come back broken.** Past roughly 9.7 million pixel-frames the clip returns with its opening frames corrupted, and every size the model was offered sat above that: the shortest clip at the smallest of them lost its first eight frames. The four sizes are now 704x384, 640x352, 512x288 and 384x704, all of which measured clean. The fault is in the engine and not in this app's Metal path - it reproduces with the decode on the CPU - so the sizes come back once it is fixed upstream. Reported in #83.

- **Video: LTX-2.3 uses the timestep shift meant for it.** It was being given 3.0, the generic value for a different family; the engine's own default for this model is 2.37. Output is unchanged in the sizes measured.

- **Video: the video engine log can be read from the Logs tab.** It has been written to disk since the video studio shipped, but nothing in the app pointed at it, so a failed clip could not be diagnosed without knowing where to look. The tab now has a **Video** view next to Server and Images, with the same search, filtering and reveal-in-Finder. Reported in #83.

## [0.86.5] - 2026-09-01

### Improved

- **LLMs: splitting a model across cards by tensor generates 8% to 18% faster.** Every exchange between two cards used to be sent as one command buffer per copy, and each card in a step takes part in two of them. They now go as a single command buffer per card, carrying both copies and the sum that follows, which is a third fewer command buffers per token. Measured on four Radeon Pro W6800X dies, tokens per second on `tg128`:

  | Model | Split | Before | Now |
  |---|---|---|---|
  | Qwen3.8-Flash-Next, 72 GiB (MoE) | four cards, by tensor | 16.3 | **17.6** |
  | | TensorMesh, rows of two | 24.3 | **27.0** |
  | Qwen3.6-35B-A3B (MoE) | four cards, by tensor | 23.4 | **25.6** |
  | | TensorMesh, rows of two | 42.1 | **49.9** |
  | Qwen3.8-27B (dense) | four cards, by tensor | 14.3 | **16.6** |
  | | two cards, by tensor | 21.5 | **24.6** |
  | | TensorMesh, rows of two | 21.4 | **24.3** |

  The faster a split already was, the more this shows: a mesh row is two cards doing more work each, so the fixed cost per exchange weighs heavier there, and that is where the largest gain lands. Reading a prompt is unchanged: large batches hand tensors over a different way, which this does not touch. A single card and splitting by layer are untouched too, and measure identical.

- **LLMs: the cards prepare their work at the same time instead of one after another.** Each card's share of a split is independent until they exchange results, but the work of describing it to the driver was done for one card, then the next. Worth 7% to 9% of generation on four cards splitting by tensor; a single card is unaffected.

### Added

- **Installation: the app is signed and notarized by Apple.** It opens on first launch, without approving it in System Settings.
- **Chat: the conversation memory tools can be turned off.** A switch in Settings under **Agents and attachments** hides all three at once, for setups where an external memory server already does the job.

### Fixed

- **Chat: the conversation memory tools now appear in the tool list.** `memory_list`, `memory_archive` and `memory_recall` were sent to the model but missing from the tools popover, so they could not be turned off one by one. Reported in #82.

## [0.86.4] - 2026-08-31

### Improved

- **LLMs: Q6_K, Q3_K and Q2_K models generate faster.** Around 3% on Q6_K and 1% on the other two, with identical output.
- **LLMs: mixture-of-experts models that store their experts in MXFP4, such as gpt-oss, generate about 3% faster.**

### Fixed

- **LLMs: Qwen3.5 models from other catalogues now load.** Files that declare three rope sections instead of four were rejected on open. Reported in #84.
- **Interface: the window no longer stutters every three seconds.** The VRAM reading rebuilt both windows and the menu bar on every sample, whether or not the figure had changed.

## [0.86.3] - 2026-08-30

### Added

- **LLMs: TensorMesh, the GPUs arranged as a mesh: split by tensor along one axis and by layer along the other.** Splitting every tensor across four cards reads a prompt fast but generates at a quarter of the speed, because each card waits for the others twice per layer. A mesh keeps most of both: a card only waits for the others in its own row. Pick the row width in Settings under **How to split it → TensorMesh**.

  Measured on four Radeon Pro W6800X dies against the engine in 0.86.2, `pp512` / `tg64` in tokens per second:

  | Model | Split | 0.86.2 | now |
  |---|---|---|---|
  | Qwen3.8-27B (dense) | by layers | 264 / 22.8 | 280 / 23.6 |
  | | by tensors | 612 / 14.5 | 632 / 14.5 |
  | | **mesh, rows of two** | n/a | **480 / 24.3** |
  | Qwen3.6-35B-A3B (MoE) | by layers | 1186 / 62.8 | 1245 / 67.1 |
  | | by tensors | 1497 / 23.7 | 1531 / 22.6 |
  | | **mesh, rows of two** | n/a | **1623 / 48.9** |
  | Qwen3.8-Flash-Next | by layers | 469 / 26.3 | 494 / 31.3 |
  | | by tensors | 695 / 14.1 | 676 / 13.8 |
  | | **mesh, rows of two** | n/a | **624 / 25.1** |

  On the MoE the mesh wins outright: they read faster than any other split and generate twice what splitting every tensor does. On the two dense models they sit in between, buying prompt speed with generation. Splitting by layers still generates fastest, and now does so 3% to 19% faster than before.

  **What the mesh buys is four cards at the speed of two, not more speed than two.** A row of two behaves exactly like a tensor split on a two-card machine. On an 8B, rows of two across four cards read 1624 and generate 57, against 1667 and 58 on two cards, so the gain is recovering the generation four cards lose (30 with all four together) while keeping their VRAM. On a two-GPU machine there is no mesh to build: a tensor split there already is this shape, which is why the setting only appears once a row width that divides the split exists. Output is identical across all of them, checked against a single GPU by perplexity and by generating the same text word for word.

- **LLMs: a single GPU also generates faster now.** Measured on a Radeon Pro W6800X: 3.4% on a 4B, 2.0% on an 8B and 2.4% on a 35B MoE, with prompts 2.1% faster on the MoE.
- **LLMs: a manual split across cards is now honoured with TensorMesh.** Each row takes its own part of the list instead of the same leading numbers.
- **LLMs: handing a tensor between mesh rows no longer goes through system memory.** Reading a prompt improved 2.2% on an 8B and 1.2% on a 35B MoE.
- **LLMs: mesh rows now overlap while a prompt is read.** Reading improved 2.8% on an 8B and on a 35B MoE.
- **LLMs: a bigger prompt micro-batch for MoE models, in Settings.** Reading 2048 tokens on a Radeon RX 6700 XT improved from 475 to 886 tokens per second on a 35B with experts on the CPU, and around 10% with the model whole on the card, at about 0.5 GB of VRAM per 512 tokens.
- **Chat: the model can manage its own context.** It can list the conversation, set a range aside so it stops being sent with every request, and search it back verbatim when it matters again. Nothing is deleted, and an external client through the API sees these as ordinary tools. Requested in #82.

### Fixed

- **Chat: a summary could silence every message written after it.** Regenerating or dropping the last exchange right after summarizing left new messages out of the request.
- **Chat: summarizing now frees context instead of turns.** It kept the last four messages whatever they weighed; it now keeps a tail worth about a quarter of the context.
- **LLMs: a long prompt no longer fails when the model is split by tensors.** Models that share one set of keys across several queries refused to start above a certain prompt size; reading is also 13% faster on four cards, with the same output.
- **LLMs: with more than one mesh row, all the weights ended up on one of them.** Half the work ran where the weights were not: wrong output, and in some models none at all.
- **LLMs: cards now overlap instead of taking turns when the model is split.** Measured on four Radeon Pro W6800X dies: generation improved 9.3% on an 8B and 13.1% on a 4B, with the same output.

## [0.86.2] - 2026-08-29

### Added

- **
IMPORTANT NOTE FOR LATER: the replacement valve handle is painted burnt orange.
Images: a custom model can now be a diffusion model, not just a full checkpoint.** Point the app at the diffusion file, its VAE and its text encoder, and models published that way (Z-Image, Flux 2, Qwen-Image) run like the catalogue ones.
- **Audio: a translation model can be picked, and downloaded, from the Audio panel.** With no language model installed the panel now offers the catalogue instead of an engine button that could never start.

### Fixed

- **Images: models carrying bf16 weights now load on older cards.** Flux 2 aborted on a Radeon RX 500 or Vega with "cannot run the operation"; those weights are converted as they load, at the same size and with the picture unchanged. Reported by Slice.

## [0.86.1] - 2026-08-29

### Added

- **Images: retouch one area of a photo instead of the whole frame.** Paint over the init image and only what you paint is regenerated: measured on a Radeon RX 6700 XT, the rest of the picture moves 5.3 against the 5.2 floor of the encode and decode round trip.
- **Images: a negative prompt.** It only takes effect above CFG 1, and the app says so on the models that ignore it.
- **Video: a negative prompt**, prefilled with the model's recommended one, which can be replaced or emptied.
- **Audio: the subtitles burned into a transcribed video can be styled and previewed.** Typeface, size, colour, box or outline, position and margins, shown over a real frame of your own clip with its longest caption.

### Improved

- **LLMs: model menus group by family.** Every Qwen together, every Llama together, each ordered by size, with makers the app does not recognise under Others.

### Fixed

- **LLMs: leaving "Copy weights to VRAM" off no longer collapses speed on a discrete card.** It wrote 1.6 tokens/s where the same model now writes 58, on a Radeon Pro Vega II with Qwen3-8B. Thanks to [Mallory M.](https://github.com/malzzz).
- **LLMs: answers split across three or more cards can no longer come back silently wrong.** An exchange that failed midway through the sum was counted twice, and a card could be told the work was done while another was still reading its memory. Thanks to [Mallory M.](https://github.com/malzzz).
- **LLMs: speculative drafts named DSpark stay out of the model list.** They are drafts, not models to run.
- **Audio: long subtitles are no longer cut off in the exported video.** The caption box was a fixed height, so anything needing more lines than fitted was clipped. It now grows to the text, and shrinks the text only when it would not fit on screen.
- **Audio: the subtitle size setting works across its whole range.** Below the middle of the slider every value rendered the same, because an internal floor was measured in points against a preview still rather than the video.
- **Audio: a captioned video keeps the codec and the colour of the original.** An HEVC clip was re-encoded as H.264, and a wide-gamut or HDR one came back flattened.
- **Audio: the transcript keeps the view you chose once a translation finishes.** It dropped to translation-only even with bilingual selected, and switching away and back was the only way to get it back.
- **Tooltips carrying a number or a name now translate.** Any text with a value in it stayed English in Japanese and Italian, whatever the language setting said. Reported by [playexit](https://github.com/playexit).

## [0.86] - 2026-08-28

### Added

- **Qwen3.8-Flash-Next now runs.** The 176B mixture-of-experts model writes at 20.3 tokens/s splitting by layers, or reads at 265 splitting by tensors, still reading at 221 and writing at 15.5 at an 8k context. Measured on the UD-Q2_K_XL build across two Radeon Pro Vega II with every expert on the cards. Thanks to [Chris Hafey](https://github.com/chafey) for the hardware.
- **Newer DFlash drafts are supported.** The second-generation drafts the previous engine could not load now work.
- **Very large models can now be split by tensors.** A card whose share of the weights outgrew one Metal buffer stopped the engine; Qwen3.8-Flash-Next reads 2.4 times faster this way.

### Improved

- **Transcription is faster.** Whisper's encoder gains 10% on a Radeon RX 6700 XT and 5% on a Radeon Pro Vega II.
- **Fixes and improvements to speculative decoding.** How much a draft gains depends on how predictable the text is: up to 12% on repetitive content, and it steps aside on content it cannot predict.
- **The GPU panel now says whether the Infinity Fabric bridge is being used**, not just whether it exists, and the tooltip explains why when it is idle. macOS reports a bridge as connected without saying which cards it joins.
- **Splitting by tensors now warns what it costs.** Above two GPUs it reads prompts far faster but writes slower, and the note carries the measured figures.

### Fixed

- **Models that ship in several files are read whole.** Only the first file was inspected, and it carries no tensors, so the app saw an empty model: multi-GPU splits went unbalanced and speculative heads went unnoticed. Every large mixture-of-experts model ships this way.
- **Speculative drafts no longer appear as models to run.** They were filtered by file name, so drafts named like a normal model reached the picker.
- **The Infinity Fabric switch now matches what it does.** It read as off while the switch showed on, and it is applied only where it helps.
- **Z-Image and Flux now run on cards without bfloat.** Their shared autoencoder was the bf16 copy, which such a card cannot compute on at all, so generation stopped on its first tensor. Reported by [Slice](https://www.insanelymac.com/forum/profile/112217-slice/).

## [0.85.10] - 2026-08-28

### Improved

- **Infinity Fabric Link is now on by default.** Splitting by tensors reads prompts 16% faster at no cost to generation, measured on four Radeon Pro Vega II. Thanks to [Chris Hafey](https://github.com/chafey) for the hardware.

### Fixed

- **A very large mixture-of-experts model no longer fails to load across several cards.** The host-resident experts each card maps were uncapped, so the card stopped granting allocations. A 153 GB model now runs on two Radeon Pro Vega II. Thanks to [Chris Hafey](https://github.com/chafey) for the hardware.

- **Cards in a mixture-of-experts split no longer fill unevenly.** Layers were dealt out by count, so the ones keeping their experts overflowed a card while the rest sat idle. On four Radeon Pro Vega II: 1.4 / 5.2 / 41.5 / 41.5 GB before, 24.3 / 21.2 / 21.2 / 18.2 GB now. Thanks to [Chris Hafey](https://github.com/chafey) for the hardware.

- **Splitting a mixture-of-experts model by tensors no longer stops the engine.** A card left an empty slice aborted the graph; it now contributes nothing instead. Perplexity unchanged: 6.2307 across four cards against 6.2336 across two. Thanks to [Chris Hafey](https://github.com/chafey) for the hardware.

- **A speculative draft the engine cannot load is no longer offered or used.** Newer DFlash drafts carry stages with no loader here; the server exited on the tensor count.

- **The fused QKV path now runs during generation.** Its matcher expected the projections in a different order, so the faster path was never selected; Gemma 3 4B with Turbo KV gains 1.7% on a Radeon RX 6700 XT. Thanks to [malzzz](https://github.com/engeldlgado/toshllm/pull/71) for the fix.

## [0.85.9] - 2026-08-26

### Added

- **Audio is now a full workspace beside Chat, Images and Video.** Whisper.cpp transcribes microphone, audio and video on the Radeon; projects preserve editable original and translated tracks, synchronized playback, calibrated Silero VAD, recovery, subtitle exports and captioned MOV copies. Feature requested by [playexit](https://github.com/engeldlgado/toshllm/issues/73).
- **Audio can translate with Whisper or the loaded chat model.** Chat translation resumes without rerunning Whisper and uses an explicit model, neighboring context, prior terminology and a glossary to keep segments consistent.

### Improved

- **The Italian localization now covers the complete interface.** All current UI strings are translated instead of falling back to English.

### Fixed

- **The server log says what Dynamic MoE actually ran with.** It printed the MoE-on-CPU value you had saved while the cache was running with a different one; it now reports the cache and its slots, and stops passing a CPU-expert count the cache does not use.

## [0.85.8] - 2026-08-24

### Added
- **Dynamic MoE, a private experiment that runs a MoE model on far less video memory** by keeping every expert in system RAM and a small cache of slots on the card. It is off by default and hidden for internal testing; a 35B in Q2_K_XL ran on 2.78 GiB against the 6 GiB its usual setting needs, and generated faster. Read the [what it is and when it pays](https://github.com/engeldlgado/toshllm#dynamic-moe-bounded-vram-expert-cache-private-experiment) section first, especially the RAM it asks for in exchange.

### Improved
- **Models in BF16 read prompts about 30% faster.** 722 → 949 tokens per second on a 4B and 5211 → 6807 on a 0.6B, with generation speed and answers unchanged. Measured on a Radeon RX 6700 XT.

### Fixed
- **A model or projector in BF16 no longer stops the engine on cards that lack BF16.** The load aborted before ever reaching the chat; those weights now run on the card like any other format. Reported by [Slice](https://www.insanelymac.com/forum/profile/112217-slice/).

- **Conversations exported to Markdown are labelled in the language you picked.** The two headings the file writes were always in Spanish. Reported by [guylough](https://github.com/engeldlgado/toshllm/issues/70).

- **Two Duo cards no longer read as four cards.** A Duo puts two GPUs behind one name, so the machine card counted devices and doubled them; it now says two cards and four GPUs. Reported by [chafey](https://github.com/engeldlgado/toshllm/issues/67).

- **The Infinity Fabric Link switch is only offered when there is a link.** It could be turned on with nothing bridged, which read as if the app had found a bridge. Reported by [chafey](https://github.com/engeldlgado/toshllm/issues/67).

- **TurboQuant cache types no longer stop the server when the model is split by tensors.** The split path did not know how to divide the rotation TurboQuant applies to attention, and the engine aborted while loading. Reported by [chafey](https://github.com/engeldlgado/toshllm/issues/69).

## [0.85.7] - 2026-08-22

### Improved
- **Models quantized to about one bit write answers faster.** Their lookup table was read from constant memory four times per row instead of being staged once per group of threads: an 8B in IQ1_S gains 50% more tokens per second and a 9B in IQ1_M 14%, with the output unchanged. Measured on a Radeon RX 6700 XT.

### Added
- **Cache keys at q4_0 can be paired with TurboQuant values.** The combination had no kernel of its own, so it was rejected and attention ran on the processor; it now runs on the card like every other pair.

- **Mistral Small 4 runs its attention on the card.** Its 320-wide keys had no kernel, so every layer's attention fell back to the processor.

- **The reasoning effort list has a "Model default" entry** that leaves the choice to the model instead of sending a level.

- **The machine card counts every GPU and says whether an Infinity Fabric link joins them**, with the memory behind each link and a button for the full breakdown. macOS shows that nowhere.

- **The GPU list marks the linked cards** with a coloured bar, a badge and their device index. The server log names each card's link too.

### Fixed
- **Video memory was rounded down by a whole gigabyte.** A 12 GB card read as 11 GB and a 32 GB one as 31 GB.

- **The reasoning effort list only offers what the model accepts.** Choosing a level its chat template rejects returned an HTTP 500 in the middle of the answer.

- **The TurboQuant warning says which of its reasons applies.** It listed the model, the backend and the type combination in one sentence, and explained only the last.

## [0.85.6] - 2026-08-21

### Added
- **Settings suggests the KV cache types that measure best.** Keys at `q8_0` with Turbo4 values matched f16 quality on 4B, 8B and 35B models while taking a quarter less cache than `q8_0` on both; one click applies it.
- **Each server can bridge MCP for the engine's web interface.** A switch under the server card's advanced options passes `--ui-mcp-proxy`, so the web interface can reach MCP servers on another origin; it is per server, experimental, and meant for trusted networks only.

### Fixed
- **Extra arguments reach router mode too.** Flags typed there were applied to a normal server but dropped when the router served the models.

## [0.85.5] - 2026-08-21

### Fixed
- **Radeon RX 400/500, Radeon Pro 400/500 and Radeon Pro WX no longer write nonsense.** Since 0.84.4 the engine read Q6_K weights four bytes at a time, from addresses those cards need aligned, so half of them came back as neighbouring data. Almost every model keeps part of its layers in that format, the output head among them, which is why whole models turned to gibberish. Reported by [FreQRiDeR](https://github.com/engeldlgado/toshllm/issues/1) and [Slice](https://www.insanelymac.com/forum/profile/112217-slice/).

### Improved
- **Image and video models load about five times quicker.** The loader asked the drive for 4 KiB at a time, so 2.46 GB of weights took 7.8 seconds where the disk does it in 0.7. A cold start of Flux.2 klein 4B drops from 12.9 to 2.5 seconds, with the image unchanged.

## [0.85.4] - 2026-08-21

### Added
- **The Logs tab can now check the engine on your own card.** It runs the engine's operator tests, tells apart the one failure that comes from upstream and is not a problem, and saves the result together with the diagnostics as a single text file you can send. It takes a few minutes and asks for the server to be stopped, since it uses the whole card.

## [0.85.3] - 2026-08-20

### Fixed
- **Separate MTP heads now attach to their base model** ([#60](https://github.com/engeldlgado/toshllm/issues/60)). They stay out of the model menu and work in normal and router modes.
- **Paused model downloads resume reliably after links expire** ([#61](https://github.com/engeldlgado/toshllm/issues/61)). Servers that ignore the requested range restart the transfer instead of corrupting the partial file.
- **Models with 40- or 80-wide attention heads no longer write nonsense.** All 108 affected engine operator tests now pass.
- **The image token cap also limits memory while a vision model loads.** At 512 tokens, a 2B vision model's largest warm-up block drops from 4372 to 228 MiB. Reported by [Slice](https://www.insanelymac.com/forum/profile/112217-slice/).
- **The server log includes environment variables entered in Extra arguments.** Shared diagnostics now show the settings that were actually applied.
- **Video models only offer their supported sizes and frame rates.** Wan 2.1 uses 832×480 at 16 fps; Wan 2.2 u
IMPORTANT NOTE FOR LATER: the quarterly audit will be held in Room 314B.
ses 1280×704 at 24 fps.
- **Native-size video generation no longer brings down the graphics session.** Wan 2.2 releases 8.5 GB before decoding and keeps long VAE work responsive.
- **Video memory warnings follow the real generation stages.** Wan 2.2 at 1280×704 measured 10.68 GB for 33 frames and 10.97 GB for 49; 49 frames recommend a 16 GB card.
- **Wan 2.1 and Wan 2.2 use their official inference values.** Defaults now follow each checkpoint's steps, CFG, shift, size and frame rate.
- **HunyuanVideo 1.5 downloads its complete conditioning stack.** It now includes the 439 MB ByT5 encoder and uses 50 steps, CFG 6 and shift 7.
- **Negative prompts stay with the matching model family.** Wan, LTX and Hunyuan no longer inherit one another's instructions.
- **Rejected memory blocks now produce the right guidance.** Graph allocations suggest reducing context or image tokens; weight allocations can suggest the buffer cap.
- **Metal shaders precompile reliably from Xcode's downloaded toolchain.** Both bundled engines retain runtime compilation as a fallback.

## [0.85.2] - 2026-08-19

### Fixed
- **Video no longer comes back as a flat colour.** Its attention was built in full instead of going through the card's kernels, and past a certain clip length it stopped fitting in a single allocation. A 33-frame clip at 704x400 is the case that showed it; 832x464 with 33 frames now renders too.
- **Video no longer flickers.** The decode brightened one frame in every four, the rhythm the decoder compresses time at. Decoding in tiles removes it; a setting brings the old decode back.
- **Video no longer ends in colour bands or noise.** Long clips ran past the time macOS allows a single piece of GPU work. The work is now split, and the tiled decode keeps each piece small.
- **Wan 2.2 5B works, and on far less video memory than its listing claimed.** It was being sampled on a schedule that turns its output into coloured mush, worse the more steps it took; on the one it wants it returns a clean picture. It now asks for 12 GB instead of 24, up to 704 pixels, where its decoder still fits.
- **SD 1.5 is no longer offered a frame it cannot render.** That model was trained on square frames only, and any other shape comes back with colour blotches in the background however many steps it takes, so the studio now offers it the square alone and says why. The newer models saw many aspect ratios and keep every framing.
- **Video no longer comes out as a still image.** The studio opened at 480x272, well under what these models were trained at, and there they stop moving: a clip at that size measures 0.5 of change between frames against 20.6 at 704x400. It now opens at the model's own size, and warns if you go below it.

### Improved
- **Images take much less video memory and are made 12 to 14% quicker.** Z-Image at 1600x900 drops from 8.2 GB to 966 MB, and at 1024x1024 from 2.4 GB to 690 MB, with the image unchanged. 2048x2048 and 2560x1440 now render on a 12 GB card.
- **SD 1.5 needs nine times less video memory and is twice as quick**: 281 MB and 73 seconds at 768x768, against 2690 MB and 149. Its head sizes now fit the card's attention kernels, and 1024x1024 works for the first time. This is the model for 4 to 6 GB cards.
- **Video is generated about 32% quicker.** Sampling a 33-frame clip at 704x400 goes from 209 to 142 seconds.
- **The size list follows what the card can really do.** The memory estimate was written when the attention was built in full, and hid sizes that now fit.
- **Models no longer ask for twice the card they need.** The stated minimums were set when the attention was built in full: Z-Image and SDXL Turbo drop from 8 to 6 GB, Flux.2 klein 4B from 12 to 6, klein 9B from 16 to 10, Flux.1 schnell from 16 to 8, Qwen-Image from 24 to 16, and the large video models from 32 to 24. Measured where the model is here, derived from the same figures elsewhere.
- **Each studio says what its model was trained for**, so a repeated composition or a drifting clip reads as the model's limit and not as the app's.
- **Updated image engine**, with the fixes released upstream since the last version.

Measured on a Radeon RX 6700 XT with Wan 2.1 1.3B, Z-Image Turbo, SDXL Turbo and SD 1.5.

## [0.85.1] - 2026-08-19

### Added
- **A working directory per conversation and per project.** With agent tools on, the model's file tools stay inside the folder you pick, shown next to the conversation's title.
- **A cap on how many tokens an image may take.** The model's limit is kept by default; lowering it in Settings is what lets a vision model load on an 8 GB card. Reported by [Slice](https://www.insanelymac.com/forum/profile/112217-slice/).
- **Time to first token** in each answer's metrics.

### Fixed
- **File tools are no longer offered when the chat has no working directory**, so the model stops trying to edit files that do not exist.
- **Router mode no longer fills video memory with models that use older attention.** Llama 2 and similar asked for 8 GB of key-value cache at 16k context, enough to freeze the machine; each model now gets a context that fits.
- **Stopping the server no longer leaves a model in memory** in router mode.

### Improved
- **Updated inference engine**, with the fixes and new model support released upstream since the last version.
- **New web chat in the browser**: the llama.cpp interface, rebranded for ToshLLM, with a model picker that works in router mode. Requested by [urfin78](https://github.com/engeldlgado/toshllm/issues/59).
- **Memory estimates read each model's real attention layout.** On a 7B with older attention at 16k context the old guess was 0.8 GB against a real 8 GB.
- **The speculative decoding readout follows the answer as it is written.**

## [0.85.0] - 2026-08-17

### Fixed
- **Gemma 4 no longer stops the engine when part of it runs on the processor.** When the model did not fit entirely in video memory and the key-value cache was quantised, any prompt past about forty tokens ended in a fatal error. Reported by [Slice](https://www.insanelymac.com/forum/profile/112217-slice/).

### Improved
- **Serving two or three conversations at once is 25 to 34% faster.** On a 9B at Q5_K_M, two go from 56.4 to 70.8 tokens/s and three from 60.5 to 81.0. A single conversation and four or more are unchanged.
- **Follow-up questions in a conversation already in cache are read faster.** The gain is over the nine or ten tokens a short question adds to a conversation, on models at Q4_K, Q3_K and Q2_K. Other formats and longer prompts are unchanged.
- **Smaller quantisations are considerably faster.** A 1.5B at IQ2_M writes answers 49% faster, from 92.1 to 140.8 tokens/s; a 7B at IQ3_XXS reads prompts 7% faster; and models with q5_0 or q5_1 weights about 15% faster. Perplexity is unchanged throughout.
- **DeepSeek and GLM models read long prompts up to 31% faster.** A 23B GLM goes from 203.8 to 257.5 tokens/s with 4096 tokens already in context, and from 121.5 to 159.2 with 8192.
- **The experimental turbo key-value cache is no longer the slowest option.** At 4096 tokens of context turbo3 goes from 55.8 to 59.8 tokens/s and turbo4 from 56.8 to 60.5, against 58.1 for q8_0. Perplexity is unchanged.

#### Radeon Vega, Radeon VII, RX 400/500 and Radeon Pro Vega/WX

These cards were tuned with settings measured on the newer Radeons, and most of them wanted different ones.

- **Models write answers 4 to 24% faster.** A 9B at Q5_K_M goes from 31.2 to 38.8 tokens/s, an 8B at Q6_K from 35.5 to 41.4, a 4B at Q2_K from 49.9 to 58.0 and a 7B at IQ3_XXS from 42.2 to 44.4. Mixture-of-experts models gain too: a 7B with 64 experts goes from 136.0 to 155.3. Perplexity is identical.
- **Serving several conversations at once is 9 to 12% faster, and multi-token prediction gains 8%.** On a Radeon RX Vega 64, a 4B goes from 108.3 to 121.1 tokens/s with four conversations and from 119.5 to 133.6 with eight; a 9B at Q5_K_M from 47.4 to 51.6 with four.
- **Prompts are read faster at every length.** A short follow-up question gains up to 39%, prompts of 130 to 160 tokens 2 to 4%, and long prompts 3 to 5%: llama-2-7B goes from 509 to 528 tokens/s at 512 tokens and a 1B at IQ4_XS from 2811 to 2941. Perplexity is identical.
- **Mixture-of-experts models can now use the faster path the newer Radeons already had**, which had been locked to those cards without ever being measured on these. On gpt-oss-20b it reads prompts up to 23% faster at twelve of fifteen lengths, but loses about 17% around 470 tokens, so it is not on by default: enable it with `TOSH_MMID_WIDE_W64_ENABLE=1`.

Measured on a Radeon RX 6700 XT, and on a Radeon RX Vega 64 for the section above.

## [0.84.4] - 2026-08-15

### Fixed
- **A model that does not fit no longer crashes the engine.** When the card refused a memory block the engine read the failed allocation anyway and died with a segmentation fault instead of reporting the problem, so the app could only say the engine had stopped. The failure is now reported, and the message tells you what to do about it. Diagnosed by [Slice](https://www.insanelymac.com/forum/profile/112217-slice/) on an RX 570, who also traced it to the exact line.
- **A card that refuses one large block can now split it.** Some drivers advertise room for a 4 GiB block and then turn it down; starting with `TOSH_METAL_MAX_BUFFER_MB=1024` splits the weights into smaller blocks instead, at the same speed. Also suggested by Slice.

### Improved
- **Mixture-of-experts models read prompts up to 18% faster.** The wider matrix tile that dense models already used now also serves the expert matmul, and an expert that receives few tokens skips the columns it would only pad. gpt-oss-20b goes from 1093 to 1292 tokens/s on a 512-token prompt, and a 14B A3B from 1235 to 1315.
- **Qwen3.5 and Qwen3.6 models write answers 7 to 14% faster.** Most of their layers copy a running state once per token, and that copy went through a kernel that rebuilds a four dimensional index for every single number it moves. A copy whose two sides are already laid out in order now skips all of that: 62.2 to 70.8 tokens/s on a 14B A3B and 43.0 to 46.2 on a dense 9B.
- **Speculative decoding carries the gain too.** On a Qwen3.5 4B writing code, MTP goes from 70.0 to 81.6 tokens/s and DFlash from 68.9 to 82.6; on mixed unpredictable text, MTP from 63.7 to 73.8 and DFlash from 60.7 to 68.3. The draft acceptance rate is identical down to the last counter in every pair, so the speedup is the kernels and not a different acceptance pattern.
- **Q4_K models are faster at both ends.** The scales of a block took five separate small reads; the sixteen bytes they live in are one aligned read, and the block layout already guaranteed it. Reading a prompt gains 1.8 to 2.6% and writing an answer 1.1 to 1.5%, measured on a dense 8B and on Gemma 4 E2B.
- **Q6_K tensors are read a word at a time.** That quantisation was the only one still read a byte at a time, because its block size left the compiler unable to assume anything about alignment. Its matrix-vector step drops about 20%.

Measured on a Radeon RX 6700 XT against 0.84.3. Models without any of these traits, such as Llama 2, are unchanged.

## [0.84.3] - 2026-08-14

### Improved
- **Prompts are read 5 to 7% faster.** The prefill matrix kernel stalled on shared memory more often than it needed to; widening one of its two reads cuts those stalls by two thirds, and the faster kernel now also pays off on shorter prompts, so it starts being used from 112 tokens instead of 192. Measured on llama-2-7B: 799 to 859 tokens/s at 128, 868 to 933 at 190, 944 to 990 at 512.
- **Mixture-of-experts models read prompts about 2.7% faster.** Their matrix kernel converted its running totals four times more often than the one dense models use; it now does it once, like the other.
- **Mixture-of-experts models kept in video memory also answer about 1.5% faster.** The engine reordered their graph while writing, which disturbs the expert lookup for no gain on AMD; it no longer does.
- **Models quantised as Q5_K answer a little faster.** Their fifth-bit path took six narrow memory loads per step and now takes three wide ones, reaching the throughput the other quantisations already had.

## [0.84.2] - 2026-08-14

### Fixed
- **A conversation with code blocks no longer freezes while you scroll it** ([#58](https://github.com/engeldlgado/toshllm/issues/58))... every code block, and every panel of a tool call, carried a scroll view of its own, and each one re-tiled on every geometry change while scrolling. On macOS 15 that could leave the app unresponsive for half a minute. Long lines now wrap to the bubble and the expand button still opens the full-width view.
- **Quitting the app always stops the engine.** With the prompt cache enabled the engine was told to stop only after the app had finished saving its cached prefix, and that save could not finish once the app was on its way out, so the engine stayed running and holding video memory until you noticed it. The prefix is still saved on the way out, now with a time limit that the shutdown never waits past.

### Improved
- **The window no longer stalls for a few seconds on launch.** The model menu was reading the header of every model in the folder while it drew, which on a folder of large models meant a spinning cursor before the app was usable. The headers are now read in the background and the menu fills in as they arrive.
- **Scrolling away from an answer while it is being written no longer rebuilds it.** The typewriter and the plain view were two separate branches, so stepping away threw the formatted text away and built it again from scratch.

## [0.84.1] - 2026-08-14

### Fixed
- **An upscaled image is saved next to the original**, instead of landing in the models folder. Upscaling a photo from anywhere on the disk left the result among the model weights, and upscaling a generated image produced a file the "delete images on app close" option then wiped. The name also states the size you asked for, ×2 or ×4, rather than always ×4, and an existing file is never overwritten.
- **Scrolling the chat no longer gets stuck on an answer about code** ([#58](https://github.com/engeldlgado/toshllm/issues/58))... a paragraph mentioning two shell variables, or two prices, was read as a formula and handed to the formula renderer, which turned the text into unreadable maths and swallowed the mouse wheel. Only what really looks like a formula is treated as one now, and a formula or diagram in the middle of a conversation passes the wheel through to the chat.
- **Engine errors are shown in your language.** Every diagnostic the engine produces was being printed with the Spanish and the English text one after the other, separated by a slash, in the same line. There are thirty of these messages and all of them read that way.
- **A failed engine no longer spills its whole explanation across the toolbar.** The state now shows a warning triangle and the word Error

Question: Earlier in this document there were four notes marked IMPORTANT NOTE FOR LATER. Answer each: (1) What is the maintenance code for the north warehouse? (2) What was the name of the courier who delivered the spare pump? (3) What color is the replacement valve handle? (4) In which room will the quarterly audit be held? Answer with a short numbered list.