Reduce translation latency (BLE round trip 45-100ms per uncached lookup) #2

New issue

Open

opened 2026-07-04 14:07:20 +08:00 by afiqzudinhadi · 0 comments

afiqzudinhadi commented

2026-07-04 14:07:20 +08:00

(Migrated from github.com)

Text emission waits one BLE round trip (GET_STRING/RESOLVE) per uncached lookup: measured 45-100 ms, timeout currently 250 ms. Cached lookups are instant, but cold words feel sluggish.

Optimization candidates, roughly in order of value:

BLE connection parameter tuning for the split link (shorter connection interval / zero peripheral latency while the steno layer is active) — biggest single win, round trip is dominated by interval alignment.
Request pipelining: fire GET_STRING/RESOLVE for the probable match while the chord is still held (speculative prefetch on chord-down instead of all-keys-up).
Bigger left-side LRU (currently 64 entries × 128 B) and persistence of the hot set.
Prefetch follow-up candidates: after a hit, batch-request translations for the most likely next windows (CONFIG_STENO_SPLIT_PREFETCH exists but is unimplemented).
Retry-once on timeout instead of falling back to raw steno immediately.

Text emission waits one BLE round trip (GET_STRING/RESOLVE) per uncached lookup: measured 45-100 ms, timeout currently 250 ms. Cached lookups are instant, but cold words feel sluggish. Optimization candidates, roughly in order of value: 1. **BLE connection parameter tuning** for the split link (shorter connection interval / zero peripheral latency while the steno layer is active) — biggest single win, round trip is dominated by interval alignment. 2. **Request pipelining**: fire GET_STRING/RESOLVE for the *probable* match while the chord is still held (speculative prefetch on chord-down instead of all-keys-up). 3. **Bigger left-side LRU** (currently 64 entries × 128 B) and persistence of the hot set. 4. **Prefetch follow-up candidates**: after a hit, batch-request translations for the most likely next windows (CONFIG_STENO_SPLIT_PREFETCH exists but is unimplemented). 5. Retry-once on timeout instead of falling back to raw steno immediately.