diff --git a/README.md b/README.md index be77abc..17d54e3 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,154 @@ # zmk-steno-engine -Clean-room stenography engine for [ZMK Firmware](https://zmk.dev). Dictionary-based lookup with multi-stroke support, optimized for nRF52840 flash constraints. +Clean-room stenography engine for [ZMK Firmware](https://zmk.dev). +Carries the **full Plover main dictionary AND the full Lapwing base +dictionary — 262,309 entries, zero trimming — on a pair of nRF52840 +keyboard halves** using the v4 "union split-section" format -**Status:** Early development — basic single/multi-stroke lookup works, formatter not yet implemented. +**Status:** working end to end on real hardware (corne, 2x nice!nano v2). +Translations, multi-stroke longest-match with retrace, number strokes, +runtime dictionary switching (Plover/Lapwing). Known issues tracked in +the issue tracker; latency optimization in progress. -## Features +## How it works -- Standard 23-key steno layout -- Sorted-array dictionary with binary search lookup -- Multi-stroke support with configurable timeout -- All-up chord detection -- HID keyboard output (ASCII) -- Build-time dictionary compilation from Plover JSON -- Test dictionary included for development +One logical dictionary structure, its sections split across the two MCUs: -## Building +| Half | Holds | Role | +|------|-------|------| +| Left (central) | MPHF displacements, membership, fingerprints, conflicts, part of the value index | Every stroke *decision* is local — found / not found / which dictionary | +| Right (peripheral) | Compressed string table, rest of the value index | Serves translation *text* over a custom BLE GATT protocol | -Add as a ZMK module in your `west.yml`: +Key measured facts: + +- One union CHD MPHF over 227,091 stroke keys — two separate dictionaries + would cost 1.8x more flash +- Lapwing's translations are 93% a subset of Plover's → the shared, + front-coded, deflate-compressed string table stores both for 193 KB +- Stroke decisions never wait on BLE; only text retrieval does + (LRU-cached on the left half) + +## Requirements + +- **A split keyboard with two nRF52840 controllers** (e.g. corne with + 2x nice!nano v2). The dictionary spans both halves — **single-MCU + keyboards are not supported yet.** +- ZMK v0.3.x build environment (GitHub Actions user-config works) +- Python 3 with `numpy` on the build host (dictionary compiler) +- **Adafruit nRF52 bootloader 0.9.0 or newer on BOTH halves — + 0.11.0 recommended.** See [Bootloader](#bootloader) below; old + bootloaders silently corrupt large firmware images. + +## Usage + +`west.yml`: ```yaml manifest: remotes: - - name: zmk-steno-engine + - name: afiqzudinhadi url-base: https://github.com/afiqzudinhadi projects: - name: zmk-steno-engine - remote: zmk-steno-engine - revision: optimize-dict + remote: afiqzudinhadi + revision: main ``` -Enable in your `.conf`: +`.conf`: ``` CONFIG_STENO_ENGINE=y +CONFIG_STENO_DICT_BOTH=y # or STENO_DICT_PLOVER / STENO_DICT_LAPWING +CONFIG_STENO_SPLIT_DICT=y +CONFIG_STENO_DICT_LEFT_MAX_SIZE=440320 +CONFIG_STENO_DICT_RIGHT_MAX_SIZE=519168 +CONFIG_STENO_SPLIT_TIMEOUT_MS=250 ``` -Use in your keymap: +The build FAILS if either half's blob exceeds its budget — entries are +never trimmed. + +Keymap: include the behavior and bind the 23 steno keys +(see `include/dt-bindings/zmk/steno_keys.h` for all key names): ```dts +#include #include -/ { - keymap { - steno_layer { - bindings = < - &steno STENO_SL &steno STENO_TL &steno STENO_PL ... - >; - }; - }; -}; +// example row: &steno STENO_NUM &steno STENO_SL &steno STENO_TL ... ``` -## Dictionary Compiler +Digits in dictionary strokes (e.g. `12K` → `12:00`) map to the number +bar plus their positional key, exactly like Plover. -```bash -# Compile test dictionary -python3 tools/compile_simple.py dicts/test.json -o steno_dict.bin --stats +## Flash budget -# Compile Plover dictionary (trimmed) -python3 tools/compile_simple.py plover-main.json -o steno_dict.bin --max-entries 120000 --stats +The app image on each half must stay below the bootloader's app-region +ceiling (`0xEA000`, i.e. **802,816 bytes** with Adafruit 0.11.0). +Firmware blocks above the ceiling are silently dropped and the half +will not boot. The default budgets above keep both halves under it — +check the `FLASH` line in your build log if you change them. + +## Bootloader + +The nice!nano ships with various versions of the +[Adafruit nRF52 bootloader](https://github.com/adafruit/Adafruit_nRF52_Bootloader). +**Versions from ~2021 (e.g. 0.6.0) corrupt large UF2 flashes** — blocks +go missing mid-image and the firmware crash-loops with no display, no +typing, no USB. This module's images are large (~1.5 MB UF2 per half), +so a current bootloader is mandatory. + +### Check your version + +Double-tap the reset button. The half mounts as a `NICENANO` USB drive; +open `INFO_UF2.TXT` on it: + +``` +UF2 Bootloader 0.6.0 ... ← too old +UF2 Bootloader 0.11.0 ... ← good +``` + +(On newer macOS the volume may not auto-mount — run `diskutil list` +and `diskutil mount diskN`.) + +### Update + +1. Download the updater for your board from the + [official releases](https://github.com/adafruit/Adafruit_nRF52_Bootloader/releases) + — for nice!nano: `update-nice_nano_bootloader-_nosd.uf2` + (the `nosd` variant keeps the installed SoftDevice, which ZMK needs). +2. Enter the bootloader (double-tap reset) and copy the updater UF2 to + the `NICENANO` drive. The device installs it and reboots. +3. Re-enter the bootloader and confirm the version in `INFO_UF2.TXT`. +4. Repeat for the other half. + +To revert, flash any older `update-nice_nano_bootloader-*.uf2` from the +same releases page the same way — the process is not one-way. + +### macOS flashing note + +Recent macOS versions (FSKit FAT driver) break Finder/`cp` copies to +UF2 bootloader drives — writes are silently dropped or fail with error +-50. Flash from the terminal against the raw disk instead: + +```sh +diskutil list # find the NICENANO disk number +diskutil unmountDisk disk4 # keep the device attached +sudo dd if=firmware.uf2 of=/dev/rdisk4 bs=512 +``` + +The bootloader flashes any 512-byte sector carrying UF2 magic, so the +filesystem layer is not needed. + +## Repository layout + +``` +src/ engine, v4 decoder, BLE split protocol, inflate +tools/compile_v4.py dictionary compiler (both JSONs → two half blobs) +tools/fetch_dict.py dictionary downloader (Plover main, Lapwing base) +tests/test_dict_v4.c host round-trip test (42,000 vectors) ``` ## License -[PolyForm Noncommercial 1.0.0](LICENSE) +PolyForm Noncommercial 1.0.0 — see [LICENSE](LICENSE). diff --git a/src/dict_v4.c b/src/dict_v4.c index 1f38530..4e57d9c 100644 --- a/src/dict_v4.c +++ b/src/dict_v4.c @@ -732,7 +732,7 @@ int steno_dict_lookup(const uint32_t *strokes, uint8_t count, ret = dict_v4_lookup(&dict_singleton, strokes, count, active_dict_id, &slot, &string_id); - LOG_INF("lookup n=%u s0=0x%06x dict=%u -> %d slot=%u sid=%u", + LOG_DBG("lookup n=%u s0=0x%06x dict=%u -> %d slot=%u sid=%u", count, strokes[0], active_dict_id, ret, slot, string_id); if (ret == DICT_V4_FOUND_LOCAL) { @@ -742,7 +742,7 @@ int steno_dict_lookup(const uint32_t *strokes, uint8_t count, } #ifdef CONFIG_STENO_SPLIT_DICT ret = split_dict_get_string(string_id, out, out_size); - LOG_INF("get_string(%u) -> %d", string_id, ret); + LOG_DBG("get_string(%u) -> %d", string_id, ret); return ret; #else return -ENOTSUP; @@ -752,7 +752,7 @@ int steno_dict_lookup(const uint32_t *strokes, uint8_t count, if (ret == DICT_V4_FOUND_REMOTE) { #ifdef CONFIG_STENO_SPLIT_DICT ret = split_dict_resolve(slot, active_dict_id, out, out_size); - LOG_INF("resolve(%u) -> %d", slot, ret); + LOG_DBG("resolve(%u) -> %d", slot, ret); return ret; #else return -ENOTSUP;