v4 union split-section dictionary: both full dicts, zero trimming
Format v4 (docs/FORMAT_V4.md): one union CHD MPHF over 226,791 stroke keys from Plover (147,424) + Lapwing (114,885), sections distributed across halves. Left blob 387KB (displacements log-class Huffman 59KB, membership 57KB, fingerprints 113KB, conflicts 9.6KB, value-index slice 157KB); right blob 515KB (strings front-coded+deflate 193KB, value-index remainder 324KB, conflicts dup). Shared string table: 73,464 unique translations, Lapwing 93% subset of Plover. - tools/compile_v4.py: encoder + full verification (all 262,309 entries byte-exact through cold decode path; hard error if either blob exceeds budget — never trims) - src/dict_v4.c/.h: decoder — canonical Huffman displacement decode, conflict binary search, FC block walk, 2-block LRU cache - src/behavior_steno.c: Plover-style sliding longest-match with retrace replaces multi-stroke timeout + prefix checks - src/split_dict.c/.h: protocol v4 — GET_STRING(string_id) / RESOLVE(slot, dict); decisions are 100% left-local, BLE only for translation text - CMake/Kconfig: STENO_DICT_BOTH default, one compiler run emits both half blobs; legacy MPHF path kept behind STENO_DICT_V4=n - Host round-trip test: 42,000 vectors left→right split path, 0 mismatches (tests/test_dict_v4.c)
This commit is contained in:
parent
dd593c1640
commit
9bf21c9db2
12 changed files with 3338 additions and 666 deletions
74
Kconfig
74
Kconfig
|
|
@ -11,29 +11,43 @@ if STENO_ENGINE
|
|||
|
||||
choice STENO_DICT
|
||||
prompt "Steno dictionary"
|
||||
default STENO_DICT_PLOVER
|
||||
default STENO_DICT_BOTH
|
||||
|
||||
config STENO_DICT_BOTH
|
||||
bool "Plover + Lapwing"
|
||||
help
|
||||
Compile both full dictionaries into the v4 union format.
|
||||
The active dictionary is selected at runtime.
|
||||
|
||||
config STENO_DICT_PLOVER
|
||||
bool "Plover main dictionary (MPHF compressed)"
|
||||
bool "Plover main dictionary"
|
||||
help
|
||||
Use Plover main.json via MPHF compression.
|
||||
Compile only the Plover main dictionary.
|
||||
|
||||
config STENO_DICT_LAPWING
|
||||
bool "Lapwing dictionary (MPHF compressed)"
|
||||
|
||||
config STENO_DICT_TEST
|
||||
bool "Test dictionary (46 entries, simple format)"
|
||||
bool "Lapwing dictionary"
|
||||
help
|
||||
Small built-in test dictionary for development.
|
||||
Compile only the Lapwing dictionary.
|
||||
|
||||
endchoice
|
||||
|
||||
config STENO_DICT_MPHF
|
||||
bool
|
||||
default y if STENO_DICT_PLOVER || STENO_DICT_LAPWING
|
||||
config STENO_DICT_V4
|
||||
bool "v4 union split-section dictionary format"
|
||||
default y
|
||||
select ZLIB
|
||||
help
|
||||
Use MPHF (minimal perfect hash) dictionary format.
|
||||
Compile the dictionary into the v4 union split-section format
|
||||
(docs/FORMAT_V4.md): two half blobs, left (central) holds the
|
||||
lookup structures, right (peripheral) holds the string table.
|
||||
Requires STENO_SPLIT_DICT. Disable to fall back to the legacy
|
||||
single-dictionary MPHF format.
|
||||
|
||||
config STENO_DICT_MPHF
|
||||
bool
|
||||
default y if !STENO_DICT_V4
|
||||
select ZLIB
|
||||
help
|
||||
Legacy MPHF (minimal perfect hash) dictionary format.
|
||||
Selects ZLIB for block-compressed string table decompression.
|
||||
|
||||
config STENO_CUSTOM_KEYMAP
|
||||
|
|
@ -75,39 +89,43 @@ config STENO_MULTI_STROKE_TIMEOUT_MS
|
|||
|
||||
config STENO_DICT_MAX_SIZE
|
||||
int "Max dictionary binary size (bytes)"
|
||||
depends on !STENO_DICT_V4
|
||||
default 473088
|
||||
help
|
||||
Max compiled dict size. 473088 = 462KB.
|
||||
Build fails if dict exceeds this — no trimming.
|
||||
Max compiled dict size for legacy MPHF single-blob builds.
|
||||
473088 = 462KB. Build FAILS if the dict exceeds this —
|
||||
entries are never trimmed.
|
||||
|
||||
menuconfig STENO_SPLIT_DICT
|
||||
bool "Split dictionary across both halves"
|
||||
default n
|
||||
select STENO_DICT_MPHF
|
||||
help
|
||||
Partition the steno dictionary across both keyboard halves.
|
||||
Left (central) gets highest-importance entries for zero-latency
|
||||
local lookup. Right (peripheral) gets remaining entries, queried
|
||||
over BLE on local miss. Both halves have their own MPHF dict.
|
||||
Store the steno dictionary across both keyboard halves.
|
||||
v4 format: left (central) holds the lookup structures
|
||||
(displacements, membership, fingerprints, conflicts, part of
|
||||
the value index); right (peripheral) holds the string table,
|
||||
queried over BLE. Legacy MPHF format: entries partitioned by
|
||||
importance, right half queried over BLE on local miss.
|
||||
Build FAILS if either half exceeds its budget — entries are
|
||||
never trimmed.
|
||||
|
||||
if STENO_SPLIT_DICT
|
||||
|
||||
config STENO_DICT_LEFT_MAX_SIZE
|
||||
int "Left half dict budget (bytes)"
|
||||
default 153600
|
||||
default 440320
|
||||
help
|
||||
Flash budget for left (central) partition. Default 150KB.
|
||||
Central has more code overhead (behavior, formatter, output,
|
||||
undo, BLE client) so gets a smaller dict partition.
|
||||
Highest-importance entries fill this first.
|
||||
Flash budget for the left (central) half blob. Default
|
||||
440320 = 430KB. Build FAILS if the compiled blob exceeds
|
||||
this — entries are never trimmed.
|
||||
|
||||
config STENO_DICT_RIGHT_MAX_SIZE
|
||||
int "Right half dict budget (bytes)"
|
||||
default 512000
|
||||
default 527360
|
||||
help
|
||||
Flash budget for right (peripheral) partition. Default 500KB.
|
||||
Peripheral has less code overhead so gets the bulk of the dict.
|
||||
Remaining entries after left partition is filled.
|
||||
Flash budget for the right (peripheral) half blob. Default
|
||||
527360 = 515KB. Build FAILS if the compiled blob exceeds
|
||||
this — entries are never trimmed.
|
||||
|
||||
config STENO_SPLIT_CACHE_SIZE
|
||||
int "LRU cache entries on central side"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue