Tokenization Articles

On-Device LLM Tokenization and Decoding on Android

How BPE tokenization, Top-K, and Top-P sampling shape quality, latency, and memory use in Android on-device AI.