TFLite Articles

Android On-Device AI Inference Warmup: From Model Loading to First-Token Latency

A practical breakdown of on-device AI cold-start latency: model loading, GPU Delegate initialization, KV cache prefill, warmup inference, long-lived contexts, and memory tradeoffs.

Designing an On-Device LLM Inference Scheduler: Priority Queues and Backpressure in Practice

This article shows how to build a scheduling layer above an on-device inference engine, using priority queues, preemption, and backpressure to avoid OOMs, unpredictable latency, and out-of-order results.

Android On-device AI Image Preprocessing: From Bitmap Pixels to Tensor Input

A full Android on-device AI preprocessing pipeline from Bitmap pixels to tensor input, covering memory layout, pixel conversion, resize choices, normalization, and zero-copy optimization.