Android Concurrency in Practice: Thread Pools, Queues, and Coroutine Dispatchers

I took over an IM module optimization last year where the production ANR rate sat at 2.3%. Opening a trace revealed a single unbounded thread pool handling all message send/receive traffic, with thread counts spiking past 200 during peak. In Android interviews everyone recites the four thread pool types, but in real engineering work, most developers stall somewhere between Handler and coroutines, only half-aware of the middle ground.

Where Thread Pools Fit in Android Concurrency

Android concurrency has three core tools:

• Handler/Looper: single-threaded serial execution, suited for UI operations and simple async tasks.
• ThreadPoolExecutor: multi-threaded parallel execution, suited for CPU-intensive and batch I/O work.
• Kotlin coroutines: structured concurrency, suited for orchestrating complex async flows.

The problem in many projects is jumping straight from Handler to coroutines, skipping the thread pool layer that sits between them. Coroutine dispatchers are backed by thread pools underneath — without understanding that mapping, using coroutines becomes “tuning by feel.”

Core Parameters: Behavior, Not Formulas

ThreadPoolExecutor(
    corePoolSize = 4,           // core thread count
    maximumPoolSize = 8,        // max thread count
    keepAliveTime = 30, TimeUnit.SECONDS,
    workQueue = LinkedBlockingQueue(128),
    threadFactory = NamedThreadFactory("msg"),
    rejectedExecutionHandler = CallerRunsPolicy()
)

When a task arrives, the pool checks three conditions in order:

1. Current thread count < corePoolSize → create a new thread immediately, regardless of idle threads.
2. Current thread count ≥ corePoolSize and queue isn’t full → enqueue.
3. Queue is full and thread count < maximumPoolSize → create a non-core thread.

There’s a counterintuitive consequence: if corePoolSize is too large, the queue barely sees any traffic, because the pool prioritizes thread creation over enqueuing. I saw a project with corePoolSize set to 32 — the queue metric was always zero, thread count held steady at 32, completely defeating the pool’s reuse purpose.

Queue Selection: Bigger Isn’t Better

// Bounded queue — the recommended engineering choice
LinkedBlockingQueue(128)

// Synchronous queue — requires immediate thread availability
SynchronousQueue()

// Priority queue — orders tasks by priority
PriorityBlockingQueue(64)

A bounded queue is the safe engineering choice. An unbounded queue (no capacity specified) lets tasks pile up indefinitely when submission rate outpaces processing, eventually causing OOM. SynchronousQueue suits burst scenarios but on Android it easily triggers RejectedExecutionException unless paired with a sensible rejection policy.

Rejection Policy: CallerRunsPolicy Is the Android Answer

Of the four rejection policies, I have a clear preference for CallerRunsPolicy in Android:

// When rejected, the calling thread executes the task directly
val executor = ThreadPoolExecutor(
    corePoolSize = 4, maximumPoolSize = 8,
    keepAliveTime = 30, TimeUnit.SECONDS,
    workQueue = LinkedBlockingQueue(64),
    rejectedExecutionHandler = ThreadPoolExecutor.CallerRunsPolicy()
)

The reasoning is practical: CallerRunsPolicy provides natural backpressure. When the pool is at capacity, the thread submitting the task (often the main thread or a producer thread) runs the task itself, naturally slowing the submission rate. The other three policies — AbortPolicy (throws), DiscardPolicy (silently drops), DiscardOldestPolicy (drops the queue head) — all risk losing tasks in Android business scenarios where delivery matters.

A pitfall to watch for: when tasks are submitted from the main thread, CallerRunsPolicy can cause the main thread to execute long-running work and trigger an ANR. The fix is ensuring pool tasks aren’t submitted from the main thread, or adding timeout control on the submission side.

Thread Factory: Name Your Threads

class NamedThreadFactory(private val name: String) : ThreadFactory {
    private val group = Thread.currentThread().threadGroup
    private val threadNumber = AtomicInteger(1)

    override fun newThread(r: Runnable): Thread {
        val t = Thread(group, r, "$name-${threadNumber.getAndIncrement()}", 0)
        t.isDaemon = false          // Android thread pools should use non-daemon threads
        t.priority = Thread.NORM_PRIORITY
        t.uncaughtExceptionHandler = CrashHandler()
        return t
    }
}

Naming threads isn’t cosmetic. In production debugging, the difference between pool-1-thread-3 and msg-dispatch-3 is that the latter lets you pinpoint the business module directly in a trace. uncaughtExceptionHandler is equally critical — exceptions in pool threads don’t propagate to the main thread automatically. Without a handler, tasks fail silently and you won’t even see a log.

Coroutine Dispatcher to Thread Pool Mapping

Kotlin coroutine dispatchers all land on Java thread pools. Understanding this mapping prevents many pitfalls:

// Dispatchers.Main — Android main thread Looper
// backed by HandlerDispatcher, not a thread pool

// Dispatchers.IO — elastic thread pool
// default max 64 threads, created on demand, reclaimed after 60s idle
// equivalent to: newCachedThreadPool but with a ceiling

// Dispatchers.Default — CPU-intensive thread pool
// thread count = CPU core count, minimum 2
// equivalent to: fixed-size thread pool + global work queue

Dispatchers.IO and Default share the same thread pool — a detail that’s easy to miss. When IO dispatcher threads are idle, Default reuses them. If you run heavy CPU computation inside an IO coroutine, you’re eating into the Default dispatcher’s thread budget.

Practical rule: IO coroutines should only do I/O. For computation, switch with withContext(Dispatchers.Default).

Custom Dispatchers: Thread Pools as Coroutine Hosts

// Create an isolated pool for a specific business domain
val businessExecutor = ThreadPoolExecutor(
    2, 4, 60, TimeUnit.SECONDS,
    LinkedBlockingQueue(32),
    NamedThreadFactory("business")
)
val businessDispatcher = businessExecutor.asCoroutineDispatcher()

// usage
viewModelScope.launch(businessDispatcher) {
    // runs in an isolated pool, unaffected by IO/Default dispatchers
}

This pattern suits high-frequency tasks that need isolation: push notification processing, log writing, batch database operations. After converting with asCoroutineDispatcher(), you keep all of structured concurrency’s benefits (cancellation propagation, exception handling) while achieving complete thread pool isolation.

Thread Governance in Practice

Monitor First

Thread pools are invisible until they break, and then they’re catastrophic. Wire up monitoring before going to production:

class MonitorableExecutor(
    corePoolSize: Int, maxPoolSize: Int,
    keepAliveTime: Long, unit: TimeUnit,
    workQueue: BlockingQueue<Runnable>,
    factory: ThreadFactory
) : ThreadPoolExecutor(corePoolSize, maxPoolSize, keepAliveTime, unit, workQueue, factory) {

    override fun beforeExecute(t: Thread?, r: Runnable?) {
        super.beforeExecute(t, r)
        val metrics = Metrics(
            activeCount = activeCount,
            queueSize = queue.size,
            poolSize = poolSize
        )
        reporter.report(metrics)
    }

    override fun afterExecute(r: Runnable?, t: Throwable?) {
        super.afterExecute(r, t)
        if (t != null) reporter.reportError(t)
    }
}

Watch three dimensions: activeCount consistently near maximumPoolSize means you need to scale up; queue.size growing steadily means processing can’t keep up; P99 task latency over 500ms needs investigation.

Thread Pool Lifecycle

// proper shutdown sequence
executor.shutdown()           // stop accepting new tasks, wait for queued tasks to finish
if (!executor.awaitTermination(5, TimeUnit.SECONDS)) {
    executor.shutdownNow()    // force-interrupt after timeout
}

After shutdown(), the pool enters SHUTDOWN state — no new tasks accepted, but queued tasks continue executing. shutdownNow() returns the list of unexecuted tasks for compensation handling. Forgetting to shut down pools in Activity/Fragment onDestroy or Application onTerminate causes memory leaks — pool threads hold references to their ThreadGroup, which indirectly holds a Context.

Tuning thread pools isn’t about memorizing parameters. It’s about understanding the three pathways: thread creation, queue accumulation, task rejection. In practice, my habits are:

• One pool per business module to prevent cross-contamination.
• Bounded queue + CallerRunsPolicy as the default combo.
• Pool monitoring as a non-negotiable launch checklist item.
• For high-frequency coroutine tasks, isolate with asCoroutineDispatcher().

The essence of concurrency debugging: check thread count first, then queue depth, then task latency. That order resolves 80% of thread pool mysteries.