Running Real-ESRGAN and NAFNet on AndroidEngineering a 100% On-Device AI Video Upscaler — Zero Cloud Infrastructure

Introduction: The Cloud Paradigm vs. Edge Intelligence

State-of-the-art super-resolution networks like Real-ESRGAN and image restoration networks like NAFNet (Nonlinear Activation Free Network) have set the gold standard in computer vision. However, deploying these architectures for consumer mobile applications has historically relied almost exclusively on cloud servers.

The cloud paradigm carries massive trade-offs:

1. Severe Privacy Hazards: Users must upload sensitive personal, family, and confidential recordings to remote cloud buckets. Footage is never truly private.
2. Bandwidth & Latency Bottlenecks: Uploading and downloading multi-gigabyte video files over cellular data is slow and expensive.
3. Escalating Cloud Costs: GPU compute hours on A100/H100 instances force cloud apps into exorbitant recurring subscription models ($9.99+/week). In contrast, local edge inference eliminates remote GPU costs entirely, making OnDevice 100% free with unlimited exports.

This article examines how OnDevice — Video Enhancer (com.nabil.ondevice.videoenhancer) bypasses the cloud entirely, deploying optimized ONNX neural models directly onto Android devices — running on consumer Qualcomm Snapdragon, MediaTek Dimensity, Samsung Exynos, and Google Tensor chipsets. Your footage is never sent to a server.

Architecture Overview

The on-device local pipeline is structured into four decoupled, memory-safe stages:

[ Input MP4 / MOV / MKV Video ]
           │
           ▼
┌────────────────────────────────────────┐
│ 1. Native MediaCodec Decoder Pipeline  │
│    Extracts raw RGB/YUV bitstream      │
└────────────────────────────────────────┘
           │
           ▼
┌────────────────────────────────────────┐
│ 2. Edge Neural Network Inference       │
│    • Real-ESRGAN Compact 4x (FP16)    │
│    • OpenCV NAFNet Deblur 1x           │
│    • MobileSR 2x (Fast Conv)           │
│    • AnimeSharp 2x (Line-Art)          │
│    • MobileClear (Laplacian DSP)       │
└────────────────────────────────────────┘
           │
           ▼
┌────────────────────────────────────────┐
│ 3. Foreground Service & Thermal Policy │
│    • Android Game Mode GPU Lock        │
│    • Chunked In-Memory Frame Buffer    │
└────────────────────────────────────────┘
           │
           ▼
┌────────────────────────────────────────┐
│ 4. Hardware Encoder & Muxer            │
│    • MediaMuxer + MediaCodec (H.264)   │
│    • Audio Stream Preservation         │
└────────────────────────────────────────┘
           │
           ▼
[ Output 2x / 4K Enhanced Video — local device storage only ]

At no stage in this pipeline is any video frame, metadata, or user data transmitted to an external server. The entire process — from frame decode to re-encode — occurs within your device's hardware boundary.

1. The Model Suite: Balancing Quality, Compute, and Footprint

A. Real-ESRGAN Cinematic 4x (Photorealistic Texture Synthesis)

• Model Size: 4.8 MB (Optimized compact architecture, FP16 precision)
• Scale Factor: 4× — upscales video to 4K resolution on-device
• Architecture: Compact Residual-in-Residual Dense Block (RRDB) network optimized for ONNX Runtime Mobile execution on Android NPUs.
• Role: Reconstructs fine high-frequency details (foliage, facial pores, fabric weave) from compressed or downscaled source clips. Textures and edges are reconstructed frame by frame entirely on your device.

B. OpenCV NAFNet Unblur & Restore 1x (Motion & Defocus Inversion)

• Model Size: 87.5 MB (Official OpenCV ONNX release)
• Scale Factor: 1× (Preserves native canvas dimensions)
• Architecture: Nonlinear Activation Free Network (NAFNet). By replacing conventional non-linear activations (ReLU, GELU) with simple element-wise multiplications (Gated Mechanism) and simplified channel attention, NAFNet achieves state-of-the-art deblurring performance with a fraction of the computational complexity of transformer-based restoration models.
• Role: Eliminates camera motion shake, vehicle motion blur, and out-of-focus camera distortion — all processed locally on your device's hardware.
• Source: Official OpenCV model: deblurring_nafnet_2025may.onnx

C. MobileSR 2x (Ultra-Fast Lightweight Super-Resolution)

• Model Size: 7.6 MB (Pre-bundled in app assets — zero download required)
• Scale Factor: 2×
• Architecture: Shallow convolutional neural network tuned for sub-second frame inference on mobile ONNX Runtime.
• Role: High-speed 2x upscaling designed for mid-range and budget devices where thermal throttling and battery consumption are top priorities.

D. MobileClear Fast 1x (Native DSP Engine — Zero Download)

• Model Size: 0 MB — no neural model file required
• Scale Factor: 1×
• Architecture: Native digital signal processing using Laplacian high-pass spatial filtering and contrast-adaptive unsharp masking executed directly via Android's DSP API.
• Role: Instant real-time clarity boosting without requiring neural tensor allocation. Runs on every Android device, including budget phones.

2. Solving Mobile Constraints: Memory, Thermal, and Throttling

Memory Management & OOM Prevention

A naive implementation might load multiple frames into a queue as uncompressed 32-bit bitmaps. A single 1080p frame consumes ~8.3 MB of RAM — buffering 60 frames would easily breach Android's per-app memory limit (256 MB–512 MB on standard devices).

Solution: OnDevice implements a strict backpressure pipeline. The decoder extracts a single frame, copies the normalized tensor directly into the ONNX Runtime input tensor, executes inference, and pipes the output directly to the hardware encoder's input surface before releasing the buffer. Peak RAM usage remains constant regardless of video duration — 10 seconds or 10 minutes.

Thermal Throttling & Android Game Mode

Long-running AI workloads cause mobile SoCs to heat up, triggering Android's thermal daemon to throttle CPU/GPU frequencies down to 30% of their base clocks — dramatically slowing frame processing speed.

Solution: The app implements the Android Game Mode API (android.game_mode_config) and utilizes foreground wake locks. By signaling the OS that the application is performing an intensive, continuous compute task, the scheduler prevents erratic frequency downscaling, ensuring consistent frame throughput across long export sessions.

3. Performance Benchmarks (On-Device)

All benchmarks run entirely on-device — footage never leaves the device boundary:

4. Why On-Device AI Is the Future of Mobile Multimedia

The success of OnDevice — Video Enhancer demonstrates that consumer mobile devices have reached the performance inflection point where desktop-class neural restoration models can run entirely locally.

By eliminating remote server infrastructure:

• Users gain ironclad privacy: Family memories and sensitive footage never traverse the internet. Footage is never sent to a server.
• Zero latency over mobile networks: The app works on a flight, in a rural cabin, or with no SIM card installed.
• Fair economics: Without multi-thousand-dollar monthly server bills, developers can offer perpetual licenses rather than predatory weekly subscriptions.
• No single point of cloud failure: No server downtime, no API rate limits, no cloud company data breach risk.

How to Try It

OnDevice — Video Enhancer is built using Kotlin, Jetpack Compose Material 3, and ONNX Runtime Mobile. Available on Google Play:

• Package Name: com.nabil.ondevice.videoenhancer
• Minimum Requirements: Android 8.0 (Oreo) or higher; 4 GB RAM recommended for 4x neural upscaling.
• Processing: 100% on-device — footage is never sent to a server.