Exploring the Orange Pi 6 Plus: A Deep Dive into ARM Board Performance

Over the past few months, I delved into the Orange Pi 6 Plus, expecting a straightforward review of another ARM board, only to find myself immersed in its intricate hardware and software dynamics. The Orange Pi 6 Plus, powered by the CIX P1 SoC, boasts 12 CPU cores, a Mali G720 GPU, and a dedicated NPU, promising a robust performance for homelab and edge AI applications. However, the real challenge lay in the software, which led me to build my own OS images to ensure optimal performance.

## Hardware Insights

The board's hardware is intriguing, featuring a mix of Cortex-A520 and Cortex-A720 cores, dual 5GbE ports, and a comprehensive set of connectivity options. The asymmetric core architecture, with varying clock speeds, offers a unique performance profile that requires careful workload management. The dual 5GbE setup, in particular, enhances its appeal for network-intensive applications.

## Custom OS Build

To unlock the board's potential, I embarked on creating a custom Debian 13/Trixie build, stripping away Ubuntu dependencies and ensuring a server-first layout. This involved significant modifications, including baking in GPU/NPU prerequisites and ensuring a deterministic first boot process. The effort paid off, as the custom image provided a stable platform for further testing.

## Software Challenges and Solutions

The software journey was fraught with challenges, particularly with the GPU and NPU support. Initial attempts with vendor images proved inadequate, prompting a deep dive into vendor-specific packages and driver configurations. Successfully binding the GPU to the vendor stack and resolving NPU userspace inconsistencies were key milestones.

## Performance Testing

Performance testing focused on local AI inference, revealing the board's strengths and limitations. While not a universal AI powerhouse, it handled specific models and runtimes effectively, particularly with Vulkan-patched llama.cpp. The tuning process highlighted the importance of runtime and model selection, with CPU inference occasionally outperforming GPU offload.

## Thermal and Power Considerations

Thermal management and power consumption were critical considerations. The board's active cooling system, while effective, was notably loud, and the power draw was higher than typical SBCs. Despite this, the board maintained stable performance under load, with no thermal throttling observed during extensive benchmarking.

## Living with the Orange Pi 6 Plus

After a month of continuous operation, the board proved reliable, hosting development projects and serving as a testbed for AI experiments. Its power consumption, though higher than average, was manageable, and the stability exceeded initial expectations.

## Conclusion

The Orange Pi 6 Plus excels in specific roles, such as edge AI and compact Linux services, thanks to its dual 5GbE and capable hardware. While the software requires significant effort to optimize, the board's potential for local AI and systems work is undeniable. Despite higher power consumption and fan noise, it offers a more polished experience than many competitors, showcasing the progress of ARM boards in recent years.