Raspberry Pi AI HAT+ 2 and the Hailo-10H: Running Local LLMs on a Pi 5

The original Raspberry Pi AI HAT+ and AI Kit, built around Hailo's Hailo-8/8L accelerators, are genuinely good at one thing: real-time computer vision. Object detection, pose estimation, segmentation — all fast, all local, all well documented on this site's Frigate and object-detection guides. What they were never built for is generative AI. A 26-TOPS vision accelerator with no dedicated memory of its own can't hold a language model's weights in any useful way. The Raspberry Pi AI HAT+ 2, announced in January 2026, exists specifically to close that gap.

What's Actually New

SpecDetail AcceleratorHailo-10H Performance40 TOPS at INT4 Onboard memory8 GB dedicated RAM on the HAT itself — separate from the Pi's own RAM InterfacePCIe, Raspberry Pi 5 only (no Pi 4 support) Price$130 Vision performanceEquivalent to the original 26-TOPS AI HAT+ for computer vision workloads

The 8 GB of dedicated onboard RAM is the part that matters. It means the Pi's own 4 GB or 8 GB of system memory isn't what's constraining model size — the HAT effectively brings its own memory pool sized for holding small-to-mid quantized model weights, which is what makes running an LLM on a $130 add-on to a $80 board plausible in the first place.

What Ships and Runs at Launch

Hailo's launch lineup targets small, INT4-quantized models rather than anything you'd mistake for a datacenter LLM:

• DeepSeek-R1-Distill (1.5B parameters)
• Llama 3.2 (1B parameters)
• Qwen2.5-Coder (1.5B parameters)
• Qwen2.5-Instruct (1.5B parameters)
• Qwen2 (1.5B parameters)

Vision-language models (VLMs) for multimodal tasks — image plus text prompts — are supported too, and Hailo has said larger models are planned as post-launch additions. The board also supports LoRA (Low-Rank Adaptation), so you can fine-tune a small base model for a narrow task — a support-ticket classifier, a shop-specific parts lookup assistant — without retraining the whole network.

Setting It Up

1. Hardware: the AI HAT+ 2 connects to a Raspberry Pi 5 over the PCIe FFC connector, the same physical mounting path as the original AI HAT+ and NVMe HAT boards. If you're already running an NVMe SSD off the Pi 5's PCIe lane, you'll need to decide which one gets it, or move to a carrier board that breaks out more than one PCIe lane.
2. Software stack: install the Hailo runtime and drivers the same way you would for the original AI Kit, then add the hailo-ollama backend, which exposes an Ollama-compatible API but routes inference through the Hailo-10H instead of the Pi's CPU.
3. Frontend: pair it with Open WebUI for a ChatGPT-style browser interface, or hit the Ollama-compatible API directly from a script or Home Assistant integration.
4. Camera stack: for VLM work, the board integrates with the existing libcamera / rpicam-apps / Picamera2 stack, so the same Pi Camera Module setup covered in this site's camera guide feeds straight into a vision-language model instead of (or alongside) an object detector.
5. Model compatibility note: models compiled for the original Hailo-8/8L are not automatically compatible — they need to be recompiled against the Hailo-10H target before they'll run.

Where This Fits Against CPU-Only Local LLMs

This site already covers running Ollama directly on a Pi 5's CPU. The honest comparison: CPU-only Ollama on a Pi 5 works, but tokens-per-second on anything above a 1–2B model gets painful fast, and the whole Pi is tied up doing it. Offloading to the Hailo-10H's dedicated 40-TOPS INT4 path leaves the Pi's CPU free for whatever else it's doing — Home Assistant, a camera pipeline, a web server — while the HAT handles inference. The tradeoff is the models themselves: INT4 quantization and a 1–1.5B parameter ceiling at launch means you're getting a fast, responsive small assistant, not GPT-4-class reasoning. For "summarize this sensor log," "classify this support message," or "describe what the camera just saw," that's plenty. For anything requiring deep multi-step reasoning, it isn't, and no amount of prompt engineering on a 1.5B model will fully close that gap.

The AI HAT+ 2 doesn't replace the original AI HAT+/AI Kit for vision work — if all you need is fast object detection for Frigate or a security camera pipeline, the cheaper Hailo-8L kit still does that job well and there's no reason to pay more. Where the HAT+ 2 earns its price is the specific case of wanting a genuinely local, offline, small-model AI assistant bolted onto hardware you already understand, without shipping a single sensor reading or camera frame to a cloud API.