AMD Ryzen AI Halo Review: A 'Batteries Included' Dev Kit for AI

The AMD Ryzen AI Halo is a truly mini-PC built around the Zen 5 AMD Ryzen AI Max+ 395 processor(16 core, 32 thread) that streamlines learning AI development with ROCm or AMD hardware. The Max+ 395 processor is equipped with AMD Radeon 8060S integrated graphics which will be doing most of the heavy lifting, and an NPU which historically doesn't do much, but we were finally able to use.

It comes in a single hardware configuration with a removable 2 TB M.2 SSD and 128 GB of unified LPDDR5x-8000 memory capable of 256 GB/s bandwidth. 2 TB is a good amount of storage to hoard local models, and 128 GB is certainly enough memory to load a couple reasonably sized models into memory while reserving some space for system operation.

The AI Halo can be purchased for $3,999.99 USD in a single hardware configuration, preloaded with either Windows 11 Pro or Linux. You are able to load your own OS on the system once you have it, but as far as we know AMD won’t be making the 'factory' Linux and Windows installs(packaged drivers, programs, and models) available.

AMD has sent us the Linux version of the Halo which is running a custom AMD Linux distribution based on Debian 13.4.

Despite the marketing images presenting it as the size of a datacentre, the Halo is an incredibly small box with only a square 15 cm(6 in) footprint and at less than 5 cm(2 in) tall. It weighs 1.2 kg, but if you’re planning on putting this in your backpack then also consider the required 240 W power brick.

The power button and all of the ports are on the back face of the chassis: four USB 3.2 Type-C ports, an HDMI 2.1 port, and a 10 GbE ethernet port. Besides the connectivity on the rear it features Wi-Fi 7 and Bluetooth 5.4. The USB Type-C port closest to the power button is dedicated to USB-C Power Delivery(PD) power input.

There aren’t any clear affordances for stacking them, but the corner feet and air intakes on all sides should make it viable if you need Windows and Linux, or if you want to cluster them. The Halo contains two blower fans to draw air in through the top and sides of the case which is then blown through the heat sink and out the back. This box usually sits quietly, but can ramp up the fans to dissipate the 120 W TDP of the processor inside.

The best feature is the white ring of light around the bottom of the case.(pulsing blue when asleep) It doesn’t cast much light, and it can be turned off, but it gives it a nice look without being gaudy.

Being a tightly integrated mini PC there isn’t too much to see inside but you only have to remove four bolts beneath the removable magnetic feet to lift off the bottom cover. The removable M.2 2280 SSD is easily accessible with no further dissection. Removing the top shell to expose the compute core only requires addressing a few more connections. The core can be pulled out but there isn’t much else to be done. The bottom metal plate visible when first removing the case bottom is removable with four bolts, but we didn’t remove it so as to not mess with the thermal compound underneath.

For previous “AI” specific hardware we’ve used MLPerf and Procyon to measure representative values of hardware performance, but as we get further into locally hosted LLMs and agentic workflows, we’re looking a little deeper. For this testing we’ve focused on using llama-bench, the benchmarking tool packaged with llama.cpp. We’re still exploring tests we can apply with llama-bench as well as other programs that can provide insight, so constructive feedback on test selection is very welcome! We’ve found that nearly every benchmark and benchmarking decision can be met with a “that only applies to these specific cases”. We can’t cover every angle, but for now we’ll at least contribute the tests we conducted on this hardware. LLM token per second performance can be extremely sensitive to compatibility differences, and have high variances; so consider results below with that in mind and consider results from multiple sources.

For those unfamiliar with llama-bench or llama.cpp itself, llama.cpp is an open source inference engine that can be used to load and run large language models(in the GGUF format). It has seen wide adoption for running LLMs because of its simplicity of operation and hardware compatibility. It takes the list of numbers constituting the LLM and orchestrates execution of the model, interfacing with hardware-specific drivers. llama-bench is one of the many command-line-interface(CLI) utilities that are packaged with–and support–llama.cpp. Along with many options and configurations, it conducts two main tests: prompt-processing(pp), and token generation(tg), also known as the pre-fill and decoding phases of inference. Prompt processing is the part of LLM inference where the LLM ‘reads through’ what the user has said to it, and token generation is when it begins outputting tokens back to the user.

The first test is the default llama-bench pp512/tg128 configuration, simulating a user providing 512 tokens, and the LLM generating 128 in response. We tested with Qwen 3.6 35B A3B(Q4_K_M), Gemma 4 31B IT(IQ4_XS), and GLM 4.7 Flash(Q8_0) models. These are 17-32 GB models that have been receiving a lot of attention recently. As with test parameters, preferred models are always changing, but these should give an idea of performance with the currently favoured LLMs. We tested the AMD Halo alongside a Framework Desktop(AI Max+ 395, 128 GB), M2 Ultra(76-core GPU) Mac Studio with 128 GB of unified memory, and M3 Ultra(80-core GPU) Mac Studio with 512 GB of unified memory. For the AI Halo and Framework Desktop we also used both the ROCm/HIP and Vulkan runtimes(backends). None of these devices are direct competitors, but they provide some context. Note that the owner of our company has invested in Framework.

The Apple Silicon Mac Studios outperform the AMD Ryzen AI Max+ 395 machines. This is likely primarily due to their much higher 800 GB/s memory bandwidth compared to only 256 GB/s for the Max+ 395. The prompt processing(pp) portion of LLM inference is typically compute-bound as the CPU/GPU/processor must perform large batches of calculations as it parses the user’s input. This can be seen with the dense Gemma 4 model, where Apple Silicon and the Max+ 395 perform more closely than just comparing memory bandwidths would suggest. We believe prompt processing of the sparse Mixture of Experts(MoE) models Qwen 3.6 35B A3B and GLM 4.7 Flash relies less on compute, allowing the memory bandwidth of the Macs to pull ahead. Token generation(tg) is typically far more memory bandwidth-bound. It requires accessing a lot of data from the LLM and only computes a single token at a time. The processor is typically waiting around for data to be delivered for computation, and with the dense model Gemma 4, the Apple Silicon devices were able to get 2-3x the token per second performance.

In our testing above there was no clear winner between Vulkan and ROCm/HIP backends on the Ryzen AI Max+ 395. The most performant backend relies on many factors like compatibilities, model architecture, context size, specific hardware, and software optimizations. This is not to mention the continuous updates that both backends receive. This testing was conducted with flash attention enabled, but even that doesn't always achieve the best performance so it is best to perform tests on your specific system.

Agentic use of LLMs is becoming far more popular so we concocted a llama-bench test in an attempt to simulate those scenarios, though it involves a slightly more complicated llama-bench command. In theory, this simulates providing instructions to an agent and having it make some tool calls or provide an answer. The important aspect is observing how performance degrades as context size increases.

llama-bench -m <model>
-p 512
-n 64
-b 2048 -ub 2048
-fa 1
-ngl 999
-r 5
-d 0,4096,8192,16384,32768,65536
-o csv

The -p and -n flags specify how many tokens should be used in the prompt processing and token generation tests respectively; this command performs a pp512/tg64 test. -b and -ub specify the batch and micro-batch sizes. The batch size is the number of tokens that llama.cpp will group together for processing, and the micro-batch size is the number llama.cpp will send to the hardware in a single group for calculation. Higher values typically allow greater speed through parallelization, but require more working memory space. It doesn’t impact this test since we’re only processing 512 tokens, but our previous testing found -b/-ub of 2048 to result in the best speeds overall. -fa enables or disables Flash Attention. This defaults to ‘auto’ but in general it should be left on for more efficient memory usage. Compatibility with models and runtimes will vary. -ngl controls the number of model layers offloaded to the GPU. We set it to 999 to specify that all calculations should happen on the GPU. -r controls the number of times each test is repeated before the results are aggregated to provide an average. The default value is 5, but we’ve included it to be explicit. -d specifies the number of tokens already in the LLM’s ‘memory’, its context. Multiple comma-separated values for most arguments allow you to run different permutations of the test in a series. Despite llama-bench’s ability to run the tests in series, we wrote a short script to create our own loop, incorporating a delay between tests to mitigate any thermal soak that could occur as these tests run.

We can see that all three models exhibit significant performance degradation as the context size increases and they all must be considered during pre-fill(pp) or decoding(tg).(Gemma 4 with the Vulkan backend and 65,536 tokens context didn’t complete within 30 minutes)

The AI Halo has a surprisingly small chassis so we were skeptical of its ability to reach and sustain the maximum power of the compute contained inside. To investigate, we ran a simple test to pull the maximum TDP(Thermal Design Power) of 120 W(with a boost of up to 140 W) from the Ryzen AI Max+ 395 package. We also tested the Framework Desktop with the Ryzen AI Max+ 395 alongside it. This is again not direct comparison hardware, but it provides context. For testing we conducted a series of twenty llama-bench prefill tests while measuring the power draw from the wall with a Quarch QTL2843. Both devices were in “Performance” mode with no other changes to power or cooling(the Halo doesn’t allow any customization anyway). We were monitoring HWInfo and AMD tools during the tests to verify package power and temperatures.

The graph above shows the power draw of the AI Halo and Framework Desktop over the test duration of approximately 20 minutes. The AI Max+ 395 in the Framework maintained a steady baseline of 120 W for the entire duration of the test with relatively frequent spikes of power up to 130 W. The Halo started the test at a constant 140 W until the boost was over after 5 minutes, and it settled back down to the 120 W TDP for the remainder of the test.

The blower style design of the Halo with air intakes on all sides kept it cool to the touch at thermal equilibrium, but the bottom became quite warm(around 50°C) and the two blower fans were pushing significant heat out of the rear exhaust. The fans spin up significantly while removing the heat, but the sound profile is a reasonable ‘woosh’ and nothing piercing.

You can get a Ryzen AI Max+ 395 in any number of miniature computers these days; choosing one based on the I/O, cooling, or even just from a favourite manufacturer. The unique offering of the Halo is the AMD Ryzen AI Developer Center, curated configurations(BKC), and promise of continued first-party support. Like NVIDIA’s DGX Spark, it has been designed to provide a development environment that gets out of the way of those who need to test against specific AMD or NVIDIA hardware. Unlike the DGX Spark, the Halo is available in both Linux and Windows variants. We received the Linux version, so all comments will reflect that experience.

The system boots into the AMD Ryzen AI Developer Center which serves as the control panel for software installation and updates, as well as front-and-center access to documentation.

Beyond the OS, the AI Halo and developer center give access to AMD’s “Best Known Configurations”(BKC). These are system configurations for which AMD have validated that all included software, packages, and drivers are intercompatible. This provides a clean, known, starting point for any of the playbooks, and a base for people interested in learning without enduring some dependency hell first. This can be invaluable to streamline the process of getting started with the system, hopefully removing hours of guesswork and following outdated tutorials on the internet. The AMD AI Playbooks then build off of these configurations to guide users to the next step.

The target user of this system is exemplified by comparing the AMD AI Playbooks for the AI Halo against AMD AI Playbooks for other AI Max+ 395 systems. The first step of most of the playbooks is to allocate memory and for regular AI Max+ systems you’re given a series of command line instructions. The AI Halo simplifies this into a slider or dropdown depending on if you’re on Linux or Windows. It is important to note that the Halo doesn’t stop you from doing it the manual way, or using a more complicated setup. It can be ‘dangerous’ to abstract away implementation and hardware details, but this provides a place to begin before diving deeper.

The AMD AI Playbooks were announced and released earlier this year, providing simple tutorials for anyone with AMD hardware to explore AI workloads. AMD are trying to catch up in market share so these are intended to entice new users, or make transitioning from their competitors as simple as possible. While the playbooks focus on the AI Halo, they also have versions available for Radeon GPUs. They’re interesting to read through even just to see what your hardware is capable of. The playbooks are also all available on GitHub along with other useful AMD resources. AMD have committed to keeping these playbooks up to date(like the best known configurations) and introduce new ones each month.

We’ve had the AI Halo for about ten days and had the chance to go through a handful of the playbooks. As intended, they’ve been an easy way to learn our way around the system and explore the software tools that are packaged with it.

This is a simple playbook but a very good place to start. It introduces AMD Sync, a streamlined method to remotely connect to the Halo over the network for live metrics, opening a VSCode project, starting a Jupyter Labs project, or just accessing the terminal. This is another prime example of the AI Halo being the frictionless choice as an AMD developer kit. The playbook was simple to follow and we ran into no issues. It only requires installation of AMD Sync on the remote machine and copying over SSH details. This is of course all possible already with SSH and some configuration, but the quick setup is very convenient. Especially if you’re often resetting this system to factory defaults.

LM Studio and Lemonade are programs for downloading, managing, serving, and interacting with local LLMs. LM Studio is already a popular program for running models, while Lemonade has been developed by AMD more recently to make running LLMs as simple as possible. The LM Studio and Lemonade playbooks are short and simple to follow, walking through installing the software, updating runtimes, and downloading first models. They both end with doing something ‘useful’ with the locally hosted LLMs, using them as a coding assistant or interacting with the OpenAI API programmatically. The VSCode playbook builds on the hosted LLMs from the previous playbooks, connecting a local LLM to a Cline agent within the IDE. If kept up to date, these tutorials could be useful to people not even using AMD hardware, providing a simple walkthrough of all factors to consider.

Getting slightly further into the intended use of this hardware, these playbooks successfully show off how the pre-installed software, drivers, and models can make it as simple as four or five steps to be running an LLM with PyTorch. Unfortunately the playbooks don’t go as in depth as a full tutorial which may walk you through the minutiae of how the scripts work or why each step is required. However, that doesn’t seem to be their main goal. They are extremely successful at quickly getting the dependencies hooked up and providing a smoke test to confirm that things are working. For people inexperienced in the subject, they also provide “Next Steps” with ideas of what to try next.

Personally, I am a huge fan of the software and compatibility support that the Halo provides. I typically learn best ‘from the ground up’, but it can be annoying to follow multiple tutorials and in the end not be able to run anything. The best known configurations and playbooks provide a state which can always be returned to if you venture too far and reach a mess of dependencies. It’s always as accessible as the “System Reset” button in the developer center. As always it is important to be wary of abstractions and disconnecting from what is going on in the background, but at least personally, this wasn’t the case. It’s still just a regular computer and you’re completely free to install and configure it however needed. The BKC and playbooks could also play a part in 3rd party tutorials where they can assume a known starting point or compatibility, making it simpler for users to go further.

As can be seen by the AI Playbook GitHub issues, there are some parts of the playbooks that are currently failing, which I ran into myself. Hopefully those are addressed quickly by AMD as they come up, otherwise this whole thing doesn’t work. The danger of buying based on promises of future value.

To close this off, we’re excited we finally found a device that uses the NPU(Neural Processing Unit). We’ve heard a lot of hype over NPUs in various devices but then every demo we see is running things entirely on the CPU or GPU. With AMD’s Lemonade, we were able to run an LLM on the XDNA 2 NPU using AMD’s FastFlowLM(FLM). Unfortunately the NPU wasn’t providing any usage telemetry, but we were able to see the AI Max+ 395 package draw up to 35 W at essentially zero CPU/GPU usage and generating 20 tokens per second with the gpt-oss-20b-FLM model.

In the defence of all the demos running on CPU/GPU instead of the NPU, NPUs aren't very flashy beyond their names. They often have far less compute than the GPU but in return they have the advantage of much greater energy efficiency. They’re great for operations like fast processing of sensors(like the camera) so that the CPU and GPU are fully available for main tasks. This is an aspect we're interested in investigating further. With this dev kit including the Ryzen AI Max+ 395 with an NPU instead of some giant GPU means that it should hopefully advance the development of more power efficient local LLMs.

The AMD Ryzen AI Halo is entirely powered over USB-C PD. This would be astonishing to someone a couple decades ago, but USB-C is now capable of delivering up to 240 W. The Halo ships with the Delta ADP-240KB BA AC/DC power adapter capable of USB-C PD Extended Power Range(EPR); up to 48 V and 5 A. It is well equipped for the task of supplying the Halo since in our testing the Halo never drew over 200 W from the power supply.

When the power adapter is first connected to power, it broadcasts its SOURCE_CAPABILITIES message to indicate to any connected sinks(devices to be powered) what kinds of voltages and currents it can provide. However, these are only the Standard Power Range(SPR) output modes, limited to 20 V(100 W at 5 A). The Halo sends an EPR_MODE message to prompt the EPR modes from the power adapter. This is responded to with the full EPR capabilities of the power adapter. The Halo requests the fixed 48 V output mode with up to 5 A available, for a total power of up to 240 W.

The new and most interesting part to us is the EXTENDED_CONTROL_MESSAGEs that are continuously sent from the Halo to the power adapter. We’ve seen ‘heartbeat’/’keep alive control messages with PD PPS and AVS, but never before with a fixed voltage output.