An AI web application firewall on GPU and DPU, with 0% host CPU

The challenge

A classic GPU-based firewall reads each packet with the host CPU, sends it to the model on the GPU, waits for the verdict and then drops or forwards it. Every packet crosses memory several times, and the inspection runs inside the system it protects. Moving it off the host was an unusual use case: the DPU has its own PCIe root complex, the GPU sits on the host’s, and the model had to run inline without the CPU and without adding latency.

What was at risk:

• Host CPU spent on inspection, malicious traffic reaching the host before a verdict, and inline ML adding latency to every request

What we did

• A fully offloaded design: the DPU handles traffic, the GPU runs the model, and the host CPU only sets things up
• A direct device-to-device path between a GPU and a DPU on separate PCIe ports, using GPUDirect RDMA and P2P DMA, with GPU memory pinned in shared mode for the DPU
• A DOCA ML-based firewall application on the DPU, with an in-house embedded TCP/IP stack that extracts HTTP request data
• Only the relevant part of each request sent to the GPU, which cuts PCIe traffic and latency
• A custom PyTorch plugin that records the model as a self-launching CUDA Graph, so inference runs unattended on the GPU
• A verdict loop on the DPU: it reads each model result over DMA and drops the packet or passes it to the host HTTP server

How it works

Traffic is inspected and filtered by the DPU and the GPU together, with the host CPU idle during processing, in five stages.

1. Initialization An initialization app on the host prepares shared buffers in GPU memory that the DPU writes into and the model reads as PyTorch tensors. The model is captured in a CUDA Graph, so the GPU can schedule its own execution. This is the only work the host CPU does for the firewall.
2. Extraction on the DPU Packets arrive at the DOCA firewall app on the BlueField DPU. Its embedded TCP/IP stack handles the connection and extracts the HTTP request data, so only that part of the packet travels on.
3. Transfer to the GPU The DPU writes the extracted data straight into the shared GPU buffer with a DMA write, over P2P DMA and GPUDirect RDMA, with no host CPU in the path.
4. Inference On arrival, the GPU runs the WAF model on the batch through CUDA Graphs tail launch. The proof of concept used the mobilebert-sql-injection-detect model to detect SQL injection.
5. Verdict The DPU reads the model’s result with a DMA read and decides whether to drop the packet or forward it to the HTTP server on the host.

The design accepts other models, as long as the DPU can fill the model’s input through memory operations. The same pattern extends to blocking whole traffic streams with DOCA Flow, and to device-to-device transfers between NICs, GPUs and storage.