ONUG Fall 2026 takeaways: Networks must change faster, and few teams have a safe place to test

After ONUG Spring we wrote about a coming "patch storm": AI finds vulnerabilities faster, so remediation windows get shorter. At the AI Networking Summit in New York on October 28–29, this was no longer a forecast. One panel was titled "The Summer of Patch Hell", and the people on stage were describing their current workload.

Tomasz Janaszka and I also spoke at the event. Our talk, "No Shortcuts: Making Agentic Network Operations Reliable Enough to Trust      ", showed an AI agent handling a network incident on a test network. We prepared it before the event, so it was interesting to hear enterprise leaders on the main stage ask for the same things: evidence, a safe place to test, and a person making the final decision.

This article combines what we heard with what we have learned from our own work.

Patching is now an operating model problem

In his keynote, Tom Gillis of Cisco said that AI tools have increased the number of vulnerabilities found by about two orders of magnitude, and that nobody can patch their way out of this. The usual habit is to test an upgrade for months, deploy it, and then skip versions. He argued for what cloud operators already do: many small changes, all the time. Cisco's short-term answer is compensating controls, which use eBPF to block a specific exploit on a running system without a reboot. Gillis was clear that this buys time and does not replace the patch.

The panel showed how hard this is in practice. The moderator quoted an average of 20 days to deploy a patch, while exploits now follow disclosure within hours. Many of these teams automated patch deployment years ago. What they lack is a quick way to know what will happen after they deploy.

Frequent change only works when each change is routine. This is what we see in firewall operations. For a network security solution provider, our upgrade automation runs more than 20 pre- and post-validation checks on each upgrade. It supports over 10,000 firewall upgrades a year, with more than 100 in one maintenance window. The numbers matter less than the method: the checks are the same every time, so an upgrade stops being a special event. We described this approach in more detail in Safe network and infrastructure changes with pre- and post-checks.

Few teams have a safe place to test

This was the strongest message for us. A healthcare CTO said they cannot afford a real copy of production, so they use virtual patching and AI-based simulation. Another infrastructure leader has no dev or User Acceptance Testing (UAT) network at all, so network patches go straight to production. An aerospace leader wants to know that a change was simulated in a lab before it can affect operations. The moderator raised the idea that companies will need a digital twin to simulate every change.

We reached the same conclusion from the other direction. When we built our agent, we could not test it on a production network, and test networks are scarce and expensive. So we built our own digital twin lab, Holodeck. It runs real routing software and real traffic. We can reset it to the exact moment a problem starts and repeat the same experiment many times. The demo in our talk was recorded there.

A twin will not reproduce every hardware detail, and it does not need to. It should show how your routing and traffic behave when a change goes wrong. That is enough to rehearse a change, or to test an agent, before it reaches production. Our earlier post on practical uses of digital twins in networking covers more options.

What rehearsing a change can look like

1. Model what matters. Build the part of the network the change touches: sites, roles, addressing and routing. You do not have to start with everything.
2. Apply the change to the twin. Use the same configuration or migration step you plan to use in production, and watch what happens to routing and traffic.
3. Break it on purpose. Add the failures that worry you, such as a flapping link or a policy mistake that withdraws routes, and see how the network and your team respond.
4. Practice the way back. Confirm that the rollback works and how long it takes.
5. Make it repeatable. Turn the scenarios into a suite that runs before each change window, as part of your CI/CD pipeline.

The result is a change board decision based on evidence. It also gives you a safe place to evaluate new tools, including AI agents, before they get near production.

Nobody lets AI do this alone

The moderator asked the audience who lets AI deploy patches without a person approving each change. No hands went up.

He also pointed out that your network already changes itself. A routing protocol reroutes traffic after a link failure, and you trust it because it is deterministic. AI is not, and it can be wrong with the same confidence as when it is right. Speakers listed what they would need before giving automation more room: deterministic actions, pre- and post-checks, instant rollback, and policy as code.

Our demo showed what that looks like in practice.

1. A BGP partner was leaking routes into our network, hours before anything would break. \
2. A forecasting model raised the alarm, exact code measured the leak, and the agent proposed fixes from a fixed menu of reviewed operations, each with its risk.
3. Before each change, it showed exactly what would change, and a person approved every one.
4. Then the agent read the router back to prove it worked: the session stayed up, and the partner’s routes dropped to what we had agreed, just as the preview predicted.

Once a procedure like this has proven itself, we turn it into a fixed, tested, versioned workflow. The agent uses a proven workflow first and improvises only when there is none. This is how we would expect the boundary to move: one proven case at a time, based on evidence.

Agents need foundations before they scale

The CISO of FedEx gave the clearest list of what scaling agents requires: clean data, identity and access management for agents, observability with a full audit trail of agent actions, and a security control plane. He said FedEx is deliberately patient about this. Sandboxes and demos are easy. Control at the scale of a large enterprise is not. Zscaler added a useful number: it has seen individual users with up to 75 agents.

We use a similar model. In our design, each agent has its own identity and an owner who answers for it. It can do only what both the user and the agent are allowed to do. Every change needs approval, and every step is recorded. In our experience, an agent that works and an agent that works securely are very different projects, and the second takes far more effort. Tomasz wrote about the details in Six MCP security gaps when AI tools access network infrastructure.

FedEx also showed a practical view of model costs. Developers get fast answers from a low-cost model, and a frontier model scans the whole code base in nightly batches. We came to a similar place: not every job belongs to a large language model. In our demo, a specialized time-series model produces a forecast in about a tenth of a second, and the language model receives the result as a fact.

Old security ideas matter more than ever

Most of the advice from security speakers was not new: segmentation, zero trust, a smaller blast radius, knowing what you have exposed. What changed is the urgency. The enterprise speakers who felt best prepared were the ones who started segmenting years ago.

Gillis said that policy management is the hardest part of segmentation, and advised starting with large boundaries and refining them over time. Network capacity can also be the limit. A global application security company we worked with had reached the ~4,094 VLAN limit. Moving to EVPN/VXLAN on SONiC gave them about 16 million segments, as part of a migration of more than 10,000 VMs to an open platform with near-zero user-visible disruption and no hypervisor license fee.

Customers are building more

The CISO of FedEx said the buy-versus-build decision has changed for his team. AI lowers the cost of building and maintaining bespoke tools, so he now leans towards building. FedEx is also training technology staff as forward-deployed engineers.

Others added a caution: getting something into production is much harder than it looks. We agree with both sides. For a global e-commerce marketplace, we helped build an intent-driven orchestrator for its SONiC data center fabrics. It reduced the time to define a new fabric from weeks to 1–2 days. No product described that network well enough, so building was the right choice. It worked because of the modeling and network knowledge behind it. AI makes the code cheaper, but not that knowledge.

The panel on what suppliers owe customers also suggested that teams will change. Network, security and compute specialists will need to work much closer together, because a critical vulnerability cannot wait for a chain of tickets and meetings. Speakers warned against treating AI as a way to cut engineers. Some companies that did this are already hiring people back. This matches how we start our own projects: with network engineers, not with AI. Before any code, they work out how a problem shows up, what data answers it, and which fixes are safe.

Trust, consolidation and budgets

Best-of-breed has been the safe choice for years, but stitching products together creates small gaps, and attackers can now find those gaps much faster. One leader said this may force more trust in fewer platforms. At the same time, nobody wants to pay more. AI is often funded by savings elsewhere, and IT is one of the first places asked. The expectation was to help in re-engineering the environment and removing cost, not a swap of one brand for another.

Whatever you buy, complexity is the common enemy. Simplifying the environment came up again and again as a condition for moving at speed. Any new tool, including an agent, also has to fit what you already run. In our talk, we showed that identity, cloud and security policies should be settings, not a rebuild.

Networks for AI: Ethernet and open ecosystems

Broadcom gave the main infrastructure talk. Its argument was that Ethernet has won scale-out networking for AI, and that the next problems are topology, power, and congestion. It presented Tomahawk 6, a 102.4 Tbps switch chip, as a way to build a cluster of up to 128,000 GPUs in two tiers, and stressed open choice of hardware and operating systems, including SONiC.

Open fabrics are attractive, but they ask more from the operator. They need the same automation, testing, and observability discipline as the rest of the network.

What we would do now

1. Create a safe place to test changes. Start with the changes that scare you most, and rehearse them on a digital twin.
2. Automate the checks, not only the deployment. Run the same pre- and post-checks on every change, with a defined rollback.
3. Fix your network data. AI cannot rank risk or find affected devices if your inventory and telemetry are scattered or wrong.
4. Segment in small steps. Start with large boundaries, such as separating dev from prod.
5. Give AI read access first. Let it forecast, analyze and recommend. Extend its permissions based on measured results.
6. Turn proven procedures into workflows. Keep them tested and versioned, and let people approve what cannot be undone.
7. Simplify before you automate. Each exception you remove is one less thing to test on every change.

Where we can help

Tell us where the gap is

We closed our talk by asking people to tell us where the gap is in their network, and the invitation stays open. Contact us if you want to:

• See the longer demo. We will walk you through the full incident, from the forecast to the approved fix, and answer questions about how it is built.
• Talk through a change you worry about. Tell us about the upgrade or migration you keep postponing, and we will show how we would test and roll it out.
• See Holodeck. If you have no realistic environment for testing changes or agents, we can show what a digital twin can do for your network.
• Review your first agent use case. If your team is starting with agents, we can discuss which case to begin with and what it needs in data, tools, and approvals.