Network Automation Forum AutoCon 2 2024: insights and summary

From November 18th to 22nd, Denver became the center for network professionals and network automation enthusiasts as the AutoCon 2 2024 conference took place. This dynamic event offered two days of hands-on workshops followed by three days of inspiring conference speeches, fostering collaboration and innovation in network automation. Representing CodiLime, Monika Antoniak, Tomasz Janaszka, Adam Kułagowski, and Patryk Cichowski actively participated, contributing to the lively exchange of ideas and expertise.

As part of the event, CodiLime led the workshop "Utilize GenAI for network troubleshooting" showcasing how generative AI can revolutionize network operations. Additionally, at our dedicated stand, we demonstrated the Net-Chat Assistant, an AI-powered tool designed to simplify network troubleshooting and automation workflows. The conference provided a fantastic platform to share knowledge, connect with industry leaders, and explore the future of network automation.

Here’s a recap of the key moments and insights we gained during AutoCon 2 2024.

AutoCon 2 workshops

During the first two days of the conference, a total of four sessions were held, each consisting of four parallel workshops. This format provided attendees with a diverse range of learning opportunities tailored to different network automation areas of interest.

We had the chance to participate in nine of these workshops, each offering unique insights and hands-on experience in various technologies and practices. Below, we provide a brief summary of ones we were able to attend, highlighting their key takeaways and what participants could learn and practice. But first, let us start with the workshop we prepared: "Utilize GenAI for network troubleshooting".

1. Utilize GenAI for network troubleshooting (by CodiLime)

   The workshop combined a theoretical overview with hands-on practice to demonstrate how Generative AI can streamline network troubleshooting. The theoretical part introduced GenAI and LLMs, covering tools like GPT-4o-mini, LangChain, and Streamlit, explaining their role in enhancing network operations, including tasks like troubleshooting, documentation, and configuration creation. The session emphasized the potential benefits of GenAI, including simplifying problem-solving and reducing reliance on senior staff, while also discussing GenAI-related challenges like hallucinations, cost, and privacy concerns.

   The hands-on session provided a dedicated VM environment for each participant, with a pre-configured Net-Chat Assistant and a simulated network topology using open-source tools like ContainerLab, FRRouting and VyOS. Participants practiced querying the AI assistant to obtain network information, debug link failures, and simulate real-world scenarios like OSPF misconfigurations. They learned to enhance the assistant by adding support for specific network commands and modifying Python scripts to tailor functionality.

   The lab was divided into four main parts: understanding network topology, handling a link failure, implementing OSPF protocol tools, and debugging configuration errors. Each part involved practical tasks, such as executing scripts to simulate failures, analyzing network states using the assistant, and validating fixes. Participants were also introduced to the process of prompt augmentation for improved AI responses and troubleshooting efficiency. At the end of the workshop, participants were invited to ask questions in their native languages and were amazed to receive valuable answers from the assistant.

   The workshop highlighted the importance of tools and documentation in troubleshooting, offering guidance on integrating AI tools into real-world workflows. It concluded with optional free exploration, feedback collection, and a discussion on the transformative potential of GenAI in network operations while recognizing the need for oversight and further refinement.

2. Data Modeling & Network Source of Truth (by OpsMill)

   The "AutoCon 2 - Workshop B2 Data Modeling" workshop aimed to teach automation builders the foundational technologies needed to store, organize, and consume data, with a focus on schema and database types. It covered essential concepts like the data schema, which defines data structure, relationships, and constraints, ensuring data consistency and integrity. Various schema formats like SQL, JSON Schema, and GraphQL have unique strengths in flexibility and data handling. The workshop highlighted trade-offs in schema design, emphasizing a balance between flexibility, performance, and integrity. The course delved into different types of databases (relational, key-value, document stores, etc.), each suited to specific use cases like scalability or complex data structures. The workshop also explored practical applications such as using flexible data models, understanding the different types of databases, and applying schema at various levels in an application. Advanced topics included schema migrations, inheritance, and polymorphism, which aid in handling complex data relationships and extending schema over time. Finally, the workshop highlighted best practices and challenges in modeling network infrastructure as a Source of Truth for automation.

3. Network Monitoring for Beginners: Getting Started with TPG stack (by Roblox)

   During this workshop, Naveen Achyuta used an example network modeled in ContainerLab to demonstrate how to configure network monitoring tools, including Telegraf, Prometheus, and Grafana (TPG stack). He explained the steps to configure these tools from scratch and integrate them using Docker Compose for efficient monitoring. Additionally, he covered configuring Alertmanager in Prometheus to send email alerts. He showed how to run show commands through the REST API when gNMI does not support certain data. Naveen also introduced Starlark, a lightweight Python-based configuration language, for parsing metrics. The session emphasized the simplicity of Starlark for embedding scripts. Each participant had the opportunity to practice each part of the presented material in a well-prepared, dedicated environment.

   Key takeaways included the ease of setting up network monitoring within a day, using open-source tools. It was highlighted that starting with essential metrics and gradually expanding is a good approach. The importance of maintaining a consistent naming convention for metrics and alerts was also stressed. The TPG stack offers a strong community and numerous integrations with external tools. Once the monitoring pipeline is set up, expanding it is as simple as adding more collections in Telegraf, configuring alerts in Alertmanager, and creating dashboards in Grafana.

4. The Fundamentals of Setting Up and Populating a Network SoT (by Network to Code)

   This workshop was focused on establishing a Source of Truth (SoT) for network automation. It began with an introduction to the concept of SoT, highlighting its role in capturing the intended state of the network for automation and ensuring consistency and reliability. Participants learned the principles of data modeling, structured data import, and onboarding data from networks, covering both manual and automated approaches. The use of Nautobot as a centralized SoT platform was emphasized, demonstrating its extensibility, integration with external systems, and ability to streamline network data management.

   Key discussions addressed the importance of clean data, governance strategies, and prioritizing data population to support short-term automation goals. Labs provided hands-on experience with Nautobot’s device onboarding and system-to-system synchronization, showcasing tools like CSV imports and single source of truth (SSoT) applications for seamless integration with CMDBs and IPAM systems. The challenges of conflicting and duplicate data were discussed, alongside strategies for ensuring accuracy through validation and structured workflows. Participants also explored the Nautobot ecosystem, including apps like Device Onboarding and SSoT, to automate network data synchronization.

   By the end of the workshop, attendees had gained a comprehensive understanding of building and maintaining an SoT, leveraging Nautobot for efficient network management, and integrating it into broader automation frameworks. Practical insights into structured data import, data normalization, and continuous improvement were emphasized as critical to successful network automation.

AutoCon 2 conference

We had the opportunity to attend all the AutoCon 2 presentations, which covered a wide range of aspects related to network automation. Below, we highlight the ones that we found most impressive or particularly relevant to what we consider the most important topics in the field. Each of these presentations stood out for its insightful content, innovative approaches, or practical applicability, and provided valuable perspectives that enriched our understanding of network automation challenges and solutions.

1. After the Network Automation Forum founders' opening remarks, Anna Claiborn (Zayo Group, University of California - Davice), in her opening keynote speech, explored why full network automation remains a challenge, focusing on history, ecosystem, and perception. Historically, monopolistic supply chains and closed systems have hindered progress, unlike the rapid advancements in computing fueled by open-source platforms like Linux. In the ecosystem, open-source network operating systems (NOS) like SONiC and FRR are driving innovation and interoperability, but infrastructure investments still lag behind software advancements. Perception is equally critical, as automation adoption is slower than it seems, following the classic innovation curve. Despite these challenges, evolving tools, shifting job roles, and growing open-source momentum signal steady progress toward fully automated networks.
2. Following this, Mark Coleman (NetBox Labs) gave an instructive talk titled “Delivering the Foundations of Open and Composable Networks”. He outlined the critical role of tools like NetBox in fostering controlled collaboration within network management. Mark highlighted the evolution from chaotic, undocumented networks to structured, well-documented environments by utilizing NetBox to gather, ingest, and validate network data. He emphasized the significant benefits of this approach, including eliminating network drift through automated discovery and ensuring confidence in network documentation. This, in turn, lays a solid foundation for building more reliable, scalable, and composable network infrastructures.
3. Next, Chris Grundemann (Network Automation Forum) shared insights from the “2024 State of Network Automation Survey”, which revealed a growing adoption of network automation frameworks (NAFs) such as Ansible, NetBox, and Nautobot, with 40% of networks reporting some level of automation. The survey highlighted diverse approaches to automation, shaped by varying investment and staffing strategies, but also uncovered a lack of consensus on which aspects of networking should be prioritized for automation. A major hurdle remains the automation skill gap, underscoring the critical need for more training and expertise to fully harness the potential of network automation.
4. The next speakers, Aaron Drydale (QuikTrip) and Joe Wesch (Network to Code), shared their journey to network automation, addressing QuikTrip’s operational challenges, such as enabling IT staff to complete complex tasks, conducting state checks, reducing human errors, and ensuring speed and consistency across a multi-vendor environment. They outlined the key role of Nautobot in implementing a design-driven automation approach, which enables seamless device configuration, life cycle management, and integration with external systems while maintaining a unified interface for diverse teams. Key features of their integrated solution included "one-click" network changes, centralized audit logs for visibility, and tailored tools for role-based access. Their lessons learned highlighted the importance of modularity, automated data imports, and user-focused design for logs and workflows.
5. Continuing the theme of innovative applications, Jeremy Schulman (Major League Baseball) gave an engaging talk on “AI-driven advanced network observability”, exploring whether AI should act as "better microscopes" or "better doctors" in network management. He highlighted the use of technologies like natural language processing (NLP), named-entity recognition (NER), and retrieval-augmented generation (RAG), combined with tools like NetBox, Selector.ai, and custom-built ChatOps for actionable insights and interactive dashboards. Schulman emphasized that building an effective observability system is complex, requiring a mix of commercial, open-source, and DIY solutions tailored to specific needs, with a strong focus on leveraging context-aware private metadata for more accurate and secure network insights.
6. Transitioning to the theme of efficiency, Michael Daly (Imperva) delivered an excellent talk titled "The Tale of 2 Henrys: Precision vs. Accuracy" comparing network automation approaches inspired by Henry Ford's focus on efficiency and Henry Royce's obsession with perfection. He explained that Imperva’s approach aligns more closely with Ford's practical, production-focused methodology to address challenges like outdated systems, lack of communication between tools, and minimal documentation. The solution involves implementing tools with single responsibilities, enforcing rigorous coding standards, maintaining a shared state for consistency, and ensuring only one system manages edge changes, creating a streamlined and reliable automation framework.
7. Adding to the conversation, Ryan Shaw (Zscaler) gave a provocative talk titled "Comprehensive Infrastructure Automation" emphasizing the need for a shared understanding and systemic thinking in network automation. He sparked lively discussions by questioning whether the automation journey leads everyone to the same goals and highlighting the importance of a "map" and "lexicon" for effective collaboration. Ryan outlined a framework for signal-driven workflows that span the network life cycle (design, build, deploy, validate, operate) using composable, reusable building blocks, while stressing that business logic must drive automation for it to be meaningful and scalable.
8. The next day was kicked off by Hari Gollapalli (PayPal). He shared PayPal's journey to network platform engineering with a focus on automating processes to meet increasing demands for reliable, secure, and scalable tools despite small teams and tight budgets. PayPal developed a framework to standardize and accelerate application development by automating common tasks like UI coding and database handling, allowing developers to focus solely on business logic. This approach resulted in 25+ applications in 18 months, with a uniform user experience, faster concept-to-release cycles, and improved operational metrics such as mean time to resolution. Hari emphasized treating infrastructure as a platform, adopting microservices architectures, leveraging open-source tools, and aligning automation efforts with modern cloud standards to deliver more with less.
9. Following this, Peter Sprygrada (Itential) shared his perspective on why network automation, despite more than 15 years of discussion, has yet to achieve the widespread adoption many anticipated. He outlined the evolution of automation, transitioning from the consumer phase, focused on operational efficiency and foundational tasks like zero-touch provisioning, to the builder phase, emphasizing value creation, self-service APIs, and service orchestration. Peter stressed the need for a product mindset, where automation and orchestration work together—stateless operations handle shared infrastructure, while stateful services enable tailored capabilities. This approach unifies isolated automation efforts into coherent workflows, paving the way for scalable and efficient network services.
10. Building on these insights, Matthew Deibel (Southern California Edison) discussed “Intent-Based MPLS Router and WAN Provisioning”, highlighting the evolution of network operations from manual processes to intent-based networking, where automated actions align networks with dynamic business needs. He outlined the challenges of migrating legacy TDM transport to MPLS, addressing aging infrastructure, and implementing a new deployment model, which was overcome through highly standardized designs and internal automation capabilities. By leveraging tools like Linux containers, React/Python web apps, and databases such as Oracle Metasolv and MySQL, they streamlined device configuration and service deployment, resulting in annual savings of $1.8 million, reduced misconfigurations, and improved employee morale. The key takeaway was that network automation is a continuous journey requiring investment in organizational competencies and culture.
11. Transitioning to testing, Danny Wade (BlueAlly) reminded us all of the importance of thorough network testing as the foundation for successful automation, highlighting various testing approaches like unit, integration, end-to-end, and regression testing. He demonstrated how tools like Nornir and Cisco pyATS enable streamlined workflows by collecting snapshot data, validating configurations, and running automated tests to ensure network functionality and stability. Success stories included implementing network access controls with Nornir and Pytest, as well as hardware refreshes across multiple sites using pyATS, showcasing how robust testing frameworks reduce errors, improve reliability, and provide actionable insights through detailed reporting.
12. Shifting focus to practical applications, John Jackson, Matt Miller, and Ryan Ring (OpenText and CloudMyLab) shared how a single use case kickstarted their network automation journey, transforming manual, error-prone processes into streamlined automated workflows. They highlighted a critical customer problem: manually updating firewalls with malicious IP lists, which was time-consuming, prone to errors, and had caused a major outage. Their solution automated every step, from importing IP lists to firewall configuration and ticket updates, reducing a full-time effort to minutes while eliminating human error and improving oversight. This approach not only resolved immediate challenges but also laid the foundation for broader automation, leveraging OpenText tools to drive efficiency and reliability across IT operations. At the end of the presentation, they threw socks to the delighted participants.
13. William Collins (Alkira) then shared strategies for driving network automation initiatives in large enterprises, focusing on overcoming organizational inertia and leveraging informal channels for influence. Using the analogy of bicycles versus trains, he highlighted the challenges enterprises face—complex communication chains, high stakeholder involvement, and slow decision-making—compared to the agility of startups. William outlined the importance of mastering "Layer 8," the human and organizational layer, by navigating politics, building cross-team relationships, and packaging technical ideas in ways that resonate with executives. He concluded with actionable advice on capitalizing on opportunities through data gathering, storytelling, and tailoring messages to different audiences to translate technical solutions into impactful organizational change.
14. In the next speech, Jose Miguel Izquierdo (Juniper Networks) discussed using advanced tools to create automated, standardized, and reusable network documentation, addressing common challenges like time-consuming manual processes, human error, and inconsistent results. He proposed combining tools such as Markdown, Mermaid, Jinja2, Ansible, and Docker to build a streamlined workflow, integrating version control and automation platforms like Git and Jenkins for effective change management. Jose emphasized starting with simple tools, even without coding expertise, and defining clear goals and workflows to ensure documentation is repeatable, consistent, and easy to maintain.
15. Transitioning to AI, Du’An Lightfoot (AWS) delivered a convincing talk on building an AI-powered network agent, emphasizing the transformative potential of generative AI in network operations. He outlined key use cases for large language models (LLMs), including documentation, health checks, threat detection, and root cause analysis, while addressing challenges like hallucinations, privacy concerns, and lack of domain knowledge. Du’An highlighted strategies for adapting to the AI era, such as starting with small, specific use cases, fostering an innovation culture, and enhancing AI capabilities through collaboration, domain-specific models, and governance frameworks, ultimately framing AI as a tool to empower humans rather than replace them.
16. Concluding the theme of AI, Phillip Gervasi (Kentik) discussed the human factor in using LLMs for network operations, emphasizing their role in augmenting human work by making tasks faster, easier, and more insightful. He explained how LLMs, leveraging transformer models and attention mechanisms, can process large datasets, facilitate programmatic data analysis, and enable autonomous network operations workflows. While highlighting benefits like faster root cause analysis and reduced manual effort, he also addressed challenges such as hallucinations, privacy concerns, and the need for human oversight to ensure reliable outputs. Gervasi encouraged starting small, using tools like LangChain, vector databases, and Streamlit to build workflows, and fine-tuning LLMs for domain-specific tasks to maximize their effectiveness in network operations.
17. The Friday session was started by Mircea Ulinic (DigitalOcean), who presented a comprehensive approach to network automation, showcasing an architecture integrating tools like NetBox, Peering Manager, and Salt for configuration management, orchestration, and monitoring. He emphasized the importance of data accuracy, prefix validation, and event-driven capabilities, highlighting the benefits of using Kubernetes-deployed Proxy Minions and Prometheus-based monitoring for scalability and regional optimization. The session stressed that automation extends beyond configuration, encompassing the entire network life cycle, and underscored the value of open-source solutions tailored to specific business needs while preparing for the AI revolution in network operations.
18. Finally, Scott Robohn (Network Automation Forum) discussed "Total Network Operations" (TNOps) as a holistic framework aimed at elevating network operations beyond traditional boundaries, emphasizing community engagement and the application of DevOps principles to NetOps. He highlighted the need to address gaps in skills, tools, and organizational readiness, urging leaders to prioritize continuous improvement, independent reviews, and investment in team development. The talk underscored the inevitability of disruption in NetOps and the necessity of adapting workflows and embracing new technologies, including AI and event-driven orchestration. Scott encouraged building frameworks that balance traditional practices with modern approaches like NetDevOps, fostering collaboration, and aligning network operations with broader business objectives. TNOps aims to provide resources, support grassroots efforts, and advocate for the evolution of NetOps as a critical, strategic component of IT infrastructure.

Net-Chat assistant live demo at the CodiLime stand

At our booth, we showcased a special demo version of the GenAI-supported tool, Net-Chat Assistant, designed to answer questions about a demonstration network. Visitors were invited to sit at a laptop and ask their own questions, which they enthusiastically embraced. They were impressed by how logically and autonomously GenAI determined which scripts to execute, which specific network devices to query, and how precisely it delivered answers to their inquiries. But there was more to explore. Visitors could simulate network failures directly through the user interface and ask questions about which devices were impacted and which had lost communication. Additionally, they could experiment with switching between different LLM models for each question. We also provided an opportunity to observe how the Net-Chat Assistant reasoned through its steps and performed autonomous actions to reach accurate answers. Smiles and nods of approval were frequent, affirming the tool’s practical value. Here’s a short video capturing the experience our visitors enjoyed.

Conclusion 

AutoCon 2 2024 brought together network professionals in Denver to explore the latest developments in network automation, including workshops and presentations on practical solutions and emerging technologies. CodiLime showcased its GenAI-powered Net-Chat Assistant, which impressed visitors with its ability to autonomously troubleshoot network issues, aligning with future expectations for GenAI agents. The hands-on workshops provided actionable skills, covering topics like generative AI for troubleshooting, network Sources of Truth, and the TPG stack for monitoring. Key conference sessions highlighted challenges such as skill gaps and legacy systems, alongside innovative approaches like intent-based networking and modular automation frameworks. CodiLime demonstrated its strong position in understanding the future role of AI in network automation through its forward-thinking attitude. Finally, we believe it is worth reflecting on a statement frequently echoed during the speeches: "AI is not going to replace humans, but humans with AI are going to replace humans without AI." – Karim Lakhani, Professor at Harvard University.