Evolution of management protocols for network devices

Connecting and powering network devices (be it in the form of physical hardware or virtualized/containerized functions) is not enough to make them operational - the devices have to be configured before using them to forward / filter / load balance, etc. the traffic. And most likely, later one will need to reconfigure, monitor, diagnose and troubleshoot faults to ensure the network is always available, reliable, and secure. All those tasks have one thing in common - network management. As its name denotes, network management is the process of administering, monitoring, and optimizing a network to ensure its efficient operation and performance. Network devices have a management plane (to compare different planes check out our article: Management vs. Control vs. Data Planes in a Network Device) allowing those actions to be performed. 

Imagine a scenario where a company's network spans multiple locations with numerous routers, switches, servers, and other devices. Each of them needs to be monitored for performance, configured for specific tasks, and troubleshot in case of issues. Without a standardized method of communication between these devices and a centralized management system, this task would quickly become unmanageable. Hence network management protocols, which provide the necessary framework to interact with network devices in a structured and efficient manner. 

What are network management protocols?

Network management protocols define the rules and procedures for information and commands exchanged between network devices and network management systems (NMS). These protocols specify how data is exchanged, what types of information can be requested or modified, and the security measures in place to protect this communication. Their goal is to allow managing (configuration, maintenance, troubleshooting, etc.) network devices in an efficient and secure manner and to provide tools to build resilient, scalable and performant networks. Typical tasks supported by these protocols include:

• Monitor device health and performance metrics
• Configure settings such as IP addresses, access controls, routing protocols
• Detect and respond to network events such as failures and security breaches
• Collect data for analysis and reporting

Network management protocols facilitate the implementation of network management systems, which serve as the centralized application, providing a single interface through which administrators can manage their network elements.

Data modeling

Network management protocols often rely on data modeling language for the definition of data exchanged between device and application. Data modeling language specifies rules and conventions which defines the entities, their attributes, constraints, relationships and hierarchical organization of data.

Nowadays, one of the most important data modeling languages is YANG (Yet Another Next Generation). It was developed as a data modeling language for the NETCONF protocol (YANG 1.0 specification was published in 2010 as RFC 6020       and the updated YANG 1.1 was published in 2016 as RFC 7950      ). However YANG is protocol independent and its data models can be used regardless of the underlying transport protocol.

There are a number of RFCs which define standard YANG data models. The OpenConfig project       (open-source project with contributions from network operators, equipment vendors, etc.) is an important source of data models (for configuration and telemetry) which are vendor-neutral (written in YANG version 1.0). Additionally vendors create their own models or extend existing ones; e.g. OpenConfig models with data specific to their solutions. 

The evolution of network management protocols

The history of network management protocols is marked by a constant evolution to meet the growing demands of increasingly complex network devices. Early protocols laid the foundation for basic monitoring and management. However, as networks expanded and became more diverse (e.g. through virtualization or containerization of network functions), the limitations of those protocols became apparent. This led to the development of more advanced protocols that offer features such as improved security, support for more complex configurations, and better integration with modern software development practices.

Today there exist numerous network management protocols, including both proprietary ones created by a specific vendor (and supported within their respective ecosystems) and industry-standard protocols supported by many vendors and applications.

In the forthcoming section, we delve into the world of network management protocols, exploring their features, strengths, weaknesses, and real-world applications. Our goal is to provide a comparison of the prominent network management protocols: SNMP, NETConf, RESTConf, and gNMI.

SNMP

SNMP stands for Simple Network Management Protocol. It is the first widely recognized network management protocol and was initially developed in the 1980s, with its first specification (known as SNMPv1) published in 1988. Since then new versions, namely SNMPv2 and SNMPv3, have been developed and published. The main characteristics of those versions are:

SNMPv1:

• Basic functionalities for monitoring and management.
• Limited security features, relying on plaintext community strings for authentication.

SNMPv2(c):

• Introduced in 1993.
• Added features like GETBULK for more efficient data retrieval.
• Improved error handling and error reporting.
• SNMPv2c (commonly known as "Community-based SNMPv2”) - version introduced for easier configuration (instead of the security model proposed by SNMP v2, uses community-base security scheme from SNMP v1), considered as the de facto SNMPv2 standard.

SNMPv3:

• Introduced in 1999.
• Adds enhanced security with the inclusion of authentication and encryption.
• Includes support for User-Based Security Model (USM) for message security and the View-Based Access Control Model (VACM) for access control.
• Secures operation with message integrity verification,  message timeliness checking, origin verification and confidentiality.

Nowadays it is common that all three versions are supported by networking devices.

In SNMP one can distinguish two types of elements:

• SNMP Agent - runs on managed devices (like switches or routers) and has knowledge about the device and its operations. It answers requests (GET requests) and generates notifications (TRAP messages) for specific events.
• SNMP Manager - software responsible for network management and monitoring through the SNMP protocol. It communicates with SNMP agents using GET, SET, and other SNMP operations.

SNMP uses Structured Management Information - SMI (a subset of Abstract Syntax Notation One, in short ASN.1) to define its notation (type of data) and employs ASN.1 BER (Basic Encoding Rules) to encode messages. SNMP stores its data in a Management Information Base (MIB). MIBs specify the structure of data, which is organized in hierarchical, tree-like structures, with each node representing a different aspect of the device or network being managed. In SNMP, a managed object is represented as a variable, and the Object Identifier (OID) is used to uniquely identify each variable. In that notation each node in the tree is represented by a series of integers separated by dots, with each integer representing a sub-branch in the tree. For example 1.3.6.1.4.1.2021.4.5.0 is the OID for total RAM in a machine. Aside from the standard OIDs, MIBs can contain vendor-specific ones (supported/available only on devices from a given vendor). 

SNMP Agents use OIDs to report the value of managed objects to the SNMP Manager, and the Manager uses OIDs to retrieve particular values from the agent. Likewise, SNMP Manager can configure a network device by setting the values of a chosen parameters (through usage of OID). Additionally SNMP supports notifications (SNMP TRAP) when SNMP Agent sends information to SNMP Manager about certain events (e.g. a change of value of a given OID). 

SNMP primarily uses UDP (User Datagram Protocol) as its transport protocol. SNMP agent receives requests on UDP port 161, notifications (TRAP messages) are sent to SNMP manager port 162. SNMP can also use TCP (Transmission Control Protocol) for some operations; SNMPv3 in particular allows for the use of TCP as a transport protocol. 

Usage of SNMP as a network management protocol protocol has a number of benefits. However SNMP has its own limitations and drawbacks. One of them is the lack of standard discovery mechanism to find proper MIB modules for a given network device - a network administrator has to manually configure the OIDs of the devices within the monitoring system (in such cases, one can for example employ snmpwalk to retrieve a subtree of management values). It is worth noting that standard MIBs are also available, not tied to a specific vendor and device (proprietary MIBs typically extend the exposed properties of functionality of the devices of a given vendor).

Another limitation stems from the fact that SNMP has not been designed to retrieve huge amounts of data efficiently. Due to performance limitations when polling, SNMP is not well suited when managing large networks. Also it has limited filtering and retrieval capabilities. SNMP's primary focus is on monitoring, and its configuration capabilities are limited compared to more modern protocols.

One of the main concerns related to SNMP is security. From that perspective, only SNMPv3 should be used as it provides authentication and payload encryption. 

NETCONF

NETCONF (Network Configuration Protocol) is an IETF standard protocol described and published in 2006 in RFC 4741       (and later superseded by RFC 6241       with updates and extensions provided by RFC 6242      , RFC 7803      , RFC 8526      ). It was designed to address the limitations of older protocols, such as SNMP, by providing a more powerful and flexible way to configure, manage and automate network devices. 

In the NETCONF protocol four layers are distinguished:

Fig.2: NETCONF Protocol LayersSource: RFC 6241      

These layers have the following roles:

1. The Secure Transport layer provides a secure and reliable communication channel between client and server. NETCONF can use  different transport protocols with SSH-based being the most common.
2. The Messages layer provides a transport-independent framing mechanism for encoding remote procedure calls (RPCs) and notifications.
3. The Operations layer specifies a set of base protocol operations which are run as RPC calls. Among others they include: , , , , , supporting a range of configuration and management tasks. Additionally, clients and servers may extend the base set of operations (customizations are supported).
4. The Content layer is responsible for configuration and notification data exchanged between devices.

NETCONF follows the client-server model, where a client can be, for example, an application running on a separate system and the server is the network device implementing NETCONF. Communication between client and server is based on Remote Procedure Calls (RPC). The NETCONF client initiates operations to manage network devices, while the server responds to client requests.

NETCONF distinguishes configuration data and state data (e.g. read-only status information and statistics). The protocol allows checking the configuration before applying it. This is possible thanks to the support of multiple datastores (like running, startup, and candidate, a vendor may implement different datastores). When a configuration from the candidate store can be applied and copied to the running datastore after being validated (and if validation fails it is not applied). Having different datastores allows managing each of them separately.

One of the strengths of NETCONF is the fact it is transaction-oriented, which allows atomic changes in configuration, meaning that when configuration changes are pushed to the network device(s), they are applied only if there are no errors /  all operations succeed, otherwise all changes are withdrawn (there is no partial configuration change in such situation). This also means that a single transaction can contain different configuration operations and be pushed to multiple devices and be applied only if successful on all of them (e.g. the addition of a new VLAN on several switches, setting options for network interface, etc.)

The NETCONF protocol uses XML-based data models to describe device configuration and state information with YANG as a data modeling language. This facilitates writing automation scripts and tools that can interact with network devices and perform tasks such as configuration management, monitoring, and troubleshooting. 

Servers (run on network devices) can send asynchronous notifications to clients about specific events, such as changes in device state or exceeded thresholds. These notifications, defined using YANG models, provide real-time updates to network management systems, aiding in proactive monitoring and troubleshooting.

NETCONF uses a secure channel (using SSH / Transport Layer Security) for communication between client and server with authentication, encryption, and data integrity. 

While NETCONF offers significant advantages, it might be considered complex as it requires understanding YANG data modeling and XML messaging as well as XML-based messages which, though verbose,  potentially increase bandwidth usage.

RESTCONF

RESTCONF is specified in RFC 8040       where is described as, “an HTTP-based protocol that provides a programmatic interface for accessing data defined in YANG, using the datastore concepts defined in NETCONF". 

RESTCONF has been created to provide the HTTP-based REST API, allowing access to configuration data, state data, data-model-specific RPC operations, and event notifications in the networking device. REST stands for representational state transfer and is a type of software architecture that has been designed to ensure interoperability between different web applications. Nowadays it is one of the most common approaches used in internet systems. This also means that many automation solutions use that design principle. From that perspective, RESTCONF simplifies management tasks as it enables automation of network infrastructure using familiar RESTful API patterns.

RESTCONF uses HTTP methods to provide CRUD (CREATE, READ, UPDATE and DELETE) operations on datastores containing YANG-defined data (compatible with a  server that implements NETCONF datastores).  This means that the operations applied by RESTCONF are the standard HTTP methods such as GET, POST, PUT, PATCH and DELETE. 

The protocol works on hierarchical resources, where each resource is a manageable entity within the network device. A resource is defined as a collection of data and available methods on that data. Examples of resources include device configurations, interfaces, routing tables, and system status. Resources are identified and accessible by URIs (Uniform Resource Identifiers), e.g. https://example.com/restconf/data/ietf-interfaces:interfaces/interface=eth1.

RESTCONF uses client-server architecture and is stateless (operations are one-and-done, no active sessions are kept open with the client). 

RESTCONF does not support the concept of transactions. In fact, each RESTCONF call (be it  GET, POST, PUT, PATCH and DELETE) is a transaction in itself and is directly applied. As a logical consequence, distributed transactions are not available - configuration is done operation by operation and device by device. Nor is a locking mechanism supported.

RESTCONF does not provide validation options without activating a given configuration (validation is implicit, part of the RESTCONF calls which succeeds or fails).

The RESTCONF protocol allows data to be encoded with either JSON or XML. As a transport, HTTP or HTTPS is used.

RESTCONF with its web-friendly architecture, use of standard HTTP methods, and lightweight data encoding make it a powerful tool for network administrators and developers. Its stateless nature allows it to scale effectively to manage large and complex networks. However, when compared to NETCONF, RESTCONF offers a less comprehensive set of operations for advanced configuration management (e.g. the handling of transactional operations and rollbacks is challenging). The verbose nature of JSON or XML encoding can result in increased bandwidth usage, especially in environments with frequent configuration changes (which might be an issue, especially in bandwidth-constrained use-cases).

gNMI

gNMI (gRPC Network Management Interface) is a network management protocol that leverages gRPC (framework implementing remote procedure call) to set, edit, and delete the configuration of network devices, and also to view operational data. The protocol is defined by the OpenConfig Working Group (gNMI specification      ) and allows for configuration and telemetry data exchange between network devices and management systems.

gRPC       is an open-source remote procedure call (RPC) framework, initially developed by Google. It uses HTTP/2 as its transport protocol, allowing for efficient communication between clients and servers. gRPC offers features such as bi-directional streaming, authentication, and multiplexing. 

Protocol Buffers       (aka protobuf) provides the data serialization in gRPC. The resulting data can be characterized by its compact binary representation, which significantly reduces the size of data payloads. This reduction in size translates to decreased bandwidth usage, thereby optimizing the efficiency of data transmission across the network. The schema of exchanged data is defined in protobuf .proto files, providing a clear specification of the structure and types of data elements. 

Moreover, Protocol Buffers are seamlessly integrated with YANG data models, which serve as the foundation for defining the structure of network configurations and operational data. In this collaboration, YANG models delineate the detailed structure and semantics of the data, while Protocol Buffers are responsible for the tasks of serialization and encoding.

gNMI uses client-server architecture (with clients establishing a connection with gNMI servers). It implements four RPC methods: 

• Capabilities - exchanged by the client and server in the initial handshake to share capability information.
• Get - used to retrieve configuration or operational data from network devices (gNMI server) by client.
• Set - enables clients to modify configurations on network devices, clients send desired configuration changes, which are applied atomically on the device(s).
• Subscribe - allows controlling subscriptions to data on the network devices by the client. Provides real-time telemetry streaming from devices to clients, which subscribe to specific paths and receive updates when the data changes.

gNMI supports transitions - when a client sends a Set request to modify configurations on a network device, it can include multiple configuration changes within a single gNM request. These changes are bundled together, ensuring that either all changes are successfully applied or none at all.

In conclusion, gNMI stands as a contemporary network management protocol offering a unified service for state management, encompassing both streaming telemetry and configuration operations. gNMI boasts efficient data serialization capabilities, resulting in significantly smaller data sizes compared to protocols such as NETCONF. This efficiency is particularly advantageous for managing significant volumes of data and accommodating numerous concurrent connections, making it well-suited for large and dynamic network infrastructures.

The protocol's support for streaming telemetry is an important advantage, enabling continuous and real-time updates without the need for frequent polling. This feature enhances the responsiveness of network monitoring and facilitates timely decision-making based on up-to-date data. However, it is essential to note that implementing gNMI requires a degree of familiarity with the underlying technologies it relies upon, such as gRPC and Protocol Buffers. While these technologies offer powerful capabilities, they may present a learning curve for those new to them. Additionally, as a newer protocol in the network management landscape, gNMI may not yet have achieved the widespread adoption of more established protocols like SNMP or NETCONF.

What is worth noting is that the OpenConfig working group has defined the specifications for two related management protocols:

• gNOI (gRPC Network Operations Interface)
• gRIBI (gRPC Routing Information Base Interface)

They can be considered as extensions to the gRPC-based network management framework, providing specific functionalities tailored to network operations (gNOI allows execution of operational commands on network devices) and routing information management (gRIBI provides an API for adding and deleting routing entries).

Protocols comparison

For a comprehensive comparison of the above-considered network management protocols, the table below presents how they compare across key aspects.

Each protocol offers some unique features, advantages and weaknesses, catering to different aspects of network management. SNMP remains a base protocol for network monitoring, with widespread adoption. However, it also poses important security challenges and has limited efficiency. NETCONF and RESTCONF provide standardized approaches to network configuration with YANG used as a modeling language. NETCONF offers transaction support and structured data handling. gNMI stands out as a modern protocol, leveraging gRPC for efficient communication and Protocol Buffers for data serialization. Its support for streaming telemetry and transactions makes it ideal for large and dynamic networks.

When determining the optimal network management protocol for a given environment, it is essential to first assess the compatibility with the network elements. Each use case requires individual consideration. A general approach may involve the following considerations:

• SNMP remains a widely adopted protocol, supported by numerous network devices. It is well-suited for traditional network environments focused on monitoring. However, taking into account security, only SNMPv3 should be considered.
• For organizations transitioning to more modern infrastructures, both NETCONF and RESTCONF offer structured configuration capabilities. RESTCONF, with its RESTful design, presents a more web-friendly approach, while NETCONF is more comprehensive and provides support for more configuration options.
• In dynamic and large networks, gNMI emerges as a compelling option. Its support for transactions and efficient data encoding ensures superior scalability and performance. Although, as a relatively new protocol, its support might be more limited. 

Summary

Management protocols act as a bridge between network management systems and network devices, enabling efficient communication, centralized control, and ultimately, effective network management.

The evolution of network management protocols reflects the ever-changing landscape of networking requirements. While SNMP laid the foundation for network management, protocols like NETConf, RESTConf, and gNMI have emerged to address the limitations and cater to the demands of modern, dynamic networks. The choice of protocol depends on factors such as network complexity, scalability, security, and the need for real-time updates. As technology continues to advance, network management protocols will likely evolve further to meet the challenges of managing complex and diverse network infrastructures.

The coexistence of various protocols will almost certainly continue, as different environments have different requirements. One of the notable trends is the move towards a more standardized and efficient approach to network management. This is also highlighted by  initiatives like OpenConfig, which not only defines network management protocol (gNMI as well as gNOI and gRIBI) but also common, vendor-neutral data models (YANG based).

In conclusion, the landscape of network management protocols is dynamic and diverse. Network administrators should assess their specific needs, consider the advantages and limitations of each protocol, and adopt a strategy that aligns with their network's requirements. With the right mix of protocols and technologies, they can navigate the complexities of modern networks with efficiency and agility.