An application, a system or even a simple device is made up of several components, each of which performs specific functions to the best of its ability, while leaving other tasks to other components which are dedicated to taking care of them. For example, a web application may consist of a HTTP server which provides the UI, a database server which keeps information about users, and a content server where images and videos are stored.
Such an approach to dividing responsibilities is no different in networking devices, where hardware and software components work together to analyze and forward traffic according to the configuration applied by the administrator.
While the number of components in a networking device is definitely large, they usually can be grouped into three main categories:
- the management plane,
- the control plane,
- and the data plane (also known as forwarding plane).
Let’s talk about each of them to see the differences, the responsibilities, and some actual examples.
The Management Plane
The management plane allows administrative access to the device, analysis of its state and health and its reconfiguration. The access components may include an SSH server and a CLI daemon where operational and configuration commands can be issued. It may include a web server that provides a graphical interface better suited for some maintenance activities than the CLI. It may also include other access methods such as NETCONF, REST API and SNMP that are especially useful for management systems and automation solutions.
The management plane would also be hosting processes that monitor the state of the device itself, such as fan speed, power supply failures, and the status of components belonging to the other planes (e.g. the STP process in the control plane or networking interface states in the data plane).
Some devices will have a dedicated management network interface with its own routing table, which allows management traffic to completely bypass the data plane and not be dependent or affected by it in any way.
When the administrator is changing the configuration, the management plane will communicate with the control plane and issue changes to appropriate control plane processes.
Usually, there is no direct access for the administrator to control plane components - all the information is gathered and all the changes are carried out by the management plane.
The Control Plane
The control plane is where the processes responsible for running network protocols are located. This is where Spanning Tree Protocol (STP) stores information and makes sure BPDUs are sent out at appropriate intervals, this is where ARP mappings between IP and MAC addresses are kept and this is where the routing table is stored and updated by protocols such as BGP.
Some control panel components will communicate with each other (for example the BFD process with dynamic routing protocol processes), but many of them will also communicate with neighboring network devices.
For example, the LLDP protocol will send out and receive information about what devices it directly connects to.
The OSPF protocol will send and receive link-state advertisement messages which carry information about the Layer 3 topology of the network, allowing the router to find the best routes to available subnets.
The LACP protocol will send out its BPDUs to dynamically create aggregated ethernet connections that provide higher throughput and redundancy.
It’s important to notice that the control plane does not have dedicated network interfaces and all the communication is carried out by the revenue ports located on the data plane. So the control plane cooperates with the data plane to send and receive packets belonging to different protocols and forward them to appropriate processes.
The control plane is also able to issue commands and to program the data plane, based on information learned by the networking protocols. For example, the STP process may instruct some interfaces to block traffic in order to avoid loops in the Layer 2 topology.
The Data Plane (Forwarding Plane)
This is the part of the device where the revenue (non-management) network interfaces are located, as well as all the hardware and software components needed to quickly process network packets and forward them from the ingress to the egress interface.
Packet processing and forwarding may be done in software using a general purpose CPU, and this is often the case for less expensive devices where high traffic throughput and low latency is not critical. However, in switches and high-end routers, for example, almost all processing is done by purpose-built ASICs or FPGAs which provide the best performance while having low power utilization.
In the case of devices such as firewalls there is often a mix of software and hardware processing, where TCP/UDP-based filtering can be done by dedicated ASICs, while deep-inspection such as web filtering could be done by a typical CPU.
Putting it all together
The diagram below shows how the different planes work together in order to create a functional networking device. While the diagram is simplified and the communication paths between components may differ among vendors, the general idea stays the same.
We get three different planes, each with its own dedicated components, resources and responsibilities.
But why should one go to all this trouble, why not just have a simple, monolithic architecture?
In the case of networking devices that would be a very problematic approach (which is quite often the case in software solutions, where in many cases microservices architecture is put in place of a monolithic one).
Let’s say a CPU core is shared between the management plane and data plane. In times of high network traffic the CPU could be 100% utilized by packet-forwarding tasks, which could lead to an inability to access the device and issue configuration changes.
Similarly, a CPU core shared between the management and control planes might prove problematic as well. We could, for example, enable process logging at the debug level in the management plane, which could lead to problems for protocols that require precise timings and fast reactions, such as BFD (on some devices the time-sensitive elements of the BFD component are actually run in the data plane to avoid similar issues and to allow for rapid neighbor fault detection).
This is why the resources assigned to networking device planes are often dedicated, or at least there are specific limits put on resource utilization for different components. This could still be a single CPU core and shared RAM, but the plane components would be assigned RAM usage limits and the OS would employ task prioritization to protect the more demanding and important processes from CPU starvation.
For high-end devices the data plane is commonly composed of actual dedicated hardware, with its own CPUs, RAM and ASICs, and sometimes even a separate operating system. This way, even if all the interfaces and forwarding chipsets are running at 100% utilization, the control and management planes are not affected.
Conclusion
The delegation of tasks to different components is nothing new in systems architecture, especially in the age of microservices and containerization. While networking devices may not yet implement the newest trends they are not that far behind either, and great care is taken by many vendors to properly split all the responsibilities between dedicated processes and chipsets.
When choosing a networking device it's good to invest some time in researching the general architecture and whether or not the planes are separated from each other and what kind of separation approach is applied. Thanks to that, we can trust our network to be reliable and stable even at times of very high utilization, letting the network administrators sleep soundly at night.
