Virtual Labs: running network topology on a plain laptop

Let's start with a bit of history. For a very long time, whenever someone wanted to create a network playground, they had to use physical equipment. Although such an approach had some pros, they were quickly outweighed by the cons, such as: the price of equipment, manual cabling required to set up the environment, noise, power consumption - the list is long. 

Everything changed with the development of Dynamips      . This allowed us to emulate some Cisco routers (such as the Cisco 7200, 3600, 3700,  2600, and 1700 series) on traditional PCs. It was not ideal - no official vendor support, only a limited number of equipment was supported (no firewalls, no switches) - but it changed a lot for network enthusiasts.  One could create an advanced topology with a few mouse clicks. Then the virtualization boom started, and the network vendors had to follow. They started to provide virtualized equipment such as Juniper vMX or Cisco ASAv. Not long after that another tool, GNS3      , became popular.

GNS3

The GNS3 tool allows for the creation of virtualized network topologies, with emulation based on virtual machines (VMs). The connections (also emulated) between the VMs are the equivalents of network connections. This approach allows for building a multi-vendor setup that reflects a physical network. That is, a VM image is provided by the vendor. Of course, one can intermix some open-source solutions based on FRR, for example.

GNS3 can be deployed in several ways:

• As a Linux system service - assuming you have a Linux server running.
• As a dedicated VM on a hypervisor platform (kvm/qemu, ESXi, etc) - this is the most flexible approach in terms of deployment but at the cost of nested virtualization.
• As a container, which is similar to the previous point but allows us to avoid nested VMs.

Currently, GNS3 allows us to create advanced topology by using Client UI (or the clientless WebUI).

Source: GNS3      

Each emulated network device is working as a separate VM, and everything is connected together by virtual links. The list of supported vendors is impressive: Juniper, Cisco, Arista, Palo Alto, Mikrotik.       As long as you have access to a virtual appliance image, there is a good chance that GNS3 already supports it. That’s why GNS3 has become so popular: it is easy to understand (thanks to the mature UI) and flexible enough to create advanced topology. However there are some drawbacks that have to be mentioned:

• Main type of appliance is a VM - although this is in line with how the image was provided by the vendor, virtual machines are heavy by nature. Without a rack full of servers you will quickly run out of CPU or memory.

• No official support for automation - this problem exists on two planes:

  • Lab setup - using UI introduces the classic pet vs. cattle problem. You can easily create the topology but integrating it with CI/CD or other automation tools can be tricky. This problem can be mitigated by using Ansible       along with a Python library called gsn3fy, located here      .
  • ZTP - there is no unified way to provide configuration for routers/switches in a newly deployed lab. Again one can use Ansible playbooks for each vendor but there is definitely room for improvement.

• Performance - VMs are deployed in userspace, virtual links are in kernel space. This (and some other decisions) has made the GNS3 rather slow in terms of network throughput. 

• Scale - but this time in terms of several long running labs, permissions, users. GNS3 was created with a single user in mind and it performs great there. However, in a shared environment (such as topologies used for training) it has several drawbacks.

EVE-NG

Some of the problems mentioned above were addressed by the EVE-NG       project which stands for: “Emulated Virtual Environment - next generation”. The first difference that the user sees is in the installation. You are presented with only two options:

• Either as a dedicated VM on a hypervisor platform - similar to GNS3.
• Or as a bare metal server installation.

Such limitation (only BMS or VM) comes from the fact that EVE-NG is using a custom kernel, so running it as a container or systemd service is no longer an option.  EVE-NG attempts to address some of the problems that affects GNS3: 

• Scalability - by using the UKSM kernel feature (that allows sharing of identical memory segments across several identical VMs) it optimizes memory consumption significantly.
• Multi-user architecture - with Web UI access only, permissions can be applied to logged-in users. This allows the creation of administrator accounts, user accounts and others with different access levels.
• Scale - running concurrent labs at the same time for a long time. Although such a feature is still available in GNS3, I believe it's better implemented in EVE-NG.
• UI - while this is not a “problem” per se of GNS3, the WebUI on EVE-NG is extremely nice: 

Source: EVE-NG      

However, the VM appliance configuration provision is still lacking. Also, creating huge topologies (>500 different nodes) still requires significant resources. The automation of lab deployment is lacking as well.

Still, it's a great piece of software for shared environment or training, supporting multiple vendors      . It's definitely worth the price. 

Containerlab

Containerlab       takes a bit of a different approach than the other projects mentioned in this article. As its name suggests, it concentrates more on containerized network appliances (such as SONiC, cEOS, Nokia SR Linux) than on VMs. This doesn’t mean that VM-type appliances are not supported - they are but as another container running qemu inside it. Given that approach, Containerlab can be installed on existing Linux, macOS, and Windows (with WSL) hosts, as long as that host is running a Docker service. After installation is performed the following differences (vs. other projects) are immediately visible:

• No UI - while GNS3 and EVE-NG both provide a nice UI, Containerlab is based on YAML definitions.
• ZTP - as long as a vendor is supported you can provide a configuration file directly for device provision.
• Fewer supported appliances - the list       is not as impressive as in GNS3 or EVE-NG but I can assure you that adding a new device is quite simple with only basic Python skills needed.

One should keep in mind that Containerlab was created with a slightly different application in mind than GNS3 or EVE-NG. Here we are not concentrating on data plane functions (as most of the appliances will be sharing the Linux kernel), but more on protocols or control plane functions. Scalability is also a very strong point of this project. On an average laptop, it is easily possible to create a topology of 200-300 nodes (based on FRR). Such a topology can be used later to get familiar with more advanced network topics such as RSVP, MPLS VPNs, segment routing, etc. Also having YAML as lab definition, Containerlab is suited for integration with CI/CD or other tools as part of a QA process.

If you want to get familiar with this project, I encourage you to read our short blogpost about Containerlab where we deploy a simple topology with two BGP routers in it.

Other projects

GNS3 and EVE-NG are third-party projects and not affiliated with any vendor, meaning they support all of them as long as the user has access to the VM image (which is tricky quite often). However, there are some vendor-specific software options that are worth mentioning here.

Cisco Modeling Labs

The biggest advantage of Cisco Modeling Labs       is the fact that it runs on-prem (which is a must when dealing with sensitive data) as well as providing the cheapest legal way to test Cisco VM appliances (with vNexus and IOS XRv exceptions, the Cisco license prohibits running other images in GNS3 or other virtual labs platforms). It allows building custom topologies and provides an API for automation      . However, there are some drawbacks as well:

• Although it allows running non-Cisco appliances (such as Palo Alto, Fortinet), full automation (with config provision) is only provided for Cisco images. 
• The use of VM images creates the same scalability issues as with GNS3. Also, the personal license is limited to 20-40 concurrent VMs. A license for a higher number of VMs is available but it's pricey.
• We encountered some stability issues as well, but those could be related to our environment.
• It can be deployed only as a VM, so nested virtualization is a must.

Juniper Networks Virtual Lab Services

With Juniper      , our options are much more limited. The provided service is cloud only, topologies are predefined (Juniper only), and access time is limited. So this solution cannot be used in an automated environment and is better suited for end user training. We would strongly recommend downloading the free vEX image       and using it with other projects, assuming you have the resources to run them.

Nvidia AIR 

Nvidia Air       is similar to Juniper (cloud only, Nvidia equipment only), but with noticeable exceptions:

• you can deploy a switch with Cumulus Linux on it or SONiC,
• custom topologies are available,
• you can emulate end hosts with the OS, applications, etc.

Of course, similar to Juniper, you can download a free Cumulus VX       image and use it outside of the Nvidia AIR project.

Summary

The following table summarizes all the projects and tools we have covered in this blog. It is not definitive but should provide a quick overview to find the best lab for your need

This blog entry covers the most popular virtual labs software options that can be found currently. The list is not exhaustive, e.g. a notable missing entry is a netlab project      , but we hope that after reading this article, you will be more familiar with the most popular projects that are available to an average user.