Testing the basic functionality of a router in your home can be relatively easy - we connect it to our PC, open an Internet speed test, and watch as basic parameters such as available throughput are measured. Things become more complicated when our task is to perform complex tests on a network device on a carrier/data center network model, which in some cases, transmits traffic at hundreds of gigabits per second. What tools can we use in such a case?
Professional traffic generators from vendors such as IXIA and Spirent are coming to our rescue.
Capabilities of modern traffic generators
Traffic generators play a crucial role in network testing, simulating various traffic types from basic Layer 2/3 to complex application-level interactions. These tools enable detailed network performance assessments under diverse scenarios, offering functionalities for configurable traffic streams, network protocol emulation, and application traffic simulation. The following discussion highlights how traffic generators facilitate comprehensive testing, ensuring networks are optimized to meet contemporary digital communication demands.
Generation of L2/L3 traffic
Generating Layer 2 or Layer 3 traffic is a base functionality of traffic generators, yet it has many useful applications. It involves generating a number of traffic streams required by a test scenario with configurable parameters such as:
- source and destination addresses,
- throughput,
- traffic characteristics - constant/running for a set period of time,
- traffic burst parameters,
- frame/packet size,
- QoS parameters (Ethernet 802.1p/MPLS EXP/IP DSCP markings)
- intentionally introduced errors such as invalid checksums.
When traffic starts, it becomes affected by the characteristics of the devices that are part of our test setup and undergoes typical network events such as delays, losses, and jitter. A traffic generator gathers this data and stores it for later analysis.
When failures occur, options exist to drill down to statistics from a single traffic sub-flow, allowing one to pinpoint what exactly is failing.
Similarly, depending on the configuration of the tested solution, traffic stream parameters may change. Some examples may be IP addressing (if NAT is in use) or QoS parameter values (as a result of remarking). The precision of these changes can also be monitored using traffic generators, either through statistics or packet captures.
Emulation of network protocols
Traffic generators' capabilities are not limited to simple packet blasting. An additional functionality of this class of devices is the ability to emulate several network protocols. This allows a user to simulate the behavior of a neighboring network node's control plane. Some examples of possible emulations are:
- routing and switching protocols,
- MPLS,
- access protocols, and
- multicast protocols.
In conjunction with the capability to generate traffic described previously, this allows traffic sending based on dynamic network information. A particular case of emulation of network protocols is the capability to simulate a topology consisting of nodes. We can fulfill the above requirements using products like IXIA IxNetwork or Spirent Test Center.
Generation of application traffic (L4 - L7)
In some cases, generating simple L2/L3 traffic may be insufficient. Tests related to functionalities based on recognizing higher layer applications (application-aware policies/routing, security, and others) may require streams that effectively emulate, for example, Gmail, BitTorrent, Facebook, and Office365.
These functionalities typically are part of security or SD-WAN solutions that allow separate treatment of selected types of traffic based on a company's requirements.
Products such as IXIA IxLoad or Spirent Cyberflood come to the rescue again. They allow the generation of a given mix of applications and analysis of statistics regarding the establishment of sessions within these applications.
L2/L3 traffic generation is less demanding on hardware than L4/L7, which means that we might achieve lower rates for the latter on a given hardware platform. Testers must take this into account when defining the test procedure.
Worth mentioning
Apart from the commercial products mentioned in the previous section, testers can use free products for the same cases. One that is worth special consideration might be TRex.
Integrating Spirent and IXIA with automated testing solutions
Spirent and IXIA products are great tools for manual testing, but using them in automated testing solutions is also recommended. Various APIs are available, such as REST API, Python, Perl, TCL, and others (vendor and product-specific), making it possible to integrate them with a given automation code. Some solutions even have an option to save a manually configured setup to a script file (Python being one option for script language) which helps to reduce automation effort.
Traffic generator use cases
Since we've already covered a description of traffic generators' capabilities, let's focus on some typical types of tests that we can perform using them.
Functional tests
The role of functional tests is to check the correctness of the behavior of an isolated functionality of the device or service being tested. It is a basic group of network tests intended to confirm that the required functionalities of the device work as expected. Functional tests can cover all aspects of a device's capabilities.
Taking QoS (quality of service) as an example, we can verify:
- classification based on different criteria,
- queueing,
- scheduling,
- shaping/policing,
- marking and remarking.
The generator is configured to send traffic destined to hit the mentioned functionalities. On the receiving side, it is then used to verify if each functionality is working as expected, which is further confirmed with verification commands run on the tested device itself. A subset of those tests are conformance tests, which check if the verified functionalities comply with the relevant standards.
Traffic generator usage in the case of functional tests may vary, but in general, the low scale of either traffic streams or network protocol emulations is in use.
Performance tests
The role of this group of tests is to examine the peak capabilities of the device under test (DUT). We should perform them in a standardized way that allows a reliable comparison of the capabilities of devices from different manufacturers. A popular example of tests belonging to this group in the world of routers and switches testing is RFC2544 throughput tests.
As part of the RFC2544 throughput benchmark, generators send test traffic (IP/UDP) through the DUT in iterations lasting a specified period, and the measured value is the maximum throughput, which ensures lossless transmission of traffic through the device. The test frame size changes during subsequent iterations (typically 64, 128, 256, 512, 1024, 1280, 1518 bytes). For more information about RFC2544 tests, check out another blog post from Codilime >> Network devices benchmarking methodology — RFC 2544 performance testing
The test that, in terms of traffic profile, better corresponds to real Internet conditions compared to RFC2544 is IMIX. In this test, traffic consists of a mix of packets of various sizes in specific proportions. The simple IMIX variety (many varieties of IMIX exist) is 40B/576B/1500B in a 7:4:1 ratio.
Planning the traffic transmission pattern through the device is important when preparing a test. It should consider its structure: the number and modularity of line cards and the internal structure (e.g. forwarding of chipsets and port layout ports within the hardware).
Performance tests require sending very large amounts of traffic through a device, which may involve many ports on the router and generator sides. The illustration below shows how the snake setup can optimally use the latter's resources.
The L2 VLAN based snake setup is presented here but an L3 VRF-based scenario can also be used.
Convergence tests
Convergence testing examines the time required to switch traffic running through a network device if the primary link or transit device fails. The illustration below shows a network diagram of a typical triangle setup used in this group of tests.
The test scenario assumes that emulations of network protocols on the generator are running on ports connected to the DUT and Helper 1 devices. Traffic generators inject network information on the required, usually large, scale and generate test traffic. Under nominal conditions, the DUT sends network traffic directly toward Helper 1 devices. In case of failure of the primary link, the scenario assumes switching to the DUT and Helper 2 link. Detecting a failure and making a switchover decision are the responsibilities of the DUT. Other devices play an auxiliary role here.
The measured values are mainly the convergence time - complete switching of the test traffic to the backup link. Other network device parameters, such as CPU utilization and memory consumption, can also be measured.
The network topology presented above is an option but may be adapted based on the test scenario.
Stress tests
The role of stress tests is to verify DUT behavior under repeated "stress" caused by rapid network events. Such events can be sudden protocol session disablements, link shutdowns, rapid prefix withdrawals, and others. Events like this put the stability of the device's control plane to the test. In ideal conditions, the device should handle such events without impacting traffic, continuing to run using unaffected sessions and ports. A temporary spike in CPU usage and memory consumption is expected but it should return to starting values after a while.
Potential problems to look for are DUT process restarts, memory leaks, increased CPU utilization, protocol instability, and traffic drops. This group of tests puts the most emphasis on the control plane.
System tests
System tests (end-to-end tests) aim to test a complete network solution or service, often consisting of multiple nodes integrated with external elements such as AAA systems, logging servers, and NTP servers. The goal here is to test elements of the solution but in a context that closely reflects the production network. System tests emphasize customer experience. Generators help to reflect service traffic run by customers or administrators.
Scalability tests
The goal of scalability tests is to check if a network element can support a given scale of control plane load that reflects what it is supposed to handle in the production network. Therefore, it is critical to state the requirements regarding the scale of each protocol that a device should handle. The current and planned architecture and buffer for the potential growth of the network must be considered when planning. Of course, a complete set of features has to run in parallel for the test to fully reflect the network environment. Traffic generators inject the required scale of control-plane information.
Network equipment vendors that want to present the scalability of their products might take a different approach. Their test scenario would focus on presenting a given parameter's highest possible scale by minimizing additional features running on the box.
Summary
Network device testing is a broad, time-consuming, and complicated topic. Thanks to their built-in features, traffic generators can make the life of a network engineer much easier. They are a great tool that provides scale, capabilities, and repeatability, which is critical since tests rely on them as a reference.
In this blog post, we've presented the capabilities of professional traffic generators along with popular use cases that readers can implement in their testing environments, preferably in an automated manner.
