It's difficult to envision the role of future network engineers without the inclusion of automation. As network environments expand and become increasingly complex, it's essential that we adapt by leveraging the tools at our disposal.
In this article, we will explore the advantages of integrating network infrastructure testing with automation.
Sandbox testing: safeguarding networks with automation
Could you imagine applying every configuration change right onto the production network, without any checks and tests? It's a guaranteed route to disaster.
Luckily, today we have many ways to test our desired configuration before it hits the production network. With the rise in popularity of NFV, we can set up a virtual network environment. Such a setup can be used as a sandbox, for testing changes that could potentially negatively impact the network. With the help of automation, such environments can be automatically scaled, and configured, so the network engineer can focus on testing the impact of configuration changes. After the reconfiguration, such environments are ready to be automatically tested to see if there are any potential problems.
Automating network recovery: using snapshot comparisons post-outage
Power outages are a common occurrence in network environments. Ideally, we would be forewarned about them, but unfortunately, unexpected outages still occur. But how do we know if our network operates correctly after the outage? We can test it manually of course, but can we automate that test? Sure!
Such tests can be based on various assumptions, for example:
- connectivity based - In such tests, we're testing reachability between two network components. If there is no connection to a specific element, or it’s disrupted, we can assume that there are problems in our network, which can be then checked by the network engineers.
- snapshot based - For the purpose of this article, let’s assume that a snapshot is a device state in specified time. Various data can be gathered for the snapshot purpose, e.g. routing table, interface states, configuration, etc. If we have a snapshot of the network before the power outage, we can compare it to the one gathered after the break. The differences between the two snapshots can be analyzed to catch any issues.
The period following a power outage can be particularly challenging for network engineers. It's common that some components are not working as expected after the breakdown, so detection and troubleshooting become essential, especially in business critical environments. With automated tests, engineers have less work, since scripts are responsible for problem detection, and engineers can focus entirely on the service bringup. Additionally, automated tests are generally much quicker than manual checks performed by engineers. Power outages are used here as a descriptive example of network environment disruption, but automated tests can be used in many other situations. Even after a minor configuration change.
Device-specific testing: enhancing network equipment before deployment
Is it necessary to run automated tests on production networks? Absolutely not! There are scenarios where we want to test the behavior of network equipment that's not released to the clients yet, or it’s already released, but new enhancements are added.
Automated tests can be used in various phases of software development:
- Feature tests - these tests allow us to make sure that a device’s features, for instance BGP with route maps configured on a device with a specific software version and some custom configuration, work as expected.
- Upgrade tests - upgrading of a device might cause issues with its functionalities. Testing upgrades helps in detecting such anomalies and allows us to eliminate them before a company releases the upgrade to end users.
- Comprehensive product testing - this can be done before and/or after the release to the client to ensure quality.
- Maintenance - to ensure that an application/device is still functioning as expected after updates, code changes, or the addition of new features.
Once written, these tests are generally quick and easy to execute and they can detect abnormalities early on. This allows for timely resolution, resulting in a more robust and reliable product that satisfies both users and providers.
Selecting the best test cases for automation: a strategic approach
One of the most basic mistakes testers make is choosing the wrong test cases for automation. It is not enough to choose any set of tests; in fact it is extremely important to carefully analyze all cases and select the best candidates for automation. It all depends on the context, i.e. the application or software being tested, the team and its knowledge, the technical and financial resources, etc.
There is a set of categories that can be used as guidelines in the selection of subsequent areas for automation:
- Tests that require a lot of effort and consume a lot of time when performed manually.
- Repetitive tests that are carried out on different versions/builds of the application or run on various hardware.
- Tests that tend to cause numerous mistakes during their execution.
- Tests that require many extensive databases and those that involve entering large amounts of data.
So, first define your automation goal and identify the types of tests to automate. The more repeatable the test process flow, the better for automation. Examples of such tests include:
-
Performance tests, of which we have many examples, among them:
- Load testing - verify how the application will behave under the expected load and how it translates into the response time of such an application.
- Stress testing - checking the maximum load that the application can handle.
-
Regression tests - check that the new code hasn't broken existing functions.
-
DDT (data-driven testing) - check the functionality of the application in the context of a large amount of different input data.
The benefits of automation are closely related to how many times a given test can be repeated. Tests that are carried out only a few times are better for manual testing. The best candidates, however, are those tests that are run frequently and require a large amount of data to perform the action itself.
Key stakeholders benefiting from network automation
Automated tests are beneficial for various stakeholders involved in network development and maintenance:
- Development teams: Automation helps development teams validate their code changes quickly and consistently. It allows them to catch issues early in the development process, leading to faster bug detection and resolution. Automated tests also enable developers to refactor code confidently, knowing that the tests will identify any regressions.
- Quality assurance (QA) teams: Automated tests empower QA teams to efficiently perform extensive testing across different network configurations and scenarios. They can focus on designing and maintaining test cases rather than spending excessive time on repetitive manual testing. This frees up their resources to perform more exploratory testing and focus on higher-level test scenarios.
- Operations teams: Automated tests contribute to the stability and reliability of networks, benefiting operations teams responsible for maintaining network infrastructure. By providing continuous feedback on network performance and identifying potential issues, automated tests help operations teams take proactive measures to maintain system stability and respond promptly to incidents.
- Product managers: Automated tests provide valuable insights into the performance and reliability of network features and functionalities. Product managers can leverage this data to make informed decisions about feature prioritization, resource allocation, and overall product roadmap. It enables them to balance development efforts effectively and deliver a high-quality product.
- Management and stakeholders: Automated tests offer visibility into the state of the network, providing confidence to management and stakeholders. They can rely on the test results to make informed decisions about project timelines, resource allocation, and risk mitigation. Automated tests serve as a confirmation of the network's stability and readiness for production environments.
Automated tests benefit development teams, QA teams, operations teams, product managers, management, and stakeholders. They enhance efficiency, reliability, and decision-making capabilities across the network development life cycle. If you're interested in how network automation can further enhance your network management, check out our network automation offering.
Summary
The networking sector is continuously evolving along with its methodologies and tools. This progression enables us to automate tests more efficiently in terms of both time and cost. Given the prevailing market trends, test automation is poised to remain a key strategy, allowing us to improve our operational workflows.
