Python, with its simplicity and versatility, has emerged as a powerful programming language for network and infrastructure automation. It is the best option for engineers and administrators to start programming and automate their work. With many useful libraries, Python allows you to quickly implement an infrastructure management strategy tailored to your situation using Python-based automation tools.
This article is an introduction to Paramiko and Netmiko, which are the most popular Python SSH libraries for automation. It should be valuable to both administrators and developers who want to start their automation journey and become aware of the differences between server host automation and network device automation using Python.
Agentless automation with SSH for Python network automation
In agentless automation, automation tasks are performed on remote systems without software agents installed on those systems. Unlike agent-based automation implemented with tools such as Puppet or Chef, agentless automation relies on remote management protocols such as SSH or WinRM to execute commands remotely. Agentless automation is easier to implement because it reuses existing management protocols. It also has lower administrative costs since one or at most several central automation agents need to be deployed and managed when automating infrastructure and networks.
The most popular tool used to establish agentless automation is SSH (Secure Shell). SSH is a protocol for secure system administration, file transfers, and other communication across the Internet or other untrusted networks. It encrypts identities, passwords, and transmitted data to prevent eavesdropping and theft. SSH is widely used for Python automation of servers and network devices because it provides security using various authentication and encryption methods and is standardized. Thanks to many existing implementations, including open source, SSH is in widespread use. SSH is used to configure services and retrieve configuration settings and state parameters. It is one of the key components of higher-level automation tools such as Ansible, Napalm, and Nornir in Python-based network automation ecosystems.
Paramiko for Python SSH automation
Paramiko link-icon 
is a handy LGPL-licensed library that implements the SSHv2 protocol for Python-based automation. It is written in pure Python, so it can be easily installed and used to connect to any system that provides an SSH service. For these reasons, it has become the first choice if someone wants to play with agentless automation using Python. However, simplicity of use is not everything. Its functionality and implementation are modeled on OpenSSH, the most commonly used SSH protocol implementation. This origin makes Paramiko versatile and very useful in more complex scenarios beyond basic Python SSH scripts.
That is possible because it provides full implementation of all internal SSH components, i.e., transport layer (RFC4252 link-icon ), user authentication (RFC4253 link-icon ), and connection protocols (RFC4254 link-icon ). Additionally, it implements Secure File Transfer Protocol (SFTP) over SSH channels, which can be used to manage remote files and transfer them in both directions as part of Python infrastructure and network automation.
Paramiko usage example for Python SSH automation
Let's skip the details of Paramiko’s low-level SSH components and focus on the fact that Paramiko provides a high-level API that simplifies the creation of SSH clients, servers or port forwarders in Python automation scripts. We can check the use of Paramiko in a simple scenario. We would like to connect to a Linux host and fetch all network interfaces. The use of Paramiko.SSHClient is shown in Listing 1.
import paramiko
ssh = paramiko.SSHClient() ssh.set_missing_host_key_policy(paramiko.WarningPolicy())
key = paramiko.ECDSAKey.from_private_key_file('host_ecdsa_256.key')
ssh.connect('192.168.0.1', username='user', pkey=key)
stdin, stdout, stderr = ssh.exec_command('ip addr show')
output = stdout.read().decode()
ssh.close()
Listing. 1 Example implementation of SSH client using Paramiko
Explanation
Looking at the code, we can see that once the SSHClient instance is created, a policy is established to warn about unknown host keys. WarningPolicy allows you to seamlessly connect to any new host, while RejectPolicy should be used in production environments when building secure Python SSH automation. Later, a private ECDSA key is loaded to connect to the host machine (Paramiko can also use RSA, DSA, or Ed25519 keys). Finally, a connection to the server is established, and the required Linux command is remotely executed on the host via Python SSH automation.
The command result is read from the standard output and decoded as UTF-8. Unfortunately, this is only a raw string of characters, so additional operations must be performed to parse it; e.g., the IP addresses of the interfaces. Extracting the required information from command results is beyond the scope of the Paramiko library and typical Python SSH automation use cases.
Executing simple commands in the SSH EXEC channel using Python Paramiko
The Paramiko usage example in the previous section uses the SSHClient.exec_command method. This method creates an SSH EXEC channel for Python-based SSH automation. An EXEC channel accepts a single command or a set of commands, but does so only at the time the channel is created. The SSH server executes the commands received from the channel and returns all generated output when execution is complete to the Python automation script.
Linux SSH servers execute commands from the EXEC channel mainly in non-interactive and non-login shells. It is important to know what environment a command is being executed in, as it can have a significant impact on the behavior of commands, especially those that are parameterized by environment variables. This may happen because some startup scripts (e.g., ~/.bash_profile) may not execute, and environment variables may be set differently. By default, the EXEC channel doesn’t use terminal emulation to avoid commands to do fancy things like adjusting output to terminal width and height, pagination, and coloring.
In summary, the EXEC channel is ideal for automating simple commands with Python SSH libraries for which you can collect output in non-interactive automation scenarios.
The SSH SHELL channel for interactive applications in Python SSH automation
When an interactive application needs to be automated, a different type of SSH channel is required. The recommended channel type is SSH SHELL when automating interactive workflows with Python. Thanks to the SHELL channel, additional input data can be provided to the application, such as additional "yes/no" confirmations, option selections, or set values.
This channel continuously puts command results into the standard output stream. Linux SSH servers execute commands from the SHELL channel mainly in interactive and login shells. This is the same environment where you log in manually using an SSH terminal client when administering servers or network devices.
In Paramiko, the output of the SHELL channel is exposed in the form of stdin and stderr streams, which require more complex handling, similar to the use of network sockets in Python SSH automation code. All command output becomes unstructured in the output stream, and the automation code must recognize different parts of the results. The time aspects of communication, such as the waiting time for specific information, should also be properly determined when automating interactive SSH sessions with Python.
Why isn’t Paramiko enough for network automation?
Most Linux and BSD systems use OpenSSH as their implementation of the SSH protocol. This implementation runs as a daemon called sshd. Compared to servers, network devices have their own SSH server implementation quirks, which cause important differences when establishing a connection during Python-based network automation. An example of such a dissimilarity is how the user is authenticated. In the case of hosts, mechanisms built into the SSH protocol are always used for this purpose. However, in the case of network devices, there are cases where user authentication takes place in a separate application launched immediately after establishing an SSH session when automating network devices with Python.
The second significant difference between network devices and hosts is the use of specialized CLI shells that allow easy navigation through a large number of available commands. Network equipment manufacturers treat shells as their intellectual property and trademark, and carefully cultivate differences in the behavior of their shells. This causes shells from different manufacturers to behave differently and have different command hierarchies. Therefore, implementing automation for network devices using Paramiko itself is much more complicated than it might initially seem in real-world Python network automation scenarios.
A few years ago, the user support for Paramiko (aka GitHub issues) was full of problems with people trying to connect to various network devices or execute specific commands. Currently, such unfortunate people are quickly redirected to another library: Netmiko, a Python library designed specifically for network automation.
Check out our other related content:
- Python Nornir for simplifying network automation: code examples
- Network automation tools comparison in code examples: Terraform, Ansible, and Python SDK
Python Netmiko library to rescue network automation
Netmiko link-icon is an open-source, Python MIT-licensed library designed to interact with a wide range of network devices (Cisco, Juniper, Arista, Huawei, and many others link-icon ) for Python-based network automation. It hides low-level differences between different vendors and provides a high-level API that exposes common behaviors of many web CLI shells (e.g., the existence of operational/configuration modes) in a consistent way for automating network devices with Python.
Netmiko uses several communication technologies that enable the automation of network devices. The most important of these is SSH, which is supported on Netmiko using Paramiko as the underlying Python SSH library. SCP is available through the scp library link-icon (it is also implemented on top of Paramiko). Netmiko also supports telnet (using telnetlib link-icon ) and serial (using pyserial link-icon ). Additionally, SNMP (pysnmp link-icon ) is used, but only for automatic device type detection, which helps select the appropriate device driver in Python network automation workflows.
Netmiko usage example for Python network device automation
Let's demonstrate using Netmiko to connect to a Cisco IOS router to obtain the IP addresses of all interfaces using Python network automation. Please refer to Listing. 2 below.
import netmiko
cisco = {
'device_type': 'cisco_ios',
'host': '192.168.0.10',
'username': 'user',
'use_keys': True,
'key_file': 'cisco_rsa.key',
'disabled_algorithms': {'pubkeys':
['rsa-sha2-512', 'rsa-sha2-256']}
}
with netmiko.ConnectHandler(**cisco) as ssh:
interfaces = ssh.send_command('show ip interface brief',
use_textfsm=True)
ip_addresses = (interface['ipaddr'] for interface in interfaces)
Listing. 2 Example implementation of SSH client using Paramiko
Explanation
This example uses the netmiko.ConnectHandler factory function. ConnectHandler requires device type information and other access arguments. Using these arguments, ConnectHandler creates a dedicated SSH client for the Cisco IOS router in a Python automation script. The appropriate Cisco command is then sent.
The use_textfsm argument is set so that send_command returns fully parsed interface information using NTC templates link-icon (this is a large collection of TestFSM link-icon templates for parsing network device text output). Since the returned output of the command is a list of Python dictionaries describing the attributes of each interface, the user of the Netmiko library can easily filter the required information (i.e., interface IP addresses) in Python network automation workflows.
If the type of network device is not known, the Netmiko library user should first try using the netmiko.ssh_autodetect module. It will most likely provide the correct device type when automating heterogeneous network environments with Python.
What problems does Netmiko solve in Python network automation?
A network automation engineer’s work is much easier using Netmiko. The list of the most important Netmiko functionalities is presented in Fig. 5.
The library includes a long list of dedicated SSH clients for many types of network devices. Each Netmiko client implements the necessary quirks enabling successful login to a specific network platform (i.e., non-standard user authentication methods, additional user confirmations immediately after the login, skipping login banners). After logging in, the library user can easily change CLI mode to privileged and configuration mode. Additionally, each dedicated SSH client includes the expected prompts for a specific mode in Python automation workflows.
Netmiko also provides a number of features required to successfully execute commands. First, device-specific window size settings are applied to the created SSH channels, and pagination is disabled, so that structured and multiline commands produce clear results in Python network automation scripts. Secondly, Netmiko supports parsing command output using TestFSM link-icon , Genie link-icon and TTP link-icon libraries.
Some network devices copy commands to output, but Netmiko can automatically remove them from the generated output. The library also provides useful commands link-icon and their options to handle commands that generate additional questions instead of the expected prompts when automating interactive network device CLIs.
Many network devices experience problems with slowdown periods when responding to commands. For this reason, Netmiko implements a hierarchy of delay factors link-icon as well as different command timing strategies. Finally, it provides a convenient method for saving/committing a new configuration.
What if performance really matters in Python SSH and network automation?
Both Paramiko and Netmiko are implemented in pure Python, so there are situations where their low performance is problematic for large-scale Python network automation. This may be important when building automation systems for hundreds or thousands of nodes. In this case, there are alternative open-source Python SSH libraries that could be a better fit for high-performance automation scenarios:
- parallel-ssh link-icon - an asynchronous SSH client library under the LGPL license. It is based on ssh2-python link-icon (Cython wrapper around C libssh2 link-icon ) and gevent link-icon .
- asyncssh link-icon - another asynchronous SSH client that is published under the EPL license. It is mostly written in Python using asyncio link-icon .
Both libraries excel when used directly to process hundreds of calls in parallel, as documented here link-icon . However, you should be prepared for installation issues and problems in successful connections to many network devices when automating networks at scale with Python.
Here, you can watch a video that provides practical insights and innovative approaches to tackling complex network firewall upgrades by presenting a real-world case study involving the automation of firewall upgrades:
Summary
This article introduced you to the two most popular Python libraries for the SSH protocol used in Python network automation. The first is Paramiko, a solid solution that will help you automate your work on servers using Python SSH automation. With Paramiko, you can automate both simple applications or scripts as well as programs with more interactive behavior over SSH.
The second library is Netmiko, which is indispensable when you want to remotely manage network devices using Python network automation tools. It supports many important behaviors of various network equipment manufacturers and provides parsed command results for automating network devices with Python.
However, when performance is a deciding factor, it's worth trying one of the asynchronous SSH implementations, such as parallel-ssh or asyncssh, for high-scale Python SSH automation.
