10k+ firewalls upgraded a year by a Professional Services automation team

The challenge

Large customers run 100+ NGFWs each, and migrations from another vendor must fit maintenance windows with a clean rollback. Every customer deployment is different, and source and target platforms never match feature for feature, so a conversion has to carry over the business logic of the original policies. The existing tool for migrating configurations did not meet the scale and reliability large projects needed, and bespoke scripting could not keep up with the timelines.

What was at risk:

• Migrations and upgrades that miss their maintenance windows, and delivery held back by bespoke scripting that could not keep up with the scale

What we did

• Acted as a global delivery center for automation: 50+ engineers on three continents, specialized in deploying and integrating network security
• Built a production-grade config conversion factory that converts NGFW configurations between vendors, run by SME engineers for large customers
• Delivered Infrastructure-as-Code with Terraform for fast, repeatable software firewall deployment in public cloud
• Built a Python library and Ansible Collection for automated OS upgrades, with 20+ pre- and post-validation checks
• Automated configuration cleanup, removing unused objects, deduplicating and reducing config size
• Moved configuration management from ClickOps to GitOps, with a single source of truth in git and integration with ITSM and IPAM systems
• Exposed the automation to the whole Professional Services organization and its partners, through open-source Terraform modules and a provider for the vendor’s OS, and a PS SaaS platform for automation workflows

How it works

The automation follows a customer’s firewalls through the whole service lifecycle: build, maintain and run.

1. Build: config conversion A modular conversion tool, built from scratch for Professional Services engineers, parses a source vendor’s configuration, translates it and outputs the target format. One tool covers several source vendors, is extensible with purpose-built scripts and is consumable through an API. Conversions account for dependencies so the converted policy does what the original did, and SME engineers run it for large customers, 1000+ NGFWs a year.
2. Build: cloud deployment Open-source Terraform code, used by the community, customers and the whole PS organization with 300k+ downloads, deploys software firewalls in public cloud smoothly and repeatably, for 100+ customers a year.
3. Maintain: cleanup Cleanup automation removes unused objects, deduplicates and shrinks configuration size, for 30+ customers a year.
4. Maintain: upgrade A Python library and Ansible Collection automate OS upgrades, with 20+ pre- and post-validation checks around each one. 100+ NGFWs are upgraded in a single maintenance window, and 10k+ a year.
5. Run: configuration as code Configuration lives in git as the single source of truth, integrated with ITSM and IPAM systems, and changes deploy as code in minutes or hours. The approach runs for airlines, MSPs, banks and industry.