Building an automation module for a network observability solution
Automation
Cloud
Monitoring & alerting
Full stack development
Business background
Client type
A provider of a network monitoring and observability platform
Business goal
Automate the onboarding process of virtual private clouds to the client’s network observability platform
Our approach
Provide a team of developers and DevOps engineers with expertise in network monitoring and observability, cloud and automation
Design and build an entire solution from the ground up: architecture, technology stack, execution, testing
Enable configuration of customer's observability platform using Infrastructure-as-Code (IaC) approach
Ensure close cooperation with the client and a fast feedback loop
Organize regular, recorded demos of new features
Ensure the client’s feedback is promptly implemented
Business benefits
Reduced number of errors: the error-prone manual process of integrating a virtual private cloud with the client’s platform was fully automated.
Saved time and labor: integrating new cloud deployment with the client’s network observability platform is faster and simpler.
Reproducibility ensured: the IaC approach allowed for an easily reproducible and trackable configuration.
Ensured adherence to industry standards: the integration was implemented utilizing widely adopted technologies.
Technical highlights
Prepared a configuration of AWS Cloud to be consumed by the client’s platform:
IAM roles and policies that provide access to the client’s platform to EC2 and S3 resources
S3 bucket to store flow logs for the client’s platform
Flow Log configuration
Prepared configuration of GCP Cloud to be consumed by the client’s platform:
Enabled flow logs for required subnets
Flow log sink to filter logs from required subnets to PUB/SUB topic
Created PUB/SUB topic
Created PUB/SUB topic subscription
Assigned roles to allow the client’s platform to pull logs from PUB/SUB
Prepared configuration of Azure Cloud to be consumed by the client’s platform:
Turned on appropriate providers to enable flow logs and network watchers
Network watcher and flow logs to get logs from resource group
Granted Azure roles access to the client's platform
Configured storage account
Prepared Terraform/Ansible modules to automatically integrate new virtual private clouds with the monitoring and observability platform.
Implemented Terraform provider (in Golang) allowing to automate configuration of flow log export from cloud deployment to the observability platform. The Terraform provider utilizes observability platform's API to implement changes.
Ensured that Terraform modules support automated export of AWS and GCP clouds to the client’s platform.