← Back to use cases
AIOps & GenAI
Malicious IPs detected from 2+ TB of telco netflow
Who the client is
A major Asian telecommunications provider. Its network produces netflow, the metadata of who talks to whom across the operator’s infrastructure, at a scale of terabytes. The operator wanted to prove that this metadata can reveal malicious IP addresses before they appear in threat intelligence databases, and to bring that evidence into government policy discussions on metadata access for national security.
The challenge
Threat intelligence databases list a malicious IP only after it has been reported. The client wanted to find those addresses earlier, from the netflow it already collects. That meant processing more than 2 TB of raw network metadata efficiently, turning it into features a model can learn from, and building ground truth from external threat intelligence, so every prediction could be checked against real-world threats.
What was at risk:
- Malicious IPs seen only once public threat databases list them, and no measured evidence for the policy discussion on metadata access
What we did
- A high-performance, multithreaded C++ tool for netflow processing and feature extraction, sized for a dataset larger than 2 TB
- An ML classification model trained on netflow features: source and target IP, ports and packet size
- Ground truth built from external threat intelligence (VirusTotal), so the model learns from and is checked against real-world threats
- Explainability analysis that shows which features drive each prediction
- Results enriched with IP geolocation data
- An MLOps pipeline with bi-weekly retraining, so the model keeps up with new data
How it works
The pipeline takes raw netflow to a list of suspect IP addresses that is checked against threat intelligence, in five stages.
- Netflow processing A multithreaded C++ tool reads more than 2 TB of netflow and extracts the features the model needs from the raw network metadata.
- Ground truth Labels come from external threat intelligence: VirusTotal tells the pipeline which addresses are known to be malicious, so the model learns from real-world threats.
- Classification An ML classification model scores IP addresses from netflow features: source and target IP, ports and packet size. Explainability analysis shows which of those features drive each prediction.
- Enrichment and validation Flagged addresses are enriched with IP geolocation data and checked against VirusTotal. Within one week, 60% of the IPs the model flagged were confirmed malicious.
- Retraining An MLOps pipeline retrains the model every two weeks on fresh data.
The PoC took about four months, most of it spent on model validation.
Outcomes
60% (confirmed malicious)
Of the IPs our model flagged in 2+ TB of telco netflow, 60% were confirmed malicious in VirusTotal within a week.
Made possible
Ahead of threat databases
threats detected from network metadata before they appear in public threat databases
Business
Evidence for policy
the results gave the client evidence for government policy discussions on metadata access for national security
Business
~4 months
PoC completed in about four months, most of it spent on model validation
Tech & stack
- C++ (multithreaded)
- Netflow
- ML classification
- Explainability analysis
- MLOps (bi-weekly retraining)
- IP geolocation
Anonymized example — fuller write-up available on request.
