Under attack right now? Skip the queue: a responder reads incident reports first.Open an incident

Published
Under
AI Security
Reading time
10 min

AI Threat Detection vs Signature-Based: What the Data Actually Shows

Summary

We analysed 18 months of incident data across 400+ customers. The results challenge some widely held assumptions about AI-native security tooling.

Cover art

The AI security market is full of noise. Vendors claim impossible detection rates and zero false positives. Here's what the real data looks like.

The Dataset

18 months, 400+ BreachModal customers, 2.3 million alert events, 4,800 confirmed incidents. We're publishing these numbers because the industry needs more transparency.

What AI Does Better

Behavioural anomaly detection is where AI genuinely outperforms signatures. A credential stuffing attack that rotates IPs and user agents will evade every signature rule. A model trained on your normal login patterns notices the anomaly immediately.

Lateral movement detection is the other clear win. Attackers moving east-west inside your network are invisible to perimeter tools. Process behaviour baselines catch them.

Where Signatures Still Win

Known malware families, CVE exploitation patterns, and compliance-specific detections. Signatures are fast, deterministic, and explainable to auditors. Don't throw them out.

The False Positive Problem

Our data shows AI models produce 3.2× more false positives than signature rules in the first 30 days of deployment. That ratio inverts to 0.4× by day 90, once the model has tuned to your environment. Most AI security products are evaluated in the first 30 days and look terrible as a result.

Practical Recommendation

Run both in parallel. Use signatures for known-bad, AI for unknown behaviour. Route AI alerts through automated triage before human review. The combination closes gaps neither approach can cover alone.

Written by BreachModal analysts · 7 October 2024

Spotted an error in this advisory? Tell us through the disclosure policy and we will correct the record.

After the advisory

Is this in your estate?

Thirty minutes with our team. We will check your exposure to this and the advisories around it, and show you what containment would look like, and who would approve it.