real-world

Real-World Vs Synthetic Eval Gap In Security

Synthetic eval benchmarks are controllable. Real-world data is messy. The gap between performance on each is usually large, and vendors prefer one over the other for a reason.