Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners
A benchmark of 170 Pickle and PyTorch artifacts across 145 specimen families (135 labeled, 10 malformed) evaluated ModelScan, ModelAudit, and Fickling. On labeled families, ModelAudit produced definitive security decisions for 135/135 (100%), Fickling for 110/135 (81.5%), and ModelScan for 67/135 (49.6%). Conditional on making a definitive judgment, ModelScan achieved 100% precision, recall, and F1. Fickling identified no unique true-positive families beyond those found by ModelAudit and ModelScan. For 48 malicious families where ModelScan failed to complete analysis, ModelAudit and Fickling generated detections.
A research paper evaluates three static scanners for AI model security: ModelScan, ModelAudit, and Fickling. Using a controlled benchmark of 170 artifacts across 145 specimen families, the study distinguishes coverage, analysis completion, definitive security decisions, non-security findings, and unsupported outcomes. ModelAudit achieved 100% definitive decision coverage on labeled families, Fickling 81.5%, and ModelScan 49.6%. ModelScan had perfect precision/recall/F1 when it did make a judgment, but failed to complete analysis on 48 malicious families where the other two scanners detected threats. Fickling added no unique true positives beyond ModelAudit and ModelScan.
The benchmark reveals that F1 alone is insufficient for security scanners because it ignores non-coverage and incomplete analyses. ModelScan's perfect F1 masks a 50.4% failure-to-decide rate on labeled families. The distinction between 'definitive security decision' and 'unsupported outcome' is critical; scanners may halt on malformed or complex artifacts. The 10 intentionally malformed unlabeled families test failure recovery, and the paper reports that ModelAudit and Fickling still produced detections on the 48 families where ModelScan failed, suggesting robustness differences in parsing or execution.
Security teams relying solely on F1 or precision/recall may overestimate scanner effectiveness. ModelScan's high precision but low coverage means it misses many malicious artifacts silently. ModelAudit's 100% coverage makes it a stronger default for enterprise ML artifact scanning, but its precision/recall on those decisions is not reported in the evidence. The lack of unique true positives from Fickling suggests redundancy; combining ModelAudit and ModelScan may be sufficient. This has implications for tool selection in MLOps and model registry security.
For organizations deploying ML models, this research provides evidence to choose scanners based on coverage and failure modes, not just accuracy. ModelAudit's complete coverage reduces the risk of undetected malicious models, potentially lowering security incident costs. The benchmark methodology can inform procurement and red-team evaluations. However, the paper does not provide precision/recall for ModelAudit's decisions, so a full cost-benefit analysis requires additional data.
Expect follow-up work to standardize coverage and failure-recovery metrics for AI security scanners. Vendors may add explicit 'unsupported' or 'inconclusive' result codes to avoid silent failures. The benchmark corpus could become a reference for evaluating future scanners. Watch for adoption of ModelAudit-like coverage in commercial tools and for improvements in ModelScan's handling of complex or malformed artifacts.