MAGIKA: AI-Powered Content-Type Detection

The task of content-type detection-which entails identifying the data encoded in an arbitrary byte sequence-is critical for operating systems, development, reverse engineering environments, and a variety of security applications. In this paper, we introduce Magika, a novel AI-powered content-type de...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings / International Conference on Software Engineering s. 2638 - 2649
Hlavní autoři: Fratantonio, Yanick, Invernizzi, Luca, Farah, Loua, Thomas, Kurt, Zhang, Marina, Albertini, Ange, Galilee, Francois, Metitieri, Giancarlo, Cretin, Julien, Petit-Bianco, Alex, Tao, David, Bursztein, Elie
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 26.04.2025
Témata:
Deep learning
Internet
Licenses
Operating systems
Pipelines
Reverse engineering
Security
Software development management
Software engineering
Training
ISSN:1558-1225
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The task of content-type detection-which entails identifying the data encoded in an arbitrary byte sequence-is critical for operating systems, development, reverse engineering environments, and a variety of security applications. In this paper, we introduce Magika, a novel AI-powered content-type detection tool. Under the hood, Magika employs a deep learning model that can execute on a single CPU with just 1MB of memory to store the model's weights. We show that Magika achieves an average F1 score of 99% across over a hundred content types and a test set of more than 1M files, outperforming all existing content-type detection tools today. To foster adoption and improvements, we open source Magika under an Apache 2 license on GitHub and we make our model and training pipeline publicly available. Our tool has already seen adoption by Gmail and Google Drive for attachment scanning, by VirusTotal to aid with malware analysis, and by prominent open-source projects such as Apache Tika. While this paper focuses on the initial version, Magika continues to evolve with support for over 200 content types now available. The latest developments can be found at https://github.com/google/magika.
ISSN:1558-1225
DOI:10.1109/ICSE55347.2025.00158