Papers for

large software repositories

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Neural model speeds and improves text file type detection by four times

Fast And Accurate Text Content File Type Identification

Abstract: A common requirement across organizations is to have a tool that can identify file types based on their contents, particularly in the cybersecurity domain where magic numbers and file extensions can not be trusted. While existing tools work well in practice, there is plenty of room for improvement either in terms of computational load and time for detection in the case of model based tools like Magika or in terms of accuracy of detection in the case of file parsing tools that use programming language constructs. In this study, we propose a neural network model for identification of types of text content files, especially source code, that is more accurate and faster than other available tools. Our experiments on open-source files indicate that it is not only more accurate on average for text-content file-type identification, but also approximately four times faster than Magika, while being 28% smaller in size.

Fri 18 SeptMachine Learning
The gist
It can be hard for computers to tell what kind of file they are looking at just by the content inside, especially when the usual clues like file extensions might be wrong. The authors created a new type of computer program that uses neural networks to recognize different kinds of text files, like source code, faster and more accurately than other tools. Their tests with open-source files show this new tool works better and is quicker than a popular existing method named Magika. It’s also smaller, which helps it run more easily on different computers.
Open → 2609.21306v1