Regex from a Local LLM: Bitrix24's Experience Without Fine-Tuning

As data volumes grow exponentially, efficient log analysis is becoming critical for keeping IT infrastructure stable and secure. Bitrix24 has found an innovative way to tackle the problem: it uses a local language model (LLM) to generate regular expressions (Regex) automatically. Instead of the traditional approach, which requires costly, labor-intensive fine-tuning of a neural network on proprietary data, Bitrix24 built a system in which the LLM generates the Regex and a script saves and applies those rules autonomously. The approach not only saves resources but also protects data, since all computation happens inside the company's perimeter.

Traditionally, writing Regex for log parsing is a routine, laborious task that demands deep knowledge of regular expression syntax and an understanding of log structure. It can take hundreds of hours of manual debugging, especially with many different log files. Cloud APIs for generating Regex can simplify the job, but they carry the risk of passing confidential data to third-party services. Cloud solutions can also prove economically unviable at large data volumes.

The architecture Bitrix24 built centers on a locally deployed LLM running on a Mac Mini. The model receives a description of the log structure and the parsing task, then generates a matching regular expression. A script written by Bitrix24 specialists automatically saves the generated Regex and uses it to parse logs. The key point is that the LLM is used out of the box, with no additional fine-tuning on Bitrix24-specific data. Instead, the emphasis is on optimizing the script that manages Regex generation and application.

The advantages of this approach are clear. First, it saves resources: no need to fine-tune the neural network significantly cuts the cost of compute and dataset maintenance. Second, it improves security: all computation happens inside the company, which rules out the risk of confidential data leaking. Third, it is flexible and scalable: the system adapts easily to new log types and can scale to handle larger volumes of data.

Deploying the system let Bitrix24 significantly reduce the time spent on log analysis and make its IT specialists more efficient. Automatic Regex generation frees up resources for harder tasks such as anomaly analysis and detecting security threats. The case shows that local LLMs can be an effective tool for practical tasks that do not require complex fine-tuning.

The approach holds broad promise for other companies that need to analyze large volumes of data. It can be applied in areas such as application performance monitoring, fraud detection and user behavior analysis. It is important to note that its success depends largely on the quality of the script that manages Regex generation and application. Companies planning a similar system should therefore pay particular attention to developing and optimizing that script.

In conclusion, Bitrix24 has demonstrated an innovative way to use local LLMs to automate Regex creation. The approach not only saves resources and improves data security, but also opens up new possibilities for practical tasks in data analysis. Going forward, we can expect this direction to develop further, along with new tools and methods that use local LLMs to automate a range of tasks.