Agile method to take preventive actions using Machine Learning
Over the past few years, several organizations started to store their operational data, believing that such data would be useful as decision support. One consequence of this was the creation of a large and complex amount of data that becomes increasingly difficult to maintain, analyze and be understood. In this scenario the decision support systems, known as BI - Business Intelligence has arised.
BI systems are robust solutions but expensive, too complex and difficult to maintain. In these systems, it is common to deal with several tools and technologies, such as data warehouse, data marts, feature extractors, data mining, ETL and analytical tools (OLAP). All this complexity makes the implementation of a decision support system a real disorder, mobilizing many people for a long period of time to produce results that justify their investment. The main motivation for implementing such a system is the ability to discover patterns and behaviors to support business decisions. However, in most cases these patterns are discovered late, ie, after a problem occurred or an opportunity has been lost. These systems are limited to the reactive decisions, which will be applied to problems that can no longer be avoided.
The use of machine learning techniques for pattern recognition has been adopted to solve this problem. In general, these techniques are applied in a timely manner in their own transactional systems. This approach maximizes results and speeds up the decision making on the issue. For this, the pattern recognition mechanisms need to be incorporated into transactional systems so that they can identify patterns in a predictive way. For example: evidence of fraud in a operations system, profile identification of users in a eCommerce portal or news and others. Many of the portals we access everyday are using artificial intelligence mechanisms to recognize patterns and make predictions. This is the case of Facebook, which seeks to recognize the profile of its users to recommend news.
In the case of fraud evidence, most organizations do not do this analysis due to the high cost of verifying the integrity of all data. By incorporating a pattern recognition engine, your system can identify fraud evidence at the time the data was being input, generating an alert to the analysts. After checking the alert, the analyst can confirm or deny the transaction, reinforcing the knowledge of the recognizer standards. Over time, with this enhanced knowledge, the recognizer standards will become an expert in identifying fraud.
So, as a decision support system, you can also use indicators to answer business questions, using a pattern recognizer to identify fraud evidence like: "Which product or service are more susceptible to fraud?", "What are the regions or segments where it happen most? ", " Are they more common for a particular profile of customers? Which one?". These business questions can be answered easily, since the system is able to distinguish a fraud from a normal record. The answers to these questions are obtained from simple queries to the database and presented in the form of reports or statistical charts in a dashboard. In addition to answering business questions, reports and dashboards are great tools to measure the effectiveness of actions taken over the business, for example, taking action to minimize fraud, you can track their effectiveness over time, in order to continue or not.
The biggest fear of incorporating pattern recognition routines in a transactional system is its maintenance. Contrary to what it seems, it is not necessary to modify the machine learning algorithms, once they are solid and very stable. Just use a good API, supported by a reliable and diligent community. To respond to new business questions, we can think of new reports and dashboards. The pattern recognizer may evolve through integrations with other systems, or even supportive environments to the decision, if your company already has one.
Recently we tried this approach on some projects and the results were quite satisfactory. By incorporating mechanisms for recognizing patterns in specific features, we proved that it is possible to strengthen business decisions with effective and quick deliveries, which allowed us to prioritize the most critical issues of the business, while minimizing risks and costs.