Mitsubishi Electric is developing AI for behavioral analysis

Posted on April 2, 2024

Mitsubishi Electric Corporation announced that it has developed artificial intelligence (AI) for behavioral analysis capable of analyzing in a few minutes the level of effectiveness of manual operations carried out in production areas, without the need for operators to prepare AI training data. This is achieved through the adoption of a special model of probability generation. Thanks to this further innovation brought to the company’s Maisart line in the field of artificial intelligence, it is possible to quickly analyze videos of people performing repetitive activities. Below we show the ways in which similar activities can be done more efficiently, to achieve greater productivity.

This is believed to be the world’s first application of a probabilistic production model capable of simulating the cyclical (repetitive) physical actions performed during factory work. The tests carried out have shown how this technology, announced for the first time by Mitsubishi Electric on 13 February 2019, can reduce the time normally required for the analysis of the work done by up to 99%.

The business is expected in the financial year ending March 2026, or later.

The technology was presented, along with practical demonstrations, at Iifes 2024 (Innovative Industry Fair for E x E Solutions 2024), which was held at Tokyo Big Sight.

Features and functionality

1) The world’s first application of a production probability model to the analysis of repetitive work in production sites

For the first time in the world, Mitsubishi Electric has used a probabilistic production model capable of simulating the process of generating signal data about the various body movements performed repeatedly during specific activities. Using a video showing how a certain task is performed, the skeletal structure of the operator is detected and their physical movements are recorded as waveform data. Artificial Intelligence (AI) for behavioral analysis analyzes data using a probabilistic model based on repetitive body movements. AI recognizes and analyzes the execution of repetitive tasks, such as moving an object or tightening a screw, based only on the estimated time it takes to perform a certain type of task once. AI can also identify non-recurring activities that differ from recurring ones in terms of time or waveform.

The results of the analysis obtained can be included in a video about the work in progress. This allows users to verify each step of the job, even assigning specific labels, such as “tightening screws.” Unlike the AI ​​currently used for job analysis, the new technology eliminates the need to create data for AI training, reducing the overall time required to analyze a job by up to 99%. Furthermore, the greatly reduced computational complexity of the technology in question eliminates the need for high-performance computers, such as graphics processing units (GPUs). Compared to manual analysis, the inspection accuracy rate is equal to or greater than 80% for work performed by unqualified operators, while it is equal to or greater than 90% for work performed by professional personnel.

Job analysis results obtained using artificial intelligence (AI) for behavior analysis

The new technology eliminates the need for training data, meaning analysis is faster even when looking at large numbers of employees. By properly matching experienced workers, AI can easily identify differences when they occur, with the goal of helping new workers acquire advanced skills. All this produces an increase in skill levels in a short period of time. In addition, the new technology can select and provide more representative examples of skilled and unskilled repetitive work through special videos, allowing new hires to easily, instantly, understand the differences, to accelerate the learning of advanced skills.

Comparison between activities performed by new employees and experienced employees

3) Rapid generation of critical data to correct incorrect work methods and thus maintain a high level of production quality

AI currently used to identify errors in work methods requires the preparation of relevant data, training the AI ​​on how to compare the work being done against correct and common ways of working. However, the working methods may differ depending on the specific version of the product designed or, in some cases, due to the uniqueness of the workplace. Therefore, it is often necessary to adjust the training data according to specific conditions. This can significantly increase the time and effort required for data preparation.

Mitsubishi Electric’s new AI creates its own training data using only job analysis results. Even if production processes or different versions of the manufactured product are changed, real-time detection of abnormal processes can be achieved quickly and with little effort. This ultimately helps prevent any quality defects in production.

Creating training data for AI to detect errors

Future development

In the near future, Mitsubishi Electric will conduct further tests of its new AI at domestic and overseas manufacturing sites, including plants operated by Sysmex Corporation and Sumitomo Rubber Industries Co., Ltd., with the goal of launching a commercial product within the year fiscal year ending March 2026, or later.

The real context

The technology and skills used in automated manufacturing have advanced greatly in recent years, but capital investment has not kept pace due to high costs, resulting in many processes still being done manually. Human performance tends to vary depending on the time and quality of the work being done. All this can lead to problems in production processes. In order to reduce the differences in terms of human performance and thus maintain a high level of quality, it is important to make a proper analysis, aimed at measuring and standardizing the times and methods required to carry out certain basic activities, such as moving objects or tightening screws. . However, manual analysis of work processes is very difficult and involves a large amount of time. In response, great efforts are being made to automate such analysis, including using artificial intelligence. But until now, the adoption of AI has been hindered by the imperative need to determine the actual formation of the training data required by the artificial intelligence to be able to interpret the differences that exist between each worker and between the different production processes implemented.

opening image: An advanced system for the analysis of manual activities in the workplace and the subsequent improvements in terms of efficiency at work.