Fujitsu Develops AI-Video Recognition Technology to Promote Hand Washing Etiquette and Hygiene in the Workplace

Fujitsu Laboratories Ltd. and Fujitsu Exploration and Improvement Centre Co., Ltd. (FRDC) these days introduced the growth of “Actlyzer” hand clean movement recognition technology, which leverages AI and equipment discovering techniques to determine complicated hand washing movements from online video info captured by digicam.

Impression credit history: Pixabay (Free Pixabay license)

Amidst the ongoing world wide COVID-19 pandemic, the significance of hand washing as a evaluate to shield people’s health and fitness from germs, influenza, and other infectious health conditions is gaining renewed focus around the world. Beneath new laws(one) prepared to come into outcome in June 2020 in Japan, foods business operators will also be demanded to implement more powerful actions to assure hygiene in accordance with worldwide HACCP(two) foods protection expectations, generating an urgent want for a non-invasive strategy to speedily and precisely ensure that handwashing is carried out in a good manner.

Anticipating this, Fujitsu has expanded the recognition functionality of its current “Actlyzer behavioral analysis technology”, which can understand a variety of refined and complicated human movements without relying on massive amounts of education info. Exclusively, Fujitsu has refined recognition capabilities for hand movements to generate technology to routinely understand complex hand movements carried out throughout hand washing.

This technology would make it feasible to simply establish regardless of whether a person is pursuing every single of the six actions for handwashing proposed by the Japanese Ministry of Wellness, Labour and Welfare, cutting down the range of male-several hours demanded for intrusive visual checks by inspectors for on-website sanitation management.

Shifting ahead, Fujitsu envisions this technology currently being utilized in a variety of other contexts, like in clinical amenities, educational facilities, inns, and venues for massive occasions, and plans to conduct subject trials and more exploration and growth into it as a opportunity remedy for its AI portfolio in the long term.

Developmental Background

The Japanese Ministry of Wellness, Labour and Welfare endorses that men and women follow six actions to assure their fingers are thoroughly washed and reduce foods poisoning as perfectly as the distribute of infectious health conditions (Figure one).

Figure one: six Steps for Correct Hand Washing. Impression credit history: Fujitsu

At existing, foods assistance industry workers are demanded to perform these six actions when washing their fingers and ensure that throughout every single move the distinct pieces of their fingers have been rubbed more than a specified range of situations. To this stop, foods assistance vendors fill out a check sheet and make a self-report, and workers on-website bear visual checks by supervisors. Since verification is however performed manually, having said that, prices involved with human error, securing sources for inspectors, etcetera. stay a persistent obstacle. As a lot of manufacturers, like in the foods industry, proceed the march towards course of action automation for inspections by applying equipment discovering systems, hand washing checks depict a primary prospect for course of action optimization through automation.


Gesture recognition applying deep discovering is a typical approach for determining hand and finger movements. This conventional approach can detect various characteristic points, this kind of as joints and fingertips, from an impression of the hand, and establish the hand gesture primarily based on the positional relationship of the characteristic points (Figure two Left). Having said that, one issue with the current technology was that when men and women clean their fingers the right way, the two fingers overlap and are lathered with soap, which obscures the detection points on the fingers and helps prevent correct gesture recognition (Figure two Appropriate).

Figure two: Traditional hand tracking tech (Left: Gesture recognition outcomes Appropriate: Hand clean conduct). Impression credit history: Fujitsu

Recently Created Technologies

To resolve this obstacle, Fujitsu Laboratories Ltd. and FRDC have designed a new AI technology that routinely and precisely recognizes hand movements below the conditions explained above, increasing the recognition functionality of the first Actlyzer behavioral analysis technology.

With the new technology, the complicated hand movements of handwashing are captured as a mixture of hand form and repetitive rubbing motions, detected by two deep discovering engines: Hand Condition Recognition and Motion Recognition (Figure three).

Figure three: Recognition of complicated two-hand finger movements as a mixture of the general form and movement styles of the two fingers. Impression credit history: Fujitsu

The two-hand form recognition motor makes use of a learned model of a fundamental form of two fingers, which is a usual sort of hand movement in which fingers are put on best of every single other, to establish the handshape for every single frame of the impression. Concentrating on the general form solves the issue of when fingertips and joint characteristic points simply cannot be the right way detected due to hand overlap or foam. In addition, Fujitsu’s exclusive AI technology “High Durability Learning (Superior Durability Learning)”, which can track info alterations, is used to assure that the fundamental form of the two fingers is identified with large accuracy even when the digicam position or lighting alterations throughout procedure.

The motion recognition motor makes use of a learned model that detects periodically altering motion from successive frames and counts the range of iterations as the range of rubs from the iteration pattern and its time period. In addition, the outcomes of these two recognition engines are fed back to every single other to boost recognition accuracy. The motion recognition motor sets a threshold worth for the magnitude of the motion to be judged in accordance with the actions identified by the two-hand form recognition motor to reduce detection of faulty durations, this kind of as hand tremors not similar to foam motion or rubbing. The two-hand form recognition motor improves detection accuracy by filtering the judgment outcome applying the repetition pattern time period detected by the motion recognition motor.


A hand-clean online video info set with around 2000 variants like men and women, digicam positions, and soap sorts, was independently filmed and collected for discovering and analysis. It was confirmed that the accuracy of six actions of appropriate hand washing is average of 95% or more, and that the accuracy of the range of fingers rubbing movements is ninety% or more. When running the technique on-website, the omission of motion could be prevented because the particular person washing their fingers generally knows how extended to scrub their fingers and which of the six stages they’ve finished because the technique determines it’s exhibited on the display right until it’s finished. In addition to this, the technique routinely records info with the starting off timestamp and size of motion of finishing every single six actions, and the range of situations of rubbing. (Figure four).

Figure four: Case in point of recognition display screen. Impression credit history: Fujitsu

This technology automates on-website handwashing checks for workplaces that have to have demanding hygiene command, reducing the range of male-several hours wanted for visual affirmation and guide recording. On top of that, because the technique does not understand incorrect or incomplete hand washing, Fujitsu expects the remedy will support teach end users and assure standardization for the proper way of handwashing.

(one) New laws Act for Partial Revision of the Food Sanitation Act

(two) HACCP Hazard Examination and Essential Command Issue. A sanitation management system that assures protection by managing procedures that are specifically important for reducing or cutting down harmful things in the whole course of action from the receipt of raw resources to the cargo of goods, primarily based on the business operator’s knowing of hazards this kind of as contamination by foods-borne germs. This is turning out to be a obligatory system generally in designed international locations.

Resource: ACN Newswire