Face Mask Detection tutorial using ML.NET

Public shaming over not wearing a face mask started almost as soon as the COVID-19 pandemic itself. Nowadays, a lot of countries made it mandatory to wear face masks when in public. Every single person in the world must recognize how important it is to wear a mask to slow the spread of SARS-CoV-2, the virus that caused COVID-19. As a result, today we have a lot of companies pioneering face mask recognition software to get people to comply for the public good.

Up until COVID-19 hit, there were just a handful applications for mask recognition. And traditionally masks have been confounding facial recognition software, until now. With the new machine learning tools, we can increase the accuracy of recognizing if a person wears a face mask.

And this is what we are going to do today. We will learn how to leverage the ML.NET Model Builder to resolve this problem.

The Face Mask Detection Problem

A Facial Recognition software analyzes the features around the eye, nose, mouth, and ears to identify the person whose picture is supplied. Like we mentioned before, wearing a mask obstructs the recognition process. Because of that a lot of companies issued an update to their facial recognition system. Instead of recognizing the person on the image, they now scan for a person wearing a mask. If the person is wearing a face mask, they are prompted to enter their password. Otherwise, the normal face recognition workflow is executed. What I am trying to convey is that a lot of companies had to solve this problem and adapt their system accordingly.

A lot of people are rightfully worried about this type of software. Is it breaking our privacy rights? Well mask recognition software in theory bypasses the privacy issues because programs do not actually identify people. And we will see the correctness of this statement in just a bit. Like others before us, we will train a ML.NET Model on two sets of pictures. One set to teach the algorithm how to recognize a person with a face mask on. And the other set will consist of pictures where the person does not wear a mask.

But before we do that, we must have another module in place. And that one is responsible for Face Detection. We will use a famous algorithm to extract faces from an image. Then the cropped face will be handed over to the face mask recognition system. As you can see our application is not identifying faces in any way. So, it can not link a face to a specific person. That is because we are not using images of faces that are linked to their identities.

ML.NET Model Builder

In this tutorial we will use the ML.NET Model Builder to create a face mask detection module. Model Builder is a simple UI tool for developers to build and train custom machine learning models in their .NET applications. This tool is especially useful for developers that have no prior knowledge on Machine Learning. Because ML.NET Model Builder supports AutoML, it is very easy to find the machine learning model best suited for our scenario. AutoML allows us to load the dataset and train the model right away. As a result, the Machine Learning complexity is delegated to AutoML. Because of this, the Model Builder can be used by anyone trying to solve a Machine Learning problem without a prior knowledge.

Keep in mind is that this tool is shipped with Visual Studio 2019. If you use an older version of Visual Studio, then the Model Builder might be in a Preview Mode.

To make sure that you have your environment set correctly you can go back and check my article on how to activate and use ML.NET with Model Builder.

This article is not about ML.NET or the Model Builder as a tool. So, I will only focus on some key points. If you want to learn more continue reading up on the following link.

Image Face Detection with C# NET – CODE-AI NET with Source Code (code-ai.mk)

C# Application

This application is continuation to the Face Detection app we built some time ago. The goal in this article is to detect whether a person on the image, wears a face mask.

First, we need to open an image. Because we are using the Haar Classifier we need to set up couple of parameters. Those parameters are explained in the previous post. The addition to this application is the ML.NET Face Mask Recognition module. Once the face is detected, it is cropped. Then the image is passed to the face mask detector and it generates one of two results. Either the person wears a face mask or not.

ML.NET: Data Set

Out dataset consists of two types of face images. Images that contain a face wearing a mask. And images having a person without a mask. This is our goal. Given a face image, we need to classify said image in one of those two categories.

This is how the training set looks like:

The dataset is very small. This is because I would like to train my model locally. You can get a bigger data set and train using Azure. We will cover how to train on the cloud soon.

Face Detection

I am not going to talk too much about Face Detection, just because we have covered it in a previous article. So, Face Detection is the process of identifying a human face in a digital image. As a result, we will get a bounding box around the desired object on the image – a human face. In this project we are using the Viola-Jones Object Detector based on Haar-like features.

Accord.NET library has implementation of this technique in the HaarObjectDetector class. This class implements the Viola-Jones object detector algorithm. This technique is used for object detection in images. But it is not only for faces. It can be used on any type of object. But we need faces so, we are going to create an instance of FaceHaarCascade class and pass it to the HaarObjectDetector instance.

One very important thing to note here, is that this algorithm might perform worse on images where a person wears a mask. This is because some key facial features like the nose and mouth are obstructed. The Haar Classifier still works fine most of the time, but do not be surprised if it misses a face. In a situation like this, you can experiment with the face detection parameters and improve your results.

Eventually we will build a face detector using ML.NET and Deep Learning. But for now, this will do.

Face Recognition

The Face Recognition module consists of the ML.NET model generated by the Module Builder. If you have followed my other tutorial on how to use it, you should have two projects. One is a .NET Core project with the Model suffix appended to it, and the other is a .NET Code Console Application.

Your project structure should be something like this:

In my scenario the FaceDetectionAppML.Model contains the trained model produced by the ML.NET Model Builder tool. The FaceDetectionAppML.ConsoleApp is a .NET Core Console Application that consumes and uses the generated model.

The Solution Structure

The Face Mask Detection application uses the .NET Framework. However, the other two projects use the .NET Core 3.1. ML.NET has a known issue with .NET Framework so we will need to find a workaround to use our trained model inside the Face Mask Detection application.

To consume the ML.NET model we will call and send commands to the .NET Core console application. We will launch it as a separate process and send the extracted faces to be processed there. The app will return a string of the category that image belongs to. In other words, it will return a “Mask” or “No Mask” string.

We will execute and control the console application via C#  System.Diagnostics.Process class. We want to be able to pass multiple images on a single application instance. This is because loading up our ML.NET Model can be time consuming. What we want to do, is load the application once and just pass image file paths for classification.

There are two ways we can easily pass an image. We can use Shared Memory Manager, or we can save the image to a disk and send the file path. In this demo the latter is used.

C# Code Walkthrough

The C# code is quite straightforward. First, we need to load up an image. The Image Selector class checks if the image is valid or not. If the image is valid the Image Processor class takes over. Its soul goal is to optimizes the image for the Face Detector. As you can see, we do a gray scale operation, followed by Histogram Equalization. This prepares the image for the Face Detector module.

Then, the Face Detector executes the algorithm, and it returns a rectangle array. A single rectangle represents the face of the image by its X, Y, Width and Height properties. As a result, we can enumerate the array. For each face rectangle, we crop the image and save it on disk. If the image is saved the Face Mask Recognition module picks it up and passes it to the .NET Core console application. This C# project uses the ML.NET model we trained to classify the face image. As you can see, we call the function PersonWearsMask. This method returns true if the person on the image wears a face mask.

Then all we have left to do is to draw the bounding box with the correct color. Green if face mask is present, red otherwise.

And that is pretty much it.

Face Mask Detection using ML.NET Conclusion

You can see how easy it is to build and train a model using ML.NET. However, using it in .NET Framework application is not that simple. The ML.NET and Model Builder modules are still work in progress. So, there are bound to have some issues at this point of time. But still, we managed to find a solution to our problem. In the next article I will describe it in more detail. Getting comfortable with the Model Builder is very important because soon we will start writing ML.NET code. This article does not contain a project for download simply because it is too big. But you have all the tools to recreate it.

One of the weakest links in this app is the Face Detection module. Please note that we are using Haar features trained on faces without a mask. This means that the face detection algorithm might miss a lot of faces. We will deal with this later. For now if you experiment with the provided parameters, you actually may get decent results

More ML.NET Tutorials

• How to use ML.NET for Image Classification
• Image Face Detection
• Article Classification using ML.NET