Yesterday, I wrote about Fawkes & Photo Ninja which can be used to protect your photos from facial recognition models and reverse image searches. This is a very interesting field and I had mentioned about creating a service that does it for free instead of charging like what Photo Ninja is doing.
The first step to that is to check if the program (Fawkes) actually works the way it is supposed to, so I downloaded a pic from the internet (my profile pic on Twitter) and ran it through Fawkes. The program takes a while to run (~20 seconds per image) depending on the no of people in the photo. It detected the faces very reliably and modified the image. When using the default settings the output is saved as a PNG file but you can override it using a command line parameter. It requires you to provide the directory you want to run it against but if you don’t pass it the directory, it doesn’t give any errors. It took me a few mins to figure out what the issue was (yes, I know… My brain is tired). The command to run it in the current directory with debug (because I like seeing what the software is doing) is:
./protection --debug --directory .
I then took the resultant, file and searched for it via Google Images, Yandex and TinEye. None of them were able to find any results with the new image. So that part of the software works great. 🙂 Now coming to how the software modifies the image, I saw that it adds 2 rows of pixelisation to the image. First is near the hairline and cuts across the hair and forehead, and the second is near the chin and is about 5-10 pixels wide. It is clearly visible in larger photos, but when zoomed out it doesn’t look too jarring. Frankly it looks like the image got damaged and is kind of obvious when you look at it.
In my very basic tests it made the same change everytime so I have a feeling that we can train image recognition software to look for this modification and ignore it. It might be more powerful to put the modifications at random locations in the image (over the faces) that way it is harder to train the software to counter it. Plus if the visual noise section can be reduced it would be great. Maybe instead of a long blur that is noticeable we can try to do multiple small changes that change the pic without making it obvious that the image was modified.
Below are the two images, the original on the left and the modified version on the right.
Sample output of the Fawkes
I then looked at running this on my webserver, but due to the restrictions there I wasn’t able to get it to run. Although, to be honest I only tried for about 20-30 mins because I was tired. If I can’t get it to run on the server then the other option is that I run it on my home computer but I will need to look at that in more detail before I commit to making this site. I have a rough draft of the requirements and feature list but still looking at the options before I start working on it. It will be a good way to take my mind of what is going on in the world so that is good.
Well this is all for now. Will keep you posted on how this project goes.
– Suramya