Artificial Intelligence Is Getting Insanely Good at Removing Shadows From Photographs of Faces

While Photoshop can do a pretty good job at removing shadows from faces, there’s a fair amount of legwork involved. One scientist has shown that neural networks and artificial intelligence can produce some very impressive results, suggesting that it will soon be a part of how we edit our photos.

Károly Zsolnai-Fehér of Two Minute Papers and the Institute of Computer Graphics and Algorithms, Vienna University of Technology, Austria, just released a video demonstrating how he has taught a neural network using large data sets to recognize and eliminate shadows from a face in a photograph. As detailed in the video, the neural network was taught by giving it photographs of faces to which shadows had been added artificially.

Given its effectiveness and the quality of the results, it seems only a matter of time before smartphones give you the option to remove shadows. In theory, you might even be able to switch on shadow removal while taking the photograph. 

Numerous university departments around the world seem to be pushing each other’s research to produce ever-improving results, with neural networks key to processing the data required to understand the image. If you’re interested in learning about how Zsolnai-Fehér is using neural networks, be sure to check out this video.

Will shadow removal soon be a one-click solution in Photoshop? Let us know your thoughts in the comments below.

Log in or register to post comments


microteck's picture

Yes, technology is very impressive. But who said a shadow is a bad thing? Sometimes it gives character to the photo. That's why it's important to know how to use your camera in manual mode.

Lee Christiansen's picture

Knowing how to use a camera in manual mode makes zero difference to placement of shadows. And although yes shadowing can add depth, awkward shadows are often unsightly.

And in many circumstances, we have no control over those shadows - particularly when shooting candid moments. Not every shot allows us to pop a reflector in, or take time to find a new position for the subject, or to reposition the sun...

And photography isn't always about recreating physical reality. Sometimes it is about recreating how our eyes perceive the world, (which is very different), or it is about presenting people at their best (particularly when trying to sell those same images).

Manual mode isn't going to lift an awkward shadow or re-point a light.

microteck's picture

Manual mode simply means you should know how to take a professional picture with total control. Some people don't understand. But that is okay, because some people just like using their smartphone.

Lee Christiansen's picture

Except there is often no total control. I'll take you on some of my Bar Mitzvah shoots with evening parties... flashing lights, smoke machines, changing colour balance, people set against backlights and often in shadow - all requiring split second captures... Very little control over this stuff.

And lack of control would apply to many other forms of photography too. I shoot studio work where I can finesse every aspect before pushing the button, but in many cases for PR, editorial, street, events - my level of control varies hugely. And for these... well PS and C-1's ability to dodge and burn is an essential part of the photography process.

Smartphone or DSLR - it's all the same. Shadows are a function of light, and we don't always get to control that light at the critical moment.

Jerry Suppan's picture

That is absolutely right Mr. Cool Cat! Some people DO just love using tgheir smartphones. Smartphone technology is advancing in leaps and bounds beyond the traditional behemoths of the industry which just wallow to or for in the same path year after year. Sales performance have been testimony to that as well as camera sales have been in descent mode, going down, down, down over the past 10 years. It's too bad the camera industry can not pick up some cues from the smartphone industry making it hugely successful over the traditional DSLR, MILC types. Algorithmic, computational-based photography is ramping up annually if not biannually. Manual control can also be done through smartphones as well if that is so desirable, but I sure love the convenience of advancements in computational photography. And what is the best camera for any event? The one you have with you. And that, is usually in my shirt pocket. The times they are a changin'.

microteck's picture

Everything you stated is true. But for many of us computational-based systems takes the fun out of photography. But hey, whatever floats your boat, right?

anthony marsh's picture

Sorry but Photoshop and AI are not photography. If one cannot make a good photograph using those are not creativity,merely "creating" a picture.

Lee Christiansen's picture

But photographers have been manipulating their images since before PS. Even with dodging and burning, the masters of photography would spend countless hours manipulating their images.

Or they could retouch skin by actually "polishing" the negatives with a rather expensive contraption... all part of the photographic process.

PS is just a more convenient and accurate way of making adjustments. And yes we can have sky replacements etc, but again this has been done with more traditional photographic methods before PS - albeit not with the same precision.

As to AI, for the main it is a bit of a marketing hype. But if we can have a computer take the grunt work out of things to allow us to be more creative, then happy days.

For me, photography does not start and end at the pressing of a button. Because when I light something I am cheating the natural light. When I dodge/burn something I am cheating the natural exposures. When I change a colour balance I am probably changing the natural colours. If I retouch then I'm certainly cheating in general and if I clone things then I'm cheating reality.

But for me, photography is a process - whether chemical, optical or electronic. And it only starts with a button press. After which it goes on a whole journey to create an image.

I guess it depends. Are we merely taking a slavish record of how light interacts with a sensor / film, or are we creating something. Both ideals are valid, and both need have no preconceived limits.

microteck's picture

Almost everything you said is true. But sometimes too much is too much. Some of us rather wait for a nice blue sky and fluffy white clouds before we take the shot. We don't mind a little more planning. You prefer the quick and dirty method of using Photoshop for sky replacement to take away the grunt work. Some of us prefer doing as much as possible in camera. We both have different ways. Just enjoy yourself.

Lee Christiansen's picture

If you're doing it for fun, then wait for days on end to get a sky. But for commercial work, we have different needs. And maybe we've already taken a great sky ourselves for our own library of images. (And yes for the record, that's what I've done in the past).

But you are referencing one of the more extreme examples. PS is used electronically for otherwise chemical and optical processes.

Post production isn't quick and dirty. It could be argued that a lack of post work is also quick and dirty.

microteck's picture

Isn't this fun Lee?

Best Regards,
Quick & Dirty

anthony marsh's picture

Of course they have including ANSEL ADAMS,however they used artistic methods that could take years in the darkroom to master. Today it is not so much about artistry but gimmickry,press a few buttons and voila! an "image",not a photograph.

Ryan Luna's picture

spoken like an artist. Nothing wrong with that, but many photogs are in the business of producing images where there may not be time to mitigate shadow placement, (events and weddings come to mind).

anthony marsh's picture

Great photographers whether photographing weddings or other events have produced extraordinary photos following years of learning and they made good livings. I may be too critical however to my mind holding a shutter button down for dozens if not hundreds or thousands of clicks does not comport with talent or artistry as anyone with a digital camera and using editing can achieve similar results.

Enrique Olivieri's picture

The more you add AI into photography, the less it becomes photography. I guess it's becoming an entire new form AI were you create an AI model and clothes. Check out on Instagram and you'll see where photography is probably headed, the models are not real on this Instagram page, they're all AI created.

Jerry Suppan's picture

"The more you add AI into photography, the less it becomes photography."

Welcome to the digital era in which anything is possible, even more so than before. Morphing in videos for another example. Whatever. ;-) I reckon is someone does not like the technology, they can choose to not use it.

anthony marsh's picture

Anything except true artistry and technical knowledge of photography is possible.

Black Z Eddie .'s picture

It would be great for family/friends get-togethers, events, and such.

jim hughes's picture

A great job of removing all the interest from a photo.

anthony marsh's picture

While not still photography ANDY DAY should watch the original THE GODFATHER to understand the value of shadows. Removal of all shadows makes for a boring sterile image.