The app appears to contain a trio of AI-powered developments on the forefront of today’s photo software research: 3D-facial mapping, automatic portrait masking and style transfers.
With 3D-facial mapping, the software can identify the facial features and make slight adjustments both in the pose and the appearance created by focal length. By collecting 3D-mapping data, the entire face can be moved, pointing the chin down for a more flattering look or tilting the face to one side — Adobe demonstrated the possibility in another teaser earlier in 2017.
The same 3D-facial mapping also allows the software to mimic the effect of a portrait lens. Wide-angle lenses tend to distort facial features, making the face appear thinner but the nose to appear larger — that is why many portrait photographers shoot with a 50mm or larger lens. In the teaser video, however, the system recognizes each facial feature and with a slider can create the look of a longer lens from a smartphone photo.
But that is not the only DSLR-like quality that Adobe Sensei is hoping to bring to smartphone photography. Using auto-portrait masking, the app will be able to mimic a shallow depth of field effect. The small sensor of a
Adobe, however, appears to be teasing an effect that can be accomplished with single lens smartphones. The company did not go into detail on just how the software separates the background but in March, Adobe created an auto clipping program by allowing the neural network to watch more than 49,000 images being clipped by actual people.
The third feature in the teaser allows the app to transfer the style of one photo onto the image, like Prisma, only a photo-to-photo edit instead of turning a photo into a painting. Just two weeks ago, Adobe Research published a paper demonstrating software that was able to mimic the style of another photo, even allowing smartphone snapshots to mimic high-end magazine photos.
For now, Adobe is only teasing the tech — it’s unclear how close the program is to reality. Adobe, however, did confirm that the app in the video isn’t a computer-generated image. But, it does take the biggest advancements in using AI to edit images from academic research and puts them all into one consumer-oriented app.
