Phancer AI Attempts to Inject DSLR Quality Into Smartphone Photos

Phancer artificial intelligence — This sample image was shot with an iPhone 6 and edited using the Phancer AI program. Camera Lab ETH Zurich / Andrey Ignatov et. al.

Smartphone cameras remain more limited than their larger DSLR counterparts, but scientists are hoping to narrow the gap through artificial intelligence. In a recently published paper, scientists from the Computer Vision Lab at ETH Zurich developed a neural network system dubbed “Phancer” that can take a smartphone photo and give it the vibrant colors and sharpness from a DSLR camera.

The team started by training a network using photos taken from the exact same spot with a smartphone and then with a DSLR and comparing the quality between the two. Attempting to transform a smartphone image to a DSLR shot, however, created a loss of content, color and texture. To correct for those changes, the researchers created a 12-layer convolutional neural network (CNN), with three of those CNNs working to correct content loss, color loss and texture loss. The group then trained the system as a whole, which means that the first CCN contains the capabilities of all those layers.

Since the system works on a comparison, the program needs to know which smartphone you used for the shot and which DSLR you would like to imitate. If shots from both are in the database, the system can apply an algorithm to improve that uploaded smartphone shot.

The result is a photo with the exposure and colors that are closer to that DSLR hardware, despite being shot on a smartphone. The system isn’t perfect — researchers say that odd color casts are common, for example, adding a green tint to the overall image. The software can also go overboard on adding contrast, the researchers said, and in images from lower quality cameras, the program can exaggerate noise.

The software only looks at a handful of the differences between smartphones and DSLRs, namely color and sharpness. The research doesn’t mention attempting to mimic the depth of field (which AI can already do with the right hardware, either a dual-lens camera like the iPhone Plus or dual-pixel sensor like the Google Pixel 2). The software in some cases also emphasized the graininess from smartphone cameras that are less pronounced from DSLRs. But, coupled with other research projects and the existing tech, the algorithm could help AI bridge some of that gap between smartphone hardware and dedicated cameras.

The software is available for users to try out online, along with a set of before and after results of what the program can do.

The group isn’t the first to attempt to use AI to give smartphone photos better quality — earlier this year, a group from UC Berkeley created a style transfer algorithm that takes the style of one image and applies it to another, but they included the attempt to transfer DSLR style to a smartphone among the list of failures.