Google Research Blog

Google explains the tech behind the Pixel 2’s Motion Photos feature

Apple was the first mobile manufacturer to popularize still/video hybrid files with its Live Photos that were introduced on the iPhone 6s. Google then launched the Motion Stills app to improve and stabilize Apple’s Live Photos, and ported the system to the Android world soon after.

For the new Motion Photos feature on its latest Pixel 2 devices Google built on Motion Stills, improving the technology by using advanced stabilization that combines the devices’ soft and hardware capabilities. As before, Motion Photos captures a full-res JPEG with an embedded 3 second video clip every time you hit the shutter.

However, on the Pixel 2, the video clip also contains motion metadata that is derived from the gyroscope and optical image stabilization sensors.

This data is used to optimize trimming and stabilization of the motion photo and, combined with software based visual tracking, the new approach approach aligns the background more precisely than we’ve seen in the previous Motion Stills system (which was purely software-based). As before, the final results can be shared with friends or on the web as video files or GIFs.

If you are interested in more technical details of the Motion Photos feature, head over to the Google Research Blog. A gallery of Motion Photo files is available here.

→ continue…

From:: DPreview

Google just made the tech behind its ‘portrait mode’ open source

Semantic image segmentation is the task of categorizing every pixel in an image and assigning it a semantic label, such as “road”, “sky”, “person” or “dog”. And now, Google has released its latest image segmentation model as open source, making it available to any developers whose apps could benefit from the technology.

The function can be used in many ways. One recent application in the world of smartphones is the portrait mode on Google’s latest Pixel 2 devices. Here, semantic image segmentation is used to help separate objects in the foreground from the image background. However, you could also imagine applications for optimizing auto exposure or color settings.

This kind of pixel-precise labeling requires a higher localization accuracy than other object recognition technologies, but can also deliver higher-quality results. The good news is that Google has now released its latest image segmentation model, DeepLab-v3+, as open source, making it available to any developers who might want to bake it into their own applications.

Modern semantic image segmentation systems built on top of convolutional neural networks (CNNs) have reached accuracy levels that were hard to imagine even five years ago, thanks to advances in methods, hardware, and datasets. We hope that publicly sharing our system with the community will make it easier for other groups in academia and industry to reproduce and further improve upon state-of-art systems, train models on new datasets, and envision new applications for this technology.

If you are interested in finding out more about DeepLab-v3+, head over to the Google Research Blog for more details.

→ continue…

From:: DPreview