Google’s New AI Models Turn Low-Resolution Images into High-Quality Pictures

Sep. 5, 2021

As researchers push their limits to develop advanced artificial intelligence (AI) technologies, we have seen several AI tools and systems to improve imaging technology. We have seen AI tools toinstantly remove backgrounds from imagesandde-blur photos. Now, Google has developed two AI-based tools based on diffusion models that can convert low-res images into high-quality photos.

Dubbed as Super-Resolution via Repeated Refinements (SR3) and Cascaded Diffusion Models (CDM), the two new technologies were recently developed by Google Research’s Brain Team. The Mountain View giant recently publishedan in-depth blog poston its AI forum, detailing both technologies. It is similar to theprevious AI algorithmthat we saw researchers at the Duke University of North Carolina develop earlier this year.

Now, starting with the SR3 model, it is essentially a super-resolution diffusion model that canconvert low-resolution images into high-res ones from pure noise. It takes a low-resolution image as input and uses an image corruption process, using which it was trained, to progressively add noise to the image until only pure noise remains. It then reverses the process and starts removing the noise to reach the target image with the low-res input image as a reference.

Now, coming to the second AI model, the Cascaded Diffusion Model (CDM) is a class-conditional diffusion model that has been trained on ImageNet data. This enables the model to churn out high-resolution natural images by chaining multiple generative models over several spatial resolutions.

In this process, theCDM model uses one diffusion model to generate data at a low resolution, followed by a sequence of SR3 super-resolution diffusion models. This gradually increases the resolution of a low-res image to its highest resolution. You can check out the GIF attached below to get a better idea of the image generation process.

Other than the two models described above, the researchers at Google AI alsodeveloped a new data augmentation techniquecalled conditioning augmentation. It further improves the sample quality results of CDM by using Gaussian noise and Gaussian blur. Moreover, it prevents each super-resolution model from overfitting to its lower resolution conditioning input. It results in better high-resolution sample quality for CDM.

So, with the above AI-based image improvement models, Google says that it has pushed the limit of diffusion models to the state-of-the-art on super-resolution and class-conditional ImageNet generation benchmark. The researchers will further test the limits of these models for more generative modeling problems going forward.

A geek at heart, you’ll find me fidgeting with an iOS device or sitting in front of the laptop, scouring the internet to find interesting technological innovations. In my free time, you’d find me playing Valorant or listening to Pink Floyd.