Unleashing Creativity: Exploring the Mysterious, Multiple Dimensions of AI Image Making

Artificial Intelligence (AI) has made remarkable strides in various fields, and one area that has captured our imagination is AI-generated images. These images, created by diffusion models such as Midjourney and Stable Diffusion, have the ability to generate new, realistic images. In this article, we will delve into the core concepts behind AI text-to-image models, such as latent space, image encoding, denoising, and text-to-image generation. So, let's embark on a journey to demystify these complex mathematical processes and understand how AI brings our imaginations to life.

Understanding Encoding and Latent Space:

At the heart of AI diffusion models lies the concept of latent space. Latent space refers to an abstract multi-dimensional space that encodes a meaningful internal representation of externally observed events. But what does that mean?

Let's unpack this concept. Consider the process of JPEG compression, where a high-resolution photo is compressed into a smaller file size. This is an example of encoding and decoding. The algorithm that compresses (encodes) and uncompresses (decodes) the photo was created by scientists through observing how vision works and deciding what parts of an image are important to human perception. Brightness is more important than color, for example, and large details are more important than small details.

In AI, the algorithm for encoding and decoding is something that is created automatically by an autoencoder, which is an unsupervised artificial neural network. The autoencoder's goal is to learn how to take some input (text, images, text & image pairs, etc.) and learn how to encode them as efficiently as possible. The autoencoder, by design, learns what is signal and what is noise, keeping the important parts and discarding the unimportant. Similar to our JPEG algorithm but much much cooler. Once it learns the algorithm for decoding, it can also decode things that were not specifically in the training data... like I said, way cooler.

The compressed representation of any particular piece of input data could be thought of as a point or set of coordinates in what is known as latent space. This space is often described as containing hidden meaning or a hidden representation. Geoffrey Hinton, one of the superstars of Deep Learning, coined the term "thought vectors." So we can imagine that when words, images, sentences, language, etc., and their relationships to each other are compressed into a shared latent space, each point, of which there are billions and billions, could be considered a thought/idea/concept.

Latent space is an unknown multi-dimensional data wonderland that even its creators don't fully understand. I'm afraid visualizing anything more than 3 dimensions doesn't make any sense. Our friend Geoff Hinton,has a tip for imagining high-dimensional spaces, e.g., 100 Dimensions. He suggests first imagining your space in 2D or 3D, and then shouting "100" really, really loud, over and over again. I.e., no one can mentally visualize high dimensions. They only make sense mathematically.

Understanding Denoising and Transforming from Text to Image:

Denoising is another crucial part of the equation. If you have ever taken a photo in a low-light situation, you may be familiar with image noise. If there's not enough light for the camera's sensor to capture the photo correctly, the resulting image may have many small dots all over the image. These dots are called noise, and in this sense, noise is basically errors. Denoising is the process of using algorithms to remove errors and restore the image to its original state.

In AI image making, a machine learning model learns how to denoise latent space image representations. This was one of the big innovations of diffusion models. It's much more efficient to denoise latent representations of images rather than pixel images.

By adding a specific amount of noise to an image then asking the model to predict how much noise was added, it can compare its prediction to the actual amount of noise that was added, evaluate its effectiveness, and improve its noise-predicting algorithm. Once it is really good at denoising, it is able to make a good prediction at denoising a completely noisy image. Which is exactly how the diffusion process starts. Like the autoencoder that can decode data that is not specifically in its training set, the diffusion process, starting with a completely noisy image, is able to generate completely unique images that were not part of its training data, thanks to its denoising and decoding.

The prompt is also encoded into the shared latent space. As the denoising process is run over several steps, it guides the denoising process so that the resulting image captures the relationships between the words in the prompt and their latent equivalents. The final image is decoded, revealing its hidden meaning from this multidimensional word-image superbrain.

As we navigate the digital era, let the enchanting mathematical brushstrokes and innovative techniques showcased in AI image making inspire us to redefine the possibilities of human imagination and artistic creation. The exploration of AI diffusion models, with ideas such as latent space, image encoding, denoising, and text-to-image generation, let us dive into a future where art leaps beyond the physical, weaving a digital tapestry through multiple dimensions. Let us know what you think and gear up for your own cosmic creative journey.

Article by @terrygates

Image courtesy of @terrygates

RED•EYE WORLD