A recent study has shown that a generation model of AI images like Stable Diffusion it can reconstruct highly accurate images from a person’s brain activity. All without the need to train or fine-tune the AI. A direct link between the human brain and artistic creationthrough artificial intelligence.
From brain activity to images, using Stable Diffusion AI
Researchers at Osaka University created a latent representation of image data from fMRI signals. An image of the image data. Which they then elaborated on by adding noise through the diffusion process.
Subsequently, they decrypted the text representations from fMRI signals in the superior visual cortex and used them as input to produce a final image. This study was the first to provide a quantitative interpretation of the pattern of diffusion from a biological perspective. In practice, revealed the correlation between stimuli and noise levels in the brain and the engagement of several neural networks for noise reduction and high-resolution image reconstruction.
The data collected by the experiment, according to the scientists, provides a model of how our brain works when creating images. “These results suggest that, early in the reverse scattering process, image information is compressed within the bottleneck layer. As denoising progresses, a functional dissociation between U-Net layers within the visual cortex: that is, the first layer tends to represent fine-scale detail in early visual areas, while the bottleneck layer corresponds to higher-order information in more ventral semantic areas.
This study isn’t the first to look for correlations between brain waves and images. In 2014 in Shanghai, the artist Jody Xiong connected balloons filled with paint to the ECG machines of some patients with disabilities. Being able to make them paint using the brain.
But this study leverages Stable Diffusion to allow images to be created using only the brain. A result that it could have major impacts for so many people around the world. Although first it will be necessary for other universities to successfully replicate the experiment.