I came across a fun experiment being done by Mitchell Lensink which he shares on one of his blogs.
He takes one of his photos, feeds it into one of the AI large language models and asks the LLM to produce a detailed description of the photo. He then continues by asking the model to create a new image based upon the description it produced as part of its analysis.
I tried this out on a few of my images and enjoyed seeing the results. Here’s an example…
Prompt (taken from one of his examples): Please describe the attached image as well as you can. Don’t skimp out on details and make sure you will be able to understand what’s in the image when you read the description. You need to be able to form a mental picture of the image, without actually seeing it.
Here’s what Google Gemini 2.0 Flash produced:
Imagine a serene lake bathed in the warm glow of either sunrise or sunset. The sun, positioned towards the upper left of the frame, casts a brilliant light that flares slightly around the edges of the tall trees lining the left bank. These trees are silhouetted against the bright sky, their dark shapes contrasting sharply with the golden light.
The lake itself occupies the majority of the frame, its surface reflecting the sky and the surrounding trees like a mirror. Gentle ripples disturb the otherwise smooth surface, creating subtle patterns of light and shadow. In the middle of the lake, a fountain sends a spray of water upwards, catching the light and adding a dynamic element to the tranquil scene.
The far bank of the lake is also lined with mature trees, their foliage appearing lush and green where the light touches them, and darker in the shadows. A low wooden fence runs along a portion of this bank, suggesting a pathway or boundary.
In the foreground, along the near edge of the lake, there’s a patch of vibrant green grasses interspersed with delicate white and yellow flowers. These plants are in sharp focus, drawing the viewer’s eye into the scene and adding a touch of natural beauty in the immediate foreground.
The overall atmosphere is peaceful and picturesque, a moment captured in time where the natural elements – water, trees, light, and flora – come together in a harmonious and visually appealing way. The air likely feels still and the sounds would probably be soft – perhaps the gentle splashing of the fountain or the rustling of leaves.
Wow! That’s a description that goes well beyond anything I might have written to describe my photograph! Lots of details and new insights that help us see more deeply into what’s in that image.
And here’s the next step – the image it produced when I asked it to create a new one based on the description above:
That’s a very different image but I can see how it got there!
This little experiment is worth playing with some more I think!