Regression to normal
But not to your normal. Would you have recognized the iconic image of the derailment in Montparnasse in this picture?
(all info about the experiment here https://apps.byatw.com/lab/i2t2i/)
I ran an experiment today in which I handed an image to an AI tool with a sophisticated prompt on how to describe images.
Then, I handed that description again to an AI tool to generate images.
My initial idea was to create a movie of the transformation, which led me to asking the loop to run around 100 times. On the first batch with Mona Lisa I noticed however that the majority of images looked the same.
After around 20 iterations, Mona Lisa was a modern Mona Lisa - and after a couple more iterations, I had a portrait of a woman with black hair looking straight into the camera - not such a big surprise.
Funnier happened for the Milkmaid, famous painting by Vermeer.
At one point the flowing milk and the face took over and then suddenly the lips became the focus:
And finally the Montparnasse image:
What we're observing here is not that the tool would create an inaccurate description of an image.
The first description of the picture is good:
Subject: A large steam locomotive has pierced and hangs out of a masonry building façade, its boiler and wheels angled downward toward the street.
And the result is not too off, neither:
But now, the narrator makes a choice - it's not a locomotive anymore:
Subject first: A large cylindrical steam locomotive-like engine dominates the image, lodged nose-first into the front of a multi-storey stone building.
And the "like" opens room for interpretation.
The next description abstracts further:
The largest, most immediately attention-grabbing element is a large, smooth cylindrical tank or capsule occupying the right two-thirds of the image, leaning diagonally from the upper right toward the lower center.
And from here on, things start to become very, very abstract but still recognizable if you know what was meant initially - but only for you, not for the AI.
Beyond "oh that's funny"
A human can absorb millions of signals per second with all senses - way too much information to make any .. sense. But we can still recognize up to 50 objects per second visually. When read written information, we read between 28-45 informations per minute(!), alas less than one per second (you read more words, but it is about the information you extract). Spoken language is even worse:
Human behaviour (typing, speaking, playing games, memorising) runs at about 10 bits per second, while our senses take in around a billion bits per second. They call this gap an open puzzle (source).
The "billion - 10" bits per second are however not lost, they trigger reflexes, feelings, give you information before you "know" about it and so on.
When it comes to describing the world around us, we are however limited to the information we gathered with our 10bit/s conscious capacity. We see something, we make sense out of it and then we can communicate about it in our language. Which is again a reduction in complexity, a compromise.
When it comes to AI, these reduced descriptions of our phyiscal world are used to train AI models. And even if they can to cool stuff, it's still based on our observations - and even if a tool would make their "own" observations, they could only do so with the words for things that we have taught them.
Then, this AI takes an image and matches it agains other images which were named by humans - and apparently, there's little content about a locomotive sticking out of a building but much more about cylindrical shapes. And the content which has been fed into the tool is the normal of the tool.
We have a hard time believing things which we don't perceive as "normal", too, but we can. The AI tool creating the description can not go beyond what we've taught it, it is literally lacking the words.
Giving this reduced content back to an AI amplifies the effect and leads to a regression to normal.
You find the experiments here: https://apps.byatw.com/lab/i2t2i/