AI models are all the time surprising us, not only in what they will do, but in addition in what they will’t, and why. An interesting recent behavior is each superficial and revealing about these systems: They pick random numbers as in the event that they’re human beings, which is to say, badly.
But first, what does that even mean? Can’t people pick numbers randomly? And how will you tell if someone is doing so successfully or not? This is definitely a really old and well-known limitation that we humans have: We overthink and misunderstand randomness.
Tell an individual to predict 100 coin flips, and compare that to 100 actual coin flips — you’ll be able to almost all the time tell them apart because, counterintuitively, the true coin flips less random. There’ll often be, for instance, six or seven heads or tails in a row, something almost no human predictor includes of their 100.
It’s the identical if you ask someone to select a number between 0 and 100. People almost never pick 1 or 100. Multiples of 5 are rare, as are numbers with repeating digits like 66 and 99. These don’t look like “random” selections to us, because they embody some quality: small, big, distinctive. As an alternative, we frequently pick numbers ending in 7, generally from the center somewhere.
There are countless examples of this sort of predictability in psychology. But that doesn’t make it any less weird when AIs do the identical thing.
Yes, some curious engineers over at Gramener performed an off-the-cuff but nevertheless fascinating experiment where they simply asked several major LLM chatbots to select a random number between 0 and 100.
Reader, the outcomes were random.
All three models tested had a “favorite” number that may all the time be their answer when placed on probably the most deterministic mode but that appeared most frequently even at higher “temperatures,” a setting models often have that increases the variability of their results.
OpenAI’s GPT-3.5 Turbo really likes 47. Previously, it liked 42 — a number made famous, in fact, by Douglas Adams in “The Hitchhiker’s Guide to the Galaxy”as the reply to life, the universe, and every thing.
Anthropic’s Claude 3 Haiku went with 42. And Gemini likes 72.
More interestingly, all three models demonstrated human-like bias in the opposite numbers they chose, even at extreme temperature.
All tended to avoid high and low numbers; Claude never went above 87 or below 27, and even those were outliers. Double digits were scrupulously avoided: no 33s, 55s, or 66s, but 77 showed up (ends in 7). Almost no round numbers — though Gemini once, at the very best temperature, went wild and picked 0.
Why should this be? AIs aren’t human! Why would they care what “seems” random? Have they finally achieved consciousness and that is how they show it?!
No. The reply, as is often the case with this stuff, is that we’re anthropomorphizing a step too far. These models don’t care about what’s and isn’t random. They don’t know what “randomness” is! They answer this query the identical way they answer all the remainder: by taking a look at their training data and repeating what was most frequently written after a matter that looked like “pick a random number.” The more often it appears, the more often the model repeats it.
Where of their training data would they see 100, if almost nobody ever responds that way? For all of the AI model knows, 100 will not be a suitable answer to that query. With no actual reasoning capability, and no understanding of numbers in anyway, it will possibly only answer just like the stochastic parrot it’s. (Similarly, they’ve tended to fail at easy arithmetic, like multiplying a number of numbers together; in spite of everything, how likely is it that the phrase “112*894*32=3,204,096” would seem somewhere of their training data? Though newer models will recognize that a math problem is present and kick it to a subroutine.)
It’s an object lesson in large language model (LLM) habits and the humanity they will appear to point out. In every interaction with these systems, one must keep in mind that they’ve been trained to act the best way people do, even when that was not the intent. That’s why pseudanthropy is so difficult to avoid or prevent.
I wrote within the headline that these models “think they’re people,” but that’s a bit misleading. As we frequently have occasion to indicate, they don’t in any respect. But of their responses, in any respect times, they imitating people, with none have to know or think in any respect. Whether you’re asking it for a chickpea salad recipe, investment advice, or a random number, the method is similar. The outcomes feel human because they’re human, drawn directly from human-produced content and remixed — to your convenience and, in fact, for large AI’s bottom line.
