Skepticism. Ignorance. User-Understanding
The all knowing super-bot
Should you haven’t heard of ChatGPT yet, you have to have been living under a rock. It’s a groundbreaking development in generative AI that has already turn out to be a go-to tool for content generation, copy-editing, and search assistance. Nevertheless, let me bask in one among my pet peeves for a moment.
In the primary scenario, I asked ChatGPT to clarify quantum mechanics to me using the primary principles of linear algebra, something well documented. It did a good job with none instruction tuning.
Now, let’s have some fun by asking something area of interest or absurd. On this case something absurd. “Explain quantum mechanics to me using first principles of political science?” The response is attached below on your examination and delight.
I realize it well, that I don’t realize it yet
Renaissance. The Renaissance was a magical time in human history. It marked a fundamental shift in the tactic of accumulating knowledge, moving away from societal seniority-based opinions to testable hypothesis-based rigor, and from unquestioning faith to the perfect working hypothesis concerning the world.
Probably the greatest features of this technique of gaining knowledge is that it doesn’t should be complete. It’s an ongoing process, with each recent understanding being our “best working hypothesis” until it’s replaced by a recent one with theoretical and experimental rigor. For instance, Bohr’s model was once our greatest understanding of the atom, but our understanding has since evolved.
In each iteration of this process, the important thing to gaining more knowledge is acknowledging what we don’t know yet. “I don’t know” is a phrase used steadily by a number of the smartest people I do know, perhaps because they push themselves to the sting of their understanding, all that more often. By following it up with “I don’t know what/how/why yet, but these are the things we will look into…,” we lay the inspiration for clear considering and the pursuit of greater knowledge.
Expressing ignorance
The bayesian loop (Part 1 and a couple of) helps you express your ignorance just as well. In essence, the loop may be expressed as follows within the log space:
while (our knowledge has not converged):
Knowledge (t) = Incoming evidence + Knowledge (t-1)
Now, let’s consider two scenarios. In the primary scenario, we’re extremely confident in our prior knowledge. On this case, we might want incoming evidence to vary our prior knowledge at a slower pace and only accept it completely when we now have seen overwhelming evidence over a protracted time frame. The above approach (and a few popular ones based on the identical idea) allow us to upweight our prior knowledge — slowing down our update speed. This approach allows us to be conservative with change.
Within the second scenario, we’re unsure about our knowledge of the world. On this case, we might want incoming evidence to vary our prior knowledge at a really fast pace. We go where the info takes us until it settles down. This approach allows us to be more open to vary and to adapt more quickly as recent information becomes available.
A Easy User-Understanding Puzzle
In most mature user-facing systems, there are typically two varieties of users: power users and cold-start users.
- Those that enjoy using the system, know their way around it, and profit from the system’s functionality.
- Those that are still learning find out how to use the system, struggle to navigate it, and don’t yet receive the complete advantages of the system’s features.
If we now have a machine learning algorithm that outputs the topics and themes a user is keen on, should we weigh this data in another way based on our level of uncertainty concerning the user? Indeed.
For power users, a couple of random decisions in a given day are unlikely to vary our fundamental understanding of their preferences that we now have amassed over multiple days. Subsequently, we should always update our understanding of their preferences more slowly.
Then again, for cold-start users, we’re less certain about their preferences and might have to take larger leaps of religion to converge towards a more accurate understanding.
