Interest in content understanding and language models is usually rooted in a real love for stories, narrative styles and linguistic nuances. A couple of years ago, I discovered myself engaged in the duty of compiling a listing of lesser-discussed difficult problems inside the field of content understanding. That’s after I stumbled upon the marginally ignored problem of finding long-range semantic relations. Ignored because there is usually little incentive to work on something, you can’t show much incremental progress on.
Consider the straightforward syntactic linguistic relationship inside the sentence “Obama was born in Hawaii.” It accommodates a straightforward relation of the type “is from” between “Obama” and “Hawaii.” Contemporary AI models would adeptly detect these concise short range relations.
Nevertheless, the problem lies in capturing more complex, long-range relations. Consider a scenario where an individual is mentioned using contextual clues several paragraphs later, corresponding to within the sentence, “The final temperature in the course of the 2008 economic crisis was indeed contrary to the weather by which the Hawaii-born lad had grown up in.” These nuanced connections pose a greater detection challenge, the more intricate the relation becomes, the harder it’s for current models to uncover.
That’s when, a friend with a background in linguistics unveiled a revelation, urging me to ponder the very limits of this conundrum. What if these relations are implicit yet unmistakably present? Consider novels with parallel tracks or themes, where the connections between different storylines only change into apparent over time.
Have you ever watched the “Forrest Gump”? Beyond its status as a memorable tearjerker, one might query whether the story primarily revolves across the protagonist’s trials and adventures or serves as a sardonic commentary on the backdrop of historical events. I shall let you select that for yourself.
Salman Rushdie’s “Midnight’s Children” offers one other compelling example. Superficially, it presents a tale of magic-realism, centered around a person with a highly sensitive nose. Nevertheless, beneath this facade, lies a private critique of Indian politics, fifty years ago, delving into the nuances of the nation’s past and its socio-economic landscape.
This led me to: “The Brothers Karamazov.” In my biased personal opinion, it’s the best work of this genre. I like this book so very much, that I even have penned down a separate commentary on it, which you might check here.
In essence, research and development in content understanding, have a protracted solution to traverse, surpassing mere correlations and delving into nuanced understanding, unraveling causality, and cultivating a modicum of reflection, before mustering the power to understand, let alone pen one other masterpiece just like the “Brothers Karamazov.” Indeed, current day generative models corresponding to ChatGPT excel in some amount of copy-editing, text completion, search assistance, and facilitating categorization.
Nevertheless, deeming them because the death of all creativity seems excessive.
