Are we experiencing the Gell-Mann Amnesia Effect with a chatbot?

Michael Crichton noticed something strange about reading newspapers. You open the paper, read an article on a subject you know deeply, and see that it is full of errors. You turn the page, read the next article on a subject you do not know, and trust it completely.

Crichton named the effect “Gell-Mann Amnesia” after his friend Murray Gell-Mann, the Nobel physicist who theorized quarks. 

Somehow you forgot what you just learned one page ago.

Are we doing the same thing with AI, at industrial scale?

You ask it a strategy question in a domain you know, and notice that the answer is generic. Then you move on to the next question, in a domain you do not know as well, and are tempted to trust it completely.

Recently researchers tested seven leading models across seven classic strategic trade-offs. Across thousands of simulations, the models gave very similar answers: some version of differentiate, augment, or collaborate. 

Answers were very similar for a tech startup, a construction firm, a rural hospital, a craft brewery. 

They call it “trendslop.”

Better prompting barely moved the bias, and even rich industry context only shifted it somewhat. In some cases, simply changing how the options were phrased or ordered nudged the model to a different side of the trade-off.

None of this should be surprising. The models are giving you the average sentiment of the internet, dressed in the vocabulary of management.

The risk is accepting a recommendation without pressure-testing it because it sounds polished.

Of course, this does not mean the tool is useless. It just means the tool is an excellent first draft but may be a terrible final answer. Use it to widen the option set, to steelman the case you would not make, to surface the assumption you are not seeing. 

Ask it to run the counterfactual: what would have to be true for the opposite answer to be right?

Importantly, keep the decision yours.

As Michael Porter put it, “The essence of strategy is choosing what not to do.” The machine cannot do that for you.

The machine averages; you choose. 

Do not confuse the two.

Insight from Harvard Business Review: https://lnkd.in/gbzMpG6m

Until next time,

Ram

— 

Ram Srinivasan

MIT Alum | Author, The Conscious Machine | Global Future of Work and AI Adoption Leader published in Business Insider, Fortune, Harvard Business Review, MIT Executive Viewpoints and more.

—

A Message From Ram:

My mission is to illuminate the path toward humanity's exponential future. If you're a leader, innovator, or changemaker passionate about leveraging breakthrough technologies to create unprecedented positive impact, you're in the right place. If you know others who share this vision, please share these insights. Together, we can accelerate the trajectory of human progress.

Disclaimer:

Ram Srinivasan currently serves as an Innovation Strategist and Transformation Leader, authoring groundbreaking works including "The Conscious Machine" and the upcoming "The Exponential Human."

All views expressed on "Substrate" and across all digital channels and social media platforms are strictly personal opinions and do not represent the official positions of any organizations or entities I am affiliated with, past or present. The content shared is for informational and inspirational purposes only. These perspectives are my own and should not be construed as professional, legal, financial, technical, or strategic advice. Any decisions made based on this information are solely the responsibility of the reader.

While I strive to ensure accuracy and timeliness in all communications, the rapid pace of technological change means that some information may become outdated. I encourage readers to conduct their own due diligence and seek appropriate professional advice for their specific circumstances.