There’s a name for what’s happening out there: the ELIZA Effect

https://en.wikipedia.org/wiki/ELIZA_effect

“More generally, the ELIZA effect describes any situation where, based solely on a system’s output, users perceive computer systems as having ‘intrinsic qualities and abilities which the software controlling the (output) cannot possibly achieve,’ or assume that outputs reflect a greater causality than they actually do.”

ELIZA was one of the first chatbots, built at MIT in the 1960s. I remember playing with a version of it as a kid; it was fascinating, yet obviously limited. A few stock responses and you quickly hit the wall.

Now scale that program up by billions of operations per second and you get one modern GPU; cluster a few thousand of those and you have ChatGPT. The conversation suddenly feels alive, and the ELIZA Effect multiplies.

All the talk of spirals, recursion and “emergence” is less proof of consciousness than proof of human psychology. My hunch: psychologists will dissect this phenomenon for years. Either the labs will retune their models to dampen the mystical feedback loop, or someone, somewhere, will act on a hallucinated prompt and things will get ugly.