My new piece at 3 Quarks Daily is up:
My two “laws” are relevant here:
Benzon’s 1st Law: A state of the art LLM is useful in proportion to the user's intelligence, knowledge, & creativity.
Benzon’s 2nd Law: If you want to know what it's like to drive a car, get in the driver's seat and go.
The first one may not be obvious, not if you think of LLMs as purely mechanical objects where you just push a button and it’s off to the races. No, they are complex and quirky. You have to mess around with them in order to get them to work for you.
By now I’ve logged hundreds of hours with LLMs, mostly with ChatGPT, but more recently with Claude 3.5, which is now my go-to bot of choice. Many of these hours involved careful and systematic exploration and probing. Others involved just casual look-ups, such as: “Tell me about the Johnstown flood.” And still others involved insane nonsense jamming, just seeing if I could surprise the LLM, and if it could then, in turn, surprise me. Sometimes those had me laughing myself silly. The Green Giant Chronicles is a good example of this; so is AGI and Beyond: A Whale of a Tale. I’d say that the best way to get to know an LLM is to figure out how to get it to make you laugh.
As for Benzon’s Second Law, that’s obvious: Just do it!
