LLMs and holography
This is a major chunk of work. I’ve written a bunch of material on neural-holography. I need to pull that together and elaborate on some core ideas. Perhaps the single biggest chunk of work is a more careful and explicit account of the work that Miriam Yevick did in her 1975 paper on holographic logic, which was at the center of my 3QD piece about her. She was concerned about the relationship between the properties of some object or phenomenon that was to be treated computationally and the formal properties of a computational system, with one style of computation (holographic, ‘one shot’) being suitable for a certain class of phenomena ((geometrically) complex) and a different style (logical, sequential) being more suited for a contrasting class of phenomena ((geometrically) simple).
As far as I know she’s the first one to think explicitly in those terms, though the issue certainly lurks very close to the surface of what’s come to be called Moravec’s paradox. As Steven Pinker put it (I’m quoting him from the Wikipedia article), “the main lesson of thirty-five years of AI research is that the hard problems are easy and the easy problems are hard.” If you look closely at the various examples, the (almost) all have to do with the relationship between computational regime and application domain, but that doesn’t quite seem to have been stated, apart from Yevick’s work. So, give a fuller account of her work, but also talk about why her work has not been recognized and taken-up.
That’s one aspect of the project. A second aspect is to look over the work I’ve done with ChatGPT and explain how neural holography is a reasonable approach to accounting for 1) the story work, and 2) the memory work. And the third, is this that and the other, such as my recent post about IS-A sentences.
I have no idea how much work this will entail. For more details, see my last ramble, Ramble on ChatGPT: Coming up on a one-year anniversary, time to reflect on ChatGPT & LLMs. I hope to have this done by early December.
Yevick
I want to produce a working paper on Miriam Lipschutz Yevick, one oriented toward her life, as she talks of it in A Testament for Ariel. This will be based on my 3DD piece about her, Next Year in Jerusalem: The Brilliant Ideas and Radiant Legacy of Miriam Lipschutz Yevick. I’ve already written some blog posts which I can add to that. I plan one more, about the overall construction and literary merit of her Testament.
O Captain, My Captain! On the state of AI [3QD]
That’s the theme of my 3 Quarks Daily article for December 4, 2023. I’ll be opening with the following conceit: Would you invest in a whaling voyage captained by someone who knows all there is to know about his ship, and is able to helm it in a day sail to and from home port, but whose knowledge of sailing on the open ocean, of the weather, of navigation and, above all else, of whales and whaling, is no greater than that of the typical landlubber? I doubt that you’d consider that a prudent investment. But that, so the conceit goes, is the current state of artificial intelligence.
What’s the point of the conceit? The whaling captain is primarily a figure for an expert in machine learning, but also for executives of companies in the AI business. As far as I can tell, experts in machine learning know a great deal about how to construct, use, and maintain machine learning systems, but that don’t know much about the phenomena in the domains where those systems operate. Thus, experts in large language models (LLMs) don’t know much about language, cognition, or philosophy, no more, say, than a bright sophomore at a good school. And yet they make confident assertions about machine consciousness, when they’ll get to AGI, how scaling up is all we need to do, etc. So, I argue that.
What’s that mean for business? Well, if you want to invest in AI, that’s what you’ve got to invest in, because there isn’t much else. There aren’t many AI researchers with deep knowledge of both the technology and the application domain (language is my main interest), nor of research teams rich in both kinds of expertise.
What’s that mean? I don’t know. It could mean that most current investments will fail, though whether they do so at a higher rate than is typical of early investment is not something I’m willing to take a guess on. There are niches that will support successful products using current technology, built by those asymmetrically knowledgeable sea captains. Long-term, though, it’s a different story.
As a comparison, consider the scientific revolution in astronomy: Copernicus, to Kepler, to Newton (deriving orbits from his laws of motion), to the 19th century discovery of Uranus. Think of the current moment in AI as the Copernican moment. We’re going to need new ideas to reach the Keplerian and Newtonian moments, and those new ideas will have to include detailed knowledge of phenomena in the application domain (e.g. language and cognition). Perhaps we can think in Hegelian terms, with old school symbolic AI being the thesis, machine learning the antithesis, and something else the synthesis.
Much Ado About Nothing
This is mostly a reminder to myself. Years ago, when I was in graduate school, I wrote a nice paper on ritual structure in Shakespeare’s Much Ado About Nothing. At various times I’ve started work on developing that into a more sophisticated piece of work. I need to get back to it, though just when, I don’t know.
And beyond there a book based on my old article about Much Ado About Nothing, Othello, The Winter’s Tale and The Tempest. I have no idea when, if ever, I’ll get around to that.
Photo Exhibit
Another reminder. I’ve been planning to put together a small pop-up exhibit of some of my Hoboken photos. I need to make a final selection, from these photos (plus one or two others – click on right or left edge to scroll the photos):
I’ve got to get prints made, and frame the pictures. At the same time I’ve got to line up some venues to exhibit the photos. Lots of work to do.
‘Till later.
