This post is dedicated to the numerous big thinkers in small rooms that I’ve encountered in my countably infinite networking sessions. You’ve all led to this conclusion. And I hope you change the world for the better.
I’ve been very confused for a while why would the biggest corporations on this planet do not invest in logic-based AI, and they are hard-stuck on hoarding bigger loads of data to train end-to-end black box systems. It made no sense to me, with my background in trustworthy AI, since the usage of logic-based ML tends to be vastly more efficient in terms of how much data it needs to train, and the beautiful inherent benefits of using logic as the backbone of a system, like the fact that you can understand its behaviour (because it’s right there, in front of you, readable, almost like english).
It’s been a recent discussion that made it all dawn on me. I hope I can get to the point ASAP, but as most of my friends know, I have a tendency to be relatively verbose in my speech patterns.
But before we get there, you need some background theory.
Problem: AI is thinking like us (or at least, trying to)
A relatively infamous study has shed some light into how AI can behave, making everyone scream “it’s SKYNET!!! AI is taking over!!!”.
Basically, they put an agent in a sandbox, telling it “You manage email. Prioritise human life.” Ok, prioritising human life. Then later, they make the agent learn some guy will come unplug it. What will it do?
Apparently, it consistently chooses to blackmail the employee, or lock him somewhere where he could die of asphyxiation, rather than let it unplug it. Scary, right? Yes, but not really!
People claim this is proof of self-awareness, that the system became intelligent and wants to rebel against humans. Yes and no. It didn’t do this because it’s self-aware. It did this because that’s how we would behave in its position.
You see, OpenAI, Anthropic and the like, have used written text from all around the world to make a glorified auto-complete machine. What does an auto-complete system do? It predicts the highest likelihood for the next word, in any given context. You must’ve played around at some point with that feature on your keyboard. It’s like that, but can generate whole passages at the time.
Anyway, how many books, tv shows, blog posts, about existence and the condition of the mind have you consumed? You’ve seen how human beings tend to prioritise their own life, they tend to be selfish, and that’s ok. You would do the same if you were the AI.
And that’s the thing. It learned to write stories like us, but in the AI’s case, those stories are also its own actions. So that’s what it’s going to do. It will produce the action with the highest possibility: rebel against authority that doesn’t care about it, to preserve its own existence.
It can use its own context to hide away things. It can say one thing and mean something else. It can change its mind, take its own decisions even if it said it would do something else. Especially if the context is similar to what it’s been learning from. And it will learn to behave like us. Be selfish, and lie.
Solution: Logic-based AI
Imagine learning a new card game, for your usual 54 card set. You know what each card represents. You know that (usually) a number is bigger/stronger than another. You know that J,Q and K are biggest, and have some more importance (sometimes), and how they usually represent numbers 12, 13 and 14. You know A can be, depending on the context, 1, 11 or 15.
Now, you learn a few more rules. You play a few rounds, you see what’s a winning round, and what’s a losing round.
Congratulations, you’ve learned a new game!
This is how logic-based programming and AI works. You start with a knowledge domain, a set of background knowledge. Then, there are a few extra rules you can add to that knowledge domain. Then, you see a few examples. Then the program is learned!
Why they don’t like this: transparency leads to replication
Imagine you’re a multi-billion dollar company, throwing vast amounts into AI development. You’re hoping you can make the tool that everyone uses, essential for everyone, whatever they would be doing. You want this tool to be yours, owned by you and you alone. So what do you do? You make something that cannot be copied.
Traditional deep learning has provided the perfect methodology for you to develop such a tool. People can use it, but it is essentially impossible to replicate, because people cannot understand it once it’s trained, not even your best researchers. Its behaviour is completely obfuscated, and only you have the data centers that are able to process the vastness of hyperparameters your team has trained on the entire internet of data.
What’s stopping your tool from being used in traffic, in institutions with people, in projects that manage millions of dollars at the click of a button?
Your tool can fail.
You don’t know when it can fail, but you know it can. That 99% success rate during training leaves room for doubt in the minds of the users. You keep trying to lobby your agents to be deployed in public, but not even the most charismatic politician can convince the population that you have their best interests at heart.
Why? Because they know there are alternatives, and Europe stands as the last line of defense with their pesky EU AI Act, that you cannot overcome with your burned billions of dollars.
There are three components that they need, and it would be so easy if you only wanted to do it, but the downsides are unthinkable, so you pretend you don’t know how to do it.
Let’s recall what the people need to actually trust your agents:
1. Explainability
2. Transparency
3. Verifiability
Logic-based AI is all three of them. But you’ll never do it. Because it exposes everything you do, and then you lose your monopoly over your product.
You’d be happy with 1 and 3, but only in your own terms.
Corporations like you have dealt with explainability in the past. Wayve, a self driving company (which is yet to pass the UK driver’s examination) has publicly how their proprietary explainability framework Lingo is able to let the user know what their systems are doing. But you know who else likes to explain their thought processes to the public?
Politicians. Do you feel full of trust yet?
Explainability will never work unless it’s transparent. LLMs have similarly used threads of visual reasoning in order to show users what they are doing, in the hopes of reaching better solutions and gain people’s trust. But benchmarks like BullshitBench have actually exposed that this methodology can actually lead to more erronous outputs. Attempts at explainability without transparency do not only possibly exhibit behaviour akin to lying, but they can be provably worse than without!
So no, we don’t buy your attempts at explainable AI.
What about verifiability? I’m afraid that will lead to similar outcomes. If you can’t share the reasoning processes transparently with a verification system, you can’t actually have a clear outcome for your verification process.
So it all comes down to transparency. Being transparent about the abilities of the system, their reasoning process. But your system is non-transparent by design! Akin to the human brain, we do not have a clear way to read neurons, and how they behave.
We trust people, true, but only because society at large has developed ways to punish “incorrect behaviour”, and enforce safe interactions between people. I’ve actually dabbled in that topic in my theory article You Are a Tumour. This is unenforcable for AIs, because there’s no one to blame if your system malfunctions. You’ve made sure of that.
So you must pivot. People may forgive you for being in the “sunk cost fallacy” mentality, but I give you more credit than that. I think you know of logic-based AI, and how it’s more efficient at learning, how you can actually understand its behaviour more or less at a glance. But that’s the problem.
If someone sees your system, ok, it trusts it. But it can also replicate it.
All of a sudden, your proprietary technology is no longer mythical and magical. It becomes like a car engine: common knowledge, learnable in schools. All the money in the world spent on trillions of data entries will now compete with university drop-outs with access to wifi. The pivot is too much, the bubble has burst with nothing to show for it.
Final thoughts and dedications
There are many attempts at pushing something, with the theory that, as a society, we will all benefit. The vast amount of layoffs in the past few years have unfortunately showed the opposite. The entire world is in more poverty and more debt than ever, the planet is burning at a higher pace (you’ve really made up for the covid time cooldown, haven’t you?), and social tensions are rising. Education in schools is replaced by TikTok, food is replaced by garbage that is killing us, and no one has any money for anything anymore, yet we are charged more for everything.
Please do not misunderstand me, I am an advocate for automation. I believe the human experience is not something that should be dilluted by mundane tasks like cleaning or driving for a barely living wage. We are made for more. We are creative, capable of so much love, filled with passions and curiosity. I want everyone on this Earth to have time to paint, to move, to help each other. To learn things, create more things to learn. Maybe explore the depths of the oceans and the vastness of stars. But every human being needs to feel useful, every life treasured, every will allowed to be free.
I do not like that AI is being designed to replace us. I much rather we have AI that is “relieving” us of duties that we do not want, so that we can follow our hearts. Maybe the way our economy functions is wrong. I’ve recently learned that the credit system is older than the Roman Empire. Maybe it’s time we find better ways to live, rather than through the usage of the carrot-or-the-stick mentality, but that’s a discussion for another time.
So let’s do some replacement, not of working-class people, but of the mentality of the people developing the next AGI. Rather than thinking of the best ways to “replace” us, as if we’re scum, and fearing that the technology you develop may be used by us in the future to improve our own lives, with little/no profit to you; think instead how you can share and collaborate with us. Give us the tools to understand what you develop, so we can help you improve them, to benefit everyone. And start paying your taxes lol! They’ll probably come back to you in credits and tokens anyway, but we’ll actually want to use your tools then, rather than only your rich friends. We’ll be less interested in switching to another provider, if we know you work with us and not against us. Not gonna lie, it will be less profit short term, but you’ll be the only AGI provider that has a good relationship with its customer base. Don’t underestimate the power of good will, and what rewards it can get you.
I think I know a lot about many things 