Episode 39

Can Rogue AI Agents Really Destroy Humanity? An AI Expert Explains

Key Takeaways

  • Fears of rogue AI are mostly hype: the examples so far are systems lacking proper protections, and people are still behind them.
  • Who is sounding the alarm matters: big AI companies may benefit from the publicity, and they keep building despite the warnings.
  • Data centers can be unplugged, have kill switches and need constant human upkeep, which limits how autonomous AI could be.
  • Thinking and acting independently is different from consciousness, which no one can define well even in humans.
  • Some data centers now include cultured human neurons in the processing, an area to watch.
  • A good first step with AI is using it for real day-to-day tasks, like synthesizing meeting transcripts and explaining medical reports in plain English.
  • Steer AI to short, simple answers, and compare tools: Claude, ChatGPT, Gemini and Perplexity each have strengths.

Main Topics

1HYPE VS RISK

Why an AI expert is not worried about rogue AI taking over, and what does deserve concern.

2CONSCIOUSNESS

What consciousness is, why it is hard to define and what biological components may add.

3PRACTICAL ROI

Everyday uses like meeting synthesis and medical record summaries that build comfort with AI.

Featured Guests

AB

A cognitive science expert who helps people develop private LLMs and writes about how AI works.

Arshavir Blackwell

Meet the Hosts

Elizabeth Gearhart
Elizabeth Gearhart, Ph.D.Chief Marketing Officer, Gearhart Law

Elizabeth Gearhart, Ph.D. is the host of Real AI Use Cases: Business Owners Roundtable, where she explores how businesses are actually putting AI to work. She is also CMO of Gearhart Law and co-host of the nationally syndicated Passage to Profit radio show and podcast.

Richard Gearhart
Richard Gearhart, Esq.Founding Partner, Gearhart Law

Richard Gearhart, Esq. is co-host of Real AI Use Cases: Business Owners Roundtable and founder of Gearhart Law, a full-service intellectual property law firm. A longtime podcaster, he hosts Intellectual Property News and is the host of the nationally syndicated Passage to Profit radio show and podcast.

Elizabeth and Richard compare ChatGPT, Claude, Copilot, Gemini and Perplexity and discuss AI answers that sound more authoritative than they are.

Episode Summary

Frequently Asked Questions

Is rogue AI likely to destroy humanity?
Arshavir Blackwell is not worried. He says the few examples labeled rogue AI are systems without proper protections, that people are still behind these systems, and that data centers can be unplugged and need constant human upkeep.
Why might AI companies warn about doomsday risks?
Arshavir Blackwell notes the warnings also generate publicity and investment interest, and that companies who truly believed in extinction would stop building rather than slow down slightly.
Can AI become conscious?
Arshavir Blackwell says independent action is possible, but consciousness is different and hard to define even in humans. He believes a conscious system would likely have a biological component mixed in.
What is a good way to start getting value from AI?
He recommends making an account and using AI for real daily tasks, such as synthesizing meeting transcripts or turning medical reports into plain English, beyond just replacing search.
How do you get shorter answers from AI?
Arshavir Blackwell suggests telling the AI to give a short answer in simple English, no more than a paragraph, which both Claude and ChatGPT will follow.
What problem do lawyers see with AI-written client emails?
Richard Gearhart says clients send long AI-generated emails that take time to answer, so the firm now asks clients which questions they truly need answered, since the firm bills for its time.

Episode Transcript

View Full Transcript

This is Real AI Use Cases: Business Owners Roundtable. I’m Elizabeth Gearhart.

And I’m Richard Gearhart.

And we’re your hosts. We’re here today with Arshavir Blackwell. He has been in AI practically his whole life, and in what was the precursor to AI. If you want to find out more about what he’s doing and see his amazing company, go to subscribe.insidetheblackbox.ai on Substack. Welcome, Arshavir.

Hi, I’m very glad to be here. Thank you.

You’re welcome. I want to dive right into a controversial subject, since you’re an AI expert. We’ve been asking people how they use AI in their business, and we could do that too, but last week there was news that rogue AI agents were going to destroy humanity. What do you think of that?

I’m not too worried about that. For one thing, we have a lot of other, much more pressing concerns in the world right now, things I’m way more worried about than artificial intelligence, and in fact I think AI may help find solutions or partial solutions to some of them. There’s a strong degree of hype and anti-hype when it comes to artificial intelligence, and there always has been, because it’s such a cool, science-fiction, Star Trek kind of technology that it lends itself to both predictions. On one side, we’ll all have so much free time that all we’ll do is play tennis all day while robots cook our meals, which isn’t happening yet. On the other side, it will be like The Terminator and take everything over and destroy it.

The few examples of what people are calling rogue AI are really just systems without the proper protections to stop them from going out and manipulating other computer systems. It’s true that you need guardrails for that. But I don’t think they’re going to be able to tap into NORAD, for example, and launch a missile, which is the image I see people hinting at when they talk about rogue AI or drones controlled by AI coming to bomb us. People could launch a whole drone attack on a population, and they do. But I’m not really concerned about AI doing it, and there are a lot of reasons why that’s just not a realistic scenario.

Let’s go into those reasons. I get what you’re saying. I do think there’s a lot of walking into the haunted house around Halloween going on, where people like to make up stuff and get everybody, including themselves, riled up. But it’s a new technology and I don’t know that anyone knows what the boundaries are. I’d be very interested in understanding why you feel the threat is not so real.

For one thing, you have to keep in mind who is saying this. A lot of the people saying it are the big corporations, and it’s not necessarily the case that they have some hidden insight into what these systems are capable of. It’s also a form of positive hype, because what comes across to people is, wow, this is incredible, powerful technology, I’d better invest in it.

I want to stop you there for a sec. I was talking to my coach this morning, and she was saying this has brought AI front and center to many, many more people.

Exactly. It’s like with publicity, the old saw, as long as they spell my name right. There’s no such thing as bad publicity. And there are other reasons. If Anthropic or OpenAI really believed this was going to end humanity, they would stop what they were doing right away. If you were working on a bomb that could destroy the entire planet, you wouldn’t keep working on it and say, but we’ll work on it a little slower.

They live here too, don’t they? They’re still human.

Exactly. I know the really wealthy ones have bunkers, but a lot of the people involved don’t. I think the way they’re reacting to their own news is a very strong clue that there’s a strong element of hype. That’s not to say one of these couldn’t get loose, but we’ve always had trouble with viruses and other kinds of rogue agents on the internet. I remember way back in the ‘80s there was a big internet virus that someone let loose, and it took a fair amount of effort to bring it back down. I don’t think a lot of what people are talking about is even that new in terms of computation. And in the background there’s always a person controlling these things. They’re not as magically autonomous as people seem to believe. Just go in there and use one for a while and you’ll see it’s an amazingly powerful tool that can do a lot, but sometimes it makes really stupid mistakes too. It’s like videos of humanoid robots that do all these incredible things and then start flipping out and falling to the floor and shaking. Those are the things I’m not particularly worried about taking over right now.

There was a movie that had exactly this scenario as its premise, not The Terminator, but an older movie called The Forbin Project, where a computer basically blackmails the entire world with nuclear weapons. But in that movie they gave the computer access to the nuclear weapons. We haven’t done anything like that, and I don’t think we’re likely to. There are still people.

You never know. It may happen.

It might, but if you look at the way these systems are designed and the way our nuclear arsenal is designed, there’s just no way. For one thing, you could unplug a data center. They all have kill switches and places where the power supply is routed through, and it would be trivial to turn those off. A friend of mine wrote an article about just this fact. He points out that the problem with the doom saying is that it doesn’t take into account how fragile these systems are. A strong burst of electromagnetic energy and they’re all fried. These are systems people spend an enormous amount of time keeping working. You always hear about uptime in these large server centers, and there’s staff who wake up at 2:00 in the morning because something’s down. If humans walked away and let them do what they wanted, they would not function on their own for very long, even if we did nothing to stop them. That’s why I’m not worried.

Do you think artificial intelligence will ever achieve consciousness? That’s really what we’re talking about here.

The singularity, they call it.

Or the singularity, so it can think and act independently.

I would argue that thinking and acting independently is possible, but it’s different from consciousness. I do have a degree in cognitive science, so I spent a lot of years in seminars talking about what consciousness is. The problem is that no one can satisfactorily define, even in human beings, what exactly consciousness is. If something is alert and responsive to you and seems to have a memory, that doesn’t make it, per se, conscious. It doesn’t make it something that has awareness, feelings and sensation, and knows what it is and what’s going on in the world. The current models are really just very, very complicated computers processing very, very complicated mathematics at lightning speed.

Can I ask a quick question? Is it true that it’s still binary code running everything?

Yes.

So keep going. Sorry.

You have very powerful systems running a lot of mathematical computations in parallel, and at the end of the day, when you see it respond to something you asked or a picture you gave it, that’s all that’s going on. The problem with consciousness is not that I think a synthetic system couldn’t have it. No one has a good definition or a good explanation for what consciousness even really is. If you want to get worried about something, and I’m not worried about it myself, but it’s a little on the sci-fi side: there is now a data center in Southeast Asia, I believe, where they have 18 million human neurons in a system that are part of the information processing. Those are biological systems. My belief is that if you do have a system that is, quote, conscious, and we still don’t really know how to define that, it will be something with some kind of biological component mixed in. It would have human or similar neurons, and on top of that maybe a synthetic computer component with millions and millions of chips and transistors. I think they culture the neurons. I don’t think they take a person’s brain.

What role do the neurons play in the system?

I don’t know much about that system, but I would assume it’s doing the same kinds of things the servers in the data centers are doing, which is training it so that when it gets some input it provides some output, exactly what the neural network paradigm is. By the way, to your question about binary, neurons are not really binary. Neurons have an analog quality, so those might actually be an example of systems that aren’t binary. I wish I knew more. I don’t know how much detail they’ve released about what task they’re working on or how they’re hooked up to the rest of the system, but I think it’s pretty remarkable.

It blows my mind that the AI we have today is all built off ones and zeros. In the very early days, I read, they would unplug it and plug it back in, so unplugged would be off and plugged in would be on, your zero and one. That’s amazing to me. I still can’t wrap my neurons around that.

I think it’s interesting that we’re kind of going full circle. We created these silicon chips to do the processing, and now we’re incorporating biological matter, with the hope that it can become quicker and more efficient than the chips.

I want to switch gears a little, Arshavir. This is supposed to be about how we’re using AI in business, but I wanted to address the elephant in the room, which was this fear-mongering terror, for maybe good reason, maybe not. I know you have a business where you help people develop their own private LLMs. What do you feel is the most powerful way to use the current AI we have right now in business to get a real ROI, a real financial benefit?

What you’re talking about, of course, are frontier models like ChatGPT and Claude and all the various incarnations of those that seem to come out with a new update every week. For someone just getting into using AI, which is most people, the first thing is getting familiar with how the system works. If you don’t have an account, make one. It’s 20 bucks a month. Go in and use it for as many day-to-day, real AI tasks as you can. By real, I mean a lot of people still use it just for search. Instead of typing a question into a search engine, which are becoming increasingly powered by AI as well, they type it into Claude or ChatGPT, and that’s really just the surface of what these systems can do.

For example, if you have a meeting you’ve recorded, I put all of my meeting transcripts into Claude, and it’s able to synthesize all of that information across all the different meetings. If I have a question about a topic that spans a bunch of meetings, or I have a meeting with somebody and I’m not quite clear on what they meant by something, I can ask the AI, and it will usually give a very good explanation of what’s going on. In these conversations we only have so much bandwidth to process what’s being said, and this allows you to go back over the meeting or call with a fresh set of eyes, so you’re not the only one figuring out the details. There are a lot of uses one can find. Every time I go to the doctor now, or get a medical report, I put it into Claude, and Claude now has a very in-depth record of all my medical visits for the past year or so, including all the tests. I can give it the tests they do and it will tell me in plain English what they mean, but it also correlates everything else in there. I’m not a doctor, so I wouldn’t know that something in one report on one day and something in another report on another day may have information when put together that I would never recognize, and that a doctor might never recognize because you’ve seen two different doctors who are very busy and can’t look through your entire medical history. Those are two really powerful examples of what I think people should start doing, not just for the value they add in your workflow and day-to-day life, but because they get you much more comfortable using these systems in other contexts as well.

Which one do you think they should start with? I think ChatGPT is probably the most popular.

That’s a very personal question. I used to use ChatGPT a lot more. I prefer Claude now. I can’t give you a cut-and-dried reason. In terms of my financial outlay, because it can get very expensive, the 20 dollars is a flat rate for a relatively small number of tokens. Most beginning users are not going to go above that, but someone using it all the time is going to spend a lot more on tokens. I think Claude is maybe slightly better value, though I’m sure there are people who would disagree. There are subtle differences between the two systems that are hard to put into words, and you have to try one and the other and see which gives you the best answers for the kind of work you’re doing. For instance, one problem I have with both of them is the really long, annoying answers. You ask, how do I do X, and a wall of text comes back and you think, I don’t want to read all this, I just wanted a simple answer. You can actually steer them by saying, give me a short answer in simple English, don’t take more than a paragraph, and put it in easy-to-understand language. It will do that, so with both systems you can mold them.

Richard, what is your favorite one right now? Are you using different ones for different things?

I use Copilot because it’s within the Microsoft ecosystem and confidentiality is important. I don’t think the results are as good as ChatGPT or Claude. I use ChatGPT for anything that isn’t confidential or client-related, mostly because I’ve been using it for a long time and it seems to know me well. I’ve been told that if you want to switch LLMs, you ask the new one to create a prompt that you put into the old one to export everything it knows about you. If I wanted to switch to Claude, I’d ask Claude to create a prompt, ChatGPT would answer, and I could copy and paste the information Claude thinks is most important into Claude and start using it. I haven’t done that yet.

I’m using four of them right now. I hadn’t been using Perplexity for a while, but it’s gotten kind of cool because it has agents that go out and do your queries. Richard gave me an intense query, I put it in Perplexity, and it gave me the spreadsheet he wanted. ChatGPT couldn’t do it. I love ChatGPT for images right now, it’s doing great images.

I think it’s very good for images. That’s true.

And I love Gemini, which gives you the most tokens, for multimodal work. We do a lot with video, obviously, and Gemini is great for video. When we’re analyzing our YouTube stats, I always ask Gemini, because Google owns YouTube and I figure I’ll get the best answer, though I do sometimes spot-check it. So I use ChatGPT, Gemini, Perplexity and Claude. The way my brain works and the kinds of things I’m doing, I don’t really get great answers from Claude. Maybe I’m not giving it the right prompts. For trying to write content, I think ChatGPT is friendlier. For facts especially I use all of them, because I think Claude is a lot more factual and doesn’t hallucinate as much as the others, maybe.

Hard to know.

Hard to know, which is why you have to check them all against each other.

Speaking of hallucinations, and relying on AI for medical information, I do think it’s a great tool for that. I do it with ChatGPT. I go to a doctor and put my test results in, and it gives me very credible-sounding answers to my questions. As an intellectual property attorney, we have clients now using LLMs to write us emails asking questions about certain topics. The challenge is you may get a five-page email that takes half an hour to respond to, and you’re not sure the client needs every single detail the LLM is asking for. In the legal profession we have to charge for our time, which is how we pay our bills. So we’ve gotten to the point where we ask clients, here’s a couple of lines we think you need to know about, and if you want us to answer the rest of the questions, we’re happy to, but we have to bill you for the time. The point I’m trying to make is that the LLMs sound so authoritative that it creates the impression they’re right, and sometimes their answers aren’t appropriate for the situation, which is what you were saying about the long answers.

Keep exploring the show

Hear how more business owners are putting AI to work in their day-to-day.