Episode 40

How to Make AI Write in Your Voice, Protect Privacy and Cut Costs

Key Takeaways

  • Frontier models do not change as a result of your prompts, so asking them to write in your voice often sounds inauthentic.
  • Local models can be trained on your own writing, so they can rewrite documents in your voice.
  • Running a local model keeps confidential information off outside servers, helping with privacy and HIPAA concerns.
  • Local models avoid token bills: you pay only for electricity, while heavy use of a frontier model can approach 1,000 dollars a month.
  • Today’s AI prices are heavily subsidized by investors, and more powerful models may eventually cost far more.
  • Data center energy and cooling are real concerns, but AI may help solve them, and the human brain runs on about 60 watts.
  • Over the next year expect evolution rather than miracles, though AI systems designing other AI systems could bring sudden jumps.

Main Topics

1YOUR VOICE

How local models trained on your own writing can sound like you.

2PRIVACY AND COST

Why local models avoid confidentiality risks and recurring token bills.

3WHAT COMES NEXT

Data centers, energy use and the pace of AI progress.

Featured Guests

AB

Develops local, private language models trained on a person’s own writing.

Arshavir Blackwell

Meet the Hosts

Elizabeth Gearhart
Elizabeth Gearhart, Ph.D.Chief Marketing Officer, Gearhart Law

Elizabeth Gearhart, Ph.D. is the host of Real AI Use Cases: Business Owners Roundtable, where she explores how businesses are actually putting AI to work. She is also CMO of Gearhart Law and co-host of the nationally syndicated Passage to Profit radio show and podcast.

Richard Gearhart
Richard Gearhart, Esq.Founding Partner, Gearhart Law

Richard Gearhart, Esq. is co-host of Real AI Use Cases: Business Owners Roundtable and founder of Gearhart Law, a full-service intellectual property law firm. A longtime podcaster, he hosts Intellectual Property News and is the host of the nationally syndicated Passage to Profit radio show and podcast.

Elizabeth and Richard ask about pricing, data center cooling, space-based computing and what AI predictions science fiction got right.

Episode Summary

Frequently Asked Questions

Why can’t ChatGPT or Claude write in my exact voice?
Arshavir Blackwell explains that frontier models’ weights do not change from your prompts, so asking for a voice or tone often comes across as inauthentic, like a computer imitating an accent.
How does a local model write in your voice?
A local model can be trained, so you can change its weights. It reads many examples of your writing and is trained on them, so when you ask it to rewrite something it uses your voice.
What are the privacy and cost benefits of local models?
You are not sending confidential information to an outside server, which helps with confidentiality and HIPAA concerns, and you pay no token fees, only for electricity.
Is AI pricing going to rise?
Arshavir Blackwell says the current prices are heavily subsidized by investors, and he has heard talk of very sophisticated models being priced at 1,000 or 2,000 dollars a month.
Can AI solve the data center energy problem?
Arshavir Blackwell suspects people are using AI to work on cooling and efficiency, mentions ideas like Arctic or space data centers, and notes that the human brain runs on about 60 watts, suggesting efficiency is achievable.
What will AI look like in the next year?
He expects an evolution of what we see now rather than miracles, though AI systems that design other AI systems could cause sudden bursts of capability further out.

Episode Transcript

View Full Transcript

There is one other question I had for you, Arshavir. You have Your Voicecraft, and what your information says is that it looks beyond the impressive output of AI to understand what is actually happening inside the model.

We should be using this and maybe answering with ChatGPT, giving ChatGPT questions with ChatGPT answers. Maybe that is the solution, I do not know.

How does Your Voicecraft know what is going on inside the model?

It is a little different from that. With the frontier models, what you have to understand is that these systems are powered by what you could call electronic guts, a series of literally billions and billions of numbers that we call the weights of the system. For the frontier models that are out there on a server and not confidential, those weights do not change as a result of what you put into them. If you put in a prompt, you are always going to get very similar responses with respect to a particular input, and there is really no way around that. You can do a little prompt engineering, putting some things into the prompt that steer it in the direction you want. But very often, if you ask it to use a particular voice or tone or way of expressing itself, you will find it does not come across as very authentic. It comes across as a computer trying to sound like it is talking with a Southwestern drawl, with little fake cactuses in the background, and not the real thing.

Local models can be trained, so you can actually change the values of those weights, because you have control over the model, which you do not have with the frontier models. We have a server that we use that is not ChatGPT or Claude but our own, and you can also run it on your own computer if it is fast enough. What our system does is take lots of examples of your own writing, read through them and train on them in a way that frontier models just cannot do. Then if you put in, hey, rewrite this, it will rewrite it in your voice because it has been trained on your voice, and that is the big difference.

There are other advantages. One is that you are not sending confidential information to some server at ChatGPT, Claude or any of those, so you do not have to worry about confidentiality, HIPAA, your marketing materials being leaked, any of that. The other is that with local models you do not pay for tokens, because it is your local model. These huge token bills people get go away, and you can keep hitting it as much as you want and all you pay for is the electricity. That is the idea of Your Voicecraft in a nutshell: a local model trained on your voice that provides you with documents and writing that sound like you wrote them.

How much does that cost?

We are still working on pricing, but it is not particularly expensive. It will probably be around 30 or 40 dollars a month for unlimited use. We are in beta right now. If you look at, say, my one month, my Claude bill was almost 1,000 dollars. It was huge. I use it not just for my own personal work but to help me with coding. It has agents as well, so I assign agents tasks like, post this every day at 3:00 PM, here is a whole bunch of notes I want you to post, and it does that. But if you rely on it as much as I do, it starts to get very expensive.

I heard a story where someone in a corporate job blew through the whole Claude budget in the first quarter.

I can well believe that, because it is not cheap. What people do not realize is that these big AI companies, as great as their valuations are, are really blowing through money. The price you are paying for those tokens is still heavily subsidized.

By investors, right?

Yes, by those investors. And that is one reason, to go back to our first topic of hype and whether this is going to kill everybody, that they really need people investing in what they are doing in order to continue providing this service at the current pricing. I have heard talk of very sophisticated ChatGPT models potentially being priced at 1,000 or 2,000 dollars a month. If you look at how much power and cooling these systems take to generate the responses, it really does get very expensive. I think that will go down and change, but it is definitely a factor right now in how they price and how they figure out what your bill is going to be at the end of the month.

Why isn’t AI solving the data center problem? Why can’t it find a recirculating coolant so it does not need to be on a river? Why can’t it develop superconductors so it can send and receive energy without loss? Why can’t it take the heat and recycle it, dump it into some sort of battery system? Is AI working on that? Do you know?

I do not know personally. I would suspect there are people working on all of those problems, and I am sure they are using AI to do it. They are not easy problems to solve. AI is very powerful, but it is not a magic bullet. I suspect it is a very good example of how AI can help AI. And by the way, data centers are not new. When you use the internet, you are using a data center. The idea that all of a sudden they are building data centers that never existed before is not true. What people are worried about is more energy usage and more heat being pumped into the atmosphere. They usually use water to cool them, and there is always going to be some evaporation, which you cannot get around. One plan has been to put data centers in the Arctic where it is already very cold, but you are still pumping heat into the atmosphere. The only other solution I have seen that I think is workable, though it will not happen tomorrow, is to put them in outer space or on the moon, with nuclear generators on the moon or solar power if it is an orbital platform. Then you do not have to worry about heat dissipation, because it is out in outer space, and you have satellite links to the data centers.

I do not understand how the heat would dissipate in space either, because there is no air.

I am not a space scientist, so I cannot tell you the exact details, but I know they have ways of dissipating heat in space. They probably have fins radiating the heat outward or something like that. The other thing that may help is to remember that data centers and LLMs right now use a huge amount of energy compared to the human brain, orders of magnitude more. The brain runs on something like 60 watts, basically just the energy you get from eating and drinking. There must be solutions to what we are trying to do. The fact that the human brain does what it does so much better than LLMs suggests that this energy problem should be conquerable at some point.

Absolutely. I was wondering where you see artificial intelligence going in the future. What new changes do you think are on the horizon, say within the next six months to a year?

I think technological progress is not linear. It goes for a while, and then there are sudden jumps. That is what we have seen with LLMs. Neural networks were in a sense moribund for a while, and then a paper came out in 2017, and ChatGPT came out in late 2022, and then all of a sudden there was this huge spike in how AI was used in the business realm. Things have continued to go upward and improve significantly since, but I do not know that there has been a jump at the same level as in 2022, when you went from no AI to suddenly having a relatively simple AI system. What AI can do now compared to a few years ago is very impressive, but it is not that jump. So I do not think we are going to see amazing miracles. Look back at the web: it started off, there was a slow build, and then suddenly everybody was on the web and everyone was selling on the web. It went in fits and bursts, and you will see the same thing. I do not think you will see anything magical in the next six months or a year. It will be roughly an evolution of what we are seeing now.

It is very hard to predict three, five or 10 years down the road, because we have now gotten to the point where these systems are designing themselves. We are beginning to create LLMs that no human being really understands how they were created, because they were not created by a human being. They were created by another AI system. Once that chain gets started, I think that may be where you see a sudden burst of innovation and capability, but it is so hard to predict. We were supposed to have flying cars and robots serving us in the ‘70s, and that never happened. Yet no one predicted the rise of personal communication devices like the iPhone, or that everyone would essentially have a supercomputer in their pocket. The things you think are going to happen very often do not, and the things that do happen are sometimes a huge surprise nobody predicted. Arthur C. Clarke was a genius who predicted a lot of this, but he was the only one. That is why I think three or five years from now things will be vastly different.

It might be fun to have an LLM read all of Arthur Clarke’s books and make some predictions.

And Heinlein too. There was one story where this man and woman were shopping and there was a 40,000-dollar diamond necklace, and he said, well, you can get that one, and she said, I do not want it, I have so much stuff already, the robots do everything for us and I am sick of it.

I know what that was. That was Frederik Pohl, a book called Midas World. It was a bunch of short stories along those lines that he put into a book. It was kind of a satire, and a very funny book. The wealthy people drove around in K cars, and the poor people had to use the really high-end stuff because they were forced to consume at maximum effort. When I read Arthur Clarke, or when I find him on YouTube in interviews, it is amazing. He knew we would have phones in our pockets and would not be able to escape. There is a book he wrote in the ‘70s in which he describes what really is an iPhone in detail, a handheld computer with buttons that can do any one of a number of functions. When you go back and read it, it is like this guy was talking about an iPhone in 1975 or ‘76. I think Arthur Clarke was an unheralded genius who does not get enough credit.

Maybe, but you do not know Arthur Clarke was right until 50 years later. I think somebody did a study of Nostradamus and said that statistically somebody could predict everything out of pure coincidence. I do not think that was the case with Clarke, though. There are a lot of examples.

I think Arthur had more on the ball and more reason to make the predictions he was making than Nostradamus did.

Arshavir, we are going to have to wind up. Thank you so much for your time. Before we wind up, what does your T-shirt say? It looks like there is an equation on there.

Have you ever watched The Twilight Zone?

I love that.

Do you remember the one where the Kanamits come to Earth and they have a book?

How to Serve Man.

To Serve Man. People are like, oh, they are here to help us. That is what it says on the front of the book in The Twilight Zone, To Serve Man, in Kanamit.

And serving man means how to cook.

Yes, and then there is that great line at the end where she runs up to him as he is getting on the ship to visit the Kanamit planet and says, it is a cookbook, and he gets pushed onto the ship. Of course I have given it away for anyone who has not seen it, but it was 70 years ago.

I love that. How can people find you if they want to talk to you, interview you, hire you or use your new platform?

You can subscribe to my Substack, which is the gateway to getting to me, at subscribe.insidetheblackbox.ai. People can also email me at just my first name, Arshavir at me dot com. Those are the two best ways to get ahold of me.

Thank you very much, Arshavir.

Keep exploring the show

Hear how more business owners are putting AI to work in their day-to-day.