Episode 42

How to Build Your Own Private AI Assistant That Works Like an Employee

Key Takeaways

  • A local AI model running on a small office box keeps client data inside the building and removes per-token costs.
  • The hardware costs about 5,000 dollars, the model itself is free to download, and a frontier model can be used only to help configure it.
  • A trained post-production bot tracks schedules, collates feedback and updates calendars when a render slips.
  • A private AI can act like an employee: monitoring channels, watching disk space and learning from every document added.
  • A personal DadBot pulls school and activity updates into one weekly printed newspaper, with a dad joke.
  • An LLM wiki acts like a second brain, tagging and linking your notes, and Obsidian makes it readable.
  • Use Tailscale to reach a private AI securely from outside the office, and check open source licenses like MIT and Apache 2.0.

Main Topics

1PRIVATE AI

How a local model on your own hardware protects data and avoids token costs.

2AI AS AN EMPLOYEE

Bots that track schedules, feedback and files and learn from company documentation.

3SECOND BRAIN

An LLM wiki and Obsidian for organizing raw material into searchable knowledge.

Featured Guests

MC

Runs a small immersive video production company and builds local AI assistants.

Matt Celia

Meet the Hosts

Elizabeth Gearhart
Elizabeth Gearhart, Ph.D.Chief Marketing Officer, Gearhart Law

Elizabeth Gearhart, Ph.D. is the host of Real AI Use Cases: Business Owners Roundtable, where she explores how businesses are actually putting AI to work. She is also CMO of Gearhart Law and co-host of the nationally syndicated Passage to Profit radio show and podcast.

Richard Gearhart
Richard Gearhart, Esq.Founding Partner, Gearhart Law

Richard Gearhart, Esq. is co-host of Real AI Use Cases: Business Owners Roundtable and founder of Gearhart Law, a full-service intellectual property law firm. A longtime podcaster, he hosts Intellectual Property News and is the host of the nationally syndicated Passage to Profit radio show and podcast.

Elizabeth and Richard share how they car-shopped with ChatGPT, why a private knowledge base is powerful and what open source really means.

Episode Summary

Frequently Asked Questions

What is a local AI model and why use one?
Matt Celia explains that a local model runs on your own hardware, so your data stays private and you do not pay per-token costs. His box costs about 5,000 dollars and the model is free to download.
What can a private AI do like an employee?
Matt’s post-production bot tracks schedules, collates feedback from Frame.io, monitors Slack and disk space, updates calendars when a render slips and learns from documentation as it is added.
Does a private LLM need an internet connection?
Matt Celia says you need the internet only to download the model initially. After that it can run offline, and he uses Tailscale to create a secure private connection from outside the office.
What is DadBot?
DadBot is Matt Celia’s personal AI that pulls updates from school and activity services into one weekly printed newspaper for his kids, with a weekly dad joke.
What is an LLM wiki and Obsidian?
An LLM wiki is a prompt-based approach that tags, links and organizes raw material into a second brain, and Obsidian is a free note-taking app that lets you view and browse it in a readable format.
Does open source mean free?
No. Matt Celia says open source allows you to modify and redistribute software, but you need to check the license, such as MIT or Apache 2.0, and you can ask your LLM to explain it.

Episode Transcript

View Full Transcript

Today we have Matthew Celia, and his company is LightSailvr.com. Matthew, what is one way you are using AI in your business, or even personally?

I would love to tell two stories, because we use AI in a lot of facets and I am very enamored with this technology. For our business, we are a very small boutique production company working in immersive video, which is really challenging. Every bit of efficiency we can find, we have to take, because there is not a huge market for immersive videos, they are expensive to produce and we want to keep making them.

What are immersive videos?

Immersive videos are either 180-degree or 360-degree videos that you watch inside a virtual reality headset. When you are watching the content, you literally turn your head to look. You are the camera, and you can absorb it as if you are really there.

Wow. So what is another way you are using AI?

The new technologies that came out in the last year record at 16K resolution, stereoscopic, 90 frames a second. For those who are not into the technology of video production, those files are extremely demanding, probably 10 times what a Marvel film is, I would estimate.

Demanding in what way?

Storage space and processing power. Just working with these files is extremely hard, which means things take a long time. Managing that is a gargantuan task here at LightSail. When I press export, it takes six hours. That requires a lot of foresight. Is this right? What could go wrong? How am I getting feedback? What do the creatives think? There are a dozen different things I have to be thinking ahead of, and AI is very good at helping you think ahead. So I have been using my experience as a post-production producer and supervisor over the last couple of decades to train what I call a post-production bot. It keeps track of all of our schedules. It scans our Frame.io to collate stakeholder feedback and sends that out securely to editors, visual effects artists and so on, so we can keep track of how we are addressing every single thing. It changes the calendars as things move. Sometimes a render fails and we need to deliver a couple of days later, which shifts color, sound and approval windows, and it notifies everyone, hey, you are going to see this on Thursday instead of Tuesday now. That matters because VR requires you to stop your life and put on a headset and consume it, so it is important to schedule with stakeholders. This bot runs completely local on LightSail hardware. It processes all that data securely and locally. Nothing ever leaves the walls of my building, no client data ever leaves, we do not have to worry about being sensitive, and it keeps everything on track in secure communication channels with everybody on the team.

What is that costing you?

The box is about 5,000 dollars. I think there is a new one out now, so it might be even cheaper. AMD is great because they are offering a lot of value for the money. The model itself is free. You can download it off the web. I use Claude to help configure the model. Claude is a frontier-level model, so I use it just to configure the setup and make sure I can talk to it, and that is where my cloud compute ends. Now I have my own version of Claude locally on my desk. I keep gesturing here, but it is a box, about this big, and it sits on my desk like a little appliance.

You are used to thinking in three dimensions.

Right. What is so phenomenal about it is that I do not pay a token cost outside of that, and it can be crushing on stuff 24/7 for me without any privacy concerns or token cost usage. I have it working all the time. It is monitoring the Slack channels, Frame.io, the hard disk space and security. Every time someone enters documentation or puts anything in, it is ingesting and learning from all of it. So it becomes an employee, just like any other employee. I can literally chat to it and say, hey, last week you did this thing, what was that again, can you give me those steps? Or, we are going to set up a new project, it is episode two, we completed episode one six months ago, can we duplicate the post calendar but have it start November 1st? And boom, it does it.

I wrote a blog post just this morning about having a private LLM. I asked, why is that any better than just dealing with the data we have now? For us, we share files on SharePoint, and I have to know where the file is, who put it there and what subdirectory they put it in. With AI you can just say, find this file.

Correct.

Is there another use you wanted to talk about?

Let me pivot to a personal use, because my wife is the ultimate technology skeptic. She is very anti-AI, and she thinks it is all going to come and murder us. So I created something called DadBot, and this is for all the parents out there. I have two kids, a seven-year-old and a 10-year-old. They are in LAUSD public school, which has a lot of messaging, and a charter school has its own messaging. They are also in dance, theater and music classes, and they have play dates and Girl Scouts. We are busy. We are an American family, in the thick of it right now, and every organization and company is using a different service. There is Band and ClassDojo and Google Classroom and LAUSD’s site. It is insane. All of these services have two-factor authentication and strong passwords, so getting updates from all of them would take me literally hours every night to collate. DadBot does it for me automatically. I gave it access, and it goes in, pulls the information and flags me. It says, tomorrow is pajama day, make sure your kids wear their pajamas or they will feel left out. It says, Evelyn has a school report due on Friday, have you checked it? And here is the best part. Every Sunday night I have it print out a physical newspaper for my kids that details the week ahead, with stories about the weather, what they are doing for homework and after-school activities, what they need to practice for piano that week, and my favorite part, a really terrible dad joke, which always gets a loud groan on Monday morning at breakfast.

That is a brilliant use.

I am using this not to get my kids more digital. It is actually the opposite. It helps me put a physical piece of paper in front of them that collates all the information they need to know, and has me stop looking at my phone endlessly trying to scroll and find things. Instead I have one sheet of physical paper I can read while munching on my Cheerios and connecting with my kids. This is the promise of AI.

I think those hybrid solutions are really powerful. I have a calendar I write everything on, but I would love to have a bot like that. I have so many people coming at me from so many directions. Richard Gearhart with Gearhart Law, what is one way you are using AI?

I use it all the time for so many different things. I am probably going to lose my ability to think in the next six months because I rely on it.

You haven’t already?

You could say that. The example I was going to give is that we decided to get a new car, so we used AI for car shopping. In the past you would go to different websites with a set of filters, and you could find a whole bunch of cars, but then you have to open each record individually and scroll through it to see if it has the features you want. It is better than driving from dealer to dealer, but it still takes a lot of time. I used ChatGPT, and it was able to identify the cars with the features I wanted at an appropriate price, and eventually find the car I wanted. I tried to purchase one online but it did not go through, and the next day I got a message that the car had already been purchased by somebody else. So I immediately found another one, and we drove to the dealership that evening and did the whole deal. It even found financing at 0.9 percent. We ran the numbers on the cost of the cars, and after factoring in the cost of the financing, buying a new car turned out to be more economical than buying a used car at current financing rates. I could have sat down and done all the math myself and I would still be working on it, but in just a few hours it made a fairly complicated process pretty straightforward.

For me, with Gear Media Studios, I was at a networking event last night, and a lot of these people do not know how to use AI in business. It got me thinking about which one use case gives the highest ROI, because there are a lot of ways to use AI in your business and they do not always come back with saving you money, and businesses have found that out. It is what Matthew was just talking about, having your own private LLM. I went to Gemini and asked how you could have a completely private one, and it is exactly what Matthew said. It has to be on your own server, or even on your own hard drive, on a computer that does not have any access to the internet. If I were going to build a knowledge database for Gearhart Law, which has a lot of knowledge other law firms do not, I would have it on its own private hard drive like Matthew has.

How would you not have an internet connection, though?

You do not need an internet connection to host a local LLM. It just uses your own computer. I have one on my MacBook, so when I am on an airplane with a bad connection, I can still chat with the LLM running locally on my machine. Everything is on that computer.

Can you put it on more than one computer?

It is not really a virtual private cloud. It is a box that sits in your office, and everyone can connect to it because the box is serving an IP address saying, hi, I have an API, you can come talk to me and send me requests.

How does the software update if there are changes, or do you have to download something initially?

You have to download the model initially. Of course you have to get it from somewhere. But once it is there, you do not have to have it connected to the internet anymore if you do not want to. I do have mine connected, using a service called Tailscale that creates a secure tunnel between my laptop when I am out of the office and my box. I can still use it out of the office, and it is securely tunneling the two together, because I do not want to put it up on the public open internet. That is a massive security risk.

What is the name of the software?

Tailscale is an amazing piece of tech. There are paid plans, but the free tier is extraordinarily generous. Think of it as creating your own private network, no matter where all your computers are. I could be at home or in the office, and the machines can still talk to each other as if they are on the same Wi-Fi network.

When we first started this podcast, early on somebody talked about having a private LLM for their company, a knowledge database everybody in the company could access. He was a CEO and super busy, and he said, if one of my employees has a question, I have taught it who I am and how to think like me, so they can ask this LLM and it will give them the answer I would give. The way you are using it is very similar, and I think that is probably the most powerful use of it: a knowledge database, and if there is one major decision-maker, that person’s thought processes in the database. Then it becomes a querying system on your own data, your own company’s processes and trade secrets too.

Are you familiar with Andrej Karpathy’s LLM wiki skill?

No.

Andrej Karpathy was an engineer at OpenAI, and he published on GitHub what you can think of as a prompt to create the LLM you are talking about. You put raw material into it, and it goes through, tagging it, linking it, finding relevant sections and organizing it, basically creating a brain. It is a second brain that forms neural pathways, so when you query it, it is not just doing a semantic search for what is tagged. It is reasoning, saying, I think you are talking about this, and this is also relevant, this is the most up-to-date thing, and here is how this article connects with that article, just like your own brain works. It has something like 50,000 stars on GitHub. It is one of the most popular. We call it a gist, a little piece of text, like a skill, and it revolutionized how I think about all of this. I modified it to become a documentary story producer for a project I am working on. I can feed it hours of interviews from my documentary subjects and use it to find thematic elements between all of them. I can ask, tell me about the water cycle, or, are trees dying, and pull those timecodes into a beat sheet I can hand my editor as a director and say, here is our story. I use a free note-taking app called Obsidian. It is open source, I believe, and free to use, and it keeps track of all of the files in my LLM wiki, with a cool graph interface so you can see how everything connects. It looks like a living brain, which is very visually cool. But the reality is you need a way to take all the raw material you throw in and look at it in a human-readable format, and Obsidian is becoming my second brain.

I agree with that, because your information and experience are part of your competitive advantage, what distinguishes you from others. If you constantly feed your information into the public LLMs, I think you make yourself a little mediocre, because it is just averaging all the information it gets from different sources when it provides output. So I think that exclusivity has a lot of value.

I think it is important for people to have digital literacy, and this is moving very fast, and none of us grew up in school learning this. What I am teaching my kids is digital literacy: know where your data is going and have ownership over it. It is okay to give your data out. That is the contract you make when you use Gmail for free. A lot of the applications I am creating, I am vibe coding with LLMs and releasing for free. I am making them open source, because the LLMs were trained on open source. They learned how to code because of the vibrant open source community, people who believe the internet should be open and free, that software is for everybody, and that we all have the opportunity to benefit from technology. Those are values I strongly believe in. I do not believe in gating things behind paywalls or cutting off access. But everyone has a choice, and you should too, about where your data goes.

You can choose to dedicate it to the public, or you can choose to keep it to yourself or try to control the rights. It is an interesting approach that all of the big tech companies file a lot of patents and fight over them all the time. Sometimes I feel it is a little hypocritical for the industry to promote open source and encourage small companies and entrepreneurs to work in open source, yet they spend hundreds of millions of dollars on intellectual property so they can control the information they create. Anthropic and all of the other AI companies guard their technology like their life depends on it.

It is why NVIDIA just put a 6 billion dollar investment into open source AI models. NVIDIA coming in and taking a stand is a really big win for open source communities, because we are all going to need open source models that we own personally and run locally to protect ourselves from other things that may be trying to get in.

Let me clarify one thing. We have an attorney who works on litigation, and he did a short video on how open source does not mean free.

That is correct. Open source has the ability to modify, change and redistribute, but you have to check the license, like MIT or Apache 2.0. These are important things to understand, and if you do not know them, ask your LLM to explain them to you.

That is another thing I use it for. I have it teach me all sorts of things, including how to use it. We were talking to somebody who said he puts what he wants to find out into one LLM and has it make the prompt, then has another LLM refine the prompt, and then he uses the prompt in another LLM. It will teach you how to use it, and it will also tell you about itself. It did not used to. I use it for marketing, and when ChatGPT first got popular and then Gemini came out, I asked Gemini how Gearhart Law could show up as the answer on ChatGPT. Since the first time I saw somebody come up as the answer, I have been banging on these LLMs, how can Gearhart Law come up as the answer? Gemini would not tell me. Now they give me all sorts of information about what you need to do so the answer engines can find your content online and offer you up. AEO is about being found. GEO, generative engine optimization, is about what you say and having the engines take your content and offer it up as an answer. I have been researching that for years, and the information is a lot better now than it was four years ago.

What did you find? I am curious, because we just recently re-optimized our site to do the same thing.

Keep exploring the show

Hear how more business owners are putting AI to work in their day-to-day.