If I were setting up a small service business around AI today, I would not start with the flashiest app. I would start with one question.
Which parts of the workflow deserve control?
That is the filter most people skip. They buy a bundle of AI subscriptions, route half their internal thinking through random dashboards, then act shocked when the system gets expensive, messy, and hard to trust.
A better stack is smaller.
For a Sterling Labs style operation, the goal is not to force every task offline. The goal is to keep sensitive work, reusable prompts, and internal knowledge in a setup that does not fall apart the second one vendor changes a plan or buries a feature.
This is the local-first stack I would use in 2026 for a small service business that wants privacy, portability, and a sane operating model.
The stack at a glance
| Layer | Tool | Why it makes the cut |
|---|---|---|
| Local model runtime | Ollama | Best backbone for running local models cleanly |
| Desktop chat and testing | LM Studio | Fastest way to validate local model workflows |
| Shared browser interface | Open WebUI | Strong self-hosted hub for model access and team use |
| Local knowledge layer | AnythingLLM | Good fit for document chat and project workspaces |
| Lightweight open-source desktop app | Jan | Nice dedicated local chat option |
| Notes and spend tracking | Ledg | Good manual visibility into tool costs on iPhone |
That is enough.
Not because more tools do not exist. Because most people do worse when they have too many moving parts.
Layer 1: local model runtime
Every serious local stack needs a dependable engine. For that, I would start with Ollama.
Ollama is the practical foundation because it gives the rest of the stack something stable to connect to. It is the part you want to forget about, in a good way. Pull the models you need, run them locally, and let the higher-level tools talk to that runtime.
That matters for two reasons:
If a business is going to build repeatable internal AI workflows, portability matters more than people think. A pretty UI is nice. Rebuilding your whole operating system because one app loses momentum is not.
Layer 2: desktop testing and quick iteration
LM Studio is the fastest way I know to get local AI into a useful desktop workflow.
Its official site says it is free for home and work use, and that alone removes a lot of friction. But the bigger win is speed. You can test models, compare outputs, and expose an OpenAI-compatible local API without turning setup into a side quest.
This is where I would do:
LM Studio is especially good when the question is not "what is the perfect stack forever" but "can this task run locally well enough to be worth keeping?"
That is an important distinction. In small businesses, a lot of expensive software decisions happen before the workflow is even proven. I would rather validate locally first and scale second.
Layer 3: the team-access interface
If the business grows past one person, or just needs a cleaner internal interface, I would add Open WebUI.
Open WebUI gives the stack a proper home base. It is self-hosted, supports local and cloud model connections, and works well as the browser layer for teams or mixed-device setups.
This is the tool that starts turning a collection of local components into something operational.
Where it helps:
I would not start here if the business is still in test mode. But once the workflow deserves a real interface, Open WebUI becomes compelling fast.
Layer 4: the knowledge layer
Businesses do not just need text generation. They need answers against their own material.
That is where AnythingLLM earns its keep.
The official site describes it as an all-in-one AI app that works locally and offline, and says it is open source and free to use. More importantly, it handles the actual business problem: project workspaces, document chat, and a usable path from raw files to grounded answers.
This is where I would use it:
The trick here is simple. Do not dump garbage into the system and expect magic back. A local knowledge layer gets stronger when the source material is clean, versioned, and scoped well.
Layer 5: lightweight local desktop chat
Jan is the optional layer, but I like having it.
It is a good fit when someone wants a dedicated local chat app that is open source, fast to understand, and not overloaded with enterprise ambition. Jan states clearly that it is free and open source. That makes it easy to recommend for focused use cases.
I would use Jan for:
It is not the center of the stack. It is the clean side door.
That matters because not every team member wants the same interface. Some want a dashboard. Some want a desktop app. A good stack has room for both without breaking the core architecture.
What I would not do
I would not build the system around ten AI wrappers.
I would not put sensitive internal thinking into a random cloud app just because the onboarding was slick.
I would not buy a premium subscription for every new category before proving the workflow saves either time or money.
And I definitely would not confuse agent demos with operating infrastructure.
A lot of small businesses get wrecked by software optimism. The demos look sharp. The stack gets bloated. Nobody can explain which tool is doing what. Six months later the team is paying for confusion.
The actual workflow
Here is the version I think holds up.
Drafting and analysis
Use LM Studio or Jan for fast local chat and draft work.
Internal knowledge
Use AnythingLLM for scoped document sets and project-level retrieval.
Broader internal access
Use Open WebUI when the workflow needs a browser interface or shared access.
Model backbone
Use Ollama underneath the stack wherever it fits.
Spend visibility
Track the software side manually instead of pretending the stack pays for itself by default.
That last point is not glamorous, but it matters. AI stacks become expensive through drift, not one giant bill.
Why Ledg stays in the picture
AI tooling does not just create outputs. It creates recurring software spend.
That is why I still like a manual, privacy-first tracker for the finance side. Ledg is useful here because it gives you a plain way to log software subscriptions and stack costs without another giant dashboard pretending to optimize your life.
I would rather have a boring truthful picture of tooling costs than a magical-looking analytics panel that hides the real total.
When local-first is the wrong move
Local-first is not a religion.
If a task truly needs frontier model performance, giant context windows, or heavy multimodal capability that your hardware cannot handle, fine, use cloud tools. Just do it on purpose.
The mistake is not using the cloud. The mistake is defaulting to it for everything, including the work that obviously benefits from tighter control.
For a small service business, the sweet spot usually looks like this:
That is a grown-up system.
The payoff
A local-first stack does three things well.
First, it reduces casual data leakage.
Second, it makes the business less dependent on one vendor's roadmap.
Third, it forces cleaner thinking about what AI is actually for inside the company.
That is the part I like most. When the stack is smaller, each tool has to justify itself. That is healthy.
Final recommendation
If I had to roll this out in order, I would do it like this:
1. Start with LM Studio to prove useful local workflows.
2. Add Ollama as the stable runtime backbone.
3. Add AnythingLLM when documents and internal knowledge start piling up.
4. Add Open WebUI when the stack needs a shared interface.
5. Keep Jan around for users who want a clean personal desktop app.
6. Track the cost side with something simple so the stack does not quietly become a tax.
That is enough to run a serious operation without drowning in AI software theater.
You do not need the loudest stack. You need one you can trust.
If you want help designing that system properly, Sterling Labs can do the cleanup and setup work.
And if you want the blunt version, here it is: local-first wins whenever the work is sensitive, repeatable, or strategically important. Which is more of your business than most people admit.