TL;DR: Three tools read your Markdown and do the work — pick the one that fits. OpenCode for simple, AnythingLLM for power and privacy, Hermes for a server that runs while you sleep.
The words everyone uses, nobody explains
AI, agent, harness, LLM — you’ve heard them everywhere. If you don’t know which is which, you can’t pick the right one, and you end up paying for something you don’t need — a Claude subscription that locks you into their ecosystem.
This article pins the words down, then shows you the tools worth your time.
Harness vs. agent vs. LLM
Three words the rest of this series throws around loosely. Pin them down now.
- The harness — the tool you install. It’s the interface: it accesses your files, runs the commands, and manages the conversation. Think of it as the workshop.
- The AI agent — the worker inside the harness. Give it a task and it follows a routine, think of it as a craftsman who always checks the workshop manual first:
- Read your
AGENTS.md— a rulebook file that tells it how to behave in your vault - Load its own specific agent instructions, if you go beyond the usual Plan/Build agent modes
- Check whether it needs a particular SKILL.md (a bundled capability)
- Explore and gather what it needs and do the work
- Read your
- The LLM (Large Language Model) — the brain the agent reasons with. The only part that costs money. We cover which one to pick in the next article.
How it actually looks: the split-screen workflow
Your daily setup is two windows side by side.
- Left: your Markdown reader — typically Obsidian. You read, browse, edit.
- Right: your AI harness’ window. You talk; it reads the same vault and acts.
Same files, two views. No copy-pasting between apps. The agent sees exactly what you see.
Three harnesses worth your time
There are more harnesses out there. These three are where I’d start — open-source, mature, and each built for a different kind of user.
- OpenCode — the simple default. Initially running into a terminal, it now offers a desktop app, perfect for all users.
- AnythingLLM — the powerful one that keeps your data on your machine by running LLMs locally. Desktop too, far more features.
- Hermes Agent — the server grade. Runs scheduled jobs and chats with you over Telegram/Slack/Discord.
One thing matters more than any feature list: these tools are interchangeable. They all speak the same standards — your Markdown files, your folder structure, your frontmatter. Pick one today, switch tomorrow, change nothing in your stack. That’s the whole point of going AI-native. No vendor lock-in. Ever.
OpenCode — the simple default
OpenCode is a command-line tool at its core, but it ships a desktop app that’s perfect for basically everyone. I’ve only run the desktop version for a while now.
Zero config. It works out of the box and even bundles free models that handle everyday business tasks well. One catch: free models train on your data, so if yours is confidential, don’t use a free model. We cover which model to pick in the next article.
Who it’s for
If you don’t yet know what an API key is (a private password that lets a tool talk to a paid AI service), what a provider is (the company selling that access — OpenAI, DeepSeek, Anthropic, OpenRouter, …), or what running a local model involves — start here. OpenCode is extremely simple. That’s the whole point, it just works.
Getting started: the OpenCode docs — install in one command, open your company vault, you’re good to go.
AnythingLLM — the local-first power user
AnythingLLM is also a desktop app. On the surface it looks similar to OpenCode. Underneath, it’s way more featured.
It’s built for users who prioritize local-first — your files and your model stay on your machine. It also works with remote models, so it’s not local-only. The choice is yours.
What “local-first” actually means
A local model runs on your computer, not on someone else’s server. Your files never leave your machine — no API call, no third party, no data leaving your network.
The trade-off is hardware. Local models need a powerful computer, ideally with a dedicated GPU (a graphics card that also happens to be excellent at running AI). AnythingLLM makes sense once you follow the LLM world a little — knowing which models are good, what’s new, what fits your hardware. The features reward that curiosity.
The standout feature: the meeting assistant
AnythingLLM’s killer feature is the meeting assistant. It records any meeting — a physical room (through your microphone) or an online call — transcribes it locally, summarizes it, and saves the result as Markdown in your vault. Agentic follow-ups can file it where it belongs — one click.
The transcript is Markdown. The summary is Markdown. Your AI agent can search it, quote it, and act on it the next morning.
And there’s more: AnythingLLM now offers live transcription — dictate to your agent directly, WhisprFlow or SuperWhisper style. That deserves its own article, soon.
Getting started: the desktop install guide — one click for macOS, Windows, or Linux.
Hermes Agent — the server grade
Hermes is for when you, or someone in IT, can maintain a server. A server matters for two reasons: scheduled tasks have to run whether your laptop is open or not, and you want to reach your agent any time of day.
Three things Hermes adds over OpenCode
- External messaging. Hermes connects to Telegram, Discord, Slack, WhatsApp, and Signal. You talk to your agent where you already talk.
- Scheduled tasks. “Scan the app logs every night — if anything looks weird, Slack me or fix it yourself.” Or: “Every 9am, read the Kanban and tell me what to work on today, then kick off the conversation in Slack.” Your agent works while you sleep.
- Skills system. Hermes ships with a skills system — it can be bundled with skills to do nearly anything. Honestly, I’m not a fan. The agent rarely knows which skill to use when, and you’ll only touch a small fraction of them. Some see it as a feature; I see it as noise.
Hermes needs access to all your vaults. Installing it on a server is out of scope here — we’ll cover that separately.
The cybersecurity warning
Any system that combines an AI, access to all your files, and a presence on the open web is a real attack surface. Understand that risk before you deploy. We don’t hand-wave it.
Getting started: the Hermes quickstart — from zero to a working setup.
OpenClaw deserves a fair word: it’s the popular alternative — full-featured, two dozen messaging channels, a huge community. For a freelancer, it can work. For a company, it doesn’t. It’s built to be independent, which is exactly the problem when several people work together: you lack the control and guardrails a team needs. Hermes is the one I use — smaller scope, the three things I need, and I stay in control.
A note on models
Every harness here can run a local model or a cloud model — the two choices are independent. AnythingLLM makes going local especially easy, but all three handle cloud and local models without friction. Pick your favorite combo. Not sure how to set it up? Ask your AI agent to deploy it the way you prefer.
Why not the locked ecosystems
Claude Cowork and Codex can replace OpenCode. Neither is open source.
The whole point of migrating was to escape lock-in — trading one SaaS hostage for another defeats the exercise. Fine to try. Not fine to build a company on.
Which one are you?
| If you… | Use |
|---|---|
| Want simple, works out of the box, files on your laptop | OpenCode |
| Have strong reservations about data leaving your machine, or want to dabble deeper into what LLMs can do | AnythingLLM |
| Need scheduled automation and chat any time of day, and can run a server | Hermes Agent |
What you gain
- One tool, not a stack of subscriptions.
- Switch any time — your Markdown doesn’t care which harness reads it.
- No vendor lock-in. All three open-source, all interchangeable, all sharing the same standards.
The data outlives the tool. That’s the bet.
Going further: isolation at scale
If you’re deploying an agent that can read every file and act on the open web — especially with a team — raw access isn’t enough. You want a fence around the agent so a bad command can’t escape. Two open-source approaches:
- Sandboxing runtimes — NemoClaw (by NVIDIA) runs agents like Hermes or OpenClaw inside hardened sandboxes, with network policy and managed inference. Still alpha, but the right idea.
- Micro VMs — a micro VM is a tiny virtual machine (a self-contained computer-within-your-computer) that boots in milliseconds to isolate one process. The building blocks: Firecracker (AWS — the canonical one), Cloud Hypervisor (Intel-led), Kata Containers (container ergonomics, VM isolation), and gVisor (Google — isolation without a full VM).
This is enterprise territory. If you’re a solo operator, ignore it. If you’re running agents in production over real data, it’s where the conversation goes next.
What’s next
Tool picked. Now configure it: AGENTS.md Template.
Markdown your next move. By Charles Henri Gayot.