Hermes Agent: Agents that grow with you |Episode #357|
Summary
- Hermes Agent’s product thesis is that useful workflow knowledge accumulates with use. Jeffrey Quesnelle describes an agent that turns successful tasks into reusable skills without being explicitly told to save them. A restaurant booking took 30 or 45 minutes initially; the following day, it reused the procedure immediately—the practical meaning of “the agent ought to get better the more you use it.”
- AI value will depend on completed work, with models and harnesses contributing jointly. Quesnelle’s analogy is “the model is the brain, and the harness is your body”: stronger execution infrastructure can let a weaker model outperform a stronger one. His economic endpoint is customers paying for outcomes delivered more cheaply or quickly, while ownership and value capture remain unsettled.
- Open models need sponsors whose businesses benefit from their adoption. Quesnelle contrasts Meta’s costly model-development setbacks with Chinese companies using open releases to gain prominence. He sees NVIDIA as a potentially significant Western sponsor because workloads run on its chips: “the house always wins.” His optimism explicitly applies to the short term.
- The enterprise opportunity is spreading one engineer’s expertise across an organization. Nous’s internal debugging agent accumulated skills over about a month, allowing support staff to investigate customer problems and obtain reports and root causes. Collaborative and multi-tenant deployments are the next direction, but Quesnelle names no partners or commercial results.
- Small teams can gain leverage without reproducing a frontier lab’s budget. Nous pursued “1,000x improvements” around expensive research bottlenecks, including scarce engineering talent. Quesnelle credits a team member he names as Technium in one passage and Teknium in another, whom he describes as not previously able to write a large application from scratch, with building what he calls GitHub’s “number one open source repository.”
- Effective delegation requires explicit success criteria and careful task selection. Quesnelle recommends workloads requiring “infinite patience but very low creativity,” rather than agents modeled on executive job titles. Users should describe outcomes and evaluation standards; human taste remains a limitation he would not delegate away “at least right now.”
- The long-term risk is that automation weakens the people using it. Quesnelle worries that users could “check out of thinking,” especially over a generation raised to assume the computer knows. Nous’s stated standard is that AI should make people “better today than yesterday and better tomorrow than today”—a goal he says also requires responsible use.
Deep dive
1. Nous reached Hermes through a search for cheaper research
Quesnelle traces Nous Research to “the homies on the Discord,” a loose community concerned that transformational AI would become locked inside a few companies. Around the arrival of GPT-3.5 and ChatGPT, capitalization looked difficult to overcome. Forming a company about two years earlier let contributors move from unpaid spare-time research to working on that mission full time.
The response was to seek “1,000x improvements” in expensive parts of the AI pipeline, with resources Quesnelle compares to larger companies’ “catering budget.” Research included training models over the internet, while the original Hermes fine-tunes emphasized models that respected users without moralizing. Those releases preceded the thinking-and-reasoning model era; the broader objective was to keep capable AI accessible.
Hermes Agent began about six months before the conversation as internal tooling for model researchers. Quesnelle compares scarce, highly paid AI research engineers to NFL quarterbacks: GPUs were not the only bottleneck. Automating research was an attempt to approach recursive self-improvement. After several months internally, Nous released the repository and encountered what he calls “a floodgate of product-market fit.”
The leverage argument becomes personal through a Nous team member whom Quesnelle names as Technium in one passage and Teknium in another. Quesnelle says he was not really a developer and previously could not have written a large application from scratch, yet built Hermes Agent using AI tools. Calling it the “number one open source repository on GitHub,” Quesnelle argues that focused individuals still have substantial opportunity; he does not regard the competitive window as closed.
2. Open-source funding follows business incentives
The hosts press the tension between openness and paying salaries. Quesnelle credits Llama 1, Llama 2 and Llama 3 with bootstrapping researchers, then characterizes Llama 4 as an architectural misstep that made people question the motivations for spending on open models. His rough framing—“$200 million, $300 million, $400 million” for “essentially a paperweight”—sets up the question of whether openness serves the sponsor’s business.
DeepSeek’s release added a geopolitical dimension because it was both open and, at the time, nearly state of the art. Quesnelle describes subsequent Chinese releases as a growth mechanism: open models could elevate companies into the top tier, with monetization deferred. He says open-source models were then coming exclusively from China as enthusiasm for that funding model weakened in America.
At its last GTC keynote, Jensen Huang said NVIDIA would try to carry the financial flag for Western open-source models, committing “something like $20” over the next several years. Quesnelle sees stronger alignment between openness and NVIDIA’s business because model workloads run on its chips: “the house always wins.” He is optimistic “at least in the short term,” while declining to predict several years ahead.
3. Outcomes determine value across the model and harness
Asked whether value moves from models into agent infrastructure, Quesnelle offers an explicitly imperfect analogy: “the model is the brain, and the harness is your body.” The model remains the enabling breakthrough, but intelligence without the ability to act has limited practical reach. Conversely, a less capable model with better tools can accomplish more: “A better harness with a worse model can beat a better model with a worse harness.”
The harness connects token generation to the world people inhabit. It supplies ways to act, maintain persistence and operate through time rather than merely return text. Quesnelle treats the two components as symbiotic; the harness matters because it turns the model’s output into changes outside the conversation.
He declines a sweeping pronouncement about which layer captures the money. The more useful comparison is hiring an employee to get a job done: customers ultimately want a materially valuable result. He expects systems to be paid according to their ability to deliver it more cheaply, quickly or cost-effectively. Ownership of models and harnesses still has to be worked out; demand for outcomes is “water going downhill.”
4. Hermes turns successful tasks into reusable procedures
Hermes’s defining decision was that it should improve through use. Rather than predesign every workflow, Nous bundled a limited set of capabilities, including running code and browsing, then encouraged behavior through prompts. Quesnelle’s instruction is to “get out of the model’s way.” Skills and memory provide mechanisms through which the model can preserve what it judges useful.
His restaurant example supplies the mechanism. While flying to a team offsite, he asked Hermes to make a reservation. Website automation barriers repeatedly blocked it before it found the booking API backend and completed the task. The process consumed about 30 or 45 minutes of agent runtime, after which it independently saved a skill called “Las Vegas restaurant booking.”
The following day, Quesnelle requested another restaurant: “Boom, instantly it knew exactly how to do it.” The point is that the procedure survived the original task. Prompts encourage the model to recognize repeated activity or an important achievement, record it, and reuse it. Nous relies on the model’s judgment about what deserves saving.
Memory follows the same philosophy. A hierarchy includes notes about users and how they want to be treated, alongside previous sessions the agent can revisit. The model largely decides when to activate those layers. Quesnelle’s claim is that a future GPT or Opus improves Hermes simply by exercising better judgment over the existing mechanisms, making model upgrades directly useful to the product.
5. Shared agents turn debugging into organizational memory
The initial audience is the power user seeking AI’s full capabilities. Local operation with local models is a “first-class option,” while Nous also offers hosting. Quesnelle says the one-person, one-agent experience is in a satisfactory place; attention is moving toward workplace collaboration and multi-tenancy, with high-level partners involved but nothing yet to announce.
Nous is testing that direction internally. Quesnelle says Hermes Agent is written “99.99% by Hermes Agent,” and several agents perform operational work. One has read-only MCP access to backend infrastructure and databases. It began as a debugging assistant for website failures, including 500 errors, across a tangled collection of databases, Vercel and other services that engineers otherwise had to inspect separately.
Over about a month, engineers’ interactions built a repertoire of troubleshooting skills. Support staff can now ask about a customer account and obtain reports and root causes without Nous coding each workflow explicitly. Quesnelle’s organizational argument is that an engineer works through something once, while later colleagues benefit from the retained procedure.
6. Delegation changes where human judgment belongs
Whitenack challenges the assumption that automation should mirror human processes, including coding conventions such as avoiding duplicated functions. Quesnelle’s operational test is to treat agents as “humans with infinite patience,” qualified immediately by “very low creativity.” Reading every log is theoretically possible for a person but unattractive in practice. He rejects copying an organization chart into CEO and CFO agents; select the work by what it requires.
Having coded since seven or eight, Quesnelle had built his identity and career around that advantage. Until about six months earlier, “I wrote all my code myself,” believing he remained better than AI. Using Hermes over the last three to six months changed his mind. He now sees an opportunity to “run the clock back and be early again,” rather than simply losing a former advantage.
His remaining boundary is aesthetic judgment. He sharpens the thought that models do not care about appearance into “they really don’t know.” Human taste, he argues, comes from shared lived experience, biology and development that models lack, whereas mathematical and computational problems are more universal. They may solve tasks in aesthetically unpleasing ways; he would not rely on them “at least right now” where human taste determines success.
Benson agrees but adds his own limitation: “the models regress to the mean.” He finds them usually less useful for genuinely novel work without related training material. That still leaves, in his deliberately expansive framing, “99.9999% of the usage out there”—a large opportunity even within the constraint.
7. Explicit goals must coexist with human agency
Asked how users should retrain themselves, Quesnelle starts with “don’t tell it how to do something.” Describe the outcome and the conditions under which it counts as achieved. He expects models to improve at long-horizon planning; users should become goal-oriented and evaluation-oriented, allowing the agent to work toward stated success criteria.
The writing example makes the problem concrete: request prose, then complain about “slop,” repeated “not X but Y” constructions and equal-length paragraphs, and the model has still fulfilled the explicit request. The missing standards were unstated. His replacement for “explain it to me like I’m five” is “explain it to me like I’m an alien”: assumptions shared between humans must become explicit instructions.
Beyond enterprise growth, Quesnelle worries about people’s relationship with work and thought. AI could extend a society already inclined toward doom-scrolling by letting users “check out of thinking.” Over 10 or 20 years, someone raised entirely within this paradigm might develop little critical judgment, accepting that “the answer is the computer knows.” He presents this as a concern, not an inevitable outcome.
Having had a baby five days earlier, he connects that concern to the world his daughter will inherit. Nous’s “human-centric AI” standard is to make people “better today than yesterday and better tomorrow than today,” but he says users bear responsibility too. His response to the risk is agency: participate in shaping the technology and “be part of the solution.”
Verification Notes
The transcript gives NVIDIA’s commitment as “something like $20” but does not state a unit. It also spells the team member’s name both Technium and Teknium.