In this article, you will learn the mechanical difference between retrieval-augmented generation and fine-tuning, when each technique is the right tool, and how to decide which one, or both, your production system actually needs. Topics we will cover include: What RAG and fine-tuning each do at a mechanical level, and what each one cannot do. Two complete, working code examples — one RAG pipeline for a knowledge-retrieval use case, and one LoRA fine-tuning setup for a structured-output use case. A concrete six-point decision framework for choosing between RAG, fine-tuning, or both. RAG vs fine-tuning is one of the most searched, most argued-about tradeoffs in applied LLM work right now, and most of the debate happens at the wrong level of abstraction. It gets framed as a single either/or decision, when the honest 2026 picture is that roughly 60% of production LLM deployments now use both together, not because teams couldn’t decide, but because retrieval-augmented generation and fine-tuning solve two genuinely different problems, and most real domain-adaptation projects have both problems at once. This article breaks that false binary down properly: what each technique actually does at a mechanical level, two complete, working, real examples — one for each approach — and then a concrete decision framework for figuring out which one your specific project actually needs, and when the honest answer is both. What RAG Actually Is (and Isn’t) Retrieval-augmented generation doesn’t touch the model at all. The model’s weights never change; what changes is what the model sees in its context window at the moment it’s asked a question. A retrieval step searches a knowledge base, pulls back the most relevant documents, and hands them to the model alongside the user’s query, so the model is answering with a briefing document in front of it rather than from memory alone. That mechanism is what makes RAG genuinely good at exactly one category of problem: information that’s large, that changes, or both. What RAG doesn’t fix is a model’s underlying behavior. If the model’s tone is inconsistent, if it won’t reliably follow a strict output format, if it uses your industry’s vocabulary incorrectly, feeding it more documents at inference time doesn’t touch any of that, because the problem was never a lack of information in the first place. What Fine-Tuning Actually Is (and Isn’t) Fine-tuning does the opposite: it changes the model itself, training the weights on real examples of the input/output behavior you want until that behavior becomes the model’s default, with no need to inject anything at inference time because the pattern is now baked in. LoRA and QLoRA are the standard approach for the large majority of projects, training a small adapter — often under 1% of the base model’s total parameters — rather than the full model, which brings a fine-tuning run down to a few hundred dollars and a few hours instead of a full retraining project. Here’s the point worth landing hard, because it’s the single most common misunderstanding in this whole debate, and it’s one several independent sources converge on with identical wording: fine-tuning doesn’t reliably add factual knowledge. A model fine-tuned on a pile of medical literature doesn’t “know” the facts in that literature the way a retrieval system genuinely does — it adjusts style, structure, and pattern recognition, but factual recall from training data is unreliable, especially for granular facts. Fine-tuning is a behavior tool, not a knowledge tool. Keep that distinction in mind, since it’s exactly what the two examples below are built to demonstrate directly rather than just assert. Retrieval Augmented Generation (RAG) The scenario: an internal engineering team wants to ask natural-language questions against their incident runbooks and postmortems — documents that get added to and edited constantly as new incidents happen. This is a textbook RAG problem: the knowledge changes weekly, and every answer needs to be traceable back to a real source document for anyone debugging at 2 a.m. Prerequisites: Python 3.10+ pip install scikit-learn anthropic An Anthropic API key First, the documents themselves — a small but real set of runbooks and postmortems: # documents.py DOCUMENTS = [ { “id”: “runbook-db-failover-001”, “title”: “Database Failover Runbook”, “text”: ( “When the primary Postgres instance becomes unresponsive, first check “ “replication lag on the standby via `SELECT now() – pg_last_xact_replay_timestamp()`. “ “If lag is under 30 seconds, promote the standby using `pg_ctl promote`. “ “Update the connection string in the config service immediately after promotion. “ “Do not attempt manual failover if replication lag exceeds 5 minutes, escalate “ “to the database team instead, since promoting a stale standby risks data loss.” ), }, { “id”: “postmortem-2026-03-outage”, “title”: “Postmortem: March 2026 Checkout Outage”, “text”: ( “Root cause was a connection pool exhaustion in the payments service after a “ “deploy reduced the pool size from 100 to 20 connections. Fix was reverting the “ “pool size and adding a minimum-pool-size alert. Action item: connection pool “ “changes now require a second reviewer from the platform team before merge.” ), }, { “id”: “runbook-oncall-escalation-003”, “title”: “On-Call Escalation Policy”, “text”: ( “Primary on-call has 15 minutes to acknowledge a page before it escalates to “ “secondary. Secondary has 10 minutes before escalating to the team lead. Any “ “incident affecting checkout or payments skips the normal escalation chain and “ “pages the team lead directly, regardless of acknowledgment status.” ), }, # additional documents omitted here for length, full set in the shared code files ] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 # documents.py DOCUMENTS = [     {         “id”: “runbook-db-failover-001”,         “title”: “Database Failover Runbook”,         “text”: (             “When the primary Postgres instance becomes unresponsive, first check “             “replication lag on the standby via `SELECT now() – pg_last_xact_replay_timestamp()`. “             “If lag is under 30 seconds, promote the standby using `pg_ctl promote`. “             “Update the connection string in the config service immediately

In this article, you will learn what tool calling and code execution are as agent action primitives, how they differ mechanically, and when to choose one over the other. Topics we will cover include: How tool calling works under the hood, and why it remains the right choice for single, time-sensitive lookups. How code execution via Programmatic Tool Calling differs from standard tool calling, and what measurable benefits it offers for fan-out and aggregation tasks. A practical decision framework for choosing between the two primitives based on call count, data sensitivity, latency, infrastructure, and auditability needs. Picture an agent asking one simple-sounding question: which of twenty employees went over their Q3 travel budget. To answer it, the agent needs each person’s expense line items, every flight, hotel, and meal receipt, compared against a budget limit tied to their level. Built the obvious way, with the model calling a tool for each person’s expenses one at a time, that’s twenty separate tool calls, each returning fifty to a hundred line items, and every single one of those items has to pass through the model’s context just so it can be added up. That’s over 2,000 line items and more than 50KB of raw data the model never actually needed to read — it needed a sum. That’s the real cost hiding behind a design decision most agent tutorials skip past entirely: how does an agent actually take action in the world. There are two real answers, tool calling and code execution, and which one you reach for isn’t a style preference — it’s an architectural choice with measurable consequences for cost, latency, and accuracy. This article breaks down both action primitives for AI agents in detail, builds a real, runnable example of each using the same underlying tool, and closes with an honest, numbers-backed framework for choosing between them. If you haven’t built a basic tool-calling agent yet, check out this article, Easy Agentic Tool Calling with Gemma 4 — it is the natural place to start before this one. What Is an Action Primitive, and Why Does the Choice Matter? An action primitive is the fundamental mechanism by which a language model turns a decision into a real effect in the world — a database write, an API call, a file read. Every agent framework, whatever else it does, is built on top of one of these primitives at its core. Tool calling is the primitive most people learn first: the model produces one structured request at a time, a host application executes it, and the result comes back into the conversation before the model decides what to do next. Code execution is the newer alternative: instead of requesting one action and waiting, the model writes an actual program — in Python or TypeScript — that performs several actions in sequence or in parallel, and only the program’s final output returns to the model. Neither one is a wrapper around the other, and neither has quietly replaced the other. They’re genuinely different mechanisms with different failure modes, different infrastructure requirements, and different cost profiles, and the rest of this article is about understanding both well enough to pick correctly. Tool Calling It’s worth understanding what’s actually happening underneath a tool call, because the mechanics explain both its strengths and its real limitations. According to Cloudflare’s detailed breakdown of the process, a model generating a tool call doesn’t produce ordinary text. It’s been specifically trained to output a pair of special tokens — one signaling “the following is a tool call” and another marking its end — with a JSON payload describing the tool name and arguments sitting between them. The application running the model watches for those tokens, pauses generation the moment it sees the closing one, parses the JSON against a schema you defined, actually executes the call, and feeds the result back into the conversation as though it were the next thing the user said. That’s a clean, auditable, one-step-at-a-time loop, and it’s exactly why tool calling became the default. Every action is a discrete, loggable event. Every result is something the model directly sees and can reason about in natural language before deciding what happens next. Code Execution Code execution takes a different starting position entirely: instead of asking the model to describe an action in a constrained JSON format, you let it write actual code that performs the action, running in a sandboxed environment separate from the model itself. Anthropic’s original code-execution-with-MCP pattern frames this precisely as presenting your tools as a code API rather than a set of directly callable functions, so the model can write a script that imports exactly the tools it needs and calls them the way it would call any other function. The mechanism that makes this genuinely different — not just a relabeled tool call — is what Anthropic now calls Programmatic Tool Calling, released alongside two companion features in November 2025. Rather than each tool result flowing back through the model one at a time, you mark specific tools as callable from code by adding an allowed_callers field to their definition, and add a code_execution tool to the request. When the model wants to act, it writes a full script — loops, conditionals, error handling, and all — that calls those tools directly inside a sandboxed execution environment. Each individual tool call the script makes still executes exactly the way it would in ordinary tool calling; you still receive a request and return a result, but that result is intercepted and processed by the running script rather than being pushed into the model’s context. Only when the script finishes does its final output — and nothing else — return to the model. That’s the entire difference in one sentence: tool calling puts every intermediate result in front of the model; code execution lets the model decide, through the code it writes, exactly what makes it back. A side-by-side flow diagram of Tool Calling and Code Execution (click to

In this article, you will learn how to build a fully local, zero-cost agentic AI workflow using Hermes Agent and Ollama, so that your files, code, and conversations never leave your own hardware. Topics we will cover include: How to install Ollama, choose the right local model for agentic work, and verify that the model is responding correctly before wiring anything else up. How to configure Hermes Agent to use your local Ollama endpoint, and how to optimize context window size and model loading for real agentic tasks. How to extend the setup with a Telegram gateway for remote access and a cloud fallback for questions the local model cannot handle well. A typical coding session against a cloud AI API runs somewhere between $0.60 and $0.80 depending on the provider, and a heavier session can climb to $5 to $20, according to Nous Research’s own cost breakdown for agentic work. That adds up fast for a hobbyist, a student, or anyone running frequent automation, and it comes with a second cost that is easy to overlook: every file, every question, every line of code gets sent to a third party’s servers. This article builds the alternative: a genuinely local, zero-cost agentic AI workflow using Hermes Agent, an open-source AI agent from Nous Research, paired with Ollama for local model serving. What Is Hermes Agent? Hermes Agent is an open-source AI agent built by Nous Research, released under the MIT license and currently at version 0.21.1 as of this writing. It ships two ways: a native desktop app for macOS, Windows, and Linux, and a terminal-first CLI you install directly. What separates it from a basic chat interface is genuine agentic capability; it edits files, runs terminal commands, browses the web, and can delegate work to isolated sub-agents with their own conversations and tools. A few features matter specifically for this article. Persistent memory means Hermes learns your projects over time and can auto-generate reusable skills from how it solved past problems, rather than starting from zero every session. Its messaging gateway connects the same agent and the same memory to Telegram, Discord, Slack, WhatsApp, and email. And its sandboxing system supports five different isolation backends — local, Docker, SSH, Singularity, and Modal — so commands it runs do not have to touch your host system directly if you would rather they did not. What Is Ollama? Ollama is the layer underneath Hermes in this setup: a tool that downloads, serves, and manages open-weight language models directly on your own hardware, exposing them through a local API that looks and behaves like a standard cloud LLM endpoint. That last detail matters more than it sounds: because Ollama’s API is OpenAI-compatible at /v1/chat/completions, Hermes can talk to a model running entirely on your laptop using the exact same integration path it would use for a cloud provider like OpenAI or Anthropic — just pointed at localhost instead of the internet. The division of labor is clean: Ollama’s only job is running the model and answering requests for it. Hermes’ job is being the actual agent — deciding when to call a tool, editing a file, running a command, browsing the web, and interpreting what comes back. Neither one replaces the other, and this tutorial needs both. What We’re Building The concrete project for this article is a private, zero-cost local assistant that can organize and answer questions about a real folder of files on your machine, search the web when a question genuinely needs current information, and — once the core setup works — stay reachable from your phone via a Telegram bot when you are away from your desk. As a final layer, it will have a cloud fallback configured so genuinely hard questions still get answered well, while the other 90% of everyday use costs nothing and never leaves your machine. Every section from here builds one real piece of that project, in the order you would actually build it. What You Need Hardware requirements scale with the model you plan to run, and it is worth knowing both ends of the range before choosing. Component Minimum Recommended RAM 8 GB (for 3B models) 32+ GB (for 27B+ models) Storage 5 GB free 30+ GB (for multiple models) CPU 4 cores 8+ cores GPU Not required NVIDIA GPU with 8+ GB VRAM CPU-only setups genuinely work; they are just slower. A 9B model on a modern 8-core CPU runs at roughly 10 tokens per second, while a 31B model on CPU drops to about 2 to 5 tokens per second, meaning each response can take 30 to 120 seconds. That is usable for a background assistant, less pleasant for an interactive back-and-forth, which is worth factoring into which model you pick. Install Ollama and Pull a Model Install Ollama with its official install script: curl -fsSL https://ollama.com/install.sh | sh curl –fsSL https://ollama.com/install.sh | sh Confirm it is actually running: ollama –version curl http://localhost:11434/api/tags # Should return {“models”:[]} ollama —version curl http://localhost:11434/api/tags   # Should return {“models”:[]} Expected output: $ ollama –version ollama version is 0.33.2 $ curl http://localhost:11434/api/tags {“models”:[]} $ ollama —version ollama version is 0.33.2   $ curl http://localhost:11434/api/tags {“models”:[]} The first command checks that the binary is installed correctly. The second hits Ollama’s local API directly, and an empty models array is the expected, correct response at this point; it confirms the server is listening — you just have not downloaded a model into it yet. Now pull a model. This is the single most consequential choice in the whole setup, because not every model can actually act as an agent: Model Size on Disk RAM Needed Tool Calling Best For gemma4:31b ~20 GB 24+ GB Yes Best quality, strong tool use and reasoning gemma2:27b ~16 GB 20+ GB No Conversational tasks, no tool use gemma2:9b ~5 GB 8+ GB No Fast chat, Q&A, cannot call tools llama3.2:3b ~2 GB 4+ GB No Lightweight quick answers only That “Tool Calling” column is the whole ballgame for

In this article, you will learn how to treat prompt templates as tunable hyperparameters for a language model, using scikit-learn’s grid search to find the best-performing prompt for a zero-shot text classification task. Topics we will cover include: How to wrap a language model in a scikit-learn-compatible classifier that accepts interchangeable prompt templates. How to define a hyperparameter grid of candidate prompts and run cross-validated grid search over them. How to interpret the results to identify which prompt yields the highest classification accuracy. Let’s not waste any more time. Introduction In traditional machine learning, a common technique used by data scientists is hyperparameter optimization via search algorithms, such as grid search or random search. Their goal is to test different settings or configurations of machine learning models to find a combination of such settings (hyperparameters) that yields optimal model behavior, e.g. maximum accuracy. This article shows how to use the same approach to test natural language, treating prompt instructions as tunable hyperparameters — in other words, trying to determine which prompt for a language model works best. We will wrap the AI model in a custom container compatible with scikit-learn, allowing us to supply plug-in models with diverse prompt templates, automate the evaluation process, and score how well they classify text. A Complete Example, Step by Step For a smoother run of this code in your own machine or notebook environment, we will consider a couple of safeguards: We will load the AI model into memory only once before initiating the test, rather than loading it inside the testing loop. This will save plenty of execution time. We will use a hard formatting of the prompt as a “chat message”, making the AI lean towards instruction-following and question-answering, rather than assuming an otherwise default text completion task. Without further ado, it’s time to start by making the required imports for our code: import numpy as np from sklearn.base import BaseEstimator, ClassifierMixin from sklearn.model_selection import GridSearchCV from transformers import pipeline import numpy as np from sklearn.base import BaseEstimator, ClassifierMixin from sklearn.model_selection import GridSearchCV from transformers import pipeline Now we initialize the model, specifying a fast and free option like “Qwen/Qwen2.5-0.5B-Instruct”: generator = pipeline( “text-generation”, model=”Qwen/Qwen2.5-0.5B-Instruct” ) generator = pipeline(     “text-generation”,     model=“Qwen/Qwen2.5-0.5B-Instruct” ) Next, it’s time to define a custom class that inherits scikit-learn’s BaseEstimator and the ClassifierMixin to act as a zero-shot text classifier. In practice, this means no explicit training on a new dataset is needed to classify — just leveraging the knowledge in the chosen pre-trained model to infer the class (positive vs. negative). class ZeroShotPromptClassifier(BaseEstimator, ClassifierMixin): # 2. Passing the generator in as a parameter def __init__(self, generator, prompt_template=”Classify as positive or negative: {text}”): self.generator = generator self.prompt_template = prompt_template def fit(self, X, y=None): return self def predict(self, X): predictions = [] for text in X: prompt = self.prompt_template.format(text=text) # 3. Formatting as a chat message to force answering instead of auto-completing messages = [{“role”: “user”, “content”: prompt}] output = self.generator( messages, max_new_tokens=5, pad_token_id=self.generator.tokenizer.eos_token_id ) # 4. Extracting the assistant’s specific reply from the chat history reply = output[0][‘generated_text’][-1][‘content’].strip().lower() if “positive” in reply: predictions.append(“positive”) elif “negative” in reply: predictions.append(“negative”) else: predictions.append(“unknown”) return np.array(predictions) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 class ZeroShotPromptClassifier(BaseEstimator, ClassifierMixin):     # 2. Passing the generator in as a parameter     def __init__(self, generator, prompt_template=“Classify as positive or negative: {text}”):         self.generator = generator         self.prompt_template = prompt_template       def fit(self, X, y=None):         return self       def predict(self, X):         predictions = []         for text in X:             prompt = self.prompt_template.format(text=text)                          # 3. Formatting as a chat message to force answering instead of auto-completing             messages = [{“role”: “user”, “content”: prompt}]                          output = self.generator(                 messages,                 max_new_tokens=5,                 pad_token_id=self.generator.tokenizer.eos_token_id             )                          # 4. Extracting the assistant’s specific reply from the chat history             reply = output[0][‘generated_text’][–1][‘content’].strip().lower()                          if “positive” in reply:                 predictions.append(“positive”)             elif “negative” in reply:                 predictions.append(“negative”)             else:                 predictions.append(“unknown”)                      return np.array(predictions) Let’s briefly explain what the three methods inside the class do: __init__() initializes the classifier, integrating the text-generation model and the prompt template to use. fit() doesn’t perform any real action, as we are using a zero-shot classification approach that doesn’t require further training. Still, it needs to be explicitly defined inside the class. predict() is where the input texts are classified, generating model answers based on prompts and extracting sentiment polarity from the responses. The classifier is ready; now we need the ingredients: some data examples. Consider the following toy dataset containing reviews with different sentiments, and their associated class labels: X = np.array([ “I absolutely love this new feature!”, “This update completely broke my workflow.”, “Best user experience I have had all year.”, “Terrible customer service and slow load times.” ]) y = np.array([“positive”, “negative”, “positive”, “negative”]) X = np.array([     “I absolutely love this new feature!”,     “This update completely broke my workflow.”,     “Best user experience I have had all year.”,     “Terrible customer service and slow load times.” ]) y = np.array([“positive”, “negative”, “positive”, “negative”]) Another couple of key ingredients are an actual instance of our classifier and a hyperparameter grid containing the candidate prompt templates to test, which adopt the role of hyperparameter values: clf = ZeroShotPromptClassifier(generator=generator) # Prompt templates to test param_grid = { ‘prompt_template’: [ “Classify as positive or negative: {text}”, “Is the sentiment positive or negative? Text: {text}”, “Analyze this review. Output ‘positive’ or ‘negative’: {text}” ] } clf = ZeroShotPromptClassifier(generator=generator)   # Prompt templates to test param_grid = {     ‘prompt_template’: [         “Classify as positive or negative: {text}”,         “Is the sentiment positive or negative? Text: {text}”,         “Analyze this review. Output ‘positive’ or ‘negative’: {text}”     ] } Now it’s time to put it all together. The following code runs cross-validated grid search with cv=2 folds: enough for a tiny, four-sample dataset like ours. We call fit() on the search object to run the process of

What’s Actually Inside 24,723 Tokens of a Search Result? We Broke It Down, Field by Field – MachineLearningMastery.com What’s Actually Inside 24,723 Tokens of a Search Result? We Broke It Down, Field by Field – MachineLearningMastery.com