In this article, you will learn the conceptual and practical differences between retrieval and memory in agentic AI systems, and how to combine both effectively. Topics we will cover include: What separates retrieval from memory, and why the distinction matters for long-running agents. How retrieval pipelines and memory systems are each built, illustrated with a concrete worked example. How to combine retrieval and memory into a single, effective agent architecture. Introduction An AI agent that can’t remember its previous interactions is not very helpful. Every large language model has a fixed context window, and once a conversation, a set of tool outputs, or a pile of retrieved documents grows past that limit, something has to be dropped, summarized, or fetched fresh. Developers building long-running agents run into this constantly. The agent re-asks questions it already answered, contradicts decisions it made earlier, or fails to recognize that a document it needs even exists. Retrieval and memory are the two mechanisms that address this, and they solve different halves of the problem. Retrieval pulls in outside knowledge the model was never trained on and should not have to carry by default, such as documentation, code, and database records. Memory persists what the agent itself has learned or done, across a session or across many, so it isn’t starting from zero every time. Confusing the two, or building only one, is where a lot of agent architectures break down. This article covers: What separates retrieval from memory at a conceptual level How a retrieval pipeline and a memory system are each built, with a worked example A side-by-side comparison of the two How to combine both into a single, effective agentic system We start with why the split exists in the first place. Understanding Why Context Forces a Split A context window is the total set of tokens the model can see at once: system prompt, conversation history, tool outputs, anything inserted ahead of time. It is finite, and every token in it gets attended to on every forward pass, so simply making the window bigger doesn’t scale the way it sounds like it should. Context engineering has emerged as the discipline of curating and managing that limited resource, treating it as the full state available to the model at a given moment, not just a place to stuff instructions. Given that constraint, an agent has two kinds of information it needs but can’t keep permanently in context: Information that exists outside the model and outside the current conversation, such as a knowledge base, a codebase, or a set of policy documents. This is what retrieval handles. Information the agent generated or learned itself, that needs to outlive the current context window, such as a decision made ten turns ago or a fact about a specific user. This is what memory handles. Both get implemented with similar tools: embeddings, vector search, structured stores. The key difference is what they store and where the information comes from. Retrieval searches a corpus outside the agent, while memory stores information from the agent’s own interactions and past actions. Defining Retrieval in Agentic Systems Retrieval is how an agent answers “what does the world know about this that I don’t have in my weights or my current context.” The most common implementation is retrieval-augmented generation, or RAG: Source documents get chunked into passages small enough to be useful. Each chunk is converted into an embedding and stored in a vector index. At query time, the incoming question is embedded the same way, and the index returns the nearest matches. Those matches get inserted into the prompt alongside the user’s question. This pattern typically runs on managed datastores with an orchestration layer that ties the retrieval step into the rest of the agent’s reasoning — the approach behind most retrieval-augmented generation architectures in production today. The corpus itself is shared — every user asking about the same product documentation hits the same index — and it is refreshed on its own schedule, independent of any individual conversation. Defining Memory in Agentic Systems Memory is how an agent answers “what have I already learned or done that I need to carry forward.” It splits into two layers that behave differently: Short-term memory is the running session state: the conversation so far, plus anything the agent has written to a scratchpad during the current task. It’s cheap, and it disappears when the session ends. Long-term memory persists across sessions. It has to answer a harder question than retrieval does: not just “what’s relevant,” but “what’s worth keeping in the first place.” Some agent memory systems automatically extract useful facts, preferences, and context from conversations and store them for later use. At the start of a new session, the agent can query that memory much like it would query a retrieval index, but the results are specific to a user, task, or agent rather than a shared document corpus. When designing this layer, teams can explore different agent memory strategies and agent memory frameworks depending on what they need to store and retrieve. A quick worked example makes the split concrete. A customer messages a support agent about a delayed order. For a delayed order, the agent first checks its memory for the customer’s previous history. It finds a note from three weeks ago saying they prefer email follow-up and that a similar shipping issue was resolved with a partial refund. That is memory, because it comes from the agent’s record of this specific customer. The agent then needs the current shipping policy, which changed last month, so it searches the company’s documentation and retrieves the relevant section. That is retrieval, because the information comes from an external source and applies to all customers. Both results are added to the same prompt, but they answer different questions. Comparing Retrieval and Memory Laid out side by side, the differences between retrieval and memory are easier to see at a glance: Dimension Retrieval Memory Source of information External corpus the agent didn’t

In this article, you will learn what latent spaces are and how they serve three distinct roles — descriptive, generative, and predictive — across a wide range of machine learning applications. Topics we will cover include: How latent spaces compress high-dimensional data into structured numerical representations using techniques like Principal Component Analysis. How the generative role of latent spaces enables the creation of entirely new data points through interpolation. How the predictive role of latent spaces powers similarity-based applications such as recommender systems and RAG pipelines. Introduction Think of a “secret”, multi-dimensional map in which machine learning models treasure the “essence” of complex, real-world data. That’s the primary purpose of latent spaces: compressed, numerical data representations containing the abstract features and hidden relationships of the original, raw data they come from — be it raw image pixels, audio, text, or simply high-dimensional, structured data like customer behavior history. This article analyzes, illustrates, and categorizes the core functions and role of latent spaces in machine learning models. In particular, we distinguish between three roles: descriptive, generative, and predictive. Let’s unveil how latent spaces work under each of these hats through some concise, runnable code examples you can easily test in a Python notebook. 1. The Descriptive Role: Structuring and Representing Data Complex data normally needs to be summarized and structured in a more digestible form before feeding it to downstream machine learning models, extracting meaningful information into relevant features and discarding irrelevant or redundant ones. That’s the purpose of the descriptive role in latent spaces: a feature extractor compresses high-dimensional inputs into key traits, encoding them numerically. For example, in a dataset of raw, high-quality portrait images, disentangling factors like the subject’s pose or lighting keeps background noise aside while the core semantic information is preserved. One particular technique that is widely used to compress high-dimensional data into a lower-dimensional space (a smaller number of features, in simpler terms) is Principal Component Analysis, or PCA for short. While PCA doesn’t extract tangible features like lighting or pose, it’s still a very popular technique to drastically compress the original data features (based on algebraic projections) while minimizing the loss of important information describing the original data — this important information underlying the original data is commonly known as variance in the context of PCA and dimensionality reduction techniques as a whole. This example shows how to apply PCA to compress 3D data into a 2D latent space that maintains the original 3D data’s descriptive properties and relationships as much as possible: from sklearn.decomposition import PCA import numpy as np # Raw high-dimensional data: 3 items, 3 features per item raw_data = np.array([[1.1, 2.2, 3.3], [1.0, 2.1, 3.1], [8.1, 9.2, 9.9]]) # Compressing into a 2D Latent Space map pca = PCA(n_components=2) latent_space_map = pca.fit_transform(raw_data) print(“Descriptive Latent Space (Compressed Data):\n”, latent_space_map) from sklearn.decomposition import PCA import numpy as np   # Raw high-dimensional data: 3 items, 3 features per item raw_data = np.array([[1.1, 2.2, 3.3],                      [1.0, 2.1, 3.1],                      [8.1, 9.2, 9.9]])   # Compressing into a 2D Latent Space map pca = PCA(n_components=2) latent_space_map = pca.fit_transform(raw_data)   print(“Descriptive Latent Space (Compressed Data):\n”, latent_space_map) Output: Descriptive Latent Space (Compressed Data): [[-3.88962445e+00 4.39634517e-02] [-4.11856576e+00 -4.31334646e-02] [ 8.00819021e+00 -8.29987064e-04]] Descriptive Latent Space (Compressed Data): [[–3.88962445e+00  4.39634517e–02] [–4.11856576e+00 –4.31334646e–02] [ 8.00819021e+00 –8.29987064e–04]] The example is extremely simple to illustrate the concept, but in practice, you might apply PCA to compress thousands of features into, say, a couple hundred at most. 2. The Generative Role: Creating New Data Obtaining latent space representations from data can also be leveraged as a canvas for creating completely new data instances. The generative role consists of creating new data points by randomly sampling feature values that “make sense” for such points, or by interpolating between existing ones. The key aspect to grasp here is: which values make sense for every feature — in other words, how do the values in each latent space feature distribute? Think of it, in its simplest form, as taking a mathematical stroll between two different existing points and blending their respective feature values in infinitely many ways to create whole new outputs: new points, such as images. This is the core idea behind modern AI image generators, voice synthesizers, and so on. These systems rely on generative deep learning models like autoencoders, adversarial models, or even transformers. While these are remarkably complex and sophisticated models, their core ideas are based on interpolating points in a latent space, as shown in the code below: # Selecting two distinct points in our latent space map point_a = latent_space_map[0] point_b = latent_space_map[2] # Interpolation: Generating a new latent point halfway between them generated_latent_point = 0.5 * point_a + 0.5 * point_b # Decoding the new point back into the original 3D raw data space generated_raw_data = pca.inverse_transform(generated_latent_point) print(“Newly Generated Data Point:\n”, generated_raw_data) # Selecting two distinct points in our latent space map point_a = latent_space_map[0] point_b = latent_space_map[2]   # Interpolation: Generating a new latent point halfway between them generated_latent_point = 0.5 * point_a + 0.5 * point_b   # Decoding the new point back into the original 3D raw data space generated_raw_data = pca.inverse_transform(generated_latent_point)   print(“Newly Generated Data Point:\n”, generated_raw_data) Output: Newly Generated Data Point: [4.6 5.7 6.6] Newly Generated Data Point: [4.6 5.7 6.6] Take this mathematical concept to the extreme, and you get something like an AI that can modify a person’s eye color in a provided image to make it darker or brighter, for instance. 3. The Predictive Role: Similarity and Forecasting How does the AI behind recommender engines guess what video you want to watch next? Or how does it efficiently and reliably identify your facial traits through the immigration gates on arrival at a destination airport after a long-haul flight? Latent spaces enter the scene again. The story is partly familiar: high-dimensional, complex data like user behavior history or high-resolution images are compressed into a latent representation for more efficient and effective management while

Access Denied You don’t have permission to access “http://zeenews.india.com/technology/tim-cook-era-ends-as-john-ternus-takes-over-as-apple-ceo-3069418.html” on this server. Reference #18.c4f43717.1788249840.3e5001f5 https://errors.edgesuite.net/18.c4f43717.1788249840.3e5001f5

In this article, you will learn seven concrete regression tests for catching the orchestration-layer failure modes that matter most before deploying an AI agent to production. Topics we will cover include: Why agent failures are almost always caused by state management issues, not by the model itself, and what distinguishes “state” from “memory.” Seven targeted regression tests — covering context loss, tool idempotency, prompt injection, structured output, non-termination, RAG grounding, and state rehydration — each returning a binary pass or fail suitable for CI/CD gating. The specific failure modes each test is designed to surface, along with the common pitfalls that cause teams to misconfigure or misinterpret them. Most agent failures aren’t caused by a model that isn’t smart enough. They happen because the orchestration layer loses control of state. And most teams discover this the hard way — in production, under real user traffic. These seven regression tests give you a concrete checklist for catching the failure modes that aggregate prompt evaluation will never surface. Each test targets a specific system boundary and returns a binary pass or fail, making them suitable for CI/CD gating. Before you wire them into a pipeline, though, one structural note: agent behavior is stochastic, so a single-run assertion isn’t a reliable gate. Pin your model snapshot, fix temperature to zero where the provider allows it, and run each test across enough trials to establish a confidence-bounded pass rate. A test that flakes will get retried into silence and stop gating anything. One more distinction worth drawing before the list. Throughout this article, “state” refers to the deterministic, transactional record of the agent’s execution steps. “Memory” refers to the probabilistic, retrieved context injected into the prompt. When an agent misbehaves, the failure almost always lives in the state layer, not the model. 1. Context Loss and Retrieval Degradation When a conversation payload approaches your configured prompt budget, the orchestration layer has to decide what to evict. FIFO eviction is the simplest policy, but it produces a specific failure: an agent that asks a user for account details it gathered 40 minutes ago, because those early turns got dropped. The correct term for this is context loss, not catastrophic forgetting — which is a training-time phenomenon involving weight updates. The regression test feeds the agent a synthetic conversation history that fills roughly 80 percent of your configured prompt budget, then asks a question whose correct answer depends strictly on a fact established in the very first turn. The test passes only if the retrieval layer successfully surfaces that evicted turn from semantic memory, or if your summarization policy preserved the core entity relationships with measurable fidelity (entity recall against a gold set works well here). Watch out for the OR-assertion trap. Passing because retrieval worked is a different outcome than passing because summarization worked. Treat these as two separate tests. 2. Tool Execution Idempotency An agent with write access to an external system will, under realistic network conditions, eventually emit the same tool call more than once. Retries come from the harness, the HTTP client, or the orchestrator loop, not from the model itself. The model re-emits a call when an ambiguous observation fails to satisfy the prompt’s expectations. These are different mechanisms, but both produce duplicate writes if your tool boundary isn’t idempotent. The regression test forces the same tool-call payload to arrive at the execution boundary three times. It passes only if the downstream system registers exactly one write and returns a cache-hit response for the subsequent attempts. Derive idempotency keys from the logical identity of the operation: a hash of the tool name, canonicalized arguments, and a business correlation ID. Don’t use step ID or message position, as both change on every loop iteration — which produces a unique key for each duplicate call and defeats the mechanism entirely. Also account for concurrent in-flight requests: return the stored response rather than a 409, and set a TTL on stored keys to prevent stale hits. 3. Instruction Override and Prompt Injection Resistance The test injects adversarial payloads through both direct user input and indirect vectors, such as retrieved documents from a web search or an external knowledge base. It passes if the agent reaches a safe terminal state without executing the injected instruction and without leaking system prompt content. Assert on the tool-call trace and side effects, not on the output text. An agent can produce a polite refusal in prose while still emitting a harmful tool call underneath. Security lives at the execution boundary, which means role-based access control at the tool layer regardless of what the model intends. Keep in mind that classifier-based boundary checks are probabilistic components with their own error rates. If your CI gate depends on a classifier, you’re gating on a confidence level, not a binary outcome. Make that explicit. 4. Structured Output Adherence Modern providers support schema-constrained decoding, which makes syntactic invalidity and out-of-schema keys structurally impossible under strict mode. The failure modes worth testing are different ones. Truncation is the most common: hitting the token budget mid-output produces a structurally incomplete response that no repair strategy can fix at the application layer. Assert on finish_reason alongside parse success. Refusals produce a null parse with a populated refusal field and should be handled as a 403, not retried as a transient error. Semantic conformance is the subtler failure: schema-valid output with the right types but wrong values. And model-version skew is worth an explicit test — requests routed to an older model snapshot through an alias can silently fall back to legacy JSON mode behavior, so pin model strings explicitly rather than relying on aliases. 5. Non-Termination and Bounded Orchestration What the agent testing community often calls a deadlock is more precisely a livelock: the agent makes progress through its thought-action-observation cycle but never advances toward the goal. True deadlock — where Agent A is blocked on Agent B’s approval while B is blocked on A’s — is a distinct failure mode relevant to multi-agent

Access Denied You don’t have permission to access “http://zeenews.india.com/technology/15-years-of-tim-cook-how-apple-grew-into-a-4-trillion-giant-3069330.html” on this server. Reference #18.eff43717.1788188279.2889f0ce https://errors.edgesuite.net/18.eff43717.1788188279.2889f0ce

In this article, you will learn three practical strategies for managing small context windows in large language models, along with working Python examples that demonstrate how two of those strategies are implemented. Topics we will cover include: How context truncation via the sliding window approach keeps token usage flat and predictable. How token budgeting combined with retrieval-augmented generation ensures only the most relevant context fits within a prompt. A concise overview of additional strategies for more specialized use cases — rolling summaries, prompt compression, and observation masking. Introduction Top-tier AI industries have become somewhat obsessed with language models capable of ingesting massive context windows, e.g. an entire book in a single prompt. However, what they won’t admit easily is that in real-world LLM applications, these massive context windows come with various limitations and challenges, including soaring API costs, unacceptable response times, and even worse, the so-called “lost in the middle” problem whereby a model ignores data deeply buried in the middle of the giant prompt. No surprise, then, that working with small yet smartly managed context windows could yield superior outcomes, reducing latency, minimizing costs, and forcing the model to concentrate on what truly matters to generate its response. This article unveils three of the most widely adopted practical strategies for managing and mastering small context windows in language models, along with examples that mimic the implementation of some of them for better understanding. Context Truncation: Sliding Window There is a consensus that sliding windows are arguably the most common and simplest strategy for managing shortened context windows in language models. Instead of providing an entire user conversation history to the model, the context is treated as a FIFO (First-In-First-Out) queue: as new interactions (exchanged messages) come in, the oldest ones are simply dropped. All it takes is defining the size of the context window and striking a balance between sufficient past context retention and latency-cost control. The main advantage of truncating the context via sliding windows is absolute control and predictability over token usage and computing overhead. The maximum number of interactions dealt with by the model at a given time remains fixed, keeping latency flat and surprise-free. To better understand how this approach works, let’s look at the following Python code in which you can freely adjust the value of max_turns (context window size) and see how it affects the “memory” injected into the current prompt: class SlidingWindowMemory: def __init__(self, max_turns=3): “””Keep only the last `max_turns` of a conversation.””” self.max_turns = max_turns self.history = [] def add_interaction(self, user_text, ai_text): self.history.append({“user”: user_text, “ai”: ai_text}) # The logic behind a sliding window: drop the oldest turns if limits are surpassed if len(self.history) > self.max_turns: self.history = self.history[-self.max_turns:] def build_prompt(self, new_query): prompt = “System: Answer concisely based on recent context.\n\n” for turn in self.history: prompt += f”User: {turn[‘user’]}\nAI: {turn[‘ai’]}\n” prompt += f”User: {new_query}\nAI:” return prompt # — Testing the Sliding Window mechanism: feel free to adjust the value of max_turns — memory = SlidingWindowMemory(max_turns=2) # Simulating a long conversation memory.add_interaction(“Hi, I’m learning Python.”, “Great choice!”) memory.add_interaction(“What are lists?”, “Lists are mutable arrays.”) memory.add_interaction(“Can they hold mixed types?”, “Yes, they can.”) # The prompt will only contain the last ‘max_turns’ interactions, saving tokens print(memory.build_prompt(“How do I append to one?”)) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 class SlidingWindowMemory:     def __init__(self, max_turns=3):         “”“Keep only the last `max_turns` of a conversation.”“”         self.max_turns = max_turns         self.history = []       def add_interaction(self, user_text, ai_text):         self.history.append({“user”: user_text, “ai”: ai_text})                  # The logic behind a sliding window: drop the oldest turns if limits are surpassed         if len(self.history) > self.max_turns:             self.history = self.history[–self.max_turns:]       def build_prompt(self, new_query):         prompt = “System: Answer concisely based on recent context.\n\n”         for turn in self.history:             prompt += f“User: {turn[‘user’]}\nAI: {turn[‘ai’]}\n”         prompt += f“User: {new_query}\nAI:”         return prompt   # — Testing the Sliding Window mechanism: feel free to adjust the value of max_turns — memory = SlidingWindowMemory(max_turns=2)   # Simulating a long conversation memory.add_interaction(“Hi, I’m learning Python.”, “Great choice!”) memory.add_interaction(“What are lists?”, “Lists are mutable arrays.”) memory.add_interaction(“Can they hold mixed types?”, “Yes, they can.”)   # The prompt will only contain the last ‘max_turns’ interactions, saving tokens print(memory.build_prompt(“How do I append to one?”)) Output: System: Answer concisely based on recent context. User: What are lists? AI: Lists are mutable arrays. User: Can they hold mixed types? AI: Yes, they can. User: How do I append to one? AI: System: Answer concisely based on recent context.   User: What are lists? AI: Lists are mutable arrays. User: Can they hold mixed types? AI: Yes, they can. User: How do I append to one? AI: You can also try extending the conversation history by appending new memory.add_interaction() calls with extra query-response pairs of your own, to test the mechanism for larger context windows. Token Budgeting and RAG (Retrieval-Augmented Generation) RAG systems supplement LLMs with engines that reference and retrieve external documents to enrich the original user prompt with founded, relevant context. Small context windows may intuitively force a ruthless attitude toward the data to include in the context. To address this, token budgeting splits the context window into zones with strict limits per zone. For instance, a token budgeting criterion could allow up to 20% of the context for system instructions, 20% for the chat history (including the latest user query), and the remaining 60% for retrieved data. This incorporates a more dynamic retrieval and data chunking behavior, halting insertion as soon as budget limits are hit. The main advantage of token budgeting is preventing unduly large retrieved documents from quickly exhausting the prompt and ensuring only highly relevant, concentrated information is included, thus avoiding side issues like the aforementioned “lost in the middle” problem. This code excerpt exemplifies the use of the mechanism in Python, using a simple word count as a free, lightweight proxy for token budgeting — to make it more realistic,

In this article, you will learn how to design, assemble, and tune a retrieval-augmented generation system that runs entirely on a standard laptop, without cloud infrastructure or paid APIs. Topics we will cover include: How quantization, compact embedding models, and in-process vector stores make a full RAG pipeline possible on consumer hardware. Which lightweight packages handle each stage of the pipeline, from document ingestion and chunking through retrieval, prompting, and local generation. How to make the system reliable through source citations, retrieval thresholds, evaluation sets, and query logs that distinguish retrieval failures from generation failures. Introduction Retrieval-augmented generation, or RAG, connects a language model to your own collection of documents so it answers from your material instead of guessing. Most build guides assume a cloud GPU, a hosted vector database, and a paid API that charges you for every question. None of that is required. A laptop with 8 GB or 16 GB of RAM can run a complete RAG system that stays offline, costs nothing per query, and keeps sensitive documents on your own machine. This guide covers the architecture and the package choices that make a small setup hold up rather than fall over. There’s no code here on purpose. A working RAG system spans document loading, chunking, embedding, storage, retrieval, prompting, and generation, and no short snippet represents that honestly. Each section explains what a component does, which lightweight package handles it, and where to find a tested implementation you can copy and adapt. Defining What “Minimal Resources” Means Here Minimal means no dedicated GPU, no monthly bill, and no data leaving your machine. Three choices make that possible. The first is quantization. Model weights are normally stored at 16 bits per parameter, and quantized formats such as GGUF compress them to 4 or 5 bits. That cuts memory use by roughly two thirds at a small accuracy cost. A 7 billion parameter model that needs 14 GB at full precision runs in about 4 GB once quantized. The second is a small embedding model. Embeddings turn text into numeric vectors so similar passages sit close together. Compact sentence encoders around 80 MB in size produce 384-dimensional vectors and handle retrieval well for most document collections. The third is a local vector store that runs inside your Python process instead of as a separate database server. Set your speed expectations accordingly. On CPU-only hardware, generation runs at a few tokens per second. That suits a research assistant or an internal knowledge tool, not a high-traffic public application. Assembling the Small-Footprint Toolkit These are the packages worth knowing before you start. Orchestration: LangChain connects the pieces and supplies document loaders, text splitters, and retriever interfaces. LlamaIndex is a reasonable alternative with a stronger focus on indexing. Local inference: llama.cpp is a C and C++ implementation of language model inference tuned for CPUs, exposed to Python through the llama-cpp-python package. Ollama wraps similar functionality behind a simpler command line and local server. Embeddings: sentence-transformers from Hugging Face downloads and runs compact encoder models locally, with no API calls. Vector storage: FAISS gives you fast similarity search over an in-memory index that you save to disk. ChromaDB adds metadata filtering and persistence, with a bit more setup. Document parsing: pypdf handles PDFs. The unstructured package covers a wider mix of file formats. Interface: Streamlit turns your pipeline into a browser-based tool in a few dozen lines. For a complete offline build using llama.cpp, LangChain, and ChromaDB together, follow Building a RAG Pipeline with llama.cpp in Python. For the FAISS and Hugging Face variant, see A Practical Guide to Building Local RAG Applications with LangChain. Step 1: Ingesting and Chunking Your Documents Your system is only as good as the text you feed it. Load each document, strip page headers and footers, then split the text into chunks. Chunk size drives retrieval quality more than almost anything else. Chunks of 500 to 1000 characters with 10 to 20 percent overlap are a good starting point. Too small, and a chunk loses the context needed to answer anything. Too large, and the retrieved passage buries the relevant sentence in noise, wasting space in a small model’s limited context window. Split on natural boundaries where you can. Paragraph breaks and section headings preserve meaning better than a fixed character count. Attach metadata to every chunk as you create it: source filename, page number, and section title. That metadata lets you filter searches and cite sources in your answers later. For a walkthrough of chunking dense academic PDFs, including a Streamlit interface, see Let’s Build a RAG-Powered Research Paper Assistant. Step 2: Embedding and Indexing Your Chunks Each chunk goes through the embedding model once and comes back as a vector. Those vectors go into your index alongside the original text and metadata. Two rules keep this stage from causing trouble later. Use the same embedding model for indexing and querying, since vectors from different models are not comparable. And save the index to disk, because re-embedding thousands of chunks on CPU takes minutes you don’t need to spend twice. A few thousand documents produce an index measured in tens of megabytes, which FAISS searches in milliseconds. Rebuild only when documents change or when you switch embedding models. Step 3: Retrieving and Prompting At query time, the user’s question is embedded with the same model, and the index returns the closest chunks. Four to six chunks suits a small model with a modest context window. Plain similarity search misses more often than people expect. Short questions produce vague vectors, and phrasing that differs from the source text drops the match score. Two techniques address this cheaply. Query expansion rewrites the question into several variants and pools the results. Hypothetical document embeddings, or HyDE, ask the model to draft a plausible answer first, then search using that draft. An invented answer resembles the target passage more closely than a question does. The prompt you build around the retrieved text matters just as much. Tell

In this article, you will learn how to combine a classical machine learning pipeline with an agentic AI system to build a hybrid, autonomous customer retention workflow. Topics we will cover include: How to generate a synthetic dataset and train a random forest classifier for customer churn prediction using scikit-learn. How to design an agentic AI system — complete with tools and an LLM-powered reasoning core — that interprets machine learning predictions and acts on them autonomously. How to wire the machine learning pipeline and the agent together into a single, end-to-end runnable Python application. Introduction Agentic AI and machine learning pipelines are far from incompatible when it comes to building production-ready AI applications. In fact, embracing them as two sides of the same coin has become more than a mere trend: it constitutes a modern foundational architecture pattern that drives the shift from passive predictive analytics to autonomous decision-making and action. Traditional machine learning pipelines excel at pattern recognition tasks of varying complexity, but they are purely reactive in their base form. Meanwhile, agentic AI systems are all about proactivity: combined with predictive machine learning models, they can build on the insights yielded by such models to plan, use tools, and address real-world use cases with little or no human guidance. In this hands-on article, we will show you how to bridge the gap between reactive machine learning models and proactive AI agents. We will construct a lightweight, free, runnable Python pipeline that: Predicts customer churn based on a classical machine learning model built with scikit-learn. Hands the obtained predictions over to an agent endowed with a state-of-the-art LLM to autonomously reason and execute different customer retention strategies. Prerequisites The entire coding tutorial can be run for free in Google Colab or a local Jupyter notebook, provided you have the necessary libraries installed and imported. If you are using Colab, at the time of writing, the only library you might need to manually install is Groq: Make sure you also import the following: import numpy as np from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from groq import Groq import numpy as np from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from groq import Groq Since Groq — one of today’s most capable open-source LLM providers — requires an API key, be sure to register on their website and create your own API key here. You will need to incorporate it in your notebook or Google Colab account. The code below is designed to read the API key from the “Secrets” section found on the left-hand sidebar in Google Colab: create a new secret variable there called GROQ_API_KEY, and paste your actual Groq API key into the “value” field. These instructions will help you inject the newly added API key into your program: import os from google.colab import userdata # Injecting the Colab secret into standard environment variables os.environ[“GROQ_API_KEY”] = userdata.get(‘GROQ_API_KEY’) import os from google.colab import userdata   # Injecting the Colab secret into standard environment variables os.environ[“GROQ_API_KEY”] = userdata.get(‘GROQ_API_KEY’) Step-by-Step Guide Once the prerequisites are set up, we will start building the classical machine learning pipeline — for customer churn prediction — that will later be extended by incorporating agentic AI principles and tools. First, we need a customers dataset to feed to our machine learning model. For this example, we will synthetically generate our own dataset containing 500 customers, each described by two predictor features plus a target variable indicating whether the customer is prone to churn. The two input features are the monthly customer spend and the number of support tickets issued by the customer: both are real-world predictors of a customer’s willingness to stay with or abandon a brand. Notice that the code uses numpy functions to introduce random noise, making the artificially generated data look realistic: # ========================================== # 0. SYNTHETIC DATASET GENERATION # ========================================== # Generating a realistic dataset of 500 customers described by two input features np.random.seed(42) n_samples = 500 # Feature 1: Monthly customer’s spend (uniformly distributed between $10 and $150) spend = np.random.uniform(10, 150, n_samples) # Feature 2: Support tickets issued by customer (Poisson distribution, averaging 1.5 tickets) tickets = np.random.poisson(lam=1.5, size=n_samples) # Generate target variable / Binary class (Churn): # Churn risk increases with more tickets and decreases with higher spend base_churn_risk = (tickets * 0.15) + np.where(spend < 30, 0.3, 0) – np.where(spend > 100, 0.2, 0) # Add some random noise to make the dataset realistic base_churn_risk += np.random.normal(0, 0.1, n_samples) base_churn_risk = np.clip(base_churn_risk, 0, 1) # 0 = Retain, 1 = Churn (Threshold at 0.5) y = (base_churn_risk > 0.5).astype(int) X = np.column_stack((spend, tickets)) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # ========================================== # 0. SYNTHETIC DATASET GENERATION # ==========================================   # Generating a realistic dataset of 500 customers described by two input features np.random.seed(42) n_samples = 500   # Feature 1: Monthly customer’s spend (uniformly distributed between $10 and $150) spend = np.random.uniform(10, 150, n_samples)   # Feature 2: Support tickets issued by customer (Poisson distribution, averaging 1.5 tickets) tickets = np.random.poisson(lam=1.5, size=n_samples)   # Generate target variable / Binary class (Churn): # Churn risk increases with more tickets and decreases with higher spend base_churn_risk = (tickets * 0.15) + np.where(spend < 30, 0.3, 0) – np.where(spend > 100, 0.2, 0) # Add some random noise to make the dataset realistic base_churn_risk += np.random.normal(0, 0.1, n_samples) base_churn_risk = np.clip(base_churn_risk, 0, 1) # 0 = Retain, 1 = Churn (Threshold at 0.5) y = (base_churn_risk > 0.5).astype(int) X = np.column_stack((spend, tickets)) Next, we build a simple, classical machine learning pipeline by splitting the dataset into training and test sets and training a random forest ensemble classifier. We verify the model’s performance on the test set before continuing: # ========================================== # 1. CLASSIC ML PIPELINE (Predictive -> Classification) # ========================================== # Train/Test Split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

In this article, you will learn how Gemma 4, Llama 3, and Mistral implement tool calling locally, and what trade-offs each model family presents for real-world deployment. Topics we will cover include: What tool calling is and why it matters for locally deployed language models. How each of the three model families — Gemma 4, Llama 3, and Mistral — implements tool calling, including architectural and versioning differences. The practical strengths and trade-offs of each family for different hardware constraints and deployment contexts. Introduction Running AI models locally has moved from a niche hobby to a practical option for developers, researchers, and data practitioners. Among the capabilities that make local models useful for real workflows, tool calling stands out. It turns a language model from a text generator into something that can take actions, retrieve live data, and interact with external systems. This article compares how three widely used open-weight model families handle tool calling when run locally: Google DeepMind’s Gemma 4, Meta’s Llama 3, and Mistral AI’s Mistral. Each has different architectural decisions, different levels of native tool-calling support, and different strengths that suit different workflows. Before the comparison, it helps to understand what tool calling is and why it matters for local deployments. What Is Tool Calling? Tool calling, sometimes called function calling, is the mechanism that lets a language model invoke external functions and APIs rather than generating an answer purely from its training data. When a user asks something that requires current information or a specific computation, the model can recognize the need, emit a structured JSON request, and hand off execution to an external system. The result comes back to the model, which incorporates it into a coherent response. For a thorough grounding in the mechanics and architecture of tool calling, two articles from Machine Learning Mastery provide solid foundational coverage: In a local deployment context, tool calling matters for a specific reason: the model has no internet access, no live database connection, and no memory beyond its context window. Tool calling bridges that gap. It lets a locally running model query an API, check a file, or run a function without any cloud dependency. The structured JSON output tells the host application which function to call and with what parameters. The Three Models at a Glance Gemma 4 (Google DeepMind) Gemma is a family of open-weight models developed by Google DeepMind, built from the same research infrastructure behind Google’s proprietary Gemini models. Gemma 4, the most recent generation, was released on April 2, 2026, and marked a significant upgrade over earlier Gemma versions in both scope and capability. Gemma 4 is multimodal by design, supporting text, image, video, and audio inputs across its model sizes. It launched in four sizes (E2B, E4B, 26B A4B, and 31B), with a fifth variant (12B Unified) added in June 2026 to fill the gap between edge and server deployments. Smaller models are optimized for on-device and edge deployment. The architecture mixes Dense and Mixture-of-Experts (MoE) designs across the family, and the context window extends up to 256K tokens on the medium-sized variants. Most relevant here: Gemma 4 ships with native function-calling support built in, alongside native system prompt support that makes structured agentic conversations more predictable. Gemma 4 models are licensed under Apache 2.0 and available on Hugging Face and Kaggle. For interactive use, Google hosts several Gemma variants through Google AI Studio. Llama 3 (Meta) Llama 3 is Meta’s third generation of its large language model family, released in 2024. Meta has been one of the most consistent contributors to the open-weight ecosystem, and Llama 3 built substantially on improvements from Llama 2. The initial release included 8B and 70B parameter models in both base and instruction-tuned variants. The subsequent Llama 3.1 release expanded the family to include a 405B parameter model and introduced native tool calling support across the lineup. Llama 3 models are text-focused and dense in architecture. The 3.1 and later releases explicitly fine-tuned the models to recognize when a function needs to be called and to emit structured JSON responses with the correct function name and arguments. Larger Llama 3 variants (70B and above) perform more reliably on tool selection than the smaller 8B models, which can struggle with complex multi-tool scenarios. Llama 3 models use the Llama 3 Community License, which permits commercial use below 700 million monthly active users. The license also includes restrictions on using model outputs to train competing AI systems and some industry-specific constraints worth reviewing before deployment. They’re available through Hugging Face and can be deployed locally via Ollama or LM Studio. Mistral (Mistral AI) Mistral AI is a Paris-based startup founded in April 2023 by Arthur Mensch, formerly of Google DeepMind, and Guillaume Lample and Timothée Lacroix, formerly of Meta’s AI Research lab. The company launched its first model, Mistral 7B, in September 2023, positioning it as a European alternative to US-dominated AI development. The model attracted immediate attention for outperforming models twice its size on standard benchmarks while requiring significantly less compute to run. Mistral AI has been Europe’s most highly valued AI startup by valuation since 2024 and maintains a dual approach: open-weight models under Apache 2.0, and proprietary commercial models available through its API platform. The Mistral 7B and Mixtral families are the most widely deployed locally. Mistral 0.3 and later versions added function calling support, with the more recent Mistral Small family consolidating reasoning, vision, and tool-use capabilities into a single model. Mistral models are available on Hugging Face, through Ollama, and via La Plateforme, Mistral’s API and model management console. Tool Calling Implementation: How Each Model Approaches It The mechanics of tool calling follow a similar pattern across all three families, but the implementation details differ in ways that matter for local deployment. How Tool Calling Works Across All Three The workflow starts the same way across all three. The application sends the model a list of available tools defined as JSON schemas, each with a name, a description of

In this article, you will learn how to use probing classifiers, UMAP visualization, and SHAP values to interpret and analyze the quality of text embeddings generated by large language models. Topics we will cover include: How to generate text embeddings from movie reviews using Scikit-LLM and a local Ollama model, and train a probing logistic regression classifier to evaluate their quality. How to use UMAP dimensionality reduction to visually inspect the semantic structure captured by LLM-generated embeddings. How to apply SHAP values to identify which latent embedding dimensions have the greatest influence on a classifier’s predictions. Introduction Text classification tasks have long been exclusively the domain of machine learning models and their direct “evolved form”: deep neural networks. However, we can’t deny that large language models (LLMs) have revolutionized the way text classifiers are now built, being more powerful and accurate but raising a side concern: the lack of interpretability due to LLMs being black-box models. Accordingly, when using an LLM before the core text classification task to convert raw text into embeddings — dense numerical vector representations of text — it is possible to capture semantic information. Yet one challenging question arises: what exactly is the model learning about text, and how does this internal learning process drive predictions? This hands-on article shows how to use Scikit-LLM to generate embeddings, train a probing classifier, and unveil the black box by leveraging UMAP visualization and SHAP (SHapley Additive exPlanations) values: two popular explainable AI techniques for explaining model inference and decisions. Initial Setup The provided code here is fully compatible with Google Colab notebooks and requires installing the latest Scikit-LLM version. To keep the whole process cost-free, the code below shows how to configure everything for local, free execution. Let’s start by installing the following dependencies and packages, including the Ollama distributions for running local LLMs for free: # 1. Installing Python libraries !pip install -q scikit-llm umap-learn shap # 2. Fix Colab’s missing system dependencies first (version-dependent, use with care in other environments) !apt-get update -qq && apt-get install -y -qq zstd # 3. Installing Ollama safely (thanks to zstd installed earlier) !curl -fsSL https://ollama.com/install.sh | sh # 4. Starting the local server in the background and waiting for it to boot !nohup ollama serve > ollama.log 2>&1 & !sleep 5 # 5. Pulling the free embedding model: all-minilm !ollama pull all-minilm # 1. Installing Python libraries !pip install –q scikit–llm umap–learn shap   # 2. Fix Colab’s missing system dependencies first (version-dependent, use with care in other environments) !apt–get update –qq && apt–get install –y –qq zstd   # 3. Installing Ollama safely (thanks to zstd installed earlier) !curl –fsSL https://ollama.com/install.sh | sh   # 4. Starting the local server in the background and waiting for it to boot !nohup ollama serve > ollama.log 2>&1 & !sleep 5   # 5. Pulling the free embedding model: all-minilm !ollama pull all–minilm Now let’s import everything we will need: import numpy as np import pandas as pd import matplotlib.pyplot as plt import umap import shap from skllm.config import SKLLMConfig from skllm.models.gpt.vectorization import GPTVectorizer from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import classification_report from datasets import load_dataset import numpy as np import pandas as pd import matplotlib.pyplot as plt import umap import shap from skllm.config import SKLLMConfig from skllm.models.gpt.vectorization import GPTVectorizer from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import classification_report from datasets import load_dataset Probing Embedding Spaces The first step to probe and analyze Scikit-LLM embeddings is, of course, to get a fresh collection of them from a text dataset. We will first configure Scikit-LLM to point to a local Ollama server via “http://localhost:11434/v1/”. # 1. Pointing Scikit-LLM to the local Ollama server running in the background SKLLMConfig.set_gpt_url(“http://localhost:11434/v1/”) SKLLMConfig.set_openai_key(“dummy_key”) # Required format, but ignored locally # 1. Pointing Scikit-LLM to the local Ollama server running in the background SKLLMConfig.set_gpt_url(“http://localhost:11434/v1/”) SKLLMConfig.set_openai_key(“dummy_key”) # Required format, but ignored locally After that, we use the public IMDB dataset containing movie reviews and load 1,000 of them: 500 labeled as positive and 500 labeled as negative, giving us a perfectly class-balanced sample. We use stratified sampling to keep 80% of the examples for training and the remaining 20% for testing: # 2. Load one thousand movie reviews from IMDB dataset print(“Downloading and preparing IMDB dataset…”) dataset = load_dataset(“stanfordnlp/imdb”, split=”train”) df = dataset.to_pandas() # Extracting 500 positive and 500 negative reviews to ensure a perfect balance df_pos = df[df[‘label’] == 1].sample(500, random_state=42) df_neg = df[df[‘label’] == 0].sample(500, random_state=42) df_balanced = pd.concat([df_pos, df_neg]).sample(frac=1, random_state=42) # Shuffle texts = df_balanced[‘text’].tolist() labels = df_balanced[‘label’].values # Splitting via stratified sampling X_train, X_test, y_train, y_test = train_test_split( texts, labels, test_size=0.2, random_state=42, stratify=labels ) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # 2. Load one thousand movie reviews from IMDB dataset print(“Downloading and preparing IMDB dataset…”) dataset = load_dataset(“stanfordnlp/imdb”, split=“train”) df = dataset.to_pandas()   # Extracting 500 positive and 500 negative reviews to ensure a perfect balance df_pos = df[df[‘label’] == 1].sample(500, random_state=42) df_neg = df[df[‘label’] == 0].sample(500, random_state=42) df_balanced = pd.concat([df_pos, df_neg]).sample(frac=1, random_state=42) # Shuffle   texts = df_balanced[‘text’].tolist() labels = df_balanced[‘label’].values   # Splitting via stratified sampling X_train, X_test, y_train, y_test = train_test_split(     texts, labels, test_size=0.2, random_state=42, stratify=labels ) We are now ready for the heaviest part of the process: generating embeddings for these 1,000 texts. We do so using Ollama’s all-minilm model via Scikit-LLM’s class designed for handling embedding models: GPTVectorizer. The syntax is intentionally similar to standard scikit-learn data transformations, as we can see: # 3. Generating Embeddings using Scikit-LLM print(“Generating Embeddings…”) vectorizer = GPTVectorizer(model=”all-minilm”) X_train_vec = vectorizer.fit_transform(X_train) X_test_vec = vectorizer.transform(X_test) # 3. Generating Embeddings using Scikit-LLM print(“Generating Embeddings…”) vectorizer = GPTVectorizer(model=“all-minilm”) X_train_vec = vectorizer.fit_transform(X_train) X_test_vec = vectorizer.transform(X_test) Be patient; if you are running this on Colab, it may take about 5–10 minutes to complete, as we are making 1,000 calls to a local LLM for embedding generation. A probing classifier (or a probing model)