In this article, you will learn how to benchmark three text classification approaches — from a classical TF-IDF pipeline to a zero-shot large language model — to understand when each is most appropriate. Topics we will cover include: How to implement and evaluate a classical TF-IDF and logistic regression text classification pipeline. How to apply zero-shot classification using a transformer-based model (BART) and compare it against the classical baseline. How to use scikit-LLM with a Groq-hosted large language model for production-ready zero-shot classification with minimal code changes. Scikit-LLM vs. Traditional Text Classifiers: When Should You Use an LLM? Introduction In recent years, generative AI models like LLMs (large language models) have gradually taken over classical machine learning ones for addressing certain tasks, for instance, text classification. But the truth is: rather than having a one-beats-all solution, there are critical trade-offs developers need to face — should we stick with fast, battle-tested conventional models, invest in fine-tuning a transformer-based LLM, or perhaps leverage LLMs’ zero-shot reasoning potential? In this article, we will implement a benchmarking between three distinct approaches for text classification: TF-IDF and logistic regression (classic baseline). Zero-shot classification with BART: a deep learning, transformer-based standard architecture. Scikit-LLM with zero-shot classification: the most modern, prompt-based approach. The tutorial below is kept entirely free for everyone to try, with no costs or API rate limits. To do so, we will use scikit-LLM alongside a model available from Groq. You will need to register at Groq and obtain an API key for evaluating the third solution below. Implementing the Benchmarking First, we install all the core libraries we will need. !pip install scikit-learn transformers scikit-llm scikit-ollama pandas torch !pip install scikit–learn transformers scikit–llm scikit–ollama pandas torch For enabling reproducibility, we create a small, synthetic dataset containing customer support messages. The tickets are categorized into five classes. Once created, we store it in a DataFrame object and split it into training and test sets. import pandas as pd from sklearn.model_selection import train_test_split data = { “text”: [ # Technical “My screen is completely black and won’t turn on.”, “The app keeps crashing every time I click save.”, “The Wi-Fi module is failing to connect to the router.”, “Data sync isn’t working across my devices.”, “My bluetooth headphones won’t pair with the app.”, “I keep getting an Error 404 on the login screen.”, “The database connection timed out during the export.”, “API rate limit exceeded even though I haven’t used it.”, “Profile images won’t load on the dashboard.”, “The software installation failed at 99%.”, # Billing “I was charged twice this month, please fix this.”, “How do I update my credit card information?”, “My invoice for last month is missing from the portal.”, “The VAT calculation on my receipt is wrong.”, “My transaction was declined but I have funds.”, “Can I change my billing cycle from monthly to annual?”, “Where can I find my official receipt?”, “My saved credit card expired and I need to swap it.”, “I was overcharged on my last statement.”, “Please remove my saved payment method.”, # Account “My account is locked and I forgot my password.”, “How do I change the email address on my profile?”, “Please delete my account and all associated data.”, “I want to update my profile picture.”, “How do I enable two-factor authentication (2FA)?”, “I didn’t receive the email verification link.”, “Can I merge two different accounts into one?”, “Is there a way to change my username?”, “I need to transfer account ownership to my manager.”, “I am locked out because I lost my 2FA phone.”, # Sales “Do you offer enterprise discounts for large teams?”, “Do you have an annual plan with a discount?”, “Can you compare the pro and basic tiers for me?”, “What is the pricing for a 50-user bulk license?”, “Is there a student discount available?”, “Can I schedule a demo with your sales team?”, “Do you sell and ship to customers in Europe?”, “How does your partner and reseller program work?”, “What are the usage limits on the free tier?”, “I need a custom quote for a government contract.”, # Refund “Can I get a refund for my last purchase? It was a mistake.”, “I want my money back for the subscription.”, “Accidental purchase, please reverse the charge.”, “I am not satisfied with the product, need a refund.”, “Cancel my subscription immediately and refund me.”, “I was charged after my free trial ended.”, “I need a prorated refund for the remaining months.”, “What is your official refund policy?”, “I was promised a refund last week but haven’t received it.”, “The item arrived broken, I want a full refund.” ], “label”: [ “Technical”] * 10 + [“Billing”] * 10 + [“Account”] * 10 + [“Sales”] * 10 + [“Refund”] * 10 } df = pd.DataFrame(data) # Stratified train-test splitting ensures all 5 categories are proportionally represented in both subsets when the dataset is small X_train, X_test, y_train, y_test = train_test_split( df[“text”], df[“label”], test_size=0.3, random_state=42, stratify=df[“label”] ) print(f”Training rows: {len(X_train)} | Testing rows: {len(X_test)}”) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 import pandas as pd from sklearn.model_selection import train_test_split   data = {     “text”: [         # Technical         “My screen is completely black and won’t turn on.”, “The app keeps crashing every time I click save.”,         “The Wi-Fi module is failing to connect to the router.”, “Data sync isn’t working across my devices.”,         “My bluetooth headphones won’t pair with the app.”, “I keep getting an Error 404 on the login screen.”,         “The database connection timed out during the export.”, “API rate limit exceeded even though I haven’t used it.”,         “Profile images won’t load on the dashboard.”, “The software installation failed at 99%.”,         # Billing         “I was charged twice this month, please fix this.”, “How do I update my credit

In this article, you will learn how to transform a basic tool-calling script into a resilient agent that gracefully handles failures from misbehaving tools, malformed model outputs, and unavailable services. Topics we will cover include: How to structure an iterative agent loop with a safety cap on iteration count. The four distinct categories of failure an agent encounters when calling tools, and how to handle each one. How to design tool error messages that teach the model how to recover, reducing wasted iterations. Building a Multi-Tool Gemma 4 Agent with Error Recovery Introduction In a previous article, we wired up Gemma 4 to a handful of Python functions using Ollama’s tool-calling API. That gave us a working single-turn dispatcher: the model picks a tool, our code runs it, the model answers. It’s a useful starting point, but it’s a long way from an agent. One of the things that turns a tool-calling demo into an actual agent is how it handles things going wrong. Tools fail. The model hallucinates a function name, or passes a string where you wanted a number, or asks about a city your lookup table has never heard of. An upstream API times out. A required argument is missing. In the previous tutorial, any of these would either crash the script or get swallowed by a try/except that prints a message and gives up. That’s fine for a single path demo. It’s not fine for anything you’d want to leave running. This article rebuilds the agent around the assumption that things will go wrong, and shows how to recover gracefully when they do. The pattern is simple: catch errors at the boundary, convert them into messages the model can read, send them back to the model, and let the model decide whether to retry, route around the problem, or explain the failure to the user. We’ll also wrap everything in a proper iterative agent loop with a safety cap on iteration count. The full script can be found here. This article walks through the parts that matter. Rethinking the Tool Loop The original dispatcher ran a single round: send the user query, collect tool calls, run them, send the results back, print the model’s reply. That’s a one-shot interaction. It works fine when the model’s first response correctly answers the user’s question, but it has nowhere to go when something goes wrong. If a tool fails, the model gets one chance to react and then we’re done. If the model wants to call another tool after seeing the first result, too bad; we already exited. A proper agent loop is iterative. The structure is straightforward: Send the current message history to the model. If the model produces tool calls, execute each one, append every result to the history, and loop again. If the model produces a plain text response, that’s the final answer. Return. Cap the loop at MAX_ITERATIONS so a confused model can’t burn through your CPU forever. That last point is non-negotiable. Small models occasionally get stuck calling the same tool repeatedly, or oscillating between two tools, and there’s nothing more demoralizing than walking back to your terminal to find your laptop’s fans screaming because Gemma decided to look up the weather in London thirty times in a row. Here’s the loop: def run_agent(user_query): messages = [{“role”: “user”, “content”: user_query}] for iteration in range(1, MAX_ITERATIONS + 1): payload = { “model”: MODEL_NAME, “messages”: messages, “tools”: available_tools, “stream”: False, } print(f”[EXECUTION — iteration {iteration}]”) print(” ● Querying model…\n”) try: response_data = call_ollama(payload) except Exception as e: print(f” └─ [ERROR] Error calling Ollama API: {e}”) print(f” └─ Make sure Ollama is running and {MODEL_NAME} is pulled.”) return message = response_data.get(“message”, {}) tool_calls = message.get(“tool_calls”) or [] # Branch A: the model wants to use tools if tool_calls: print(f”[TOOL EXECUTION — {len(tool_calls)} call(s)]”) messages.append(message) tool_messages = print_tool_calls(tool_calls) messages.extend(tool_messages) print() continue # Branch B: the model produced a final answer print(“[RESPONSE]”) print(message.get(“content”, “”) + “\n”) return # Safety rail: we exhausted MAX_ITERATIONS without a final answer print(“[RESPONSE]”) print( f”Hit the {MAX_ITERATIONS}-iteration cap without a final answer. “ “This usually means the model is stuck in a tool-calling loop. “ “Try simplifying the query.\n” ) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 def run_agent(user_query):     messages = [{“role”: “user”, “content”: user_query}]       for iteration in range(1, MAX_ITERATIONS + 1):         payload = {             “model”: MODEL_NAME,             “messages”: messages,             “tools”: available_tools,             “stream”: False,         }           print(f“[EXECUTION — iteration {iteration}]”)         print(”  ● Querying model…\n”)           try:             response_data = call_ollama(payload)         except Exception as e:             print(f”  └─ [ERROR] Error calling Ollama API: {e}”)             print(f”  └─ Make sure Ollama is running and {MODEL_NAME} is pulled.”)             return           message = response_data.get(“message”, {})         tool_calls = message.get(“tool_calls”) or []           # Branch A: the model wants to use tools         if tool_calls:             print(f“[TOOL EXECUTION — {len(tool_calls)} call(s)]”)             messages.append(message)             tool_messages = print_tool_calls(tool_calls)             messages.extend(tool_messages)             print()             continue           # Branch B: the model produced a final answer         print(“[RESPONSE]”)         print(message.get(“content”, “”) + “\n”)         return       # Safety rail: we exhausted MAX_ITERATIONS without a final answer     print(“[RESPONSE]”)     print(         f“Hit the {MAX_ITERATIONS}-iteration cap without a final answer. “         “This usually means the model is stuck in a tool-calling loop. “         “Try simplifying the query.\n”     ) The pattern is worth committing to memory because it shows up in every agent framework you’ll ever read: the message history is the state. For each iteration we send the entire conversation (the original user query, the model’s tool-call request, our tool results, any follow-up model messages) back to the model. The model is stateless; the list is the agent’s memory. This iterative structure is also what makes error recovery possible. When a tool fails and we send the error back as a tool message, the model gets to see that error and react to it on the next

In this article, you will learn how logits, temperature, and top-p sampling work together to control next-token prediction in large language models. Topics we will cover include: What logits are and how they are produced by a transformer’s final linear layer. How temperature and top-p (nucleus sampling) shape the probability distribution used for token selection. How these three components fit into a sequential pipeline that governs LLM output generation. The Statistics of Token Selection: Logits, Temperature, and Top-P Walkthrough Introduction When large language models, or LLMs for short, produce outputs, several criteria are at stake, including not only overall response relevance but also coherence and creativity. Since deep inside the models operate by building their response word by word — or more precisely, token by token — capturing these desirable properties is a matter of mathematically adjusting the output probability distributions that govern the next-token prediction process. This article introduces the mechanics behind LLM decoding strategies from a statistical vantage point. In particular, we will explore how raw model scores, known as logits, interact with two other model settings — temperature and top-p — which are three key parameters utilized to control the token selection process. While we will focus on exploring what happens inside the very final stages of the LLMs’ underlying architecture, a.k.a. the transformer, you can check this article if you need a concise overview of the whole process and journey made by tokens from beginning to end. Token selection process in LLMs What Are Logits? In neural networks, the raw, unnormalized scores produced (typically at final linear layers) before converting them into probabilities of possible outcomes (e.g. classes) are known as logits. While logits have been used since the era of classical machine learning classification models like softmax regression, the same principle still applies to the final linear layer of transformer models. This final layer processes hidden states — which contain gradually accumulated linguistic knowledge about the input text gathered throughout the transformer — and outputs a vector of logits. How many? As many as the model’s vocabulary size, i.e. the number of possible tokens the model can generate. See the diagram at the top, for instance. If an LLM trained for English-to-Spanish translation is predicting the next word after the generated sequence “me gusta mucho” (the translation of “I really like to”), it might output a raw logit score of 12.5 for “viajar” (travel), 8.2 for “jugar” (play), and -3.1 for “dormir” (sleep). These raw values are unbounded, making them difficult to interpret directly; hence, a softmax function is applied on top of the final linear layer to transform these logits into a standard, interpretable probability distribution over vocabulary tokens, such that all values sum to 1. What Are Temperature and Top-p? Once we have a probability distribution over the target vocabulary, do LLMs simply choose the token with the highest probability as the next one to generate? Not exactly, but the true process closely resembles that scenario. The next token is sampled from the distribution, and how this sampling works depends on several decoding parameters, two of the most important being temperature and top-p. Temperature is a scaling factor applied to the logits before the softmax step. A high temperature (e.g. above 1) flattens the resulting probabilities, making them more uniform. As a result, uncertainty and unpredictability increase, and the model behaves more creatively. A low temperature (e.g. well below 1) sharpens the differences between high- and low-probability tokens, increasing certainty and strongly favoring the most likely tokens in the original distribution. More about temperature can be found in this related article. Top-p, also called nucleus sampling, is another approach to controlling the randomness of next-token selection. Rather than scaling probabilities, it limits the pool of candidates to sample from. While similar strategies like top-k consider only the k highest-probability tokens, top-p identifies the smallest set of tokens whose cumulative probability meets or exceeds a threshold p, making it more adaptive and flexible. In other words, if we set p=0.9, top-p sorts tokens by probability and keeps adding them to a candidate pool until their cumulative probability reaches 0.9. The Full Walkthrough: How Do These Concepts Relate to Each Other? Logit-to-probability calculation, temperature, and top-p can be combined into a sequential multi-step pipeline for producing LLM outputs, i.e. next-token predictions. First, the model generates raw logits for all possible tokens, as described above. Temperature then enters the picture by scaling these raw logits — note that this happens before the softmax function converts them into probabilities. Depending on the temperature value, the resulting distribution will look more uniform (high temperature, more uncertainty) or sharper (low temperature, higher certainty). Token selection walkthrough based on logits, temperature, and top-p Once the scaled logits are converted into probabilities, top-p is applied to filter the resulting distribution, calculating cumulative probabilities to retain only a core “nucleus pool” of the most likely tokens (see step 3 in the image above). Finally, the model samples randomly from within that pool to select the next token. Closing Remarks Now that we have demystified the statistical process behind token selection in LLMs, it is useful to consider how to choose values for temperature and top-p in practice. As a developer, you will want to define the right balance between predictability and creativity for your use case. For factual, high-stakes scenarios like coding or legal analysis, a low temperature and a stricter top-p are advisable — e.g. t=0.1 and p=0.5 — which yields highly deterministic model responses. For creative domains like poetry generation or brainstorming, a higher temperature and top-p, such as t=0.8 and p=0.95, allow for a richer variety of candidate tokens in the selection pool.