In this article, you will learn how to use a pre-trained large language model to extract structured features from text and combine them with numeric columns to train a supervised classifier. Topics we will cover include: Creating a toy dataset with mixed text and numeric fields for classification Using a Groq-hosted LLaMA model to extract JSON features from ticket text with a Pydantic schema Training and evaluating a scikit-learn classifier on the engineered tabular dataset Let’s not waste any more time. From Text to Tables: Feature Engineering with LLMs for Tabular DataImage by Editor Introduction While large language models (LLMs) are typically used for conversational purposes in use cases that revolve around natural language interactions, they can also assist with tasks like feature engineering on complex datasets. Specifically, you can leverage pre-trained LLMs from providers like Groq (for example, models from the Llama family) to undertake data transformation and preprocessing tasks, including turning unstructured data like text into fully structured, tabular data that can be used to fuel predictive machine learning models. In this article, I will guide you through the full process of applying feature engineering to structured text, turning it into tabular data suitable for a machine learning model — namely, a classifier trained on features created from text by using an LLM. Setup and Imports First, we will make all the necessary imports for this practical example: import pandas as pd import json from pydantic import BaseModel, Field from openai import OpenAI from google.colab import userdata from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import classification_report from sklearn.preprocessing import StandardScaler import pandas as pd import json from pydantic import BaseModel, Field from openai import OpenAI from google.colab import userdata from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import classification_report from sklearn.preprocessing import StandardScaler Note that besides common libraries for machine learning and data preprocessing like scikit-learn, we import the OpenAI class — not because we will directly use an OpenAI model, but because many LLM APIs (including Groq’s) have adopted the same interface style and specifications as OpenAI. This class therefore helps you interact with a variety of providers and access a wide range of LLMs through a single client, including Llama models via Groq, as we will see shortly. Next, we set up a Groq client to enable access to a pre-trained LLM that we can call via API for inference during execution: groq_api_key = userdata.get(‘GROQ_API_KEY’) client = OpenAI( base_url=”https://api.groq.com/openai/v1″, api_key=groq_api_key ) groq_api_key = userdata.get(‘GROQ_API_KEY’) client = OpenAI(     base_url=“https://api.groq.com/openai/v1”,     api_key=groq_api_key ) Important note: for the above code to work, you need to define an API secret key for Groq. In Google Colab, you can do this through the “Secrets” icon on the left-hand side bar (this icon looks like a key). Here, give your key the name ‘GROQ_API_KEY’, then register on the Groq website to get an actual key, and paste it into the value field. Creating a Toy Ticket Dataset The next step generates a synthetic, partly random toy dataset for illustrative purposes. If you have your own text dataset, feel free to adapt the code accordingly and use your own. import random import time random.seed(42) categories = [“access”, “inquiry”, “software”, “billing”, “hardware”] templates = { “access”: [ “I’ve been locked out of my account for {days} days and need urgent help!”, “I can’t log in, it keeps saying bad password.”, “Reset my access credentials immediately.”, “My 2FA isn’t working, please help me get into my account.” ], “inquiry”: [ “When will my new credit card arrive in the mail?”, “Just checking on the status of my recent order.”, “What are your business hours on weekends?”, “Can I upgrade my current plan to the premium tier?” ], “software”: [ “The app keeps crashing every time I try to view my transaction history.”, “Software bug: the submit button is greyed out.”, “Pages are loading incredibly slowly since the last update.”, “I’m getting a 500 Internal Server Error on the dashboard.” ], “billing”: [ “I need a refund for the extra charges on my bill.”, “Why was I billed twice this month?”, “Please update my payment method, the old card expired.”, “I didn’t authorize this $49.99 transaction.” ], “hardware”: [ “My hardware token is broken, I can’t log in.”, “The screen on my physical device is cracked.”, “The card reader isn’t scanning properly anymore.”, “Battery drains in 10 minutes, I need a replacement unit.” ] } data = [] for _ in range(100): cat = random.choice(categories) # Injecting a random number of days into specific templates to foster variety text = random.choice(templates[cat]).format(days=random.randint(1, 14)) data.append({ “text”: text, “account_age_days”: random.randint(1, 2000), “prior_tickets”: random.choices([0, 1, 2, 3, 4, 5], weights=[40, 30, 15, 10, 3, 2])[0], “label”: cat }) df = pd.DataFrame(data) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 import random import time   random.seed(42) categories = [“access”, “inquiry”, “software”, “billing”, “hardware”]   templates = {     “access”: [         “I’ve been locked out of my account for {days} days and need urgent help!”,         “I can’t log in, it keeps saying bad password.”,         “Reset my access credentials immediately.”,         “My 2FA isn’t working, please help me get into my account.”     ],     “inquiry”: [         “When will my new credit card arrive in the mail?”,         “Just checking on the status of my recent order.”,         “What are your business hours on weekends?”,         “Can I upgrade my current plan to the premium tier?”     ],     “software”: [         “The app keeps crashing every time I try to view my transaction history.”,         “Software bug: the submit button is greyed out.”,         “Pages are loading incredibly slowly since the last update.”,         “I’m getting a 500 Internal Server Error on the dashboard.”     ],     “billing”: [         “I need a refund for the extra charges on my bill.”,         “Why