---
title: "DSPy — Automatic Prompt Generation"
url: "https://romankryvolapov.com/en/dspy/"
description: "Replacing hand-written prompts with DSPy signatures, a scoring metric and MIPROv2 tuning: when the LLM changes, the prompt is re-tuned instead of rewritten by hand."
language: en
published: 2025-09-22
updated: 2026-11-05
---
Many engineers working with LLMs and agents — and not just AI specialists — face tasks where the correctness of the agent's behavior largely depends on the prompt.

For simple tasks a prompt like "You are a smart agent, answer the user's questions" is enough. But if the agent has to make complex function calls and handle non-standard requests, writing a stable prompt for the calls and for the final answer is far from easy, and when you switch to a different LLM model, a prompt that used to work may stop working. How to write such prompts by hand is covered in the [article on prompt engineering](/en/prompt-engineering/), and how an agent calls tools in the [article on RAG and function calling](/en/rag-and-function-calling/).

[DSPy](https://dspy.ai) ([GitHub](https://github.com/stanfordnlp/dspy)) turns this from writing text into a measurable process. What it does, in simple terms:

- Instead of a long text you describe the task as a form: what inputs there are and what outputs are needed.
- The rules for calling tools are fixed in code, not in "words" inside the prompt.
- Quality is measured by a metric, and DSPy picks the wording to fit that metric.
- When you change the model or the functions, you run the tuning again instead of rewriting everything by hand.

What this gives you:

- Scalability: it is easier to add tools and new answer formats.
- A cheap model switch: it becomes a re-run of the tuning, with a score to compare before and after.
- Reproducibility: the prompt is stored as an artifact and can be restored.
- Control: behavior is defined by structure and a metric, not by "magic text".

The price is the data set and the time a tuning run takes; both are covered below. The example is an agent that searches a local database, to show that agents can be quite complex.

## Signature and tools

The signature says which fields arrive at the agent and what it must return. Tools are ordinary functions with clear inputs and outputs:

```python
import sqlite3
from contextlib import closing

import dspy

# A local model in LM Studio; any model LiteLLM can reach will do.
dspy.configure(
    lm=dspy.LM(
        "openai/google/gemma-3n-e4b",
        api_base="http://localhost:1234/v1",
        api_key="lm-studio",
        temperature=0,
    )
)


def search_kb(query: str) -> list[str]:
    """Searches the knowledge base and returns the matching fragments."""
    with closing(sqlite3.connect("kb.db")) as db:
        rows = db.execute(
            "SELECT text FROM articles WHERE text LIKE ? LIMIT 5", (f"%{query}%",)
        ).fetchall()
    return [row[0] for row in rows]


class AnswerFromKb(dspy.Signature):
    """Answer the question using only the knowledge base."""

    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="a short answer, no explanations")


agent = dspy.ReAct(AnswerFromKb, tools=[search_kb], max_iters=5)
```

## Data

The data set is a list of questions with correct answers:

```json
[
  {
    "question": "...",
    "answer": "..."
  }
]
```

According to the [DSPy documentation](https://dspy.ai/learn/optimization/overview/), 30 examples often bring substantial value, but you should aim for at least 300, and for MIPROv2 it [advises](https://dspy.ai/learn/optimization/optimizers/) 200 or more to prevent overfitting. What matters is that they are diverse.

The split is the opposite of what you are used to from neural networks: 20 percent for training and 80 for validation, because prompt optimizers easily overfit to a small training set. On top of that comes a held-out test set the optimizer never sees, and its score is the one to trust:

```python
import json
import random

with open("dataset.json", encoding="utf-8") as f:
    examples = [dspy.Example(**row).with_inputs("question") for row in json.load(f)]

random.Random(0).shuffle(examples)
holdout = len(examples) // 5
testset, rest = examples[:holdout], examples[holdout:]
split = len(rest) // 5
trainset, valset = rest[:split], rest[split:]
```

## Metric and tuning

MIPROv2 runs the agent to collect good traces, proposes variants of the instructions and few-shot examples, and searches among them by the metric:

```python
def metric(example, prediction, trace=None):
    return example.answer.lower() in prediction.answer.lower()


evaluate = dspy.Evaluate(devset=testset, metric=metric, num_threads=4)
print("before:", evaluate(agent))

optimizer = dspy.MIPROv2(metric=metric, auto="light", num_threads=4)
tuned = optimizer.compile(agent, trainset=trainset, valset=valset)

print("after:", evaluate(tuned))
tuned.save("agent.json")
```

A run is not cheap. With `auto="light"` this agent gets 20 trials on mini-batches of 35 examples, plus a full pass over up to 100 validation examples every 5 trials, and every agent run is several model calls. On a local model that is a long wait, on a cloud one a bill. The proposals can go to a stronger model than the one the agent runs on, through `prompt_model` and `task_model`.

## Using the result

`agent.json` is an artifact: it goes into the repository and loads without any tuning. When the data or the model changes, run the tuning again and compare the scores on the test set.

```python
agent = dspy.ReAct(AnswerFromKb, tools=[search_kb], max_iters=5)
agent.load("agent.json")
print(agent(question="How long is the refund period?").answer)
```