I Asked My Recipe RAG for Dinner Without Milk. It Found Milk Pudding

MealRAG began as a playground. My initial goal was to work on retrieval itself: inspect which recipes a query brought back, try different retrieval methods, and benchmark them against the same evaluation set. I assembled that set, and the experiments and evaluation work are still ongoing. Recipes seemed perfect for it: I could ask for something vague, like a “quick lazy dinner,” or state a concrete restriction, like “no milk,” and see what the search did with each. I expected to learn something about ranking. Along the way, another question became interesting enough to deserve its own article: where, exactly, does the meaning of a request get handled in a RAG system?

Recipes gave me a familiar testbed where I could inspect a result and understand why it missed the request. The broader problem is how a retrieval system connects a person’s intent to the information and operations available to it. Food makes that problem concrete; similar questions arise when searching products, documentation, or an internal knowledge base.

The first experiment on my list was to add a reranker: retrieve a set of recipes with vector search, then use a second model to reorder those candidates. I had not reached that step when the baseline search began raising questions about which parts of a request similarity ranking could handle. A side experiment with an agent then widened the question to the rest of the system. This article follows that detour from the original retrieval benchmark. The broader comparison of retrieval methods and evaluation results can have its own space later.

I collected 719 recipes from TheMealDB. Each became one Markdown document with YAML metadata. I kept recipes whole rather than splitting them into chunks: their titles, ingredients, and instructions are short enough to travel together.

For the embedding model, I chose BGE-M3 because I wanted to run it locally through Ollama. It turns text into a numerical representation that can be compared with representations of other texts. BGE-M3 is a multilingual model capable of dense, sparse, and multi-vector retrieval; this experiment used only its dense embeddings. They have 1,024 dimensions. The model has other retrieval modes I could explore later, but they were not involved in the results below.

I stored those vectors in Qdrant, a database built to retrieve nearby vectors. In my collection, each recipe was a point with a vector and a payload of recipe data; Qdrant ranked points by cosine similarity to the query vector. This let me ask a question in ordinary language, embed it with the same model, and inspect the nearest recipes. Qdrant can also filter by payload fields — a capability I would only appreciate after seeing where similarity search failed.

I did not embed the raw Markdown file. I assembled labeled text from its title, cuisine, category, measured ingredients, and instructions. A Poutine document, for example, produced text shaped like this:

Title: Poutine
Cuisine: Canadian
Category: Miscellaneous

Ingredients:
Dash Vegetable Oil
1 Can Beef Gravy
5 thin cut Potatoes
2 cups Cheese Curds

Instructions:
Heat oil in a deep fryer or deep heavy skillet...

The Qdrant payload held the title, source, tags, original ingredients and instructions, plus a separate ingredient_names array. At the time, I mostly cared about the vector. I would come back to that array much later.

First, the search was delightful

I started by querying Qdrant directly, without an LLM in front of it. curry returned curries. “Something vegan but tasty” returned vegan and vegetarian dishes. Fine, but those results did not tell me much about what the model had learned. Then I tried a less literal request:

I want a quick lazy dinner

1. Drunken noodles (pad kee mao)  0.576
2. Quick gazpacho                0.572
3. Lamb Tzatziki Burgers        0.571
4. Lancashire hotpot            0.561
5. Vegetarian Casserole         0.557

Drunken Noodles came first. That made me pause. The recipe never calls itself “lazy”; the word does not appear in the text I embedded. But the instructions describe a brief stir-fry: garlic and chilli for 20 seconds, steak and broccoli for about a minute each, then noodles and sauce for another minute or two. There is preparation before those steps, so the data do not prove a total cooking time. Still, it was a plausible answer to a very human, imprecise request.

This was the fun part of experimenting with embeddings: a result can feel as if the database has understood the mood of a question. I wanted to find the edge of that ability. What would happen if I told it what I didn't want?

Then it found milk pudding

I tried a dinner request for someone who is lactose-intolerant. Qdrant returned this:

Fancy dinner for lactose-intolerant

1. Firni                         0.603
2. Flamiche                      0.583
3. Lancashire hotpot             0.573
4. Keto Banitsa                  0.563
5. Flija Layered Pancake / Crepe 0.558

I opened the first recipe. Firni is a milk pudding. Its ingredient list begins with six cups of whole milk. That was the moment the experiment became much more interesting. The model had found something semantically close to part of my request, yet missed the part that mattered most.

Perhaps “lactose-intolerant” was just the wrong wording? I tried variants including dairy-free, no milk, and without milk, cheese, cream or butter. Milk-heavy recipes kept appearing. This was not a matter of finding the magic synonym that would make an ordinary similarity search obey a prohibition.

Lactose-free, dairy-free, and “no milk” are not interchangeable dietary requirements. I am using these searches to examine how retrieval handles exclusions, not to judge whether a recipe is safe for a particular person.

I then put a positive food category next to the negative constraint. The two searches below show why that was so intriguing (scores omitted for readability):

fast dish no milk
1. Firni
2. Dulce de Leche
3. Jamaican Instant Pot Rice and Beans
4. Drunken noodles (pad kee mao)
5. Leche Asada

fast seafood dish no milk
1. Seafood rice
2. Seafood fideuà
3. Fish pie
4. Nasi lemak
5. Meang Nem

With fast dish no milk, the top two results are milk desserts. Add one word — seafood — and the list starts with seafood. It looked as if the positive topic gave ranking a direction, while no milk failed to act as a gate. That is an observation from these two searches, not a measured rule about BGE-M3. I also did not inspect every ingredient in the second list, so I cannot claim it satisfies the exclusion. The striking change is in what the search seems to optimize for.

To make sense of that result, I had to separate two tasks that sound similar in natural language. A dense bi-encoder turns both the query and the document into vectors, then ranks by proximity. If a query mentions milk, a recipe about milk may be close even when the user said “no milk.” Negation and exclusion are known difficulties in neural retrieval: researchers have built dedicated benchmarks and training sets, including NevIR and ExcluIR, to study them. Training for these cases can help. It still does not turn vector similarity into a hard exclusion rule.

That gave me a more useful way to read the results. The embeddings were doing real work on the kind of dish I asked for. I had also asked them to do a different job: enforce a condition. The next question was whether an agent could bridge that gap.

Watching the agent search

I already had an agent built with the Claude Agent SDK ready to try, so exposing the same Qdrant search as an MCP tool was an easy experiment. I was curious what would happen when a model could write its own retrieval queries, inspect candidates, and try again. Having that agent available is what took me down this branch of the project.

This is an agentic RAG setup: the model can decide how to use retrieval as it works toward an answer. It is one way to build RAG; a system can also follow a fixed pipeline that retrieves context and passes it to a model for generation. My original goal was to benchmark retrieval, and the agent gave me another way to explore its behavior. I had not yet compared those architectures or chosen an agentic approach as the best fit for MealRAG.

The initial tool description was minimal: “Search the recipes database for relevant information.” The tool took a natural-language query and optional search settings, embedded the query with BGE-M3, and returned candidates with their recipe payloads.

I asked for “some fancy dish” while saying I was lactose-intolerant. Claude made three searches:

dairy-free fancy elegant dish
seafood dish olive oil garlic no dairy
elegant dinner party main course dairy free

The sequence was fascinating to watch. Claude changed the positive direction from a general elegant dish, to seafood, to a dinner-party main course. But every search still carried dairy-free or no dairy in the free-text query. It had another move available — search again — yet it was still using the same weak mechanism for the exclusion. I did not retain the candidate lists for this run, so I cannot say which particular result caused each reformulation. The search calls themselves show the pattern.

I became curious about another question: what would Claude search for if I supplied almost nothing except the constraint? For “I want a dinner for lactose intolerant,” it tried:

lactose-free dairy-free dinner
chicken dinner no milk no cheese no cream
dairy free vegetable stir fry dinner

I had not asked for chicken or a vegetable stir-fry. Claude was testing plausible directions, and its final answer offered several options rather than forcing me into one. That is reasonable behavior, but the log lets us see a decision that is usually hidden: a constraint narrows the valid set without saying which valid dish should rank first. To make the search useful, the agent began supplying positive themes of its own.

At this point I stopped treating the search queries as isolated prompt mistakes. What had I actually told Claude about the tool? It knew that the tool searched recipes. It did not know which kinds of requests the search was good at, or which kinds it could not be trusted to enforce.

I changed the description, then ran the same request again

I put the lesson from the direct Qdrant searches into both the MCP server instructions and the tool description. The important part said:

The search works best with positive descriptions of the desired food, such as ingredients, cuisine, dish type, style, texture, occasion, or qualities like “quick”, “filling”, “light”, “spicy”, or “fancy”.

Negative constraints such as “no dairy”, “without nuts”, “gluten-free”, or “not spicy” are NOT reliably handled by vector similarity. Including excluded ingredients in the search query may actually retrieve recipes containing those ingredients.

I also told Claude to use positive preferences for retrieval, ask for a reasonably broad set, and inspect candidates against negative constraints. A high similarity score, the description warned, was not evidence of dietary compliance. I had changed no embedding model, no database, and no recipe corpus. Would the agent behave differently just because it now knew more about its tool?

I ran the same user request before and after this change: “Suggest good, filling dish for lactose intolerant.” The saved agent logs preserve the search calls and final answers. To recover the candidate lists, I reran both queries against the current index with a limit of ten. These are replayed search results, rather than archived tool responses from the original agent runs. The current MCP implementation defaults to ten results when limit is omitted; the command-line search defaults to five, so reproducing the tool search there requires an explicit --limit 10.

With the short description, the visible search call was:

{"query":"filling lactose-free dinner dish"}

Replaying that query returned:

1. Firni                                                    0.572
2. Chicken Shawarma with homemade garlic herb yoghurt sauce 0.561
3. Stuffed Lamb Tomatoes                                    0.557
4. Meang Nem                                                0.556
5. Fish pie                                                 0.556
6. Flija Layered Pancake / Crepe                             0.554
7. Three Fish Pie                                           0.553
8. Egyptian Fatteh                                          0.553
9. Lamb Biryani                                             0.552
10. Full English Breakfast                                  0.552

Claude recommended Egyptian Fatteh, describing it as a filling dish that “needs only one easy swap.” It proposed replacing the butter used in the recipe. It also offered Firni as a dessert option, with the milk replaced. In the replay, Fatteh ranks eighth and Firni first: the agent’s main recommendation differs from the vector ranking’s top result. The milk pudding had returned, this time as something the agent could adapt.

This was not a useless answer: the main recommendation matched the request for a substantial meal. The word filling, supplied by the user, gave the query a positive direction, much as seafood had in the earlier searches. That is a plausible explanation for the more useful recommendation, though this comparison does not isolate the effect of that word. The answer still relied on changing a listed ingredient to meet the constraint as Claude interpreted it.

With the expanded description, the visible search call was:

{"query":"hearty filling dish for dinner, protein and vegetables","limit":10}

The exclusion had disappeared from the similarity query, and Claude explicitly requested ten candidates. The replay returned:

1. Chicken Marengo           0.603
2. Creamy Mustard Chicken    0.598
3. Jiggs Dinner              0.597
4. Chicken Alfredo Primavera 0.594
5. Fish pie                  0.591
6. Moussaka                  0.591
7. Chicken Parmentier        0.590
8. Egyptian Fatteh           0.586
9. Beef Wellington           0.586
10. Creamy Tomato Soup       0.585

Chicken Marengo now ranks first, while several recipes the agent would describe as unsuitable remain highly ranked. Its final answer made the selection more visible:

• Top pick: Chicken Marengo. Claude described it as having no dairy in the original recipe.
• Reported exclusions: Creamy Mustard Chicken, Chicken Alfredo Primavera, Fish Pie, Moussaka, Chicken Parmentier, and Creamy Tomato Soup. Claude said it had filtered these out because they relied heavily on cream, milk, or cheese in ways it considered difficult to substitute.
• Backups: Jiggs Dinner and Egyptian Fatteh. Both still came with instructions to omit or replace butter.

I checked the Chicken Marengo recipe: its listed ingredients are olive oil, mushrooms, chicken legs, passata, a chicken stock cube, black olives, and parsley. No milk, cream, cheese, or butter is explicitly listed. This time the top recommendation did not require rewriting a listed dairy ingredient. The backups show that the agent still treated substitutions as an acceptable route to an answer.

That felt like a small breakthrough. The replays connect the search calls to concrete candidates: all six recipes Claude said it rejected appear in the second list, along with its top pick and both backups. The final answer supplies its stated reason for those exclusions. This makes the example traceable from query through retrieved candidates to recommendation, while the agent’s actual checking process remains visible only through its account. Both replays use ten candidates, so they show a change in query and selection, not evidence that the candidate pool became larger.

I changed both description fields, so I cannot say which one mattered, and this was one before-and-after comparison, not a controlled evaluation. A chicken stock cube may also contain details absent from my recipe record. The useful observation is narrower: after receiving an explicit description of the tool's failure boundary, Claude changed how it searched in this run.

That changed how I thought about the tool description. It was not merely documentation for a developer reading the code. For this agent, it was part of the algorithm: information about the tool shaped which query it wrote and what it did with the results.

But should the agent have to do this every time?

I was pleased with the changed behavior, then uneasy about what I was asking the agent to do. “Find a recipe without X” is hardly an exotic request in a recipe database. The agent could inspect ten recipes and reject the unsuitable ones. But why make it carry out that check in prose every time, when the domain tells us in advance that exclusions will be common?

I considered whether another retrieval method would settle it. Sparse ranking could supply a different relevance signal. A full-text system might offer an explicit Boolean NOT operator. But those are distinct capabilities: a sparse score alone does not mean “exclude milk.” Even a text-level NOT milk can miss variants, composite ingredients, or recipes with incomplete descriptions. The question had moved beyond which ranking method I chose.

There are two separate questions here: whether a request uses a negative phrase, and whether it states a mandatory condition. “The dish must be vegan” is a hard requirement expressed positively; “I would prefer something not too heavy” is a soft preference expressed negatively. Even “quick” can be a loose ranking signal or a strict “no more than 20 minutes.” My milk searches exposed a failure around negation, but the design problem is broader: the system needs to identify which parts of a request should influence ranking, which must determine eligibility, and which need clarification.

Then I remembered what was already sitting in my payload: ingredient_names. I had put structured ingredient names into Qdrant, but the actual MCP tool offered Claude only a natural-language query, an optional result limit, and an optional score threshold. In simplified form, its input was:

type SearchInput = {
  query: string;
  limit?: number;
  scoreThreshold?: number;
};

There was no place to express “exclude dairy” as a condition. Claude could put those words into the similarity query or inspect recipes after retrieval. I have not implemented ingredient exclusions yet, but the next step became concrete: normalize the names, define groups such as dairy, and let the tool accept a condition directly. A proposed interface might look like this:

searchRecipes({
  query: "hearty filling dinner",
  excludeIngredients: ["milk", "cream", "cheese", "butter"],
  limit: 10
})

With a carefully defined food taxonomy, the intent might instead become excludeCategories: ["dairy"]. Qdrant supports payload filtering, including exclusions. The hard work is deciding what counts as dairy in the data and how to handle ingredients that cannot be classified confidently. I have not built or evaluated that layer yet.

This gave the LLM a clearer role. It could translate “something filling for dinner without dairy” into a search query expressing the preference and a formal exclusion expressing the requirement. The food taxonomy would encode which ingredients count as dairy. Application logic could enforce that condition against the normalized data, and the search tool could rank eligible recipes. Meaning would be represented in the interpretation of the request, the category definitions, and the tool interface; the filter would enforce the condition those choices had made explicit.

For a medically sensitive request, that logic would also need an explicit policy for unknown ingredients, compound products, and substitutions. Neither a vector score nor a confident answer is a substitute for such a policy.

Beyond exclusions: formalizing intent

Negation is only one example of the meaning a request can carry. The milk example made the mismatch easy to inspect. Consider other requests: “something more heavily spiced than this curry,” “a dish without sharp flavors,” “comfort food that still feels light,” or “a special-occasion dinner using familiar ingredients.” These involve comparisons, sensory qualities, combinations of preferences, and personal expectations. They suggest further cases to explore beyond the searches I ran here.

The difficult part often comes before the database query: formalizing what the person means. “Without sharp flavors” could mean less acidity, less chilli heat, less bitterness, or fewer strong aromas. “More heavily spiced than this curry” needs a reference dish and a useful interpretation of “more.” A database can filter or rank relevant properties when those properties and operations are available. Defining them, representing them in the data, and translating the request into them can be the hard engineering work. Some qualities may remain better suited to an approximate relevance judgment than a precise filter.

This gives the agent a role as a formalizer of intent. It can identify preferences and requirements, resolve references and comparisons, ask for clarification, and translate its interpretation into searches, conditions, and checks supported by the system. For “without sharp flavors,” that might begin with a question about which flavors the person wants to avoid. It also needs to recognize when the data cannot support a meaningful check. The tools then carry out the operations they expose, while the data and domain definitions determine what can actually be verified.

The same design question extends beyond recipes. “A quieter alternative to this laptop for travel” combines a comparison with a use case. “A setup guide for someone who knows Python but has never deployed a service” asks the system to interpret the reader’s background and the material’s assumed knowledge. These are examples of the broader problem, rather than capabilities established by my recipe experiments: how much of an informal request can the system turn into useful, supported operations?

That is what makes experimenting with LLMs exciting to me. Having an agent available made it easy to start exploring this interpretation layer: how everyday language could become searches, checks, and follow-up questions. Evaluation can then tell me whether those translations preserve what the user wanted.

The question I took away

I started with a plan to add a reranker. Before I got there, a milk pudding had changed the question I wanted to ask. Meaning was being handled in several places: the embedding model ranked recipes, the agent interpreted my request and chose how to act on it, and the tool interface determined which parts of that request could become explicit operations. Structured conditions would add another place for meaning to live, through the definitions encoded in the data. Even the tool description changed how the agent used the search.

I have not yet tested whether structured exclusions improve MealRAG across the evaluation set. The reranking experiment and systematic evaluation are still part of the work; their detailed comparisons belong in a different article. But these exploratory runs left me with a design question: where does each kind of meaning live, and when should a recurring user intention move from text into structure?

Practical takeaways for building a RAG system

The rest of this article turns those observations into a practical checklist: what I would measure first, where I would consider making changes, and what each change could realistically solve.

What I would measure before changing the system

These runs were exciting because each one exposed something I had not expected. They were also a reminder of how easy it is to optimize an LLM system by anecdote: a new prompt looks better on one request, a different model looks worse on another, and the next run surprises you again. That is useful for discovering failure modes. It is a poor way to decide which architecture to ship.

For a real system, I would establish the evaluation loop before choosing among the fixes below:

1. Build a realistic test corpus. It should contain the kinds of records, imperfect fields, and edge cases the system will face. My 719 recipes served as a small corpus for exploration; a production corpus would need to reflect its actual domain and data quality.
2. Create an evaluation set. Write representative user requests and define what success means for each: which results are relevant, which constraints are mandatory, and which outcomes are unacceptable. I assembled one for MealRAG; its detailed design belongs to the ongoing evaluation work rather than this article. The point is to cover plain positive requests, exclusions, and combinations such as “fast seafood dish without milk.” To evaluate the broader interpretation layer, I would also add comparisons, subjective preferences, and ambiguous requests where a useful clarification is part of success.
3. Automate the evaluation. Run the same requests through the pipeline after each change and record more than a few appealing examples. For retrieval, check whether relevant candidates appear and where they rank. For constraints, count violations. For the complete agent flow, check whether the agent preserved the user’s intent when choosing queries and conditions, and whether the final recommendation respects the request. Some judgments need human labels or review, but the runs and comparisons should be repeatable.

Only then would I swap embedding models, add a reranker, change prompts, or introduce filters and compare the results. The evaluation set gives those changes a shared target. Without it, an LLM's flexible behavior makes “this feels smarter” an unreliable measure of progress.

Who can improve the result?

With an evaluation baseline in place, the following interventions become options to test. They act at different points in the pipeline, and they solve different parts of the problem. The distinction I kept coming back to was relevance versus eligibility: which dinner seems appealing, and which dinners are allowed to be considered at all?

• A different embedding model. A model trained on exclusionary queries might rank “without milk” more sensibly; datasets such as ExcluIR exist partly to improve that behavior. I could test one against BGE-M3. Better ranking would still be a score, not a guarantee that every returned recipe satisfies a hard constraint.
• A reranker or relevance judge. This was my original next experiment. A second model could compare each retrieved recipe with the full request and reorder the candidates, or reject those it judges irrelevant. Cross-encoders have performed better than bi-encoders on the negation-focused NevIR benchmark. But a reranker can examine only the candidates the first stage found, and a model's judgment remains fallible.
• The agent’s interpretation and planning. Claude could search again, inspect recipes, suggest substitutions, and sometimes find a good answer. I saw that flexibility firsthand. I also saw it spend queries on excluded terms and supply preferences I had not specified. A clearer role would make the agent responsible for translating intent into supported operations, asking for clarification where needed, and identifying checks the data cannot support. That translation needs evaluation too: the agent can misinterpret the request before retrieval even starts.
• More tools, including web search. A broader source could help when the 719-recipe collection lacks a suitable dish. It would not, by itself, make exclusions reliable. It would also change the task: the answer might come from outside the recipe base I was trying to study.
• Better prompts and tool descriptions. The same user request produced a more useful strategy after I described the search tool's limitations. This is a relatively cheap improvement and helps the agent choose its next action. It still asks the agent to enforce the restriction by reading and reasoning over recipe text.
• Sparse, full-text, or hybrid retrieval. Lexical signals can help when exact words matter; hybrid search combines them with semantic results. A Boolean NOT can also exclude text matches when a search system exposes that operation. Neither sparse scoring nor result fusion automatically understands that “no milk” is a prohibition, and text-level exclusion can miss ingredient variants.
• Domain-aware tools and data. A search API could accept excludeIngredients or excludeCategories, while normalized recipe data and deterministic logic apply that condition. This is the clearest place to put a guarantee, provided the data and category rules are good enough. It costs more engineering work up front, and uncertain ingredient information still needs an explicit policy.

Use the evaluation results to identify where a request fails, choose an intervention at that point, and run the same cases again. The practical test is whether the change improves relevance, reduces constraint violations, and leads to better final recommendations.