Bayer trusts its decades of drug research to AI agents.

In a deep dive on building an agent system at Bayer, Thoughtworks principal consultant Sarang Sanjay Kulkarni explains why we don’t need a better model to build reliable agent systems, but rather a better harness and workflow. The post describes how they built PRINCE, a Preclinical Information Center that helps scientists sift through decades of safety study reports. This is a great glimpse into how to build a real agent system that works for real users on real data.

They broke their project down into three parts, each corresponding to a typical agent adoption journey: Search (filtering with metadata), Ask (retrieval augmented generation, allowing natural language question asking), and Do (multiple specialised agents for planning, researching, reflecting, writing, and completing complex tasks). At each stage, they increased functionality while increasing opportunities for error.

1The harness matters more than the model

The key quote of the article is this: “Reliability comes from engineering both the context the model sees and the harness within which the model acts.” There are two main concepts here: context engineering, where you actively send specific information to specific agents at specific times (e.g., providing planning context to the planner, research context to the researcher, etc.) instead of throwing all the information in a huge prompt and hoping for the best; and harness engineering, which involves building a framework around the model (e.g., orchestration, boundaries, persisting state, automatic retries, fallbacks, validation, reflection, observability, human intervention).

They implemented three types of reflection: process reflection when planning, data reflection to see if the retrieved evidence is enough, and draft reflection to make sure the answer is fully developed. They also built in resilience features like storing state in PostgreSQL, automatic model and node retries, allowing users to rerun from the point of failure, and falling back to other LLM providers if one fails.

They describe their RAG workflow in detail as well: keyword extraction + metadata filtering, query expansion to generate 5 semantically different versions, weighted hybrid search (0.7 semantic, 0.3 keyword), and cross encoder reranking (which reduces ~20 chunks to the top 7). They also set max SQL retries to 3 and record fetches to 50 per query. This may seem unglamorous, but that’s kind of the point.

2Why it matters for you

If you’re struggling to build an agent system that works in practice, then this is your guide to getting over the final hurdle. You won’t find success by improving your model — instead, you’ll succeed by controlling the information that your steps receive and creating observable, retryable, and recoverable workflows. That’s good news because it involves engineering, not waiting for a breakthrough model.

In my opinion, the most valuable takeaway from this article is learning how to break a big prompt into stage-specific contexts. I’ve seen my own agent systems get messier as I tried to cram more information into a single prompt. Sending each step the exact information it needs is a low cost way to improve reliability and something you can do right now with your current model.