I want to have this conversation with you before you build anything, because it’s the one that quietly determines whether your project succeeds or turns into an expensive science experiment. There’s a temptation, once you’ve read a few articles about agentic AI, to build every AI feature as a full-blown agent. Don’t. For a very large fraction of real-world tasks, an agent is genuinely the wrong tool. What you want is something simpler, more predictable, and cheaper.
This article is about honestly choosing between three levels of AI sophistication: chatbots, workflows, and agents. Each solves a specific class of problem well. Each fails on the classes the others are built for. Getting this choice right is one of the highest-leverage decisions in the whole track, and it’s the one people most often skip.
The three levels, from simplest to most autonomous
Let me define each one crisply, so we’re working from the same picture.
A chatbot is a language model that answers your question and stops. You ask, it responds, done. Every turn is independent (or nearly so). A retrieval-augmented chatbot that answers questions about your documentation is still a chatbot: it does one thing per turn, well.
A workflow is a fixed sequence of steps involving one or more language model calls, wired together in a predictable way. The steps are decided by you at design time, not by the model at runtime. The model might do the writing, the classification, the extraction, or the summarising, but the shape of the process is hard-coded. Extract the key facts from an email, classify the intent, look up the customer, draft a reply, and hand it to a human to review is a workflow.
An agent is a system where the language model itself decides what to do next at each step, using tools it can call, until a goal is reached. The path through the task is discovered as the agent runs, not planned in advance.
Notice the pattern: as we go from chatbot to workflow to agent, we hand more of the decision-making to the model. Which is exactly the axis that matters, and it’s what Figure 1 shows.
Figure 1: The three levels sit on an autonomy axis. A chatbot decides nothing about the flow; you drive every turn. A workflow decides everything at design time; the model plays specific roles inside a fixed shape. An agent decides at runtime; the shape emerges from the model’s choices.
That autonomy axis is the whole game. More autonomy means more flexibility and, honestly, more risk. Less autonomy means more predictability and, honestly, less power on the hard tasks. There is no universally “better” position; there’s only the right position for the problem you have.
When each level is the right tool
Let me give you the plain-English rules I use, because this is where the article earns its keep.
Reach for a chatbot when: the task is one interaction at a time, and each interaction is self-contained. Answering questions from documentation. Explaining a concept. Rephrasing something. Providing a targeted piece of help. If a competent human could handle it in a single message, a chatbot probably can too. Chatbots are cheap, fast, and easy to keep reliable.
Reach for a workflow when: the task has multiple steps, but the steps are the same every time. You know exactly what needs to happen, in what order. The variability lives inside each step (the language model handles a fuzzy input like an email or a document), but the sequence is fixed. Most enterprise “AI automation” is this. Extract, classify, look up, generate, send. Workflows are predictable, testable, cheap, easy to monitor. If you can draw the flowchart, build the flowchart. Don’t overengineer.
Reach for an agent when: the path through the task genuinely varies, and you can’t predict the steps in advance. The agent has to figure out what to do based on what it finds. Researching a topic and delivering a briefing. Troubleshooting an issue where the diagnosis depends on what’s found. Multi-tool tasks where “what to do next” depends on the result of the last thing. Agents earn their keep when the task is variable enough that hard-coding the steps would either not work or take you longer than just letting the agent decide.
Figure 2 puts this decision into one flow you can actually use.
Figure 2: Start with the simplest tool that could work. If the task is one interaction at a time, a chatbot is fine. If it’s multiple steps that repeat the same way, a workflow will beat an agent on cost, reliability, and debuggability. Only reach for an agent when the path itself genuinely varies.
The single most valuable habit is starting simpler than you think you need to. The number of times I’ve seen a team plan an agent and end up with a workflow that does the job better, and vice versa, is basically equal. Both mistakes cost you.
The honest trade-offs
Because the pattern I keep coming back to is choose the simplest tool that fits, let me be clear about what you actually give up as you move down the autonomy axis, and what you gain.
Moving from chatbot to workflow: you gain the ability to do multi-step work reliably. You give up simplicity: more moving parts, more prompts, more places to have bugs. You’re still very predictable, though. You know what your system does; it’s just that “what it does” is now a small pipeline.
Moving from workflow to agent: you gain the ability to handle variable, unpredictable tasks. You give up quite a lot: predictability (two runs of the same task may take different paths), debuggability (when something goes wrong, “which step failed” is less obvious), cost (agents typically make more model calls per task), and consistency (they can go off-piste in ways a workflow simply can’t). Agents are powerful and are also the most expensive thing on this list to run, monitor, and maintain. Treat that decision with respect.
There’s one more honest point worth making. You can and often should combine these. A really good real-world system might be a workflow that calls an agent for one specific step where the path varies, or an agent that calls a workflow as a tool for a well-defined subtask. Nothing forces you to pick one for the whole system. And often, “80% workflow, 20% agent for the fuzzy part” is a far better design than either “everything is an agent” or “everything is a workflow.”
A concrete case, worked through
Let me make it real with an example. Suppose you’re building an AI feature for customer support.
If the goal is to answer questions from the knowledge base: chatbot with RAG. One turn, one answer, one lookup. Simple, cheap, reliable. Don’t build an agent for this.
If the goal is to triage incoming customer emails: workflow. Classify intent, extract fields, look up the customer, decide the routing, either draft a suggested reply or escalate. Same shape every time, model handles the fuzzy bits. Workflow, without question.
If the goal is to research a customer’s complex complaint across multiple systems and produce a diagnosis: this is where an agent shines. What the agent looks up depends on what it finds. It might check the order system, then the payment system, then a call log, then correlate all three, and the sequence depends on the case. Hard-coding that flow would be brittle or impossible. An agent lets the path be discovered per case.
Three related tasks, three different right answers. That’s the shape of good judgement here.
The temptation to over-agentify
One last honest observation, because it comes up in almost every team I’ve watched build these. There is a real temptation, when agentic AI is the exciting topic, to build everything as an agent. This has two costs worth naming.
First, it makes things worse on the tasks that could have been chatbots or workflows. Slower, more expensive, harder to keep reliable. An agent doing a well-defined task is often a workflow with extra steps and less predictability. Second, it makes it harder to notice when you actually need an agent, because if everything is an agent, you can’t tell which ones needed to be.
The teams that ship the most reliable AI systems tend to use chatbots by default, workflows when they clearly help, and agents only when the task genuinely demands it. That’s not a lack of ambition; it’s engineering judgement. Aim for that.
From choosing to understanding
You now have a real, working framework for deciding which level of AI system to build. This is a bigger deal than it might seem, because most projects that fail with AI fail on this exact choice: they built an agent for a task that was really a workflow, or a chatbot for a task that really needed multi-step handling.
The rest of this track assumes you have decided you need an agent for something. So we now dive into how one actually works, mechanically. What happens inside the loop that makes an agent an agent? That’s what the next article opens up.
Next in the series: Inside the Agent Loop, the mechanism that makes an agent an agent.