Build Production AI Workflows with TypeScript and Temporal

Lena: You know, Miles, I was looking at some AI agent demos recently, and they all seem to work perfectly... until they don't. The moment a process crashes or an API times out, the whole thing just vanishes into thin air.

Miles: Exactly! That’s the "production problem." Most people treat orchestration like a simple LLM loop, but as of April 2026, we have to treat it like a distributed systems problem. If you don't have a way to resume from a mid-run checkpoint, your incident response is basically just guesswork.

Lena: Right, and it's fascinating that the fix isn't just "more AI"—it's actually about strict architectural separation. We’re talking about keeping your pure orchestration logic in Workflows while pushing all that messy I/O, like LLM calls and DB queries, into a dedicated Step layer.

Miles: That’s the key. By using a stack with TypeScript, Temporal, and the Vercel AI SDK, you can ensure every step is cached for replay-safety. So let's dive into how we actually build this durable framework.

Lena: So, if we’re moving away from those simple "if-then" loops and into this more robust territory, where do we actually start? I mean, when I think about a production environment, the first thing that comes to mind is stability. How do we translate that "distributed systems" mindset into actual TypeScript code?

Miles: It starts with a fundamental shift in how you view your code’s execution. In a standard setup, if your server restarts, your function dies, and whatever progress you made is gone. But with a framework like Temporal, we use what’s called "durable execution." Imagine your code is running in a stateful virtual machine that just doesn't care if the physical hardware fails. It’s like having an "undo" button for your entire system’s state, except it’s actually a "resume exactly where you left off" button.

Lena: That sounds like magic, but I know it’s grounded in some pretty heavy-duty engineering. You mentioned separating "Workflows" from "Steps." Why is that distinction so critical? Why can’t I just put my LLM calls right inside the main logic?

Miles: That is the number one mistake that causes systems to fail in production. Workflows in a durable system like this must be deterministic. That means if I run the same code ten times with the same inputs, it must take the exact same path every single time. But LLM calls—and really any I/O, like a database query or an HTTP request—are inherently non-deterministic. The API might be down, the response might change, or the network might lag.

Lena: Oh, I see. So if the workflow engine tries to "replay" the logic to recover from a crash, and it hits a new LLM call that returns a slightly different answer, the whole execution path diverges?

Miles: Precisely. The engine gets confused because the "history" it recorded doesn't match the code it’s running. That’s why we push all that non-deterministic stuff into "Steps" or "Activities." When a Workflow calls a Step, Temporal records the result of that step in its event history. If the system crashes and needs to recover, it doesn't actually re-run the Step. It just looks at the history, sees "Oh, last time we ran this LLM call, we got this JSON," and it just injects that result back into the workflow.

Lena: That’s brilliant. It’s essentially a transparent caching layer for every single interaction with the outside world. It makes the "Workflow" a pure coordinator of logic, while the "Steps" handle the heavy lifting of talking to the internet.

Miles: Exactly. And because we’re using TypeScript, we can use Zod to define strict contracts between these two layers. Every Step has a Zod schema for its input and its output. So, even before the code runs, we have compile-time and runtime guarantees that the data flowing through our AI pipeline is exactly what we expect. No more "undefined is not a function" halfway through a twenty-minute AI reasoning task.

Lena: It’s like building a high-speed railway where the Workflows are the tracks—solid, predictable, and unchanging—and the Steps are the specialized trains that carry the data. If a train breaks down, the tracks are still there, and the system knows exactly where the next train needs to start from.

Miles: That’s a great way to put it. And because we’re building this in 2026, we’re seeing that this isn't just a "nice to have." As multi-agent systems grow, the communication protocol layer becomes the absolute bottleneck for reliability. One recent study—ProtocolBench—actually found that the choice of protocol can change overall completion time by over 36 percent. If your orchestration isn't durable, you’re not just risking a crash; you’re leaving a massive amount of efficiency on the table.

Lena: So we’ve got our tracks and our trains. But how do we actually tell the trains what to do? In an AI context, that means managing prompts. And I've heard that just hard-coding strings into your TypeScript files is a recipe for disaster.

Miles: Oh, it’s a total time bomb. Treating prompts like "sticky notes" instead of production code is how you end up with a mess. We need to treat them like the versioned, mission-critical assets they are.

Lena: I love that phrase—"prompts are production code." It feels like we're finally giving the "instructions" part of AI the respect it deserves. If we aren't hard-coding them, where are they living? And how do we keep track of them when things inevitably change?

Miles: In a professional-grade framework, you want to store your prompts as standalone `.prompt` files. Think of these as a hybrid between a config file and a template. We use YAML for the metadata at the top—things like the model version, the temperature, and the author—and then Liquid templates for the actual prompt text.

Lena: Liquid templates? Like what’s used in Jekyll or Shopify? That seems really smart because it allows for conditional logic right inside the prompt.

Miles: Exactly! You can have a single prompt file that says, "If the user is a premium subscriber, use this tone, otherwise use that tone." But the real magic is in the versioning. You don't just "edit" a prompt. You create a new version, like `v1.1.0`. And your code doesn't just ask for "the prompt"; it asks for a specific version.

Lena: That solves the "who changed what" problem. I’ve seen so many teams where someone tweaks a prompt to be "friendlier," and suddenly the agent stops outputting valid JSON, and nobody can figure out why because the old prompt is just... gone.

Miles: Right. It’s a process failure. By keeping these in version control—right next to your TypeScript code—you get a full audit trail. And since we’re using that Step/Workflow split we talked about, the Step can load the specific `.prompt` file it needs, populate the Liquid variables, and send it off to the LLM. If the LLM call fails, Temporal handles the retry. If the prompt version was wrong, you can roll back the config in seconds.

Lena: It makes the whole thing feel so much more... engineering-led. It reminds me of the "Prompt Deck" approach I read about, where you manage these as disk-stored templates. It makes A/B testing so much easier, doesn't it? You could run one version of a prompt through a Step and compare it against another.

Miles: Absolutely. And when you combine that with the Vercel AI SDK, you can swap providers—going from Claude to Gemini or GPT-4o—in literally one line of code. The prompt stays the same, the metadata might change, but the business logic in your Workflow remains untouched. You’re effectively decoupling your "Intent" from the "Provider."

Lena: That "Provider Independence" is a huge deal. With models evolving so fast—I mean, we’re seeing new releases every other month in 2026—you don't want to be locked into an SDK that only works with one company. You want your architecture to be future-proof.

Miles: It’s about high-level policy not depending on low-level details. That’s the Dependency Inversion Principle from the SOLID world, applied to AI. Your Workflow shouldn't care if it’s talking to Anthropic or OpenAI. It should only care that it’s getting a valid response that matches its Zod schema.

Lena: So, we’ve got this durable execution, we’ve got versioned prompts, and we’re using Zod to make sure the data is clean. But how do we know if the AI is actually doing a good job? In a production workflow, you can't just "vibe check" every response.

Miles: You can't. And that’s where the next layer of our framework comes in—the "Evaluator" layer. This is where we use the LLM itself as a judge to score the quality of our own production outputs.

Lena: "LLM-as-Judge"—it sounds a bit like having the fox guard the henhouse, doesn't it? If the model is making the mistakes, can we really trust it to catch them?

Miles: It sounds counterintuitive, but it’s actually one of the most effective patterns we have. The key is that the "Judge" model is often a more capable, slower model—or it's given a much more specific, narrow rubric than the "Worker" model. Think of it like a senior editor reviewing a junior writer's work. The senior editor has a checklist of exactly what to look for.

Lena: So in our framework, we’d have a Step dedicated specifically to evaluation?

Miles: Exactly. After your main "Reasoning Step" finishes, the Workflow passes that output to an "Evaluator Step." This step doesn't just ask, "Is this good?" It uses a Zod-defined rubric. It might check for factual accuracy against a source document, verify the tone, or ensure no sensitive information was leaked.

Lena: And since this is happening inside a Temporal workflow, if the Evaluator gives a low score, the Workflow can actually loop back and tell the first agent to try again, right?

Miles: You’ve hit the nail on the head. That’s a "self-correcting" workflow. You can define a retry policy that says, "If the quality score is below 0.8, re-run the reasoning step with the evaluator’s feedback as a new input." This is how you move from "stochastic AI" to "reliable software."

Lena: I love that. It’s like the "Cognitive Operating System" concept—COCO—where you have continuous oversight. It’s not just a one-and-done prompt; it’s a conversation with a supervisor built directly into the code.

Miles: And it’s not just about the "Judge" model’s opinion. We can also use "Safety Tech" probes. In medical or financial apps, we might have specific evaluators that look for TLS downgrades, session hijacking tokens, or metadata leakage. The ProtocolBench research showed that different protocols like ANP or Agora handle these security probes differently. By building this into our framework, we make security an active part of the workflow, not just an afterthought.

Lena: It’s fascinating how this all connects. The durability of Temporal ensures the loop doesn't break, the versioned prompts ensure we can tune the "Judge," and the Zod schemas ensure the feedback is structured. It’s a total safety net.

Miles: It really is. But there’s a missing piece here. To run all these agents, evaluators, and loops at scale—especially if we’re talking about thousands of concurrent users—we need to talk about secrets. We’re passing API keys, customer data, and JWTs through these workflows. How do we keep that side of things from becoming a security nightmare?

Lena: Right, because if Temporal is recording the "history" of every step, isn't it also recording all those sensitive payloads in its database? That seems like a huge liability if someone gets access to the Temporal server.

Miles: It’s a massive liability if you don't handle it right. That’s why the next pillar of our framework has to be selective payload encryption.

Lena: Okay, so if the Temporal server is essentially a "black box" that stores our execution history, we need to make sure that history is encrypted before it ever leaves our worker, right?

Miles: Exactly. We use AES-256-GCM for this. The trick is to implement what’s called a "Data Converter" in the Temporal SDK. This acts as a middleman. Before any data—like a prompt or a result—is sent to the Temporal server, the Data Converter encrypts it. And when the worker pulls data back from the server, the Data Converter decrypts it.

Lena: So the Temporal server itself only ever sees scrambled gibberish? It knows there’s a workflow running, it knows which steps are complete, but it has no idea what the agents are actually saying?

Miles: Precisely. This is "Zero-Knowledge" orchestration. Even if your Temporal cluster was compromised, the attacker wouldn't be able to read your customer’s PII or your proprietary prompts. But there’s a nuance here—"selective" encryption. You don't necessarily want to encrypt everything.

Lena: Why not? Wouldn't it be safer to just scramble the whole thing?

Miles: Well, think about debugging. If everything is encrypted, the Temporal UI—which is normally this amazing tool for seeing exactly what happened in a workflow—becomes useless. You can't see the state of your system. So, we use a pattern where we only wrap "Sensitive" values.

Lena: Oh, like a TypeScript wrapper? I wrap my `apiKey` or `userSocialSecurityNumber` in a special `Sensitive` class, and the Data Converter only targets those specific objects for encryption?

Miles: That’s it. It’s the "secure payload" pattern. It gives you the best of both worlds: you get the observability for your logic and non-sensitive data, but your secrets are protected by industry-standard AES-256. It’s a crucial step for any production AI app, especially in 2026 when data privacy regulations are getting even stricter.

Lena: And since we’re using TypeScript, we can probably even use some type-level magic to ensure that we *can't* accidentally pass a sensitive value without it being wrapped.

Miles: You definitely can. You can define your Zod schemas to expect that `EncryptedPayload` type for certain fields. It’s all about creating "pit of success" architecture—making it easier to do the right thing than the wrong thing.

Lena: This whole stack—TypeScript, Temporal, Zod, AES encryption—it feels very "traditional software engineering," which is exactly what’s been missing from the AI world. We’re applying 20 years of distributed systems wisdom to these new, unpredictable models.

Miles: That’s the only way to ship at scale. I mean, look at what companies like Snap or Coinbase have done. They moved away from homegrown "SAGA" patterns and into durable execution because they realized they couldn't spend all their time writing retry logic and state-tracking code. They needed the infrastructure to handle the "how" so they could focus on the "what."

Lena: It’s about velocity, really. If I know my system is fault-tolerant and secure, I can iterate on my agent’s reasoning much faster. I’m not afraid to deploy a new prompt because I have a versioned rollback and a durable execution engine backing me up.

Miles: Exactly. But there’s one more challenge when you move from one agent to multiple agents. They have to talk to each other. And just like we need a protocol for the internet, we need a protocol for agents. This is where the world of AGENTS.md and CLAUDE.md comes in.

Lena: I’ve been seeing these `.md` files in the root of almost every major AI repo lately. It’s like a `README.md`, but specifically for the AI agent that’s browsing the code, right?

Miles: That’s exactly what it is. It’s "AI-readable documentation." In 2026, we’ve moved past just hoping the agent "gets" our project structure. We’re being explicit. And there are two main competing standards right now—AGENTS.md and CLAUDE.md.

Lena: Okay, so if I’m building our production framework, which one should I use? Or do I need both?

Miles: It depends on your team’s tools. AGENTS.md is the open, tool-agnostic standard. It was built by a huge coalition—Sourcegraph, OpenAI, Google—and it’s now part of the Agentic AI Foundation. It’s great if your team uses a mix of tools, like Cursor, GitHub Copilot, and custom scripts. It provides a universal baseline for code style, testing rules, and project boundaries.

Lena: And CLAUDE.md is the Anthropic-specific one?

Miles: Right. It’s purpose-built for Claude Code. It’s a bit more "opinionated" but also much more powerful. It supports hierarchical rules—so you can have different instructions for different subdirectories in a monorepo—and it has this amazing "import" system where you can compose instructions from multiple files.

Lena: That seems perfect for a complex AI workflow. You could have a `CLAUDE.md` that imports your "Security Standards" and your "Zod Validation Rules," and then a specific one for your "Payment Agent" that only activates when the agent is working in the `services/billing` folder.

Miles: Precisely. It’s "contextual documentation." Instead of flooding the agent with a 10,000-word prompt, you use "progressive disclosure." The agent only reads the rules that are relevant to the files it’s actually touching. It saves tokens, reduces "context rot," and drastically improves accuracy.

Lena: I love that term—"context rot." It’s so true; if you give an LLM too much irrelevant info, it starts losing track of the important stuff. By using these structured `.md` files, we’re essentially giving the agent a map of our framework’s "rules of the road."

Miles: And the best part is, they can coexist. A common pattern now is to put your universal rules in `AGENTS.md` and then have your `CLAUDE.md` just import that file and add a few Claude-specific tweaks. It’s about creating a "handbook" for your agents, so they know exactly how to interact with your Workflows and Steps without you having to re-explain it in every prompt.

Lena: It’s another layer of the "Prompts as Code" philosophy. We’re moving the instructions out of the "runtime" and into the "filesystem." It’s more maintainable, more auditable, and just... cleaner.

Miles: It really is. But even with the best documentation and the most durable execution, you’re eventually going to hit a scale where you need to coordinate dozens of specialized agents—maybe an "Analytics Agent," a "Translation Agent," and a "Compliance Agent"—all working on the same task. How do you manage that without it becoming a chaotic mess?

Lena: That’s the "Orchestration vs. Choreography" debate, right? Do we have one central brain, or do we let them all talk to each other?

Lena: When I think about a "mesh" of agents, I imagine something like a Slack channel where everyone is shouting at once. That sounds like a nightmare for a production system. How do we keep the coordination "durable" when we have so many moving parts?

Miles: You’re right to be wary. In distributed systems, there are two main ways to handle this. "Choreography" is that Slack channel idea—each agent reacts to events and talks to others directly. It’s very flexible, but it’s almost impossible to debug because there’s no single source of truth.

Lena: And "Orchestration" is more like a conductor in an orchestra?

Miles: Exactly. One central "Workflow" (our Temporal workflow) owns the state machine. It knows that Agent A must finish before Agent B starts. In 2026, for production AI, orchestration is almost always the better choice. It gives you that "centralized observability" the Akka team talks about. You can look at one execution log and see exactly how the "Compliance Agent" handed off data to the "Writing Agent."

Lena: But does a central orchestrator become a bottleneck? What if I need to run 50 agents in parallel on a massive data-processing task?

Miles: That’s where Temporal shines. You can use its "Child Workflows" or "Parallel Activities." Your main workflow can spawn 50 "Worker Steps," and Temporal handles the scheduling, the retries for each one, and the final "Join" where it aggregates all the results. It’s like having a project manager who can perfectly coordinate 50 people without ever losing a single status update.

Lena: And because each of those workers is a "Step," their results are cached. So if the 49th worker fails, you don't have to re-run the first 48!

Miles: Spot on. That’s the "Efficiency" play. The ProtocolBench research found that in a "Streaming Queue" scenario—where a coordinator is feeding tasks to workers—the choice of communication protocol can significantly impact throughput. Using something like the "Agent Communication Protocol" or ACP can minimize latency while keeping the load balanced.

Lena: It’s interesting that we’re seeing "Protocol Routers" now, too. The idea that a system can look at a task and say, "For this high-security medical task, I’ll use the ANP protocol with its DID-based identity, but for this high-speed data task, I’ll swap to A2A for the throughput."

Miles: It’s the next level of maturity. We’re not just building "an AI app" anymore; we’re building "AI networks." And our framework needs to be flexible enough to handle those different "interaction modes." Maybe one agent needs to stream its thoughts token-by-token, while another just needs to submit a final report.

Lena: It’s a lot to take in, but it all points to one thing: the era of the "unreliable AI demo" is over. If you want to build something that actually survives the real world, you have to build it on these durable, structured foundations.

Miles: You really do. And it’s not as daunting as it sounds once you have the right mental model. Workflow for logic, Step for I/O, Zod for contracts, and Temporal for the "durable pulse" that keeps it all alive.

Lena: So, we’ve covered the architecture, the prompts, the quality gates, the security, and the scaling. I think it’s time to put this all into a concrete "Playbook" that our listeners can actually use.

Lena: Okay, Miles, let’s get practical. If someone is sitting down tomorrow to start building this, what does the "Day One" checklist look like?

Miles: Step one: Get your environment right. Install the Temporal CLI and the Vercel AI SDK. Don't even think about writing business logic until you have your "Workflow" and "Activity" folders separated. That’s your structural foundation.

Lena: Right. And for the prompts?

Miles: Step two: Create a `prompts/` directory. Every prompt gets its own `.prompt` file with YAML frontmatter. Use Liquid for your templates. And here’s the rule: no prompt is allowed to be used in code unless it has a version tag. If you’re just "editing the string," you’re doing it wrong.

Lena: I love that rule. It forces discipline from the start. What about the "I/O Layer"?

Miles: Step three: Every LLM call, every database fetch, every external API request must be wrapped in a Temporal Activity—our "Step." And every one of those steps must have a Zod schema for its output. If the LLM returns a hallucinated field, Zod catches it at the boundary, and the Step fails gracefully instead of crashing your whole system.

Lena: And Step four has to be security, right?

Miles: Definitely. Configure your Temporal Data Converter with AES-256-GCM encryption. Wrap your sensitive data in that `Sensitive` type we talked about. This ensures your prompts and customer data stay private, even in your history logs.

Lena: And Step five? How do we handle the "Vibe Check" problem?

Miles: Build an "Evaluator Step" into your main Workflow loop. Don't just trust the output. Use a "Judge" model with a specific Zod rubric to score every reasoning task. If the score is low, use Temporal’s "Retry" logic to give it another go. This is how you guarantee quality.

Lena: This feels like a complete "System of Record" for AI. It’s not just code; it’s a way of operating. I can imagine this being a huge relief for developers who are tired of their agents being "flakey."

Miles: It’s about peace of mind. When you have this stack, an "incident" isn't a disaster. It’s just a workflow that’s "waiting for retry." You can see exactly where it is, you can inspect the (encrypted) state, and you can tune your prompts to fix the underlying issue without ever losing the work that’s already been done.

Lena: It’s a complete paradigm shift. We’re moving from "hope-based engineering" to "durable engineering." It’s the difference between a toy and a tool.

Miles: Exactly. And the best part is, this stack scales. Whether you’re running one agent or a mesh of 100, these principles remain the same. The architecture protects you from the complexity.

Lena: Well, Miles, this has been an absolute deep dive. I feel like I have a whole new perspective on what "production-ready" actually means for AI.

Miles: I’m glad. It’s a fascinating time to be building. We’re finally seeing the "software engineering" side of AI catch up to the "research" side.

Lena: You know, Miles, as we bring this to a close, what sticks with me most is how much of this "AI future" is actually built on very solid, traditional engineering values. Separation of concerns, version control, strict typing, encryption—it’s like we’re taking the "wild west" of LLMs and putting a proper infrastructure around it.

Miles: That’s the real takeaway. The models are powerful, but they’re just engines. To build a car that actually goes somewhere reliably, you need the chassis, the brakes, the dashboard, and the seatbelts. This framework—this "durable" stack—is the rest of the car.

Lena: It’s about building trust. If we want AI to handle our medical records, our financial trades, or our business operations, we need to know that the system is resilient. We need to know that it doesn't just "forget" what it was doing because a server blinked.

Miles: Right. And as we move toward 2027 and beyond, the teams that succeed won't just be the ones with the "best" prompts. They’ll be the ones with the best *systems*. The ones who can iterate safely, scale effortlessly, and recover from failures without anyone even noticing.

Lena: It’s an exciting challenge. For everyone listening, I hope this gives you a concrete path forward. Whether you’re starting a new project or trying to fix a "flakey" agent, maybe try applying just one of these patterns this week. Start with the Workflow/Step split, or move your prompts into version-controlled files.

Miles: Even small steps make a huge difference in the long run. Building with durability in mind changes how you think about every line of code you write.

Lena: Absolutely. Well, thank you so much for joining us for this deep dive. It’s been a blast exploring the "production problem" and its solutions with you.

Miles: Same here, Lena. It’s always a pleasure.

Lena: To our listeners, take a moment to reflect on your own AI workflows. Where is the "fragility" in your system? And how could a bit of durable execution or a structured quality gate help you build something that truly lasts? Thanks for listening, and happy building.