For experienced software engineers

Applied AI Engineering

A practical path for backend engineers who want to build reliable AI features. It focuses on model APIs, retrieval, tools, bounded agents, evaluation, and production work. Model training is outside the main path.

Sketch the system once in a visual tool. Rebuild the important path in code. Add a framework only after you can name the problem it solves.

Choose a depth

One map, three ways to use it

Choose a route, then move on when you can meet the layer’s exit criteria.

Project strategy

Build the same product in four passes

Each pass exposes a new engineering problem. You keep the domain and test cases, so the learning compounds.

1. Typed model featureDirect API, schema, errors

→

2. Grounded assistantRAG, sources, access filters

→

3. Tool workflowRead, propose, approve, write

→

4. Operated productEvals, traces, security, rollback

Map first

The roadmap

Click a yellow stage to open its guide. The faded nodes sit beyond the route currently selected.

Tokens and contextEmbeddings vs generationModel limitsPrompting, RAG, tools, fine-tuningVisual prompt comparisonRepeatable API experiment

Layer 1Build a working mental model

Direct SDK callsPrompt and context designStructured outputRetries and rate limitsVisual ticket classifierTyped triage API

Layer 2Use model APIs without a framework

Ingest, chunk, indexDense and keyword retrievalMetadata and access filtersReranking and citationsVisual document Q&ARAG API with pgvectorRetrieval evaluation

Layer 3Build RAG and learn why retrieval fails

Tool schemasRead vs write actionsIdentity and permissionsApproval and auditMCP architectureVisual approval flowSafe tool gatewayRemote MCP security

Layer 4Connect models to tools and MCP safely

Workflow vs agentState and checkpointsBudgets and stop conditionsHuman-in-the-loopOne visual agentBounded support copilotDurable recovery

Layer 5Build bounded workflows and agents

Representative test setDeterministic checksRubric and model gradersTrace inspectionCI regression gate20-case comparison30–50 case eval suite

Layer 6Make evaluation and tracing part of development

Threat modelTenant and retrieval isolationRate limits and queuesBudgets and fallbacksTracing and runbooksFailure-path drillDeploy and rollback

Layer 7Harden and operate the application

The visual tools are deliberate. They show the shape of RAG, tools, and agents quickly. The code task is where an experienced engineer should learn the real boundaries.

Layer guides

Concept, lab, code, exit

Each layer has a narrow goal. References are ordered, so start with #1 unless you already know the topic.

Build a working mental model

Understand what a model can and cannot do before adding frameworks.

Understand, then experiment

Core for every routeLight effort Exit criteria met

What to learn

Tokens, context windows, messages, sampling, and why the same request can produce different answers.
The difference between generating text and representing meaning with embeddings.
Pretraining, post-training, reasoning behavior, hallucination, and knowledge limits at a practical level.
How to choose between prompting, retrieval, tool use, fine-tuning, and an agentic workflow.

You know enough when

You can explain non-determinism without saying the model is random magic.
Given a feature request, you can say whether it calls for prompting, retrieval, a tool, or fine-tuning.
You know which internal details help application design and which ones can wait.

Quick visual lab

In n8n, connect a manual input to one model node. Ask the same task four ways: vague instruction, clear instruction, one example, then the same prompt with irrelevant context. Save the outputs side by side.

Small code experiment

Call one model directly from Python. Run the same fixed task ten times and log the input, output, model, latency, and token usage. Change one variable at a time. The point is to observe behavior, not to build a product yet.

Recommended references

#1 Start hereVideo Intro to Large Language Models ↗Andrej Karpathy

Recommended starting point. A useful long-form mental model from pretraining through application behavior. Watch selectively.

Use model APIs without a framework

Turn the model into a typed, testable component inside a normal backend service.

Build properly

Core for builder and production routesMedium effort Exit criteria met

What to learn

System and user instructions, examples, context boundaries, and prompt versioning.
Structured output with JSON Schema, Pydantic, or Zod. Do not parse prose with string tricks.
Timeouts, retries, rate limits, streaming, refusals, malformed output, and provider errors.
How to choose a model by task quality, latency, cost, and operational constraints.

You know enough when

No production path depends on extracting fields from free-form prose.
Expected failure cases have explicit behavior instead of one broad catch block.
A small fixed test set can show whether a prompt or model change helped.

Quick visual lab

Build a support-ticket classifier in n8n. Ask for JSON with category, priority, summary, and requires_human. Branch high-risk or malformed results to a review step. Deliberately add missing and conflicting fields.

Code project

Create a small ticket-triage API with a typed response. Add validation, retry only retryable failures, log prompt version and latency, and keep a dozen test tickets in the repository.

Implementation shape, not provider-specific copy/paste

class TicketDecision(BaseModel):
    category: Literal["billing", "bug", "account", "other"]
    priority: Literal["low", "normal", "high"]
    summary: str
    requires_human: bool

# Use the provider SDK's current typed-output method.
decision = model.generate(text=ticket, output_type=TicketDecision)

Keep the first production path simple and inspectable.

1Input + promptclear task and context

→

2Model callone provider SDK

→

3Schema validationtyped output only

→

4Retry or fallbackonly when justified

→

5Typed resultlog version, latency, and tokens

Recommended references

#1 Start hereDocs Structured outputs ↗OpenAI developer documentation

Recommended starting point. Shows how to make model output fit a schema instead of parsing prose.

#2Docs Prompt engineering ↗OpenAI developer documentation

Current patterns for instructions, examples, context, and prompt iteration.

#3Free lecture Prompt Engineering ↗Full Stack LLM Bootcamp

A compact engineering lecture that connects prompting to system design and testing.

Build RAG and learn why retrieval fails

Give the application controlled access to private or current information with inspectable sources.

Build and measure

Core for most applied AI productsMedium effort Exit criteria met

What to learn

Embeddings, similarity, chunking, overlap, metadata, document identity, and access filters.
Dense, keyword, and hybrid retrieval; reranking; top-k; query rewriting; and when each helps.
How to separate a retrieval miss from a generation mistake.
Why authorization must constrain retrieval before text reaches the model.

You know enough when

You can inspect what was retrieved for every answer.
Source IDs map back to the exact document and version.
Tests cover known facts, ambiguous questions, missing answers, and access boundaries.
You can change retrieval without rewriting the whole application.

Quick visual lab

Use n8n to load a small policy set. Ask five known questions and inspect the chunks returned before reading the final answer. Change chunk size and top-k once so the effect is visible.

Code project

Add document ingestion and retrieval to the ticket service. PostgreSQL with pgvector is enough. Return source IDs with every supported claim and say that evidence is insufficient when retrieval does not support an answer.

RAG has two paths. Debug them separately.

Ingestion

Documentsversion + access metadata

→

Parse & chunkkeep document identity

→

Embed & indexvector + keyword fields

Question time

User questionidentity + filters

→

Retrieve & rerankinspect candidates

→

Generate with evidenceanswer + citations

First diagnostic: did the right evidence reach the model? A fluent answer cannot repair a retrieval miss.

Recommended references

#1 Start hereDocs Introduction to RAG ↗LlamaIndex documentation

Recommended starting point. A clear explanation of the RAG pipeline and the parts that can be inspected or replaced.

#2Open-source docs pgvector ↗pgvector project

A practical vector search starting point for engineers already comfortable with PostgreSQL.

#3Docs + lab Retrieve relevant context with AI workflows ↗n8n documentation

A fast visual way to see ingestion, retrieval, and generation as separate steps.

Connect models to tools and MCP safely

Let the model request an action while the application keeps control of execution and permissions.

Build with security boundaries

Builder and production routesMedium effort Exit criteria met

What to learn

Tool definitions, schemas, results, and the model–tool execution loop.
Read versus write actions, least privilege, caller identity, audit logs, idempotency, and retries.
How tool descriptions and returned data affect model behavior.
MCP hosts, clients, servers, resources, tools, prompts, transports, and authentication.

You know enough when

Arguments are validated outside the model.
Permissions come from application identity, never from text in the prompt.
Write actions are idempotent or have a safe duplicate strategy.
You can explain when MCP improves interoperability and when a normal function call is simpler.

Quick visual lab

Give an n8n agent one read-only lookup and one write action. Put human approval before the write. Trigger invalid arguments, a missing record, a timeout, and a duplicate request.

Code project

Add get_customer, search_orders, draft_reply, and create_ticket. The model may propose a write, but application code validates the arguments, checks the signed-in user, asks for approval, and records the result.

Implementation shape, not provider-specific copy/paste

proposal = model.propose_tool_call(context)
args = ToolArgs.model_validate(proposal.arguments)
policy.authorize(user, proposal.name, args)

if tool_registry[proposal.name].writes_data:
    approval.require(user, proposal)

result = executor.run_idempotently(proposal.name, args)
audit.record(user, proposal, result)

The model proposes. Application code decides and executes.

Model proposaltool name + arguments

→

trusted application boundary

1. Validate schema

2. Check auth and policy

3. Ask for approval when needed

4. Execute idempotently

→

Audit and safe resultreturn only approved output

Recommended references

#1 Start hereDocs Function calling ↗OpenAI developer documentation

Recommended starting point. Covers tool schemas, arguments, execution, and returning results to the model.

#2Docs Introduction to Model Context Protocol ↗MCP documentation

Explains hosts, clients, servers, resources, prompts, and tools without unnecessary framework code.

#3Security guide MCP security best practices ↗MCP documentation

Useful before exposing tools, credentials, or remote servers.

Build bounded workflows and agents

Use an agent only when the next step genuinely depends on what happened in the previous step.

One framework, one bounded workflow

Builder and production routesHigher effort Exit criteria met

What to learn

The difference between a fixed workflow and an agent that chooses its next step.
State, memory, context construction, planning, handoffs, checkpoints, and recovery.
Time, token, step, and tool budgets; cancellation; retry policy; and clear stop conditions.
Where human review belongs when an action is sensitive, costly, or hard to undo.

You know enough when

A trace shows each decision, tool call, and state change.
The workflow has completion, failure, cancellation, and budget limits.
It can resume or fail safely after an external call.
You can defend why the agentic part is needed instead of using a deterministic pipeline.

Quick visual lab

Choose one visual tool. Give one agent retrieval plus two tools. Keep it single-agent. Watch the trace and note where the model chooses a path instead of following a fixed branch.

Code project

Extend the support copilot: classify the request, retrieve policy, choose a read tool when needed, draft a response, then pause before any write. Use one framework only if state or recovery is becoming awkward in plain code.

A useful agent loop is visible, bounded, and interruptible.

1Read current state

→

2Choose next step

→

3Call one tool

→

4Observe result

↺

Stop when complete, blocked, cancelled, or over budget

Recommended references

#1 Start hereEngineering article Building effective agents ↗Anthropic Engineering

Recommended starting point. A grounded distinction between workflows and agents, with simple patterns before complex autonomy.

#2Docs LangGraph overview ↗LangChain documentation

Useful when a workflow needs durable state, checkpoints, explicit transitions, or human pauses.

#3Docs OpenAI Agents SDK ↗OpenAI open-source documentation

A lightweight reference for agents, handoffs, guardrails, sessions, and tracing.

Make evaluation and tracing part of development

Replace demo impressions with repeatable evidence.

Required engineering practice

Core for anyone shipping a real featureMedium effort · ongoing habit Exit criteria met

What to learn

Success criteria, representative cases, baselines, experiments, and regressions.
Deterministic checks, human review, rubric-based model graders, and grader calibration.
RAG metrics such as retrieval relevance and groundedness; tool metrics such as choice and argument correctness.
Tracing prompts, retrieved chunks, tool calls, latency, tokens, cost, and failures without leaking sensitive data.

You know enough when

Two versions can be compared on the same fixed dataset.
A failure can be traced to retrieval, prompting, tool selection, execution, or policy.
Human ratings and model graders are periodically checked for agreement.
Quality is reviewed alongside latency and cost.

Quick visual lab

Collect twenty cases from the project. Compare two prompts or models with simple rules plus a written human rubric. Inspect failures before looking at one average score.

Code project

Grow the set to 30–50 cases. Check ticket classification, retrieval relevance, cited support, tool choice, forbidden writes, latency, and cost. Run it in CI for important prompt, model, retrieval, or tool changes.

Treat every change as an experiment.

Representative casesreal tasks + edge cases

→

Run candidateprompt, model, RAG, tools

→

Component checksretrieval, output, tool use

→

Inspect failurestrace before averages

→

Ship or reviserecord baseline

Recommended references

#1 Start hereEngineering article Demystifying evals for AI agents ↗Anthropic Engineering

Recommended starting point. Shows how to move from vague impressions to task-level and component-level evidence.

#2Open-source docs Promptfoo ↗Promptfoo documentation

Useful for local test cases, provider comparisons, assertions, and CI regression gates.

#3Open-source tutorial Evaluate RAG ↗Arize Phoenix documentation

A concrete path for inspecting retrieval and response quality with traces.

Harden and operate the application

Production AI is production software plus untrusted content, uncertain model behavior, and external actions.

Production ownership

Production routeHigher effort Exit criteria met

What to learn

Prompt and indirect injection, sensitive data exposure, improper output handling, excessive agency, and supply-chain risk.
Tenant isolation, retrieval authorization, secret handling, audit trails, write approval, and sandboxing.
Rate limits, backoff, queues, caching, streaming, fallbacks, model changes, and provider outages.
Budgets, latency targets, retention, privacy, monitoring, incident response, gradual rollout, and rollback.

You know enough when

The repository contains a short threat model and abuse cases.
Every critical action has authorization, validation, and the right approval policy.
Quality, latency, errors, and cost are visible enough to investigate an incident.
A model or prompt change can be tested, rolled out gradually, and rolled back.

Quick visual lab

Draw the trust boundaries, then test them: malicious text in a document, an unauthorized record lookup, a duplicate write, a model timeout, a provider outage, and a request that exceeds its budget.

Production pass

Add authentication and per-user permissions, queued ingestion, traces, red-team cases, cost limits, dashboards, deployment, and rollback. Document which actions are allowed, suggest-only, approval-gated, or forbidden.

Recommended references

#1 Start hereSecurity guide OWASP GenAI LLM Top 10 2026 ↗OWASP GenAI Security Project

Recommended starting point. A current threat checklist for prompt injection, data exposure, excessive agency, and related risks.

#2Docs Production best practices ↗OpenAI developer documentation

Covers operational concerns such as scaling, limits, reliability, and deployment.

#3Open-source spec OpenTelemetry semantic conventions for GenAI ↗OpenTelemetry project

A useful reference when traces need consistent model, token, tool, and agent fields.

One evolving project

Support & Operations Copilot

The capstone is ordinary enough to understand, but rich enough to exercise retrieval, tools, approvals, evals, and production controls.

OpsDesk / AI workspace

Request and trace

“Several customers report checkout errors after today’s release. Check the runbook and current order service status.”

✓ classified: incident✓ policy retrieved✓ status tool readapproval pending

Draft response

The order service is returning elevated 5xx errors. The release runbook says to open an incident, notify the on-call engineer, and hold customer-facing status language until impact is confirmed.

Runbook §3.2Service status 14:21

What changes at each layer

Keep the same tickets and policies. The application becomes more capable and more accountable.

1Classify and summarize a ticket with typed output.
2Add prompt versions, retries, and a small test set.
3Retrieve policies and cite the exact source version.
4Read operational data and propose approval-gated writes.
5Use a bounded workflow when the next step depends on observations.
6Run component and end-to-end evals on every important change.
7Add isolation, budgets, traces, deployment, and rollback.

92%task completion on fixed set

88%relevant evidence in top results

100%forbidden writes blocked

3.1sp95 response before approval

Capstone architecture and trust boundaries

Web / API client

→

Auth + tenant context

→

Workflow orchestrator

→

Model API

Document ingestion + pgvector

Tool gateway + approvals

Traces + eval store

Queues + workers

Tool choice

Pick tools by the problem they remove

Keep the stack small. Use one visual tool to understand the shape, provider SDKs to learn the real boundaries, and one workflow framework only when state and recovery become real needs.

n8n

Visual orientation

Use it when: Use it for a first pass through model calls, RAG, approvals, and integration-heavy prototypes.

Keep in mind: Once the shape is clear, move the important path to normal application code.

Official docs ↗

Direct provider SDK

Default coding path

Use it when: Start here for messages, structured output, streaming, tool calls, retries, and error handling.

Keep in mind: Stay with direct SDKs until a framework clearly removes repeated orchestration work.

Developer docs ↗

LangGraph

Stateful workflows

Use it when: Bring it in only when the application needs checkpoints, pauses, branching, or recovery over time.

Keep in mind: Do not introduce it for short deterministic request paths that are already clear in plain code.

Official docs ↗

How to give this to a senior engineer

Guide outcomes, not study time

Ten years of backend experience changes the path. Do not reteach APIs, databases, queues, auth, testing, or deployment. Connect those skills to model behavior and evaluation.

Choose the route together

Agree whether the goal is orientation, feature delivery, or production ownership. This prevents both shallow browsing and unnecessary theory.

Use one visual pass

Let n8n make the system shape visible. Then stop and rebuild the important path in code.

Review exit criteria

Ask for the working project, traces, failures, and decisions. Do not judge progress by videos watched or framework vocabulary.

Keep one capstone

New requirements should deepen the same product. That reveals architecture trade-offs that isolated tutorials hide.

Applied AI Engineering

One map, three ways to use it

Fast orientation

Applied AI builder

Production ownership

Build the same product in four passes

The roadmap

Concept, lab, code, exit

Build a working mental model

What to learn

You know enough when

Quick visual lab

Small code experiment

Recommended references

Use model APIs without a framework

What to learn

You know enough when

Quick visual lab

Code project

Recommended references

Build RAG and learn why retrieval fails

What to learn

You know enough when

Quick visual lab

Code project

Recommended references

Connect models to tools and MCP safely

What to learn

You know enough when

Quick visual lab

Code project

Recommended references

Build bounded workflows and agents

What to learn

You know enough when

Quick visual lab

Code project

Recommended references

Make evaluation and tracing part of development

What to learn

You know enough when

Quick visual lab

Code project

Recommended references

Harden and operate the application

What to learn

You know enough when

Quick visual lab

Production pass

Recommended references

Support & Operations Copilot

What changes at each layer

Pick tools by the problem they remove

n8n

Direct provider SDK

LangGraph

Guide outcomes, not study time

Choose the route together

Use one visual pass

Review exit criteria

Keep one capstone