Production failure modes
Job-oriented pages for the failures demos hide. Each page diagnoses the problem, then points to a Knackline capability and related reports.
AI Agents
Loops, evals, permissions, harnesses
Why AI agents loop in production
AI agents loop in production when the system keeps calling tools without proving task completion. Demos hide this because happy-path prompts finish quickly; real users trigger retries, ambiguous goals, and missing stop conditions.
How to evaluate AI agent workflows
Evaluating AI agent workflows means measuring whether the agent completes the job under realistic constraints: tools, permissions, approvals, and failure modes. Latency and chat quality alone do not prove production readiness.
Tool permissions and approval gates for AI agents
Tool permissions and approval gates constrain what an AI agent may do. Without them, a convincing demo becomes an unbounded automation with production blast radius.
RAG / Retrieval
Drift, evals, productionizing retrieval
RAG evaluation metrics that matter
RAG evaluation metrics that matter measure whether the system retrieved the right evidence and answered without inventing facts. Fluency and embedding similarity alone miss the failures users feel in production.
How to detect retrieval drift
Retrieval drift is the quiet decline in RAG quality after corpus, chunking, embedding, or filter changes. Dashboards may stay green while users get the wrong version of the truth.
How to productionize a RAG application
Productionizing a RAG application means turning a retrieval demo into an operable data path: owned indexes, ACL-safe retrieval, citations, evaluation, and change management.
AI Observability
Traces, taxonomies, SLOs
Generative BI
Metric governance and groundedness
Catalog last updated 2026-09-11.