AI Chatbot Development for Businesses: RAG, Guardrails & Handoff

Retrieval-Augmented Generation (RAG) — the foundation

RAG is the pattern that makes AI chatbots trustworthy: instead of letting the LLM answer from its training data (which may be wrong, outdated, or hallucinated), you retrieve relevant chunks from your own knowledge base and inject them into the prompt. The model can only answer from what you gave it.

1export async function POST(req: Request) {2  const { question } = await req.json();3 4  // 1. Embed the question5  const embedding = await openai.embeddings.create({6    model: "text-embedding-3-small",7    input: question,8  });9 10  // 2. Search your knowledge base11  const context = await searchKnowledgeBase(embedding.data[0].embedding);12 13  // 3. Answer ONLY from the context14  const stream = await openai.chat.completions.create({15    model: "gpt-4o-mini",16    stream: true,17    messages: [18      { role: "system", content: `Answer ONLY from the context below. If the answer is not in the context, say "I don't have that information — let me connect you with a team member."\n\nCONTEXT:\n${context}` },19      { role: "user", content: question },20    ],21  });22  return new Response(stream.toReadableStream());23}

Guardrails — keeping it on topic

Without guardrails, users will jailbreak your support bot into writing code, telling stories, or giving legal advice. A strict system prompt is the first layer; a classifier that detects off-topic questions before they reach the LLM is the second.

Human handoff — the trust signal

The chatbot should proactively offer human escalation when: confidence is low, the user has asked the same question twice, the question contains keywords like 'urgent', 'refund', 'legal', or 'cancel'. A visible 'Talk to a person' button at all times is not an admission of failure — it's a trust signal.

Building and maintaining the knowledge base

The chatbot is only as good as its knowledge base. We build a sync pipeline that re-indexes your docs, FAQs and help articles whenever content changes — so the chatbot always reflects the current version.