Every client wants an "AI chatbot" now. Most tutorials show you a demo that breaks the moment real users touch it. Here's the setup I actually use in production — simple, but it handles the parts demos skip.

The architecture

Keep it boring:

• Next.js API route (/api/chat) — your server-side proxy. Your LLM API key never touches the browser.
• Streaming responses — the bot "types" token by token. Users perceive this as 3x faster even when it isn't.
• A small React chat component — input, messages, done.

1. The API route

// app/api/chat/route.ts
export async function POST(req: Request) {
  const { messages } = await req.json();

  const response = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    },
    body: JSON.stringify({
      model: "gpt-4o-mini",
      stream: true,
      messages: [
        {
          role: "system",
          content:
            "You are a helpful assistant for an e-commerce store. Keep answers short.",
        },
        ...messages,
      ],
    }),
  });

  return new Response(response.body, {
    headers: { "Content-Type": "text/event-stream" },
  });
}

Note the system prompt — that's 80% of "bot quality" right there. Tune it per client, not per tutorial.

2. The frontend

Fetch the stream and append chunks as they arrive:

const res = await fetch("/api/chat", {
  method: "POST",
  body: JSON.stringify({ messages }),
});
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let done = false;

while (!done) {
  const { value, done: d } = await reader.read();
  done = d;
  if (value) {
    const chunk = decoder.decode(value);
    // parse the SSE "data:" lines and append each delta.content
  }
}

Parsing SSE chunks by hand is fiddly — in production, use a library like Vercel's ai package, which handles the streaming protocol for you.

3. Three things demos skip

1. Rate limiting. One shared API key plus a public endpoint equals a surprise bill. Add per-IP or per-user limits (Upstash Ratelimit takes about ten minutes to wire up).
2. Failure states. LLMs time out. Show a retry button, not a spinner of death.
3. Conversation memory. Sending the full history on every turn gets expensive fast. Summarize or truncate old messages server-side.

Going further

Once text chat works, the same pattern extends nicely: retrieval-augmented generation over the client's own docs, and voice agents (speech-to-text → LLM → text-to-speech) for hands-free support. I build these for clients regularly — you can see the kind of work I do at theabubakar.dev.

TL;DR

API route as proxy, stream the response, nail the system prompt, and handle rate limits and failures before launch. That's a production chatbot — everything else is polish.