Knowledge Base for AI Agents: How to Structure SOPs, FAQs and Policies

The most common mistake we see when a company builds an AI agent is feeding it everything. The shared drive gets exported, years of chat threads get uploaded, the old returns policy sits next to the new one, and then everyone is surprised when the agent tells a customer they have 60 days to return an item when the real window is 30. The model did not invent that number. It found it, in a document nobody deleted.

A knowledge base for AI agents is not a file dump. It is a small, curated, deliberately structured set of documents written so that a system reading one fragment at a time can find the right answer and nothing that contradicts it. This guide covers how to structure the three things most business agents answer from: SOPs, FAQs and policies.

Why agents give wrong answers from the right documents

Most business agents work by retrieval. When a question comes in, the system searches your knowledge base, pulls the handful of passages that look most relevant, and hands them to the model to write an answer. That means the agent rarely sees a whole document. It sees fragments. Three things go wrong at that stage:

• Contradictions. Two versions of the same fact exist and the search returns the wrong one, or both.
• Missing context. A passage says “this does not apply to sale items” but the fragment that was retrieved never says what “this” refers to.
• Buried answers. The fact exists, but it sits in paragraph nine of a meeting note, phrased in a way no customer would ever search for.

Every structural rule below exists to fix one of those three problems.

Sort before you write: three document types

Before touching formatting, sort what you have into three buckets. Each has a different job, and mixing them is where confusion starts.

Policies are the source of truth. SOPs and FAQs should point back to a policy rather than restating its numbers. If your return window changes, you want to edit one line, not hunt through forty documents for every mention of it.

Everything that does not fit these buckets, such as meeting notes, brainstorms, old campaign briefs and draft decks, stays out. Archive it somewhere the agent cannot reach.

A standard shape for every entry

Agents retrieve better when every entry follows the same pattern. A simple header block at the top of each document works well:

1. Title written as the thing a person would search for: Returns and exchanges policy, not RP-v4-final.
2. Applies to: which products, regions, customer types or channels.
3. Does not apply to: the exclusions, stated plainly.
4. Owner and last reviewed date.
5. Summary: two or three sentences with the core answer.
6. Detail: the full content, broken into short sections with descriptive subheadings.

The summary matters more than it looks. When retrieval pulls the top of the document, the answer is right there, together with its scope. When it pulls a passage from the middle, a descriptive subheading carries enough context for that passage to make sense on its own.

Write every passage so it stands alone

Assume any paragraph might be read without the paragraphs around it. That means repeating the subject instead of leaning on pronouns. Compare these two versions of the same rule:

Weak: These can be exchanged once, but not after the window above has passed.

Strong: Final-sale items can be exchanged once for a different size within 14 days of delivery. Final-sale items cannot be refunded.

The second version reads a little repetitive to a person. To an agent reading one passage, it is the difference between a correct answer and a guess.

Writing SOPs an agent can follow

An SOP written for a trained employee skips steps because the employee already knows them. An agent does not. Agent-ready SOPs share a few habits:

• Numbered steps, one action each. Check the order date, confirm the item is eligible and issue the label is three steps, not one.
• Decision points written as if/then. If the order is older than 30 days, go to step 7. If not, continue to step 4.
• Named systems. Say which tool each step happens in: the store admin, the helpdesk, the shipping app.
• A clear stop condition. State when the agent must hand off to a person, for example refunds above a set amount, damaged goods, anything involving a chargeback, or a frustrated customer on their second contact.

That last point is the one most teams skip. An agent that knows when to stop is far more useful than one that tries to answer everything.

FAQs: one question, one answer, several phrasings

FAQs are where most customer-facing agents find their answers, so treat them as the front door. Start from real questions, not ones you imagine. Pull the last few months of support emails, DMs and chat logs and group them by topic. The ten or twenty topics that come up most often are your first FAQ set.

For each entry:

• Write the question the way customers phrase it, then add two or three alternate phrasings underneath. Where is my order, has my package shipped and I never got a tracking number are the same question.
• Answer in the first sentence. Add detail after.
• Link to the policy the answer depends on rather than copying its numbers.
• Keep one question per entry. Combined entries like Shipping and returns retrieve badly because they match everything a little and nothing well.

Policies: write the exceptions down

The gap between a policy on paper and a policy in practice is where agents get into trouble. Your team may quietly extend return windows for repeat customers, or waive a shipping fee when a delivery runs late. If those exceptions are not written down, the agent will either refuse something your team would approve or approve something it should not.

For each exception, decide whether the agent may offer it, may mention it but must escalate, or should never raise it. Then write that into the policy in plain language. Agents may not offer discounts or store credit; pass these requests to the support team is a perfectly good policy line.

Say what the agent should not say

Some topics carry legal or reputational risk: health claims, medical advice, pricing promises, anything about a dispute. List them in a short guardrails document that the agent's instructions reference directly, with the exact wording to use instead, such as That is a question for your doctor, or I will connect you with the team for that.

Test it like an exam before launch

Once the knowledge base is structured, test it with a fixed set of questions before any customer sees the agent. A practical process:

1. Write 30 to 50 test questions from real support history, including awkward ones: edge cases, exclusions and questions the agent should decline.
2. Write the correct answer for each, and note which document it should come from.
3. Run them through the agent and mark each answer right, wrong, or right but sourced from the wrong document.
4. For every miss, fix the document before you touch the prompt. Most wrong answers trace back to a contradiction, a missing exclusion or a buried fact.
5. Re-run the full set after each round of changes, since a fix in one place can break an answer somewhere else.

Keep this question set. It becomes your regression test every time a policy changes.

Keeping it true after launch

A knowledge base goes stale the moment a policy changes and nobody updates it. A few habits prevent that:

• Give every document a named owner and a review date. Quarterly is a sensible default for policies, monthly for FAQs during busy seasons.
• Tie updates to business events. When a price, shipping rate or policy changes, updating the knowledge base is part of the same task, not a follow-up.
• Review the questions the agent escalated or could not answer each week. Each one is either a missing FAQ or a sign that something is unclear.
• Delete superseded versions instead of keeping them for reference. Old versions are the single biggest source of confident wrong answers.

From clean documents to a working agent

The structure above is most of the work. Once your SOPs, FAQs and policies are clean, connecting them to an agent that answers staff questions, drafts replies or handles routine requests becomes a much smaller project. Our company agent builder service covers the whole job, from auditing and restructuring your documents to building, testing and handing over an agent your team can maintain. If the goal is answering shoppers on your site or in Instagram DMs, the same foundation powers our AI concierge chatbot for websites and DMs.