AI web chat is only as good as what you tell it. Give an AI agent a vague instruction and your whole website to read, and it will answer confidently from an old blog post about a service you stopped offering. Give it a short, current knowledge base and clear rules, and it answers the routine questions well and passes the rest on. This article answers one question: what should go into the instructions and knowledge base of an AI agent answering website chat on the levelpartner.ai platform, and how do you test and maintain it?
The agent knows only what you give it
An AI agent on web chat works from two things. Its instructions say who it represents, how it should speak, what it may do and what it must never do. Its knowledge base holds the facts it answers from: services, areas, hours, process, common questions. Everything it says should trace back to one of those two sources.
That makes the job editorial, not technical. The person who sets up the agent needs to know the business well enough to write down the answers a good receptionist would give, and to spot when the agent gets one wrong.
What goes in the knowledge base
Keep it short and current. A focused set of sources beats a large pile of everything.
- A one-page summary of services, with who each is for and what it includes.
- Areas or markets covered, such as the countries, states or regions you serve.
- Opening hours and how quickly a person replies outside them.
- How a first meeting or call works, how long it lasts and what to bring.
- The ten to twenty questions visitors ask most, each with a short approved answer.
- How to book, with the calendar the agent can offer.
Leave out old brochures, blog posts with outdated detail, internal documents and anything that mentions a price you would not quote in a chat.
Writing the instructions
Write the instructions as plain sentences, as if briefing a new member of staff on their first day. Say who the agent represents and that it is an assistant, not a person. Say what tone to use: short replies, plain English, no jargon. List what it may do, such as answer questions from the knowledge base, ask qualifying questions and book a call. List what it must hand over, such as fee negotiations, regulated advice, complaints and anything it cannot answer from its sources.
For a mortgage broker, for example, the agent can explain which documents a first appointment needs, but must never suggest whether someone will be approved. A real estate agent's assistant can describe the valuation process, but must not give a figure.
Before you start
You need access to AI Agents, Calendars and Conversations, and the chat widget installed on your site. Gather the sources listed above and check each one is current. Write your list of handover topics. Pick one person who owns the agent's content and will review its conversations each week.
How to set it up
Write the instructions before you load any sources: who the agent represents, its tone, what it may do and what it must hand over. Then build the knowledge it answers from.
- Open AI Agents and select the Knowledge Base tab.
- Create a new knowledge base or edit an existing one.
- Open the FAQ tab and add your common questions with approved answers.
- Click + Add Source and select Web Crawler.
- Choose Exact URL so only the pages you pick are read.
- Enter the URL and click Extract Data.
- Click View All Pages and tick only current, relevant pages.
- Click Train Bot.
- Open the Knowledge Base Retrieval Tester and ask a realistic question.
- Return to the bot's Bot Training tab and select the knowledge base.
Once the knowledge base is attached, turn on the Chat Widget channel on the agent's Deploy tab. Write twenty test questions, five of which should be handed over, ask each one in a private browser window, and correct the instructions or the sources for every wrong answer before the agent goes live.
Testing before visitors see it
The twenty test questions are the most valuable hour you will spend. Include the obvious ones, such as hours and services. Include awkward ones: a question about a competitor, a request for a price, an angry message, a question in the wrong language, and a question about a service you do not offer. Every wrong answer is a gap in the instructions or the knowledge base, and is far cheaper to find now than after a prospect has seen it.
Imagine a first test run that answers 13 of 20 questions correctly. Say four errors trace back to an old services page in the knowledge base, two to instructions that never mention fees, and one to a missing answer about remote meetings. Remove the old page and add two lines to the instructions, and a second run could reasonably reach 19 of 20. If the last miss is a handover, the team may decide that is the right behaviour and leave it alone.
Feeding the agent the whole website and hoping it sorts the good from the bad. It cannot tell an outdated page from a current one. Choose the sources yourself, and remove pages as soon as they go out of date.
What it takes to maintain
Plan an hour a week for the first month and about an hour a fortnight after that. Read a sample of conversations, especially the handovers. Add an answer to the knowledge base whenever the same question is handed over twice. Update the sources the same day a service, area or hour changes.
When not to use AI web chat
Skip it if nobody will own the content, because an unmaintained agent drifts out of date. Hold off if your services are bespoke and almost every question needs a specialist; the agent will hand over nearly everything and add a step. And never use it for regulated advice.
How the agent splits work with your people is covered in AI live chat with a handover to people. The widget it answers through is set up in configuring the live chat widget. And handed-over chats land in one inbox the whole team works from.
Troubleshooting
The agent invents details. Its instructions do not tell it to stay within the knowledge base. Add a line saying it must only answer from its sources and hand over anything else.
It gives an outdated answer. An old source is still attached. Remove it and add the current version.
It refuses simple questions. The handover rules are too broad. Replace general rules with a specific list of topics.
It books calls in the wrong calendar. The agent is connected to an old calendar. Check the connected calendar and its availability.
Frequently asked questions
How big should the knowledge base be?
Small enough that one person can read all of it in half an hour. If it is larger, much of it is probably out of date or irrelevant.
Can the AI web chat agent answer in other languages?
Decide that in advance. If visitors sometimes write in another language, choose whether the agent replies or hands over, and write that rule into its instructions.
Should the agent ask for contact details?
Yes, but after it has answered the first question. Visitors give details more readily once they have had a useful reply.
Who should own the agent?
Someone who knows your services and clients well, usually a sales or operations lead, not whoever set up the website.