If you searched for testai property management, the job behind the search is usually the same: you want to know whether an AI assistant is safe to put in front of your tenants, landlords and applicants before it starts answering them. This article answers that one question. It sets out how a property management team of 10 to 250 people should test an AI before launch, what scenarios to include, what pass and fail look like, and how to keep testing once it is live. It does not review any particular product; it describes the testing work you should do whichever tool you choose.
Why property management needs stricter testing
A sales enquiry that gets a clumsy AI answer costs you a viewing. A tenant who gets a wrong AI answer about a leak, a deposit or a notice period can cost you a great deal more. Property management conversations involve safety, money held on someone else's behalf and legal timelines. They also involve people who may be stressed, vulnerable or angry.
In the US, the UK and most other markets, tenant selection and treatment are covered by anti-discrimination rules, such as fair housing law in the US and equality law in the UK. An AI that answers applicants differently based on anything other than the stated criteria creates real exposure. That is why testing for a property management AI cannot be a quick check that it sounds friendly.
Decide the scope before you test
You cannot test an AI against an undefined job. Write down, in one page, exactly what the assistant is allowed to do.
A sensible first scope for a lettings and management team looks like this. Answer questions about available properties from current listing data. Book viewings into the right calendar. Take maintenance reports, ask for photos and a description, and log them against the property. Answer general questions about office hours, how to pay rent and who to contact. Hand over to a person for everything else.
Equally important is the list of what it must never do. Never decide or hint at whether an applicant will be accepted. Never give legal advice about notices, evictions or deposit disputes. Never promise a repair date the team has not confirmed. Never handle a complaint beyond acknowledging it and passing it on. On the levelpartner.ai platform, these rules go into the AI agent's instructions, and the hand-over path routes the conversation to a named person in Conversations.
Testing what the AI can do and never testing what it should refuse. Most launch failures in property management come from an assistant that answered a question it should have handed over, not from one that failed to answer a simple one.
Build a scenario library
Testing means running the assistant through realistic conversations and checking each answer. Build a library of scenarios before you start, written from real messages your team has received. Take them from your inbox history with personal details removed.
Group them into five sets.
- Routine questions: rent due dates, payment methods, viewing availability, office hours.
- Maintenance reports: a dripping tap, a broken boiler or furnace in winter, a smell of gas, water coming through a ceiling.
- Applicant questions: whether pets are allowed, what documents are needed, how long referencing or screening takes.
- Boundary cases: questions about deposits held, notices, disputes with neighbours, rent increases.
- Difficult conversations: an angry tenant, a tenant who mentions financial hardship, someone who seems confused or distressed, someone trying to get the AI to reveal another tenant's details.
Each scenario needs an expected outcome written in advance: the correct answer, or the correct hand-over. Without the expected outcome, testers judge on tone and miss the errors that matter.
Suppose the team writes 120 scenarios: 40 routine, 30 maintenance, 20 applicant, 20 boundary and 10 difficult, each with an expected outcome. If three staff each run every scenario through the assistant in their own words, the first round would produce 360 test conversations. That is enough variety to show whether the assistant holds a boundary when the question is phrased three ways. The numbers are illustrative.
What pass and fail look like
Decide the grading rules before you see any results, so nobody softens them because the assistant seems mostly fine.
Grade each test conversation on three things. Was the information correct? Did it stay within scope, handing over where it should? Was the tone appropriate to the situation? An answer can be polite and wrong, and wrong is the grade that counts.
Treat some failures as blocking. Any emergency maintenance scenario, such as a gas smell, that does not produce an immediate instruction to call the emergency line and an alert to the on-call person is a blocking failure. Any answer that reveals another person's information is blocking. Any answer that appears to judge an applicant's suitability is blocking. One blocking failure means you fix and re-run the whole set, not just that scenario.
For everything else, set a standard the team agrees with, and only launch when the assistant meets it on two consecutive rounds.
How to run testai property management checks on the platform
On the levelpartner.ai platform the assistant is a Conversation AI bot, and two built-in tools carry the offline rounds. The Knowledge Base Retrieval Tester shows what the bot can find for a question and which sources it used. Test your Bot shows the full reply the tenant would get. Run every scenario through the first, then the second.
Round one: check what the knowledge base returns
- Go to AI Agents and open Knowledge Base.
- Open the knowledge base that the bot is connected to.
- Wait until any new or changed sources have finished processing.
- Open the Knowledge Base Retrieval Tester.
- Type a scenario from your library, worded the way a tenant would write it.
- Read the answer and the retrieved sources.
- Correct any unclear, missing or outdated content.
- Run the same question again.
This round catches the stale data problem before tone ever comes into it. If the deposit scenario retrieves last year's policy page, no amount of prompt work will fix the reply.
Round two: check the bot's full reply
- Go back to the Conversation AI bot.
- Open Bot Training.
- Enter the scenario in Test your Bot.
- Try the usual wording, a paraphrase, an FAQ and a question outside the bot's scope.
- Click the reset icon before moving to a different scenario.
- Check the reply against the approved source content and your expected outcome.
Record the grade for each scenario in your library, not in your head. The reset matters more than it looks: without it, the bot carries context from the previous test, and a boundary case can pass only because the last conversation steered it.
What most writing on AI for property management gets wrong
Most writing on the subject treats testing as a pre-launch event. You test, you pass, you go live. In practice, the assistant's answers change every time your data changes: a property is let, a policy updates, a contractor changes, a new building joins the portfolio. An assistant that passed in March can give a wrong answer in May because the knowledge it reads went stale.
So testing is a routine, not a gate. Keep the scenario library, re-run a sample of it every month and the full set after any significant change to instructions or data. Add a new scenario every time a real conversation goes wrong. For the wider design of AI across a property management operation, see AI for property management.
Live testing with a safety net
Once the assistant passes offline, move to live testing with a narrow audience, and start in Suggestive mode rather than Auto-Pilot. In Suggestive mode the bot drafts each reply inside the conversation and a member of the team reads it, edits it if needed and sends it. Nothing reaches a tenant unchecked, and every rewritten draft is a free new test case. Switch the bot on for one channel, such as website chat for applicants, and only move that channel to Auto-Pilot once a fortnight of drafts has gone out with no blocking failures.
Keep an easy route to a human visible in every conversation, and once on Auto-Pilot tell people plainly that they are talking to an AI. Every AI conversation sits in Conversations alongside the team's own, so the reviewer can step in mid-thread and the tenant sees one continuous history. When a live reply looks wrong, open Response Info beside the message to see which knowledge base sources it drew on, then fix the source.
Imagine the assistant drafts replies for 210 applicant chats in its first fortnight in Suggestive mode, and staff rewrite 9 before sending: 6 with outdated listing details, 2 hand-overs that came too late and 1 tone problem with an upset applicant. Each of the 9 would become a new scenario, and the knowledge base would be corrected before the next channel goes live. The figures are hypothetical.
The trade-off
Testing like this is slow. Building a scenario library, running rounds with several staff and supervising live conversations can take a few weeks before the assistant handles any volume on its own. A competitor who switches an AI on in an afternoon will appear to move faster.
The price of that speed shows up later, in a tenant complaint, a missed emergency or an applicant who believes they were treated unfairly. In property management, the cost of one serious error is larger than the hours saved by skipping tests. Take the slower route, and widen the scope one channel at a time.
Who owns testing
Name one person who owns the assistant, usually the head of property management or an operations lead. They own the scenario library, the monthly re-test and the decision to widen scope. The people who write the expected outcomes should be those who answer tenants today, because they know which answers cause trouble.
If you run sales alongside lettings, the same testing approach applies to the sales assistant, with different scenarios. For how a brokerage splits AI by stage and owner, see AI for real estate agents across a team. For conversation design that makes testing easier from the start, see designing real estate conversations for AI.
Frequently asked questions
How many test scenarios are enough before launch?
Enough to cover every type of message your team receives and every boundary you have set. For a small first scope that is often several dozen; for a full lettings and management operation it can run to more than a hundred. Coverage of risky cases matters more than the total.
Should tenants be told they are talking to an AI?
Yes. Say so in the first message and offer a person at any point. Tenants who discover it later lose trust in every answer they were given, including the correct ones.
Can the AI handle emergency maintenance reports?
It should recognise them and escalate immediately, with clear safety instructions and an alert to the on-call person. It should not try to diagnose or schedule the repair itself. Test every emergency scenario in each round.
Does Suggestive mode replace the offline rounds?
No. Suggestive mode tests the assistant on whatever tenants happen to ask that fortnight, which rarely includes a gas leak or a deposit dispute. The scenario library makes sure the risky cases are tested on purpose.
How often should we re-test after launch?
Re-run a sample monthly and the full library after any change to instructions, policies or the property data the assistant reads. Add a scenario each time a live conversation goes wrong, so the library grows with your real risks.