After creating or updating your Knowledge Base, you should test how Novus AI uses the information when answering real Discord questions.
A Knowledge Base entry may look complete when you read it, but testing helps determine whether the information is clear enough for the AI to use correctly.
Why Test Your Knowledge Base?
Testing helps you identify problems such as:
Missing information
Unclear instructions
Incomplete conditions
Conflicting Knowledge Base entries
Outdated information
Incorrect AI Routing
Questions that unnecessarily trigger fallback responses
Answers that include unsupported information
The goal is not to make Novus AI memorize exact questions.
The goal is to make sure your Knowledge Base contains enough clear information for the assistant to answer different versions of the same question.
Start With a Known Answer
Choose a Knowledge Base entry with a clear answer.
For example, if you have an entry explaining account registration, ask:
How do I create an account?
Review the response and compare it with the information stored in your Knowledge Base.
The important details should remain accurate.
Ask the Same Question Differently
Members will not always use the same wording as your Knowledge Base.
Test several variations.
For example:
How do I register?
Where can I make an account?
I need an account. What should I do?
Can you tell me where registration is?
These questions have similar intent even though they use different wording.
Novus AI should be able to use the relevant Knowledge Base information when answering them.
Test Short and Informal Questions
Discord users often ask very short questions.
For example:
registration link?
where download?
how to recover password
event time?
Include these types of messages in your testing.
Your Knowledge Base should contain enough context for Novus AI to understand what information is relevant.
Test Detailed Questions
You should also test more specific questions.
For example:
I already downloaded the full client but the launcher isn't opening. What should I do?
Detailed questions can reveal whether your Knowledge Base contains enough troubleshooting information for specific situations.
Verify Important Details
Pay particular attention to information where accuracy matters.
Check:
Prices
Dates
Times
Percentages
Requirements
Rewards
Limits
Durations
Commands
URLs
Discord mentions
Procedures
Restrictions
Novus AI should not change these values unless your Knowledge Base or instructions provide a reason to do so.
Test Conditions and Exceptions
If your Knowledge Base contains conditions, test both eligible and ineligible situations.
For example, suppose your Knowledge Base says:
Members receive the reward only after completing the required objective.
Test:
What do I get after completing the objective?
and:
Can I get the reward without completing the objective?
This helps verify that Novus AI understands the condition rather than only remembering the reward.
Test Missing Information
Ask a question related to a topic where the requested detail is intentionally not provided.
For example, if your Knowledge Base lists a service but does not provide a confirmed price, ask:
How much does this service cost?
If your AI Instructions prohibit guessing, Novus AI should not invent a price.
It should use the configured fallback or escalation behavior instead.
Test Conflicting Information
If Novus AI provides different answers to similar questions, check whether multiple Knowledge Base entries contain conflicting information.
Search for duplicate:
Prices
Dates
Rules
Requirements
Links
Procedures
Schedules
Remove or update outdated information and test again.
Test AI Routing
Sometimes the Knowledge Base itself is correct, but the wrong route is handling the question.
If you use AI Routing, verify that the question is being handled by the appropriate route.
For example, a technical troubleshooting question should not accidentally use instructions intended for payment support.
If routing is incorrect, improve the route description instead of unnecessarily rewriting accurate Knowledge Base information.
Test Different Languages
If your community is multilingual, ask Knowledge Base questions using the languages commonly used by your members.
Verify that:
The correct information is used.
Important numbers remain unchanged.
Commands remain correct.
Links remain correct.
Discord mentions are preserved.
Conditions and restrictions remain accurate.
You can also test mixed-language questions when they are common in your community.
Test Follow-Up Questions
Members may continue a conversation instead of repeating the entire question.
For example:
Member: How do I register?
After Novus AI responds:
Member: What if I already have an account?
Testing follow-up questions helps you evaluate how the assistant uses conversation context together with your Knowledge Base.
Identify the Source of an Incorrect Response
When a response is wrong, first determine which configuration needs improvement.
If the fact is incorrect, check your Knowledge Base.
If information is missing, add or clarify the relevant Knowledge.
If the AI is behaving incorrectly, review your AI Instructions.
If the wrong support topic is being used, review AI Routing.
If the assistant guesses missing information, review your fallback instructions.
If the bot does not respond at all, review your AI provider, response settings, and Discord permissions.
Changing the correct configuration is better than repeatedly adding duplicate Knowledge Base entries.
Don't Fix Everything by Adding More Knowledge
When an answer is incorrect, adding another Knowledge Base entry is not always the solution.
Too much duplicated information can create additional problems.
Before adding new content:
Check whether the information already exists.
Review whether it is clearly written.
Check for conflicting information.
Verify the correct route is being used.
Review the AI Instructions.
Make the smallest necessary change.
Test again.
Build a Test Question List
For important Knowledge Base topics, maintain a collection of realistic questions.
For example:
Account Registration
How do I register?
Where is registration?
Can I make an account?
Registration link?
Download
Where can I download?
Do you have a full client?
Is there a manual patch?
How do I install it?
Support
Where can I get help?
How do I create a ticket?
Who should I contact?
Use these questions whenever you make major changes to your Knowledge Base or AI configuration.
Test After Every Important Update
You do not need to test your entire Knowledge Base after every small change.
However, when changing important information such as:
Prices
Rules
Services
Procedures
Event details
Account instructions
Support processes
test several related questions immediately after saving the update.
Knowledge Base Testing Checklist
Before considering an important Knowledge Base topic ready, verify that:
The correct information is returned.
Different question wording still works.
Important values remain accurate.
Conditions and exceptions are respected.
Missing information is not invented.
The correct AI Route handles the question.
Fallback behavior works when needed.
Multilingual questions remain accurate.
Follow-up questions work appropriately.
No outdated information is being used.
Continue Improving With Real Questions
Your members will eventually ask questions you did not anticipate.
Use those conversations to improve your Knowledge Base.
When Novus AI cannot answer a legitimate question:
Verify the correct answer.
Determine whether the information belongs in the Knowledge Base.
Add or improve the relevant content.
Test the original question again.
This creates a continuous improvement cycle based on the actual needs of your community.
Knowledge Base Setup Complete
You now understand how to create, write, organize, edit, remove, and test Knowledge Base information.
The next step is controlling how Novus AI handles different types of questions using specialized instructions and routing.
Continue to the AI Routing documentation.