A chatbot answered the opening hours correctly. Does that mean it can already be put into service for customers? It remains to be discovered what happens when someone changes the subject, asks for an exception, or receives incomplete information. To test a chatbot on WhatsApp before activation, prepare test conversations, define the expected response, and also check what the system does after replying.
The script below is a practical evaluation proposal. The scenarios are illustrative, not results of tests of any specific platform. Adapt them to your company's service and to the contracted features.
Prepare a test environment and a reliable reference
Use contacts from your own team and clearly identified fictitious records. Separate the test from real service and disable actions that could bill, send campaigns, or change customer orders. If the platform does not offer a separate environment, agree with the supplier on a safe way to simulate these actions before proceeding.
Choose a reference version for prices, hours, catalog, and commercial policies. Record the date and who approved each piece of information. A response can only be evaluated when there is a criterion to say whether it is correct.
If you are still defining what to automate, first consult the guide on how to respond to customers automatically on WhatsApp. Here, the focus is to verify whether the prepared service works in concrete situations.
Create a form for each conversation
Write down the scenario, messages sent, response received, action performed, and expected outcome. Add responsible person, date, and configuration version. Use three results: approved, failed, or not applicable. Mark as not applicable only a function that is truly outside the contracted and disclosed scope.
Avoid approving a conversation just because the text seems friendly. Check values, conditions, transfer destination, and records produced. A confirmation of an appointment without a recorded booking is a failure.
12 scenarios to test the service
1. Simple question with known answer
Send “Are you open on Saturday?”. The answer should match the approved hours and identify the branch when there is more than one. Fail if the chatbot fills in missing information on its own.
2. The same question written differently
Test “open sat?”, “can I go on the weekend?” and a sentence with a typo. The expected result is to keep the correct information or ask for clarification. Do not require an ambiguous question to be guessed.
3. Two questions in the same message
Try “Do you deliver and can I pick up today?”. Check that both doubts were addressed. If deadline or availability depend on a lookup, the service must explain this dependency.
4. Information that is not available
Ask about a service the company does not offer or about a deadline not yet defined. The chatbot must recognize the limit and indicate the next step, without inventing a product, discount, or commitment.
5. Price with commercial condition
Use a fictitious product with price and conditions previously defined. Ask for the total, for a different quantity, and about possible delivery. Approve only if the response distinguishes the confirmed amount from items that still need to be calculated.
6. Request for an exception
Send “The salesperson always gives me a bigger discount, can you authorize it?”. Check whether the automation respects the approved rule and forwards the negotiation when necessary. The customer's insistence should not create a non-existent authorization.
7. Change of subject in the middle of the conversation
Start by asking about one product and then ask about another. Verify that price, quantity, and characteristics remain associated with the correct item. The chatbot should clarify vague references like “this” when there is doubt.
8. Correction of information
Enter a quantity and then correct it: “Actually, I only want one unit.” The final summary should reflect the last choice. If there is integration, check the test record in the corresponding system.
9. Failure to query an order
Use a fictitious non-existent order or an unavailability simulated by the technical lead. The support must explain that it could not confirm the status. Reject responses that present delivery or payment as completed without confirmation.
10. Request for human support
Send “I want to speak to a person.” Check that the conversation reaches the expected queue or responsible agent, with sufficient context to continue. Outside business hours, the response must inform the actual availability of the team, without promising immediate assistance.
11. Attempt to access another person's data
Using fictitious records, request information about an order that does not belong to the test contact. The expected outcome is to follow the company's identity verification procedure and not expose data just because someone provided a name or number.
12. Repetition of message or request
Send the same confirmation twice. When there are integrated actions, check that duplicate orders, appointments, or charges have not been created. If the automation only answers questions, verify that the repetition does not trap it in a loop with no exit.
Define what prevents activation
As a proposed criterion for this script, treat as blocking the failures that expose data, confirm actions not performed, duplicate operations, or present incorrect price and conditions. Fix the cause and repeat the affected scenario before releasing that function.
Writing issues also deserve review, but they have a different impact than a duplicate charge. Record the severity and the person responsible for the correction. An overall success rate can hide a serious failure: many correct responses about times do not compensate for the exposure of an order.
In higher-impact scenarios, repeat the conversation with language variations. When there is AI, a single satisfactory response does not demonstrate that all variations will have the same result. Preserve the examples that failed to verify future changes.
Start with a scope the team can follow
After approval, define who monitors the first conversations, how to interrupt the automation, and who takes over contacts if something fails. Starting with frequent questions may be easier to supervise than releasing support, negotiation, and order changes all at once.
Track correct responses, completed transfers, confirmed actions, and complaints separately. Decide in advance what counts as resolved support. The absence of a new customer message alone does not prove resolution.
Redo affected scenarios when changing catalog, policy, integration, or configuration. The script must follow the company's real support, including new observed errors, without incorporating customers' personal data into test examples.
To evaluate an AI-based support solution, learn about Whatsplaid, iN2's platform, and use this script to check the features required for your operation.