It's a good idea to introduce your AI agent in stages and monitor the impact as you go. Before you set Fin AI Agent live, you'll have plenty of opportunities to review the end-to-end experience for both your customers and your teammates.
Below are some ways to test Fin AI Agent internally and gradually roll Fin out to your customers.
Preview Fin AI Agent
Preview your content with Fin AI Agent
As soon as you start adding content for Fin AI Agent, you'll be able to use the interactive Messenger preview on the right.
For previewing Fin AI Agent, we use a test user called "Preview User". If your articles are restricted to certain audiences only (and the Preview User does not match these rules), Fin AI Agent will not be able to answer these questions in preview.
You can add a filter "Email contains preview-user" in order to make your articles visible to the Preview User.
Preview your Workflow with Fin AI Agent
When you add Fin to a Workflow, you'll be able to experience a conversation with Fin AI Agent using your Workflow.
Simply click the Preview button at the top of the Workflows builder.
This will give you an interactive preview of the Messenger experience when talking to Fin. Simply start asking questions you would expect to find answers to in the support content you added.
If you want to test a Fin workflow over email, send an email to your support email address configured to receive email in Intercom or to the email address in Settings > Email > Email forwarding.
You can also preview Fin AI Agent from any website where you have the Messenger installed. Click Preview from a website and enter the address, then click Open website to load the Messenger with Fin on that website.
To preview Fin AI Agent from a website, it'll need to be set live and you must have the Messenger installed.
Target Fin AI Agent at a specific audience
The audience rules for Fin AI Agent are configured differently depending on how you've set Fin up:
From the quick setup
Audience rules from the quick setup are configured in the Who will see Fin section.
From a Workflow
Audience rules for a Workflow are configured in the trigger settings by clicking on the trigger or first step in the Workflows builder.
Without Workflows
If you don't have access to Workflows, you can still set Fin AI Agent live for your team by targeting them in the audience rules under the Who will see Fin section from Fin AI Agent > Overview > Set up and go live.
Note:
Audience rules will apply to all channels you’ve enabled for Fin AI Agent.
You'll need to have a live Messenger installed before you can test Fin on your website.
Set Fin AI Agent live for your team
To share Fin AI Agent with your team and allow them to test it themselves, you can set Fin AI Agent live on your website and use audience rules to target your team only. This will also allow you to check that your handover rules are running as you intended.
For example, you could select ‘Users’ and use the filter ‘Email contains…’ to target teammates with your company email address.
Now when you click Set live, Fin AI Agent will be live for the users you specified. Those users will be able to interact with Fin by asking it questions in the Messenger.
Create a set of control questions to share with your team and ask them to note whether Fin provided the best response or not. This can also be a great way to identify gaps in your support content.
If you test Fin in your live environment, Fin will be billed per resolution.
However, if you or your teammate has asked to "Get more help" or speak to your team, the conversation is considered 'unresolved' and you would not be charged for this.
Roll Fin AI Agent out to customers
We recommend rolling Fin AI Agent out in stages by customer segment, such as starting with lower-tiered customers who would benefit the most from self-service, or by region. This approach allows you to supervise Fin closely and feel confident with how Fin works in your product.
Here are some ways you can identify and target customers:
Specific tiers
For example, you could enable Fin AI Agent for everyone except VIP customers by specifying ‘Users’ and using a custom data attribute you’re tracking such as ‘VIP customer is false’.
Or, target only self-serve customers or users on a specific price plan by using a custom data attribute such as ‘Plan is Free’.
An audience can become super targeted by combining And / Or filters. For example, you could use the ‘Last seen’ and ‘Region’ attributes to target only new users in a specific region.
Users or leads only
You can also make Fin AI Agent only available to logged in users, or only available to visitors to your website who reach out through the Messenger and become leads. Simply select this audience from the dropdown.
You can target users of a particular part of your product by selecting ‘Users’ and a custom data attribute such as ‘Projects is greater than 1’.
Specific page URLs
You can also enable Fin AI Agent only on specific pages by using the ‘Current Page URL’ attribute and entering the URL for that page. For example, you could target it at customers who are visiting your support page.
Or, if you want to see how much impact it has on reducing conversation volume for your team, you could target a page which drives a high volume of conversations.
Make sure you understand how audience rules and filters work to ensure you’re capturing the right audience and Fin AI Agent is able to get involved in these conversations.
See the impact of Fin AI Agent
Using reporting
You’ll have access to an AI agent performance report in Reports > Automation > Fin AI Agent.
In this report, you’ll be able to see:
The AI agent’s involvement in all conversations.
The answer rate, deflection rate, and resolution rate of conversations in which the AI agent is involved.
The AI agent’s impact over time.
Breakdown of resolved and unresolved conversations.
List of conversations the AI agent didn’t provide an answer for.
Identify gaps in content by reviewing unanswered conversations in the report and create new support content which covers these.
Learn more about the Fin AI Agent report.
Monitoring responses
The Fin AI Agent section in Help Desk allows you to monitor all conversations where your AI agent was involved. You can easily review these conversations and take note of any content gaps to update or create new articles.
Learn more about managing AI Agent conversations.
Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts