Agent and AI Insights

For a conceptual overview of Agent and AI Insights, see Agent and AI Insights in Product Concepts.

​

7.1 Accessing Agent and AI Insights

​

Navigation

1. Click Agent and AI Insights in the sidebar
2. The chat interface opens

​

Permissions Required

​

Interface Overview

​

7.2 Asking Effective Questions

​

Question Types

• “What’s my conversion rate this week?”
• “How many impressions did we get yesterday?”
• “Show me CTR for the last 30 days”

• “Compare treatment A vs treatment B”
• “How does this week compare to last week?”
• “Which journey is performing best?”

• “Is engagement trending up or down?”
• “Show me the daily conversion trend”
• “What happened to purchases over the past month?”

• “Why did clicks drop on Monday?”
• “What’s driving the conversion increase?”
• “Which surface has the highest engagement?”

​

Tips for Better Answers

• Less effective: “How are things going?”
• More effective: “What’s the conversion rate for the Welcome Journey this week?”

• Mention specific journeys, treatments, or time periods
• Reference goals by name

• Dig deeper into initial findings
• Ask “why” to understand causes

​

Example Conversations

You: How did our New User Onboarding journey perform this week?

AI: The New User Onboarding journey had strong performance this week:
- 45,230 impressions (up 12% from last week)
- 2,156 clicks (CTR: 4.8%)
- 312 conversions (conversion rate: 0.69%)
- Uplift vs control: +23%

Top performing treatment: "Welcome Modal v2" with 5.2% CTR.

You: Why did engagement drop yesterday?

AI: Looking at yesterday's data, I see a 35% drop in engagement.
Potential causes:
1. The "Holiday Promo Banner" was paused at 2pm
2. This treatment typically drives 40% of daily engagement
3. Other treatments maintained normal levels

Recommendation: Review whether the pause was intentional,
and consider reactivating if appropriate.

​

7.3 Understanding Responses

​

Response Format

Total Conversions: 1,234
Conversion Rate: 3.2%
Week-over-Week Change: +15%

​

Interpreting Results

• Uplift above baseline
• Improving trends
• Above-average CTR

• Declining metrics
• Below-average performance
• Unexpected changes

​

7.4 Common Use Cases

​

Daily Check-In

• “What happened overnight?”
• “Any anomalies in the last 24 hours?”
• “How are my key journeys performing today?”

​

Journey Monitoring

• “How is the Holiday Journey doing?”
• “What’s the CTR on our new treatment?”
• “Are we on track for our conversion goal?”

​

Performance Investigation

• “Why did journey X underperform?”
• “What’s different about treatment A vs B?”
• “Which audience segment converts best?”

​

Report Preparation

• “Summarize this month’s performance”
• “What were the top 5 treatments by conversions?”
• “Give me weekly trends for Q4”

​

Optimization Planning

• “Which treatments should I focus on improving?”
• “Where are we losing users in the funnel?”
• “What optimization opportunities do you see?“

​

7.5 Prompt Templates

​

Performance Review

How did [journey/treatment name] perform in [time period]?

​

Comparison

Compare [item A] vs [item B] for [time period]

​

Trend Analysis

Show me the [metric] trend for [time period]

​

Investigation

Why did [metric] [increase/decrease] on [date/period]?

​

Top Performers

What were the top [N] [treatments/journeys] by [metric] in [time period]?

​

Recommendations

What optimization opportunities do you see for [journey/goal]?

​

7.6 Best Practices

​

Start Broad, Then Narrow

1. Begin with high-level questions
2. Identify areas of interest
3. Ask follow-up questions for details

​

Be Clear About Time Periods

• “Last 7 days”
• “This month”
• “December 1-15”

​

Reference Specific Entities

• Treatment names
• Journey names
• Goal names

​

Save Important Findings

• Screenshot or copy important findings
• Use for presentations and reports
• Share with team members

​

Verify Unusual Results

• Ask clarifying questions
• Cross-check with Analytics dashboards
• Consider data timing/delays

​

7.7 Limitations

​

What Agent and AI Insights Can Do

• Answer questions about your data
• Provide summary statistics
• Identify trends and patterns
• Offer optimization suggestions

​

What Agent and AI Insights Cannot Do

​

Data Freshness

• Most metrics: 1-4 hours
• Some aggregations: Up to 24 hours

​

7.8 Troubleshooting

​

”I don’t have data for that”

• Time period is too recent (data delay)
• Journey/treatment doesn’t exist
• You may not have access to that data

• Use a slightly older time period
• Verify the name of the journey/treatment
• Contact admin about data access

​

Generic or Unhelpful Responses

• Be more specific in your question
• Include time period
• Name specific treatments or journeys
• Ask simpler, more focused questions

​

Slow Responses

• Complex queries may take 10-30 seconds
• Large time ranges take longer

• Try a shorter time period
• Ask a simpler question first
• Refresh and try again

​

7.9 Privacy and Data

​

What Data Can Agent and AI Insights Access?

• Your company’s Console data
• Treatment performance metrics
• Aggregated user statistics

​

What Agent and AI Insights Cannot Access

• Individual user data (PII)
• Other companies’ data
• System configuration details

​

Data Security

• All queries are processed securely
• Results are scoped to your permissions
• Conversation history is private to you

​

7.10 Advanced Features

​

Multi-Turn Conversations

You: How did Banner A perform last week?
AI: Banner A had 10,000 impressions and 450 clicks (4.5% CTR).

You: And this week?
AI: This week, Banner A had 12,000 impressions and 600 clicks
    (5.0% CTR) - a 0.5 percentage point improvement.

​

Comparative Analysis

You: Compare all my active treatments by CTR for the last 30 days

AI: Here are your active treatments ranked by CTR:
| Treatment | CTR | Impressions |
|-----------|-----|-------------|
| Modal B   | 6.2% | 15,000     |
| Banner A  | 4.5% | 45,000     |
| Push v2   | 3.8% | 28,000     |
| *(additional rows omitted)* | | |

​

Anomaly Detection

You: Were there any anomalies in the last week?

AI: I detected the following anomalies:
1. Dec 15: Push notification CTR dropped 40% (7pm-11pm)
2. Dec 17: Conversion rate spiked 85% (likely due to promo)

​

Next Section