Agent and AI Insights
For a conceptual overview of Agent and AI Insights, see Agent and AI Insights in Product Concepts.
7.1 Accessing Agent and AI Insights
Navigation
- Click Agent and AI Insights in the sidebar
- The chat interface opens
Permissions Required
You need the Agent External Use permission to access Agent and AI Insights. Contact your admin if you can't access this feature.
Interface Overview
| Element | Description |
|---|---|
| Chat Area | Conversation history |
| Input Box | Where you type questions |
| Send Button | Submit your question |
| History | Previous conversations |
7.2 Asking Effective Questions
Question Types
Performance Questions:
- "What's my conversion rate this week?"
- "How many impressions did we get yesterday?"
- "Show me CTR for the last 30 days"
Comparison Questions:
- "Compare treatment A vs treatment B"
- "How does this week compare to last week?"
- "Which journey is performing best?"
Trend Questions:
- "Is engagement trending up or down?"
- "Show me the daily conversion trend"
- "What happened to purchases over the past month?"
Investigation Questions:
- "Why did clicks drop on Monday?"
- "What's driving the conversion increase?"
- "Which surface has the highest engagement?"
Tips for Better Answers
Be Specific:
- Less effective: "How are things going?"
- More effective: "What's the conversion rate for the Welcome Journey this week?"
Include Context:
- Mention specific journeys, treatments, or time periods
- Reference goals by name
Ask Follow-Up Questions:
- Dig deeper into initial findings
- Ask "why" to understand causes
Example Conversations
Example 1: Performance Check
You: How did our New User Onboarding journey perform this week?
AI: The New User Onboarding journey had strong performance this week:
- 45,230 impressions (up 12% from last week)
- 2,156 clicks (CTR: 4.8%)
- 312 conversions (conversion rate: 0.69%)
- Uplift vs control: +23%
Top performing treatment: "Welcome Modal v2" with 5.2% CTR.
Example 2: Investigation
You: Why did engagement drop yesterday?
AI: Looking at yesterday's data, I see a 35% drop in engagement.
Potential causes:
1. The "Holiday Promo Banner" was paused at 2pm
2. This treatment typically drives 40% of daily engagement
3. Other treatments maintained normal levels
Recommendation: Review whether the pause was intentional,
and consider reactivating if appropriate.
7.3 Understanding Responses
Response Format
Agent and AI Insights presents information in various formats:
Summary Statistics:
Total Conversions: 1,234
Conversion Rate: 3.2%
Week-over-Week Change: +15%
Tables:
| Treatment | Impressions | CTR |
|---|---|---|
| Banner A | 10,000 | 4.5% |
| Banner B | 8,500 | 5.2% |
Insights: Key observations highlighted in natural language.
Recommendations: Suggested actions based on data analysis.
Interpreting Results
Positive Indicators:
- Uplift above baseline
- Improving trends
- Above-average CTR
Areas for Investigation:
- Declining metrics
- Below-average performance
- Unexpected changes
7.4 Common Use Cases
Daily Check-In
Start your day with quick questions:
- "What happened overnight?"
- "Any anomalies in the last 24 hours?"
- "How are my key journeys performing today?"
Journey Monitoring
Track active journeys:
- "How is the Holiday Journey doing?"
- "What's the CTR on our new treatment?"
- "Are we on track for our conversion goal?"
Performance Investigation
Dig into issues:
- "Why did journey X underperform?"
- "What's different about treatment A vs B?"
- "Which audience segment converts best?"
Report Preparation
Gather data for reports:
- "Summarize this month's performance"
- "What were the top 5 treatments by conversions?"
- "Give me weekly trends for Q4"
Optimization Planning
Get improvement ideas:
- "Which treatments should I focus on improving?"
- "Where are we losing users in the funnel?"
- "What optimization opportunities do you see?"
7.5 Prompt Templates
Use these templates as starting points:
Performance Review
How did [journey/treatment name] perform in [time period]?
Comparison
Compare [item A] vs [item B] for [time period]
Trend Analysis
Show me the [metric] trend for [time period]
Investigation
Why did [metric] [increase/decrease] on [date/period]?
Top Performers
What were the top [N] [treatments/journeys] by [metric] in [time period]?
Recommendations
What optimization opportunities do you see for [journey/goal]?
7.6 Best Practices
Start Broad, Then Narrow
- Begin with high-level questions
- Identify areas of interest
- Ask follow-up questions for details
Be Clear About Time Periods
Specify time ranges explicitly:
- "Last 7 days"
- "This month"
- "December 1-15"
Reference Specific Entities
Use exact names when possible:
- Treatment names
- Journey names
- Goal names
Save Important Findings
When you get valuable insights:
- Screenshot or copy important findings
- Use for presentations and reports
- Share with team members
Verify Unusual Results
If something seems unexpected:
- Ask clarifying questions
- Cross-check with Analytics dashboards
- Consider data timing/delays
7.7 Limitations
What Agent and AI Insights Can Do
- Answer questions about your data
- Provide summary statistics
- Identify trends and patterns
- Offer optimization suggestions
What Agent and AI Insights Cannot Do
| Limitation | Alternative |
|---|---|
| Make changes to treatments | Use Treatment editing |
| Create new journeys | Use Journey Portfolio |
| Access real-time data | Data has processing delay |
| Predict future outcomes | Use historical analysis |
Data Freshness
Agent and AI Insights uses processed data, which may have a delay:
- Most metrics: 1-4 hours
- Some aggregations: Up to 24 hours
For real-time counts, check the main dashboards.
7.8 Troubleshooting
"I don't have data for that"
Possible Causes:
- Time period is too recent (data delay)
- Journey/treatment doesn't exist
- You may not have access to that data
Try:
- Use a slightly older time period
- Verify the name of the journey/treatment
- Contact admin about data access
Generic or Unhelpful Responses
Tips:
- Be more specific in your question
- Include time period
- Name specific treatments or journeys
- Ask simpler, more focused questions
Slow Responses
Normal Wait:
- Complex queries may take 10-30 seconds
- Large time ranges take longer
If Too Slow:
- Try a shorter time period
- Ask a simpler question first
- Refresh and try again
7.9 Privacy and Data
What Data Can Agent and AI Insights Access?
- Your company's Console data
- Treatment performance metrics
- Aggregated user statistics
What Agent and AI Insights Cannot Access
- Individual user data (PII)
- Other companies' data
- System configuration details
Data Security
- All queries are processed securely
- Results are scoped to your permissions
- Conversation history is private to you
7.10 Advanced Features
Multi-Turn Conversations
Agent and AI Insights remembers context within a conversation:
You: How did Banner A perform last week?
AI: Banner A had 10,000 impressions and 450 clicks (4.5% CTR).
You: And this week?
AI: This week, Banner A had 12,000 impressions and 600 clicks
(5.0% CTR) - a 0.5 percentage point improvement.
Comparative Analysis
Ask for direct comparisons:
You: Compare all my active treatments by CTR for the last 30 days
AI: Here are your active treatments ranked by CTR:
| Treatment | CTR | Impressions |
|-----------|-----|-------------|
| Modal B | 6.2% | 15,000 |
| Banner A | 4.5% | 45,000 |
| Push v2 | 3.8% | 28,000 |
| *(additional rows omitted)* | | |
Anomaly Detection
Ask about unusual patterns:
You: Were there any anomalies in the last week?
AI: I detected the following anomalies:
1. Dec 15: Push notification CTR dropped 40% (7pm-11pm)
2. Dec 17: Conversion rate spiked 85% (likely due to promo)
Next Section
Continue to Section 8: QA & Testing for quality assurance documentation.