Section 7: Agent and AI Insights (Analyst Agent)
Agent and AI Insights is an AI-powered assistant that helps you understand your data and answer questions about your marketing performance.
7.1 What is Agent and AI Insights?
Overview
Agent and AI Insights (powered by the "Analyst Agent") is a conversational AI interface that lets you ask questions about your data in natural language.
Instead of navigating complex dashboards or building reports, you can simply ask:
- "How are my purchase conversions trending?"
- "Which treatments are performing best this week?"
- "Why did engagement drop yesterday?"
Key Capabilities
| Capability | Description |
|---|---|
| Data Queries | Ask about metrics, trends, and performance |
| Analysis | Request comparisons and deep dives |
| Insights | Get AI-generated observations |
| Recommendations | Receive optimization suggestions |
How It Works
- You ask a question in natural language
- The AI interprets your intent
- It queries your data
- It presents findings in an easy-to-understand format
7.2 Accessing Agent and AI Insights
Navigation
- Click Agent and AI Insights in the sidebar
- The chat interface opens
Permissions Required
You need the Agent External Use permission to access Agent and AI Insights. Contact your admin if you can't access this feature.
Interface Overview
| Element | Description |
|---|---|
| Chat Area | Conversation history |
| Input Box | Where you type questions |
| Send Button | Submit your question |
| History | Previous conversations |
7.3 Asking Effective Questions
Question Types
Performance Questions:
- "What's my conversion rate this week?"
- "How many impressions did we get yesterday?"
- "Show me CTR for the last 30 days"
Comparison Questions:
- "Compare treatment A vs treatment B"
- "How does this week compare to last week?"
- "Which journey is performing best?"
Trend Questions:
- "Is engagement trending up or down?"
- "Show me the daily conversion trend"
- "What happened to purchases over the past month?"
Investigation Questions:
- "Why did clicks drop on Monday?"
- "What's driving the conversion increase?"
- "Which surface has the highest engagement?"
Tips for Better Answers
Be Specific:
- Less effective: "How are things going?"
- More effective: "What's the conversion rate for the Welcome Journey this week?"
Include Context:
- Mention specific journeys, treatments, or time periods
- Reference goals by name
Ask Follow-Up Questions:
- Dig deeper into initial findings
- Ask "why" to understand causes
Example Conversations
Example 1: Performance Check
You: How did our New User Onboarding journey perform this week?
AI: The New User Onboarding journey had strong performance this week:
- 45,230 impressions (up 12% from last week)
- 2,156 clicks (CTR: 4.8%)
- 312 conversions (conversion rate: 0.69%)
- Uplift vs control: +23%
Top performing treatment: "Welcome Modal v2" with 5.2% CTR.
Example 2: Investigation
You: Why did engagement drop yesterday?
AI: Looking at yesterday's data, I see a 35% drop in engagement.
Potential causes:
1. The "Holiday Promo Banner" was paused at 2pm
2. This treatment typically drives 40% of daily engagement
3. Other treatments maintained normal levels
Recommendation: Review whether the pause was intentional,
and consider reactivating if appropriate.
7.4 Understanding Responses
Response Format
Agent and AI Insights presents information in various formats:
Summary Statistics:
Total Conversions: 1,234
Conversion Rate: 3.2%
Week-over-Week Change: +15%
Tables:
| Treatment | Impressions | CTR |
|---|---|---|
| Banner A | 10,000 | 4.5% |
| Banner B | 8,500 | 5.2% |
Insights: Key observations highlighted in natural language.
Recommendations: Suggested actions based on data analysis.
Interpreting Results
Positive Indicators:
- Uplift above baseline
- Improving trends
- Above-average CTR
Areas for Investigation:
- Declining metrics
- Below-average performance
- Unexpected changes
7.5 Common Use Cases
Daily Check-In
Start your day with quick questions:
- "What happened overnight?"
- "Any anomalies in the last 24 hours?"
- "How are my key journeys performing today?"
Journey Monitoring
Track active journeys:
- "How is the Holiday Journey doing?"
- "What's the CTR on our new treatment?"
- "Are we on track for our conversion goal?"
Performance Investigation
Dig into issues:
- "Why did journey X underperform?"
- "What's different about treatment A vs B?"
- "Which audience segment converts best?"
Report Preparation
Gather data for reports:
- "Summarize this month's performance"
- "What were the top 5 treatments by conversions?"
- "Give me weekly trends for Q4"
Optimization Planning
Get improvement ideas:
- "Which treatments should I focus on improving?"
- "Where are we losing users in the funnel?"
- "What optimization opportunities do you see?"
7.6 Prompt Templates
Use these templates as starting points:
Performance Review
How did [journey/treatment name] perform in [time period]?
Comparison
Compare [item A] vs [item B] for [time period]
Trend Analysis
Show me the [metric] trend for [time period]
Investigation
Why did [metric] [increase/decrease] on [date/period]?
Top Performers
What were the top [N] [treatments/journeys] by [metric] in [time period]?
Recommendations
What optimization opportunities do you see for [journey/goal]?
7.7 Best Practices
Start Broad, Then Narrow
- Begin with high-level questions
- Identify areas of interest
- Ask follow-up questions for details
Be Clear About Time Periods
Specify time ranges explicitly:
- "Last 7 days"
- "This month"
- "December 1-15"
Reference Specific Entities
Use exact names when possible:
- Treatment names
- Journey names
- Goal names
Save Important Findings
When you get valuable insights:
- Screenshot or copy important findings
- Use for presentations and reports
- Share with team members
Verify Unusual Results
If something seems unexpected:
- Ask clarifying questions
- Cross-check with Analytics dashboards
- Consider data timing/delays
7.8 Limitations
What Agent and AI Insights Can Do
- Answer questions about your data
- Provide summary statistics
- Identify trends and patterns
- Offer optimization suggestions
What Agent and AI Insights Cannot Do
| Limitation | Alternative |
|---|---|
| Make changes to treatments | Use Treatment editing |
| Create new journeys | Use Journey Portfolio |
| Access real-time data | Data has processing delay |
| Predict future outcomes | Use historical analysis |
Data Freshness
Agent and AI Insights uses processed data, which may have a delay:
- Most metrics: 1-4 hours
- Some aggregations: Up to 24 hours
For real-time counts, check the main dashboards.
7.9 Troubleshooting
"I don't have data for that"
Possible Causes:
- Time period is too recent (data delay)
- Journey/treatment doesn't exist
- You may not have access to that data
Try:
- Use a slightly older time period
- Verify the name of the journey/treatment
- Contact admin about data access
Generic or Unhelpful Responses
Tips:
- Be more specific in your question
- Include time period
- Name specific treatments or journeys
- Ask simpler, more focused questions
Slow Responses
Normal Wait:
- Complex queries may take 10-30 seconds
- Large time ranges take longer
If Too Slow:
- Try a shorter time period
- Ask a simpler question first
- Refresh and try again
7.10 Privacy and Data
What Data Can Agent and AI Insights Access?
- Your company's Console data
- Treatment performance metrics
- Aggregated user statistics
What Agent and AI Insights Cannot Access
- Individual user data (PII)
- Other companies' data
- System configuration details
Data Security
- All queries are processed securely
- Results are scoped to your permissions
- Conversation history is private to you
7.11 Advanced Features
Multi-Turn Conversations
Agent and AI Insights remembers context within a conversation:
You: How did Banner A perform last week?
AI: Banner A had 10,000 impressions and 450 clicks (4.5% CTR).
You: And this week?
AI: This week, Banner A had 12,000 impressions and 600 clicks
(5.0% CTR) - a 0.5 percentage point improvement.
Comparative Analysis
Ask for direct comparisons:
You: Compare all my active treatments by CTR for the last 30 days
AI: Here are your active treatments ranked by CTR:
| Treatment | CTR | Impressions |
|-----------|-----|-------------|
| Modal B | 6.2% | 15,000 |
| Banner A | 4.5% | 45,000 |
| Push v2 | 3.8% | 28,000 |
| *(additional rows omitted)* | | |
Anomaly Detection
Ask about unusual patterns:
You: Were there any anomalies in the last week?
AI: I detected the following anomalies:
1. Dec 15: Push notification CTR dropped 40% (7pm-11pm)
2. Dec 17: Conversion rate spiked 85% (likely due to promo)
Next Section
Continue to Section 8: QA & Testing for quality assurance documentation.