[Epic] Redesign azd telemetry: retention, user surfaces, cohorts, and dashboards

[hemarina]

Goal

Improve azd telemetry so we can make design decisions from evidence, and grow adoption.

Our cooked tables and Power BI dashboards were built around templates, provision, and deploy. That no longer matches how azd is used:

• Most traffic now comes from CI/CD and agents, but MAU counts local/desktop only, so our headline number describes a shrinking slice of our users.
• Users increasingly start from a custom template or their own codebase, so the template pivot our dashboards depend on is going stale.
• New features ship with no way to see whether anyone uses them, because reporting is provision/deploy shaped.
• Raw events expire after 90 days, so no long-horizon question can be answered.
• When telemetry breaks we find out late, by noticing a chart looks wrong.

This epic tracks the work to close those gaps.

Workstreams

Compliance — unblocks everything else

• [telemetry] Complete GDPR review and approval of existing azd telemetry #9386 Complete GDPR review and approval of existing azd telemetry
• [telemetry] add GDPRBot to classify events at PR time #9384 Add GDPRBot to classify events at PR time (already filed)

Data foundation

• [telemetry] Retain raw events into cooked tables and share Kusto queries as team notes #9387 Retain raw events into cooked tables and share Kusto queries as team notes
• [telemetry] Split user tables: local (MAU), CI/CD, and agent, plus a combined total #9388 Split user tables: local (MAU), CI/CD, and agent, plus a combined total
• [telemetry] Move to year-over-year metrics (same month across years) #9389 Move to year-over-year metrics
• [telemetry] Cohort analysis by project initialization type #9390 Cohort analysis by project initialization type

Dashboards

• [telemetry] New azd data dashboard, separate from the template dashboard #9392 New azd data dashboard, separate from the template dashboard
• [telemetry] Power BI dashboard for feature usage outside provision and deploy #9393 Power BI dashboard for feature usage outside provision and deploy
• [telemetry] Categorize builder extension metrics #9394 Categorize builder extension metrics
• [telemetry] Agent/MCP telemetry: cooked tables, data quality, and dashboard cleanup #9395 Agent/MCP telemetry: cooked tables, data quality, and dashboard cleanup

Analysis and reliability

• [telemetry] Analyze AI/agent impact on migration to custom templates #9391 Analyze AI/agent impact on migration to custom templates
• [telemetry] Telemetry validation and monitoring #9396 Telemetry validation and monitoring

Related

• [telemetry] Review and redesign azd telemetry end-to-end, and close the gaps #9345 — review and redesign azd telemetry end-to-end, and close the gaps

Kusto functions and Power BI models live in the internal azure-dev-tools repo under product-telemetry/azd/.