Purpose
Tracking issue for reviewing and redesigning azd telemetry across the whole pipeline — from what the CLI emits, through RawEvents, to the cooked tables and reporting. The goal is to close existing gaps and come out of this with a concrete set of TODOs (follow-up issues) for the improvements. This issue is intentionally exploratory: I'll use it to drive the review and design, then break the outcomes into actionable work items.
Scope
- CLI side (azure-dev): events/fields emitted, coverage per command, consistency.
- Downstream (azure-dev-tools): RawEvents → cooked tables → Power BI/dashboards.
Areas to explore
1. Privacy review process for new telemetry
We expect a privacy review for each new feature's telemetry, but in practice that isn't consistently happening. Rethink how privacy reviews are triggered and enforced so they're a reliable part of adding telemetry rather than an afterthought.
2. A better path to adopt new telemetry data
New telemetry currently just lands in RawEvents with no defined path to becoming usable, reviewed, documented data. Figure out a repeatable way to promote/adopt new data (schema, classification, docs, cooked-table surfacing) instead of it sitting unused in RawEvents.
3. Validation / monitoring to detect telemetry breakage
We don't have a good way to detect when telemetry breaks (missing events, renamed/empty fields, ingestion regressions). Explore a validation or monitoring pipeline so telemetry is reliable and breakage is caught early.
4. Rethink cooked tables around the story we want to tell
Most existing cooked tables are provision/deploy-focused. As I update existing tables and add new ones, step back and design cooked tables from the perspective of what story we want the telemetry to tell — i.e., what data we actually want to collect and surface, not just what's easy to aggregate today.
Tasks
Purpose
Tracking issue for reviewing and redesigning azd telemetry across the whole pipeline — from what the CLI emits, through RawEvents, to the cooked tables and reporting. The goal is to close existing gaps and come out of this with a concrete set of TODOs (follow-up issues) for the improvements. This issue is intentionally exploratory: I'll use it to drive the review and design, then break the outcomes into actionable work items.
Scope
Areas to explore
1. Privacy review process for new telemetry
We expect a privacy review for each new feature's telemetry, but in practice that isn't consistently happening. Rethink how privacy reviews are triggered and enforced so they're a reliable part of adding telemetry rather than an afterthought.
2. A better path to adopt new telemetry data
New telemetry currently just lands in RawEvents with no defined path to becoming usable, reviewed, documented data. Figure out a repeatable way to promote/adopt new data (schema, classification, docs, cooked-table surfacing) instead of it sitting unused in RawEvents.
3. Validation / monitoring to detect telemetry breakage
We don't have a good way to detect when telemetry breaks (missing events, renamed/empty fields, ingestion regressions). Explore a validation or monitoring pipeline so telemetry is reliable and breakage is caught early.
4. Rethink cooked tables around the story we want to tell
Most existing cooked tables are provision/deploy-focused. As I update existing tables and add new ones, step back and design cooked tables from the perspective of what story we want the telemetry to tell — i.e., what data we actually want to collect and surface, not just what's easy to aggregate today.
Tasks