A new view into which agents are being used
On August 7, GitHub said its Copilot usage metrics API would report activity from recognized third-party agent apps separately. The new data includes an agent identifier, user-initiated job starts, and session counts in aggregated enterprise and organization reports. Some per-user reports include job starts but omit session counts. The update requires the Copilot usage metrics policy and appropriate metrics permissions. GitHub also cautions that display names can change, so the stable agent identifier is the safer key for comparisons.
For a business owner, this fills an adoption visibility gap: an organization can see whether a newly introduced agent is being used and compare recognized agents over reporting periods. The metric counts starts and sessions. It does not say whether a job completed, whether the code was accepted, whether a reviewer had to repair it, or whether a customer received value. Treat it as evidence of activity, not return on investment.
Separate adoption, completion, and quality
Set up a small reporting table with three distinct layers. Adoption asks who tried the tool and how often. Completion asks whether the intended work reached its defined stopping point. Quality asks whether the result met the team’s standards after review. If these layers collapse into one “productivity” number, a rising usage count can look like progress even when the work is abandoned or generates extra correction.
For a coding workflow, pair the agent counts with a few existing measures the engineering team already trusts: tasks accepted after review, rework associated with those tasks, and time from assignment to merge. Keep a consistent definition for each measure and record changes in team size, project type, or policy. Those comparisons can guide questions, but they do not prove that the agent caused a faster outcome. A short before-and-after period is a signal to investigate, not a controlled experiment.
Read the reporting fields carefully
GitHub notes that third-party agent activity is grouped by recognized agent. Unknown agents may be omitted, while multiple apps from the same agent may appear as one entry. The API’s nested job-start count is different from the top-level count for other supported telemetry; adding them together would double count unlike activities. These details matter when a spreadsheet or dashboard turns fields into a single total.
Before building a dashboard, inspect the raw response for your organization and reporting window. Confirm the optional field is present, identify the reporting grain, and preserve the stable agent ID. If a field is absent, do not assume the value is zero: GitHub says the third-party array may be omitted when there is no recognized activity in the period. Verify the policy and permissions, then label unavailable data separately from a measured zero.
Make the next decision small and reversible
Use the new data to answer one purchasing or rollout question, such as whether a pilot team has adopted a second agent enough to justify a deeper comparison. Ask the team what work it used the agent for, select a small sample of outputs, and review them against the normal quality bar. If usage is low, learn whether the tool was hard to access, poorly matched to the work, or simply unnecessary. Do not push adoption merely to increase a chart.
A useful next step is to export one 28-day report, document what each field measures, and compare it with a short sample of completed work. Keep the activity count, review result, and business outcome in separate columns. That gives an owner a more honest basis for a license or rollout decision while avoiding the false comfort of treating “agent started” as “work done.”