Table of Contents
Copilot Usage Reports Weak on Detail
Announced in message center notification MC877369 (29 August 2024, Microsoft 365 roadmap item 396562), the Microsoft Graph beta usage reports API now includes support for Microsoft 365 Copilot tenant usage data. All tenants with Microsoft 365 Copilot licenses should now have access to the usage data.
Microsoft says that the availability of this information will “facilitate the creation of customized reporting and analytics,” but the fact is that the data exposed by the API is bare-bones. On the upside, the data matches what’s available in the report section of the Microsoft 365 admin center (Figure 1).
- Tenant-level summary of Copilot-enabled (licensed) users and active users.
- Adoption trend (tenant summary) over time.
- Last activity date for Copilot interaction in different apps for each user.
Accounts accessing the Graph data must have a Copilot for Microsoft 365 license.
User Count Summary
The user count summary report returns a count of the user accounts licensed for Microsoft 365 Copilot (enabled users) and a count of the users with an active interaction with Copilot in each app during the reporting period (7, 30, 90, or 180 days). Unsurprisingly, when someone is enabled for Copilot in one app, they’re usually enabled for all:
- Teams
- Outlook (classic, new Outlook for Windows, OWA).
- Excel.
- PowerPoint.
- Copilot Graph-grounded chat (aka Copilot Chat).
- OneNote.
- Loop.
$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountSummary(period='D90')" $Data = Invoke-GraphRequest -Uri $Uri -Method Get $Data.value.adoptionByProduct Name Value ---- ----- loopEnabledUsers 100 reportPeriod 90 oneNoteActiveUsers 3 wordEnabledUsers 100 powerPointEnabledUsers 100 microsoftTeamsActiveUsers 97 oneNoteEnabledUsers 100 excelActiveUsers 43 loopActiveUsers 2 copilotChatEnabledUsers 100 outlookEnabledUsers 100 anyAppEnabledUsers 100 anyAppActiveUsers 97 microsoftTeamsEnabledUsers 100 excelEnabledUsers 100 wordActiveUsers 61 powerPointActiveUsers 12 copilotChatActiveUsers 73 outlookActiveUsers 18
User Activity Detail
The user activity detail report is the most interesting because it details the last activity date for Copilot interaction by users with each of the various Copilot-enabled apps. In addition, the last activity date for any Copilot interaction with any of the supported apps is published (lastActivityDate). An array (value) holds a separate usage report for each Copilot-enabled account.
The user principal name and display name is obfuscated if the tenant data privacy control is enabled. In the following extract, we see that the user has never used Copilot for Loop and OneNote and hasn’t used Copilot with PowerPoint since April 11, 2024:
$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period='D90')"
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.value[0] | Format-Table Name, Value -AutoSize
Name Value
---- -----
copilotActivityUserDetailsByPeriod {System.Collections.Hashtable}
reportRefreshDate 2024-11-04
oneNoteCopilotLastActivityDate
loopCopilotLastActivityDate 2024-09-11
microsoftTeamsCopilotLastActivityDate 2024-09-27
powerPointCopilotLastActivityDate 2024-09-22
wordCopilotLastActivityDate 2024-10-29
outlookCopilotLastActivityDate 2024-10-09
excelCopilotLastActivityDate 2024-09-05
lastActivityDate 2024-10-31
copilotChatLastActivityDate 2024-10-31
userPrincipalName Tony.Redmond@office365itpros.com
displayName Tony Redmond
Adoption Trend over Time
This report returns an array called adoptionByDate with entries for each day during the reporting period (7, 30, 90, or 180 days). The purpose of the report is to track progress in Copilot adoption over time and to note if any specific action had an effect. For instance, you might run an education campaign to teach users how to generate effective results using Copilot in Excel. Over the weeks following the campaign, you’d expect to see the number of users who use Copilot in Excel to grow.
$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountTrend(period='D90')" $Data = Invoke-GraphRequest -Uri $Uri -Method Get $Data.Value.copilotActivityUserDetailsByPeriod reportDate 2024-06-17 excelEnabledUsers 100 wordActiveUsers 51 powerPointActiveUsers 11 copilotChatActiveUsers 66 outlookActiveUsers 15 loopEnabledUsers 100 oneNoteActiveUsers 1 wordEnabledUsers 100 powerPointEnabledUsers 100 microsoftTeamsActiveUsers 86 oneNoteEnabledUsers 1 excelActiveUsers 21 loopActiveUsers 1 copilotChatEnabledUsers 100 outlookEnabledUsers 100 anyAppEnabledUsers 100 anyAppActiveUsers 86 microsoftTeamsEnabledUsers 100
Track Copilot Activity Using Audit Records instead of Copilot Usage Reports
Although it’s nice to have Copilot usage reports included in the Graph API, the information exposed isn’t very informative in terms of how people use Copilot. The data tells you that someone used Copilot in an app during a day. At least, they clicked a Copilot button. The information doesn’t reveal any more insight than that. Any enterprise who invests large sums of money in expensive Microsoft 365 Copilot licenses will find a dearth of detail here in terms of understanding whether the investment is justified. In many cases, you will be better off analyzing the audit records captured for Copilot interactions to figure out what’s really going on.
Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.
what are the API Permissions required inorder to use from Service Principal ?
I’m sure that you can search for Graph Usage Reports API and find that information yourself… They’d be the application permissions specified in the documentation.
thanks for the script. very useful. is there a limit of 200entries for the report?
https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period='D90‘)
some users on my report last week arent showing today.
I think i see the issue. Using the syntax in your script it is only sending the first 200 entries. I have to use .’@odata.nextLink’ to get the next set. Is there a better way to do this if my usage keeps growing?
Yep, it’s an example that doesn’t include pagination. Here’s code from https://github.com/12Knocksinna/Office365itpros/blob/master/Find-UnderusedCopilotLicenses.PS1 that might help:
# Fetch usage data for Copilot
Write-Host “Fetching Microsoft 365 Copilot usage data…”
$Uri = “https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period=’D90′)”
[array]$SearchRecords = Invoke-GraphRequest -Uri $Uri -Method Get
If (!($SearchRecords)) {
Write-Host “No usage data found for Microsoft 365 Copilot”
Break
}
# Store the fetched usage data in an array
[array]$UsageData = $SearchRecords.value
# Check do we have more usage data records to fetch and fetch more if a nextlink is available
$NextLink = $SearchRecords.’@Odata.NextLink’
While ($null -ne $NextLink) {
$SearchRecords = $null
[array]$SearchRecords = Invoke-MgGraphRequest -Uri $NextLink -Method GET
$UsageData += $SearchRecords.value
Write-Host (“{0} usage data records fetched so far…” -f $UsageData.count)
$NextLink = $SearchRecords.’@odata.NextLink’
}
I just discovered the same 200 user limitation myself after thinking it was because that was exactly the same number of licences we have!
I don’t understand what the time period is doing. If I select 7, 30, 90 or 180 days in the API, I get the same results in my report. The OOTB report from the admin centre always returns data for the last 180 days regardless of whether the user has had their licence removed or not. The docs refer to data being aggregated but these are last activity dates so I don’t see how that is related. Does anyone understand how this time period is working (or not!)?
As discussed earlier, the limitation is imposed by the need to paginate to fetch all available data. It’s just a Graph thing.
As to the data, because usage data reflects a snapshot of activity over the selected period, you should see differences (like a user either used Copilot or not). Unfortunately, the Copilot usage data basically tells you if someone used Copilot or not over the selected period and what apps they used Copilot in. If you want more data, you need to look at audit records. Here’s an example.
I do see differences in the data for my tenant over the various periods. If you’re positive that data is not changing and that the data is inaccurate, log a support call with Microsoft and ask them to check. I suspect that it’s simply a case that the data hasn’t changed because the same last usage date pertains no matter what period you measure over.
Yes understood on the Graph paging. In my case I did this:
https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period='D7‘)?$top=999
as I never expect to go above 999.
Regarding the data if the time period for the API is truly meant to refer to the last x time period then its not working properly. For my 7 day query, I see a row in the report that looks like this:
reportRefreshDate userPrincipalName displayname lastActivityDate Report Period Last activity date of Copilot chat(UTC) Last activity date of Teams Copilot (UTC) Last activity date of Word Copilot (UTC) Last activity date of Excel Copilot (UTC) Last activity date of Power Point Copilot (UTC) Last activity date of Outlook Copilot (UTC) Last activity date of OneNote Copilot (UTC) Last activity date of Loop Copilot (UTC)
3/24/2025 redacted@redacted.com redacteddisplayname 11/02/2024 8/12/2024 7 8/12/2024 8/4/2024 1/10/2024 4/10/2024
As you can see from the dates this is def. not in the last 7 days. I have an open ticket with MS on this and they tell the report covers 180days and I know that is true for the OOTB reports, but I am still querying the on the API as it doesn’t make sense.
The usage data for Copilot is basic. I don’t use it because the audit data is much more insightful.