Change Applies on July 1 to Tenants that Integrated SharePoint with Entra ID B2B Collaboration

The announcement in message center notification MC1089315 (6 June 2025) that Microsoft is dumping the old one-time passcode (OTP) authentication mechanism for SharePoint Online and OneDrive for Business sharing is unexpected, but only because it took Microsoft so long to make the change.

After July 1, 2025, external users who have received a sharing link from a user in a tenant that uses OTP authentication will discover that they have lost access to the shared content (files, folders, or sites). Microsoft says that they’re making the change to “enhance security.” I think this is correct, and the change delivers an additional benefit to Microsoft because it gets rid of an old feature.

A History of One-time Passcodes in SharePoint Online

OTP-based sharing links (aka, the “Secure external sharing recipient experience”) predates the support of Entra ID B2B Collaboration (guest accounts) within SharePoint Online. That support arrived as a result of guest access to Office 365 groups (now Microsoft 365 groups) in September 2016. Guest accounts took a while to catch on, and Office 365 groups only became really popular after the advent of Teams in early 2017. Indeed, Teams didn’t surpass 20 million active users in November 2019 before massive growth occurred in Teams usage during the Covid-19 pandemic.

Although Teams growth propelled similar growth in groups and SharePoint usage, there was no great push to move tenants off OTP authentication to SharePoint and OneDrive integration with Azure AD (now Entra ID). External sharing worked, so why bother?

Microsoft began the process to get off OTP by integrating OTP with Entra ID B2B Collaboration in October 2021. Essentially, the change ensured that external users who received OTP sharing links had guest accounts created for them in the tenant directory. The next step made sure that new tenants created after March 31, 2023, could only use B2B collaboration.

The plan now revealed “only impacts organizations that have already enabled or plan to enable SharePoint and OneDrive integration with Microsoft Entra B2B.” In other words, nothing changes for tenants that did not link SharePoint Online and OneDrive for Business to Entra ID B2B Collaboration. I wonder what proportion of the SharePoint community still use one-time passcodes exclusively for sharing.

The Result of the Change

MC1089315 rates this change to be “highly relevant.” In other words, it will affect how users work because:

• After July 1, all new sharing links generated for external people will use Entra ID B2B Collaboration and the sharees will receive email containing the sharing link generated by the Entra ID Invitation Manager service. This shouldn’t cause too much upheaval because the process is reasonably painless. I use it all the time to share documents with several other Microsoft 365 tenants and haven’t had any issues with sharing links that I can remember.
• After July 1, all previously issued sharing links based on one-time passcodes generated by SharePoint Online and OneDrive for Business will stop working. Obviously, this aspect of the change could cause confusion when a link sent to users doesn’t work. July 1 is a Tuesday, and it’s entirely possible that many sharing links with one-time passcodes arrive in user mailboxes on Monday, June 30. If the recipients action the links immediately, they can access the shared content. If they delay, the links stop working. It’s as simple as that.

Microsoft says that users will be told “Sorry, something went wrong. This organization has updated its guest access settings. To access this item, please contact the person who shared it with you and ask them to reshare it with you.” What’s gone wrong is that Microsoft decommissioned one-time passcodes. However, the statement is accurate that the only way to resume access to the shared content is to receive a new sharing link generated based on B2B collaboration. The potential for impact on users and the knock-on effect on help desks is clear.

MC1089315 notes that users will be required to complete multi-factor authentication (MFA) registration as part of the Entra ID B2B onboarding process. That’s strictly only true if the tenant that hosts the content requires MFA, most likely with a conditional access policy to block access unless an MFA challenge is satisfied. Even if your tenant doesn’t use MFA today (which it should), it is the hosting tenant that gets to choose whether MFA is required.

A Good Change

I bet this change will cause confusion and some upheaval in the weeks after July 1. After that, everything should calm down as the old OTP-based sharing links work their way out of the system. It’s good to have consistency and security and having one method to secure sharing links seems like a good change to make. At least, it is in my book.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Can Attackers Use Copilot for Microsoft 365 to Help Find Information?

An article by a UK-based security penetration test company titled “Exploiting Copilot AI for SharePoint” drew my attention to see what weaknesses testing had found. I was disappointed. Although the article makes some good points, it doesn’t tell reveal anything new about the potential issues that can arise due to poor protection of information stored in SharePoint Online sites. Let’s discuss the points raised in the article.

A Compromised Account

Copilot for Microsoft 365 always works as a signed in user. Before an attacker can use Copilot for Microsoft 365, they must be able to sign into a licensed user’s account. In other words, that account is compromised. That’s bad for a tenant because any compromise can lead to data loss or other damage, and it’s probably indicative of other problems that attackers can exploit without going near Copilot.

Organizations should protect themselves with strong multifactor authentication (MFA). That message seems to be slowly getting through, and you’d imagine that any tenant willing to invest in Copilot is also willing to protect themselves by insisting that all accounts are protected by MFA.

Seeking Sensitive Information

The authors make a good point that people often store sensitive information in SharePoint Online. Attackers like to search for information about passwords, private keys, and sensitive documents. Copilot undoubtedly makes it much easier for attackers to search, but I don’t think that the default site agents create any vulnerability because these agents are constrained to searching within the sites they belong to.

Custom agents might be more problematic, but that depends on the information accessed by the agents. It also depends on the penetrated user being able to run the custom agents. The big thing to remember here is that Copilot can only access data available to the account being used. Custom agents in the hands of an attacker can’t automagically get to some hidden data. Anyway, organizations should monitor the creation of agents and have some method to approve the use of those agents.

Accessing Password Data

The penetration team reported that they had found an interesting file (an encrypted spreadsheet) that appeared to contain passwords that SharePoint blocked access to because “all methods of opening the file in the browser had been restricted.” This sounds like SharePoint’s block download policy was in operation for the site. However, Copilot was able to fetch and display the passwords stored in the file.

It’s likely that the spreadsheet was “encrypted” using the default Excel protection applied when a user adds a password to a spreadsheet. However, the encryption is no match for Microsoft Search, which can index the information in the file, and that’s what Copilot for Microsoft 365 Chat was able to display (Figure 1).

Excel’s encryption is very poor protection in the era of AI. Sensitivity labels should be used to secure access to sensitive information, specifically labels that do not allow Copilot to extract and display information from files found by searching against Microsoft Search. Even better, use the DLP policy for Microsoft 365 Copilot to completely hide sensitive files against Copilot so that not even the file metadata is indexed.

Alternatively, use Restricted Content Discovery (RCD) to hide complete sites so that casual browsing by attackers (or anyone else looking for “interesting” information). Apart from RCD, Microsoft makes other SharePoint Advanced Management (SAM) features available to Microsoft 365 Copilot tenants. There’s no excuse for failing to use the access control and reporting features to secure sensitive sites.

Copilot for Microsoft 365 is a Superb Seeker

Copilot for Microsoft 365 is superb at finding information stored in SharePoint Online and OneDrive for Business. With good prompting, an attacker with access to a compromised account can retrieve data faster than ever before, and unlike previous methods of trawling through SharePoint files, Copilot access doesn’t leave breadcrumbs like entries in the last files accessed list.

Copilot access can be constrained by making sure that suitable permissions are in place for documents, deploying the DLP policy for Microsoft 365 Copilot, and limiting access to confidential sites through Restricted Content Discovery. The DLP policy and RCD are recent Copilot control mechanisms that I don’t think the authors of the penetration test report considered (even though they refer to blocking agents with RCD). But available mechanisms are worthless unless implemented, and the real value of reports like this is to prompt administrators to use available tools, including MFA to reduce the likelihood of a compromised account.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Ignite Announcement About SAM for Copilot Customers Misinterpreted by Many

At the Ignite 2024 conference, Microsoft announced that “Microsoft 365 Copilot will now include built-in content governance controls and insights provided by SharePoint Advanced Management.” At the time, and still broadly believed, the assumption was that Microsoft would provide customers with Microsoft 365 Copilot licenses with SharePoint Advanced Management (SAM) licenses. Maybe even a single SAM license would be sufficient to license SAM technology alongside Copilot. That’s not the case.

If you’ve been waiting for a SAM license to appear in your tenant, you’ll be disappointed and won’t see SAM listed in the set of tenant subscriptions. Don’t be swayed by the banner in the SharePoint Online admin center to announce that your SharePoint Advanced Management subscription is enabled (Figure 1). It’s not. Access to SAM features is granted through a check enabled in code for the presence of Copilot. The necessary update is now broadly available to customers.

SAM Features for Microsoft 365 Copilot Customers

The facts are laid out in the SAM documentation. Customers with eligible Copilot licenses can use some, but not all, SAM functionality without a SAM license. Here’s the list:

• Site Lifecycle Policy
  • Inactive SharePoint sites policy
  • Site Ownership Policy
• Data Access Governance (DAG) Insights
  • “Everyone Except External Users” (EEEU) insights
  • Sharing Links and Sensitivity Labels
  • PowerShell: Permission state report for SharePoint and OneDrive Sites, and Files
  • Sharing links report
• Site Access Review
• Restricted Content Discovery (RCD – enabled via PowerShell)
• Restricted Access Control (RAC) for SharePoint and OneDrive for Business.
• Recent Admin Actions and Change History
• Block Download Policy
  • SharePoint and OneDrive sites
  • Teams recordings

There’s some good stuff here, particularly Restricted Content Discovery (RCD), the Site Lifecycle Policy to manage inactive sites, and the Block download policy. Every tenant with Microsoft 365 Copilot should consider enabling RCD to block Copilot access to sites containing sensitive Office and PDF files and sites containing old and obsolete material (the digital rot or debris that clutters up so many tenants).

The problem with Copilot reusing sensitive material in its responses is obvious. The issue with Copilot reusing old, obsolete, and potentially misleading content in its responses is equally problematic, especially if human checks don’t catch errors in responses. Copilot doesn’t know when a Word document written ten years ago is outdated and inaccurate. All Copilot sees is words that can be processed and reused.

When SAM is Needed

All of which brings me to a point where a SAM license is required. In my case, I wanted to test the extend SharePoint protections with a default sensitivity label feature. The idea here is to make sure that unlabeled files receive protection when downloaded by applying a sensitivity label with equivalent rights to those enjoyed by site users. Defining a default sensitivity label for a document library already requires an Office 365 E5 license or equivalent. Why this slight extension wanders into the need to have SAM is another example of bizarre Microsoft licensing.

The documentation notes that Copilot can’t currently open files with sensitivity labels applied in this manner. This means that Copilot cannot extract the protected content to use in its responses because it doesn’t have the right to do so. However, Copilot can search the metadata of labeled files and show that metadata to those who perform searches. Restricted Content Discovery is the right way to block Copilot access to files.

Anyway, without a SAM license, I can’t test. Do I want to pay Microsoft for a license for the privilege of testing their software? I don’t think so.

Copilot in Word for iOS

In closing, I attempted to use a new feature in Word for iOS (and Android) to dictate some notes for this article for Copilot to reason over and produce a draft. The feature is covered in MC1060866 (23 April 2025) and deployment has begun, which is why I guess I could use it. The dictation part worked, even if some of my words were misunderstood (Figure 2). But any attempt to have Copilot do some magic failed utterly. I guess that AI can’t help me…

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Use Audit Records to Find Who Creates SharePoint Agents

Another day, another event on the TEC 2025 European Roadshow. This time we were in Paris and just like in London’s discussion about how to protect old confidential files from Copilot access, attendees posed many good questions. Following the discussion about how to manage agents, I was asked how an organization could discover if SharePoint agents are in use. It’s a good question that isn’t answered in Microsoft’s documentation about how to manage agents in SharePoint. Microsoft gives some details about planned administrative features for agents are in an April 8, 2025 blog post, but there’s nothing available today.

I hadn’t thought about the problem up to now. SharePoint agents are limited in scope and don’t seem to pose too many administrative challenges. Each agent exists as a file in a document library with an .agent extension (originally, agents had a .copilot extension). Except for sites marked for Restricted Content Discovery, SharePoint Online sites have a default agent that reasons over the entire site. Site members can create other agents that focus on specific parts of the site. Agents created by site members are available to all site members and can be amended by them.

Approved Agents

To mark agents as being particularly useful, site owners can approve agents to highlight the agents in the agent picker. The files for approved agents are moved to the Site Assets library where they’re stored in the Approved sub-folder of the Copilots folder. Only site owners can edit approved agents. Figure 1 shows the details of an approved agent.

Auditing Agent Creation

The creation and updating of custom SharePoint agents is evidence that people are using agents. Because the agent files are treated like other SharePoint files, audit records are captured in the Microsoft 365 audit log when these actions occur. By interrogating the audit log, we can discover who is creating agents and the sites where agents are used.

Here’s some PowerShell to use the Search-UnifiedAuditLog cmdlet to find SharePoint FileUploaded audit records for files with an .agent extension:

[array]$Records = Search-UnifiedAuditLog -StartDate (Get-Date).AddDays(-30) -EndDate (Get-Date) -Formatted -ObjectIds "*.agent" -Operations FileUploaded -ResultSize 5000 -SessionCommand ReturnLargeset
If ($Records) {
    $Records = $records | Sort-Object Identity -Unique
    Write-Host ("{0} audit records found" -f $Records.Count)
} Else {
    Write-Host "No audit records found"
    Break
}

$AgentReport = [System.Collections.Generic.List[Object]]::new()
ForEach ($Rec in $Records) {
    $AuditData = $Rec.AuditData | ConvertFrom-Json
 
    $ReportLine = [PSCustomObject][Ordered]@{
        TimeStamp       = Get-Date ($AuditData.CreationTime) -format 'dd-MMM-yyyy HH:mm'
        User            = $AuditData.UserId
        Action          = $AuditData.Operation
        SiteURL         = $AuditData.SiteURL
        Agent           = $AuditData.SourceFileName

    }
    $AgentReport.Add($ReportLine)
}
$AgentReport = $AgentReport | Sort-Object {$_.TimeStamp -as [datetime]} -Descending
$AgentReport | Out-GridView -Title "Custom SharePoint Agent Creation"

Write-Host ""
Write-Host "Custom agents created in these SharePoint Online sites"
$AgentReport | Group-Object SiteURL -NoElement | Sort-Object Count -Descending | Format-Table Name, Count
Write-Host ""
Write-Host "Custom agents created by these users"
$AgentReport | Group-Object User -NoElement | Sort-Object Count -Descending | Format-Table Name, Count

Figure 2 shows some sample output seen through the Out-GridView cmdlet.

Some basic statistics are also produced about the sites where custom agents were created and the user accounts which create agents. To track agent usage, you can use the same technique to fetch and analyze FileAccessed audit events.

Microsoft Reports to Come?

Once again, the Microsoft 365 audit log is the source to answer questions. I’m sure that Microsoft will eventually get around to generating better-looking reports about agent creation and activity in the future. The usual course of events is that these kinds of gaps are filled sometime after functionality becomes available. Given that SharePoint agents reached general availability in November 2024, we’re still only finding out what reporting is needed for operational purposes, so it might take a while yet before we see any Microsoft reports.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Filter, Refine, and Report File Sharing Events from the Audit Log

A recent article about auditing file sharing activities in Teams generated some questions. The accompanying script searches for FileUploaded events, which have nothing to do with sharing. SharePoint Online captures FileUploaded events when users create new files in SharePoint sites.

In any case, after reading the article, it makes a case to keep an eye on files uploaded to Teams channels because it’s possible that someone might share information that results in a data leak. It’s a tenuous proposal that only makes sense in a weird sort of way. I am not saying that no one has never uploaded a file to a Teams channel that they shouldn’t have. Some mistakes will happen given that people create billions of files in SharePoint Online daily. But the sheer volume of FileUploaded events created in the unified audit log means that a simple report detailing these events is never going to be valuable. Filtering and analysis are required to extract value.

Most file activity logged by SharePoint Online is innocuous. To find value in the audit log, administrators need to know the data they want to find. As an example, it seems like it would be good to know who shares files from SharePoint Online, both through Teams and the SharePoint browser interface, and who they share the files with (internal and external).

Microsoft documents how to use audit data to track sharing activities. There’s lots of good information in that article to help you understand how SharePoint Online generates the content of the audit events generated to track sharing activities.

Finding File Sharing Events in the Audit Log

When I begin to figure out what audit data might be valuable for investigative purposes, I usually use several accounts to perform the activities I’m interested in (in this case, sharing documents), wait about 30 minutes, and then go through the events that turn up in the audit log. Searching the audit log with a command like this returns SharePoint sharing events. Make sure that the start and end dates are limited to the period when the actions of interest occur:

[array]$Records = Search-UnifiedAuditLog -StartDate '2-Apr-2025 19:00' -EndDate (Get-Date) -Formatted -SessionCommand ReturnLargeSet -ResultSize 5000 -RecordType SharepointSharingOperation

Analyzing the audit data revealed that SharingSet events happen to set up a sharing link. UserExpirationChanged events are also found if the sharing link policy sets expiration dates for sharing links. If you cast the audit net wider and look for other events, you’ll also find Send events logged when SharePoint Online sends notification messages to inform people that someone has shared a file with them.

Filtering File Sharing Events

The audit log is a rich source of information that can be overwhelming because of the amount of logged data. When searching for answers, it’s important to focus. In this instance, I extracted only the SharingSet events and then filtered the returned set to remove sharing events that I wasn’t interested in. These events included:

• Sharing for SharePoint embedded applications such as Loop and Outlook Newsletters.
• Sharing performed by the background app@sharepoint app. For instance, when SharePoint Online shares the recording of a Teams meeting (stored in the OneDrive of the meeting organizer) with meeting participants.
• Sharing set operations to adjust SharePoint lists. When a user shares a document, SharePoint Online adjusts the group that controls access to that item within the site, which results in audit events being logged for groups like “Limited access system group for list.” A Microsoft article covers permission levels and explains what these groups mean.

Essentially, the only sharing events I am interested in are those involving member and guest Entra ID accounts (i.e., humans).

The lesson here is that retrieving a set of events from the audit log seldom delivers useful results. It’s usually the first stage in a process to remove unwanted events to focus on the valuable information.

Parsing and Reporting Sharing Audit Events

The next step is to parse the information contained in the remaining audit events to answer the questions who shared what with whom and what level of access did they grant? Most of this information is hidden in plain sight in the AuditData property of audit events. The data must be extracted, cleaned up, and enhanced.

For example, if your organization uses sensitivity labels to protect files (and you should), the audit events note the GUID of the label applied to the shared file and the GUID of the label applied to the host site (container management label). Resolving the GUIDs to label names makes this information more accessible. Knowing that a shared file has a sensitivity that will block unauthorized access is always a nice feeling.

The result is a report of file sharing events (Figure 1) that answers the question of who shared files from SharePoint Online with whom and what access was granted.

In addition, because the script extracts the email addresses of sharees, you can analyze the volume of sharing to external domains:

$AuditReport | Group-Object TargetDomain -NoElement | Sort-Object Count -Descending | Format-Table Name, Count

Name                        Count
----                        -----
microsoft.com                  11
o365maestro.onmicrosoft.com     4
contoso.com                     2
proton.me                       1

Report File Sharing Events to Meet Your Requirements

Like anything published on the internet, the script (available from GitHub) might or might not satisfy your requirements. But it’s PowerShell, so you can change the code to meet your needs. I used the Graph AuditLog Query API to retrieve audit data. The same data is available by running the Search-UnifiedAuditLog cmdlet.

The takeaway is that real value is seldom extracted from audit logs without some additional processing to refine, filter, and interpret the information. Articles that merely extract and report audit data don’t add much value because they don’t tell the full story and reveal the actionable data that administrators need.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Restricted Content Discovery Hides SharePoint Content from Copilot and Agents

The problem of poor permission management has surfaced from time to time in the history of SharePoint. The Office Delve app caused the last big upheaval within Microsoft 365 when it demonstrated an uncanny ability to surface sensitive documents to user view. Of course, Delve was never the problem. The issue is due to careless permission assignment, usually at site level.

When Microsoft launched Copilot in March 2023, it soon became apparent that Copilot is even better than Delve at finding and reusing documents, including files that an organization would prefer to remain restricted. Microsoft’s short-term answer was Restricted SharePoint Search, a horrible but expedient solution that works on the basis of an allow list for enterprise search which restricts users to only being able to search approved sites. Copilot always works as the signed in user, so the limits applied to users apply to Copilot to stop the AI using material stored in unapproved sites in its responses.

Restricted Content Discovery (RCD) is the latest solution to control unfettered access to confidential information stored in SharePoint Online sites. RCD is part of the SharePoint Advanced Management (SAM) suite. Microsoft is making SAM available to tenants with Microsoft 365 Copilot licenses via a code update that’s slowly deploying.

How Restricted Content Discovery Works

Restricted Content Delivery works by adding a flag to files stored in designated SharePoint Online sites. When an administrator marks a site for RCD through the SharePoint admin center or PowerShell. Figure 1 shows the “restrict content from Microsoft 365 Copilot” option in the admin center. When a site is selected for RCD, SharePoint sets a site-level property that causes index updates for every file in the site. Although RCD is applied at a site basis, SharePoint indexing happens at the file level, so a fan-out process must find and reindex every file in a site before RCD becomes effective for that site.

The time required to update the index for a site is highly dependent on the number of items in the site. Microsoft says that “for sites with more than 500,000 items, the Restricted Content Discovery update could take more than a week to fully process and reflect in search and Copilot.”

The indexing update does not remove items from the tenant index. If it did, items would be unavailable for eDiscovery searches, auto-label policies for retention and sensitivity labels, and other solutions. Instead, the flag set on files instructs Copilot to ignore those files when it consults the Graph to find matching content to help ground user prompts. The same approach is used by the Data Loss Prevention (DLP) policy to block Copilot access to files assigned specific sensitivity labels.

The block applies to anywhere Copilot for Microsoft 365 can use SharePoint Online files, including Copilot agents. It doesn’t affect how site-level search works, nor does it interfere with other Purview solutions like eDiscovery, content searches, or DLP. However, content from sites enabled for RCD don’t appear in enterprise level searches.

RCD Management with PowerShell

PowerShell can be used to manage RCD for sites. Make sure that you use a recent version of the SharePoint Online management module (I used Microsoft.Online.SharePoint.PowerShell version 16.0.25715.12000). For example, to enable RCD for a site, run the Set-SPOSite cmdlet to set the RestrictContentOrgWideSearch property to $true.

Set-SPOSite -Identity https://office365itpros.sharepoint.com/sites/rabilling -RestrictContentOrgWideSearch $true

To remove RCD from a site, set the value for RestrictContentOrgWideSearch to $false:

Set-SPOSite -Identity https://office365itpros.sharepoint.com/sites/rabilling -RestrictContentOrgWideSearch $false

Much the same reindexing process must occur before files in sites where RCD is disabled after being enabled before files become available to Copilot.

To generate a list of sites with RCD enabled, run the Start-SPORestrictedContentDiscoverabilityReport command to create a job on a queue for processing. The Get-SPORestrictedContentDiscoverabilityReport cmdlet reports the status for the job, which eventually reports “Completed.”

Start-SPORestrictedContentDiscoverabilityReport

Generating the report will take some time. Are you sure you want to proceed?
Continue with this operation?
[Y] Yes  [N] No  [?] Help (default is "Y"): y

RunspaceId           : 1d839c7e-c0bf-4c11-be94-20179f2335e2
Id                   : 02aa91ea-5e12-43de-91a1-a58275d3b201
CreatedDateTimeInUtc : 03/31/2025 16:09:52
Status               : NotStarted

Get-SPORestrictedContentDiscoverabilityReport

RunspaceId           : 1d839c7e-c0bf-4c11-be94-20179f2335e2
Id                   : 02aa91ea-5e12-43de-91a1-a58275d3b201
CreatedDateTimeInUtc : 03/31/2025 17:03:52
Status               : Completed

To download the RCD insights report, run the Get-SPORestrictedContentDiscoverabilityReport cmdlet and pass the GUID (id) for the report. This value is shown in the Get-SPORestrictedContentDiscoverabilityReport output:

Get-SPORestrictedContentDiscoverabilityReport –Action Download –ReportId 02aa91ea-5e12-43de-91a1-a58275d3b201
Report RestrictedContentDiscoverabilityReport_1743437651407.csv downloaded successfully

Microsoft documentation says that “the downloaded report is located on the path where the command was run.” This is incorrect. The file ends up in whatever folder the PowerShell session starts up in. In my case, I ran the job when positioned in c:\temp and the file ended up in c:\windows\system32. The easy fix here is to use a PowerShell profile to define the folder where PowerShell starts up.

The contents of the “insights” report aren’t too exciting (Figure 2) and could be easily generated by looping through sites with PowerShell to find those with the flag set.

Restricted Content Discovery for All

It’s a reasonable guess that any Microsoft 365 tenant that’s interested in Copilot has some sensitive information stored in SharePoint Online sites. If you’re in this situation, you should consider RCD as the front-line method to prevent that information leaking out through Copilot. I’d also deploy the DLP policy to restrict Copilot access as a backup. Between the two lines of defence, it’s unlikely that inadvertent disclosure of confidential data will happen, and that’s a good thing.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Old-Fashioned Identity Client Jettisoned for OAuth

Message center notification MC1028318 (March 11, 2025) says that the SharePoint Online PowerShell module will replace the IDCRL authentication protocol with OAuth (modern authentication). Microsoft says that the replacement is “part of our ongoing efforts to enhance security and adopt modern authentication practices.”

Some might ask why it’s taken so long for Microsoft to make the decision to switch the module to OAuth. Microsoft has not given the SharePoint Online PowerShell module much tender loving care over the last few years. For instance, the module hasn’t been upgraded to PowerShell 7 and remains an outlier in this respect within the set of PowerShell modules used within Microsoft 365.

It’s not as if an adequate Graph-based replacement exists. The SharePoint Settings Graph API appeared in mid-2022 and hasn’t made much progress since. It’s just one of the reasons why the SharePoint PnP module is so popular.

The Identity Client Run Time Library

IDCRL is the Identity Client Run Time Library. It’s a very old authentication protocol that was used by products like Lync 2010 Server to authenticate with Exchange Online and Lync Online. IDCRL was also used by the Office desktop apps. Microsoft replaced IDCRL in the Microsoft 365 Apps for enterprise in September 2020 (MC222132).

More pertinently, SharePoint Online used IDCRL for authentication until recently, including with CSOM-based applications.

Upgrade in Modules Released after March 28, 2025

Microsoft issues new versions of the Microsoft.Online.SharePoint.PowerShell module regularly, mostly to add cmdlets or parameters needed to manage features like intelligent versioning. In this case, the change to OAuth is effective for modules released after March 28, 2025 (versions higher than 16.0.25814.12000).

You can download the latest version of the SharePoint Online management module from the PowerShell gallery (Figure 1). Once installed, the Connect-SPOService cmdlet automatically uses modern authentication (also called “modern TLS protocols”) instead of IDCRL. Although the implementation is designed not to affect how scripts work, you might see warning messages because Microsoft will deprecate the ModernAuth parameter in the future (the parameter is now obsolete).

Although I accept Microsoft’s statement that the upgrade to OAuth-based authentication should not affect scripts, it’s always wise to test and verify in case the specific use of the module in a tenant is an edge case that Microsoft doesn’t test. Given some of the recent problems with other PowerShell modules, testing an updated module before putting it into production is always wise.

One Small Step Forward

Given Microsoft’s focus on removing outdated authentication protocols from across Microsoft 365 workloads, it’s surprising that the SharePoint Online management PowerShell module is only now being updated. It’s well behind the modules to manage other major workloads like Exchange and Teams. But then again, as I keep on saying, the signs over the last few years is that Microsoft really doesn’t devote too much attention to the SharePoint Online management module, and that’s a real pity.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

On Demand Classification Processes Cold Files to Find Sensitive or Confidential Information

Message center notification MC1013459 (21 February 2025) announces that Microsoft will introduce a new Information Protection service to find and classify “historical data in SharePoint Online and OneDrive for Business,” or as Microsoft 365 roadmap item 475062 says “scan cold files.”

The idea seems to be that tenants might have a bunch of files in SharePoint and OneDrive that have never been scanned or not been scanned in some time. Purview solutions like Data Loss Prevention (DLP) and Information Protection (sensitivity labels) tend to operate against active files created and edited by users. Cold files gather the digital equivalent of dust and might never be accessed, and it’s possible that those files contain many types of sensitive data.

Closing the gap by finding and classifying the information discovered in “cold files” is what the new On Demand Classification service is all about. By processing historical data, an organization can make sure that the files present in the tenant are classified appropriately. Once classified, policies can be automatically invoked to apply actions like assigning sensitivity or retention labels to files.

Preview of On Demand Classification in March 2025

Perhaps because trainable classifiers are an important method for tenants to find information that’s specific to their business, On Demand Classification is available through the classifiers section of the Purview DLP solution.

Microsoft expects to begin the deployment of a preview version of the code to targeted release tenants in late March 2025 with the goal of attaining general availability in mid-June 2025. Dates have been known to slip in the past, especially with new services, so treat these dates with caution.

Pay As You Go On Demand Classification

The On Demand Classification service is an example of the kind of new functionality that Microsoft bundles into its Office 365 E5 and Microsoft 365 E5 products to tempt customers to upgrade. That’s not the situation in this case. On Demand Classification is the first Purview Pay-as-you-go (PAYG) service.

PAYG is not unknown in Microsoft 365. SharePoint Online has PAYG services like Microsoft 365 Backup, Microsoft 365 Archive, and Document Translation, all of which require customers to have an Azure subscription attached to a valid credit card to pay for metered resource consumption. Other examples include the Graph API to apply sensitivity labels programmatically to Office documents and PDF files in SharePoint Online and OneDrive for Business and the export APIs for Teams and Exchange Online.

The case can be made that all the PAYG offerings are optional and not mainline services. They fill niches that not every Microsoft 365 customer wants or needs. It would be possible to include these services in the Microsoft 365 E5 product, but likely at an increased cost for everyone. Overall, it seems fairer for Microsoft to do the software engineering to create the solutions and realize a return by selling metered services to customers that need these capabilities.

No Details about Service Limitations

Microsoft hasn’t released any details about potential service limitations like the 100,000 items per day maximum an auto-label policy (sensitivity labels) can process within a tenant. We don’t know what kind of cold file types can be processed or if specific policy-invoked actions are restricted to certain types. The screenshot released by Microsoft (Figure 1) doesn’t give any insight into how long a scan might take or how long the subsequent processing might require.

All will be revealed when we get the software to test sometime in March 2025. Of course, some historic data must be found for On Demand Classification to process, but I’m sure that I have some old articles or book files hanging around that deserve to be heated up.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Opens Access to UDP-Protected Files to Search, eDiscovery, and DLP – but not Copilot

Originally announced in preview in an August 1, 2023 technical community post, message center notification MC1013467 (21 February 2025) contains the good news that SharePoint Online will deploy support for sensitivity labels with user-defined permissions (UDP) in mid-March 2025. The reason why this development is important is that SharePoint Online support for UDP enables support for these files in content searches, Purview eDiscovery, and Purview Data Loss Prevention (DLP).

Configuring Permissions for Sensitivity Labels

Most sensitivity labels that protect files with rights-management based encryption use permissions configured by administrators. Permissions are formed by a set of usage rights that dictate what level of access an authenticated user has to a file. The same permissions apply to all files that receive a label with preconfigured access.

User-defined permissions allow file owners to assign different permissions for different files. To allow this to happen, administrators must configure a sensitivity label to support UDP (Figure 1).

After the label is published to make it available to users, they can assign the label and configure permissions for files (Figure 2). UDP labels are visible in Office web applications but can only be set by Office desktop applications.

Clicking more options reveals additional controls for a user to assign to protect a file, including an expiration date (which doesn’t pick up the date format configured for the workstation) for the permissions, a contact email address to request additional permissions, and whether a user must be online to validate their permission before they can open a file. The last option, to access content programmatically, allows Word and Excel to run code within a protected document.

Support for Microsoft Search

The initial SharePoint support for UDP-protected files previewed in August 2023 was limited. The big issue remained that files with UDP labels stored in SharePoint Online or OneDrive for Business couldn’t be indexed by Microsoft Search because Search had no way to gain access to file content (metadata for UDP-protected files is always indexed). This is important because Microsoft Search is an essential component for other services such as eDiscovery. In a nutshell, no indexing meant that UDP-protected files were invisible outside SharePoint Online.

The news announced in MC1013467 addresses the problem, but in a very focused manner. Although the number of UDP-protected files stored in SharePoint Online is likely a very small percentage of the billions of new files created daily, there’s no way that a trawl across all sites to find and process UDP-protected files could work in a practical sense.

To solve the problem, SharePoint Online processes newly-created UDP-protected files from mid-March 2025 to make their content accessible to Microsoft Search. Once indexed by Search, the file content is available to other Microsoft 365 workloads like eDiscovery. During the indexing process, SharePoint interprets the permissions assigned to a file by the author to ensure that those with relevant permissions can engage in co-authoring. In addition, SharePoint Online and the Office apps need permission to access the file before the autosave feature can work. It takes a little time to process a new file after it is uploaded to SharePoint Online. Microsoft reckons on ten minutes, but I have experienced longer delays before features like autosave work.

Older files stored in sites remain inaccessible to SharePoint Online until the next time they are edited. At this point, SharePoint processes the file content to make it searchable. Over time, the idea is that the number of inaccessible UDP-protected files will gradually decrease, and the problem will go away. Once a file is processed by Search, it becomes available to content searches, eDiscovery, and DLP.

Even when UDP-protected files are processed by Microsoft Search, MC1013467 says that “files with labels configured for user-defined permissions will continue to not be available for Microsoft 365 Copilot processing.” In other words, although Search can find UDP-protected files, Copilot still does not have the necessary permissions to load content from those files to use when generating responses to user prompts.

No Big Change for Users in the Immediate Future

From a user perspective, the update for how SharePoint Online processes UDP-protected files won’t mean dramatic change in the immediate future. UDP sensitivity labels might become more popular and widespread, but that’s a process that needs time because it must be factored in the organization’s information protection policy, which is probably currently based on preconfigured permissions. Administrators will need time to absorb the news and figure out how and if UDP-protected files bring value to the business before they create and publish UPD labels.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

No Microsoft 365 Archive Fee to Reactivate Sites After March 31, 2025

In a February 20, 2025 announcement, Microsoft said that they will remove reactivation fees for archived SharePoint Online sites. Some tenants will see the reduction in fees in early March and the change will roll out gradually worldwide for completion by the end of March 2025.

When Microsoft launched Microsoft 365 Archive, they charged $0.60 per GB to reactivate a site by moving its content from “cold” (long-term, archived) storage to “hot” (online, immediately-accessible) storage. Reactivation is immediate for sites archived within the last week, while sites archived for longer take approximately 24 hours to come back online.

Following the removal of the site reactivation fee, Microsoft will only charge the ongoing monthly storage cost of $0.05/GB. Storage fees don’t apply when they can be offset against the tenant’s unused SharePoint Online storage quota, so depending on how many sites they archive and how much content exists in those sites, some organizations might be able to use Microsoft 365 Archive for free.

A restriction does apply in that reactivated sites cannot be moved back into the archive for four months after reactivation. Microsoft says that the restriction is there to stop constant movement in and out of archive storage.

Keep Material Online but Prevent Copilot Access

One of the nice things about archiving sites is that it makes site content inaccessible for Microsoft 365 Copilot. There’s nothing worse than having AI-generated results being polluted by old, obsolete, and probably misleading information, and even if steps are taken to stop Copilot using content in its responses, Copilot can still find and use document library metadata because it exists in Microsoft Search.

I can’t think of a downside to moving old sites into the archive if you want to keep the material stored in the sites. Archived sites are still accessible for eDiscovery, the storage costs are a lot lower than hot online SharePoint storage, and now you can reactivate archived sites free when necessary.

Archived OneDrive Accounts

But you won’t be able to reactivate archived OneDrive for Business accounts free of charge because Microsoft excludes these objects from the removal of reactivation fees. The big idea behind automatically archiving unlicensed OneDrive for Business accounts is to force organizations to do something with accounts that might have been around for ten years or more. The unlicensed OneDrive accounts occupy valuable online storage and because Microsoft encourages the use of OneDrive for Business to hold all manner of files from Teams meeting recordings to PowerShell modules, a significant amount of storage can be occupied.

Microsoft released a report to help tenant administrators decide how to deal with unlicensed OneDrive for Business accounts in August 2024. By now, administrators should have a good handle on the unlicensed accounts within the tenant and know whether they will let automatic archiving happen (and be willing to pay the ongoing storage fees) or take action to remove the unlicensed accounts.

According to recently-revised Microsoft documentation, unlicensed accounts fall into two categories: those unlicensed before February 17, 2025, and those unlicensed afterward. The first batch includes all the historically unlicensed accounts. By April 25, 2025, these accounts will be in read-only mode to prepare them to move into the archive. This process will happen in the background and the unlicensed accounts will be archived by May 16, 2025, including the set shown in Figure 1. Once archived, tenants must pay to reactivate unlicensed OneDrive for Business accounts.

OneDrive for Business accounts that become unlicensed now are placed into read-only mode sixty days after they become unlicensed (for instance, the owning user account is deleted but the OneDrive data is kept by a retention policy, or the user account loses its license to allow them to use OneDrive for Business).

Thirty-three days afterward (93 days after the removal of the license), SharePoint Online will either move to the OneDrive account into the archive or into the recycle bin. Movement into the archive happens when a retention policy applies to the owner’s account.

Lower Fees are Always Appreciated

It’s good that Microsoft has removed the site reactivation fee. While not a lot in the overall scheme of things, getting rid of fees always encourage more use of facilities, and using Microsoft 365 Archive to store old material that the organization cannot remove is a good tactic. Some might question why the same logic doesn’t apply to archived OneDrive for Business accounts. That’s for Microsoft to answer, but I bet that they just want to get people used to the idea of paying to keep old OneDrive content online before they move on charges.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Custom Columns are One Part of the Mystery of SharePoint Search

Understanding how SharePoint Online search works is one of the learning curves faced by many Microsoft 365 tenant administrators. Because search is so important to SharePoint, this topic is well-covered ground for people who worked with SharePoint Server (on-premises). The magic that melds managed properties, crawled properties, mapping, custom cpolumns, indexes, and so on into search doesn’t hold any mysteries.

Things are different for those who come to SharePoint because of its core role within Microsoft 365. SharePoint Online is not the center of an ecosystem like SharePoint Server is. Like Exchange Online delivers email services, the cloud version of SharePoint takes on a completely different role as the provider of document management services to other workloads, like Teams. The different role doesn’t make search any less important. People still want to find files quickly and easily but competing demands from across Microsoft 365 mean that administrators sometimes pay less attention to the finer details of search. After all, search just works in the cloud…

Generally, SharePoint search does just work. Sometimes complexities do exist, like finding out how to find files with a specific sensitivity label. Although users mightn’t want to look for labelled files, administrators might need this knowledge to find labelled files in eDiscovery searches, and that’s why some knowledge about how search works is a good skill to acquire, even for non-SharePoint people. In my case, it helped me to maximize the advantage of creating a custom column for a site.

Creating a Custom Column

SharePoint Online is basically a big Azure SQL application based on a database holding many lists (tables). The lists hold data in items with the information for each item stored in columns (fields). SharePoint allows site administrators to define new custom columns at a site or library level. I use this feature to track the topic areas for articles I publish on different sites.

In general, defining custom columns at site level is recommended because the custom columns can then be used in any library. Figure 1 shows the properties of a custom site column called RAInfo.

After creating the site custom column, use the Add from existing site columns option in Library settings to add the column to a document library. Once the column is added to a library, it can be added to the view that exposes file metadata and edited there (or updated programmatically using a Graph API or SDK cmdlet).

Remaining in library settings, go to Indexed columns and add the new custom column as an index. SharePoint libraries support up to 20 indices.

Using Custom Columns

One of the nice things about custom columns is that SharePoint supports their use in filters. In Figure 2 we see that the filter picker shows the values entered for files in the RAInfo custom column. Selecting the desired values for the RAInfo column helps SharePoint to find and display files that meet the filter criteria. Of course, filtering only works if users remember to enter the necessary information for files.

Users can also input a custom column value (for instance, “RA001”) into the search box to search the library for matching files. Because the custom column is part of the document metadata, SharePoint search can use it to find files.

Making Custom Columns More Searchable

The search against document metadata finds matches against any property containing the value. A further step is required to allow searches against the custom column be more specific.

When you create a custom site column, SharePoint Online creates a managed property named using the form CustomColumnName + “OWSText.” In this case, the managed property is called “RAInfoOWSText.” SharePoint also creates a crawled property with the name OWS_Q_TEXT_CustomColumnName, or OWS_Q_TEXT_RAInfo in our case. The crawled property is what SharePoint search extracts from a site.

To allow search to use the managed property, it’s critical to map the managed property to the crawled property and wait for indexing to complete. Once indexing is complete, you can input RAInfoOWSText:RA001 into the search box. This command instructs SharePoint to search against the RAInfoOWSText managed property.

Not everyone will appreciate working with what seems to be odd column names. To solve the problem, go to Site Information, then View all site settings, and select the Search schema. You can now create an alias for the custom column, hopefully giving the column a name that makes more sense to regular users. In Figure 3, I’ve assigned RAInfo as the alias for the RAInforOWSText managed property.

Adding an alias to disguise some of the complexities of SharePoint search is a small but useful step to take. Now users can input search terms like RAInfo:RA003 (Figure 4) into the search box instead of RAInfoOWSText:RA003 and find the same information.

Take Your Time

One thing that you’ll discover when tweaking SharePoint search is that it takes hours and sometimes days for changes to become active in a site. Users need to populate values in the custom column and search needs to index those values, including respecting the changes that you might have made like adding an alias for a managed property. You can force the issue somewhat by requesting a reindex of the document library (in library advanced settings), but SharePoint Online can’t be rushed too much.

Take your time and everything will work in the end. At least, it did for me.

Create and Publish SharePoint Pages API with the Microsoft Graph PowerShell SDK

In April 2024, Microsoft announced the General availability for the Graph API for SharePoint Pages (also in message center notification MC789609 and Microsoft 365 roadmap item 101166). Despite Microsoft proclaiming that they were thrilled with the new API, I never got around to looking at it, largely because other work got in the way.

Given the period since general availability, it is no surprise that cmdlets for the SharePoint Pages API are available in the Microsoft Graph PowerShell SDK. However, some functionality is missing, and the Get- cmdlet to fetch pages for a site doesn’t work very well and some cmdlets are missing.

Get SharePoint Pages for a Site

Using the API requires the Sites.ReadWrite.All permission to read details of site pages and to create new pages, so the first step is to run the Connect-MgGraph cmdlet with Sites.ReadWrite.All as the selected scope.

The Get-MgSitePageAsSitePage cmdlet retrieves details of the pages for a site. You’ll need to fetch the site identifier for the target site first. The site identifier is not the site URL. A full site identifier looks something like this:

office365itpros.sharepoint.com,8e0a5589-b91d-496e-a5be-3473a75f2fe2,22d7a59d-d93c-498e-a806-6c9475717c88

If you know the URL for a site, you can compute a form of the site identifier that SharePoint will accept to lookup a site like this:

$Uri = "https://office365itpros.sharepoint.com/sites/BlogsAndProjects"
$SiteId = $Uri.Split('//')[1].split("/")[0] + ":/sites/" + $Uri.Split('//')[1].split("/")[2]

$Site = Get-MgSite -SiteId $SiteId

With the site identifier, you can run Get-MgSitePageAsSitePage. Here’s how to return the set of site pages sorted in date created order:

[array]$Posts = Get-MgSitePageAsSitePage -SiteId $Site.Id -All | Sort-Object {$_.CreatedDateTime -as [datetime]} -Descending

Unfortunately, the cmdlet doesn’t return values for many interesting properties, such as createdByUser. Better results are obtained by using the Graph API request:

$Uri = ("https://graph.microsoft.com/V1.0/sites/{0}/pages/microsoft.graph.sitepage" -f $Site.Id)
$Data = Invoke-MgGraphRequest -Uri $Uri -Method Get
$Pages = $Data.Value

Create a Page (a News Post) with the SharePoint Pages API

The example of creating a SharePoint page features see a large JSON structure composed of many properties. I wanted to simplify things to create a simple News Post page by running the New-MgSitePage cmdlet.

In PowerShell terms, the JSON structure is represented by a set of hash tables and arrays. It’s usually easier to manipulate the contents of hash tables and arrays programmatically, so that’s what I do here to create a page with a news item about a recent Office 365 for IT Pros article featuring the top five SharePoint features shipped in 2024.

$PostTitle = 'Microsoft Describes Top Five SharePoint Features Shipped in 2024'
$PostName = ("News Post {0}.aspx" -f (Get-Date -format 'MMddyyy-HHmm'))
$PostImage = "https://i0.wp.com/office365itpros.com/wp-content/uploads/2025/01/Top-Five-SharePoint-Features.png"
$PostContent = '<p> An interesting article by Mark Kashman, a Microsoft marketing manager, lists his top five SharePoint features shipped in 2024. Four of the five features involve extra cost. Is the trend of Microsoft charging extra for most new features likely to continue in 2025? The need to generate additional revenues from the Microsoft 365 installed base probably means that this is the new normal.</p><a href="https://office365itpros.com/2025/01/07/top-five-sharepoint-features-2024" target="_blank">Read full article</a>'

# The title area
$TitleArea = @{}
$TitleArea.Add("enableGradientEffect", $true)
$TitleArea.Add("imageWebUrl", $PostImage)
$TitleArea.Add("layout", "imageAndTitle")
$TitleArea.Add("showAuthor",$true)
$TitleArea.Add("showPublishedDate", $true)
$TitleArea.Add("showTextBlockAboveTitle", $true)
$TitleArea.Add("textAboveTitle", $PostTitle)
$TitleArea.Add("textAlignment", "center")
$TitleArea.Add("imageSourceType", $null)
$TitleArea.Add.("title", "News Post")

# A news item only needs one web part to publish the content
$WebPart1 = @{}
$WebPart1.Add("id", "6f9230af-2a98-4952-b205-9ede4f9ef548")
$WebPart1.Add("innerHtml", $PostContent)
$WebParts = @($WebPart1)

# The webpart is in a single column
$Column1 = @{}
$Column1.Add("id", "1")
$Column1.Add("width", "12")
$Column1.Add("webparts", $webparts)

$Columns = @($Column1)
$Section1 = @{}
$Section1.Add("layout", "oneColumn") 
$Section1.Add("id", "1")
$Section1.Add("emphasis", "none")
$Section1.Add("columns", $Columns)

$HorizontalSections = @($Section1)
$CanvasLayout = @{}
$CanvasLayout.Add("horizontalSections", $HorizontalSections)

# Bringing all the creation parameters together
$Params = @{}
$Params.Add("@odata.type", "#microsoft.graph.sitePage")
$Params.Add("name", $PostName)
$Params.Add("title", $PostTitle)
$Params.Add("pagelayout", "article")
$Params.Add("showComments", $true)
$Params.Add("showRecommendedPages", $false)
$Params.Add("titlearea", $TitleArea)
$Params.Add("canvasLayout", $CanvasLayout)

$Post = New-MgSitePage -SiteId $site.Id -BodyParameter $Params
If ($Post) { Write-Host ("Post {0} successful" -f $PostTitle) }

Update (Promote) a SharePoint Page to be a News Post

After creating a page, we might need to update it. In this case, I update the page to promote it to be a news post so that it will appear in the News section of the site. I also add a description to appear under the title in the card shown for the item in the News section.

The Update-MgSitePage cmdlet reported an “API not found” error, so I used the Graph API request:

$UpdateBody = ‘{
  "@odata.type": "#microsoft.graph.sitePage",
  "promotionKind": "newsPost",
  "description": "Microsoft Lists Top Five SharePoint Online features shipped in 2024"
}’
$Uri = ("https://graph.microsoft.com/V1.0/sites/{0}/pages/{1}/microsoft.graph.sitePage" -f $Site.Id, $Post.Id)
$Status = Invoke-MgGraphRequest -Uri $Uri -Method Patch -Body $UpdateBody
If ($Status) { Write-Host 'Post Updated'}

Publish the News with the SharePoint Pages API

The news item that’s created is in a draft state. It must be published to make it visible to other site members. I couldn’t find a cmdlet to publish a news item, so I used the Graph API request:

$Uri = ("https://graph.microsoft.com/V1.0/sites/{0}/pages/{1}/microsoft.graph.sitePage/publish" -f $Site.Id, $Post.Id)
Invoke-MgGraphRequest -Uri $Uri -Method Post

If an error isn’t reported, we can assume that SharePoint has published the page. Figure 1 shows how the page appears as a news item. I still have some bugs to figure out because the image I selected isn’t visible. There’s always something to do!

Acceptable SharePoint Pages API but Problematic Cmdlets

As far as I can tell, the SharePoint Pages Graph API works pretty well but the Microsoft Graph PowerShell SDK cmdlets generated from the API isn’t in great shape. I admit that some of the issues might be due to my lack of experience with SharePoint pages, but you do expect to be successful when you follow the documentation. I expect things to improve over time. At least, I hope improvement comes…

Need more help to write and manage PowerShell scripts for Microsoft 365? Get a copy of the Automating Microsoft 365 with PowerShell eBook, available standalone or as part of the Office 365 for IT Pros eBook bundle.

Intrazone Podcast Host Outlines His Top Five SharePoint Features

Mark Kashman, a well-known SharePoint marketing manager and host of the Intrazone podcast, recently published a wrap-up of SharePoint Online for 2024. I’ll leave you to read the full text if you’re interested, but my attention was attracted by two of the topics covered.

Top Five SharePoint Features Shipped in 2024

First, Mark listed his “top five SharePoint and related tech items that shipped in 2024,” namely:

#5: Copilot in OneDrive.

#4: SharePoint Premium.

#3: Microsoft 365 Backup.

#2: Microsoft Lists: New forms experience.

#1: SharePoint agents. I agree that this is the #1 new feature (trivia note: Microsoft recently changed the extension used for agents from .copilot to .agent).

All are worthy advances, but what drew my attention is that four of the five represent opportunities for Microsoft to increase license revenue. Consider the following:

• You can’t use Copilot in OneDrive unless you have a Microsoft 365 Copilot license ($360/user/year).
• SharePoint Premium has a pay-as-you-go (PAYG), so the cost depends on usage of features like eSignature, OCR, document processing models, and translation. It’s easy to accrue quite a bill for functions like document translation, albeit at a much lower cost than doing the work manually.
• Microsoft 365 Backup is also funded on PAYG. My experience is that Microsoft 365 Backup costs are quite reasonable. Some ISVs have integrated the Microsoft Backup API into their own products, so you can choose between the Microsoft implementation and an ISV solution.
• Access to SharePoint agents (Figure 1) are controlled by Microsoft 365 Copilot licenses. However, a trial running between January 6 and June 30, 2025, allows up to 10,000 free SharePoint agent queries per month for unlicensed users. See this page for more details. If you don’t have Microsoft 365 Copilot, you should definitely take advantage of the trial to establish if agents can do a job for your tenant.

If 80% of the top five SharePoint features shipped in 2024 cost tenants extra, it’s fair to ask if Microsoft has lost sight of delighting the Microsoft 365 installed base to chase additional revenues? I think this is true, and it’s driven by two factors: the need to get a return on its investment in AI coupled with a decline in the growth rate for new customers, something that almost became inevitable because of the size of Microsoft 365. I expect the trend to continue in 2025.

Archival of Unlicensed OneDrive for Business Accounts Begins on January 27

One thing that Kashman didn’t mention is Microsoft 365 Archive, a solution that’s about to become a lot more obvious when Microsoft begins the archival of unlicensed OneDrive for Business accounts from January 27, 2025. Tenants will need an Azure subscription linked to a credit card to pay for the storage of unlicensed OneDrive accounts subject to a retention hold (policy, label, or legal hold) and any operations to access data from the unlicensed accounts.

One point to note here is that the retrieval of data from any account automatically means that a tenant must begin to pay for the storage of every archived account. For instance, if a tenant has 100 unlicensed accounts that are in Microsoft 365 Archive and they need to retrieve data from one, they must pay for the storage of all 100 accounts before they can retrieve data from any of the accounts. Unlike the archival of SharePoint Online sites, Microsoft doesn’t offset the fees charged for storage of unlicensed OneDrive accounts against the unused portion of the tenant SharePoint storage quota. The entire consumption amount is billed. $0.05/GB/month mightn’t sound a lot, but it can mount up.

Twenty-Two Local Microsoft Datacenter Regions

The blog also featured the introduction of a new Microsoft 365 cloud region in New Zealand in December 2024 and pointed to similar developments over the last 20 months in  Poland (April 2023) Italy (June 2023), Mexico (May 2024), Spain (May 2024), and Taiwan (November 2024). In recent years, Microsoft has steadily built out the collection of in-country cloud regions to make it easier for customers to keep their data at rest within their preferred country using either multi-geo deployments or Advanced Data Residency (ADR).

The difference between the two data residency solutions is that multi-geo deployments can operate at a per-account level (the tenant has a home geography while users can be spread across multiple satellite regions) while ADR focuses on guaranteeing that tenant data for core workloads is stored at rest in a designated region. The core workloads include Exchange Online, SharePoint Online, OneDrive for Business, and Teams.

When Microsoft launched Office 365, the original focus was to serve customers from large regional datacenter deployments like the U.S. and EMEA. That focus gradually changed to deliver services from countries like the United Kingdom, Germany, Japan, India, and France. Microsoft doesn’t say what selection criteria it uses to choose where it deploys local datacenters, but it’s probably a mixture of the demand for cloud services and the desire for data sovereignty within a country. With the addition of New Zealand, there are now 22 county-level regions:

• Australia
• Brazil
• Canada
• France
• Germany
• India
• Israel
• Italy
• Japan
• Mexico
• New Zealand
• Norway
• Poland
• Qatar
• South Africa
• South Korea
• Spain
• Sweden
• Switzerland
• Taiwan
• United Arab Emirates
• United Kingdom.

It’s an impressive list that is likely to grow in 2025.

Top Features Cost Money

Microsoft 365 is a very healthy and successful ecosystem. It helps hundreds of millions of people get their jobs done daily. The level of functionality in some of its applications is staggering and the ingenuity and dedication behind delivering a reliable service across so many datacenters worldwide is commendable. The cost of Microsoft 365 base licenses is reasonable. It’s just a pity that the goodness is tinged by a blunt desire to extract more revenue per user. I guess it’s the way of the future.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Trimming to the 500 Version Limit Works Well to a Point

A reader asked about intelligent versioning, the new SharePoint Online method of controlling the amount of storage consumed by file versions created for Office documents. Intelligent versioning uses algorithms to decide what versions must be kept for file recoverability and discards (trims) unnecessary versions. The issue raised was how SharePoint Online deals with versions created past the limit of 500 versions set for sites when intelligent versioning is used.

I’ve already covered the question of how SharePoint Online removes the versions deemed to be unnecessary to recover content. If Purview data lifecycle management (retention policies) are not in force, SharePoint Online can trim versions back to the set needed to recover content. However, if retention is in force for items in a site via a retention policy or eDiscovery hold, the need to retain all versions trumps trimming and SharePoint Online cannot remove versions. According to Microsoft documentation, version trimming can occur when individual files have retention labels and aren’t under the control of a retention policy. I have not observed this to be the case because retention policies apply to all the sites in my tenant.

Retention Requirement Trumps Version Trimming

The same applies to file versions created past the 500 limit. SharePoint Online cannot remove any versions to stay within the limit when retention is in force. For instance, if a file reaches version 501, SharePoint Online will normally remove version 1 to trim the set back to 500. But if retention is in force, version 1 and all other versions must be kept so that eDiscovery processes work.

Figure 1 shows the version history for the source document for the Automating Microsoft 365 with PowerShell eBook (part of the Office 365 for IT Pros bundle). The document is updated very frequently to add new code examples and explanations or to refine existing text and has accumulated 519 versions since the creation of the original file on 10 April 2024. At the date of writing, that’s roughly two versions created each day since.

Finding that the number of versions for a file exceeds 500 is unsurprising. Given the way that the Office applications auto save automatically, several versions can be created during an editing session. For instance, creating the Word document for this article generated ten file versions. Generally speaking, the more changes are made to a file, the more versions are created, especially when new text or other elements are added to the file.

The Influence of Retention

The net result is that the current implementation of intelligent versioning does not contribute to any reduction of storage consumption when data lifecycle management is used. This is disappointing but understandable. If a tenant chooses to deploy retention policies, they obviously have a need to retain content. Being able to retrieve the current version of a document is interesting for eDiscovery investigators, but being able to retrieve earlier versions is often even more valuable.

Searching for a Solution

Whether Microsoft can do anything to resolve the conflict between storage consumption and retention remains to be seen. On the surface, it seems like this is an intractable problem. However, if algorithms can be found to discard file versions on the basis that they are not required to recover content, the ingenuity of software engineering knows no boundaries.

Perhaps the key is to offer tenants a choice between conserving storage by removing unnecessary file versions or maximum retention by keeping every available version. After all, if a version is deemed unnecessary for recovery purposes, it’s might not be of much use for eDiscovery because the differences between the preceding and following versions probably aren’t large. Of course, no self-respecting eDiscovery specialist will countenance the thought of losing any data that might possibly be of interest during an investigation, but sometimes practicality has to come first.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Easier Than Running Set-SPOSite to Configure Individual Sites with the Block Download Policy

In February 2023, I wrote about the SharePoint Online Block Download policy. At the time, the only way to assign the block download policy to a site was to run the Set-SPOSite cmdlet to update site settings. This isn’t a difficult operation and it’s likely that relatively few sites contain the level of confidential information that creates the need to block users from downloading documents.

Using PowerShell to maintain the block access policy for individual sites is not a problem. However, managing which sites have a block download policy is easier with a container management label. If configured for a container management label, SharePoint Online applies the block download policy automatically along with all the other controls set in the label to each site that receives the label.

Configuring a Container Management Label to Block Downloads

When Microsoft introduces new controls to control site behavior through container management labels, PowerShell is often the initial method chosen to apply the settings. External sharing is an example of a setting first enabled in PowerShell and later configurable through the GUI for sensitivity labels in the Microsoft Purview portal. Even though the block download policy has been available for a while, the block download policy is still considered an advanced setting that must be configured through PowerShell.

The first step is to connect to Exchange Online PowerShell and to the compliance endpoint:

Connect-ExchangeOnline
Connect-IPPSSession

To update the block download policy, run the Set-Label cmdlet to configure the advanced settings for the labels that will apply the control. In the following example, I configure settings for a label with the display name Limited Access to:

• Set the block download policy to True (the policy is active).
• Exclude site owners from the block download policy.
• Exclude the members of a selected group from the block download policy. Pass the object identifier for the group. If you want to specify several groups, separate the object identifiers with commas. You can use security or Microsoft 365 groups.

Set-Label -Identity 'Limited Access' -AdvancedSettings @{BlockDownloadPolicy="true"}
Set-Label -Identity 'Limited Access' -AdvancedSettings @{ExcludeBlockDownloadPolicySiteOwners = "true"}
Set-Label -Identity 'Limited Access' -AdvancedSettings @{ExcludedBlockDownloadGroupIds="2c2f5287-a88a-4e14-ba22-503d8b0bf3b3"}

The Effect on Users

After updating label settings, it takes about 24 hours for the policy to become effective. A background timer job detects that new label settings are available and applies them to the sites with the label. Afterward, when a user opens a site with a label that contains the block download policy settings, SharePoint Online checks if the user is a site owner or is on the excluded user list. If not, SharePoint Online applies the block download policy, and the user is restricted to working with content online (Figure 1).

The most obvious effect of the block download policy is that users must edit Office files with the browser apps. This is because SharePoint cannot download temporary copies of files for the Office desktop apps to work with.

If you don’t know if sites have assigned container management labels, run the script described in this article. For more details about how the Block Download policy works, read Microsoft’s documentation.

Advantages of Policy Assignment via Sensitivity Labels

The advantage of applying settings through sensitivity labels is obvious: once an administrator assigns a label to a container, the container inherits all the label settings. There’s no need for administrators to discover the syntax to apply individual settings to sites. You still can apply the block download policy to sites by running the Set-SPOSite cmdlet, but policy assignment via sensitivity labels is more convenient and less prone to error. It also means greater consistency in site settings because administrators know that once they apply a container management label to a site, the site automatically picks up all the settings from the label.

Using the Block Download policy requires the Microsoft SharePoint advanced management license for all users who “benefit from the policy.” In other words, anyone who connects to a site where the policy is active must have an appropriate license.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Odd Document Mismatch Notifications For No Apparent Reason

Sensitivity label mismatches occur when a user applies a sensitivity label to a document in a SharePoint Online site that has a higher priority to the container management label applied to the site. When this happens, SharePoint Online sends a document mismatch notification email to the user who caused the mismatch and to the site owners.

It’s a simple and effective way to draw attention to the potential danger of data leakage caused when sensitive information is stored in sites intended for material that perhaps isn’t so confidential.

A Flood of Document Mismatch Notifications

Recently, I noticed that some accounts were receiving a flood of document mismatch notifications. This seemed strange. The accounts receive document mismatch notifications for the entire tenant because I use a mail flow rule to centralize processing of mismatch notifications, but the volume was abnormal (472 in a week). It’s not as if many people in the tenant apart from me apply sensitivity labels to protect content!

When I examined the email, I saw that the mismatch was accurate (the Confidential -User Assigned label has a higher priority than the Confidential access container management label), but the notifications were for Word documents with odd names that humans were unlikely to have created (Figure 1).

Clicking the link to open the document brought me to the SharedVersions folder in the preservation hold library of the owning site. This is the location used by SharePoint Online to hold copies of cloudy attachments (aka “modern attachments”, or the sending of links rather than actual files) when an auto-label retention policy is in place to capture copies of cloudy attachments for eDiscovery purposes. The auto-label policy covers cloudy attachments shared in Exchange Online email and Teams and Viva Engage conversations. It also covers situations where Microsoft 365 Copilot extracts and uses content from a document in its responses, such as creating a set of key points from a document.

For instance, Figure 2 shows Microsoft 365 Chat (BizChat) extracting key points from a document. If a retention policy for cloud attachments is in force when this happens, a background SharePoint Online job captures a copy of the referenced document as a cloudy attachment and assigns the retention label defined in the policy. It can take up to an hour before SharePoint creates the copy of the cloudy attachment in the preservation hold library.

The purpose of retaining copies of cloudy attachments is to make sure that eDiscovery can find the exact content at the time it was shared through email, Teams, or Viva Engage rather than the current content. A document might be very different now to what it was when its author circulated it to peers for their review and comment. Because SharePoint Online knows what version of the file was shared, it can locate the correct copy for eDiscovery. In Figure 3 we can see that this copy of a cloudy attachment is for version 5.0 of the shared file.

The Problem with Document Mismatches in Cloudy Attachments

The idea behind retaining copies of cloudy attachments is great, but the implementation runs into a problem when a sensitivity label mismatch exists. SharePoint captures a complete copy of cloudy attachments, including the assigned sensitivity label and that’s what provokes the document mismatch notification.

There’s no way to fix the problem. You cannot change the assigned label for a file captured in the preservation hold library when a retention policy is in force because SharePoint Online blocks any attempt to change the label. Likewise, SharePoint blocks any attempt to delete (or move) labelled items, even by site or global administrators.

In summary, you can open the document and view its content, but you can’t change anything. If this wasn’t the case, it would be possible to compromise the integrity of files retained in the preservation hold library. You can exclude the site(s) from the cloudy attachment retention policy, but this only prevents the capture of future cloudy attachments.

The result is that SharePoint Online keeps on sending document mismatch notifications to the author of the cloudy attachments and the site owners. The flood of notifications continues until the retention period set for the label finishes and SharePoint Online moves the copies of the cloudy attachments to the second stage of the site recycle bin and eventually permanently deletes the files.

The simple solution would be for SharePoint Online to ignore document mismatches for anything stored in the preservation hold library.

Fix Cloudy Attachment Storage Before the Problem Gets Worse

No one seems to have protested (in a public forum) about the problem of protected cloudy attachments ending up in the preservation hold library. I guess not many tenants that use a cloudy attachment retention policy have hit the problem with document mismatches. Maybe they don’t use sensitivity labels or perhaps their users are very disciplined at how they assign sensitivity labels to files. However, as time goes on, sensitivity labels are likely to become more popular and more Microsoft 365 apps might generate cloudy attachments.

Now’s a good time to fix this particular problem. I’ve made that point to Microsoft. Let’s see if they fix the issue.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Exploit File Operations Audit Events to Find Who Accessed a Document Last

I’m speaking about how to master the unified (Microsoft 365) audit log at the European SharePoint Conference (ESPC) event in Stockholm in early December. At this point in the proceedings, the normal panic about putting together a presentation is in full swing, and I’ve been busy creating slides and examples.

In May 2024, I published an article about how to use the Microsoft Graph PowerShell SDK to create a report of files in a SharePoint Online document library. The idea is that it’s hard to understand everything that’s in a document library by scrolling through file details in the SharePoint browser app. Sometimes it’s just easier to see things in a report, and it’s definitely easier to figure out which files can be removed to clean up the document library. The temptation to leave well alone is deep in us all, but cleaning out old files from SharePoint has two benefits: it returns some storage quota, and it eliminates some of the potential for digital rot that can affect AI results.

A reader asked if the SharePoint files report could include the last accessed date for documents. The Graph API to List children of a drive item (folder) or the equivalent SDK Get-MgDriveItemChild cmdlet doesn’t return a last accessed date as far as I can see, so some other method must be used.

Analyzing SharePoint Online File Operations Audit Events

The unified audit log is a feature available to all tenants with Office 365 E3 or higher licenses. SharePoint Online creates a profusion of audit events that the audit log ingests on an ongoing basis. In this case, we’re interested in the FileAccessed event, which is logged when someone opens a file. Other events are logged for creation (FileUploaded), modification (FileModified), downloaded (FileDownload), and so on. You might be surprised at how many file operation events are logged for a busy SharePoint Online site. Figure 1 shows the count of file operations for some of documents used to generate the Office 365 for IT Pros eBook over the last six months.

Scripting a Solution Based on File Operations Audit Events

The outline of the PowerShell script to answer the request is:

• Connect to Exchange Online with an administrator account.
• Run the Search-UnifiedAuditLog to find SharePoint file operations audit events for the target site over whatever period is required. Office 365 E3 tenants store audit events for 180 days. E5 tenants store events for 365 days. Remove any duplicates that might have been fetched from the audit log. You could also interrogate the audit log with the Graph AuditLog Query API, but richer information is fetched by Search-UnifiedAuditLog.
• Filter out file events logged by human users. SharePoint Online has many background processes to do things like clean out the recycle bin, preserve files for retention, and so on. We’re not interested in system events.
• The full set of file operation events can be used to generate statistics, such as the count of user activity over the period, or the number of operations for individual files. We’re interested in file access events and use FileModified and FileAccessed events to generate this information, so the script populates a separate array with those events.
• By grouping the file access events by file name and sorting the events by date, we can easily extract the last accessed date for each file. The result is something like this:

File                                                    User                                 Timestamp
----                                                    ----                                 ---------
01 Introduction and Overview.docx                       paul.robichaux@office365itpros.com   31-Oct-2024 12:34:06
02 Managing Identities.docx                             tony.redmond@office365itpros.com     31-Oct-2024 14:12:54
03 Tenant Management.docx                               paul.robichaux@office365itpros.com   31-Oct-2024 20:21:47
04 User Management.docx                                 paul.robichaux@office365itpros.com   31-Oct-2024 20:21:48
05 Managing Exchange Online.docx                        Andy.Ruth@office365itpros.com        29-Oct-2024 20:45:03
06 Managing Mail Flow.docx                              James.ryan@office365itpros.com       29-Sep-2024 15:07:31
07 Managing SharePoint Online.docx                      tony.redmond@office365itpros.com     14-Oct-2024 13:00:56
08 Managing Tasks.docx                                  paul.robichaux@office365itpros.com   29-Oct-2024 19:40:47
09 Managing Video.docx                                  paul.robichaux@office365itpros.com   29-Oct-2024 19:40:47
10 Managing Microsoft 365 Groups.docx                   brian.weakliamoffice365itpros.com    20-Oct-2024 17:49:23
11 Teams Architecture and Structure.docx                tony.redmond@office365itpros.com     16-Oct-2024 15:02:20
12 Managing Teams.docx                                  Lotte.Vetler@office365itpros.com     04-Nov-2024 19:01:57

Two odd user identifiers for bdc6105c-4e11-4050-82e6-6549f9b99b89 and eba15bfd-c28e-4433-a20e-0278888c5825 can appear in file operation events. I assume these identifiers belong to background SharePoint Online processes, so the script filters these events from the set.

You can download the complete script from GitHub.

Good Example of the Power of the Audit Log

Finding who last accessed SharePoint Online documents and when that access occurred is a good example of why the unified audit log is a great repository of information for tenant administrators and forensic investigators alike. If you’re at ESPC 24 in Stockholm, come along to my session on Decoding the Microsoft 365 Audit Log on Tuesday, December 3 at 10:30am. I’ll share more useful tips about exploiting the audit log there.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Trimming Unwanted Versions Stopped by Retention Requirements

Last month, I wrote about the introduction of Intelligent Versioning for SharePoint Online. I think this is a great feature because its automated management of versions created during editing sessions reduces the storage quota consumed to store file versions. The advent of AutoSave for Office increased the number of versions created for files, and keeping 500 or so versions for a file, when some versions only include minimal changes, is effective but expensive.

Microsoft allows tenants a default storage quota for SharePoint Online that’s consumed by items stored in sites and Loop workspaces (containers). If a tenant exceeds their SharePoint storage quota, they must buy more from Microsoft or use Microsoft 365 Archive to move the storage consumed by inactive sites to cheaper “cold” storage.

As I noted in the article, the big issue with the current implementation of intelligent versioning is that it doesn’t work with Purview Data Lifecycle management, aka Microsoft 365 retention policies. If SharePoint Online sites come within the scope of one or more retention policies, then the requirement to retain information about files for use by workloads such as eDiscovery trumps the desire of intelligent versioning to remove unwanted versions for those files.

Checking Expired Versions Trimmed by Intelligent Versioning

Microsoft’s documentation explains how retention works with document versioning. I decided to check out what happens when versions expire for documents in a site with a retention policy in force. On November 6, I noted that several versions were in an expired state (Figure 1).

The next day, the expired versions were gone from the list. In one respect, this is what you might expect to happen. A background SharePoint Online job detected the existence of expired versions and removed them, which is what intelligent versioning is all about (the process is called trimming).

But the retention policy applied to the site set a five-year retention period and the document had a retention label with a ten-year retention period. The document is a source file for the Office 365 for IT Pros eBook, and you can never be too careful with source material. The longest retention period wins, so SharePoint Online should retain the file for ten years. However, no trace could be found of the removed versions.

Microsoft’s documentation says that versions for items subject to a hold imposed by a retention policy are not automatically purged. In addition, users cannot delete versions from the Version history. When intelligent versioning trims versions in a site without retention policies, the files bypass the recycle bin. This didn’t apply, so it seemed like the site preservation hold library is the logical place to look. However, nothing was found in the preservation hold library except the copy of the file containing all versions prior to the implementation of intelligent versioning in the tenant.

Reappearing Versions

Then the removed versions reappeared in the version history complete with a new expiration date (Figure 2). Interestingly, SharePoint Online adjusted the expiration date for some other versions to make sure that full coverage of changes to the file is available.

After chatting with Microsoft engineering, I understand that the observed behavior is quite normal. The expired versions are removed by a background job, only for retention processing to detect that the removed versions are still within their retention period and required by the retention policy. This causes SharePoint to add a week to the previous expiration date for each version and make the versions available again. The cycle then repeats until the retention period for removed versions lapses to allow SharePoint Online to permanently remove the unwanted versions from its store.

The problem does not happen for retention labels because labels do not impose in-place holds on content. Instead, retention labels are designed to keep (or remove) content after a certain period. The retention action used at the end of the period applies to the current version and doesn’t involve any versions, so intelligent versioning can trim unwanted versions for files with retention labels. Of course, some files have retention labels and come within a scope of a retention policy. In this scenario, the retention hold keeps all versions.

More Intelligence in the Future?

It’s unfortunate that a clash exists between storage management and retention policies. Microsoft’s current approach is probably the best that can be done for now. I’m sure that they have an eye on the potential to extend intelligent versioning to interact with retention processing better. One possibility is to allow organizations to decide if selective version trimming is permissible, perhaps at a less aggressive level. For instance, it’s OK to remove versions that only contain formatting changes but not OK to remove any that contain text additions or deletions. Perhaps some storage savings are possible without compromising retention. It’s a hard nut to crack.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Time to Consider the Impact on User Profiles for Microsoft 365 Tenants

Microsoft announced the demise of the Delve browser app in December 2023 and on October 1, they issued a reminder in MC902780 that December 16, 2024 is when the curtain finally descends on Delve. Microsoft’s formal guidance on the Delve retirement is available online along with a support document.

The Microsoft 365 User Profile

I think it’s undeniable that the management of user profile information, including photos, within Microsoft 365 has been a mess for a long time. The underlying reason is simple: Microsoft 365 is built from the foundation of on-premises servers like Exchange and SharePoint, each of which had its own directory and method to store profile photos. Throw in cloud services like Yammer (Viva Engage) and Teams and the water became even muddier.

It’s also fair to say that Microsoft has taken far too long to rationalize the situation. At one point, Delve seemed to be a potential solution, but things didn’t work out with the app. However, it’s disappointing that Microsoft didn’t see the issue and do something about the problem more quickly.

One thing is obvious. Entra ID is the directory of record for Microsoft 365. If you want to store information about people, store it in Entra ID, which supports a wide range of properties for user accounts that can be surfaced on the profile card. If you want to store custom information about people that’s specific to your tenant, use the predefined custom attributes for the job. If you need more than fifteen custom attributes, consider using Entra ID custom security attributes. The downside is that these attributes can’t be added to the profile card.

The Microsoft 365 User Profile Card

Even as different services competed to store profile data, Microsoft 365 introduced the user profile card. This is a common component used across Microsoft 365 to display properties of user accounts, including customizable properties. After December 16, Microsoft will redirect from Delve profiles to the Microsoft 365 search experience, which displays the same data as user profiles. The sample URLs described in the document seem more complex than what’s needed. I use the following format. Figure 1 shows the result

https://www.microsoft365.com/search/?q=sean.landy@office365itpros.com

The redirects will take care of casual browsing for user information. What it won’t do is allow users to upload their preferred choice of profile photos, nor will it allow users to update profile details in the same way as is possible with Delve.

Updating User Profiles

Microsoft says that they are working on a new “edit profile experience” that is “tightly coupled with the profile card.” This work is due for release in November 2024 and should allow users to edit their profile information “across Microsoft 365.” Only properties that can be edited today with Delve will be exposed and editable via an Update your profile button in the profile card.

Other details that can be set in SharePoint profiles today won’t appear in the Microsoft 365 profile card because the profile card is designed to work across the service. It’s time to move this kind of information about people out of SharePoint and into Entra ID.

Of course, it will take time for the upgraded profile card to appear in apps, but at least we know the direction in which Microsoft is going.

User Profile Photos

Microsoft doesn’t mention user profile photos in their guidance for the Delve retirement. This is odd because Delve is one of the places where Microsoft 365 users can upload profile photos.

My assumption is that the new edit profile experience will include the ability to upload photos. We won’t know if this is the case until the new UI is available. I hope that the current controls over who can upload photos will be used rather than the non-granular Entra ID Photo Update Settings policy that’s coming into view. I’ve no doubt that the photo update settings policy will be the long-term control for Microsoft 365, but it would be nice if Microsoft made it optional until the policy works in the way that it should.

And a Mention for Copilot

It seems like Microsoft sometimes operates an edict that Copilot for Microsoft 365 should be mentioned in all documentation. Copilot appears in the support documentation, which solemnly informs the reader that Copilot can find information about your colleagues, possibly as a replacement for browsing Delve. The thing is that Delve was free to tenants with Office 365 E3 and Copilot costs $30/user/month. It’s hardly a comparison.

In any case, the summary of the Delve retirement is that Entra ID is the directory of record, the user profile card feeds off Entra ID and will have a way for users to update their details. The new profile card will appear in apps gradually. After all these things happen, we will forget about Delve and its retirement next December.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Revoke-SPOUserSession is No Longer Fit for Purpose

Microsoft’s announcement in message center notification MC903785 (3 October 2024) that they will retire the Revoke-SPOUserSession cmdlet (in the SharePoint Online PowerShell module) in early November 2024 was expected. There’s no purpose served by having a workload-specific cmdlet to revoke user access to an app when the job can be done across all workloads with a single cmdlet built for the job. That cmdlet is Revoke-MgUserSignInSession, which I discuss in an article about the right way to revoke access from Entra ID accounts.

The Roots of Revoke-SPOUserSession

Microsoft introduced the Revoke SPOUserSession cmdlet in January 2016. That’s an aeon in cloud terms. Teams hadn’t yet appeared, Azure AD delivered a much simpler directory and authentication service, with no notion of features like continual access evaluation (CAE), and SharePoint Online wasn’t trying to deal with nearly 4 billion files created daily.

At the time, the primary access to SharePoint Online was through the browser (now I suspect primary access is via Teams), and I’m sure that it made perfect sense to create a cmdlet to force the sign-out of a user from SharePoint Online across all devices.

Retiring Revoke-SPOUserSession

Microsoft says that their telemetry indicates that only a few organizations are active users of Revoke-SPOUserSession. I’m surprised that even a few tenants exist that might still use the cmdlet because better options have existed for some time, cumulating with the Revoke-MgUserSignInSession cmdlet the Microsoft Graph PowerShell SDK.

The critical difference is that the SDK cmdlet forces a sign-out from all Microsoft 365 sessions, not just SharePoint Online. It’s an essential part of any administrator action to secure an account because of suspected compromise or because an employee is leaving the organization. If you’re in the category of those who have scripts that use Revoke-SPOUserSession, it’s time tio change before the curtain comes down.

Securing an Employee Account

All of which brings me to the second annual PowerShell script-off at TEC 2024 (in Dallas). It’s quite a challenge to strut your PowerShell skills in front of a sometimes-boisterous crowd, and I admire the folks (Figure 1) who stepped up to take part.

The first challenge was to write a script to automate the securing of my account (I make a great victim) after my forced ejection from the organization at 9AM on Monday. You’d imagine that this is a well-trodden path with many sample scripts available on the internet, so it was surprising the difficulty some had with the challenge. Competitors couldn’t use ChatGPT and Microsoft 365 Copilot to avoid any hint of generative AI spoiling the responses, and it was interesting to see how people approached the issue without that kind of help.

Most immediately focused on disabling the Microsoft 365 account. This is undoubtedly an important step, but there’s more to be done, like:

Forcing a sign out with Revoke-MgUserSignInSession is a great next step, but only after changing the account password. You don’t want to have someone be prompted to reauthenticate because their access tokens are invalid only to be able to sign in again because their account password is changed. Yes, disabling the account should stop the sign-in, but let’s be sure.

Securing devices is another step. It all depends on what device management software a tenant uses, but it should be possible to wipe corporate data from devices to prevent ex-employees having continued access to local copies. Sensitivity labels help here by making sure that even if an ex-employee takes copies of sensitive files, they won’t be able to authenticate and gain the right to access the content. Sensitivity labels put a stop to the tactic often seen when people just about to leave exfiltrate large amounts of confidential documents and email (in PSTs) to removeable devices. Exfiltration might work, but once the ex-employee can no longer authenticate, the confidential material becomes no more than an interesting collection of bytes.

It’s Hard to Revoke Access

No one quite delivered a script to totally secure an ex-employee’s account in the 20 minutes allotted for the task (one solution was delivered that removed access from every account in the tenant). Even with access to the internet, it takes time to find, assess, and decide what code to base a solution on. The difficulty is compounded when people are looking over your shoulder to criticize every move, or even when you find a great cmdlet to revoke access that Microsoft’s just about to deprecate…

Learn more about how the Microsoft 365 applications really work on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

SharePoint Advanced Management Focusing on the Challenges of the AI Era

An interesting TEC 2024 session covering SharePoint Online security, reporting, and artificial intelligence given by Sanjoyan Mustafi, principal program manager for SharePoint and OneDrive provoked more questions than it answered.

Sanjoyan covered the current and some future capabilities of SharePoint Advanced Management (SAM), a premium add-on license announced in March 2024. SAM includes solutions to address the problems of oversharing, data governance, and lifecycle management for SharePoint Online sites. Sanjoyan noted that nearly 4 billion documents are uploaded to Microsoft 365 daily, a substantial increase in the 2.5 million often cited by Microsoft spokespeople.

SAM spans reports and policies. Some of the reports generated by SAM depend on audit records and reflect historic actions such as people sharing using anyone links. Others use current state data, meaning that they reflect near real time data. Policies include the block download policy and a conditional access policy to restrict access to sensitive SharePoint Online sites using authentication contexts. Another policy restricts access to OneDrive for Business accounts to specific users. These are all useful features to help manage access to SharePoint content.

But the discussion about oversharing made me think that Microsoft is taking an opportunity to sell yet another add-on ($3 user/month) to fix flaws revealed by Microsoft 365 Copilot that are a direct result of poor decisions made by Microsoft in the past.

The Grave Error of Unfettered Group Creation

The biggest example I can give is the decision made in November 2014 not to impose control over who could create Office 365 Groups (now Microsoft 365 Groups). The idea was to foster collaboration. Despite strong argument against the decision based on knowledge of the disaster Exchange public folders became when users were allowed free rein, Microsoft persisted and launched the era of open collaboration at the Ignite conference in May 2015.

The mistake was compounded in November 2016 when Microsoft released the preview of Teams and allowed anyone to create a new team. Even worse, when Entra ID (then Azure AD) introduced a policy to allow tenants to dictate who could create Microsoft 365 groups, they insisted on making this a feature covered by the Entra P1 license. This control should have been part of the base product since day 1.

The result is plain to see with massive team sprawl in many tenants. Sanjoyan said that approximately 90% of the SharePoint sites created in Microsoft 365 are team-enabled. Many of those teams are inactive, badly managed, or ownerless, all of which are factors that contribute to poor data governance. The question must be asked if the same situation would exist had Microsoft had seen sense and allowed tenants to control group creation from the start. I say no, but we are where we are.

The Era of Copilot

None of this mattered too much until Microsoft 365 Copilot arrived. Being grounded in the Graph means that Copilot can access and use any document available to the signed-in user when it responds to user prompts. That doesn’t mean documents containing accurate and useful information. It means any document stored in sites where the user is a member or can be accessed through a sharing link. The corpus of documents available to Copilot can contain misleading, inaccurate, and just plain information. Copilot doesn’t care and can’t tell the difference between an accurate and incorrect fact.

Reasoning over files that contain bad data means that Copilot can include bad information in its responses. This is why Microsoft has rushed to limit the free access Copilot enjoys via Graph queries with solutions like Restricted SharePoint Search and the sensitivity label setting that blocks access for individual documents to Microsoft Content Services. A new solution called Restricted Content Discoverability (RCD) is in private preview. RCD allows tenants to exclude sites from Copilot access. It seems like a much better approach than Restricted SharePoint Search, which limits Enterprise Search to 100 curated sites.

Restricted Access Control (RAC) for SharePoint Online and OneDrive for Business is already available. RAC means that no matter whet sharing links are present on files in a site, the only people who can access the files are users in groups specified in an access list. Microsoft 365 Copilot respects RAC and won’t access files in protected sites unless the signed-in user is in the access list.

Maybe Bundle SharePoint Advanced Management with Microsoft 365 Copilot

SharePoint Advanced Management isn’t all about Microsoft 365 Copilot, but the need to control oversharing for Copilot seems to be the current focus for SAM. Given that, wouldn’t it make sense for Microsoft to bundle SharePoint Advanced Management with Microsoft 365 Copilot? It sure seems like a good idea to me.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Critical Need to Update Scripts Using PnP PowerShell Before September 9 2024

On August 21, 2024, the Pattern and Practices (PnP) team announced a major change for the PnP PowerShell module. To improve security by encouraging the use apps configured with only the permissions needed to process data within the tenant, the PnP PowerShell module is moving away from the multi-tenant Entra app (the PnP Management Shell, application identifier 31359c7f-bd7e-475c-86db-fdb8c937548e) used up to this point to require tenants to register a unique tenant-specific app for PnP.

Reading between the lines, the fear is that attackers will target the current PnP multi-tenant app and attempt to use it to compromise tenants. The multi-tenant app holds many Graph API permissions (Figure 1) together with a mixture of permissions for Entra ID, SharePoint Online, and the Office 365 service management API. Being able to gain control over such an app would be a rich prize for an attacker.

Swapping out one type of Entra app for another might sound innocuous, but it means that the sign-in command for PnP in every script must be updated. The PnP team will remove the current multi-tenant app on September 9, 2024, so any script that isn’t updated will promptly fail because it cannot authenticate. That’s quite a change.

The Usefulness of PnP PowerShell

I don’t use PnP PowerShell very often because I prefer to use Graph APIs or the Microsoft Graph PowerShell SDK whenever possible. However, sometimes PnP just works better or can perform a task that isn’t possible with the Graph. For instance, creating and populating Microsoft Lists is possible with the Graph, but it’s easier with PnP. SharePoint’s support for Graph APIs is weak and PnP is generally a better option for SharePoint Online automation, such as updating site property bags with custom properties (required to allow adaptive scopes to identify SharePoint Online sites). Finally, I use PnP to create files in SharePoint Online document libraries generated as the output from Azure Automation runbooks.

Creating a PnP Tenant Application

The first thing to do is to download the latest version of the PnP PowerShell module (which only runs on PowerShell 7) from the PowerShell Gallery. The maintainers update the module regularly. I used version 2.9.0 for this article.

The easiest way to create a tenant-specific application for PnP PowerShell is to run the Register-PnPEntraIDApp cmdlet:

Register-PnPEntraIDApp -ApplicationName "PnP PowerShell App" -Tenant office365itpros.onmicrosoft.com -Interactive

Make sure that you sign in with an account that has global administrator access. The cmdlet creates an Entra ID app and populates the app with some default properties, including a default set of Graph API permissions and a self-signed certificate for authentication. It doesn’t matter what name you give the app because authentication will use the unique application identifier (client id) Entra ID creates for the new app. The user who runs the cmdlet must be able to consent for the permissions requested for the app (Figure 2).

The Graph permissions allow read-write access to users, groups, and sites. Other permissions will be necessary to use PnP PowerShell with other workloads, such as Teams. Consent for these permissions is granted in the same way as for any other Entra ID app. Don’t rush to grant consent for other permissions until the need is evident and justified.

Using the Tenant App to Connect to PnP PowerShell

PnP PowerShell supports several ways to authenticate, including in Azure Automation runbooks. Most of the examples found on the internet show how to connect using the multi-tenant application. To make sure that scripts continue to work after September 9, every script that uses PnP PowerShell must be reviewed to ensure that its code works with the tenant-specific application. For instance, a simple interactive connection looks like this:

Connect-PnPOnline -Url https://office365itpros.sharepoint.com -ClientId cb5f363f-fbc0-46cb-bcfd-0933584a8c57 -Interactive

The value passed in the ClientId parameter is the application identifier for the PnP PowerShell application.

Azure Automation requires a little finesse. In many situations, it’s sufficient to use a managed identity. However, if a runbook needs to add content to a SharePoint site, like uploading a document, an account belonging to a site member must be used for authentication. This example uses credentials stored as a resource in the automation account executing the runbook.

$SiteURL = "https://office365itpros.sharepoint.com/sites/Office365Adoption"
# Insert the credential you want to use here... it should be the username and password for a site member
$SiteMemberCredential = Get-AutomationPSCredential -Name "ChannelMemberCredential"
$SiteMemberCredential
# Connect to the SharePoint Online site with PnP
$PnpConnection = Connect-PnPOnline $SiteURL -Credentials $SiteMemberCredential -ReturnConnection -ClientId cb5f363f-fbc0-46cb-bcfd-0933584a8c57

[array]$DocumentLibraries = Get-PnPList -Connection $PnpConnection | Where-Object {$_.BaseType -eq "DocumentLibrary"}
 
# Display the name, Default URL and Number of Items for each library
$DocumentLibraries | Select Title, DefaultViewURL, ItemCount

Ready, Steady, Go…

September 9 is not too far away, so the work to review, update, and test PnP PowerShell scripts needs to start very soon (if not yesterday). Announcing a change like this 19 days before it happens seems odd and isn’t in line with the general practice where Microsoft gives at least a month’s notice for a major change. I imagine that some folks coming back from their vacations have an unpleasant surprise lurking in their inboxes…

Understand Why Unlicensed OneDrive Accounts Exist

At the end of July, I reported Microsoft’s plan to charge for unlicensed OneDrive for Business accounts. The idea is simple. Ninety days after a OneDrive for Business account enters an unlicensed state, SharePoint Online will move the account into Microsoft 365 Archive. The tenant must then decide what to do with the accounts with the options being to manage the accounts or leave accounts to rot in the archive. Unlicensed accounts arise when an account no longer has access to a service plan for OneDrive (see the product names and service plans reference). Usually, an account enters the unlicensed state for OneDrive when an administrator deletes an account or removes a license like Office 365 E3 or E5 from the account.

Managing accounts requires the tenant to link Microsoft 365 Archive to an Azure subscription to pay for ongoing storage and restore operations. Storage costs $0.05 per month per gigabyte while retrieval costs $0.60 per gigabyte. Restored accounts remain accessible for 30 days. During this time, someone has to review the material in the account and move it to another repository, such as a different OneDrive for Business account or a SharePoint Online site. Once the 30-day period lapses, SharePoint Online archives the account again.

The OneDrive Report

So good, so far. Archiving old OneDrive accounts that clutter up storage is a good idea. It stops artificial intelligence tools like Copilot for Microsoft 365 using the content held in the obsolete accounts in its response to users and helps to better manage information belonging to ex-employees.

When Microsoft issued MC836942 on July 26, they said that by August 16, 2024, SharePoint administrators would be able to access a new report detailing unlicensed OneDrive for Business accounts. The OneDrive report should now be available through the Reports section of the SharePoint admin center in all tenants (Figure 1).

Note the warning that if accounts are left in Microsoft 365 Archive for more than 180 days after becoming unlicensed and the tenant does not take out an Azure subscription to pay for the Microsoft 365 Archive storage costs, SharePoint Online can delete the accounts. No documentation is currently available to cover this point, but it seems reasonable that Microsoft should remove old and unwanted OneDrive accounts if the owning tenant is unwilling to pay the storage costs to keep them in the archive.

Four Categories of Unlicensed OneDrive Accounts

Unlicensed OneDrive accounts fall into four categories:

• Retention period: The owning account is unlicensed but SharePoint Online has retained the OneDrive account because the retention period configured in the SharePoint admin center has not expired.
• Retention policy: A Microsoft 365 retention policy or retention labels prevent the deletion of an unlicensed OneDrive account. It is quite common for tenants to apply a blanket retention policy to all SharePoint Online sites and OneDrive accounts to retain information for multiple years. If this happens, the unlicensed OneDrive accounts cannot be removed until the retention period defined by the policy lapses.
• Active user with no license: The account that owns the OneDrive account is still active (is not deleted), but no longer has access to a service plan for OneDrive.
• Duplicate accounts: The account that owns the OneDrive account has several OneDrive accounts. This used to happen more often several years ago when account provisioning was not as good as it is now. I have not seen a duplicate account created in the recent past.

Figure 1 shows that my tenant has 34 unlicensed OneDrive accounts held by a retention policy. This is expected because I use a broad retention policy to govern removal of material from SharePoint Online and OneDrive for Business. Currently, you cannot see details of the accounts within each of the four categories on-screen. Instead, you must download the CSV file containing the details. In their documentation, Microsoft promises that an interactive UI will be available from January 2025, saying that “You can select a username to view the details.” Presumably, this means that the various sections in the on-screen report will expand to show usernames, and you can then expand a username to see its details, such as those available in the CSV file (Figure 2).

Time to Review Unlicensed OneDrive Account Information

Now that information about unlicensed OneDrive accounts is available in the SharePoint admin center, tenant administrators should check the report and review its content to determine if anything unexpected is present. I don’t imagine that anything strange will turn up, but you never know. Following the review, administrators might decide to adjust retention periods and policies to allow the removal of OneDrive accounts belonging to deleted Entra ID accounts or prepare for long-term storage in Microsoft 365 Archive.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Controlling Stream Video Versions Designed to Consume Less Disk Storage

Microsoft 365 message center notification MC797116 (30 May 2024, Microsoft 365 Roadmap item 395380) addresses the question of storage consumption in SharePoint Online and OneDrive for Business for videos managed by Stream. The issue is simple. Any time a video owner updates the non-video content, Stream creates a brand-new version of the video that consumes the same amount of storage as the original.

Many reasons exist to change something for a video, like editing the metadata (title, description (Figure 1), or chapters), editing the transcript to correct flaws in the automatic text generated by the transcription bot, adding callouts through the interactivity feature, and so on.

Many Stream Video Versions

Behind the scenes, SharePoint Online or OneDrive for Business treat changes to non-video content in the same way as they handle changes made to Word documents or Excel spreadsheets and create new versions. The impact on storage is obvious if you look at the version history for a video. Figure 2 shows the version history for a 402 MB video that consumes 5,226 MB for the 13 versions stored by OneDrive.

Usually, this method of storing versions doesn’t affect OneDrive for Business accounts. Given that most videos are likely Teams meeting records, few videos are updated, and the version count remains small. In addition. The large storage quotas assigned to OneDrive for Business accounts accommodate a few extra versions without a problem.

The issue is more obvious in SharePoint Online where the tenant-wide storage quota comes under pressure from user demand for document storage, retention processing, and versioning. Buying additional SharePoint Online storage is expensive, and few tenants want to go down that route.

Microsoft announced intelligent versioning for SharePoint Online in July 2023, but according to Microsoft 365 roadmap item 145802, the rollout won’t happen until August 2024. Good things take time to get right.

The Change in the Creation of Stream Video Versions

The change Microsoft is introducing to Stream starting mid-July 2024 with the intention to complete worldwide deployment by late August 2024 is to stop generating new versions of videos for changes that do not affect video content. This is a reasonable approach, and it will prevent the kind of video version sprawl seen in the past (as obvious in Figure 2).

The downside is that metadata changes made to Stream videos are irrecoverable. If you restore a version of a video, you get the metadata available at that time. Any subsequent changes made to video metadata are ignored.

These actions no longer create a new version:

• Editing the title or description from within the Stream browser client.
• Adding or editing chapters, transcripts, captions, or interactivity (callouts or forms).
• Toggling media settings (show/hide about video, chapters, interactivity, comments, analytics, etc.).
• Adding audio tracks.

Any change that affects the video content, like trimming some seconds from the start or end of a video, will force Stream to generate a new version of the video. Once the change reaches your tenant, it goes into effect and cannot be reverted to the previous behavior. The change has no effect on existing videos and will not remove any versions that are already being stored. Microsoft says that if you want to remove extraneous versions, you’ll need to wait for SharePoint Intelligent versioning to appear in your tenant and use that to clean up unwanted video versions stored in SharePoint sites.

Storage is Not a Pressing Problem for OneDrive

At this point, I am unsure if the same approach can be taken to clean up video versions in OneDrive for Business accounts. However, given that storage is much less of an issue in OneDrive than it is in SharePoint Online, and that Teams meeting recordings age out over time, this is probably not a big problem. If you’re worried about OneDrive, run the OneDrive for Business account storage and quota report and see if any account needs attention. I bet hardly any will.

Make sure that you’re not surprised about changes that appear inside Microsoft 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Microsoft Will Remove-SPOExternalUser Between July 29 and August 9

Message center notification MC806103 (27 June 2024) reports the deprecation of the Remove-SPOExternalUser cmdlet from the SharePoint Online management PowerShell module. Microsoft suggests that administrators replace the cmdlet with the Remove-AzureADUser cmdlet, which is a perfectly reasonable strategy if only the cmdlet isn’t part of the retired and soon-to-be deprecated AzureAD module.

Between July 29. 2024 and August 9, 2024, Microsoft will disable the Remove-SPOExternalUser cmdlet. When the block arrives in a tenant, attempts to run the cmdlet will be greeted with:

“To streamline scope and permissions for external users, enhance access management, and strengthen our security posture, this cmdlet has been deprecated. Alternatively, please use the Remove-AzureADUser cmdlet in Microsoft Entra ID for user management.”

Microsoft 365 is so Large that No One Understands Everything

MC806103 is a classic example of Microsoft being such a large organization that no one knows what’s happening across the board, or even what’s happening within Microsoft 365. In this case, the SharePoint Online people want to deprecate the Remove-SPOExternalUser cmdlet. That’s a good idea because the cmdlet has low usage (I don’t think I have ever used it) and doesn’t really make sense inside the Microsoft 365 ecosystem where external access for applications like SharePoint Online is now governed using guest accounts. It makes perfect sense to remove overlapping or conflicting features and replace them with what you’d consider a component that’s closer to the core.

Entra ID is the directory of record for Microsoft 365. Individual workloads like SharePoint Online have their own directory, but everything flows back to Entra ID. Replacing the SharePoint Online cmdlet with an Entra ID cmdlet is the right thing to do. The problem is that the program manager in charge of making the transition obviously doesn’t know that the Entra ID team has been trying to deprecate the AzureAD and AzureADPreview modules since 2020. For the last few years, Microsoft has conducted an ongoing campaign to move tenants off these modules to use the Microsoft Graph PowerShell SDK.

What makes this laughable is that Microsoft launched the Entra PowerShell module in preview on June 27 in the hope that a dedicated Entra module (built on top of the Microsoft Graph PowerShell SDK) would help the remaining customers who have scripts that use the AzureAD and AzureADPreview modules to move to a modern platform. Obviously, whoever wrote MC806103 had no idea that this development was in train.

The Right Way to Replace Remove-ExternalSPOUser

The Get-SPOExternalUser cmdlet reports the external users registered for a SharePoint Online tenant. The last time I discussed its use, I observed that the Get-SPOExternalUser cmdlet is an odd cmdlet in some ways, but it does generate a list of external users from the SharePoint directory.

An external user record looks like:

RunspaceId    : 9630573b-c675-4697-a029-72d535e48613
Email         : charu.someone@microsoft.com
DisplayName   : Charu Someone
UniqueId      : 100320009C9C6789
AcceptedAs    : charsomeone@microsoft.com
WhenCreated   : 20/02/2020 19:45:02
InvitedBy     :
LoginName     :
IsCrossTenant : False

Remove-SPOExternalUser works like this:

Remove-SPOExternalUser -UniqueIDs ($User).UniqueId -Confirm:$false
Successfully removed the following external users
100320009C9C6789

The cmdlet removes the external user entry from SharePoint Online. It also removes the matching guest account, if one exists, from Entra ID. In my tenant there are quite a few lingering external accounts that don’t have matching Entra ID guest accounts. These accounts are just another form of digital debris that needs to be cleaned up.

The right way to remove an external account is to use the Remove-MgUser cmdlet from the Microsoft Graph PowerShell SDK:

$User = Get-MgUser -filter "mail eq 'andrew@proton.me"
Remove-MgUser -UserId $User.Id

Or, if you decide to use the preview Entra module:

$User = Get-EntraUser -SearchString 'AdamP@contoso.com'
Remove-EntraUser -ObjectId $User.Id

Either cmdlet has a much longer future ahead of it than the Remove-AzureADUser cmdlet has. In both cases, SharePoint Online synchronizes with Entra ID and removes the matching external user record.

It’s Just Hard to Keep Up

I don’t blame the individual program manager responsible for MC806103. It’s hard to keep up with everything that goes on within Microsoft 365 and all too easy to assume that a solution that works (for now) is the right long-term recommendation. Perhaps Microsoft needs a clearing house to cross-check dependencies outside the control of an individual development group before they publish information to customers?

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Support for Sharing Links Expiration Added for Company-wide and Specific People Links

The change announced in message center notification MC799277 (6 June 2024) to make expiration dates available for all types of sharing links should now have reached targeted release tenants. General availability will follow soon afterwards.

Until now, SharePoint Online and OneDrive for Business have supported expiration dates for anyone sharing links. Microsoft was correct to start with these links because they are transferable. In other words, anyone in possession of the link can access the file or folder pointed to by the link.

Although anyone links are revocable and therefore can be annulled if a link becomes too widely available, making them the first sharing link to support expiration was a good thing. Even with expiration dates, many organizations prohibit anyone links because they consider these links to be too dangerous. Users forget to set expiration dates, the links circulate in email and can easily escape outside the organization, and so on.

Company-wide and Specific People Sharing Links

Company-wide (aka people in your organization) and specific people links deliver tighter control over sharing because SharePoint Online validates the account that attempts to redeem and use a link to make sure that they meet the sharing criteria. For example, if you’re not signed into a tenant account, SharePoint Online won’t allow you to use a company-wide link.

Specific people links are usable with people inside and outside an organization. External people must have a guest account in the tenant to authenticate, either an account created to access other resources like Teams (or most recently, Microsoft Loop), or an account created during the process of gaining access to the shared content. During this process, depending on the conditional access policies active in the tenant, an external person might be asked to configure multi-factor authentication to protect their account.

All of this sounds good, and it means that specific people links are usually a safe way to share externally, especially if coupled with a sensitivity label with encryption to stop any inadvertent leakage of confidential information.

Sharing Links Expiration for All

Using a sharing link that SharePoint Online imposes controls over who can use the link can sometimes do with a little extra help and that’s where the expiration controls come in. You can now set a date (Figure 1) for company-wide and specific people sharing links to expire.

When a link expires, it can no longer be used to access the shared content. The owner of the content must then reshare the content if they wish.

It seems like Microsoft has some loose ends to clean up before you could consider this feature to be complete. For instance, although SharePoint Online shows the expiration date after copying a link (Figure 2), the Manage access dialog for an item doesn’t display expiration dates. This might be due to an incomplete software deployment and the missing bits for an updated Manage access dialog might be still on the way.

More importantly, the SharePoint Online admin center has a setting for Anyone links to set a maximum expiration length in days (Figure 3). However, similar controls aren’t yet available for company-wide and specific people links. That seems like an oversight.

Being picky, I could also point out that setting an expiration period for a sharing link does not affect the SharingSet audit record generated when SharePoint Online or OneDrive for Business configure a sharing link. This is a pity. Microsoft needs to improve the information captured in audit records for sharing events to make them more administrator friendly. For instance, a value like “EventData    : <PermissionsGranted>Contribute</PermissionsGranted><MembersCanShareApplied>False</MembersCanShareApplied>” is meaningful to a computer but less so to a human. If you’re interested in learning how to interpret audit records for sharing events, try this script from GitHub.

Sharing Links Expiration for All Link Types is a Good Change

Even though I think Microsoft has some things to work on to complete the feature, I like that SharePoint Online supports expiration dates for all types of sharing links. It’s a good change and one that should be popular with users, even if administrators can’t find out the kind of usage the feature gets because of the lack of detail in audit records. According to MC799277, Microsoft is due to refresh the documentation, but that hasn’t happened yet. More bits lost in transit!

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Why Some People Can’t Use Shared Files with Copilot for Microsoft 365

After reading the article about the new sensitivity label advanced setting to block access for Microsoft content services to confidential Office documents, a reader asked why some users can use some documents shared using company-wide links with Copilot for Microsoft 365 while others cannot. The situation seemed a little strange because it happened for documents shared with everyone in the organization. The problem couldn’t be due to a sensitivity label because the capability only just rolled out and is limited to the Office applications.

The answer is in Microsoft’s documentation for secure file sharing, which says: “Creating a People in your organization link will not make the associated file or folder appear in search results, be accessible via Copilot, or grant access to everyone within the organization. Simply creating this link does not provide organizational-wide access to the content. For individuals to access the file or folder, they must possess the link and it needs to be activated through redemption.”

In other words, sharing a file with everyone in your organization is only the first step in the process of making information available to Copilot for Microsoft 365. A company sharing link that arrives in your inbox or is shared through a Teams chat is dormant until you redeem it by using the link. At that time, SharePoint Online checks that your account belongs to the organization to conform your access to the file. If confirmed, the file joins the set of “shared with you” information, which makes it available to Copilot for Microsoft 365.

Testing Company-wide Sharing Links with Copilot

A simple test proves the point. Create a file that contains some information that’s unlikely to exist elsewhere within the company. In my case, I created a Word document about a fictional digital SLR camera called the Bunsen BX7. Now share the file with a company-wide link (Figure 1).

After signing into another account, open Copilot for Microsoft 365 chat and attempt to find some information about the topic in the file. Copilot should return nothing because a Bing search of the internet and a Microsoft search of company resources available to the account turn up no mention of the topic. But if you now go and use the link to open the file, Copilot can find the information and use it in its responses.

Figure 2 shows a Copilot for Microsoft 365 chat session. The first prompt about the Bunsen BX7 turns up nothing and Copilot responds with some generic text about digital cameras. The second prompt is after redemption of the company-wide sharing link. Copilot is able to find the document and use the information in its response. You can see that the shared document is listed as a source for the response.

The Desirability of Company-wide Links

The mystery of why some people can use shared documents with Copilot for Microsoft 365 is solved, but thoughts now turn to whether organizations should restrict the use of company-wide links for sensitive documents. The value of these links is that they allow anyone in the organization to access content. The downside is that it’s too easy to create and use company-wide links, which then creates the temptation for people to use these links to share confidential files wider than the organization wants the information to be known.

To guide users away from company-wide links to create sharing links for specific people instead, you can modify the SharePoint tenant configuration to make direct links the default option. Even better you can update individual site settings to disable company-wide links (anyone links are also disabled). For example, the first command sets direct links as the tenant default; the second disables company-wide links for a specific site.

Set-SPOTenant -DefaultSharingLinkType Direct

$Site = "https://office365itpros.sharepoint.com/sites/BlogsAndProjects"
Set-SPOSite -Identity $Site -DisableCompanyWideSharingLinks Disabled

If your organization uses sensitivity labels, you could also consider applying a label that restricts access to a small group of users. That way, even if someone sends a document outside the organization as an email attachment, external recipients won’t be able to open it.

The Challenge of Managing Information in an AI World

The advent of AI assistants creates new information governance challenges for Microsoft 365 tenants. Slowly but surely mechanisms are being developed to help organizations cope and manage the potential for information leakage and misuse. Some Microsoft solutions are no more than sticking plasters to allow customers to progress their Copilot deployments, but overall, the situation seems to be improving. Let’s hope that the trend continues and the current AI hype lives up to its promise.

An Overlooked Feature to Block Downloads for Channel Meeting Recordings

Given the size of Microsoft 365 and the fact that you might not use some app features very often, a good likelihood exists that you’ve never heard of some features. Or you might have forgotten that a feature exists. I experienced this feeling during the review of the Managing Videos chapter for the Office 365 for IT Pros eBook (2025 edition) when Vasil Michev, our technical editor, pointed out that the chapter didn’t mention the ChannelRecordingDownload setting for the Teams meeting policy.

This oversight on our part is inexcusable. The setting has existed for quite a while. It’s mentioned in message center notification MC222640, updated August 2, 2021, and our sole excuse is that this update occurred with a bunch of other changes intended to facilitate the migration of Stream classic to Stream on SharePoint. We’ll address the deficit here.

Updating Meeting Policies to Block Download Access for Channel Meeting Recordings

By default, the value of the ChannelRecordingDownload setting in Teams meeting policies is “Allow.” This means that the SharePoint app (app@sharepoint) saves recordings of channel meetings in the Recordings folder for the channel. For instance, recordings for meetings hosted in the General channel end up in General\Recordings while recordings for meeting in a shared channel end up in the Recordings folder. Private channels don’t have a mailbox and don’t support channel meetings. All channel members have access to the MP4 files created for recordings, including the ability to download the files.

Updating the ChannelRecordingDownload setting to block access to channel meeting recordings is not supported by the Teams admin center Instead, use the Set-CsTeamsMeetingPolicy cmdlet from the Teams PowerShell module to update Teams meeting policies. For example:

Set-CsTeamsMeetingPolicy -Identity "Allow Meeting Recording" –ChannelRecordingDownload Block

You might also need to run the Grant-CsTeamsMeetingPolicy cmdlet to assign the updated policy to the set of accounts that you want to block from downloading.

Grant-CsTeamsMeetingPolicy -PolicyName "Allow Meeting Recording" -Identity Ken.Bowers@office365itpros.com

Where Channel Meeting Recordings Go

After updating the meeting policy, SharePoint applies different rules to the storage of and access to channel meeting recordings. Remember that a channel meeting doesn’t really have an owner or organizer. The person who creates a channel meeting acts as the organizer in that they manage invitations and other meeting settings. However, they do not own the recording in the same way as they would for a personal meeting.

When saving channel meeting recording in SharePoint Online, the SharePoint app checks the meeting policy assigned to the organizer and finds that the policy blocks downloads for channel meeting recordings. Instead of saving the MP4 file in the Recordings folder, it saves the file in the Recordings\ViewOnly folder and alters permissions on the file so that only channel owners can download the recording. Everyone else, including the meeting organizer (unless they are also a channel owner) are limited to online view-only access (Figure 1).

The Microsoft documentation offers sparse information on this point and a quick internet search didn’t turn up much else. I guess everyone else missed this change.

The Can View Only Permission

In other related news, an associated change described in message center notification MC699712 (19 December 2023) might also have escaped your attention. This is the update for the share link settings for Stream videos stored in OneDrive for Business and SharePoint Online.

The change introduced the “Can view, but not download” option to the permission drop down (Figure 2) to make it easier for people to share videos that they didn’t want downloaded. It was possible to block downloads for video files beforehand, but it required additional steps to block downloads.

channel member creates a sharing link for a video file in the \ViewOnly folder, they’re restricted to sending a sharing request to the channel owners to request access. This occurs even when sharing with other channel members, even though those channel members already have access.

The Joy of Finding an Overlooked Setting

I’m not particularly worried about the downloading of my videos that other users have access to through Stream. However, I can understand that this might be a concern for others, and it’s good that Microsoft 365 includes controls to limit access in a reasonably painless manner, even if I have totally overlooked the settings for years.

Learn about using Stream on SharePoint and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Support for Office 365 Connectors Ceasing for Microsoft 365 Groups and SharePoint Online

Message center notification MC798683 (4 June 2024) announces the retirement of Microsoft 365 Groups connectors, a form of what are called Office 365 connectors. The retirement process commences on August 5, 2024, and finishes on September 5, 2024. After that time, connectors will no longer be supported within Outlook (Win32), OWA, and the new Outlook for Windows (aka Monarch).

Connectors take notifications from online data sources and post messages into a target destination. In this case, the target is the Inbox in the mailbox of the Microsoft 365 group configured with the connector. These connectors are used with Outlook groups rather than Teams. You can’t configure a connector for the other folders in a group mailbox, and you can’t configure a connector for any other type of mailbox.

Messages delivered through an Office 365 connector are limited to 28 KB and aren’t intended to be complete articles. Instead, they let users know that something has happened, give them a short snippet about the event, and provide a link to follow for more complete information. Using a connector to post messages from an RSS feed is one of the most common uses, but third-party companies like Asana and Trello have created connectors to bring snippets about information from their services to Outlook and other Microsoft 365 targets.

Microsoft recommends that organizations replace group connectors with the Power Automate app, which has its own set of connectors for different data sources, including the ability to create a cloud flow to post messages to the group mailbox. Some of the Power Automate Connectors (like Salesforce and Jira) require a Power Automate premium license.

Connectors and SharePoint Online

A further blow for Office 365 Connectors comes in message center notification MC793656 (16 May 2024), which announces the retirement of connectors from SharePoint Online webparts. Microsoft says that this is due to “limited usage.” Based on anecdotal evidence and personal experience, I can’t recall ever seeing an Office 365 connector configured with a SharePoint Online webpart.

In any case, from June 15, 2024, site owners are unable to add connectors to SharePoint Online. On August 1, 2024, they’ll be unable to update or manage existing connectors and the connectors will stop receiving inbound notifications.

Teams, Office 365 Connectors, and Workflows

Teams still supports Office 365 connectors, which are configured on a per-channel basis because the target for new notifications are channel conversations. Each notification creates a new conversation.

MC798683 points out that Teams channels also support workflows created using the workflows app (“powered by” Power Automate), and workflows recently turned up in the […] menu for Teams chats (MC683929, last updated 24 May 2024).

I shall have to pay more attention to workflows in the future. I know that the basic stuff works very well (like bringing an RSS feed into a channel). I’m more interested in finding out how to replace the incoming webhook connector, which is used in many ways to bring information from applications into Teams.

So far, my experiments with the Post to a channel when a webhook request is received workflow have not been successful. This seems to work in the same way (publish a URL to post messages to) and it’s easy to find the URL, but more difficult to get the workflow to run. I eventually managed and published my experience about posting an adaptive card to Teams.

Moving to a Single Answer for No-Code Automation

All of this seems to be part of a cunning plan to turn Microsoft 365 users into citizen developers by popularizing the use of Power Automate and the Microsoft Power Platform (Figure 1) for no-code automation wherever possible. According to Microsoft (January 2024), Power Automate has 33 million monthly active users in 350,000 organizations. My assumption is that PowerShell and the Graph are the answer for code-based automation.

It’s hard to argue against rationalization and it does make sense to settle on a single no-code automation platform for Microsoft 365, something that wasn’t viable when Office 365 Connectors appeared around 2016. As always, don’t be surprised when change happens inside Microsoft 365. Just be prepared to cope with the change.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

SharePoint Online Storage, OneDrive for Business, and SharePoint Embedded

Given the vast numbers of files created in SharePoint Online daily (Jeff Teper cited 2.3 billion in December 2023), it must be the case that the storage quotas assigned to tenants are being consumed at an alarming rate. However, I suspect that a large proportion of the files end up in OneDrive for Business and don’t impact storage so much.

These thoughts came to mind when I perused the OneDrive files report for my account to discover just how many applications now store their data in OneDrive for Business. Microsoft has truly made OneDrive for Business the personal storage system for Microsoft 365 holding anything from Office documents to Teams meeting recordings and transcripts to Whiteboards.

But coming back to storage, I often hear confusion in how Microsoft charges for SharePoint storage. Let’s review the current situation.

Three Major Storage Partitions

SharePoint Online covers three major storage partitions:

• SharePoint Online sites.
• SharePoint Embedded applications, like Loop and Designer.
• OneDrive for Business accounts.

The SharePoint Online storage quota assigned to a tenant (1 TB plus 10 GB per licensed user) covers only the first category. The storage consumed by SharePoint sites is well understood because it’s highlighted in the SharePoint admin center and is easy to report with PowerShell. A Graph usage API is also available for SharePoint Online, but currently suffers from a longstanding data issue that prevents site URLs from being shown.

Understanding the storage consumption of SharePoint Embedded applications is less clear. These applications use file storage containers (like document libraries). First-party applications like Loop charge their storage against the tenant storage quota for SharePoint Online. If the applications support SharePoint Online PowerShell or another API to report storage, it’s possible to generate a report about the storage consumed by an app.

Third-party apps built on top of SharePoint Embedded are billed separately through an Azure subscription using a pay-as-you-go metered model. Charges are accrued for API calls and the storage consumed.

OneDrive for Business Storage

The OneDrive service description says that “the default storage space for each user’s OneDrive is 1 TB. Depending on your plan and the number of licensed users, you can increase this storage up to 5 TB.” The default storage assigned to OneDrive for Business accounts is defined through the Settings section in the SharePoint Online admin center (Figure 1).

In a Microsoft 365 enterprise tenant, the storage for OneDrive can be increased to more than 5 TB. The documentation states: “Before requesting an increase you need at least five licenses that include OneDrive Plan 2, you must assign at least one license to a user, and a single user must have already filled 90% of their 5 TB storage.”

The problem here is that Microsoft stopped offering OneDrive Plan 2 in August 2023, apparently to stop offering the “unlimited storage capacity” that was once available for licenses like Office 365 E3 and E5. No official notice was given, and the plan slipped away. Office 365 and Microsoft 365 licenses no longer include a OneDrive service plan.

In any case, if you want to keep an eye on OneDrive storage consumption, it’s easy to generate a report with PowerShell.

Microsoft 365 Archive

Microsoft 365 Archive is a solution that moves SharePoint Online sites from “hot” storage (immediate access) to “cold” storage. The idea is that organizations can keep data online in a form that’s available for eDiscovery but not for user access. Archiving sites also helps to remove information from consumption by AI solutions like Copilot for Microsoft 365 to stop the results generated by AI being affected by old and possibly obsolete information.

Organizations pay for the storage consumed by archived sites through an Azure subscription. The cost per GB is much less than having to pay for regular SharePoint storage and Microsoft doesn’t charge for archive storage if the tenant has regular storage available. If the tenant runs out of regular storage, Microsoft 365 archive switches to its pay-as-you-go model.

Retention Storage

Microsoft 365 Retention Policies and Retention Labels can dictate how long content stored in SharePoint Online (including OneDrive for Business and SharePoint Embedded) is kept before it can be deleted. If files coming within the scope of retention are deleted by users, SharePoint Online keeps them in the site’s preservation hold library. Depending on the settings of retention policies and labels, it’s possible that preservation hold libraries can consume a large amount of storage (Figure 2).

Retained content can be easy to overlook. Microsoft plans to introduce intelligent versioning (originally planned for November 2023), which should help.

Summarizing SharePoint Online Storage

In summary, traditional SharePoint site storage is only one of the ways that tenant storage quota can be consumed. OneDrive for Business stores more data than ever before, but Microsoft has renounced unlimited storage. New applications and retention can consume storage unexpectedly, and Microsoft 365 Archive can help by moving data to cheaper cold storage. What could be easier to understand?

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Folder Deletion with Items in Place Makes it Easier to Clean Out Old Material

In the past, SharePoint Online used to block deletion of files with retention labels. In late 2021, Microsoft decided to make the deletion behavior consistent across SharePoint Online and OneDrive for Business by allowing deletions to occur. Files with retention labels went into the site recycle bin and progressed into the preservation hold library until their retention period expired. After that point, a timer job finds and removes the expired files.

Another welcome change to SharePoint deletion behavior is now rolling out (MC791878, 11 May 2024, Microsoft 365 roadmap item 394689) and should be fully deployed worldwide around this time. The change allows users to delete folders in document libraries that aren’t empty in sites covered by a Purview retention policy.

Removing Old Material with Folder Deletion

This doesn’t sound important, but being able to delete folders without having first to open the folder and remove all the files stored there is the way things should have worked all along. A case can be argued that allowing people to delete folders without checking what’s stored in the folder could lead to inadvertent removal of information.

However, the case is undermined by the fact that the deleted folder (and its items) goes into the recycle bin from where it can be recovered. Even if the deleted folder passes through the normal SharePoint Online recycle bin cycle, administrators can still rescue the files from the site preservation hold library. When a deleted file is restored from the recycle bin, SharePoint Online recreates the folder in the original location if necessary.

The only problem I met testing deletions is when attempting to delete an empty folder and a non-empty folder together. For some bizarre reason, SharePoint Online used the old behavior and refused to remove the non-empty folder (Figure 1). SharePoint Online was quite happy to remove the same folder if processed individually.

Very importantly, after deleting a non-empty folder, OneDrive for Business will not attempt to synchronize the deleted folder back from its offline copy.

Checking Retention Status for SharePoint Sites

If a tenant uses multiple retention policies, it can be challenging to determine which policy governs an individual site or mailbox. To help, the Data lifecycle management section of the Purview compliance portal includes a policy lookup option. At first glance, the list of retention policies shown in Figure 2 seems overwhelming, but several different types of policy are present, including some to publish retention labels to the site and auto-label policies that use trainable classifiers to label files with certain characteristics.

Because multiple policies can have a specific site within their scope, it’s important to note the purpose of each policy in the description.

Why is it Important to have easier Folder Deletion?

Some people never delete any material from SharePoint Online. At least, they don’t until they’re forced to because the tenant storage quota is nearly exceeded, and they want to avoid purchasing some expensive additional storage. The problem here is that deleting non-empty folders in sites governed by retention policies won’t help with a storage quota issue because files retained in the preservation hold library count against the quota. In some cases, the preservation hold library can occupy 40% or more of the storage used by a site.

It’s wise to keep an eye on the storage consumed by sites and then investigate the sites where storage consumption seems excessive. I use a Graph-based PowerShell script to generate a report of individual files in a document library to help understand where storage is eaten up. Obviously, after identifying unwanted files and folders, being able to remove those files more easily is a good thing.

Another reason why it’s good to clean up document libraries is that it stops Artificial Intelligence tools like Copilot for Microsoft 365 using old, obsolete, and potentially inaccurate information. Removing digital debris is something I think we’re all going to become more serious about as the AI era unfolds.

Keep up to date with developments like the AI era for Microsoft 365 by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

SharePoint Online is a Huge Success But Dark Clouds Lurk Ahead

March 27 marked the 23^rd anniversary of SharePoint Portal Server 2001, the forerunner of what we have today in SharePoint Server and SharePoint Online. The date in SharePoint history was marked by several tweets, including one from the urbane Mark Kashman, the well-known SharePoint marketeer. The tweet included an updated timeline for SharePoint (Figure 1), refreshed from an original version issued to celebrate the product’s 20^th anniversary.

I debate the accuracy of some of the dates listed in the SharePoint history. For instance, Delve and the original Office 365 Video solution are listed for 1 January 2024. My recollection is that these solutions were revealed at the first Ignite conference in May 2015 as part of the “next generation knowledge” portals promised by Microsoft at the time. As we know, marketing promises don’t always transfer into actual technology at the predicted date. Delve and Office 365 Video arrived, but the next generation knowledge portals never did. There’s also no mention of Office 365 Groups (now Microsoft 365 Groups), something that has had a huge impact on SharePoint Online.

Personal SharePoint History

Although I am probably better associated with Exchange, I have a long history with SharePoint going back to Portal Server 2001, which I had deployed at Compaq soon after its release in a nascent attempt to persuade technologists to share their knowledge with their peers. I even helped Microsoft Latin America launch SharePoint Portal Server 2001 at an event in Cancun.

SharePoint Portal Server 2001 worked well at a certain level and I took it forward into HP after the HP-Compaq merger in 2002 where it displaced a large UNIX cluster that HP Services used for document management.

As SharePoint Server developed I became exasperated at the development group’s attempts to build what seemed to be everything into a single server instead of focusing on document management. I thought that SharePoint Server 2007 was a mess and expressed that view quite strongly, something that didn’t make me many friends in Microsoft. The 2010 and 2013 release weren’t much better. The zenith of incompatability within the Office server lineup was reached when Microsoft tried to make Exchange and SharePoint work together in the ill-fated site mailbox project. Only 53 operations had to be carried out with absolute precision to make the two servers co-operate.

The Cloud Made the Difference

SharePoint achieved its full potential in the cloud. Administrators were freed from the task of looking after server farms and could concentrate on leveraging the product’s strengths in document management.

The introduction of Teams in 2017 helped enormously by providing a more user-friendly face for document storage. The growth in Teams usage to 320 million monthly active users propelled SharePoint Online usage into the stratosphere to a point where petabytes of data are added monthly.

The introduction of SharePoint Embedded as a platform for developers to build on is an interesting evolution to encourage even further usage. The Loop app is a good example of an app that uses SharePoint Embedded for storage with a UI that has no connection to what people might think of as traditional SharePoint.

Dark Clouds on the Horizon

Everything seems to be on the up in the SharePoint world, but I see some clouds on the horizon. The fact that Microsoft has been forced to introduce Restricted SharePoint Search to allow customers to progress Copilot for Microsoft 365 projects is an admission of failure in information governance.

Restricting users to searching 100 curated sites might seem like a good answer, but it admits that the tens of thousands of sites created by Teams are an unmanageable tangle. Inside those sites obsolete, misleading, and erroneous information might lurk in documents ready to corrupt the results generated by Copilot. It’s perhaps the greatest challenge faced by those considering Copilot deployments.

Digital debris is a big black cloud over SharePoint. Copilot is an accelerant that highlights the issue, but Microsoft 365 customers without Copilot should also focus on gaining control over the information held in SharePoint. This a wake-up call for tenants to ask questions about how they control the creation of sites (with or without Teams), how documents are stored and managed, how they use retention policies to remove old information, and so on. The issue won’t go away. It grows worse every day as users add petabytes of documents to SharePoint Online and OneDrive for Business.

The Microsoft 365 conference takes place in a month’s time. I’m sure that the SharePoint community will applaud the achievements and popularity of the platform. I hope that they take some time to address the information governance issue and that the current threat to continued success in SharePoint history abates.

Keep up to date with developments in SharePoint Online by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

No Evidence that Microsoft Cares as Pnp.PowerShell Fills the Gap

I last wrote about the state of SharePoint Online PowerShelll in 2020. At the time, I focused on Microsoft’s PowerShell module (Microsoft.Online.SharePoint.PowerShell), which is downloadable from the PowerShell Gallery. Based on the gallery statistics, the module is popular as each version attracts hundreds of thousands of downloads. Microsoft also updates the module monthly. On the surface, everything seems wonderful, and the module is in rude health.

If only this was true, but it’s not. It’s true that Microsoft updates the module to add tenant settings to control new features as they appear (like request files), but there doesn’t seem to be a coordinated plan about how Microsoft plans to support management of SharePoint Online through PowerShell.

Lack of Progress with Graph API

In 2022, Microsoft released the initial (beta) version of a Graph API to access and update SharePoint tenant settings. Apart from supporting the SharePoint settings API through the production (V1.0) endpoint, Microsoft doesn’t seem to have made much progress with the API since 2020. At least, the same set of tenant settings are visible two years on.

On the upside, SharePoint Online tenant settings are accessible using the Microsoft Graph PowerShell SDK. For instance, the Get-MgAdminSharepointSetting cmdlet reports the supported settings:

Connect-MgGraph -NoWelcome -Scopes SharePointTenantSettings.Read.All

Get-MgAdminSharepointSetting | Format-List

AllowedDomainGuidsForSyncApp                    : {}
AvailableManagedPathsForSiteCreation            : {/sites/, /teams/}
DeletedUserPersonalSiteRetentionPeriodInDays    : 60
ExcludedFileExtensionsForSyncApp                : {*.rar, *.zip}
Id                                              :
IdleSessionSignOut                              : Microsoft.Graph.PowerShell.Models.MicrosoftGraphIdleSessionSignOut
ImageTaggingOption                              : enhanced
IsCommentingOnSitePagesEnabled                  : True
IsFileActivityNotificationEnabled               : True
IsLegacyAuthProtocolsEnabled                    : True
IsLoopEnabled                                   : True
IsMacSyncAppEnabled                             : True
IsRequireAcceptingUserToMatchInvitedUserEnabled : True
IsResharingByExternalUsersEnabled               : False
IsSharePointMobileNotificationEnabled           : True
IsSharePointNewsfeedEnabled                     : False
IsSiteCreationEnabled                           : True
IsSiteCreationUiEnabled                         : True
IsSitePagesCreationEnabled                      : True
IsSitesStorageLimitAutomatic                    : True
IsSyncButtonHiddenOnPersonalSite                : False
IsUnmanagedSyncAppForTenantRestricted           : False
PersonalSiteDefaultStorageLimitInMb             : 5242880
SharingAllowedDomainList                        : {Microsoft.com…}
SharingBlockedDomainList                        : {Gmail.com}
SharingCapability                               : externalUserAndGuestSharing
SharingDomainRestrictionMode                    : none
SiteCreationDefaultManagedPath                  : /sites/
SiteCreationDefaultStorageLimitInMb             : 26214400
TenantDefaultTimezone                           : (UTC) Dublin, Edinburgh, Lisbon, London
AdditionalProperties                            : {[@odata.context, https://graph.microsoft.com/v1.0/$metadata#admin/sharepoint/settings/$entity]}

And the Update-MgAdminSharepointSetting cmdlet updates a setting:

$Body = @{}
$Body.Add("IsResharingByExternalUsersEnabled",$true)
Update-MgAdminSharepointSetting -BodyParameter $Body

SharePoint Online PowerShell is Windows PowerShell

Getting back to the PowerShell module, Microsoft has not updated it to support PowerShell 7. This might not be a problem if you always use Windows, but it does limit platform coverage. Attempting to load and use the module with PowerShell 7 usually fails, especially when multifactor authentication is involved.

The Community Approach to SharePoint Online PowerShell

This brings me to the Pnp.PowerShell module, also available from the PowerShell gallery. Based on the download numbers, Pnp.PowerShell seems to be four to five times more popular than the official Microsoft SharePoint Online module. This state is probably due to:

• Development driven by a committed set of community advocates.
• Wider coverage of SharePoint commands. The module spans over 650 cmdlets while the Microsoft.Online.SharePoint.PowerShell module has 250. Part of the reason for the dramatic difference in cmdlet count is that Pnp.PowerShell dips into other Microsoft 365 workloads associated with SharePoint Online like Teams, Planner, Flow, and Entra ID. Another is that Pnp.PowerShell includes cmdlets to create objects like files in SharePoint Online document libraries (here’s an example) that aren’t within the scope of the administrator-centric SharePoint module
• Frequent updates to introduce new features and support for changes within SharePoint Online.
• Solid documentation.

Because Pnp.PowerShell is a community effort rather than something produced by Microsoft, some organizations are reluctant to use it. They fear that support for bug fixes will be limited or that some catastrophic bug will creep in due to a lack of testing. My experience is that the community developers are very responsive and do better testing than many Microsoft development groups (an example being the recent bugs afflicting the Microsoft Graph PowerShell SDK). There’s no reason to avoid using Pnp.PowerShell, subject to the normal requirements to test new versions and ensure that every cmdlet does what you expect.

Moving Forward with SharePoint Online PowerShell

Pnp.PowerShell wins the contest for popularity and coverage when it comes to PowerShell access to SharePoint Online. The official module appears stuck in time, and I know of no advocate within Microsoft who wants to bring it forward. The Graph tenant settings API started but hasn’t done much since 2022. Perhaps Microsoft should simply take Pnp.PowerShell over? Or maybe not, because then we might have three modules in a static state instead of two.

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Restricted SharePoint Search Confines Copilot for Microsoft 365 Access to Curated Sites and User Content

The lights are obviously burning late in SharePoint engineering as Microsoft figures out new methods to help customers avoid inadvertent disclosure of confidential information through text generated by Copilot for Microsoft 365. Last month, we discussed how to exclude SharePoint Online sites and document libraries from search results to block Copilot access. Now, Microsoft has created Restricted SharePoint Search to deliver a more elegant (but possibly flawed) solution to allow organizations to control the sites accessible to Copilot.

Tenants with Copilot for Microsoft 365 licenses can enable Restricted SharePoint Search “from April onward.” By default, Restricted SharePoint Search is disabled and administrators will have to run some PowerShell commands to enable it for a tenant.

What is Restricted SharePoint Search?

Essentially, Restricted SharePoint Search disables enterprise-wide search. Instead, tenants can select up to 100 SharePoint Online sites that users can access (subject to the access defined for the sites). In this context, “users” includes Copilot for Microsoft 365 because when Copilot is active, it always operates as the signed-in user.

The restriction placed on enterprise search is pretty severe. Given the profusion of Microsoft 365 Groups and Teams, each of which has their own SharePoint Online site (and possibly several sub-sites for shared and private channels), limiting search to a curated list of 100 sites excludes most sites in anything but small tenants. Even my own small tenant supports over 400 sites.

Easing the Restrictions to Accommodate User Data

Microsoft will no doubt argue that the impact of limiting search to 100 sites is ameliorated by allowing users to search content from:

• Frequently visited SharePoint Online sites. However, Microsoft has not defined how many sites are in this category and how users can add sites to the list. For instance, does marking a site as a favorite put it on the list?
• Files in OneDrive for Business accounts that they have access to. This includes the user’s own OneDrive account, so they have full access to all their personal documents and other information stored in OneDrive.
• Files that are directly shared with users.
• Files that users create, edit, or view. In other words, if you touch a file stored in SharePoint Online, Copilot for Microsoft 365 can find and use that content.

I don’t know how Microsoft determined that 100 was a good number for the curated sites list. Determining what sites go onto the list and what sites are excluded will be an interesting exercise for many organizations. It seems like the intention is for tenants to include important corporate sites that everyone needs access to, like those holding HR information or details of released products and public documentation while relying on the frequently visited sites lists to deliver user-specific search results. It would be interesting to know how Microsoft uses Restricted SharePoint Search and if so, how they selected the 100 sites.

Restricted SharePoint Search does not affect how Microsoft Purview solutions like eDiscovery work. One way of thinking about the restriction is that it’s a form of trimming similar to the security trimming that Search already does to make sure that users only ever see sites and files in search results that they are entitled to access. This trimming further limits results to the 100 curated files plus the user’s OneDrive for Business account and files shared with them.

The Impact of Restricted SharePoint Search

It could be that this scheme will work well, but as Microsoft points out, Restricted SharePoint Search “limits the content Copilot can search and reference when responding” and “may impact its ability to provide accurate and comprehensive responses to prompts.”

Microsoft says that the new solution will help customers review and audit site permissions while continuing to deploy Copilot for Microsoft 365. A cynic might say that Restricted SharePoint Search is a cobbled together patch rushed out to assuage the concerns of customers who have heard about potential data disclosure problems and slowed the planning process for Copilot. It’s absolutely the right thing for Microsoft to address those concerns, but Restricted SharePoint Search seems like a sticking plaster that’s been applied until Microsoft can come up with a more flexible long-term solution. I guess we’ll know more when the software reaches customers in April and can assess just how well the 100 site limit works.

Longstanding Service Issue Retrieving SharePoint Usage Data

The Microsoft 365 ecosystem is so large that it’s hard to keep track of everything that changes that show up in different workloads. We’ve always known about the difficulties of tracking new features, deprecations, and other issues, but sometimes it takes a user to report something to focus on a specific problem.

An example is when a reader noted that the Graph-based script to report the storage quota used by SharePoint sites no longer included site URLs in the output (Figure 1). The original script (from 2020) used a registered Entra ID app to authenticate and use the Graph getSharePointSiteUsageDetail API to fetch site detail data.

Problems in the Graph APIs Accessing SharePoint Usage Data

When I investigated the problem, I decided to update the script code to use the Microsoft Graph PowerShell SDK instead. The update did nothing to retrieve the missing data. This isn’t surprising because the problem lies in the Graph API rather than the way the API is called.

The Microsoft 365 admin center uses the same Graph API for its SharePoint site usage report and the same problem of no site URL data is seen there (Figure 2).

Even worse, the SharePoint site activity report in the Microsoft 365 admin center displays no data (Figure 3).

This problem is because the getSharePointActivityUserDetail API returns no data whatsoever. Here’s an example of using the API in PowerShell in an attempt to retrieve SharePoint Online user activity for the last 180 days. The retrieved data should end up in the SPOUserDetail.CSV file.

$Uri = "https://graph.microsoft.com/v1.0/reports/getSharePointActivityUserDetail(period='D180')"
Invoke-MgGraphRequest -Uri $Uri -Method GET -OutputFilePath SPOUserDatail.CSV

However, the output file is perfectly empty apart from the column headers (Figure 4).

The same approach works perfectly with other usage data. For instance, this query works nicely to fetch Exchange Online usage data:

$Uri = "https://graph.microsoft.com/v1.0/reports/getEmailActivityUserDetail(period='D180')"
Invoke-MgGraphRequest -Uri $Uri -Method GET -OutputFilePath $EmailUsage.CSV

A Known Service Issue with SharePoint Usage Data

It’s not surprising that an API should have a problem. The APIs haven’t changed recently, so the root cause is more likely due to a change in the SharePoint Online back end. This feeling is reinforced by service health report SP676147 filed on 21 September 2023 (last updated 9 February 2024) that blithely says that “SharePoint and OneDrive URLs may not be displayed in some usage reports.”

Microsoft goes on to note that:

“We’re continuing our work through the validation of multiple potential mitigation strategies to display the URLs of the affected site usage reports. Due to the complexity of the scenarios involved we anticipate this may take additional time.”

The next update for the service health announcement is due on 1 March 2024. What I’m struggling with is that the usage reports included site URLs without any difficulty for years. Why it should suddenly become an issue is inexplicable. And taking over six months to find a solution is even more so.

Microsoft suggests that developers use the Graph Sites API to retrieve the site URL. For example:

$Uri = ("https://graph.microsoft.com/v1.0/sites/{0}" -f $Site.'Site Id')
$SiteData = Invoke-MgGraphRequest -Uri $Uri -Method GET

This works, but only when using an application permission. Using delegated permissions restricts access to sites that the signed-in user is a member of.

SharePoint PowerShell Still Works

Fortunately, it’s possible to get the site storage quota information using the SharePoint Online management PowerShell module. The Graph APIs read from a usage data warehouse that’s populated using background processes. The data is always at least two days old, but it’s much faster to access than using PowerShell to check the storage for each site. But needs must, and at least the old method still works.

I admit forgetting about the service health announcement, perhaps because it’s been ongoing for so long. I’m genuinely surprised that Microsoft is still working on something that seems so innocuous. And I’m even more surprised that customers aren’t making more of a fuss because the URL is the fundamental way to identify a SharePoint site.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Handling SharePoint Online Document Mismatch Notifications

I first wrote about SharePoint Online document mismatch notifications soon after their introduction in 2020. SharePoint sends these messages when users upload (or create) a document in a site that has a higher-priority sensitivity label than the label assigned to the site. The message (Figure 1) is intended to alert the author and site administrators about a potential issue caused when sensitive information is stored in a site intended for lower-level content.

Although SharePoint Online creates audit events when document mismatches occur, the audit events don’t contain details of the author who provoked the mismatch. To find that information, you need to correlate the notification with other audit records. This method works and it’s a good way to keep an eye on trends.

Suppressing Document Mismatch Notifications with a Mail Flow Rule

So far, so good. However, some organizations dislike document mismatch notifications because they cause users to call the help desk to ask what’s happening and what they should do next. The easy answer is to say that this is a simple matter of lack of user training. Before an organization implements sensitivity labels, they should educate users about the meaning and usage of the labels published to accounts. Just because a user doesn’t understand the difference between “General access” and ”Confidential” is not a reason to suppress informational emails. Another point I’ve heard made is that users tend to ignore system-generated messages, especially if they don’t understand why they received the email.

Another way of thinking about the situation is that if mismatches occur, fast intervention should occur to coach users about how to label information correctly. But to be able to intervene, the help desk (or another body) must know about the mismatch. It’s possible to scan for audit events daily (or more frequently) to find the data. Another solution is to use a mail flow rule to redirect mismatch notifications to an email address where those who help users to understand how to handle sensitive information can learn about issues as they happen.

The mail flow rule is straightforward (Figure 2). The rule looks for messages originating from a specific sender, which is the email address used for the document mismatch notifications (see discussion below). Only messages that match the subject of the notifications are diverted to one or more selected recipients. The targets can be individual recipients or groups. They do not have to be known Exchange Online mail-enabled objects.

Once the rule is enabled, the Exchange Online transport service intercepts document mismatch notifications and redirects the emails to the nominated recipient.

Domain-Specific Address for Service Emails

In both the figures, you’ll notice that a custom email address is used for the sender of the document mismatch notifications. This is a feature known as bring your own domain address for service messages. In other words, you can replace the default addresses used by Microsoft (like noreply@sharepoint.com) with an address that makes sense to those who receive notifications and other service messages. The address must be functional because users might respond to service messages requesting more information. I use a shared mailbox for this purpose.

According to message center notification MC705761 (10 January 2024, Microsoft 365 roadmap item 375694), Teams will soon give tenant administrators the option to send information messages from a selected domain. The update is coming to standard release tenants in mid-February 2024.

No Right Answer

There’s no right or wrong answer for how to deal with document mismatch notifications. It all depends on the individual circumstances within a tenant. Some users are very conscious about information protection and understand the most appropriate label to apply in any condition. Others are unsure. Coaching helps, but sometimes a little extra help doesn’t go amiss.

Learn about using SharePoint Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Track Technology Coverage with Custom Document Properties in Word Documents

Since mid-2018, I have published nearly 1,500 articles on this blog and perhaps another 500 articles on other sites, notably Practical365.com, which is where I publish long-form articles (more than 1,200 words).

I’m often asked how I track what I write to make sure that I cover the full breadth of Microsoft 365 instead of remaining focused on just one or two applications. Given my history of writing about Exchange Server, it would be easy to stay engrossed in Exchange Online. Personal bias and interest results in some applications receiving more attention than others from any author, but I do try to cover a reasonable mix of topics drawn from across Microsoft 365. Apart from anything else, writing about a new topic is a great way to become acquainted with that technology.

Using SharePoint Metadata

Keeping track of articles isn’t difficult. The source text for articles are in Word files stored in a document library in a SharePoint Online site. The only change to “normal” document processing is some additional columns to capture metadata that helps track the areas of technology covered by an article and the publication where the article appears.

The custom columns are defined in library settings (Figure 1). The Publication column stores the name of the web site hosting the article and the Published date column captures when the article is online. The Technology column stores the major technical focus (like SharePoint Online), while the Technology sub-area property captures an additional level of detail, if needed.

Newly updated columns are available immediately to the SharePoint web app, and that’s where most updating occurs using the details panel (Figure 2). Usually I wait until the text is finalized and published before completing the details to make sure that the data is accurate.

You can see that I use retention labels and sensitivity labels in the document library. The document library has a default sensitivity label and a default retention label to make sure that everything is protected.

Updating SharePoint Metadata in Word

It’s also possible to update SharePoint metadata in the Word (Win32) app (not the web version). After adding or modifying a column, the Word application must download the new metadata from the document library. This process can take a little time and it’s often easiest to force a download by exiting the document and reopening it.

Access to document properties is through the Quick Parts option of the Insert menu. Click Quick Parts and choose Document Property to see the list of available properties, including the custom columns added to the library (Figure 3). The majority of the properties shown are defined in Word and have no connection to SharePoint Online.

While possible, I don’t suggest that you update custom document columns using this method unless you want the text stored in the column to be part of the document content. The problem is that Word treats the property like any other quick part (such as a field used for figure or table numbering) and includes it in its text. Figure 4 shows updating a choice property in Word.

Because it’s a quick part, Word supports updating of the property at any time. Following an update, SharePoint learns about the new value and updates its metadata.

Generating Output from the Custom Metadata

The advantage of adding custom columns to a SharePoint Online document library and populating the columns with data is to create the basis for reporting. The easiest way is to use the Export to Excel feature to create a worksheet or CSV file. In both cases, the custom columns are in the exported data (Figure 5).

The file can be analyzed or imported into Power BI for visualization, or even imported into PowerShell for analysis and reporting there. I do that, but I suspect that most will be happier to remain in Excel.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Sensitive by Default Blocks External Access Pending DLP Scanning

The ability to mark documents as “sensitive by default” achieved general availability in July 2020. Despite covering the topic in the DLP chapter of the Office 365 for IT Pros eBook, I never paid the feature much attention because sensitivity labels are often a better way to protect confidential material.

Not every tenant deploys sensitivity labels. Sometimes this is because they want to avoid the complications that can come when dealing with encrypted information, such as how to move encrypted documents between tenants during a migration. And sometimes it’s because the work to prepare to deploy and manage sensitivity labels is incomplete and pending completion. These are the kind of circumstances when the sensitive by default control is useful for tenants that have the necessary Office 365 E3 (or above) licenses to use Data Loss Prevention (DLP) policies with SharePoint Online.

The idea is simple. SharePoint Online uses background processes to implement the instructions in DLP to detect sensitive information in documents and take whatever action the policy settings dictate, such as to block sharing. Because DLP processing does not happen immediately for new files uploaded to to SharePoint Online and OneDrive for Business libraries, a short period exists when it’s possible for users to share sensitive data outside the tenant and inadvertently leak data. The sensitive by default control stops this happening by forcing SharePoint Online to consider all files as sensitive until DLP processes their content.

In effect, this means that SharePoint Online blocks external access to documents until DLP scans the contents. If external users, including guest members of a team, attempt to access a document before DLP scans its content, they see a page to tell them that scanning is in progress (Figure 1). After a few minutes, the scan should complete and access is possible.

Implementing Sensitive by Default

To implement the Sensitive by default control, you:

• Implement at least one DLP policy to scan the SharePoint Online sites that store information intended for external access.
• Run the Set-SPOTenant cmdlet in the SharePoint Online PowerShell module to block access to new files. It can take up to 15 minutes before the change is effective. The block applies to all sites in the tenant and you can’t exclude sites from its effect.

Here’s the command to implement the sensitive by default control:

Set-SPOTenant –MarkNewFilesSensitiveByDefault BlockExternalSharing

With the block in place, users can still share documents externally (if not blocked by the tenant’s sharing settings). However, external people with a sharing link cannot access the content until the document is scanned by a DLP policy.

To revert the block, run Set-SPOTenant to allow sharing without waiting for DLP processing:

Set-SPOTenant -MarkNewFilesSensitiveByDefault AllowExternalSharing

DLP Processing for Sensitive by Default

Any DLP policy that has a “contents contains” condition to process information in SharePoint Online sites can perform the check and release the block. Normally, DLP scanning either passes the document for external access (because DLP doesn’t detect a policy violation) or blocks it (because DLP detects some content that violates the policy if shared externally).

The Microsoft documentation for the feature discusses creating a form of “catch-all” DLP policy to cover all SharePoint Online sites and OneDrive for Business accounts in a tenant. The policy contains a rule to check new content for some arbitrary value. As shown in Figure 2, I use a check for the blood test sensitive information type.

It doesn’t matter that DLP is unlikely to detect this data in my tenant. Apart from that, the DLP policy doesn’t perform any action or notify anyone if it matches content. The sole purpose of the policy is to make sure that DLP processes every file uploaded to SharePoint Online and OneDrive for Business. Other DLP policies handle any problems lurking in documents.

Sensitive by Default and Sensitivity Labels

Applying the sensitive by default control is an effective way to stop external sharing from SharePoint Online and OneDrive for Business. However, it’s a broad-brush policy that covers the entire tenant. Using sensitivity labels to restrict access to documents containing important information might be a better approach, especially when auto-label policies are used to find and apply labels to documents at rest. The two approaches are not mutually exclusive and it’s a good idea to use sensitivity labels to control access to an organization’s most confidential information, including documents shared with external guests in Teams.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Preserve Copies of Cloudy Attachments for eDiscovery

A “cloudy attachment” is the term used when people send a link to a file stored in SharePoint Online or OneDrive for Business instead of attaching individual copies of the file with messages generated by Outlook, Teams, or Viva Engage. The idea is that recipients can work with the original content stored in SharePoint or OneDrive rather than making changes to their personal copies and then attempting to reconcile updates. Cloudy attachments can also result in the output generated through interactions with Microsoft 365 Copilot.

Exchange Online users have had the ability to send cloudy attachments for many years. However, it takes time for people to change the habits of a working lifetime, and it’s only relatively recently that I see more cloudy attachments in use, including when sharing documents across tenants. The passing of time and better internet access allows us to catch up with Microsoft’s vision. Attachments sent through Teams has always been based on sharing, so the percentage of cloudy attachments found in Teams is much higher than email.

The eDiscovery Problem Caused by Cloudy Attachments

Efficient as cloudy attachments are, they pose a problem for eDiscovery searches. The results for Purview eDiscovery standard cases and content searches include links for cloudy attachment sent in messages. However, they do not include the actual content of the linked file. eDiscovery investigators can follow the link to find the content of the attached file, but this can be an onerous task when search results include many messages with attachment links.

Microsoft’s Solution: Auto-label Copies of Cloudy Attachments

Microsoft solution depends on two factors. First, a method is needed to mark cloudy attachments so that they can be easily found by searches. Second, the marked files must be integrated into search results. This is what Microsoft does through a combination of auto-label retention policies and Purview eDiscovery (premium). You’ll need Office 365 E5 licenses to take advantage of their solution.

The first step is to deploy an auto-labeling policy using the content condition “apply labels to cloud attachments and links shared in Exchange, Teams, Viva Engage and Copilot” (Figure 1). The auto-label policy won’t be effective immediately because it must be deployed to all the sites covered with the policy. This can take up to a week, depending on the number of sites to cover.

The auto-label policy forces SharePoint Online to capture copies of files shared through messaging. When active, the a background job monitors for cloudy attachments that come within the scope of the policy (attachments selected and sent from the SharePoint Online sites and OneDrive for Business accounts specified in the policy). When the policy detects an in-policy cloudy attachment, it creates a copy of the file and stores the file in the SharedVersions folder of the Preservation Hold library for the host site (Figure 2).

Because auto-labeling happens using a background timer job, it can take up to an hour before the copy of a cloudy attachment is captured and labeled.

If someone modifies a file after sharing, a new version is captured in the Preservation Hold library. This step ensures that it’s possible for eDiscovery to find an attachment in the exact state at the time it was shared. Given that document content often changes as people work on it, knowing what a recipient sees in an attachment is a critical part of the eDiscovery process.

Copies of cloudy attachments labelled by auto-label policies remain in the Preservation Hold library until their retention period lapses. At that time, the normal method of processing the retention action occurs. Depending on how many cloudy attachments an organization generates, the preservation of captured copies might have a significant impact on the consumption of SharePoint Online storage.

To ensure that the current version of the original shared file is preserved, any files moved or deleted in the locations within the scope of the auto-label policy are automatically copied to the Preservation Hold library. These are temporary copies kept for one day to allow auto-label processing to happen and then removed. This form of temporary retention is unique to files within the scope of auto-labeling policies for cloudy attachments and is a simple safeguard to preserve all the copies of these files that might be needed for eDiscovery.

Unlike other auto-labeling policies which process data at rest to apply retention labels to content that already exists, auto-labeling of cloudy attachments is not retrospective. The only attachments that are captured and retained are those sent once the policy is in force.

Retention Labels Stamped on Captured Copies of Cloudy Attachments

The auto-label policy stamps the captured copies of cloudy attachments with the retention label defined in the policy. Because they have no access to the Preservation Hold library, users who send the messages with the cloudy attachments are unaware that the captured copies have retention labels.

To avoid problems with attachments that are shared multiple times, Microsoft recommends that the retention label chosen for the auto-label policy starts its retention period from the time when the policy applies the label to the copy of the shared attachment. The retention label applied by the policy does not have to be published to users or locations. In fact, it’s probably a good idea to create a retention label specially created for use with cloudy attachment auto-labeling policies.

Discovering Information About Captured Cloudy Attachments

As you can see, the captured copies have obfuscated file names. To discover more about a file, use the Version history option. As you can see in Figure 3, the name of the document and its original location are clearly visible, as is the date when the policy captured the copy.

More information about a captured attachment is available by checking its compliance details. In Figure 4 we can see details of the retention label assigned to the file. If the auto-label policy works, the label should be the one defined in the policy.

The existence of captured attachments means that it now becomes possible to retrieve copies of the attachments during eDiscovery operations. A preview feature in the workflow for Purview eDiscovery premium cases leverage this capability to collect copies cloudy attachments in the state they were shared. Investigators can review content either at the time when a file was shared or its current state.

Good for eDiscovery People

Obviously this is a feature that is of interest to those working with eDiscovery cases, specifically with access to Purview eDiscovery (premium). For all that, it’s an interesting example of how a change made in applications (cloudy attachments) creates issues down the line for other technology. More information about the retention of cloud attachments and how auto-label policies work is available in Microsoft documentation.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Embedded Latest Idea to Exploit SharePoint Storage

Microsoft executives love to cite statistics to prove the power of technology during conference keynotes. Thus we heard from SharePoint overload Jeff Teper that Microsoft 365 users currently generate new 2.3 billion documents for SharePoint Online daily. Teper delivered the statistic with great gusto during his keynote at the European SharePoint Conference (ESPC 23) in Amsterdam last week, and it, along with the announcement about the public preview for SharePoint Embedded, got me thinking about the role of SharePoint Online within the Microsoft 365 ecosystem.

Over two billion documents created daily seems like a lot of content. Given that Office 365 has over 400 million paid seats and (possibly more importantly) Teams has more than 320 million monthly active users, it works out at about six new pieces of content per user daily. Microsoft didn’t say if the figure was for business days, so we’ll assume it is a daily average. Given that people work every day, that’s a reasonable assumption.

Six new pieces of content is a lot for some people and not for others. It’s not just Office documents because SharePoint Online has become the storage engine for many Microsoft 365 applications. Of course, classic SharePoint and OneDrive for Business usage drives a lot f consumption, but then there’s Teams meeting recordings, whiteboards, Stream videos, files migrated from on-premises and other platforms, Microsoft Lists, Loop workspaces, and all the other information introduced into SharePoint through apps and connectors.

As always, Microsoft provides very little context for a headline figure used in conference sessions, so we assume that very item created in SharePoint Online storage counts towards the total. The actual number of Office documents created in SharePoint Online by humans daily is probably much smaller than 2.3 billion, but there’s no doubt that more content is created than ever before.

Apps and Technology Improve the SharePoint Experience

I believe that Teams exerts huge influence over the demand for SharePoint Online storage. Soon after its introduction, I said that Teams delivers a human interface for SharePoint (and OneDrive for Business). Sure, people use the SharePoint browser client to work with documents, but it’s much easier to use Teams and store files in SharePoint sites without realizing that’s what’s happening.

The OneDrive sync client is an important factor here too. Microsoft’s first sync client was awful, but the current generation works very well and makes it easy to keep local copies of files synchronized with the server. Features like auto save build on OneDrive synchronization and make sure that work done in Office documents is now hardly ever lost, even after catastrophic PC failures.

SharePoint Embedded, Apps, and Costs

To return to the original announcement, SharePoint Embedded is a renaming of Syntex Repository Services, revealed earlier this year as the storage location for Loop workspaces. SharePoint Embedded is also used by Microsoft Designer (Figure 1). The big news is that Microsoft is making SharePoint Embedded available to customers and ISVs to store their application data.

Holding data in SharePoint Embedded is an attractive notion because it allows applications to take advantage of Microsoft security and compliance features and the availability delivered by the Microsoft Cloud datacenter network. The downside is ceding some level of control over applications to Microsoft and an uncertain cost model.

Computing services must be paid for and SharePoint Embedded uses the pay-as-you-go model for storage and the Graph API transactions executed by applications to interact with SharePoint. Cost and licensing details are available online but the raw detail must be converted into actual costs based on an application profile before developers understand how much they need to charge to turn a profit. Experience will help people estimate costs more accurately over time. For now, Microsoft outlines the costs listed in Table 1. These are for the preview of SharePoint Embedded and may change when the service is generally available.

I can find no detailed information about what class A and class B transactions mean, unless these refer to the same definitions as used for the Teams Graph APIs where class A means transactions that perform a security or compliance function and class B do not.

Interpreting the potential costs for an application based on SharePoint Embedded will require some testing and analysis of the information reported for the Azure subscription used to pay.

More Business Opportunities

The large and increasing size of the Microsoft 365 installed base creates significant business opportunities for Microsoft to upsell services to its customers. SharePoint Premium (introduced at Ignite 2023 and tagged as the “next evolution for Syntex“) includes advanced content management and some new services, like SharePoint Signature (apply eSignatures to documents). Microsoft 365 Backup and Microsoft 365 Archive are other monetization opportunities, both of which use the pay-as-you-go model.

SharePoint Embedded is another way for Microsoft to create value from its Microsoft 365 infrastructure. It seems like a good idea. Time will tell if it is.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Understand the Impact Loop Workspaces Have on SharePoint Online Quota

Message center notification MC678308 (updated 2 November 2023) explains that the storage consumed by Loop workspaces (created with the Loop app rather than Loop components in Teams and Outlook) will count against tenant storage quotas. During the preview, Microsoft allowed people to use the Loop app without a license and create as many workspaces as they liked. The only limitation was on the size of an individual workspace, which was capped at 5 GB. Workspace data is held in a special form of SharePoint storage called Syntex repository services and Microsoft didn’t limit the storage occupied by the workspaces.

All good things come to an end. As the Loop app approaches the end of its public preview stage and moves toward general availability (I expect an announcement at the Ignite conference), Microsoft has revealed its hand with respect to licensing and storage. Only people with certain Microsoft 365 product licenses will be able to create new workspaces

Loop Counts Against Storage Now

According to MC678308, Microsoft will start counting Loop workspaces against tenant storage quotas between late October and late November 2023. When the change goes into effect for a tenant, the maximum size of a workspace increases from 5 GB to 1 TB.

The exact impact on a tenant is hard to know unless you use the Get-SPOContainer cmdlet in the SharePoint Online management module to fetch details of each tenant. For example, this command fetches details of existing workspaces:

[array]$LoopWorkspaces = Get-SPOContainer -OwningApplicationID a187e399-0c36-4b98-8f04-1edc167a0996
If (!($LoopWorkspaces)) {
    Write-Host "Can't get Loop workspaces - exiting"; break
}

The details reported by Get-SPOContainer miss some important information. For instance, while the creation date for a workspace is available, the last updated date is not, nor is detail about the person who last updated the workspace. Understanding the date when a workspace was last changed is critical to knowing if a workspace is in active use.

Reporting Loop Workspaces

This code generates a report with details of the storage used by each workspace and whether the workspace owners have one of the four licenses required to create new Loop workspaces:

$Report = [System.Collections.Generic.List[Object]]::new()
$TotalBytes = 0; $LicenseOK = 0; $i = 0
ForEach ($LoopSpace in $LoopWorkspaces) {
    $i++
    Write-Output ("Analyzing workspace {0} {1}/{2}" -f $LoopSpace.ContainerId, $i, $LoopWorkspaces.count)
    # Get detail of the workspace
    $LoopSpaceDetails =  Get-SPOContainer -OwningApplicationID a187e399-0c36-4b98-8f04-1edc167a0996 -Identity $LoopSpace.ContainerId
    # Get detail about the owner
    [array]$Owners = $LoopSpaceDetails.Owners
    ForEach ($Owner in $Owners) {
        $LicenseFound = $Null; $LoopLicenseStatus = "Unlicensed";  $LicenseName = $Null
        # Find if the Loop service plan is successfully provisioned for the account
        [array]$UserLicenseData = Get-MgUserLicenseDetail -UserId $Owner
        $LoopLicense = $UserLicenseData | Select-Object -ExpandProperty ServicePlans | `
             Where-Object {$_.ServicePlanId -eq $LoopServicePlan} | Select-Object -ExpandProperty ProvisioningStatus
        If ($LoopLicense -eq 'Success') {
            $LicenseOK++
            $LoopLicenseStatus = "OK"
        }
        # Find what SKU the Loop service plan belongs to
        $User = Get-MgUser -UserId $Owner -Property Id, displayName, department, UserPrincipalName
        [array]$SKUs = $UserLicenseData.SkuId
        ForEach ($Sku in $Skus) {
            $LicenseFound = $LoopValidLicenses[$Sku]
            If ($LicenseFound) {
                $LicenseName = $LicenseFound
            }
        }
    }
    [array]$Members = $Null
    [array]$Managers = $LoopSpaceDetails.Managers
    ForEach ($Manager in $Managers) {
        $Member = Get-MgUser -UserId $Manager
        $Members += $Member.DisplayName
    }

    $StorageUsed = "{0:N2}" -f ($LoopSpaceDetails.StorageUsedInBytes/1MB)
    $TotalBytes = $TotalBytes + $LoopSpaceDetails.StorageUsedInBytes

    $ReportLine = [PSCustomObject]@{
        ContainerId    = $LoopSpace.ContainerId
        App            = $LoopSpaceDetails.OwningApplicationName
        Name           = $LoopSpace.ContainerName
        Description    = $LoopSpace.Description
        Owner          = $User.DisplayName
        UPN            = $User.UserPrincipalName
        License        = $LoopLicenseStatus
        Product        = $LicenseName
        Members        = ($Members -Join ", ")
        Created        = $LoopSpaceDetails.CreatedOn
        SiteURL        = $LoopSpaceDetails.ContainerSiteUrl
        "Storage (MB)" = $StorageUsed
    }
    $Report.Add($ReportLine)
}

Figure 1 shows an extract of the information captured by the script. You can see that the James Ryan account is deemed to be unlicensed. This is because the account doesn’t hold a product licenses containing the Microsoft Loop service plan. Also note that new users all receive the Ideas workspace to help get them started with the Loop app. The workspace isn’t large (0.11 MB), but it’s a bit cheeky for Microsoft to charge for it.

Checking individual workspace containers is not a fast operation. The script can be sped up by removing the Get-MgUser commands used to fetch details about the licenses possessed by workspace owners.

You can download the complete script from GitHub. Remember that the intention of the script is to illustrate a principal rather than being a complete solution. Feel free to make whatever changes you deem to meet the circumstances of your tenant.

Update: The original script was limited to reporting the first 200 workspaces in a tenant. An updated script handles pagination to find and report all workspaces.

No Immediate Impact

It’s unlikely that Loop workspaces will have much of an impact on SharePoint Online tenant storage quotas in the immediate future. Documents will continue to be the major consumer of quota, even when tenants have the Microsoft 365 licenses necessary for users to create new Loop workspaces. Even so, it’s a good idea to keep an eye on how Loop is being used and how much space its files occupy.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Loop Components in Channel Conversations

Message center notification MC681251 (13 October 2023, Microsoft 365 roadmap item 123491) announces support for using Loop components in channel conversations in both the classic and new Teams client. Targeted release is due in November 2023 with standard release tenants getting the new capability toward the end of the year.

About two years ago, Teams chat became the first application to support Loop components, and the components are currently supported in other clients like OWA and Outlook desktop in addition to the full Loop app. We’ve also seen developments such as Microsoft enabling support for the Loop task list component in Planner and the code block component and recently had the surprise that Microsoft will only include access to the Loop app in certain Microsoft 365 product SKUs. You can say that there’s been a lot of activity since November 2021.

Pop-up Announcement for Loop in Channels

Recently, I’ve complained about the number of “helpful” (aka irritating) pop-ups that Teams insists in showing to users. The arrival of Loop components in channels was dutifully announced in a new pop-up (Figure 1).

In this instance, the pop-up was helpful because informed me how to invoke a Loop component in a channel message by using the Actions and apps button to find the Collaborate with Loop option (Figure 2). The same set of Loop components available in Teams chat can be used in channel messages.

If an organization has disabled Loop components for Teams, the option to use them in channel conversations won’t appear.

Including Loop Components in Channel Messages

When composing a channel message, you can incorporate Loop components with text and other elements. Figure 3 shows a message being composed with a Loop table component, some explanatory text, and the compulsory emoji. The image used to highlight the announcement post was created using Microsoft Designer (see this article).

When a user posts a message including a Loop component, Teams stores the Loop component in the channel folder of the SharePoint site belonging to the team. All team members have access to the Loop component. The other message elements are in the Teams message store.

Working with a Loop component in a channel conversation is just like working with these objects in other applications. Anyone with access to the component can change its content and the other users with access see the changes in almost real-time (Figure 5).

Anyone participating in the channel conversation can copy the Loop component into another app and reuse the information there. For instance, they could copy the component into an Outlook email sent to someone for their comment. That recipient can update the component without being a member of the team. This is a really nice way to bring additional wisdom into a discussion.

To allow access non-team members who receive a component through another app, I suspect that Teams generates a sharing link. The link uses the default type of sharing link for the organization, which in my tenant is “only people in your organization” and the default permission, which is edit. I base this on Microsoft’s documentation for known issues with Loop, one of which is what happens when the default sharing link is for specific people only. I could be wrong, but that’s what it seems is happening.

Support for Loop components is available in all channel types (regular, private, and shared). However, guest members of the team cannot access Loop components stored in private and regular channels and external members can’t use them in shared channels.

Mobile Support

The Teams iOS and iPadOS clients support view-only access to Loop components in channel messages. The same support isn’t currently available for the Android client. No mobile client can create a Loop component in a channel message.

Loop for Whiteboard

According to message center notification MC681250 (13 October), Microsoft is also bringing Loop components (task lists, tables, voting tables, progress trackers, and checklists) to Whiteboard for the web, the Whiteboard app in Teams (apart from Teams Room devices), and Whiteboard for Windows. Deployment should be complete by the middle of November 2023.

No Access for External Collaborators

The continuing lack of support for external access to Loop components is a major frustration with the technology. Support for sharing with external people is obvious available because it works for guest access to Office documents stored in the SharePoint Online sites used by Teams. Those documents support co-authoring where the applications synchronize content frequently to update co-authors.

The current restriction might be due to the more frequent synchronization model used by Loop. Extending to people outside the tenant might create synchronization difficulties and detract from the Loop experience. Another obvious issue is that guest accounts don’t have to belong to Microsoft 365 tenants, so the necessary infrastructure might not exist for some guests to interact with Loop. For instance, what does a guest account with a Gmail account do?

Collaborating through Loop components in Teams channels is a powerful tool for working with people within the same organization. Regretfully, its usefulness for some tenants is limited by a lack of support for external team members. I hope that Microsoft lifts that restriction soon, perhaps initially for guests from other Microsoft 365 tenants. That would certainly make Loop components in channel conversions much more interesting to me.

Learn about using Loop and the rest of Microsoft 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Easier to Create SharePoint Lists with PnP.PowerShell

Updated 14 August 2024

Last week, I wrote about how to use cmdlets from the PnP.PowerShell module to create and populate a list in a SharePoint Online site using data generated by the Teams Directory script. As benefits a module deeply rooted in SharePoint history, the cmdlets worked well and the script wasn’t too difficult to write.

The Microsoft Graph is supposed to be “the gateway to data and intelligence in Microsoft 365. It provides a unified programmability model that you can use to access the tremendous amount of data in Microsoft 365…” I don’t have much argument with this assertion because in most cases, it’s true. That is, until you come to SharePoint Online where the coverage of SharePoint objects and data is not as good as for other workloads.

This article describes some of the challenges involved in writing a script based on Microsoft Graph PowerShell SDK cmdlets and Graph API requests to create SharePoint lists similar to what I did using PnP.PowerShell. Let’s see how I got on.

How to Create SharePoint List Script in a Nutshell

The script is simple in concept. The data comes from a CSV file generated by the Teams Directory script. The script creates a list in a SharePoint Online site and populates the list with items imported from the CSV file. The plan was to use cmdlets from the Microsoft Graph PowerShell SDK (V2.8) because there appears to be cmdlets available for everything the script needs to do.

Connecting to the Graph and the Target Site

The first steps connect to the Graph with the relevant permissions and retrieve details of the site holding the list. The script then checks if the list already exists and if found, removes the list. Rebuilding a list from scratch is easier than attempting to synchronize changes.

Connect-MgGraph -Scopes Sites.ReadWrite.All, Sites.Manage.All -NoWelcome
$ListName = "Teams Directory - Graph"
# Get target site 
Write-Host "Fetching details of the target site and list..."
$Site =  Get-MgSite -Search 'Office 365 for IT Pros Communications'
# Get List
$List = Get-MgSiteList -SiteId $Site.Id -Filter "displayName eq 'Teams Directory - Graph'"
If ($List) {
    # Delete the list
    Write-Host ("Removing previous version of list {0}" -f $List.DisplayName)
    Remove-MgSiteList -SiteId $Site.Id -ListId $List.Id
}

Removing a list like this won’t work if a retention label applies to the list.

Create SharePoint List with New-MgSiteList

The next step creates the list. The Graph SDK includes the New-MgSiteList cmdlet, but no matter what I did with the cmdlet, it refused to co-operate. Even the example from the Microsoft documentation failed with the following error:

New-MgSiteList_Create: Unable to determine type of provided column definition
 
Status: 400 (BadRequest)
ErrorCode: invalidRequest
Date: 2023-10-20T16:44:06

As described in this SDK bug report, the problem is that the columns shown in the example define the data type for each column but not what’s acceptable in the column (see this page for more detail about the supported types for list columns). For instance, the text data type can be plain text or rich text or both. If you don’t want to be this specific when creating a list (because you want to customize the list through the GUI afterwards), you can run the Invoke-MgGraphRequest cmdlet to create the list as shown below:

Write-Host "Defining the new list"
$Uri = ("https://graph.microsoft.com/v1.0/sites/{0}/Lists" -f $Site.Id)
$ListDetails = '{
    "displayName": "Teams Directory - Graph",
    "description": "Discover teams to join in Office 365 for IT Pros",
    "columns": [
      {
        "name": "Deeplink",
        "description": "Link to access the team",
        "text": { }
      },{
        "name": "Description",
        "description": "Purpose of the team",
        "text": { }
      },
      {
        "name": "Owner",
        "description": "Team owner",
        "text": { }
      },      
      {
        "name": "OwnerSMTP",
        "description": "Primary SMTP address for owner",
        "text": { }
      },
      {
        "name": "Members",
        "description": "Number of tenant menbers",
        "number": { }
      },
      {
        "name": "ExternalGuests",
        "description": "Number of external guest menbers",
        "number": { }
      },
      {
        "name": "Access",
        "description": "Public or Private access",
        "text": { }
      },
    ],
  }'
Invoke-MgGraphRequest -Uri $Uri -Method POST -Body $ListDetails | Out-Null

The Graph request creates a blank list. The new list includes the specified columns and a single column called Title inherited from the template. If you want to use a column called Title, you can leave it as is. If not, you can rename the column, which is what the script does to make the Title column to be TeamName. The internal name of the column remains Title, which is important to remember when updating records.

$List = Get-MgSiteList -SiteId $Site.Id -Filter "displayName eq 'Teams Directory - Graph'"
$ColumnId = (Get-MgSiteListColumn -SiteId  $Site.Id -ListId $List.Id | `
    Where-Object {$_.Name -eq 'Title'}).Id
Update-MgSiteListColumn -ColumnDefinitionId $ColumnId -SiteId $Site.Id -ListId $List.Id `
  -Description 'Name of the team' -DisplayName 'Team Name' -Name 'TeamName' | Out-Null

Adding Records to the List

After preparing the list, the script populates it with data imported from the Teams Directory. I ran into issues with the New-MgSiteListItem cmdlet. This could be a documentation issue, but some internet forums (like this example) indicate that this cmdlet has not had a happy history. I ended up creating each item as a custom object, wrapping the item data inside another custom object, converting it to JSON, and using the JSON content as a payload to post to the items endpoint:

$Uri = ("https://graph.microsoft.com/v1.0/sites/{0}/lists/{1}/items" -f $Site.Id, $List.Id)
ForEach ($Team in $TeamsData) {
  Write-Host ("Adding directory record for team {0} {1}/{2}" -f $Team.Team, $i, $TeamsData.Count)
  $i++
  $FieldsDataObject  = [PSCustomObject] @{
        Title          = $Team.Team
        Deeplink       = $Team.Deeplink
        Description    = $Team.Description
        Owner          = $Team.Owner
        OwnerSMTP      = $Team.OwnerSMTP
        Members        = $Team.Members
        ExternalGuests = $Team.ExternalGuests
        Access         = $Team.Access
  }
  $NewItem = [PSCustomObject] @{
        fields         = $FieldsDataObject
  } 
  $NewItem = $NewItem | ConvertTo-Json
  $Status = Invoke-MgGraphRequest -Method POST -Uri $Uri -Body $NewItem
  If ($Status.Id) {
     Write-Host ("Record added to list with id {0}" -f $Status.Id)
  }
}   

This approach works, but I could never write to a hyperlink field (something that the Add-PnPListItem cmdlet can do). Apparently, the Graph doesn’t currently support list hyperlink fields, so I ended up writing the deeplink to a team to a text field. The result is the list shown in Figure 1 where users see deeplinks that are not clickable. Users can copy the link to a browser tab and navigate to Teams that way, but that’s not very user-friendly. For small lists, you can create a hyperlink field in the list and copy deeplinks to that field. Users can then click on the link in the hyperlink field. Such a solution is unacceptable at any scale.

You can download the full script from GitHub.

Choose PnP.PowerShell to Create SharePoint Lists

What I learned from the exercise is that the PnP.PowerShell module is a more robust and reliable tool to use when working with SharePoint Online lists. PnP has its own quirks, but it works. I spent far too long chasing Graph SDK cmdlets that didn’t work as documented or couldn’t do what I wanted, so I recommend that you use PnP until Microsoft sorts out the SDK cmdlets and documentation.

In closing, I asked Bing Chat Enterprise to write a script to create and populate a list in a SharePoint site Online based on the Microsoft Graph PowerShell SDK. The results were impressive (Figure 2).

After this experience, I might use Bing Chat Enterprise more often in the future to sketch out the basics of scripts. In this case, Bing Chat Enterprise was helpful. In others, it’s been awful. But that’s the nature of generative AI in respect of its ability to regurgitate errors couched in what seems to be impressive terms.

Keep up to date with developments like how to create SharePoint lists with the Microsoft Graph PowerShell SDK by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Create SharePoint List from Data Extracted from Teams

The article discussing a PowerShell script to generate a Teams directory explains how to create output files in different formats that can be used to make the directory available to users. For instance, you could post a HTML format version of the directory in a SharePoint Online site. Discussion about the post generated some nice ideas, amongst which was the suggestion to output the directory as a SharePoint list (aka Microsoft Lists).

I haven’t done much to manage SharePoint lists with PowerShell, so this seemed like a nice opportunity to explore the idea and increase my knowledge.

Choosing the Right Module to Create SharePoint List

The first order of business is to choose a PowerShell module for the task. I started off with the Microsoft Graph PowerShell SDK, which includes cmdlets like New-MgSiteList and Get-MgSiteList. Unhappily, I ran into several problems with SDK cmdlets (V2.8) that I’ve reported to Microsoft. The documentation and examples for these SDK site cmdlets are not as good as other areas covered by the SDK, so the problems could be due to misunderstanding on my part.

This brought me to the Pnp.PowerShell module (aka “Microsoft 365 Patterns and Practices PowerShell Cmdlets”). PnP is a community effort to create resources that help people to build app on the Microsoft 365 platform. The big advantage of PnP is that its cmdlets can interact with SharePoint Online content like list items where the Microsoft SharePoint management module is limited to tenant and site settings.

Basic Steps in the Script to Add Teams Directory Records and Create SharePoint List

The basic steps in the script are:

• Connect to the site that stores the list. I created a communications site for this purpose.
• Look for the list and if found, remove it because it’s easier to create and populate a new list instead of attempting to synchronize changes since the last update for the team directory.
• Create the list and the columns used to store team directory information. Many templates are available for Lists. I used the Links template and removed one of the two default columns.
• Populate the list with new items. To do this, the script reads the information in from the CSV file created by the original script and writes them as new list items.

PnP.PowerShell Cmdlets Used to Create SharePoint List

Translating the above into PnP PowerShell, the script uses the following cmdlets:

• Connect-PnpOnline to connect to the target site. PnP supports different forms of authentication. For the purpose of this demonstration, the script prompts for credentials of a site administrator and uses those to connect.
• Get-PnPList to check if the target list already exists and Remove-PnPList to remove the list if found.
• New-PnPList to create the target list.
• Add-PnPField to define the set of fields used to store directory information.
• Remove-PnPField to remove the standard Notes field inherited from the Links template. Here’s how the script creates the list and the fields used to store Teams directory information:

New-PnpList -Title $ListName -Template Links -EnableVersioning -Connection $Connection | Out-Null
# Add fields
Add-PnpField -List $ListName -DisplayName 'Team Name' -Internalname TeamName -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Description' -Internalname Description -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Owner' -Internalname Owner -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Owner SMTP Address' -Internalname OwnerSMTP -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Member count' -Internalname MemberCount -Type Number -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'External count' -Internalname ExternalCount -Type Number -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Access' -Internalname AccessMode -Type Text -AddToDefaultView | Out-Null
# Remove the Notes field inherited from the Links template
Remove-PnPField -List $ListName -Identity Notes -Force

• Add-PnPListItem to populate the list with items imported from the CSV file. Here’s how the script populates the list:

[array]$TeamsData = Import-CSV -Path $CSVFile
[int]$i = 0
ForEach ($Team in $TeamsData) {
    $i++
    Write-Host ("Adding record for team {0} {1}/{2}" -f $Team.Team, $i, $TeamsData.count)
    Add-PnPListItem -List $ListName -Values @{
        "URL" = $($Team.Deeplink);
        "TeamName" = $($Team.Team);
        "Description" = $($Team.Description);
        "Owner" = $($Team.Owner);
        "OwnerSMTP" = $($Team.OwnerSMTP);
        "MemberCount" = $($Team.Members);
        "ExternalCount" = $($Team.ExternalGuests);
        "AccessMode" = $($Team.Access);
    } | Out-Null
}

The original version of the Teams Directory script generates a directory record for each team including a clickable deeplink to allow users to open Teams in the selected team. They can then join the team (public teams) or request the team owner to join (private teams). The deeplink generated by the script is formatted to make it clickable when exported to a HTML report. I updated the script to include a simple deeplink because SharePoint list entries don’t need the formatting.

Figure 1 shows the Teams directory records in a SharePoint Online list. I’m sure that the visual appearance of the list could be improved by tweaking the columns, but what’s here is sufficient to demonstrate the principles behind creating and populating a list.

You can download a copy of the full script from GitHub.

Lots to Explore in Lists

The SharePoint community understands and takes full advantage of lists (here’s an example). Others in the Microsoft 365 world might not. Perhaps this example of extracting information from one area of Microsoft to create a SharePoint list and populate the list with Teams directory information might get your creative juices flowing.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Improving the Speed of reporting Teams SharePoint URLs by Replacing the Get-UnifiedGroup Cmdlet

Last week, following a response to a reader question, I updated an article describing how to create a report of Teams and the URLs for the SharePoint Online sites used to store shared files. The only real improvement I made to the script was to use the Get-ExoRecipient cmdlet to resolve the members of the ManagedBy property to output display names instead of mailbox names. This change is necessary since Exchange Online moved to using the External Directory Object ID (EDOID) as the mailbox name to ensure uniqueness. Not everyone can recognize a mailbox GUID and know what mailbox it refers to.

The script uses the Get-UnifiedGroup cmdlet to find team-enabled groups. After reviewing the code, I wondered if it was possible to speed up processing by replacing the Exchange Online cmdlets with Microsoft Graph PowerShell SDK cmdlets or API requests. It’s always been true that the Get-UnifiedGroup cmdlet is relatively slow. This situation is explainable because the cmdlet fetches a lot of data about a Microsoft 365 group from multiple workloads. Microsoft has improved the performance of Get-UnifiedGroup over the years, but it’s still not the most rapid cmdlet you’ll ever use.

Converting to Graph SDK Cmdlets

Converting the script to use Microsoft Graph PowerShell SDK cmdlets isn’t very difficult. Here’s the code:

# Check that we are connected to Exchange Online
$ModulesLoaded = Get-Module | Select-Object -ExpandProperty Name
If (!($ModulesLoaded -match "ExchangeOnlineManagement")) {Write-Host "Please connect to the Exchange Online Management module and then restart the script"; break}
 
Connect-MgGraph -NoWelcome -Scopes Group.Read.All, Sites.Read.All
Write-Host "Finding Teams..."
[array]$Teams = Get-MgGroup -Filter "resourceProvisioningOptions/any(x:x eq 'Team')" -All
     
If (!($Teams)) {
   Write-Host "Can't find any Teams for some reason..."
} Else {
  Write-Host ("Processing {0} Teams..." -f $Teams.count)
  $TeamsList = [System.Collections.Generic.List[Object]]::new()    
  ForEach ($Team in $Teams) { 
   $SPOSiteURL = (Get-UnifiedGroup -Identity $Team.Id).SharePointSiteURL  [array]$Channels = Get-MgTeamChannel -TeamId $Team.Id
   [array]$Owners = (Get-MgGroupOwner -GroupId $Team.Id).AdditionalProperties.displayName
   $DisplayNames = $Owners -join ", "
   $TeamLine = [PSCustomObject][Ordered]@{
      Team      = $Team.DisplayName
      SPOSite   = $SPOSiteURL
      Owners    = $DisplayNames  }
   $TeamsList.Add($TeamLine)
  }
  $TeamsList | Out-GridView
  $TeamsList | Export-CSV -NoTypeInformation c:\temp\TeamsSPOList.CSV
}

Figure 1 shows the result.

You’ll notice that I still use the Get-UnifiedGroup cmdlet to fetch the Teams SharePoint URL. It’s possible to retrieve this information using the Graph with code like:

   $Uri = ("https://graph.microsoft.com/v1.0/groups/{0}/drive/root/webUrl" -f $Team.Id)
   $SPOData = Invoke-MgGraphRequest -Uri $Uri -Method Get
   [string]$SPODocLib = $SPOData.Value
   $SPOSiteUrl = $SPODocLib.SubString(0, $SPODocLib.LastIndexOf("/"))

Or:

   $Uri = ("https://graph.microsoft.com/v1.0/groups/{0}/sites/root" -f $Team.Id)
   $SPOData = Invoke-MgGraphRequest -URI $Uri -Method Get
   $SPOSiteUrl = $SPOData.WebURL

The Problem with Permissions when Fetching Teams SharePoint URLs

In both cases, the code works. However, the code fails for some teams due to the restriction placed on interactive use of the Graph SDK. When you connect an interactive session to the Graph, you’re restricted to using delegate permissions. The only data that the Graph SDK cmdlets can access is whatever the signed-in user can access. This is very different to the permissions model used by modules like the Exchange Online management module, which allow access to data based on RBAC controls, meaning that a tenant administrator can access everything.

The restriction disappears when running the SDK cmdlets using a registered app or an Azure Automation runbook. Now the cmdlets can use application permissions, so they can access any data permitted by the Graph permissions assigned to the service principal of the app.

Using either version of the code shown above works perfectly and returns the SharePoint site URL, but only for sites accessible to the signed-in user. Attempts to access any other site returns a 403 forbidden error.

I even tried using the Teams Graph cmdlets:

[array]$Channels = Get-MgTeamChannel -TeamId $Team.Id
$Files = (Get-MgTeamChannelFileFolder -TeamId $Team.Id -ChannelId $Channels[0].Id).WebURL
$SPOSiteUrl =  $Files.SubString(0,$Files.IndexOf("sites/")) + "sites/" + $Team.MailNickName

Again, this approach works for teams that the signed-in user is a member of, but not for other teams.

Going Back to Pure Exchange Cmdlets to Report Teams SharePoint URLs

The problem with permissions meant that I had to use a hybrid of Graph SDK cmdlets to get everything except the SharePoint site URL. And while this approach works, it’s slower than the original implementation using only Exchange Online cmdlets. In several runs against 88 teams the hybrid version took an average of 42 seconds to finish. The Exchange version required an average of 31 seconds.

The learning here is that Graph SDK cmdlets aren’t always the best choice for speed, no matter what you read on the internet. It’s always worth testing to find which approach is the most functional and fastest. Sometimes both boxes are ticked, and that’s a result.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

No Good Reason why SharePoint Limits Administrator Access to Document Libraries

A reader asked if a programmatic method exists to set the default sensitivity label for a SharePoint Online document library. The simple answer is “yes,” because the only way initially available to set a default sensitivity label when the feature was in preview was to use the SharePoint REST API. Microsoft subsequently updated the SharePoint browser GUI to allow site owners to set a default sensitivity label for a document library.

Using the REST API still works, but my reader wanted something like a nice simple PowerShell cmdlet. Something like this would be nice:

Set-SpoSite -Identity $SiteURL -DocumentLibrary "Documents" -DefaultSensitivityLabel c29e68f9-bc4f-413b-a741-6db8e38ad1c6

The command would be nicer if you could pass the name of a sensitivity label, but the display names for sensitivity labels can be translated into multiple languages, which might cause some issues in multilingual tenants.

In any case, the Set-SPOSite cmdlet doesn’t support the functionality today and I haven’t heard of any plans to change in this area.

Reasonable to Allow Administrator Access to Some SharePoint Online User Data

I think it’s perfectly reasonable for SharePoint Online administrators to be able to update the default sensitivity labels for document libraries, especially because assigning a default sensitivity label incurs the requirement for Syntex-SharePoint advanced management licenses. An unwitting site owner could decide to assign a default sensitivity label to a document library (Figure 1) without realizing that the organization is now on the hook for some licenses, and that’s not a good thing. SharePoint administrators should be able to review, assign, and remove default sensitivity labels.

But this stance goes against the general approach Microsoft takes to SharePoint Online administration which holds that administrators can operate at the site level but cannot interact with objects within the site. Apparently, a site can have up to 255 document libraries, all of which are invisible to SharePoint administrators unless they’re a member of the site.

I understand the perspective that drives the approach. Administrators shouldn’t have access to user data. However, while Exchange Online administrators can see the folders inside user and shared mailboxes and Teams administrators can remove user data such as chat threads. It’s also possible for administrators to analyze and report the tasks in Planner plans. And sometimes even SharePoint Online administrators can take action with user data, like removing the sensitivity label for protected files using the Unlock-SPOSensitivityLabelEncryptedFile cmdlet. Inconsistency is rife across the Microsoft 365 workloads.

Greater Flexibility Required

I’m not advocating for SharePoint Online administrators to be able to open and examine documents and other files held in document libraries. The ability to report the contents of document libraries is already possible, albeit with some effort. What I would like to see is greater access to document library settings through PowerShell or a Graph API (which means that PowerShell support becomes available through the Microsoft Graph PowerShell SDK). For instance, why shouldn’t an administrator be able to do this to create a simple listing of all files found in the document libraries for a site:

$DocumentLibraries = Get-SpoSite -Identity $SiteUrl -DocumentLibraries
ForEach ($DL in $DocumentLibraries) {
   $Documents = Get-SPODocumentLibrary -Identity $DL 
   ForEach ($Doc in $Documents) {
    Write-Host (“Document found {0} in folder {1}” -f $Doc.Title, $Doc.Folder)
  }
}

SharePoint Online is not the center of its own universe as is the case with on-premises SharePoint Server. SharePoint Online is a highly capable document management service that’s consumed by other Microsoft 365 workloads. As such, its administrative capabilities should be on a par with other workloads, and that means greater flexibility and access to the settings for document libraries. Being able to report, configure, and remove the default sensitivity label for a document library is just the start.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Really Just Sending SharePoint News in an Email

Whover wrote MC671563 (29 Aug 2023) titled “SharePoint News in Outlook” needs some help composing headlines. Microsoft 365 roadmap item 124803 has nothing whatsoever to do with Outlook. Reading the headline, I anticipated something like a new OWA control (available also in the Monarch client, and for Outlook desktop via OPX) that allowed users to browse news items posted to their favorite SharePoint Online sites.

Instead, it’s simply a way to send news items from SharePoint Online via email to allow recipients to read the news using whatever email client they like. Although sending news via email is functional, it’s a bit of a damp squib when you consider that people have exchanged news via email since the dawn of messaging. Something more adventurous would have been nice.

Rollout to targeted release tenants has already happened. Standard release tenants will start to see the new feature in mid-September with full deployment due by late September 2023.

New Emailable News Templates

Essentially what’s happened is that SharePoint Online has six new templates to compose news items that are both posted to their host SharePoint site and emailed (Figure 1).

The templates intended for both posting and email support a limited set of web parts. With that exception, creating a new item is as before (Figure 2).

SharePoint News in Outlook Messages

After the content is ready, click Post and send. SharePoint posts the item to the site and displays a screen to allow the user to add the email addresses to receive the post (Figure 3).

The message that arrives in a user inbox gives the recipient the option to read the information in their favorite email client or in SharePoint (Figure 4). The link to SharePoint Online only works if the recipient can access the host site.

The mechanism used by SharePoint Online is rather like the Teams Share to Outlook feature and shouldn’t cause anyone to kill too many brain cells to master the feature. Some points worth noting are:

• To make sure that the information stays within the tenant, SharePoint Online doesn’t allow external addresses to receive the post. All addresses added to the message must belong to the tenant. The set of valid addresses includes user accounts, Microsoft 365 groups, and distribution lists.
• The feature connects to the mailbox of the author of the news item and creates and sends the message from there (you can do the same thing using Graph APIs or the Graph SDK). A copy of the outbound message is in the Sent Items folder. Using this mechanism ensures that the message travels through the Exchange Online transport pipeline. Exchange Online can then apply any transport rules or DLP policies that match the message. The full path of the message is available through message trace, including any transport events that happen such as the application of transport rules.

One exception exists to the rule that limits transmission to internal recipients. If you operate in a Microsoft 365 multi-tenant organization (MTO), user accounts from other tenants in the MTO synchronize to your tenant as member accounts. SharePoint Online allows news items to be emailed to MTO synchronized accounts from other tenants. It might be that the SharePoint developers decided to support MTO accounts because they are deemed trustworthy because they come from a tenant that has a cross-tenant synchronization arrangement with your tenant. Or they simply didn’t realize that MTO accounts exist. I fear that the latter is the true reason.

Analytics for SharePoint News in Outlook

Page analytics are available for each news item. Microsoft says that the analytics reflect total page reads sourced from SharePoint Online and Outlook (email). News sent by email can be reported in terms of page views but SharePoint can’t capture how long people spend reading news items received by email.

A Feature Seeking a Problem

As I played with sending SharePoint news items via email, the question crossed my mind about what demand exists for such functionality. It’s easy to copy and paste interesting news snippets into regular email if you want to. No analytics are available, but again you wonder if this is important. Perhaps organizations exist that place great importance on SharePoint news items and insist on the ability to email the latest information. If so, I haven’t met them.

Microsoft’s blog on the topic isn’t particularly illuminating until you read the comments from real people who know more about SharePoint news than I do. Those comments are worth reviewing before you decide to dedicate any effort to deploying this feature.

Learn about using SharePoint Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Stop Non-Group Members Accessing Restricted Sites

For years, I have been under the impression that SharePoint Online imposed access control for sites connected to Microsoft 365 through group membership. But then MC671823 came along on 29 August 2023 on the topic of Restricted Access Control for SharePoint and OneDrive Sites. According to Microsoft 365 roadmap item 163991, the deployment date is December 2023, but MC671823 says that the feature is now rolling out through both the SharePoint Online admin center and PowerShell. Confusion reigns.

Microsoft says that the feature allows administrators to “restrict site access to specified users using Microsoft 365 group or AAD security groups. Users not added in the specified group(s) will not be able to access the site even if they were previously granted site access.” The idea is “to reduce the risk of oversharing or permission sprawl within their organizations.” Both are good aspects to restrict, especially for organizations considering the implementation of Microsoft 365 Copilot, where the danger exists that Copilot might use sensitive information in its responses.

I think the basic thinking behind restricted sites is that it’s possible for site administrators to add extra users to sites that might expose confidential information to those people. By restricting access to known groups, you remove the risk. Of course, there’s nothing to stop a administrator for a site connected to a Microsoft 365 group adding someone to the group membership (by definition, they’re a group owner), but then everyone else in the group can see that a new member is present and could ask some awkward questions.

Configuring Restricted Sites

The first thing to do is to configure the tenant to support restricted sites. Download the latest version of the Microsoft.Online.SharePoint.PowerShell module from the PowerShell gallery (or use our script to update your Office 365 modules). I used version 16.0.24009.12000. Connect to the module and run:

Set-SPOTenant -EnableRestrictedAccessControl $True

If you don’t take this step, you won’t be able to configure restricted access for individual sites using either the SharePoint Online admin center or PowerShell. Like any setting applied to a SharePoint Online tenant, wait for a couple of hours to allow the change to take effect. You can then update individual sites. For instance, to update a site with PowerShell, run the Set-SPOSite cmdlet:

Set-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/ProjectHiddenSecret" -RestrictedAccessControl $True

Restricted access control has been enabled on the site https://office365itpros.sharepoint.com/sites/ProjectHiddenSecret. The site access is restricted to members of the group b248090e-2bca-4d14-8aa6-3969a157a2a6.

Get-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/ProjectHiddenSecret"  | Format-List Restrictedaccess*

RestrictedAccessControl       : True
RestrictedAccessControlGroups : {b248090e-2bca-4d14-8aa6-3969a157a2a6}

The GUID (b248090e-2bca-4d14-8aa6-3969a157a2a6) returned by the Get-SPOSite cmdlet is the group identifier pointing to the group SharePoint Online uses to control restricted access. Because this is a site connected to a Microsoft 365 group, the GUID resolves to that group:

Get-MgGroup -GroupId b248090e-2bca-4d14-8aa6-3969a157a2a6

DisplayName           Id                                   MailNickname        Description                      GroupTy
                                                                                                                pes
-----------           --                                   ------------        -----------                      -------
Project Hidden Secret b248090e-2bca-4d14-8aa6-3969a157a2a6 ProjectHiddenSecret A project full of hidden secrets {Uni...

Alternatively, edit the site settings to enable restricted site access (Figure 1).

Restricted Sites Not Connected to Microsoft 365 Groups

An extra step is needed to configure sites that aren’t connected to Microsoft 365 groups. In this case, you must specify the identifier for one or more (up to 10) comma-separated Entra ID security groups to use for access control. Dynamic security groups are supported. Here’s an example:

Set-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/TestSite001" -RestrictedAccessControl $True -RestrictedAccessControlGroups d347eec5-62f1-4436-af41-e53fa18090be

Restricted access control has been enabled on the site https://office365itpros.sharepoint.com/sites/TestSite001. The site access is restricted to members of the group d347eec5-62f1-4436-af41-e53fa18090be.

Finding Restricted Sites

To find sites with restricted access, you must scan each site and then resolve the group identifiers using code like this:

[array]$Sites = Get-SPOSite -Limit All
ForEach ($Site in $Sites) {
    $SiteDetails = Get-SPOSite -Identity $Site.Url
    If ($SiteDetails.RestrictedAccessControl -eq $True) {
       [array]$Groups = $SiteDetails.RestrictedAccessControlGroups
       ForEach ($G in $Groups) {
         $Group = Get-MgGroup -GroupId $G
         Write-Host ("Site {0} owned by group {1}" -f $SiteDetails.Title, $Group.displayname) 
       }
    }
}

Licensing Restricted Sites

Making a SharePoint Online site subject to restricted access requires the Syntex-SharePoint advanced management license. At least, that’s what we learn from Microsoft’s documentation, which says that “some features” require the license without offering any further guidance. My assumption is that any user accessing a restricted site needs the license.

Another Layer of Protection

Restricted sites add another security layer to protect confidential information stored in SharePoint Online. If you pay for the advanced management license, you can also assign the block download policy to sites to stop site members downloading files from the site. Add sensitivity labels to block access unless people have the right to open files, and you’ve got a nice set of protections to prevent unauthorized access to information.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

FileRecycled Audit Event Replaces FileDeleted

In December 2021, I wrote about using events captured in the unified audit log to analyze file deletion activity in SharePoint Online and OneDrive for Business. Recently, some readers complained that the script (available from GitHub) wasn’t finding events for file deletions. A major advantage of PowerShell is that you see all the code and can modify the code to meet your needs. This also means that you can debug the code. My usual response to people who report problems with scripts is to prompt them to do some basic debugging by running the code line-by-line until the problem becomes apparent. Apart from learning how the script works, debugging is a great way to improve PowerShell skills.

In any case, a quick check revealed the problem. Microsoft changed the name of the operation captured in file deletion audit events from FileDeleted to FileRecycled. The change seems to have come into force in March 2023. At least, that’s the date of the first FileRecycled audit event generated by SharePoint Online I can find in my tenant. Microsoft didn’t say anything about the change. It just happened without warning.

File Deletion or Recycling

A case can be argued that FileRecycled is a more accurate description of the action than FileDeleted is (see this documentation update request from August 2022). SharePoint Online doesn’t actually delete an item until it goes through the two-stage recycle bin and exceeds the 93-day retention period that items remain in the recycle bin. The initial action is to move an item from a document library to the site recycle bin, hence the justification to use the FileRecycled name in audit records.

I wouldn’t have a problem if Microsoft told people about the change. Not everyone scans the documentation to detect name changes for audit log activities. Unless you checked the data returned by the Search-UnifiedAuditLog cmdlet or noticed the details for file deletions (or rather “recycled file”) operations returned by the audit log search in the Purview compliance portal (Figure 1), the change would probably have escaped undetected.

Microsoft also changed the UI of the audit search solution so that if you select “Deleted file” from the list of activities to search for, you’ll find events logged when SharePoint Online removes files from the recycle bin.

The Impact of Unannounced Changes

The problem here is that when Microsoft makes unannounced changes to audit data, it potentially affects scripts written by organizations to move data from the audit log to an external repository like Splunk. Among the reasons why organizations populate external repositories with audit data are:

• Long-term retention of audit data. Until recently, Microsoft only kept audit data for 90 days. On July 19, 2023, Microsoft announced a doubling of the audit data retention period to 180 days for Audit standard (Office 365 E3) customers. Audit premium customers have a 365-day audit data retention period with an optional add-on license available to increase the period to 10 years.
• Better search and investigation facilities. Although organizations have built tools to interrogate the unified audit log, the fact remains that the contents of audit log entries often need processing to extract useful information (like this example of extracting information about changes made to Entra ID account properties).

Obviously, if a new name is introduced for a common auditable activity like file deletion, it’s likely that processes to export audit data will ignore these events. I haven’t found any other activity renames but suspect that some might be lurking in the audit log.

Updates without Warning Reduce Confidence

The bottom line is that reliable audit data is an important part of a compliance ecosystem. If audit data is missing or becomes difficult to interrogate, those who work with audit data lose a little faith because it isn’t as comprehensive and accurate as they expect. And that’s a great pity.

Making Sharing of Files and Folders Easier

Apart from Microsoft 365 roadmap item 124933, I can’t find a formal announcement about the Simplified Sharing Experience, but I have been aware that Microsoft recently updated the share dialog used by Microsoft 365 apps to make it easier and more straightforward to use. According to a LinkedIn post, (Figure 1) Microsoft ran an A/B experiment to test the new dialog. I guess I was one of the testers! In any case, the new sharing dialog is now available in all Microsoft 365 tenants. Users of OneDrive consumer will see the upgraded dialog in the second half of 2023.

The Role of the Share Dialog

The share dialog is what people see when they share a document or folder with others inside or outside their organization. According to Microsoft, the dialog is used over 800 million times monthly across 52 different Microsoft 365 experiences (desktop, browser, and mobile). In other words, Microsoft 365 apps offer users the opportunity to share in 52 different places across the suite. The most common of the experiences are likely in SharePoint Online, OneDrive for Windows, and Teams.

Microsoft says that they focused on creating a dialog that makes it simpler for users to perform core sharing tasks. When someone invokes the new screen (Figure 2) to share a file or folder, they see a simpler layout pre-populated with the default sharing link as specified by the tenant or site policy (in this case, the sharing link allows access to people within the organization). The name of the sensitivity label assigned to the document is also shown to provide a visual indicator about its relative confidentiality.

To complete the link, add the people to notify and enter a note to tell them what to do, and click Send to have the message sent by email or Copy link to copy the sharing link to the clipboard.

If you need to change the type of sharing link, select the cogwheel to expose the link settings (Figure 3). Again, everything is very straightforward and simple. If you choose a link that allows external sharing, I’m told that the new design “makes users more comfortable with sharing.” I’m not quite sure what this means, but any of the sharing that I’ve done with people outside the organization has worked smoothly.

Microsoft has also overhauled the Manage access dialog to help people manage the set of users and groups that have access to a file or folder (Figure 4).

Microsoft says that customer feedback about the new dialog is very positive. It’s worth noting that this is not the first time that Microsoft has revamped the sharing dialog. The last major overhaul was in 2020-21 when Microsoft rationalized on a common sharing dialog for all apps, notably for Teams.

The Importance of Sharing

Getting sharing right is clearly important. When Microsoft launched the Delve app in 2015, it resulted in a crescendo of protest from tenants who suddenly found that Delve suggested documents to users when the organization thought that Delve should not. Of course, the software did nothing wrong. Delve respected the access rights given to users when it computed the set of interesting documents to suggest (using an early version of Graph document insights). The problem was entirely down to poor management and access control, often at the level of complete SharePoint Online sites. Users might not have realized that they had access to the documents in poorly-protected sites, but software can’t be blamed if it goes looking for documents to suggest to a user and finds some that are available.

We’re heading for a similar situation with Microsoft 365 Copilot. The Copilot software depends on finding information with Graph queries to help satisfy user prompts. Like Delve, Copilot will find files that are available to the user who prompts for help, and the results generated for the user might include some confidential. And if the user doesn’t bother to check the content generated by Copilot, the information might then be revealed with people who shouldn’t have it. This is the danger of oversharing, and it’s certainly an issue for organizations contemplating Microsoft 365 Copilot need to resolve before implementation.

Simplified Sharing Experience One Step Along the Path

The new sharing dialog won’t solve oversharing. It’s just one step along the path to help users share information with the right people in the right way.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Clipchamp for Work, but Only for Microsoft 365 Commercial Customers

In a July 31 announcement, Microsoft says that they will make Clipchamp for Work available to Microsoft 365 commercial customers through the targeted release program in the coming weeks. The news is covered by Microsoft 365 roadmap item 124826.

Although the advent of Clipchamp for Work is good news, it’s tempered by Microsoft’s statement that

“Clipchamp will be added to the following Microsoft 365 SKUs: Microsoft 365 E3, Microsoft 365 E5, Microsoft 365 Business Standard, and Microsoft Business Premium. Clipchamp will not be added to Office 365 SKUs.”

Ignoring the Office 365 enterprise SKUs is part of Microsoft’s tactics to force customers to upgrade to Microsoft 365 SKUs to drive the average revenue per user (ARPU) and increase the profits flowing from its cloud business past the $111.6 billion mark announced for Microsoft’s 2023 fiscal year.

The same method is being used with Microsoft 365 for Copilot, which only supports the Microsoft 365 E3 and E5 enterprise SKUs. Moving from Office 365 E3 to Microsoft 365 E3 costs an extra $13/user/month. Although a case can be argued that features like Windows 11 Enterprise and Enterprise Mobility and Security which are bundled into the Microsoft 365 enterprise SKUs represent good value for the extra cost, the fact remains that some customers don’t want to be forced to upgrade. Adding a very capable video editor to the Microsoft 365 enterprise SKUs probably won’t make any difference when it comes to deciding whether to upgrade, but it is one more factor to consider.

Adding on to Stream

Making Clipchamp for Work a new Microsoft 365 service plan makes sense. Like Stream for SharePoint, Clipchamp for Work stores its videos in SharePoint Online and OneDrive for Business, so it’s very different to the Clipchamp consumer version. Microsoft won’t provide a migration mechanism to move videos from the consumer to commercial version.

Stream for SharePoint added features steadily since its release, recently adding inline playback in Teams, automatic transcript generation, and a teleprompter when recording videos. However, Stream for SharePoint lags behind Stream classic in terms of video editing capabilities. Stream classic never offered much, but at least it could trim some excess from the start and end of videos.

Clipchamp fills the void and adds a lot more functionality besides such as a gap remover (merge videos together seamlessly) and a video cropper. Microsoft also promises that “Filters, effects, and text overlays give your editing a professional and personalized look. With the green screen filter in Clipchamp, you can adjust your backgrounds to suit your professional context and environment. Transitions can easily be added as a finishing touch to give your video that high-quality look.”

Clipchamp for Work includes an in-browser video editor. You’ll be able to create a new video by launching Clipchamp from the Microsoft 365 app launcher or from OneDrive for Business. Clipchamp can edit videos stored in OneDrive for Business and SharePoint Online, including Teams meeting recordings. The point about using SharePoint storage for videos is to take advantage of integration with the rest of the Microsoft 365 suite such as sharing, compliance, data lifecycle management (retention), and information protection.

Clipchamp Brings Proper Video Editing to Microsoft 365

When Microsoft bought Clipchamp in September 2021, the speculation was that Microsoft would add Clipchamp quickly to Office 365. Well, that hasn’t happened. Apart from needing to get used to the Microsoft 365 ecosystem so that single sign-on works and Clipchamp can process videos stored in SharePoint Online, the delay in reaching this point is likely due to waiting for Stream on SharePoint to mature and the migration from Stream classic to complete, which it now mostly is.

I’m glad to see a proper video editor in Microsoft 365. Video communications are becoming more important all the time. It’s just a pity that Microsoft won’t make the Clipchamp functionality available to their loyal Office 365 enterprise customers. We’ll just have to keep on using Camtasia instead.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

More Details Revealed About Microsoft 365 Backup and Microsoft 365 Archive During Inspire session

A week or so after the launch of Microsoft 365 Backup and Microsoft 365 Archive at their annual Inspire conference (for partners), I took the time to listen to the recording of the session covering the topic delivered at the event. It’s hard to get much detail from a 14-minute session after filtering out all the marketing messages delivered by the avuncular Chris McNulty, but I found some interesting points to ponder.

As a reminder, these products are not yet available. They might be toward the end of 2023. Then again, product dates have an unfortunate habit of slipping, especially when they’re for solutions in a new area. This is doubly true when dealing with solutions targeted at backup and restore and touted as a great solution for ransomware because of their “unprecedented speed and scale.”

McNulty started with some statistics:

• Microsoft 365 users add two billion documents and emails daily. I assume this figure includes Office documents, PDFs, Loops, OneNote notebooks, emails, Teams messages, and everything else that can be stuffed into SharePoint Online, OneDrive for Business, and Exchange Online. In September 2022, Microsoft said that Exchange Online processes 9.2 billion messages daily, 2.4 billion of which are spam. However, it’s unclear if these figures include system messages that are transient and not stored.
• Microsoft 365 user activity consumes 200 petabytes of storage monthly. Much of the data is unstructured. I assume that imports from SharePoint Server and other non-Microsoft 365 sources consume some of this storage. While providing such a large amount of storage is a heavy expense for Microsoft, its existence inside Microsoft 365 creates opportunities. For instance, it is the raw material for Microsoft 365 Copilot.

Microsoft also said that the estimated annual cost of ransomware is $20 billion (2021). They also noted a 74% increase in password attacks in one year, which is yet another good reason for Microsoft 365 tenants to make better use of multi-factor authentication even if attacker tactics like password sprays are less effective due to the removal of basic authentication.

Microsoft 365 Backup

The basic value proposition for Microsoft 365 Backup is simple: the ability to backup and restore data more rapidly than any other backup solution. This is because the data remains within Microsoft 365 and therefore doesn’t have to be copied across an internet connection. Partners have access to the Microsoft APIs for backup, restore, and archiving to allow them to integrate Microsoft 365 in their solutions. In this context, Microsoft will take care of the background processing and the partner looks after the user interface and integration with backup and restore solutions that handle other non-Microsoft workloads to create a single pane for all backup and restore operations.

Of course, keeping backups of your SharePoint Online, OneDrive for Business, and Exchange Online data within the Microsoft trust (security) boundary is a double-edged sword. Keeping all your data eggs in the one Microsoft basket is convenient, enables fast restore, and easy to use because operations are integrated in the Microsoft 365 admin center.

Jacklynn Hiranaka’s demonstration of backup and restore showed how easy it is to configure full backup for a tenant (Figure 1). She made the point that once backup is enabled, it becomes effective immediately. This is likely because Microsoft can utilize techniques like capturing SharePoint changes in the Preservation Hold Library or Exchange changes in Recoverable Items to generate backup items. You can imagine how restores operate like a supercharged version of the SharePoint Restore this library feature or Exchange’s Recover deleted items.

Even more impressive was the assertion that Microsoft 365 Backup can perform parallel restores for SharePoint Online, OneDrive for Business, and Exchange Online to restore information very quickly.

Microsoft 365 Archive

Brad Gussin covered details of Microsoft 365 Archive. This is a SharePoint Online option (Exchange Online has its own archiving). You can already archive Teams and put the associated SharePoint Online sites into a read-only mode. Microsoft 365 Archive puts inactive SharePoint sites into a state where administrators can still manage the sites (to bring them back into an active state) but the data is no longer “hot” (available for immediate user access).

The major advantage gained by moving sites to an archived state is that the storage they consume is no longer charged against the tenant’s SharePoint storage quota. The data is still in SharePoint, but just like the storage consumed by Syntex Repository Services to hold Loop app data, it’s not accessible in the normal way.

Administrators will be able to search for inactive sites and decide which sites to archive. Site owners can protest this action and negotiate with administrators to keep their sites online. Once the final decision to archive, the process to archive sites takes a couple of hours. Actions to archive or reactivate sites are available through the SharePoint Online admin center (Figure 2) or PowerShell. Microsoft hasn’t specified how the PowerShell option will work, but it could be through an updated Set-SPOSite cmdlet or perhaps dedicated cmdlets to archive and reactivate sites. Long-term, Microsoft plans to enable finer granularity by supporting archival at the file level.

Microsoft 365 features such as data loss prevention, data lifecycle management (retention processing), information protection, and search remain in place for archived sites. eDiscovery can find items in archived sites (using the search indexes) and retrieve items using search exports.

A cynic might say that Microsoft created the need for an archive solution by restricting the amount of storage made available to tenants (1 TB plus 10 GB per eligible license) and the way that retention processing consumes quota. The more intelligent versioning planned for document libraries might help restrain storage consumption, but overall it’s still true that SharePoint Online storage is expensive when compared to the abundant storage made available to OneDrive for Business accounts.

No Pricing Available

Microsoft hasn’t revealed how much Microsoft 365 Backup and Microsoft Archive will cost. I’ve been surprised by some recent Microsoft pricing decisions (like the $7/user/month demanded for slightly more intelligent Entra ID access reviews). The good thing is that backup for Microsoft 365 is a competitive market. Microsoft has some strong advantages, but if it goes too far in terms of inflated pricing, customers will vote with their wallets and go elsewhere.

Learn about using SharePoint Online, Exchange Online, and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Sensitivity Label PDF Support Increases Coverage for Protection

In my review of sensitivity labels for 2023, I noted that the only way to apply a sensitivity label direct to a PDF was with:

• The paid-for versions of Adobe Acrobat.
• Generating PDFs from Office documents (subscription apps only).
• Applying a label through the unified labeling client.

Unlike retention labels, it wasn’t possible to apply a sensitivity label to a PDF using the SharePoint Online browser client. Now it is, and it’s an important update given the widespread use of PDFs within Microsoft 365. Between Office documents and PDFs, sensitivity labels can now protect over 90% (my estimate) of all files stored in SharePoint Online and OneDrive for Business. It’s another step to making PDFs a fully functional format within the Microsoft Information Protection ecosystem.

What Sensitivity Label PDF Support Means for SharePoint Online

In an update announced by principal program manager Sanjoyan Mustafi on LinkedIn, the preview of SharePoint Online support for PDFs is available to all commercial tenants worldwide. Support extends to sensitivity labels with predefined permissions. Labels with user-defined permissions or those that use Double Key Encryption (DKE) are unsupported.

Supporting sensitivity labels for PDFs means that people can use SharePoint Online and OneDrive for Business to:

• Apply sensitivity labels to PDFs through the browser interface (Figure 1) and amend or remove the label afterwards, including forcing the user to provide justification if required by policy. This includes applying the default sensitivity label defined for a document library to PDFs as users load them into the library (requires the SharePoint-Syntex advanced management license).
• Apply sensitivity labels to PDFs stored in SharePoint Online and OneDrive for Business through auto-label policies. This feature is covered in message center MC644060 (14 July, 2023).
• Apply sensitivity labels to PDFs using the assignSensitivityLabel Graph API (if your app has permission to do so).
• Display the names of sensitivity labels for protected PDFs in document libraries.
• Index the content of PDFs protected by sensitivity labels. This supports Microsoft Purview solutions like Data Loss Prevention, content searches, and eDiscovery.

Like Office documents protected by a sensitivity label with encryption, SharePoint Online can’t display a thumbnail of a protected PDF (Figure 2). I believe that this has something to do with the inability to fetch the necessary use license to decrypt the file. Thumbnails are shown for PDFs assigned a sensitivity label with no encryption. To open a document, use the Edge browser (which supports reading protected files) or download the file and use an app that understands how to open protected PDFs (like Acrobat).

I hear that Microsoft is working on the viewing issue and expects to have a fix by the end of 2023.

Enabling Sensitivity Label PDF Support for SharePoint Online

By default, SharePoint Online support for PDFs is disabled. To enable support, load the SharePoint Online administration PowerShell module and run the Set-SPOTenant cmdlet. You’ll need a recent version of the module (use this script to update your Microsoft 365 modules to the latest version):

Set-SPOTenant -EnableSensitivityLabelforPDF $True

To revert, run the command to update the setting to $False.

Set-SPOTenant -EnableSensitivityLabelforPDF $False

Disabling SharePoint support for PDFs has no effect on PDFs with sensitivity labels. It will stop users being able to assign or update labels through the SharePoint Online and OneDrive for Business browser interfaces and SharePoint Online will cease indexing protected PDF content.

If you don’t want to use PowerShell, check the Information protection section of the Purview compliance portal, and go to Auto-labeling. You might see a message inviting you to turn on support for PDFs. If you do, select Turn on now and the job is done.

More information about PDF support for sensitivity labels in SharePoint Online is available in Microsoft documentation.

Sensitivity Label PDF Support is an Important Step Forward

I don’t think it is an exaggeration to say that some organizations have been waiting years for PDF support to arrive in SharePoint Online. Given the widespread use of PDFs in many organizations, this is an important step forward for those wishing to protect their most sensitive information stored in SharePoint Online and OneDrive for Business.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Intelligent Versioning Based on Usage Coming Soon

Update: 14 June, 2024: The SharePoint version history feature is rolling out in preview.

In a recent article describing changes SharePoint Online made to how it stores retained files in the Preservation Hold Library, I mentioned the effect of retention on SharePoint storage and hoped that the changes would reduce this impact. Now it appears that Microsoft plans further changes to help.

Microsoft 365 roadmap item 145802 posted on June 30, 2023 discusses proposed changes for version history limits in SharePoint Online document libraries. Today, SharePoint Online requires versioning to be enabled for document libraries and lists and uses the following values to control major versions:

• Default: 500 versions.
• Minimum: 100 versions.
• Maximum: 50,000 versions.

Versioning is a critical feature for SharePoint Online. It underpins capabilities such as being able to restore a document library to a point in time, the autosave feature used by the Office apps to make sure that people don’t lose work, and updating of local file copies by the OneDrive sync client. It’s easy to accumulate a large number of versions. For instance, the PowerShell chapter for the Office 365 for IT Pros (2024 edition) eBook already has 81 versions (Figure 1) since its creation in early May. A single editing session to create this article created seven versions.

For these and other reasons, SharePoint Online sets the default number of versions to 500. It’s therefore not a good idea to reduce the number of versions for a document library by editing the number in Versioning Settings (Figure 2). On the other hand, increasing the number of versions retained by a document library can increase the storage consumed by the Preservation Hold Library.

Moving to Automatic Versioning Management

Microsoft says that they plan to “increase version history limits” for SharePoint Online and OneDrive for Business document libraries. Site administrators will be able to choose two types of version limits:

• Automatic mode: SharePoint Online “intelligently” adjusts the versions kept for files based on age and the probability that a version will be required for a restore.
• Manual mode: Site administrators set version expiration and count limits for document libraries.

Tenant administrators will be able to set version limits that apply to newly created document libraries.

The big change is a movement away from simple count-based version limits (i.e., SharePoint Online keeps up to 500 versions of files) to a system where SharePoint Online manages version counts automatically depending on the usage of documents and the site.

Different Update Patterns

For instance, if people edit a document daily, it might generate ten versions every business day or fifty-plus versions a week. Under the present scheme, SharePoint Online begins to discard versions after ten weeks or so. The new mechanism might note the update pattern and decide that it should keep more than 500 versions to allow for a longer restore window than 10 weeks and remove versions after six months.

On the other hand, a static document that’s edited twice a year might have a much lower version count. And SharePoint Online might dynamically adjust the version count downwards after a document moves from the phase where people actively work on its content to when the file becomes stable and is no longer being actively edited.

All of this is speculation based on the description in the Microsoft 365 roadmap item. We won’t know the exact details about how automatic versioning management works until we see the new mechanism in practice. More will become known when the preview appears (currently scheduled for November 2023). General availability is scheduled for March 2024.

Update: According to the Microsoft 365 roadmap item, intelligent versioning won’t appear until August 2024.

The Manual Alternative

As noted above, if site administrators believe that a document library needs to use a specific version count (and a new expiration limit), they can opt for manual management instead of automatic versioning.

Intelligent Versioning Needed

Features like Autosave mean that SharePoint Online makes heavier use of versions than the on-premises servers. This factor plus (I assume) pressure on SharePoint storage means that it makes sense to employ a more intelligent management system for versions. No file is worked on in the same way, so taking usage into account seems like the right approach. We’ll see when the preview starts in November.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Preservation Hold Library Now Holds Files With Version History

During a recent content search for some documents, I noticed that the search found far fewer versions of SharePoint Online files than was the case in the past. This is the effect of the change introduced in mid-2022 (or rather, delayed until August 2022) documented in message center notification MC288633 (Microsoft 365 roadmap item 82062). I discussed the potential for the change when Microsoft first announced their intention to implement it in October 2021. Now it’s time to see how things work in practice.

In a nutshell, from the time Microsoft deployed the change to a tenant, SharePoint Online stops storing multiple copies of retained files in the Preservation Hold Library. Instead, SharePoint follows the same approach as taken for regular files stored in document libraries and stores a single file containing its full version history. Files end up in the Preservation Hold Library when SharePoint must retain them because of a retention policy applied to the site, a retention label applied to individual files, or eDiscovery holds. In all cases, files remain in the Preservation Hold Library until the hold applied by retention or eDiscovery lapses.

Site owners don’t have to create the Preservation Hold Library as SharePoint creates it automatically when required. To view Site items in the Preservation Hold Library, add /PreservationHoldLibrary to the site URL in the browser bar.

Difference in File Storage

Figure 1 shows how SharePoint used to store Office documents in the Preservation Hold Library. Each version is stored as a separate file with no versions. If the user updates the file, SharePoint creates a new version in the Preservation Hold Library to capture the changes and make them available for eDiscovery.

The example in Figure 1 comes from January 2022, before Microsoft deployed the change. Examining a more recent entry in the Preservation Hold Library, we see that different versions of the file are available (Figure 2).

It’s easy for changes like this to pass by without being noted, especially when Microsoft delays the deployment of an update for one reason or another (in this case, it was to give customers more time to prepare).

A practical effect of the change is that eDiscovery searches find the latest version of retained files. If investigators want to see previous versions, they can view or restore that version.

Storage Quotas

One of the issues with the way that SharePoint Online captures files for retention is the large amount of storage consumed for this purpose. Given that SharePoint storage is expensive, this is bad news.

In Figure 3, we see that SharePoint uses 20.6 GB or 21.87% of the overall site storage. Even though I know that this site holds many copies of large chapter and book files for the Office 365 for IT Pros eBook, 20.6 GB of retained content is quite a chunk.

The good news is that the advent of multi-version storage seems to have reduced the storage used by retention by a couple of percentage points over what it was in 2021. It’s probably too early to be definite on this point, but the signs are good. My expectations are that the overall storage used by the preservation hold library should reduce over time as older files reach the end of their retention period and SharePoint removes them from the library. We’ll see.

It’s taken me too long to comment on the Preservation Hold Library change. Sometimes life gets so busy that good changes go by ignored. This is a good change.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Find SharePoint Documents to Decrypt Before Tenant Divestiture

A reader wanted to know the best way to find a bunch of files protected by a sensitivity label. The scenario is that the organization had divested an operating division. Sites used by that division had protected files that needed to be decrypted before they moved to a new tenant. If this failed to happen, the protected files would be inaccessible in the new tenant because the users signing into that tenant didn’t have the right to access their content. The question therefore is what’s the best way to find SharePoint documents protected by sensitivity labels so that administrators can remove the labels before the divestiture.

Office documents store label information in their file attributes, so the basic task is to search those attributes to find files protected with one or more specific labels. You could try and do the job with PowerShell and the Graph API. For instance, I have a script to report the files in a SharePoint document library, including the labels assigned to files. Another script uses the Unlock-SPOSensitivityLabelEncryptedFile cmdlet from the SharePoint Online management module to remove labels from documents. The two could be combined to find and remove labels from protected files.

The PowerShell approach is viable if the exercise spans several thousand documents in a few sites. Things become more problematic as the numbers scale up. For instance, sites with document libraries configured to apply default sensitivity labels to new documents (requires Office 365 E5 licenses) could accumulate thousands of protected documents in each library.

Using eDiscovery Searches to Find SharePoint Documents Protected by Sensitivity Labels

eDiscovery searches could solve the problem. Microsoft Purview eDiscovery (Premium) supports finding protected content. The documentation says that files “located on a SharePoint or OneDrive account are searchable and decrypted when the search results are prepared for preview, added to a review set in eDiscovery (Premium), and exported.” Figure 1 shows search preview displaying a protected document found by eDiscovery (Premium).

eDiscovery Premium can’t process documents protected by sensitivity labels with user-defined permissions (permissions assigned by the document author when they apply the label to the document) or when user access granted by the sensitivity label has an expiration date. In addition, eDiscovery Premium can’t decrypt files protected by the Azure Information Protection unified labeling client that are subsequently uploaded to SharePoint Online or OneDrive for Business.

Purview eDiscovery (Standard) and content searches can also find items protected with sensitivity labels. However, these solutions do not decrypt the content unless an unprotected document is an attachment for a protected email. That’s OK, because if you find and export the protected files, an Azure Information Protection (AIP) super-user can remove labels from files using the Set-AIPFileLabel cmdlet from the Azure Information Protection module. Although this is feasible, if you’re contemplating processing thousands of documents, I would buy some Office 365 E5 licenses and use Purview eDiscovery (Premium).

Configuring Content Searches to Find SharePoint Documents with Sensitivity Labels

To search for SharePoint files through Microsoft Search or a Purview content search, you use sensitivity label identifiers (GUIDs). The SharePoint Online search schema includes a managed property called InformationProtectionLabelId, which holds the GUID (identifier) for the sensitivity label assigned to a document. You can use this property to search for documents with a specific sensitivity label in SharePoint search or content searches by using the form InformationProtectionLabelId:GUID. For example, InformationProtectionLabelId:2fe7f66d-096a-469e-835f-595532b63560. The search results are trimmed and only display documents whoever performs the search can access.

An alternative approach is to remap the Sensitivity property, which stores the local language value of the label, to one of the 200 customizable RedefinableString managed properties available in SharePoint Online. This approach allows users to search using label names like “Public” and “Confidential,” but the downside is that it’s possible to assign multiple local language values for sensitivity label display names. If this happens, the searches would need to look for all defined values. By comparison, the identifier is unique and immutable, so using label identifiers is a better choice for search criteria.

To find the label identifiers, connect a PowerShell session to the compliance endpoint and run this command:

Get-Label | Format-Table ImmutableId, DisplayName

ImmutableId                          DisplayName
-----------                          -----------
2fe7f66d-096a-469e-835f-595532b63560 Public
8b652c9a-a8b7-40ec-bb1a-c5334b1b7fef No Encryption
a49e1277-93db-4a2f-8105-43c5196b4fef Non-business use
fb0975b2-1ea1-4c3c-850c-e859e690d282 Partner-Accessible Content
e42fd42e-7240-4df0-9d8f-d14658bcf7ce General Access

Now create a content search and input the label identifier into the search conditions, prefixed with InformationProtectionLabelId, just like shown in Figure 2:

To search for documents with different sensitivity labels, separate the label identifiers with OR. For example, here’s the Keyword Query Language (KQL) query to find documents with either of two labels created between 19 May 2023 and 23 June 2023:

InformationProtectionLabelId:1b070e6f-4b3c-4534-95c4-08335a5ca610 OR InformationProtectionLabelId:2fe7f66d-096a-469e-835f-595532b63560(c:c)(date=2023-05-19..2023-06-23)

Dealing with Protected Content

Searching for protected files isn’t difficult. The real question is what you do with the files that the search uncovers. Having a bunch of encrypted files (with or without the new and improved encryption cipher) isn’t much good unless you can decrypt them. That’s where most of the problems lie, which is why Microsoft might have included the feature in Purview eDiscovery (premium).

Learn about using sensitivity labels, eDiscovery, and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

OneDrive File Type Exclusions Control Synchronization for Sync Clients

Microsoft 365 message center notification MC597037 (updated June 27, 2023) brings news that the OneDrive sync client will display information about files blocked by synchronization by tenant administrators. Worldwide deployment of the updated sync client should finish by mid-July. In the past, users have been left in the dark when they discovered that some files wouldn’t synchronize, but now they can go to the Advanced settings section of the client to see what file types the tenant doesn’t allow them to synchronize.

Oddly, the description for Microsoft 365 roadmap item 124868 takes a different perspective and says:

“This feature will allow you to configure OneDrive Sync Setting to exclude selected files and selected file types from syncing to OneDrive. When available the configuration settings will be located in the OneDrive admin center.”

It seems like a little copy and pasting mistake because it’s long been possible for tenants to exclude file types from synchronization. Microsoft’s documentation explains how to achieve the goal using group policy. It’s also possible to impose a block by running the Set-SPOTenantSyncClientRestriction cmdlet from the SharePoint Online administration module. For example, this command blocks three file types:

Set-SPOTenantSyncClientRestriction  -ExcludedFileExtensions "mp4;rar;zip"

TenantRestrictionEnabled   : False
AllowedDomainList          : {}
BlockMacSync               : False
ExcludedFileExtensions     : {mp4, rar, zip}
OptOutOfGrooveBlock        : True
OptOutOfGrooveSoftBlock    : True
DisableReportProblemDialog : False

Running the Set-SPOTenantSyncClientRestriction cmdlet is the same as blocking file types through the Settings section of the SharePoint Online admin center (Figure 1). Both update the same configuration, which the OneDrive for Business sync client downloads and applies when it synchronizes files from the user’s OneDrive for Business account and whatever SharePoint Online document libraries are synchronized locally.

The Effect of OneDrive File Type Exclusions on Synchronization

Introducing a block on a file type isn’t something to do without thinking. After I ran the cmdlet to block the MP4 file type, my OneDrive for Business client complained bitterly because it could no longer synchronize any Teams meeting recordings and other videos stored in OneDrive (Figure 2).

Teams meeting recordings are possibly a bad example. According to Microsoft, few people go back and view a meeting recording after it is made, which is the reason why Teams applies an expiration tag to recordings after creating the files in OneDrive for Business or SharePoint Online. But I have many other MP4 files for which I want to keep a local copy, so maybe MP4 shouldn’t be on the file exclusion list.

It took the OneDrive for Business sync client several days to recover after updating the SharePoint policy to allow the synchronization of MP4 files, but eventually everything settled down and the client is now happy to process MP4 files again.

OneDrive File Type Exclusions for Personal Sync Client

In any case, file exclusions for OneDrive for Business are old news. What’s new is that Microsoft allows OneDrive Personal users to set their own exclusion list in the latest version of the client (I am using version 23.124.0613.0001). Because the client is for personal use, there’s no system-provided values. Instead, it’s up to the user to input the set of file types they want to exclude through the Advanced Settings section of the client (Figure 3).

File type exclusions are specific to a device rather than an account. You’ll find the information you enter in the client in a text file at

c:\users\<user>\AppData\Local\Microsoft\OneDrive\settings\Personal>odignore.txt.

If you run OneDrive Personal on multiple workstations, you’ll need to configure the settings on all workstations.

OneDrive FIle Type Exclusions are Client-Specific and Don’t Affect the Browser

Th summary is that both the OneDrive for Business and OneDrive Personal sync clients now display details about file types excluded from synchronization. OneDrive for Business users can’t do anything to affect the set of excluded files (except persuade an administrator to change the tenant configuration) while OneDrive Personal users can make their own minds up. In both cases, remember that these settings only affect the OneDrive sync clients. They have no effect on the OneDrive browser client, meaning that users can upload and download whatever OneDrive content they like using a browser.

Learn about using OneDrive for Business and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Managing OneDrive Storage Quotas Through Groups

A reader asked if it is possible to control the assignment of OneDrive for Business storage quotas using groups using a mechanism like group-based license management. The simple answer is that Microsoft 365 doesn’t support such a feature, but like many administrative operations, it’s relatively easy to automate with PowerShell.

Another article covers the basics of reporting and assigning OneDrive storage. OneDrive for Business accounts are personal SharePoint Online sites. Assigning a new storage quota to a user’s OneDrive account is done using the Set-SPOSite cmdlet from the SharePoint Online administration module. This is one of the Microsoft 365 modules that receives frequent updates, so make sure that you use the most recent version. It’s a good idea to check for updates monthly, either manually or using a PowerShell script to process the Microsoft 365 modules typically used by tenant administrators.

Creating a Script to Update OneDrive Storage Quotas

The steps required in the script to update OneDrive storage quotas based on group membership are:

• Connect to SharePoint Online and the Microsoft Graph PowerShell SDK.
• Read information about the target OneDrive storage allocations from some source. I used a CSV file with columns for the group name, group identifier, and storage allocation in megbytes, The names of the columns are group, groupid, and allocation.
• Figure out the service domain for the tenant to calculate the root of OneDrive account URLs. This will be something like: https://office365itpros-my.sharepoint.com/personal/. Later, we combine a modified version of user principal names (replacing dot and @ characters with underscores) to form the URL for each account. An example is https://office365itpros-my.sharepoint.com/personal/James_Ryan_office365itpros_com.
• For each group, get the group members. For each member, figure out the user’s OneDrive account URL and run the Get-SPOSite cmdlet to check its current storage quota. You can use any of the group types supported by Entra ID including dynamic Microsoft 365 groups. With some adjustments to the code, it would also be possible to use an Exchange Online dynamic distribution list.
• If the assigned quota is less than the desired quota, run the Set-SPOSite cmdlet to increase the quota.
• Create a report about what happened (Figure 1).

The script includes nothing complicated in terms of code. You can download the script I wrote from GitHub. Remember that the script is not bulletproof in terms of error handling. Its intention is to prove the principle of what is possible. The script should run without a problem if you sign in with a tenant administrator account. I have not tested the code in an Azure Automation runbook (to run the script on a schedule), but I think that adapting the code for Azure Automation would not be difficult.

Use Azure AD Administrative Units Instead of Groups

Azure AD administrative units are the current flavor of the month in Microsoft Purview with many solutions, including Data loss prevention (DLP) and Data lifecycle management (retention) supporting the use of administrative units to scope policies. If you have the necessary Azure AD Premium licenses, you could use administrative units as the basis for storage assignment.

This article explains how to use PowerShell to retrieve information from administrative units. Instead of fetching a set of user principal names for group members, you’d fetch the same information for the members of an administrative unit, like this:

[array]$GroupMemberUPN = (Get-MgBetaAdministrativeUnitMember -AdministrativeUnitId 150dccad-f8b8-4e54-9246-89834b8b5a25).AdditionalProperties.userPrincipalName

PowerShell Automation Scores Again

It would be nice if Microsoft included group-based OneDrive storage management in SharePoint Online. However, this functionality is probably not high on their priority list for new development. This is yet another example of how PowerShell fills in the cracks and gaps left in Microsoft 365 management and underscores why tenant administrators should have the ability to perform at least simple tasks with PowerShell.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Embedded is a Special Form of SharePoint Online

Updated 18 March 2024

Last March, I wrote about the preview of the much-awaited Loop app, and noted that the objects generated by the Loop app “are stored in hidden SharePoint Online containers with names like https://office365itpros.sharepoint.com/contentstorage/CSP_GUID.” By comparison, the content of Loop components created in apps like Teams chat and Outlook are stored in .loop files in the author’s OneDrive for Business account.

At the time, Microsoft pointed to the support SharePoint Online gave to the Loop app but were non-specific about any detail. Microsoft revealed more information at the BUILD conference, where they said that the Loop app uses Syntex repository services, described as “the fastest way for developers to build and manage file and document centric apps that leverage the rich content platform services of Microsoft 365.” Syntex repository services are in private preview at present. Subsequently, Microsoft renamed Syntex Repository Services to be SharePoint Embedded.

Powered by SharePoint

The developer blog post on the topic gives more information. It seems like Syntex repository services (“powered by SharePoint”) is a document management services Microsoft sells to app developers. Apps developed on the platform store their files in “repository containers,” a SharePoint Online storage partition within a customer tenant (you could think of OneDrive for Business as a SharePoint partition). Repository containers created by apps remain under the control of the app that create containers and cannot be accessed through regular Microsoft 365 interfaces, including administrative interfaces like the SharePoint Online admin center. Apps access their files through Microsoft Graph APIs and present the information through their own GUIs.

The big advantage put forward by Microsoft is that by managing their files through Syntex, app developers get access to Microsoft 365 capabilities like sensitivity labels and eDiscovery in addition to the scalability of the Microsoft cloud platform. In other words, don’t worry about storing files, compliance, and all that stuff because Syntex will do that for you – focus on delivering app functionality to end users. It’s a good example of creating new capabilities off a proven platform.

Loop and Syntex Repository Services

Coming back to Loop, Microsoft asserts that the Loop app is “100% powered by Syntex repository services.” What I thought were hidden SharePoint containers are repository containers that the Loop app accesses to store and manage its workspaces and pages. Microsoft noted that the Designer app also uses Syntex repository services (Figure 1).

We’re still in the early days and everything doesn’t work as promised. Loop is in preview and Microsoft 365 content searches cannot find its content, nor can retention policies target app repository containers. The Loop app doesn’t support assignment of sensitivity labels to workspaces or pages, and data loss prevention policies don’t intervene of someone shares a Loop page link outside the organization. All of this is expected when apps and underlying services are evolving.

SharePoint Embedded is an Interesting Choice for App Developers

If Microsoft delivers what it says will happen, app developers have an interesting choice to make. Should they create everything from scratch and have total control over a solution or use Syntex repository services and get a lot of off-the-shelf functionality. Cost will obviously be part of the equation. There’s no information available about who pays for Syntex repository services when it’s used to host app content.

Backup is another aspect to consider. Microsoft 365 tenants are accustomed to backing up documents and other elements stored in SharePoint Online. How will they backup documents and lists stored in app repository containers? Will backup vendors be able to read a list of app containers in a tenant and stream their contents out to a backup target? This is an example of an issue that needs better understanding before a new service becomes 100% operations-ready.

In passing, I note that the Syntex backup solutions for SharePoint Online, Exchange Online, and OneDrive for Business announced in November 2022 still haven’t made an appearance. Developing and bringing new cloud services into production is hard, even for Microsoft.

Learn more about how the Office 365 applications really work on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Similar but Different to Request Files in OneDrive for Business

In January 2020, I wrote about the feature that allows OneDrive for Business users to ask people to upload files to a folder. Time moves on and message center MC495329 (7 January 2023) announced the arrival of a similar feature for SharePoint Online document libraries. According to Microsoft 365 roadmap it 103625, rollout started in February. It’s taken a while for it to show up in my tenant, or maybe I just haven’t looked hard enough.

In any case, Microsoft says that the feature is “an easy and secure way to request and obtain files from anyone.” Essentially, you select a folder in a document library that you want to use as a target for uploads. You then create a request files link that you give to the people who have the information you want. For instance, these might be professional advisors working on some documents relating to a project. They use the link to upload the files to the target folder, which site members can then interact with as normal.

Any site member can generate a link by selecting the target folder and choosing the Request files option from the […] menu. SharePoint Online generates a link (Figure 1), which the user can share using whatever method they like.

People who upload files don’t have any visibility into site contents and can’t see the files once they upload them to the site. This is a one-way transmission.

Getting SharePoint Online Ready for Request Files

The support documentation for the Request Files feature is available online. I don’t intend to repeat it here. However, some points from the feature documentation deserve emphasis.

First, the Request Files feature depends on Anyone sharing links. If your tenant doesn’t allow people to create Anyone links, they won’t be able to request external people to upload files to a folder. The permissions allowed for the link must include upload rather than just view and edit.

Second, Microsoft checks if Anyone links are enabled in a tenant when they deploy the software update for the Request files feature. If the tenant allows Anyone links, Microsoft enables all sites to support the feature. Originally, my tenant blocked Anyone links, which meant that the default condition applied (disabled) for all sites. After enabling Anyone links, I had to explicitly enable Request files for sites to make the option available.

Other restrictions can interfere with the ability of users to create Request Files links. For example, if you apply the file download block policy to a site, the option to request a link is unavailable.

Apart from enabling Anyone links (through the SharePoint Online admin center), control over how the Request files work is via PowerShell. The Set-SPOTenant cmdlet enables or disables the feature across the entire tenant. This command makes sure that the feature is enabled for the tenant and sets the expiration for request files links to seven days:

Set-SPOTenant -CoreRequestFilesLinkEnabled $False -CoreRequestFilesLinkExpirationInDays 7

While this command disables the feature for a specific site:

$SiteURL = "https://office365itpros.sharepoint.com/sites/SecureSite"
Set-SPOSite -Identity $SiteURL -RequestFilesLinkEnabled $False 

To check the site settings, run:

Get-SPOSite -Identity $SiteURL -Detailed | Select-Object Request*

RequestFilesLinkEnabled RequestFilesLinkExpirationInDays
----------------------- --------------------------------
                  False                                7

Like any change to SharePoint Online settings, it can take up to a day before updates are effective.

By default, the site inherits the value for the link expiration setting from the tenant configuration, but you can define a more restrictive expiration period if you like. You can’t override the tenant configuration and define a less restrictive expiration period for a site. The link expiration period can be anything from 0 (zero) to 730 days (two years). Usually, the more secure the site, the lower the link expiration period.

OneDrive for Business Settings

As noted above, OneDrive for Business also supports the Request Files feature. The OneDriveRequestFilesLinkEnabled setting in the tenant configuration controls if the feature is available in OneDrive for Business accounts while the OneDriveRequestFilesLinkExpirationInDays sets the expiration period for the sharing links. You can’t prohibit Request Files for selected OneDrive for Business accounts. The feature is either enabled or disabled for all.

Set-SPOTenant -OneDriveRequestFilesLinkEnabled $True –OneDriveRequestFilesLinkExpirationInDays 7

Using Request Files

When someone uses a Request Files link, SharePoint redirects them to a special page where they can select files to upload together with some personal details (First and Last Name) to let the requestor know who uploaded files to the folder (Figure 2).

The person who created the request files link receives email from SharePoint when someone uses the link to successfully upload files to the document library (Figure 3).

Figure 4 shows a set of files uploaded to a folder in a document library. SharePoint Online doesn’t validate the details of a person who uploads a file, so the name recorded as a prefix for the filename could be incorrect or false. That’s not important because it’s assumed that the person who requests file uploads will process whatever comes in afterward to decide what’s useful (or not), rename files, and so on.

In terms of tracking the use of the Files Request feature, SharePoint Online captures when a link is used and a file is uploaded in the audit log. This PowerShell code finds the events for the last 14 days and reports them.

Connect-ExchangeOnline
[array]$Records = (Search-UnifiedAuditLog -StartDate (Get-Date).AddDays(-14) -EndDate (Get-Date) -Operations FileRequestUsed, FileUploaded -ResultSize 1000)
If (!($Records)) {Write-Host "No File upload records found - exiting" ; break}

$Report = [System.Collections.Generic.List[Object]]::new()
Write-Host "Processing" $Records.Count "audit records..."
ForEach ($Rec in $Records) {
  $AuditData = ConvertFrom-Json $Rec.Auditdata
  Switch ($AuditData.Operation) {
    "FileUploaded" {
       $FileName  = $AuditData.SourceFileName.SubString(7,($AuditData.SourceFileName.Length-7))
    }
    "FileRequestUsed" {
       $FileName = $Null 
    }
  } # End Switch
  $ReportLine = [PSCustomObject]@{
      TimeStamp    = Get-Date $AuditData.CreationTime -format g
      UploadedBy   = $AuditData.UserId
      Action       = $AuditData.Operation
      ClientIP     = $AuditData.ClientIP
      Folder       = $AuditData.SourceRelativeUrl.Split("/")[1] 
      FileName     = $FileName
      SiteURL      = $AuditData.SiteURL
      Site         = $AuditData.SiteURL.Split("/")[4]           }
  $Report.Add($ReportLine)
  
} #End Foreach Record

# Remove normal uploads
$Report = $Report | Where-Object {$_.UploadedBy -notlike "*@*"}
$Report | Select-Object Timestamp, Site, Folder, FileName  -Unique

Control Over Files Request

Some people might be cautious about using a feature that allows external people to upload files to SharePoint Online. It could, after all, be a vector that an attacker could abuse to upload infected files. On the other hand, is it any more dangerous than asking external people to email attachments to an internal user so that they can upload the files to SharePoint Online.

Control is available by

• Limiting the number of sites that support Files request.
• Limiting the validity of file request links.
• Training users to use the Files Request feature sparingly, and if they use it, they should take the responsibility of restricting access to the upload link and checking whatever files external people upload before making those files available more broadly within the tenant.

Like any new feature, it will take time for tenants to operationalize Files Request. Happy uploading!

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Block Download Policy covered by Syntex-SharePoint Advanced Management License

Microsoft launched the Syntex-SharePoint Advanced Management license into preview in late January 2023. The license is now generally available and cost $3/user/month. Since news about the license emerged, people have been figuring out if the features covered by the license are worth the cost by examining details of the features it enables. Now a new block download file policy is available for Teams meeting recordings.

Blocking Downloads and Teams Meetings

In February, I covered the Block Download Policy for SharePoint Online, a feature in Syntex-SharePoint Advanced Management to limit users to browser access when interacting with content stored in sensitive sites. Blocking downloads for Teams recordings is a similar feature that’s now available in preview. The big difference is that the block download policy applies tenant-wide for all Teams recordings created after the block comes into force in both SharePoint Online sites (for channel meeting recordings) and OneDrive for Business (for personal meeting recordings).

Clearly Microsoft is responding to a customer need to make Teams meeting recording more secure. Blocking downloads removes the worry that someone with access to a recording of a sensitive meeting can download it before the meeting file automatically expires.

Site-Wide Block Download Policy Applied With PowerShell

As noted above, the block is tenant-wide. No GUI is currently available in the SharePoint Online admin center, so management of the block is by running the Set-SPOTenant cmdlet from the SharePoint Online management module.

Make sure that you run an up-to-date version of the module (I used 16.0.23408.12000) as otherwise the Set-SPOTenant won’t support the necessary parameters. Keeping modules like Exchange Online management, Teams, SharePoint Online, and the Microsoft Graph PowerShell SDK up to date is an important task. Ideally, you should check and update modules monthly. As it’s always nice when PowerShell looks after PowerShell, here’s a script to automate the process, including tidying up by removing old module files afterward.

To impose the block, use Set-SPOTenant to set these parameters:

• BlockDownloadFileTypePolicy from $False (the default) to $True.
• BlockDownloadFileTypeIds to “TeamsMeetingRecording.” This is the only value currently supported by the cmdlet.
• ExcludedBlockDownloadGroupIds to the identifiers of security groups whose members you want to exclude from the block download policy. You can’t use Microsoft 365 groups to exclude accounts. This parameter can be left blank if you want the policy to apply to all accounts. If you want to specify multiple security groups, do so in a comma-separated list.

Here’s the command I ran in my tenant to enable the block policy and check its settings afterward:

Set-SPOTenant -BlockDownloadFileTypePolicy $True -BlockDownloadFileTypeIds TeamsMeetingRecording -ExcludedBlockDownloadGroupIds "dc637020-4b0f-4f65-bdf0-3c7dbe8a83e7"

Get-SPOTenant | Format-List BlockDownLoadFile*, ExcludedBlock*

BlockDownloadFileTypePolicy   : True
BlockDownloadFileTypeIds      : {TeamsMeetingRecording}
ExcludedBlockDownloadGroupIds : {dc637020-4b0f-4f65-bdf0-3c7dbe8a83e7}

It can take up to a day before a policy update becomes effective across SharePoint Online. Before it is effective, anyone can download a Teams meeting recording (Figure 1).

When the block download policy is effective, users don’t see the download options for recordings created after the effective date (Figure 2).

It’s important for users to understand that they are only blocked for new recordings. At least, while the feature is in preview. However, when the block download policy is generally available, a background agent will search for older Teams meeting recordings stored in SharePoint Online and OneDrive for Business and mark the files as blocked for download. Although I can see why customers would want this to happen, the fact is that many of the Teams recordings will age out and disappear in a relatively short period unless users take explicit action to retain the files.

Available in Preview Now

SharePoint Online’s block download policy for Teams recordings is available in preview. After Microsoft makes the block download policy generally available, you’ll need to buy some Syntex-SharePoint Advanced Management licenses to continue using the policy or the block download policy will stop working (perhaps much to the relief of some users!).

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Azure AD B2B Collaboration and Guest Accounts for SharePoint Sharing

Two recent message center notifications highlight closer integration between SharePoint Online and Azure AD. MC526130 (11 March) says that new tenants created after March 31, 2023 will automatically enable the SharePoint Online integration with Azure B2B integration. Existing tenants aren’t impacted by this change. The associated update, also scheduled for roll-out in late March, is MC525663 (10 March). The news here is that SharePoint Online site sharing will use the Azure B2B Invitation manager instead of the legacy SharePoint Invitation Manager (Microsoft 365 roadmap item 117557).

Rationalization Around Azure AD

The two updates rationalize existing sharing methods with external users and focus on Azure AD as the driving force for managing invitations. The journey toward Azure AD B2B Collaboration started in 2021, so it’s been a while coming. The project makes a lot of sense for both customers and Microsoft (their gain is through reduced engineering expenses).

Ten years ago, it was reasonable for SharePoint to manage site sharing invitations. Today, when the site collection-based architecture is replaced by single-sites and most sharing occurs through Microsoft 365 groups and Teams, it’s illogical for SharePoint Online to have its own mechanism. 280 million monthly active Teams users create a lot of work for SharePoint.

Another factor is that site sharing with external users is a relatively uncommon action today. Most external users join groups or teams and gain access to the group-connected site. Although non-group connected sites do exist, they’re in the minority and some of those sites (like hub and communication sites) aren’t candidates for sharing with external people. And of course, even site owners might be blocked from sharing sites by a sensitivity label.

Time to Review Applicable Policies

Overall, I don’t think the change will disrupt many organizations. As Microsoft notes “You may want to review your Azure B2B Invitation Manager policies.” Two policies are worthy of note. The first is the Azure B2B Collaboration policy, which includes an allow or deny list (but not both) of domains.

The policy is now found under Collaboration restrictions in the External Identities section of the Azure AD admin center (Figure 1). It is commonly used to block sharing with consumer domains (deny list) or to restrict collaboration to a set of known domains belonging to partner organizations (allow list). If the organization already supports guest accounts, it’s likely that the collaboration policy already exists. Even so, changes like this are useful reminders of the need for regular review of any policy that affects how external people access tenant resources.

Azure AD cross-tenant access policies are a more powerful and flexible mechanism to control external access through both Azure B2B collaboration and Azure AD direct connect (used for Teams shared channels). Cross-tenant access policies are still relatively new and don’t need to be implemented unless required for a specific reason, so your tenant might not use them yet.

Although the Azure AD B2B Collaboration policy is likely to dominate for the immediate future, over time, I expect a slow transition to take advantage of the granular control available in cross-tenant access policies. When an organization changes over, SharePoint Online will take advantage. Leveraging advances made in Azure AD is an excellent reason for SharePoint Online to embrace Azure AD more fully.

Review Guest Accounts Too

Azure AD B2B collaboration works but that doesn’t mean that you don’t need to manage guest accounts. As more sharing happens, more guest accounts end up in your Azure AD. Some guest accounts are used once to share a document. Others are in ongoing use as guest members of groups and teams access shared documents. It’s a good idea to keep an eye on guest accounts and remove them as they become obsolete.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Block Download Policy Licensed by SharePoint Advanced Management and Managed with PowerShell

One of the features covered by the new SharePoint Advanced Management license blocks users from being able to download files from a SharePoint Online site or OneDrive for Business account. The idea is to protect sites that store very confidential material by forcing users to work with the files stored in the site using browsers. Users can’t even use the Office desktop apps because those apps download a temporary copy of files to work on them locally.

The block files from download feature is currently in preview. To enable a block download policy for a site, you’ll need to use the Set-SPOSite cmdlet from the latest version of the SharePoint Online management PowerShell module.

Restricting Download Access

I tested the feature by creating a new team called Project Aurora. I then configured the SharePoint Online site belonging to the team by running these commands to find all sites, select the URL for the Project Aurora site, and use it to configure a block download policy with an exclusion for site owners. In other words, site members can’t download files from its document libraries, but site owners can.

[array]$Sites = Get-SPOSite -Limit All
$Site = ($Sites | Where-Object {$_.Title -eq "Project Aurora"}) | Select-Object -ExpandProperty Url
Set-SPOSite -Identity $Site -BlockDownloadPolicy $True -ExcludeBlockDownloadPolicySiteOwners $True

The preview documentation says that site owners can grant exclusions to groups by passing the group identifiers in the ExcludedBlockDownloadGroupIds parameter. I see some issues here because Microsoft has long coached customers not to update membership of group-connected sites through SharePoint Online. In addition, adding a Microsoft 365 group to site membership creates an unsupported condition of nested Microsoft 365 groups. For now, I would avoid using group-based exclusions and concentrate solely on site owner exclusions.

After populating the default document library with some documents, I signed into the site with a member account. The site flagged the restrictions in place and removed the options to download files (Figure 1).

The Teams Files channel tab also removes the download option but doesn’t display a banner to inform the user about the restrictions. The Files channel tab does remove the option to use an Office desktop app to open a document. Before restricting downloads by policy, Microsoft recommends that you check any potential effect that the block might have on other applications, including Power Apps and Power Automate.

The file download restrictions are the same as when using a conditional access policy to limit access when users attempt to access SharePoint content from an unmanaged device. That’s the point of this feature: you don’t need to deploy conditional access policies to get equivalent protection. Although conditional access policies are a good way to control what people can do after they connect to a Microsoft 365 tenant, there’s no doubt that organizations can end up with many different policies to manage. Replacing a conditional access policy with a relatively simple download block applied at the site level might be a good thing to do, especially if you want to have finer-grained control over what sites block file downloads.

Applying the SharePoint Block Download Policy to Multiple Sites

As a practical example of how you might deploy block download policies, let’s assume that you want to stop downloads for all sites assigned the most stringent sensitivity label. In my tenant, that’s a label called “Confidential Access.” The important thing is to know the label identifier (GUID) because that’s how Microsoft 365 workloads connect to sensitivity labels. In this case, the GUID is c99e52c6-f5ff-4050-9313-ca6a3a35710f.

This script applies the SharePoint block download policy to all sites assigned the Confidential Access sensitivity label. First, we find the set of sites associated with Microsoft 365 groups. Because the Get-SPOSite cmdlet does not return all site properties when it processes multiple sites, we need to loop through the site of sites to check the sensitivity label for each site and apply the policy after detecting a matching label:

# Process sites and set the SharePoint block download policy
[array]$Sites = Get-SPOSite -Template "GROUP#0" -IncludePersonalSite:$False -Limit All
Write-Host ("Scanning {0} sites to find those with the Confidential Access label" -f $Sites.count)
[int]$i = 0
ForEach ($Site in $Sites) {
   $SiteData = Get-SPOSite -Identity $Site.Url
   If ($SiteData.SensitivityLabel -eq "c99e52c6-f5ff-4050-9313-ca6a3a35710f" -and $SiteData.BlockdownloadPolicy -eq $False ) {
      Write-Host ("Applying site download block policy to {0}" -f $SiteData.Title)
      Set-SPOSite -Identity $Site.Url -BlockDownloadPolicy $True -ExcludeBlockDownloadPolicySiteOwners $True; $i++
   }
}
Write-Host ("Finished processing. {0} sites updated with a block download policy" -f $i)

Remember Your Syntex Licenses

Remember that every member of a site that uses a block download policy to restrict downloads to site owners or groups must have a SharePoint Advanced Management license. Given that you’ll probably only apply this kind of restriction to a limited number of sites, that shouldn’t be a big issue.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Syntex-SharePoint Advanced Management Covers Secure Collaboration for SharePoint Online

Updated 2 March 2022

I know that many Microsoft 365 organizations don’t use sensitivity labels, even if they have the necessary licenses to use labels to protect content. All Office 365 licenses allow users to read protected content, but you need Office 365 E3 or above to apply labels to files, and Office 365 E5 or Microsoft 365 Compliance E5 for auto-label processing. At least, that’s been the case up to now.

Applying a default sensitivity label for a SharePoint Online document library (Figure 1) counts as automatic processing. Apparently, Microsoft considers the fact that new and modified documents in the library pick up the sensitivity label (unless previously labeled) as reason enough. In late January 2023, Microsoft revealed that this feature was one of the set to be licensed through a new Microsoft Syntex-SharePoint Advanced Management license.

Features Enabled by the Microsoft Syntex-SharePoint Advanced Management License

The new license is in preview and includes other elements to improve secure collaboration based on SharePoint Online and OneDrive for Business, including:

• Using sensitivity labels with Azure AD authentication contexts to limit access to SharePoint Online sites. This feature has been in preview since 2021.
• Restricting access to a SharePoint Online site to members of a Microsoft 365 group. This restriction blocks users who have received access to a file in the site.
• Blocking the download of files from SharePoint Online sites or OneDrive for Business accounts without the need to use Azure AD conditional access policies. In other words, users are forced to use a browser to access the site or account and cannot download, print, or synchronize files. The restriction also blocks access to the Office desktop apps because these apps need to download files to work on them locally.

In addition, Syntex-SharePoint Advanced Management includes some management and governance features. The three examples cited appear to be instances where it’s possible for administrators to do the same thing with some effort. Microsoft is making it easier. For example, the ability to limit access to OneDrive for Business to those who are members of a specific security group stops people licensed to use OneDrive but who aren’t members of the security group from using the app. The same effect is possible by simply removing the OneDrive service plan from their assigned licenses.

I haven’t seen what actions are included in the feature to export recent SharePoint site actions, but it might be possible to replicate the functionality by fetching SharePoint management events from the unified audit log.

My assumption is that any user who takes advantage of a feature licensed by Syntex advanced management requires a license. For instance, site members of a site where a document library uses a default sensitivity label all require Syntex-SharePoint Advanced Management licenses.

I can’t find a public announcement by Microsoft about the Syntex-SharePoint Advanced Management license. Cynics will say that this is another example of how Microsoft creates licenses for new functionality to generate additional revenue from its installed base. A more benign view is that the new license allows people with Office 365 E3 licenses to use the security and governance features enabled by Syntex Advanced Management. When I find out more details about licensing, including if some features covered by Syntex Advanced Management are also available through other licenses, I shall share the information.

Viewing Metadata for Protected Files

On an associated topic, I was asked why the metadata of documents protected by sensitivity labels remains visible to people who have no right to access these files. It’s a good question because some get confused when they notice an interesting document in a library but can’t open it because they’re blocked by the rights assigned in the label. For instance, who wouldn’t want to open a document with a title like “Proposed Pay Rises for Staff”?

When you enable SharePoint Online and OneDrive for Business to support sensitivity labels, it allows the workloads to deal with protected (encrypted) content. SharePoint Online stores protected files in an unencrypted format to allow functions like indexing and data loss prevention policies to work. Any access to a document, such as a user opening or downloading a file, causes SharePoint Online to encrypt the document so that the application used to open the file (like Word) can apply the rights assigned to the user. Everything works very nicely and those who have access to files can work with that content and those who don’t cannot.

When browsing items in a document library, site members can see metadata like the titles and authors of protected documents. Attempts to open these documents fail if the user doesn’t have the necessary rights. Because SharePoint Online doesn’t encrypt or obscure the metadata, those users know that documents with potentially very interesting content are available.

How SharePoint Online Stores Documents

The reason why document metadata is visible to all site members is rooted in how SharePoint Online stores documents. SharePoint Online uses Azure SQL as its storage platform. Blob storage holds documents and other files while metadata is in a separate table (list). The Azure SQL data is heavily protected against illegal access. Once a user has access to a document library, the assumption is that SharePoint can show them all the items, which is what they see in the list shown in a browser or the Teams files channel tab. It’s only when a user attempts to access a protected document that SharePoint Online validates their right to open that content.

You can argue that SharePoint Online and OneDrive for Business should hide the existence of protected documents that the user can’t open, but this would require SharePoint Online to check that access before displaying documents in a library. Such a check would incur a huge performance penalty because SharePoint Online cannot assume that the rights assigned in a sensitivity label are the same as the last time it checked.

New Functionality, New Costs

Although the news about the Syntex-SharePoint Advanced Management license will disappoint some, it’s reasonable that Microsoft should charge extra for security and management features that not every Microsoft 365 tenant will want or need. Those that need the functionality will simply have to pay the $3/user monthly cost. Hasn’t that always been the way?

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Document Management Par Excellence

Browsing through Twitter (as some still do), I came across a “What is SharePoint” article. Given that I’ve used SharePoint since the initial release of SharePoint Portal Server in 2001, I opened and read the content. (Fun fact: SharePoint originally used the Exchange ESE database engine. The move to SQL happened with SharePoint 2003. Around the same time, the Exchange “Kodiak” project dabbled with the idea of moving to SQL. That project never proceeded).

In any case, the article sets out to explain what SharePoint is and how people use it, which is a worthy purpose. Some good points are made, especially about the transition from the old-style SharePoint to the new UX and architecture. Inevitably, a couple of points of contention exist, so here’s what I think about the role played by SharePoint Online today inside the Microsoft 365 ecosystem.

SharePoint Online Has Always Been Part of Office 365

First, the article asserts that SharePoint Online joined Office 365 in 2012 following the release of SharePoint 2013 Server. This is inaccurate. SharePoint Online has always been part of Office 365 and was included in the beta released in April 2011 and the initial version released on June 28, 2011. Microsoft based the initial release of SharePoint Online on SharePoint 2010 Server. There’s no doubt that the subsequent upgrade to the Wave 14 servers (Exchange 2013 and SharePoint 2013) helped Office 365 enormously, but that came later.

SharePoint’s Toolbox

The article covers the attempts of SharePoint to be all things to all customers by providing features like task management and conversations. One undoubted truth for SharePoint is that it failed to be the “Swiss army knife of collaboration.” That’s a good thing because we learn through failures, and I think SharePoint learned that its strengths are in content management and not collaboration or workflow.

Then again, you can argue a good case that other developments in the Microsoft 365 ecosystem left the capabilities available in SharePoint behind. The big difference between on-premises and the cloud is that on-premises servers are often the fulcrum of a complete ecosystem. Once servers like SharePoint and Exchange become part of a cloud solution, they are no longer at the center and must instead function as a productive part of the ecosystem. Teams, Yammer, and Outlook are better points for collaboration (each with its own strengths). Planner and Project are better at task management, and Power Automate offers better workflow capabilities. A common point is that all these apps contribute to and use services from other apps and Azure, including SharePoint Online. All contribute to the ecosystem, as does SharePoint Online.

Once Teams gathered speed, there was no stopping it, especially after the acceleration in demand for its services during the pandemic. SharePoint Online wisely dropped working on solutions that were never going anywhere and concentrated on what it does best, which is to deliver an enterprise-class document management service to Microsoft 365. After SharePoint focused, its developers were able to exploit other areas based on existing capabilities, like what is now Microsoft Lists.

SharePoint and Teams

I fundamentally disagree with the article’s assertion that SharePoint is the backbone of Microsoft Teams. You could say the same about Azure (Teams uses many Azure services, including Azure Cosmos DB for its message stores), or Exchange (Teams uses Exchange Online for its calendar and to store compliance records). It’s true that every new team comes complete with a new SharePoint Online site. The same is true for private and shared channels, each of which has a site associated with the site belonging to the host team. But this simply reflects an app’s use of SharePoint Online for document management. It’s just like the way Yammer stores documents for its communities.

This brings me to the true backbone of Teams: Microsoft 365 groups. Without the identity management, membership model, and resource provisioning of Groups, Teams wouldn’t work the way the app does today.

In December 2017, I wrote an opinion piece saying that Office 365 Groups saved SharePoint Online. I was wrong: although Outlook groups demonstrated how users could have easy access to SharePoint without having to navigate SharePoint’s browser interface, it was Teams saved SharePoint Online by providing users with a reason to use SharePoint Online. I said “People don’t think about using SharePoint. They think about using Teams, or Planner, or Yammer, or Outlook” and “if they have a file to store, they put it wherever the application dictates, like in the Files section of Teams. It is a natural and easy way for people to use document management and it is the engine driving SharePoint usage. That assertion is truer now than it was in 2017. Accessing SharePoint Online files through the Teams Files channel tab (Figure 1) is an area that Microsoft has improved over the years and is now as functional as the SharePoint browser interface in practical terms.

The growth in Teams to 270 million monthly active users (likely higher now because Microsoft hasn’t updated the figure since January 2022) propelled SharePoint usage to new heights. When Microsoft announced the new Syntex backup solution at Ignite 2022, they said that “Every workday, on average, our customers add over 1.6 billion documents to Microsoft 365.” Those documents go into SharePoint Online sites and OneDrive for Business accounts, and users create many of those files using the connection between Teams and SharePoint Online (here’s Microsoft’s description of that connection).

OneDrive for Business

SharePoint Online deals with business users. OneDrive for Business is the personal side of SharePoint Online. Microsoft uses the consumer version of OneDrive as the document management solution for consumer apps, including Teams Personal.

Microsoft didn’t break out the percentage of the 1.6 billion documents added daily so we don’t know how many ended up in OneDrive for Business. I suspect that the proportion is roughly half and half. OneDrive for Business stores files shared in Teams chat and Outlook messages, including Loop components. It stores user files created in the Documents folder on Windows desktops, and so on. OneDrive for Business is everywhere.

One of the reasons why OneDrive for Business does so well is its excellent sync client. I would not have said that some years ago because the original OneDrive sync client was awful. Synchronization challenges have been encountered and overcome since and the current sync client does a wonderful job of keeping files synchronized across devices. The addition od differential synchronization in 2020 was an important step in this process. I depend on OneDrive synchronization and document auto-save to preserve my work.

SharePoint is a Basic Microsoft 365 Workload

Microsoft considers three workloads to be the foundation of Microsoft 365: Exchange, SharePoint, and Teams. SharePoint Online is the critical document management service for Microsoft 365 and it fulfils that role extremely well. As time passes, the connections and dependencies between the base workloads grow and deepen, something that never happened in the on-premises world.

It’s been interesting to observe the development of SharePoint from a small department-level server to a massive worldwide service for hundreds of millions of users. Many people never realize that they use SharePoint Online because they interact through other apps. That’s just fine. No application is the center of anything these days. Services are what’s important and SharePoint Online delivers a great service, and that’s what’s important.

Move Videos from Azure Storage to SharePoint

Publicized in message center post MC437552 (September 23), the Stream migration tool to move videos from the classic Azure-based storage to SharePoint Online and OneDrive for Business became available in public preview in early October. Microsoft says that they expect the migration tool to be generally available in early 2023.

 Microsoft says that the migration tool will “transfer audio and video files to Stream (on SharePoint) while also bringing over all critical metadata and permissions associated with your Stream (Classic) content. Additionally, the tool will ensure that links and embeds of Stream (Classic) content will be redirected to the same content on Stream (on SharePoint).”

Originally launched in June 2017, Stream Classic took over from the original Office 365 Video portal in early 2020. That lengthy transition testified to the difficulties involved in moving content from one repository to another. The Stream transition to SharePoint Online is quite a strategic move for Microsoft as it further consolidates Microsoft 365 application content within SharePoint Online (and OneDrive for Business). Using a common repository for multiple data types makes it easier to engineer and maintain services like search. Microsoft says that they expect to announce a retirement date for Stream classic in early 2023 and plans to keep the service available for a year afterward.

The Migration Tool

Microsoft uses the Migration Manager tool to migrate videos from Stream classic. The tool is available in the SharePoint Online admin center (Figure 1). It’s the same technology Microsoft uses to move other data sources into SharePoint. Levering a proven migration framework rather than building a bespoke tool is an intelligent approach.

The Migration Process

I don’t intend to go through Microsoft’s step-by-step documentation for the Stream migration process. Instead, I’ll highlight the major phases and note my experience of running a migration. The migration process is divided into three stages:

• Scan: Look for Stream classic videos stored in Azure storage. The videos are categorized as Stream group (a channel), a Microsoft 365 group, or user content. The scan automatically populates the known locations in Stream classic. You can decide to migrate some or all of the locations (Figure 2).
• Prepare migration: After all the Stream classic videos are found, they can be added as migration targets in either SharePoint Online sites or OneDrive for Business accounts. User content videos move to OneDrive while group content moves to SharePoint.
• Migrate: This stage processes the videos by moving them from the Stream classic repository to the nominated targets. During this process, any links and embeds pointing to the old classic locations are redirected to the new locations.

Background jobs perform the scanning to find video files. The time required depends on the load on the service and the number of videos in the various locations. Suffice to say that it can take some time before the scan results are available. Apart from reviewing the results on screen (Figure 3), you can download reports to analyze the information and decide what content to include in the migration.

Adjusting Settings Before Moving Content

One of the big advantages of moving Stream storage to SharePoint is that users can take advantage of larger storage quotas. However, this isn’t a good reason to migrate everything as there’s no doubt that some older video content can probably be left to rot in Stream classic until the service closes down. This is especially true for Teams meeting recordings. When Microsoft introduced automatic expiration for Teams meeting recordings, they noted that 99% of all recordings are not watched again 110 days after the meeting. Applying this rule to old videos of Teams meetings waiting to be migrated seems like a good idea.

Before migration, you can adjust the target location if necessary (Figure 4). For instance, you might decide to bring the videos belonging to multiple Microsoft 365 groups together in a single SharePoint Online site.

Migrating

After making whatever adjustments are required, you can migrate the content. Be sure that you’re ready because there’s no way to reverse course once the migration process moves videos from Stream classic to SharePoint.

Once again, this is a background process that will proceed at its own pace. I launched the job overnight and came back in the morning to find that everything had gone well with only one hiccup (Figure 5).

The hiccup seemed to be a glitch in the migration process. Selecting the line displays a flyout pane with details of the migration. This reported that the migration couldn’t download one video owned by a Microsoft 365 group. However, when I looked at the group through the classic Stream portal, no videos were present and a set of videos were in the destination SharePoint Online site. This is a preview and it is notoriously difficult for migration tools to process 100% of content right exactly right. Even with the glitch, the right outcome occurred as the process moved videos from Stream classic to SharePoint Online.

It’s important to realize that once the migration tool moves videos from Stream classic, it soft-deletes the files to hide them from the Stream classic portal. The soft-deleted files do not appear in the Stream recycle bin. Microsoft will remove the files permanently “180 days after admin disables the tenant but not before classic end of life,” implying that there might be some way to restore videos if necessary.

Don’t Forget the Pre- and Post-Migration Work

Migrating from Stream classic to Stream for SharePoint isn’t just a matter of moving video files. Up-front planning is needed to determine what the target locations should be. Just because a Microsoft 365 group owned some videos in Stream classic doesn’t mean that its SharePoint Online site is the right target location.

Critical metadata moves to SharePoint Online (title, description, thumbnails, transcripts, and permission), but the metadata for some videos might need updating after the transition. Equally, the need might exist to adjust other video settings (like add chapters or set an expiration date) or to edit a transcript. But if you just want to move videos across, the Stream migration tool does a good job.

Keep up to date with developments like the transition to Stream for SharePoint by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Another Functionality Gap Plugged

Microsoft 365 notification MC400977 (updated August 31) covers the introduction of the Recording video feature in the Stream for SharePoint browser client (Microsoft 365 roadmap item 88522). This is part of the work to replace the old Stream classic browser interface by introducing a new Stream portal. In this instance, the upgrade allows users to create 15-minute videos by recording themselves or their screen.

Users in targeted release tenants should now have this functionality. General availability roll-out is ongoing and should be complete by the end of October.

In passing, it’s worth noting that the beta version of the Stream 2.0 for iOS and Android apps (Figure 1) are available for testing. This version allows users to play videos stored in Stream Classic and Stream for SharePoint. Although, the app doesn’t yet support recording, it’s good to see the ecosystem building out.

Recording a Stream Video

Getting back to the Stream for SharePoint browser client, Microsoft says “Users will now be able to use the new Stream camera to record their webcam, record their screen, add edits (think ink, text, backgrounds, and filters) and upload to their OneDrive. Future iterations of the camera will include more features, such as adding music clips.”

Update: the Stream browser app now offers two options for recording: camera and screen.

In other words, Stream can use the technologies built into a workstation to record video (webcam) and screen, and then do some basic editing (some applied before recording starts), before storing everything in OneDrive for Business.

To begin, select the big New recording button in the Stream client. This launches a new browser tab ready to record video. Like Teams, Stream supports background effects (referred to as a backdrop), and offers the set of default background images available in Teams along with background blur and the ability to upload an image. Unfortunately, there doesn’t seem to be a way to save a custom background the way you can with Teams, nor does Stream offer the chance to use any custom background images you’ve already uploaded for Teams. As shown in Figure 2, the same green-screen technique is used to place the user in front of the background image. Interestingly, grab handles are available for the user image to allow the user to drag and place their image anywhere on the recording canvas. They can also resize their image to make it larger or smaller as appropriate to the content being recorded.

You can have great fun playing with the effects built into the Stream camera. Anyone who’s accustomed to working with video apps on mobile phones or other platforms will find nothing challenging here. In my case, I limited myself to moving my picture to the bottom right of the backdrop and inserting some text (Figure 3).

When everything’s ready, click the big round record button. Stream starts a three-second countdown (to settle your nerves) and then starts to record. When you’re finished (or come to the end of the 15 minute maximum supported for recordings), hit the stop button. You now have an opportunity to review what Stream captured (Figure 4). If you’re happy to keep the content, click Publish.

If you have an app like OBS VirtualCam or Snap camera that appears as a valid device camera, you can use these devices instead of a standard webcam.

Recording Files

When it publishes a video, Stream writes it into the top-level of the user’s OneDrive for Business account. It would be nice if Stream allowed you to defined a folder to store these recordings. The files are named after the date and time of the recording, so you end up with files like 20220913_203811 (recorded on 13 September, 2022 at 20:38:11). Files have a .wbem extension, indicating that the files are saved in the WebM format.

Updating Recordings

Once stored in OneDrive for Business, you can update the properties of recordings to generate a transcript and captions, add some text to describe what the video is about, and allow or disable comments (Figure 5), or share the recording with other people.

One thing I do is rename the file to give the recording a title that’s more appropriate to its content. Renaming has a consequence. The Stream client caches information about videos and will continue to display the old file name for a while after the rename happens. Any attempt to access the video at this point will fail because Stream tries to open the file with the old name. However, after a few minutes (or a browser refresh), the cache should catch up with actuality and display the new name.

Relationship with Clipchamp

Microsoft acquired Clipchamp in late 2021. Since then we’ve been waiting to see how Microsoft will make Clipchamp available to Microsoft 365 commercial customers (it’s already included in the Microsoft 365 family and personal plans). It seems reasonable to assume that Microsoft will include Clipchamp Essentials in Office 365 SKUs at some point in the future to allow users to edit the videos they record with Stream (the trim feature available in Stream classic is unavailable for the new Stream) or import from other sources, or indeed stitch segments captured in individual files together to create a longer video.

Stream Continuing to Evolve

Microsoft is making steady progress on the transition to Stream on SharePoint. The new web player is 100% deployed to Office 365 commercial tenants (not yet GCC) to play videos stored in Teams, SharePoint Online, and OneDrive for Business. Being able to record videos is another important part of the puzzle and it’s nice to see that it’s available now.

Keep up to date with developments like the transition from Stream Classic to Stream for SharePoint by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Document Label Mismatches With Sensitivity Labels of Different Priorities

Two years ago, Microsoft launched support for sensitivity labels in SharePoint Online, including the ability to detect a mismatch between the label assigned to an Office document and the label assigned to the site storing the file. The mismatch occurs when the document library has a higher priority than the site label. For instance, someone might upload a document labeled Highly Confidential to a site labelled General Access, or they might update a document to assign it with a label with a higher priority than the site label.

A document label mismatch mightn’t be a problem. Storing sensitive material in a site designated for less sensitive information could be exactly what the user intended to do. However, a mismatch might also create a potential issue when users with access to a site might see highly confidential material. In practical terms, the users might not be able to open the files because they don’t have the necessary rights, but they can see file metadata such as titles, authors, and so on.

Audit Record for Mismatch Missing Important Data

When it detects a document label mismatch, SharePoint Online generates a DocumentSensitivityMismatchDetected audit record in the Office 365 (unified) audit log. The audit record contains information about the:

• The file name.
• The site URL and relative location (full URL).
• Sensitivity label and priority for the document label.
• Sensitivity label and priority for the site label.

The big piece of missing information is the account name (user principal name) of the user who caused the document label mismatch. It’s not as if SharePoint Online doesn’t know who caused the problem. After all, SharePoint Online sends the miscreant an email notification (Figure 1) about the issue to prompt them to consider if a label change is necessary.

Dealing with Missing User Information

The solution exists in other audit data. When someone updates or uploads a document, SharePoint Online captures an audit event for the action. These events capture user information. Later, SharePoint detects the mismatch. SharePoint Online stores documents in lists, and each item in the list has a unique identifier. The identifier is in the audit event for the upload or change. It’s also in the event generated when SharePoint finds the mismatch. Therefore, we can reference the upload/change event to find who created the mismatch.

To illustrate the point, I wrote a PowerShell script to:

• Connect to the compliance endpoint to collect information about the labels used in the tenant.
• Build a hash table of the label identifiers and display names. The audit events log label identifiers, so we can use the hash table to find the display name.
• Search the audit log for FileUpdated, FileModified, and DocumentSensitivityMismatchDetected events. The script looks back over the last 80 days. Given the volume of FileUpdated events often found in tenants, you could reduce this period.
• Split the audit records into those for document mismatches and the other events.
• Create a hash table composed of list identifiers and usernames from the document upload and change events.
• For each of the document mismatch events, lookup the hash table to match against the list identifier and return the username responsible for the mismatch. Also resolve the sensitivity labels assigned to the document and site to the label display names.
• Report the results. Figure 2 shows typical results as viewed through Out-GridView.

The full script is available from GitHub.

Some people like to block the messages sent by SharePoint Online using an Exchange Online mail flow rule so that they can send their own notifications to users. It would be easy to take the report data generated by the script and use that information to create and send appropriate messages, perhaps using the Microsoft Graph PowerShell SDK.

Blocking Email Notifications

To stop SharePoint Online sending emails to advise users about label mismatches, you can update the tenant configuration:

Set-SPOTenant -BlockSendLabelMismatchEmail $True

The setting affects all sites. It isn’t possible to block the notification emails about mismatched labels for selected sites. Blocking emails also stops SharePoint Online writing audit events to record document label mismatches. Microsoft plans to break the link between the two actions so that a tenant can block emails without stopping the creation of the audit records, but no date is available for this update.

Audit Mystery

It’s a mystery why Microsoft decided that the DocumentSensitivityMismatchDetected shouldn’t contain the user information, I see no logic in that decision, but once you know about it, you can compensate. Isn’t PowerShell wonderful?

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Default Sensitivity Label for SharePoint Document Libraries Now Rolling Out Worldwide

Update 2 April 2023

In January 2022, I explained the process to assign a default sensitivity label to a document library in a SharePoint Online site. At the time, Microsoft was in the early days of the feature’s development and the configuration was a very manual process. Now, the public preview software is generally available worldwide. Documentation to set up and use the feature also available.

Setting a default sensitivity label is very simple. Select Library settings from the cogwheel menu and choose the desired sensitivity label (Figure 1). Naturally, you can only select a label that’s configured for file and email protection rather than those set up for container management. If you have multiple document libraries in a site, each library can use a different default sensitivity label. That’s a nice touch because usually if a site has multiple libraries, the libraries serve different purposes, and the chosen label can reflect that purpose instead of being a one-size fits-all selection.

Licensing

Although Microsoft hasn’t confirmed this, assigning a default sensitivity label to a document library will follow the usual line of regarding anything that performs an automatic action as a premium feature. Accordingly, you’ll need Office 365 E5 or Microsoft 365 E5 Compliance licenses to use the feature when it is generally available.

Update: The feature is now available and requires one of these licenses:

• Microsoft Syntex – SharePoint Advanced Management
• Microsoft 365 E5/A5/G5
• Microsoft 365 E5/A5/G5/F5 Compliance
• Microsoft 365 F5 Security and Compliance
• Microsoft 365 E5/A5/G5/F5 Information Protection and Governance
• Office 365 E5/A5/G5

PDF Files and Existing Documents

As Microsoft’s documentation explains, the reference to support for PDF files in the UI is incorrect. New Office documents uploaded by users receive the default label within a few minutes, but PDFs are ignored for now. It’s likely that Microsoft will address this issue when the feature is generally available toward the last quarter of 2022.

New documents that have labels are ignored. Existing documents already present in the library are also ignored. In other words, SharePoint Online doesn’t scan all documents and apply the default sensitivity label to any without an assigned label. However, when users edit Office documents that don’t have an assigned label, SharePoint Online will apply the default sensitivity label defined in the policy applicable to the site. This change is due in mid-October 2022.

More Changes Coming for PDFs

In June, Microsoft announced that Office applications would maintain sensitivity label support when used to create PDFs. This is part of Microsoft’s work to remove the need for organizations to deploy the now-deprecated unified labelling client to apply sensitivity labels to PDFs. According to MC387639, the public preview for this functionality should be available around about now.

An associated message center notification (MC411677, August 10 2022) lets Visual Basic for Applications (VBA) developers know that soon PDFs generated when VBA scripts use Office features will also maintain sensitivity labels for the output files. This is Microsoft 365 roadmap item 93406. Microsoft is warning that some VBA add-ins will need to be updated when the change is effective in December 2022.

Meanwhile, Adobe is running a preview program to allow its Acrobat product to apply, remove, and update sensitivity labels to PDFs. The free Adobe Acrobat DC reader product has been able to read protected PDFs (if the user has the appropriate rights granted by the sensitivity label) for several years. The new functionality is currently understood to be limited to Adobe’s paid-for products.

Sensitivity Labels Increasingly Mainline

It takes time for a new technology to become mainline. Sensitivity labels are getting there. Native (built-in) support for encryption, decryption, and rights management within apps are important steps forward. Office and PDF documents are the most common formats used within Microsoft 365. Their increasing embrace of sensitivity labels makes it easier for people to protect their most sensitive information, and that’s a good thing, even if it makes it a little harder for ISVs to process encrypted user data.

Keep up to date with developments like the app support for sensitivity labels by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

SharePoint External Users From Guest Members and Sharing

A SharePoint external user is someone who doesn’t have an account in your tenant. Because of the influence of Teams, most SharePoint Online external users are guest accounts, created when external people join the membership of Microsoft 365 Groups (teams). If the organization uses the SharePoint Online integration with Azure AD B2B collaboration, SharePoint also creates guest accounts when people share files or folders with external people.

As discussed in this article, it’s reasonably easy to generate a report of the membership of all Microsoft 365 groups in a tenant. The report includes guest accounts and can be used to figure out if guests from the wrong places (like competitors) have access to information in your tenant. However, the script that creates the report relies on cmdlets like Get-UnifiedGroupLinks or Graph API requests to return details of group members, and these exclude any mention of guest accounts in a SharePoint site that aren’t members of the group which owns the site.

PnP Samples Repository

This brings me neatly to a script to report external users posted in the PnP Samples repository (a useful place to go for SharePoint-centric code examples). Reflecting that there are usually multiple ways to solve a problem, three versions are available (CLI for Microsoft 365, SharePoint Online PowerShell module, and PnP PowerShell).

Unhappily, there doesn’t appear to be a good way to retrieve the external users for a site using a Graph API request. You can certainly find the set of all guest accounts in a tenant, or the guest accounts for a team/group, but these methods exclude the guest accounts added for sharing purposes.

The Oddness of Get-SPOExternalUser

The lack of a better method is why the scripts found on the internet use the Get-SPOExternalUser cmdlet. It’s an odd cmdlet in some ways.

For example, Get-SPOExternalUser has a PageSize parameter to limit the number of external users returned. The maximum is 50, which means that if more than this number of external users exist for a site, you must continue fetching until all are retrieved (the Position parameter controls the start of the page of users to fetch). You end up with commands like:

[array]$users = Get-SPOExternalUser -SiteUrl $SiteId -PageSize 50 -Position 50

And after fetching a page of user data, you must combine it with the other pages to get a complete set. Although pagination is common with Graph API requests, it’s unusual to see it used like this with a cmdlet that could surely benefit from a parameter to fetch all matching items, like:

Get-SPOExternalUser -Limit All

Moving onto the output, here’s an example of the data returned for an external user (guest account):

Email         : vasil@michevxx.com
DisplayName   : Vasil Michev (MVP)
UniqueId      : 1003BFFD9AF15B76
AcceptedAs    : vasil@michevxx.com
WhenCreated   : 05/11/2018 18:46:40
InvitedBy     :
LoginName     :
IsCrossTenant : False

As far as I can tell, the InvitedBy and LoginName properties are not used. Across all the sites in my tenant, I found one instance of the InvitedName property being populated. In that case, the property held the user principal name of the guest account, and I couldn’t figure out how this happened.

The AcceptedBy property holds the name of the account that accepted the invitation to the site (to share a document or as a guest member). This property is not populated for sites belonging to shared Teams channels. Instead, a LoginName property captures the account used to connect to the channel site.

The WhenCreated property also deserves some comment. It seems like Microsoft reset this value for many accounts at around 18:46 UTC on 5 November 2018. Many accounts across multiple sites have this creation date. It’s an unnatural concentration of external users created at a specific time on that date. I can’t explain it.

Creating a SharePoint External Users Report

Your account needs to hold the Global tenant administrator or SharePoint administrator role to run this script and generate a SharePoint external users report. The steps are straightforward, which is probably why so many versions are available online. This version captures some extra information about the channel-connected sites used by Teams.

• Find all sites.
• For each site, get its external members.
• Create a report file.

Here’s the script:

$Sites = Get-SPOSite -Limit All | Sort-Object Title

$ExternalSPOUsers = [System.Collections.Generic.List[Object]]::new() 

#Iterate through each site and retrieve external users
$Counter = 0
ForEach ($Site in $Sites) {
    $Counter++
    Write-Host ("Checking Site {0}/{1}: {2}" -f $Counter, $Sites.Count, $Site.Title)
    [array]$SiteUsers = $Null
    $i = 0; $Done = $False
    Do {
      [array]$SUsers = Get-SPOExternalUser -SiteUrl $Site.Url -PageSize 50 -Position $i
      If ($SUsers) { 
        $i = $i + 50
        $SiteUsers = $SiteUsers + $SUsers }
      If ($SUsers.Count -lt 50) {$Done = $True}   
    }  While ($Done -eq $False)

    ForEach ($User in $SiteUsers) {
       $ReportLine    = [PSCustomObject] @{  
         Email        = $User.Email 
         Name         = $User.DisplayName
         Accepted     = $User.AcceptedAs
         Created      = $User.WhenCreated
         SPOUrl       = $Site.Url
         TeamsChannel = $Site.IsTeamsChannelConnected
         ChannelType  = $Site.TeamsChannelType
         CrossTenant  = $User.IsCrossTenant
         LoginName    = $User.LoginName }
        $ExternalSPOUsers.Add($ReportLine) }
} #End ForEach Site

Playing with PSWriteHTML

Now that we have some data to report, I’ll reveal that the real reason for this article is to mention the PSWriteHTML module. The module is maintained by Przemyslaw Klys and its job is to make HTML output easier to generate for PowerShell scripts. The ImportExcel module is another example of a community-created module to help people generate nicer output.

In any case, to create a HTML report, I used these commands:

Import-Module PSWriteHTML.psd1 -Force
$ExternalSPOUsers | Sort Email | Out-HtmlView -HideFooter -Title "SharePoint Online External Users Report"

Figure 1 shows the output, which at first glance looks like a nicer version of the output generated by the Out-GridView cmdlet. The important difference is that you can export the HTML report in different formats, including a nice PDF file.

Having different options to share information is a nice thing. If you create reports from PowerShell, consider having a look at the PSWriteHTML module. It might solve some problems for you. After all, it created a prettier SharePoint External Users report for me!

Learn more about how the Office 365 applications really work on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Introducing the Tenant Admin Namespace for SharePoint Graph Settings

Despite being the two basic Microsoft 365 workloads, one of the notable gaps in Microsoft Graph API coverage has been administrative interfaces for SharePoint Online and Exchange Online. A small but valuable step in the right direction happened with the appearance of the settings resource type in the TenantAdmin namespace. For now, the coverage for tenant settings is sparse and only deals with some of the settings that administrators can manage using the Set-SPOTenant PowerShell cmdlet, but it’s a start, and you can see how Microsoft might develop the namespace to handle programmatic access to settings that currently can only be managed through an admin portal.

Options to Manage SharePoint Online Settings

SharePoint Online tenant-wide settings apply to SharePoint Online sites and OneDrive for Business accounts. Like all Graph APIs, apps must have permissions to be able to make requests. The read-only permission is SharePointTenantSettings.Read.All while you’ll need the SharePointTenantSettings.ReadWrite.All permission to update settings.

Three methods are available to use the new API:

• The Graph Explorer.
• A dedicated app registered in Entra ID
• The Microsoft Graph PowerShell SDK.

The Graph Explorer is acceptable for testing or one-off commands. However, given that the Set-SPOTenant cmdlet is available, it’s unlikely that you’d use the Graph Explorer as your preferred method to update settings.

Creating a dedicated app just to manage SharePoint Online settings is unlikely too unless you use the same app to manage multiple tenants. This points to the most likely use of the TenantAdmin API, which is to allow MSPs to create apps to manage multiple tenants on behalf of customers.

The Microsoft Graph PowerShell SDK could be used to replace the SharePoint Online management module. An organization might want to do this to rationalize the number of PowerShell modules its developers work with and maintain. I can see this happening in the future when Microsoft has developed the TenantAdmin API to match the capabilities available today through the Set-SPOTenant cmdlet. For now, I’d stay with the SharePoint module and keep a close eye on what happens with the API.

Updating SharePoint Online Settings with the Microsoft Graph PowerShell SDK

As an example of using the new API, let’s update the setting controlling Loop components in Microsoft 365 apps. This seems appropriate given the recent appearance of Loop components in OWA. The setting controlling the availability of Loop components is IsLoopEnabled, which is True by default. Here’s the code to retrieve the current setting:

Connect-MgGraph -Scopes SharePointTenantSettings.ReadWrite.All
$Uri = "https://graph.microsoft.com/V1.0/admin/sharepoint/settings"
$SPOSettings = Invoke-MgGraphRequest -Uri $Uri -Method Get
$SPOSettings['IsLoopEnabled']
True

To change the setting to False (and disable Loop components), we use the same URI and run a Patch request. To make the command slightly more interesting, we’ll also update the SharePoint News feed setting at the same time and set a new default time zone for new sites created in the tenant. The time zone for new sites is an example of a setting that cannot be set using the Set-SPOTenant cmdlet. Currently, the time zone can only be set in the SharePoint admin center, so this is an example of how the Graph API will expose new settings.

First, we create a payload object.

$NewSettings = @{
    "isLoopEnabled" = "false"
    "isSharePointNewsFeedEnabled" = "true"
    "tenantDefaultTimezone" = "(UTC) Dublin, Edinburgh, Lisbon, London"
}

Then, we patch the settings.

Invoke-MgGraphRequest -Uri $Uri -Method Patch -Body $NewSettings

SharePoint responds by listing all the settings available to the API: You can see that the two settings have the values contained in the payload.

Name                           Value
----                           -----
isFileActivityNotificationE... True
isCommentingOnSitePagesEnabled True
sharingBlockedDomainList       {Gmail.com}
sharingAllowedDomainList       {hotmail.com, live.com, locklan.com.au, Microsoft.com...}
siteCreationDefaultManagedPath /sites/
deletedUserPersonalSiteRete... 60
isSiteCreationUIEnabled        True
isSyncButtonHiddenOnPersona... False
isSitePagesCreationEnabled     False
tenantDefaultTimezone         (UTC) Dublin, Edinburgh, Lisbon, London
isLoopEnabled                  False
personalSiteDefaultStorageL... 5242880
allowedDomainGuidsForSyncApp   {}
isSiteCreationEnabled          True
availableManagedPathsForSit... {/sites/, /teams/, /containers/}
isResharingByExternalUsersE... False
isSharePointMobileNotificat... True
sharingDomainRestrictionMode   none
sharingCapability              externalUserAndGuestSharing
isMacSyncAppEnabled            True
imageTaggingOption             basic
isUnmanagedSyncAppForTenant... False
isSitesStorageLimitAutomatic   True
isSharePointNewsfeedEnabled    False
excludedFileExtensionsForSy... {*.exe, *.zip, *.rar, *.pst...}
@odata.context                 https://graph.microsoft.com/beta/$metadata#admin/sharepoint/settings/$entity
siteCreationDefaultStorageL... 26214400

Something to Monitor

I suspect that the new API will not be heavily used for now and won’t until it attains feature comparability with the Set-SPOTenant cmdlet. But that’s not the important thing to take away. This is the start of the development of Graph API support for tenant administrative settings, and that’s certainly something to welcome.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Grid View and Details Now Available

One of the things people often forget about Teams is that the app is built from other apps. Some apps, like the Activity Feed and Chat, feature in the navigation rail. Others are limited to a channel, like the Wiki and Files channel tabs. The Wiki tab is losing its default status, meaning that Teams will no longer install it automatically for new channels, but the Files channel tab is going nowhere. In fact, Microsoft is steadily improving its functionality. Given that it’s one of the two default tabs added to all channels, this is a good thing.

The last overhaul was in early 2020, and recently the Files channel tab received another useful upgrade. Microsoft didn’t post an announcement in the Microsoft 365 message center, but the changes are clear and obvious. The possibility might exist that I totally missed the changes when they first appeared, but I thought I’d discuss them anyway.

The Role and Functionality of the Files Tab

The role of the Files channel tab (aka the FileBrowser app) is simple. It allows team members to view the files in the channel folder of the default document library in the SharePoint site belonging to the team. Each channel has its own folder used to store files shared in channel conversations.

The view presented by the Files tab has always lagged the range of options available to users in the SharePoint Online browser interface. In some respects, this didn’t matter. Although SharePoint aficionados know about the missing options, people using Teams might not know that SharePoint manages team files and often didn’t notice. The options available in the Files tab are enough to get the job done, and that’s all that mattered.

But the lack of some features like check-in/check-out bothered some people. The last overhaul addressed many issues, but some remained, like the inability to see the version history and apply retention labels. The latest version of the Files tab (Figure 1) brings some notable enhancements, two of which are particularly worthwhile.

Document Details Pane

First, the details pane is now available for a selected document. This allows you to update metadata for the document and assign a retention label. Apparently, the SharePoint browser interface might soon get the ability to assign a sensitivity label to a document. Today, this can only be done by the Office online and desktop apps (and according to MC395209, soon in the paid-for version of Adobe Acrobat), but when SharePoint gets sensitivity label support, it will probably appear in the details pane and so turn up in the Files tab.

Grid View

SharePoint’s Grid View used to be called Quick Edit. In February 2021, Microsoft renamed the capability to bulk edit items in lists and document libraries. Grid View appeared in the SharePoint browser interface, but not in the Teams Files tab. Now it has turned up, and it works very nicely.

One of the nice things you can do with Grid View is to add a new column easily. For example, the document view we use to organize the chapter files for the Office 365 for IT Pros eBook shows the person who last edited a file but doesn’t show the author responsible for the chapter. Using Grid View (Figure 2), I added a new text column called Author and populated it with the names. The nice thing was that the new column shows up with the populated values in the SharePoint browser interface too.

Not Much Else to Do

I typically use the SharePoint browser interface to work with files. However, the updates to the Files channel tab have made me reconsider whether I should use it. The only thing I use regularly that’s missing is version history, but I only use that feature very occasionally. There’s no sharing option in the view and you can’t add a link to OneDrive. Sharing can be managed through the Details pane (not as easily, but possible) and I seldom create a OneDrive link. So, It seems like Microsoft has not much else to do for the Files tab – except sensitivity labels, of course.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Useful Spreadsheet to Understand Changes

Two weeks ago, I reviewed the new Stream (on SharePoint) client. The client is incomplete, and Microsoft still has lots of work to do to round it out. An online spreadsheet helps customers compare the functionality available in the new Stream client against the classic client, and it’s worthwhile checking out to make sure that your favorite feature is in Microsoft’s plans.

Before reviewing what Microsoft intends to do as it builds out Stream on SharePoint, it’s worthwhile considering the fundamental nature of the change that’s in motion. Microsoft designed Stream classic as a standalone app. The original idea was that Stream could function independently of Office 365 to allow Microsoft to sell it to customers who didn’t have a tenant or allow consumers to use Stream to store video analogous to the way Outlook.com and OneDrive consumer work. Inside Office 365, Stream would replace the old Office 365 Video portal.

Stream Services Rather than a Portal

The app idea is dead. Stream on SharePoint is really a set of services that manage video files stored in SharePoint Online and OneDrive for Business. The obvious manifestation of the services is revealed in the web audio and video player, while the new Stream client is really just an adaptation of the OneDrive client modified to manage video files.

The transition means that there is no video portal. People interact with videos they have access to through the new Stream client, but there’s no organization-wide portal to highlight selected videos or publish material to end users. If organizations want this kind of functionality, they need to use a different approach. For example:

• A SharePoint site tailored to highlight and feature selected videos.
• A channel in a team dedicated to the same purpose.
• Organization videos published through Viva Learning.
• Videos published through a Yammer community.

In other words, there’s more work to do to create an organization-wide video portal. On the plus side, you now have the option to select your preferred approach instead of being limited to a video portal conceived and delivered by Microsoft.

Video Organization

Another related change is in how Stream organizes videos. Stream classic can collect videos into channels and Microsoft 365 groups. These concepts don’t exist in the new Stream because SharePoint organizes its files into sites, libraries, and folders. However, something called SharePoint video collections page is coming soon that appears to take the place of channels. The notion of using Microsoft 365 Groups to organize videos is present because many SharePoint sites are backed by a Microsoft 365 group, but it’s a less direct connection than what happens in Stream classic.

Stream Audit Events

On the compliant front, the Stream-specific audit events logged when users uploaded and viewed videos are no more. Microsoft says that “Audit log schema will change and be logged against the file in ODB or SPO.”

In other words, audit capture for video uploads and other activity is treated in the same way as other SharePoint file operations. When someone uploads a video, SharePoint captures a FileUploaded event; when they modify a video, SharePoint captures a FileModified event. Some Stream-specific events remain, such as those for transcript generation (FileTranscriptCreated), but the majority of the logged events for Stream actions are likely to look like SharePoint file operations.

There’s no harm in using SharePoint file operations audit events until the time comes to extract Stream events from the audit log. You’ll now need to search for the standard SharePoint file operations and then extract the Stream events from that set, probably based on file type (here’s an example PowerShell script I wrote to report Teams meeting recordings).

The problem here is the sheer volume of SharePoint file operations, especially FileModified events. Office documents stored in SharePoint and OneDrive generate vast quantities of these events because of the way the AutoSave feature works, so 5,000 FileModified audit events might only include two or three relevant to Stream. Some won’t care about this change at all, but you will if you use audit events to track video uploads.

Lots of change is in the air. It will be interesting how Microsoft 365 tenants take to the new Stream.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

No Good Way to Clean up Unwanted ex-Teams Channel Folders

Some years ago, I wrote about why Teams leaves the SharePoint folder intact when it deletes a channel. In a nutshell, this is to avoid data loss. The deletion of a channel can be reversed for up to 21 days afterward, and it’s important to be able to restore the complete channel including its files in the Teams channel folder during the 21-day retention period.

But then the question of what happens after the retention period arises. The channel is gone and irrecoverable, but its folder lingers on in SharePoint as a reminder of a now-gone collaborative space. The natural thing for administrators is to clean up the unwanted folder, but that’s not possible because the Delete option is missing from the folder menu. Figure 1 shows an example. There’s no Delete or Move to options in the menu for a folder connected to a Teams channel.

In a Microsoft technical community discussion, some suggest using the Move to option to move the complete folder to somewhere else, like a personal OneDrive account, and delete it from there. Others put their faith in the Move-PnPFolder cmdlet, and some other innovative solutions are offered.

The point is that administrators want to remove the unwanted folders belonging to deleted channels and don’t understand why Teams makes this difficult to do.

Teams Takes Control

After playing around with several channel deletion scenarios in my tenant, it seems to me that when Teams creates a channel folder, it updates the folder properties to remove the options to delete and move the folder. This wasn’t always the case. An unpublicized change seems to have made the change sometime late in 2019. I’ve been able to delete folders belonging to old channels, even immediately after deleting the channel in Teams by following the same approach as used to delete SharePoint folders that have no relationship with Teams:

• Removing all subfolders (delete or move the items).
• Deleting the channel folder.

An example is in Figure 2. In this case, the deletion was of a channel created in November 2016, which is right at the start of the Teams era. The connection with SharePoint Online was looser and this is probably what allowed the deletion to happen. You can see that the menu for this folder includes both Delete and Move to options.

As time went by, Teams became more proscriptive in how it dealt with SharePoint Online. For instance, you can’t modify the settings of the sites used for Teams shared and private channels because Teams will overwrite what you do with settings from the host team.

Prevent Accidents

But I think the reason why Teams doesn’t allow site administrators to delete these folders is to avoid the possibility of data loss both during normal operation and while a deleted channel is in a soft-deleted and restorable state. Removing the Delete option stops an accident happening that leads to data loss and removing the Move to option prevents someone moving files that might be required to restore a deleted channel.

Break the Connection with Teams

What’s missing is a step in the code Teams uses to permanently remove a deleted channel. When the 21-day retention period lapses, Teams cleans up by removing the channel from its soft-deleted cache. It would be good if it also reset the properties of the channel folder to break any connection to Teams and to allow site administrators to manage the folder as normal. In other words, restore the Delete and Move to options in the folder menu and stop telling people they must manage the folder through Teams.

I don’t mind Teams wanting to keep its channel folders under control, but there’s a time to let go, and it’s obvious that Teams hangs on too long in this instance.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Loss of Valuable Feature

In a surprising announcement, Microsoft said in MC394933 (June 24) that they plan to retire the SharePoint Inside Look part of the file preview card. Office 365 tenants will lose the feature starting in late July and the Inside Look will be gone by early August.

I think this is a pity. Inside Look estimates the time required to read a document and an extract of what it believes the author’s key points to be (at a glance – Figure 1). Some background process generated the key points and usually did a reasonable job, at least for documents written in English, which is all the feature supports.

Composing a Custom At a Glance

Sometimes, the process did not work so well, especially for larger documents and no inside look is available. Our main file for the Office 365 for IT Pros eBook is around 33 MB and SharePoint stays mute when it comes to the “at a glance” section. Fortunately, the option exists to create your own “at a glance” by composing three points of up to 100 characters each. The “Edit at a glance” option is available through the […] menu under “See details” in Figure 1, which and reveals an input form to compose the three points (Figure 2).

At a Glance Gone from Sharing Emails Too

In addition, Microsoft is retiring the insertion of the “at a glance” text in the email notification sent when someone shares a document (Figure 3). This is especially regrettable because the text gives recipients some immediate insight into the content within a document.

The Language Issue

Of course, I write documents in English and therefore get value from the feature. The problem might be because Microsoft says that the feature worked exclusively for Word documents written in English. This is surprising because Microsoft certainly has the translation capability to handle other languages. The obvious conclusion is that the issue lies in extracting the three “at a glance” points from the text of a document.

No doubt this is a machine learning task, probably based on something like creating points from sentences at the start of a document where summaries are most often located. I’m sure that scaling this capability up to handle the intricacies of non-English languages plus the resources needed to perform the processing are factors driving Microsoft’s decision to retire the Inside Look feature.

Removing Features is Hard

One thing that’s not clear is if the retirement covers the estimated time necessary to read a document. I’m unsure how Microsoft computes this number but can report that the Office 365 for IT Pros eBook (2022 edition) apparently takes 34 hours to read. Not all at one time, as that would leave you boggle-eyed and incapable of sensible conversation. I’m sure that the computation is based on factors such as the number of pages and words with other influences like the number of paragraphs and headings probably thrown in for good measure.

SharePoint Online has added some good features recently, like the document library drop-down menu and the ability to set a default sensitivity label for a document library (just like you can set a default retention label). It’s a pity to see something like the Inside Look disappear, even if it is English-only and only works for Word documents. Removing features is hard, but the cloud can take away functionality as quickly as new capabilities appear.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Fills Gap in Current Implementation

Updated: August 15, 2022

Message center notification MC393822 (18 June – Microsoft 365 roadmap item 93209, and updated in MC412375 on August 11, 2022) informs tenants about an important change to the way sensitivity label policies apply default sensitivity labels. Up to now, if you define a default label in a policy to apply to documents (Figure 1), SharePoint Online and OneDrive for Business assign the label to new documents created in sites that come within the scope of the policy. MC412375 says that the public preview for the change will roll-out in late August and be available everywhere by mid-October.

Change Applies to Modified Files

The change Microsoft is rolling out in public preview from mid-June is to make sure that when people edit unlabeled (existing) Word, PowerPoint, or Excel files, SharePoint and OneDrive stamp the default label on the file. The functionality already works for the Office web applications and is now extending to Office on Windows and macOS.

Obviously, this is a good change for organizations that want to ensure that all documents have at least a default sensitivity label. Until now, the default label guaranteed that new documents received sensitivity labels, but this left a huge gap in terms of all the files created prior to the implementation of sensitivity labels.

Auto-label policies help close the gap because background processes can scan sites for documents and apply labels to the files if they don’t already have a label. The problem is that auto-label policies are a premium feature. However, if you have the necessary licenses, auto-label policies are a good way to achieve coverage of a large number of preexisting files.

Another change that’s coming soon is the ability to configure a default sensitivity label for a document library, much like you can do with retention labels. Again, this is a premium feature and it’s likely to require Office 365 E5 or Microsoft 365 Compliance E5 licenses.

API to Bulk Apply Sensitivity Labels

One missing piece in the puzzle is the lack of an API to allow organizations and ISVs to create applications to apply sensitivity labels in bulk. Microsoft’s AIP Scanner is an example of such an application. The scanner can apply sensitivity labels to protect information found on file shares or SharePoint on-premises sites. Other use cases include tenant-to-tenant migrations where the need might exist to apply sensitivity labels to a set of documents inherited from a tenant belonging to a company being acquired. There’s nothing off-the-shelf that can handle such a scenario today, and the prospect of having to apply labels manually is unattractive.

Apparently, an API is coming, but it will be a paid-for consumption-based API like that available for Teams Export. In other words, you’ll be able to build an application to apply sensitivity labels to a bunch of files (probably throttled at a certain level to reduce strain on the service), providing you have an Azure subscription to pay the bills.

Making Sensitivity Labels Mainstream

Sensitivity labels are still relatively uncommon inside Office 365 tenants. Microsoft is the only source that can definitively say what percentage of tenants use sensitivity labels or how much of their content have labels. Changes to allow tenants apply sensitivity labels more effectively by default, or to spread sensitivity label support more widely (like the work done to make it easier to protect PDFs) help to encourage more organizations to consider sensitivity labels to be a mainstream part of their overall information protection strategy. However, it’s still going to take time before sensitivity labels become the norm inside Microsoft 365.

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Update the App Tile to Point to the New Stream

Message center notification MC381948 appeared on May 18, but I confess to not having paid much attention to it due to other more important topics. In any case, the notification informs tenants that they can configure the target for the Stream tile in the Office 365 app launcher to direct users to Stream powered by SharePoint rather than the classic Stream. Targeted release tenants should have the update now, with standard release tenants getting it in early July. By late August, all tenants should be able to update the app tile.

Classic Stream stores its video files in Azure blob storage. The plan of record is to move everything to SharePoint-based storage. Personal videos will be in OneDrive for Business while shared (group videos) will be in the document libraries of the SharePoint Online team sites belonging to Microsoft 365 groups (teams). Progression is already obvious as all newly-created Teams meeting recordings are now in the new location. Using SharePoint-based storage means that many features become available for videos, like the application of expiry dates for meeting recordings. In addition, OneDrive for Business offers a lot more storage quota for videos.

Migration a Work in Progress

The big piece of work remaining for Microsoft to do is the migration of old video content from classic Stream. The latest information is that Microsoft has the migration tool in private preview with some customers. Dates for when the migration tool will become generally available are unavailable.

Diverting the App Tile to the New Stream

In the interim, tenants might find it useful to divert users away from the old Stream and have them upload any new video content into OneDrive for Business. And that’s where MC381948 comes in. A new setting is available in the SharePoint admin center to control the behavior of the app tile for Stream. Three values are available

• The default option is to Automatically switch to Stream (on SharePoint). Microsoft controls this option and will set it after the migration of existing Stream content is complete.
• Stream (on SharePoint) directs users to the preview GUI for the new Stream. The user can switch to the classic Stream GUI if they want.
• Stream (Classic) forces people to use the classic Stream GUI.

In Figure 1, I chose to switch to the new Stream. After saving the choice, it takes about ten minutes for the option to ripple across the tenant (and maybe a browser refresh, just in case).

In effect, the target URI for the app tile changes from https://web.microsoftstream.com/ to https://www.office.com/launch/stream. The new GUI (Figure 2) displays any video files found in the user’s OneDrive for Business plus any video attachments for Outlook messages. This ability to highlight video attachments leverages the new messages search vertical and highlights the role of the new Stream in managing video content stored anywhere in Microsoft 365 instead of just in a dedicated repository.

Feel the Power of an Updated App Tile

There’s not much more to say about the new option (nor any nuggets to glean from the Microsoft documentation). On the one hand, it makes sense to begin using the new video storage and management platform for new content. On the other hand, you can argue that it’s best to keep all video content in one place until the migration is ready. The fatal flaw in that argument is the storage of Teams meeting recordings in OneDrive for Business. I switched to embrace change. What will you do with your app tile?

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new developments as they happen.

View Teams and Outlook Messages in Search Results

Microsoft Search and the results it delivers to users are in a state of constant flux. This is usually a good thing because it means that Microsoft is upgrading search capabilities to help users find information more effectively. Sometimes, things get out of step, and you can see extra results in one place that don’t appear in another. A little consideration usually comes up with a reason why this is so.

Take the example of the Messages vertical that Microsoft has added to Office.com. When you search from Office.com, the results include Teams and Outlook messages (Figure 1). In search parlance, the set of results exposed by the messages tab is referred as a “search vertical.” You can add custom search verticals to SharePoint search, but not to Office.com.

The Teams messages come from both chats and channel conversations. Selecting a Teams or Outlook message uses a deeplink to bring you to the source loaded in the Teams client or OWA.

Microsoft Search trims the search results so that users only see information from resources they have permission to access.

Why Messages from Deleted Teams Appear in Search Results

Sometimes search results resurrect messages from deleted groups. Take the second message listed in Figure 1, which comes from a conversation in the Project Athena group (a team). Selecting this message does nothing because it doesn’t have a deeplink to bring it to the source conversation.

Some investigation found that the team doesn’t exist anymore. I deleted the team since the conversation happened in 2018. However, the messages persist because the team came within the scope of a hold imposed by a retention policy. Microsoft Search relies on the compliance records the Microsoft 365 substrate captures for Teams chats and channel conversations, and these records remain in mailboxes until the retention period for the policy lapses. Therefore, the conversation remains available for search to find while the deeplink pointing to the source conversation is unavailable.

Microsoft Search in Bing

The interesting thing is that the ability to return messages in search results isn’t available in SharePoint search. You might expect this to happen because it’s a search for Microsoft 365 data. However, it’s a search of SharePoint resources, so the results only cover the information available to SharePoint Online and OneDrive for Business. Personally, I think Search should deliver the same results in SharePoint Search as it does in Office.com, even if SharePoint Online doesn’t manage the items found. The lines between applications continue to blur and it seems strange to have artificial barriers where they’re not needed.

Where messages do turn up is in search results from Bing.com if you configure Microsoft search in Bing through the Search & Intelligence section of Org settings in the Microsoft 365 admin center. In effect, when you do this, you connect Microsoft 365 content to Bing to expose “work” results alongside results for internet sources. Accessing the work tab exposes results from different Microsoft 365 sources, including messages (Figure 2).

This capability has been available for at least six months. At least, we updated the coverage about Microsoft Search in the Office 365 for IT Pros eBook about six months ago to report its availability!

Loop Components in Search Results

While looking at the various results now available through Microsoft Search, I noticed that Loop components show up. I probably missed this in the past but felt that it’s worth noting that even though Loop components pose some eDiscovery challenges, the information in the components is fully indexed and discoverable as evident in the first two search results shown in Figure 3.

There’s nothing surprising here because the Loop components in Teams chats (and soon in OWA messages) exist as files in OneDrive for Business.

Nice to See Messages in Search

Given the amount of data people now store in the cloud, effective search facilities are increasingly important. Adding the new search vertical for messages to Office.com is very useful. It’s just a pity that the same capabilities aren’t available elsewhere.

Storage and Sites Grow Strongly

At the recent Microsoft 365 Conference (April 5-7, Las Vegas), Microsoft CVP Jeff Teper spoke about Hybrid Workplace Innovations (see the video posted on YouTube). This event is rooted in the SharePoint Conference and the event is heavily supported by the SharePoint development group because of the audience it attracts. In that respect. Teper was talking to base, which he labeled as the “best community in tech.” I guess other communities would debate that point, but when you play to your audience, you need to make them feel good.

In any case, two interesting points about SharePoint Online growth are in the video. First, Teper said that SharePoint Online ingests 100 petabytes of new content monthly (Figure 1). That’s a lot of new storage for Microsoft to add to their data centers every month. But it’s not all documents, and it’s not all for SharePoint Online because OneDrive for Business is in the mix.

Storage Buckets

Microsoft doesn’t say what activity consumes storage. Thinking about where such a large quantity of new content comes from is an interesting exercise. I think the content can be divided into these buckets:

• User activity to create and update documents stored in SharePoint Online sites and OneDrive for Business accounts. This kind of growth is evidence that organizations have moved from older file shares and that people are comfortable with storing their personal files online rather than on local disks.
• Retention policies and labels keeping copies of documents for set periods. Unlike Exchange Online, SharePoint Online charges storage used for retention purposes against organizational storage quotas. In some cases, tenants might use over 20% of their SharePoint storage for retention (Figure 2).
• The effect of Microsoft Lists is probably small on such a large number, but Microsoft has put considerable effort into publicizing Lists and this effort will have some impact, especially for lists storing graphic elements.
• Transition of application storage from Azure to SharePoint Online and OneDrive for Business. Stream is still in mid-transition and Whiteboard is the latest application to move. Teams meeting recordings are the big application consumer of storage at present.
• The storage of Loop components in OneDrive for Business. Because Loop components are available only in Teams chats (soon in OWA), this has minimal impact.

Teams Meeting Recordings

In April 2020, I looked at how much storage a Teams meeting recording consumes and concluded that a recording generates about 7.65 MB per minute. At the time, the videos were in Stream’s Azure-based storage. More recent videos stored in OneDrive for Business appear to consume approximately the same amount of storage.

Microsoft hasn’t given numbers for Teams meeting minutes and users since April 2020 when 75 million Teams users generated 4.1 billion minutes per month, or 55 minutes per user.

The last number for Teams users is 270 million. Applying 55 minutes per user, we get current generation of 14.76 billion. Not every Teams meeting is recorded. Let’s say that 30% are, which gives 4.428 billion minutes requiring 33,210 million MB, or 33.82 petabytes (using this calculator).

Based on these numbers, Teams meeting recordings might account for a third of the monthly storage growth in SharePoint Online. With that kind of consumption, it’s no wonder that Microsoft has implemented a default 120-day expiration period for Teams meeting recordings.

Growth in Sites

The other statistic that took my interest is the creation of 8 million new sites per month (Figure 3). Teper was careful to emphasize that these are active sites and don’t include sites created for testing, development, etc.

Eight million new sites is a big number, but again it’s very understandable in the context of the size of Office 365. The latest Microsoft number for Office 365 users is 345 million paid seats, so eight million is not a big number when placed against the size of the overall base.

The ongoing growth in Teams and the way that Teams creates new sites for each team and for new private and shared channels drives a lot of site creation. Again, Microsoft didn’t give a detailed breakdown of the types of sites that Office 365 tenants are creating so we can only make guesses about what’s happening. What’s for sure is that demand for SharePoint Online services and storage is rising strongly to reflect its status as one of the core workloads in the suite.

Learn about SharePoint Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Just in Time for Outlook

Updated: March 22, 2023

Microsoft Loop components have been available in Teams chat since November 2021. I haven’t heard about widespread usage, but that might be because people need time to adjust their collaboration habits. Access to Loop components in other applications is also a gating factor, but availability in OWA and Outlook for Windows (current channel preview) should help to address this concern. According to MC360766 (April 18, Microsoft 365 roadmap item 93234), Microsoft will roll out this feature to tenants configured for targeted release in early May.

Update: It took a little longer than predicted, but Loop components are now available in OWA.

So far, there’s no sign of Loop components in Outlook desktop, but I’m sure the components will arrive in my email any day now to deliver the same kind of functionality as available in Teams chat (Figure 1). In a nutshell, if an email contains a loop component, it exists as a file in the sender’s OneDrive for Business account that is shared with the email’s recipients. We’ll report more when the software is available.

IsLoopEnabled

This brings me to MC371268 (May 2), where Microsoft announces that “in response to customer feedback,” they’re retiring the existing settings to control the availability of Loop components and introducing a new control called IsLoopEnabled.

The control is part of the SharePoint Online tenant configuration and is set using the Set-SPOTenant cmdlet. You’ll need to upgrade the SharePoint Online management module to version 16.0.22413.12000 or later. Microsoft posted this version in the PowerShell Gallery five days ago. You can install or update the module from the PowerShell gallery or download an MSI file from Microsoft.

The replaced control is IsFluidEnabled, which enables the Fluid Framework within a tenant. Microsoft plans to retire the IsFluidEnabled setting on November 25, 2022. Going forward, the relevant settings in the SharePoint Online configuration are:

• IsLoopEnabled: Controls if Teams can use Loop components. The default is True (Enabled).
• IsCollabMeetingNotesFluidEnabled: Controls if fluid components are available in OneNote collaborate meeting notes.

Update: Following the availability of the preview version of the Loop app, the control for the Loop app, Outlook, Whiteboard, and the Office Online apps is via settings in the Cloud policy.

eDiscovery and Compliance Issues

Although eDiscovery searches can find Loop component files stored in OneDrive for Business, Microsoft acknowledges “limited eDiscovery workflow support.” With the additional of Loop support in Outlook, this aspect might become more problematic. For example, today, the preview feature for search results can render the full content of emails. This isn’t possible when an email contains a loop component because the preview window needs a software upgrade to fetch the content from OneDrive and display it inline within a message.

Another issue is with exports of search results. Today, Microsoft Purview can export emails (and the compliance records captured for Teams chats) found by searches as individual message files or in PST files. Microsoft says that the export format is “not consumable by existing tools,” and that they’re working on “an offline consumable export format.” Taken together, these statements make me think that the exported emails contain references (links) to OneDrive files that aren’t accessible to investigators working offline or independent experts who review eDiscovery results without access to the source tenant.

Making the content of search results available offline probably involves replacing the embedded link in messages containing Loop components with a static version of the content extracted from OneDrive.

This topic deserves a more comprehensive test, which I will get to once Outlook support for Loop components is available. In the meantime, organizations that don’t want to run into potential eDiscovery problems should strongly consider disabling Loop components for both Teams and Outlook by setting the IsLoopEnabled control to False.

Set-SPOTenant -IsLoopEnabled $False

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Advanced Setting Manipulated by PowerShell

For the last year, Microsoft has steadily added to the ability of sensitivity labels to manage different aspects of SharePoint Online sites. Possibly because of a desire not to clutter up sensitivity label settings in the GUI, the developers chose to manage the settings via PowerShell. Adding to the ability to manage the external sharing capability and default link settings, administrators can now control site sharing permissions (a preview feature) via a new advanced sensitivity label setting.

In the SharePoint Online browser interface, this option is available through Site Permissions – Site Sharing (Figure 1).

It’s possible to set site sharing permissions to block all but site owners with PowerShell by running the Set-SPOSite cmdlet with the DisableSharingForNonOwners switch. For example

Set-SPOSite -Identity https://office365itpros.sharepoint.com/sites/Office365Adoption -DisableSharingForNonOwners

However, the Set-SPOSite cmdlet doesn’t allow administrators to enable site sharing for non-owners. It’s a very simple off switch that cannot go back or set site sharing permissions to the option where only site owners can share the site. The new capability for sensitivity labels delivers a way to address these shortcomings, but only for sites assigned sensitivity labels with the advanced setting defined.

Available Site Sharing Permissions

Three site sharing permissions settings are available (the descriptions are from the GUI shown in Figure 1):

• MemberShareAll: Site owners and members can share files, folders, and the site. People with edit permissions can share files and folders. This is usually the default setting assigned to new sites.
• MemberShareFileAndFolder: Site owners and members, and people with edit permissions, can share files and folders, but only the site owners can share the site.
• MemberShareNone: Only site owners can share files, folders, and the site.

Updating the Site Sharing Permission

To assign a new site sharing permission, connect to the compliance endpoint by first connecting to Exchange Online (Connect-ExchangeOnline cmdlet) and then running the Connect-IPPSSession cmdlet. You then have access to the compliance cmdlets and can run the Set-Label cmdlet to update the MembersCanShare advanced setting. For example:

Set-Label -Identity 'General Access' -AdvancedSettings @{MembersCanShare= 'MemberShareFileAndFolder'}

To ensure that the update worked, run the Get-Label cmdlet:

Get-Label -Identity "General Access" | Select-Object -ExpandProperty Settings

[contenttype, Site, UnifiedGroup]
[tooltip, General access to information in a team, group, or site that's available to anyone in the organization plus guest members.]
[displayname, General Access]
[memberscanshare, MemberShareFileAndFolder]

Note that the Get-Label cmdlet only lists advanced settings that apply to a sensitivity label. For instance, the external sharing capability setting doesn’t appear here because it is not set for the General Access label.

Wait and Verify

The new label setting must propagate to SharePoint Online before it applies to the sites assigned the sensitivity label. The synchronization process usually takes about 24 hours, but it can take longer. After waiting for a day or so, to verify that the change worked, select a site with the sensitivity label you updated and check its site sharing permissions. Because we selected ‘MemberShareFileAndFolder’ as the value for the setting, you should see permissions as shown in Figure 2.

If the permission doesn’t show up as expected, check that the label settings are correct and wait another day before checking again. If nothing budges after a week, it’s time to seek assistance from Microsoft Support.

GUI Updates Take Time

Some will ask why Microsoft doesn’t expose advanced sensitivity label settings in the (now renamed) Microsoft Purview compliance portal. After all, many settings are managed through sensitivity labels in the GUI, including external sharing capability (Figure 3). This setting was originally only settable through PowerShell.

Although I don’t know for sure, I suspect that the answer is “development time.” In other words, after a new sensitivity label setting becomes generally available, extra development effort is necessary to update the GUI and make sure that everything works as it should. Patience is a virtue…

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

And the Joined Teams Feature That Should Have Been Available Sooner

Sometimes Microsoft releases a feature that is so obviously useful that you wonder why it never existed earlier. SharePoint’s document library dropdown menu is in this category. The unfortunate thing is that the feature has arrived too late.

Announced in MC301473 (updated February 3, Microsoft 365 Roadmap item 81990), the dropdown menu is generally available. When a site contains multiple document libraries. You see a dropdown menu to navigate between the different libraries. Take the example in Figure 1. We’re positioned in the default Documents library, but the site has four other document libraries. The dropdown menu makes it very easy to navigate from one library to another.

Will People Use the Document Library Dropdown Menu?

Despite the dropdown menu being generally available, my guess is that many people will never use the menu. This feeling is based on the experience that most SharePoint Online sites created today are linked to Teams and only have the default document library. Although the dropdown menu is available in these sites, it’s not very useful (Figure 2). It would be better if SharePoint Online hid the menu in these sites.

The other thing is that if a large proportion of SharePoint activity is via Teams, people don’t ever go near the browser interface and therefore won’t see the document library dropdown. MC301473  says that Microsoft is planning work to bring the dropdown to Teams, probably with an update to the Files channel tab. They haven’t shared a date for this work, but when it is delivered, Teams users will be able to switch between the available document libraries in a team-enabled site.

Teams Admin Center Shows User-Joined Teams

The Teams feature announced in MC332869 (February 15, Microsoft 365 roadmap item 87969) is another example of something that should have been in a product a long time ago. In this case, the message center notification tells us that the Teams admin center can now show the list of teams a user account is part of (Figure 3).

This is welcome news because there are many examples where tenants have written their own reports to list what teams (or Microsoft 365 groups) people belong to (here’s my version of a PowerShell script to create such a report). In fact, some like the idea of being able to print off the membership of a specific group or team (so here’s a script to do that job).

My point is that the necessary support to list the set of groups or teams a user belongs to has been around in the Graph transitiveMemberOf query for quite a while. Shown below is a simple code example which uses the Get-MgUser and Invoke-MgGraphRequest cmdlets from the Microsoft Graph PowerShell SDK to retrieve the set of groups a user belongs to. A filter extracts the set of teams, which we can then list. In the Teams admin center implementation, the code uses the team identifier to fetch details like its privacy, active state, and so on for display.

$User = Get-MgUser -UserId James.Ryan@office365itpros.com
$Uri = "https://graph.microsoft.com/v1.0/users/" + $User.Id + "/transitiveMemberOf"
[array]$UserGroups = Invoke-MgGraphRequest -Uri $Uri -Method Get
[array]$UserTeams = $UserGroups.Value | ? {$_.resourceProvisioningOptions -eq "Team"}

$UserTeams | ForEach{Write-Host $_.Id, $_.DisplayName}
34d68904-9d7c-4ef7-b715-eed283e80243 Industry News
c055da06-f21d-4381-9c51-f5a239d36329 Plastic Production (Team)
204e3211-4d07-4fde-95f9-227a8827742d Organization Planning (Team)
18aa8f1b-3bdf-41f7-b14b-a3be217478e8 Baden Workers
5348781d-52a8-490f-b75b-a72e702114d1 PL Test

In any case, the feature is now rolled out and available in your closest Teams admin center.

Fit and Finish Features

Microsoft will likely say that they work on features in priority order and items like a document library dropdown menu are not high on that list. That’s true, and it’s unfortunate when features show up long after they would have been useful.

Small but important features like the document library dropdown menu and revealing the set of teams someone belongs to are fit and finish items. No one will decide to use SharePoint Online or Teams because these features exist, but current users of these products will like that they do.

Keep up to date with new features in SharePoint Online and Teams by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Perhaps Not the Biggest Problem for OneDrive to Solve

Featured in the set of OneDrive announcements at the Microsoft Ignite conference in November 2021, the ability to move a OneDrive shortcut from the Files root to a public or shared folder is now rolling out. The change is described in message center notification MC316147, first published on January 19 and updated on March 4.

The original announcement limited movement to private folders, and this is also the case in Microsoft 365 roadmap item 82166. However, something obviously changed since November because MC317147 explicitly states “when moving a shortcut to a folder into a shared folder, the short cut does not change its sharing permissions. People who don’t currently have access to the shortcut won’t be able to access its content but can rename or remove the shortcut.”

OneDrive Shortcuts

Originally launched in 2020, OneDrive shortcuts are a useful way to add pointers to folders that users commonly access so that they appear in OneDrive for Business. The shortcuts might be to folders in SharePoint Online document libraries or other OneDrive folders. When OneDrive shortcuts first appeared, I thought they were pretty good and used them for a while, but then I ran into a problem that still lingers today.

The OneDrive sync client is a critical component for users who keep files in the cloud. The sync client synchronizes files from cloud folders to local copies, and that updates made to the local copies synchronize back to the cloud. The original OneDrive sync client (Groove.exe) wasn’t very good, but a rewrite to create a new client fixed the problems and the current client is very stable. Interestingly, while the OneDrive sync client takes care of synchronization for non-Office files, to enable features like autosave and co-authoring, the Office apps perform the synchronization when actively working on documents.

The Office 365 for IT Pros eBook team depend heavily on the OneDrive sync client to synchronize changes made to the source Word documents used for book chapters. These and other files for the book are stored in a SharePoint Online document library. The OneDrive sync client makes sure that changes made by authors on Windows and Mac workstations synchronize with SharePoint Online.

Synchronization Problems with OneDrive Shortcuts

Which brings me to the synchronization problem with OneDrive shortcuts which stop me using shortcuts. Everything works well if you create OneDrive shortcuts and then set up synchronization with SharePoint Online. However, if you use the OneDrive sync client to synchronize both OneDrive for Business and SharePoint Online folders and then add a OneDrive shortcut to a folder in the same document library, it creates a sync issue.

Figure 1 shows a SharePoint Online folder in a document library. I don’t synchronize this folder to my workstation because it contains large book files. However, I synchronize other folders from the library. I also synchronize my OneDrive for Business account.

If I take the option to add a shortcut to OneDrive, SharePoint Online creates the shortcut and adds it to OneDrive for Business (Figure 2). Everything looks good and I can use the shortcut to access the files in the SharePoint Online folder.

However, the OneDrive sync client reports that it has a sync issue (Figure 3) saying that it cannot sync the shortcut because it conflicts with other folders. The client reports that the fix is to stop syncing two folders, both of which come from the same SharePoint Online document library.

The sync client offers to fix the problem by unsynchronizing the conflicting folder. Do not do this. The action breaks the connection between the local copy on the workstation and the cloud files, which means that you’ll have to re-establish synchronization afterwards, which could involve a lot of work to make sure that local copies are accurate.

However, the issue is only a warning about a single file (the OneDrive shortcut) and doesn’t affect synchronization for any other file. Changes made locally continue to upload to the cloud and updates made to cloud files by other workstations flow down to the local copy on my workstation.

The solution is simple. Go back to OneDrive for Business and remove the offending shortcut. The sync client is happy immediately and the warning disappears.

The problem doesn’t occur if you create a OneDrive shortcut to a SharePoint Online folder when no folders from that document library are synchronized. However, if you attempt to synchronize a folder from the document library, OneDrive fails and says that it can’t synchronize the folder because you’re already syncing a shortcut to a folder from this shared library (Figure 4).

I can’t imagine that this is the kind of experience that Microsoft would design into OneDrive shortcuts. What’s more, the problem has been in place since the introduction of shortcuts, so perhaps no one has complained too much.

Moving of Shortcuts Not The Biggest Problem

The clash between OneDrive synchronization and OneDrive shortcuts is the reason why I won’t use shortcuts. Although it’s great that Microsoft has done the work to make it possible to move shortcuts, it’s odd that they haven’t sorted out the obvious clash between two OneDrive components. When they do, I’ll consider using shortcuts again.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

It’s All About the Substrate

I must be slowing down. At least, that’s the thought which ran through my mind as I tried to make sense of Microsoft’s post about SharePoint Online proxy addresses and Exchange Online mailboxes. Specifically, I couldn’t understand this sentence “To ingest SharePoint Online content into a mailbox, we establish SharePoint Online routing information to the mailbox.” This sounds awfully like the way site mailboxes worked, but thankfully those abominations are long gone. And then I realized that the text wasn’t as clear or precise as it could have been, despite discussing an interesting aspect of the Microsoft 365 ecosystem. Here’s what I think Microsoft meant to say.

The Microsoft Substrate and Digital Twins

As anyone who’s listened to Microsoft Fellow Jeffrey Snover talk about the Microsoft 365 substrate knows, the substrate plays a key role in making Microsoft 365 shared services work. The substrate is what captures compliance records for Teams, Planner, and Yammer. It handles the ingestion of audit records generated by multiple workloads. And the substrate creates “digital twins” of SharePoint Online and OneDrive for Business documents and lists. A digital twin is not necessarily a full copy of an item; it’s enough to allow shared processes to operate against the data. If access is required to the complete data, a link redirects to the owning workload.

The substrate does this work because assembling digital twins gathered from across Microsoft 365 workloads into one place makes it much easier for shared services like compliance processing or search to operate. Instead of a service needing to communicate with multiple repositories, it needs to deal with one. And the physical representation of that repository is a special form of Exchange Online mailboxes.

SharePoint Online Proxy Addresses

Which brings me back to the subject of the blog point: the SPO (SharePoint Online) proxy addresses stamped on user mailboxes. If you examine a mailbox, you see the proxy addresses assigned to the mailbox. For example, four proxy addresses exist for this mailbox:

DisplayName    : Steve Gippy (Operations)
EmailAddresses : {SPO:SPO_20876de2-3b1c-44ce-8773-34499caaa16c@SPO_a662313f-14fc-43a2-9a7a-d2e27f4f3478, 
SIP:steve.gippy@office365itpros.com, 
SMTP:Steve.Gippy@office365itpros.com, 
smtp:Steve.Gippy@office365itpros.onmicrosoft.com}

One is the primary SMTP address used for email routing (the one with capitalized SMTP), another is a secondary SMTP address belonging to the service domain for the tenant. Then there’s the SIP address used by Teams for calls and meetings. And finally, there’s SPO, the SharePoint Online proxy address, which means nothing to anyone because this address is created and maintained by background Microsoft 365 processes. The address includes a unique identifier for the user and the tenant identifier.

As the post says, administrators should leave the SPO addresses alone as “several internal cloud processes rely on them” not to mention that “Admins should never modify the SharePoint Online proxy address as it is an internal Microsoft service concept.” In other words, keep your greasy hands away from SPO proxy addresses. If you don’t, things break, and you won’t be able to fix them. In fact, you probably won’t know what broke and where it broke.

Without the SharePoint Online proxy address in place, the link between Exchange Online and SharePoint Online is broken, and the substrate can’t ingest digital twins from SharePoint Online into Exchange Online. In other words, the SharePoint Online proxy address stamped on user mailboxes is a connection back to SharePoint Online (and OneDrive for Business).

Hard and Soft Deletes

Now the opening of the post makes sense. It discusses why administrators see mailbox objects they believe are permanently removed (hard deleted) persist in a recoverable (soft deleted) state. After all, if you run the Remove-Mailbox cmdlet and use the PermanentlyDelete switch to tell Exchange Online to erase all trace of a mailbox, you’d like to think that the service would do your bidding.

But because Exchange Online is the foundation for the Microsoft 365 substrate, it has more to do than simply blow away a mailbox. In particular, because the search results generated by Microsoft search depend on mailbox content, some adjustment is necessary to reflect a mailbox deletion. That’s why Exchange Online signals SharePoint Online so that background processing can adjust the search results shown to users. While this processing proceeds, it’s possible to see erroneous results featuring a deleted user, but eventually processing completes and search is 100% accurate again.

Exchange Online keeps the mailbox in a soft-deleted state until the deleted mailbox retention period expires (183 days). By then, background processes have adjusted indexes and SharePoint Online is content. Exchange Online can then tidy up by hard-deleting the mailbox, unless of course it’s under the control of a retention hold (litigation hold or otherwise), in which case the mailbox is inactive and kept until all retention holds expire.

Life is More Complicated in the Cloud

All of this proves that cloud objects lead a more complicated existence than on-premises objects. The Microsoft 365 substrate connects objects together in a way that simply doesn’t exist on-premises, so when you remove an object, it might just have an effect elsewhere that must be dealt with. Which is why some mailboxes that you might want to hard delete have to stay soft-deleted until background processes can adjust connections.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Saturated Colors and New Designs for Teams Emojis – Windows Emojis Useful for Channel Naming Too

Last year, Microsoft refreshed the set of emojis designed for use in its applications, lauding the “bright, saturated colors and bold forms” available for more than 1,800 emojis. The new set is already available in Windows and OWA, with plans in place to bring the new emojis to Yammer and other places within Microsoft 365 during 2022. Of course, you don’t need an explicit insert emoji option to use the Windows emojis. In any editor and in most places where you can input characters, just press the Windows key and period and you should be able to browse the emoji set and insert whatever you like.

New Animated Emojis for Teams

Which brings me neatly to the latest Teams public preview feature: Fluent emojis (“a modern and delightful new version of the emojis we use every day” are available for use in Teams chat and channel conversations. The new emojis replace the set last refreshed in April 2021, which increased the number of available emojis to 800. Adding another thousand is an impressive graphical feat, and Microsoft says:

• Emojis in chat and channels messaging will update to the new Fluent style, along with delightful animations for applicable emojis.
• Reactions in chat, channels, and live meeting reactions will update to the new Fluent style.

Animations are the difference between the Windows emojis and those now available in Teams, which is why I guess some refer to 2D emojis (Windows) and 3D (Teams). With so many emojis to choose from (Figure 1), no doubt some conversations will now be conducted entirely through emojis (I’m not kidding).

Everyone will have their own favorite emoji. Mine is surely a candidate to become a new favorite to communicate a state of mind or opinion of a piece of work. The pile of poo emoji looks innocuous when inserted into a message (Figure 2), but the brown smelly material becomes animated when viewed by the recipient. It makes you think about the thought process necessary to come up with a suitable animation for such an object…

Not every emoji is animated, and some animations are very subtle (like the moving legs on the prawn emoji). You can see which emojis are animated by hovering over emojis when browsing the set in the picker.

The new emojis are available for Teams desktop, browser, and mobile clients, but user accounts must be enabled for Teams preview before they’ll be able to add the new emojis to chats and channel messages.

Using Emojis as Chat Reactions

According to MC296204 (updated February 8), people will soon be able to use the new emojis as reactions to chat messages. Microsoft 365 roadmap item 88080 says that users will be able to select any emoji as a reaction to a chat message (but not a channel conversation). The roadmap item lists review for April 2022 and general availability starting for standard release tenants in May 2022.

Update: According a May 26 update to MC296204, Microsoft is pausing plans for this update to make some changes.

Emojis in Channel Names

Another good use of emojis is to highlight important channels. Given that an individual team can have up to 200 regular channels and 30 private channels (with shared channels coming soon), there’s no doubt that it’s easy to overlook channels in a list, even when team owners pay attention to giving channels appropriate and helpful names.

Increasingly, I see people using emojis in channel names. Usually, the emoji is at the start of the name to make it more visible and catch the eye, but I’ve seen emojis placed at the end too. For instance, the channel for discussions about the current version of the Office 365 for IT Pros eBook has a jolly roger (skull and crossbones) emoji to display its important status (Figure 3).

You can’t use Teams emojis in channel names but the Windows emojis are available, so there’s no shortage of choice. In any case, if you use emojis in channel names, it’s best to put the emoji at the start of the channel name rather than the end to make sure that the selected highlight is always visible. As evident in Figure 4, channels highlighted in this manner stand out from other channels.

You can also include emojis in the display names of teams and Microsoft 365 groups, but maybe that’s going too far.

Ensuring SharePoint Online and Teams Agree

In the past, renaming a channel has wreaked havoc with the folder in the SharePoint document library associated with the channel. This month, Microsoft released the update titled Pairing naming convention between Teams channels and corresponding SharePoint folders (described in MC306666, December 18, Microsoft 365 roadmap item 72211).

Microsoft has been working on the issue for some time and originally planned to release the update last year. The fix makes sure that the SharePoint folder gets the same name following the rename of a regular channel. For private channels, both the site name and folder are updated. All of which means that you can include emojis in channel names without any qualms.

Keep up to date with even small changes by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Search in Outlook Has Never Been Great

On January 11, MC313286 brought the news that Outlook searches might return no result if messages are stored in PST and OST files. I’ve zero sympathy for those who store email in PST files, but the loss of search in OST files handicaps offline operation for those of us who keep email in Exchange Online mailboxes. I realize that some persist in using POP3 and IMAP4 to access mailboxes (hopefully, the loss of basic authentication in October 2022 will stop this), but it’s time to move on use more modern messaging protocols.

In any case, the problem affects people who upgrade PCs to Windows 11 because the upgrade removes the search index. Over time, Windows rebuilds the search index, and all is well. At least, it’s as well as Outlook searches ever are. Over the years, my expectation that Outlook delivers reliable search results has never been high, so my level of disappointment is never severe. To be fair, searches performed by latest version of Outlook desktop (click to run) are better than before, but force of habit makes me depend on OWA when I need to search for something.

New Search Capabilities Include Outlook and Teams

Behind the scenes, Microsoft Search powers the search facilities in Outlook and OWA. Microsoft Search indexes and can search the Microsoft 365 substrate, meaning that it can find documents, email, tasks, and the compliance items for Teams, Planner, and Yammer. Recently, Microsoft upgraded the search UI in Office.com and SharePoint Online to add a “Conversations” tab to search results. This tab reveals Teams and Exchange Online messages (Figure 1) while other tabs deal with news, people, sites, files, and so on. The change is documented in MC299210 (last updated December 8) and Microsoft 365 roadmap item 68779.

If you select an item, a deeplink takes you to the original message in the underlying workload. For example, if you find a Teams message you want to see, the deeplink offers to open the Teams browser client but will open the item in the desktop client if that client is available. Outlook items open in OWA.

According to the roadmap item, the new search became generally available in January 2022. It should therefore be available in all tenants now.

Microsoft 365 Search in Bing Now Covers Outlook

The roadmap item refers to Bing.com too, which covers the scenario when Microsoft 365 results are integrated with results from Bing searches. It’s long been possible to see Teams and Yammer messages in Bing results. Now Outlook messages are included (Figure 2). As in other features powered by Microsoft Search, filters make sure that the person performing the search only sees the information they can access. This means that a search covers the user’s own mailbox but won’t reveal items in shared mailboxes or other user mailboxes they have delegate access to.

The presentation of Outlook content differs in Bing. In the past, Bing had a Conversations tab covering Teams messages and Yammer. Now, Teams and Outlook show up under Messages and Yammer is moved out to its own tab. I’ve heard speculation that this is because Yammer messages are slower to index. Curiously, the search results available in neither SharePoint Online nor Office.com include Yammer content, so perhaps Microsoft is doing some work to integrate Yammer better.

Integrated View is Best

The obvious advantage of using Office.com or SharePoint Online for searching is access to integrated results. OWA delivers good results for Outlook messages. However, given that we live in a world where communications aren’t restricted to email, the integrated search across SharePoint, OneDrive, Teams, and Outlook is very attractive. It’s now my favorite way to look for Microsoft 365 content.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Potentially a Play to Extract More Revenue from OneDrive Consumer

Being mostly concerned with the happenings in Office 365, our team doesn’t usually take much notice of developments in the consumer side of Microsoft. However, sometimes developments happen which are worth noting, especially when Microsoft marketing is excited about an announcement. Take January 31 for instance, when the avuncular Mark Kashman handcrafted text to announce the preview of Microsoft Lists for MSA. In other words, you can use your consumer Microsoft account to work with a “lightweight version of the Microsoft Lists app designed for small business and individual use.” All good, if you’re one of first 200,000 Microsoft account holders to head to the preview page to try out lightweight lists on a first-come, first-served basis.

Off I headed to lists.live.com to see what all the fuss was about. And I can report that it is possible to create a Microsoft list using a Microsoft Service account (Figure 1).

The process is painless, won’t kill any brain cells, and works like it does in the enterprise version. At least, it does from the user perspective. Those who do interesting and skillful things with Lists using Power Automate and other tools are likely underimpressed.

During the preview, Microsoft imposes a limit of 50 lists with up to 2,000 items per list. There’s also a 200 MB storage limit per list. That’s more than enough to test things out without doing anything more serious (always a bad idea with preview software).

The Teper View

On LinkedIn, Jeff Teper, who heads up ODSP (OneDrive, SharePoint, and Teams), had his say in another post. He asserts that making Lists available to consumer accounts is the next big technical bet for SharePoint. Under the covers, SharePoint has “user shards” (discrete segments of storage) to support consumer access and needed “a lot of engineering” to support authentication for MSA in addition to Azure AD. Lists for MSA uses a SharePoint MySite, which Teper notes is “just like we use in OneDrive for Business.” Microsoft suppresses the MySite UX, but the functionality is there, which Teper says “gives us a lot of flexibility for the future.”

A Premium Feature

Microsoft seldom undertakes large engineering efforts for zero return. In this case, I expect that, when it’s generally available, Lists for MSA will be a premium feature of OneDrive consumer, like the way that Outlook consumer is available in free and premium versions. In the same way, OneDrive consumers will use a common platform with some UX tweaking to hide or reveal features based on how much they pay. Lists is probably the first of these features, possibly coupled with Nucleus-powered offline capabilities and 100 GB storage (available today for $1.99/month).

Planner and Lists

In terms of Lists in SharePoint Online, an interesting post makes the case that Microsoft should replace Planner with Lists. Or perhaps, replace the underpinnings of Planner with Microsoft Lists (keeping the UX is easy). I don’t agree with the idea.

Planner and Lists are two different entities. In fact, Planner uses Tasks, one of the fundamental entities managed by the Microsoft 365 substrate shared across multiple applications. Lists are more complex objects, well suited for use as a development platform in many circumstances (including by Microsoft, such as the way Lists store Teams webinar information). Although a list can certainly manage a set of tasks, it’s a minor example of the kind of solutions people use Lists for today.

Moving Planner from Tasks (very simple items) to Lists is not straightforward, especially with the impact rippling across multiple applications and UIs. For instance, think of the way you can manage the same tasks through To Do, Tasks by Planner in Teams, and Outlook. If you moved Planner to use Lists, what impact would this have on To-Do and Outlook? The answer is “a lot.”

Over-engineering is as serious a problem as under-engineering, and it seems to me that any attempt to replace the fabric of Planner with Lists is an example of radical over-engineering.

Planner and Project – The One Development Group

There’s no doubt that Lists offers better support for customer-facing APIs today. The lack of application permissions for the Planner Graph API is regrettable, as is the slow pace of development in the Planner app overall when measured against the rest of Microsoft 365. That pace might be because the Project development team is responsible for Planner, and they want to keep clear blue water between Project and Planner.

Holding back Planner to enable Project to prosper might be regrettable but understandable in the context of the Microsoft 365 business. It’s no reason to jettison the Tasks underpinning for Planner and replace it with Lists.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Feature Became Generally Available in July 2022

According to a LinkedIn post by Microsoft Principal Program Manager Sanjoyan Mustafi, administrators will soon be able to assign default sensitivity labels to document libraries in SharePoint Online and OneDrive for Business. The capability is in private preview at present, but Microsoft 365 tenants can sign up to join the preview here.

Update: According to message center notification MC391948 (June 13), rollout of the public preview of setting a default sensitivity label for a document library will roll out in late June. This is Microsoft 365 roadmap item 85621.

Update 2: On July 29, Microsoft announced that the roll-out for the public preview code had begun and that all tenants would receive the update within 90 days. The documentation is also available.

Today, you can require that users add a sensitivity label to documents and define a default label to use. This is done through settings of the sensitivity label publishing policy which makes labels available to users. Requiring documents to be labelled works, but you don’t know what labels users will choose. Sometimes, it might be necessary to ensure that every document in a library receives the same sensitivity label to reflect the level of confidentiality of the library, and that’s where the new capability comes in.

The Backend to Apply Sensitivity Labels

The preview includes the back-end code to define a default label and apply it to new Office documents uploaded or copied to or saved in a library. An asynchronous thread examines new items to check if they already have a sensitivity label. The stamping of the default sensitivity label on new items by the thread can take a few minutes.

If a new item already has a user-applied sensitivity label, the thread ignores the document based on the principle that explicit assignment by users always takes precedence over automatic assignment. If the item has a label of a lower priority (sensitivity labels have a priority order from 0 to n, with 0 being the lowest) received through automatic assignment (usually because a label publishing policy mandates the application of a default label), the thread replaces the label and applies the default label defined for the library.

For now, labeling only happens for new Office documents (support for PDFs will come later). Existing documents remain untouched, and you must apply labels manually if you want all documents to have the same label. However, in the future, Microsoft plans to update the code so that SharePoint will apply labels whenever a user opens an unlabeled document in a library with a default label.

Note that a user can remove the default label assigned for the library or replace it with a label of higher or lower sensitivity. In these cases, the user-assigned label remains, again following the principle of user precedence.

Update: Figure 1 shows the UX to configure a default sensitivity label for a document library. To access this screen, go to Library settings.

Configuring for Default Sensitivity Labels

Prior to Microsoft delivering the UX to configure a default sensitivity label for a document library, you had to update the configuration of the target document library using the SharePoint API. You can do this with Postman (the tool favored by Sanjoyan), but I prefer PowerShell, which is what I used. Sanjoyan explains the procedure in his post, but briefly is:

• Get a bearer token to authenticate with SharePoint Online. You can copy the token if you’re logged into SharePoint Online by using the developer tools (F12).
• Create a header structure to hold details of the transaction, including the bearer token.
• Create a body structure to define the GUID of the sensitivity label you want to add as the default for the library. Use Connect-IPPSSession to connect to the Compliance center endpoint and run Get-Label to find the list of labels. The GUID for each label is in the ImmutableId property.

Get-Label | Format-List DisplayName, ImmutableId

• POST to the URL for the document library using the header and body defined earlier.

The commands I used to update a document library were:

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept", "application/json;odata=verbose")
$headers.Add("Content-Type", "application/json;odata=verbose")
$headers.Add("X-HTTP-Method", "MERGE")
$headers.Add("If-Match", "*")
$headers.Add("Authorization", "Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkRya21Mczl1akhnMkp1SE5CRm5vOERicXBJSSJ9.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDBAYjY2MjMxM2YtMTRmYy00M2EyLTlhN2EtZDJlMjdmNGYzNDc4IiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTY0MzMxNTU5MiIsImV4cCI6IjE2NDMzMTkxOTIiLCJ1cG4iOiJ0b255LnJlZG1vbmRAcmVkbW9uZGFzc29jaWF0ZXMub3JnIiwicHVpZCI6IjEwMDNCRkZEODA1Qzg3QjAiLCJjYWNoZWtleSI6IjBoLmZ8bWVtYmVyc2hpcHwxMDAzYmZmZDgwNWM4N2IwQGxpdmUuY29tIiwidmVyIjoiYnJvd3NlciIsInRpZCI6ImI2NjIzMTNmLTE0ZmMtNDNhMi05YTdhLWQyZTI3ZjRmMzQ3OCIsImFwcGlkIjoiYzU4NjM3YmItZTJlMS00MzEyLThhMDAtMDRiNWZmY2QzNDAzIiwiYXBwX2Rpc3BsYXluYW1lIjoiU2hhcmVQb2ludCBPbmxpbmUgQ2xpZW50IEV4dGVuc2liaWxpdHkiLCJzY3AiOiJGaWxlcy5SZWFkV3JpdGUuQWxsIFNpdGVzLk1hbmFnZS5BbGwgU2l0ZXMuRnVsbENvbnRyb2wuQWxsIFRlcm1TdG9yZS5SZWFkV3JpdGUuQWxsIiwiaXBhZGRyIjoiNTEuMTcxLjIxMi4xMjkiLCJzZXNzaW9uaWQiOiIzYzdiMjUzYy0zNmJjLTQ1ZTctYmQ4OS1mNGRhZjdkZjZhNmEiLCJzaWduaW5fc3RhdGUiOiJbXCJrbXNpXCJdIn0.m0VNYiAPfu7GKuTcnAi0hc4ay7TAQ-KzlH1g3hRzRzJZccoLeRepey8k7ydNHsvdhO8N0E4mMEEz3dD8Tk-1qreBzNrqPkB6p2s8hGF1J04RaR6vkyTqJypFXLRXgmSsVrPsX1huNnkwZ0d_ShmPowUToZk_HN0MrDRIEleCks32pg1nQs2Umk63BkWAaUHJy_pLhYJOea0uzSc7iPeVpPaAQ8PbK8K4eRJX__DEByQueUSOd21V9O6KJ9ey-JasryPiqtncFUDGrofQ6EZztjwaCAjQubRv7RjOkMYeucgsgiI7cvfuvuCzcXjc6oqdosZwc-18Uurq_8r8ks9c4A")

$body = "{
`n `"__metadata`": {
`n `"type`": `"SP.List`"
`n },
`n `"DefaultSensitivityLabelForLibrary`": `"27451a5b-5823-4853-bcd4-2204d03ab477`"
`n}
`n"
$Uri = 'https://office365itpros.sharepoint.com/sites/Office365Adoption/_api/web/lists/GetByTitle(''Documents'')'
$Update = Invoke-RestMethod -Method 'Post' -Headers $Headers -Body $Body -Uri $Uri

Formatting of these commands must be precise, and the bearer token must be valid or the update will fail (I know, because I made many mistakes before doing it just right). The easiest way to make sure is to open the site you want to update in a private browser window to force a recent authentication and then copy the token (use F12 in Edge and access Local storage, then copy the value of the key for the identity for SharePoint Online as shown in Figure 2).

After configuring a default sensitivity label, it’s a good idea to change the default view for the library to include the sensitivity label to remind users that documents now have labels.

Steady Progress

Sensitivity Labels and SharePoint Online had a rocky start. There was a time when the content of protected Office documents was inaccessible to search and eDiscovery. That’s in the past (if you enable support) and Microsoft is busy filling out all the details that make software more useful. Adding a default sensitivity label to document libraries is a nice step forward but remember that using this capability will require Office 365 E5 or above, just like all the other auto-label application features in Microsoft 365.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Now Available in SharePoint Online and OneDrive for Business

Message Center Notification MC302489 (December 8) brings news of yet another tweak made by Microsoft to the dialog used to create new Sharing Links. The update means that the settings for sharing links for “most video and audio” files now block download by default (Figure 1).

Previous tweaks to the dialog include making it easier to update sharing link settings and highlighting the edit setting. Because many workloads use the sharing link dialog, the benefit of the changes ripple across Microsoft 365.

Understandable Change in Line with Previous Updates

The change is understandable. Sharing a video or audio is often just an invitation to consume final content (using the recently-upgraded web viewer) and you don’t want people to be able to download the files. By comparison, sharing a document, spreadsheet, or presentation is often for review and editing purposes, and the recipient might need to download a local copy to edit the file offline.

Interestingly, Microsoft 365 roadmap item 82193 makes explicit reference to Microsoft Stream, probably reflecting the ongoing motion to move Stream away from its old Azure-based platform to storing videos in OneDrive for Business and SharePoint Online. This transition has already happened for Teams meeting recordings, and the migration for other Stream content is in preview. Teams meeting recordings restrict download access to the recording owner, so setting sharing links to no download by default is in line with that philosophy.

Not All Video or Audio Files

Noting the caveat that the change applies to most video and audio files, I checked the content of my OneDrive for Business account and discovered that OneDrive blocks downloads in sharing links created for Teams meeting recordings. The same doesn’t happen for other MP4 files that I uploaded to OneDrive where the download control is missing when creating sharing links (Figure 2).

The BlockDownloadLinksFileType setting for my tenant (managed through PowerShell with the Set-SPOTenant cmdlet) is WebPreviewableFiles, which means that download blocks are available for all supported files. Given that audio and video files are now in the supported category, something else is going on.

OneDrive recognizes both sets of files as MP4s, so the difference in behavior might be because the uploaded files didn’t have the same PROGID tags as the Teams recordings (these tags make it possible to apply an auto-label retention policy to Teams meeting recordings). Alternatively, it could be because some background job hasn’t yet processed the other MP4 files. Requiring extended periods to process files is not unknown in SharePoint Online and OneDrive for Business. In any case, I’ll keep an eye to see if things change.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Knowing When Sharing Happens

A natural question flowing from the discussion about implementing the SharePoint Online expiring access policy for external users is how administrators know if people use the feature. Equally naturally, the first place to look is the Office 365 or “unified” audit log to see if SharePoint Online generates any helpful events when users extend sharing links.

Unhappily, although SharePoint Online captures a UserExpirationChanged audit event when someone extends a sharing link close to its expiration, the information stored in the event is not enough to easily identify the content the sharing link grants access to. If you look at the sample audit event shown below, the SiteUrl property tells us that this event relates to sharing some OneDrive for Business content. Apart from that, we can see:

• The user principal name of the user who extends the validity of the sharing link (Jane.Sixsmith@office365itpros.com).
• The user principal name of the target user being granted access (Jsmith_yandex.com#ext#@office365itpros.onmicrosoft.com). The form tells us that this is a guest account (JSmith@yandex.com).

It would be nice if the name of the actual folder or document being shared was captured, but that’s not the case.

RecordType   : SharePointSharingOperation
CreationDate : 15/11/2021 13:17:04
UserIds      : Jane.Sixsmith@office365itpros.com
Operations   : UserExpirationChanged
AuditData    : {
                 "AppAccessContext": {
                   "AADSessionId": "bfe559aa-a811-488b-828d-a1fa90062133",
                   "CorrelationId": "b45e03a0-50df-3000-73a8-a6b7cbd31cc0"},
                 "CreationTime": "2021-11-15T13:17:04",
                 "Id": "5ee7b4d0-97ca-476d-c7ef-08d9a83a37aa",
                 "Operation": "UserExpirationChanged",
                 "OrganizationId": "a562313f-14fc-43a2-9a7a-d2e27f4f3478",
                 "RecordType": "SharePointSharingOperation",
                 "UserKey": "i:0h.f|membership|1003bffd805c87b0@live.com",
                 "UserType": "Regular",
                 "Version": 1,
                 "Workload": "OneDrive",
                 "ClientIP": "51.171.212.129",
                 "ObjectId": "https://office365itpros-my.sharepoint.com/personal/jane_sixsmith_office365itpros_com",
                 "UserId": "jane.sixsmith@office365itpros.com",
                 "CorrelationId": "b45e03a0-50df-3000-73a8-a6b7cbd31cc0",
                 "EventSource": "SharePoint",
                 "ItemType": "Web",
                 "Site": "cc191cff-670a-4740-8458-e6067537c747",
                 "UserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36 Edg/95.0.1020.44",
"WebId": "551065f1-04a6-4979-8b19-2c8a0c16319f",
                 "TargetUserOrGroupType": "Guest",
                 "SiteUrl": "https://office365itpros-my.sharepoint.com/personal/jane_sixsmith_office365itpros_com",
                 "TargetUserOrGroupName": Jsmith_yandex.com#ext#@office365itpros.onmicrosoft.com

Investigating SharePoint Sharing Events

To see if it was possible to find some other information that would allow me to link the UserExpirationChanged events back to other sharing events, I wrote a script to extract the events from the audit log and parse their content. The results are not what I hoped. You can track the progress of sharing an item through:

• SharingSet: A user shares an item.
• SecureLinkCreated: A sharing link is created for the item. This is what is sent to the recipient.
• UserExpirationChanged: The expiration date for the sharing link is adjusted in line with policy.
• SecureLinkUsed: The recipient uses the sharing link to access the shared content.

The audit records for the first three events often have the same date and time because they occur close together (within milliseconds). For this reason, they can appear in a different order when viewing the report (Figure 1).

In due course, if the sharing link validity is extended further, SharePoint logs another UserExpirationChanged event. The cycle continues until the sharing link expires.

Download the Script

The script isn’t all that interesting. It finds the relevant audit events, extracts information, and reports its findings (you can download the script from GitHub). Unless you focus on UserExpirationChanged events which happen outside the initial creation of sharing links, I don’t think it helps much in terms of understanding the extent of sharing link extensions. However, someone who is smarter than I might be able to tweak the script to derive better results.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

SharePoint Expiring Access Policy Controls Sharing Links Issued to Guests

In the summer, Microsoft introduced an expiring access policy for external users in SharePoint Online sites and OneDrive for Business accounts. In a nutshell, a tenant can set a policy to control the number of days a sharing link lasts after a user shares some content with an Entra ID guest account (created automatically when sharing with an external user). The expiring access policy doesn’t apply to guest accounts who access content through their membership of Microsoft 365 groups (teams). Their ability to work with content in SharePoint Online is controlled by the guest’s membership instead of a sharing link.

By default, the expiring access policy is not set. A tenant or SharePoint administrator must enable it and define the sharing period in the Sharing section of the SharePoint Online admin center (Figure 1). The period can be from 30 to 730 days.

Once set, the policy applies to new sharing links. The policy defined in the SharePoint Online admin center applies to all SharePoint sites and OneDrive for Business accounts. You can override the expiration period on a per-site basis.

Unlike other expiration policies used in Microsoft 365, like the Teams meeting recording auto-expiration policy or even retention policies and labels, content remains unaffected when an expiration period lapses. The only effect is on the sharing link which becomes invalid and unusable for access.

What Happens When Sharing Links Expire

As sharing links approach expiration, users receive warnings through two means. First, a banner appears in OneDrive for Business (Figure 2). The text could be better as it’s a sharing link which expires rather than a user. The guest account remains in Entra ID and can be used for other purposes, such as other sharing links or as a member of a group or team. The logic here might be that people manage sharing access on a user-by-user basis, so it’s appropriate to refer to users expiring.

The second method is email. SharePoint sends a note to people to advise them when sharing links are within ten days of expiration (Figure 3). In both cases, the Manage (or Manage access) link allows the user to update the soon-to-expire sharing links.

Clicking the link brings up the Access Expiration fly-out pane (Figure 4), which lists all sharing links created by the user subject to the expiring access policy. As you can see, some of the links are quite a long way off because the tenant has a 120-day expiration policy.

To extend the validity of a sharing link, select a user and click Yes, extend (Figure 5). SharePoint Online will then extend the sharing link by the maximum period allowed, in this case 120 days from the current date. You can also remove a sharing link if it’s no longer needed.

Good Practice to Implement Expiring Access Policy

It’s good practice and makes good sense for Microsoft 365 tenants to implement an expiring access policy. Many expiring sharing links will need no intervention by content owners when they expire. Other links will need an extension, which is a quick and low friction action. Overall, there’s nothing much to dislike about implementing an expiring access policy where links expire after a reasonable period, like 90 to 120 days. Organizations which store more sensitive content in SharePoint could reduce the expiration period and couple expiration with the targeted availability to content available with sensitivity labels.

Learn how to exploit the Office 365 data available to tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Joins the Controls for Teams Meeting Recordings

Now that Microsoft has created the transition of storage for Teams meeting recordings (TMRs) from Stream (classic) to OneDrive for Business and SharePoint Online (ODSP), attention is focused on how to manage these files. Microsoft plans to introduce an auto-expiration policy for TMRs in January 2022 to allow organizations dictate how long these files exist in ODSP. The auto-expiration policy will work for any Microsoft 365 tenant which has licenses for Teams.

If you have Office 365 E3, users can apply retention labels to TMRs to gain more control over their retention, and if you have Office 365 E5 or Microsoft 365 E5 licenses, you can deploy an auto-label retention policy to find and label TMRs (and track the success of the policy in finding and labeling TMRs). In short, over time, organizations are gaining ways to exert compliance control over TMRs.

Blocking Sharing with Data Loss Prevention

Data Loss Prevention (DLP) for SharePoint Online and OneDrive for Business is included in the Office 365 E3 SKU. The value of DLP is that you can use a policy to protect against inadvertent data leakage caused when someone shares a TMR outside the organization. Imagine what would happen if a competitor got hold of a recording of a discussion, complete with slides, about the development of a new product!

Using much the same approach as taken to identify TMRs for the auto-labeling retention policy, we can build a DLP policy for TMRs which looks for recording files and stamps them with metadata to stop sharing happening. The DLP policy to block external sharing for TMRs is very simple. It is a custom DLP policy (i.e., not created using a template) consisting of:

• A name and description.
• Target locations. For maximum coverage, choose all SharePoint Online sites and OneDrive for Business accounts. This will stop any sharing of TMRs created for personal meetings (OneDrive) and channel meetings (SharePoint).
• A single rule. The rule looks for any file with the property value ProgId:Media.Meeting that is shared with someone outside the organization. The rule action blocks sharing with people outside the organization. Figure 1 shows what the rule conditions look like. Optionally, the rule can allow users to override the block by providing a justification to explain why they need to share a recording with an external person.

Other rule settings which you might consider include creating a custom policy tip to explain why users can’t share TMRs externally or generating an incident report to alert administrators or other people when a rule violation occurs.

The Effect of DLP

It can take up to an hour before a new DLP policy is effective. When the policy is active, the indexing process for new files detects that TMRs come within the scope of a policy and applies the policy settings to block external sharing. There might be a few minutes before the block is effective for a new file during which it’s possible to create and send a sharing link. However, once the block is in place, the sharing link is nullified.

The effect of the policy is obvious because any document which matches the policy conditions now has a small icon (circle with a line in the middle). In Figure 2, the icon is shown alongside all the TMRs in the Recordings folder. Other video files that don’t have the property set are not marked. Hovering over a TMR reveals information about the file, including a link to a DLP policy tip if set. In this case, the link reveals some custom text to explain that external sharing is not permitted for TMRs.

If the user ignores the warning and goes ahead to try and share the recording anyway, they won’t be able to do this because OneDrive for Business blocks the attempt to create and send a sharing link (Figure 3).

Easy Update

Even if internal users don’t often go back to relisten to what was discussed in a conference call, there’s no doubt that some external people might find that content interesting, perhaps even to the detriment of your company. The time required to create and deploy a DLP policy to block external sharing of TMRs is roughly ten minutes (including a pause to drink coffee). It’s a quick and easy update to make it easier to manage the security of information contained inside these files. This is a good example of the value of DLP.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Investigating Unlabeled SharePoint Sites

Microsoft is fond of equipping its administrative consoles with cards containing insights which administrators might action. Yesterday, I noticed that the SharePoint Online admin center highlighted that my tenant had many sites had no sensitivity label (Figure 1).

As you might recall, Microsoft 365 uses sensitivity labels to apply settings to “containers” (teams, groups, and sites). Controlling the external sharing capability of SharePoint Online sites is a good example of the power of this approach. By default, I assign sensitivity labels to when creating new Microsoft 365 groups and teams, so it surprised me to discover the unlabeled state of so many sites.

Explaining Unlabeled Sites

Using the Manage unlabeled sites link, I examined the sites. Because I use sensitivity labels for the sites used for groups and teams, I expected to find that some sites in the tenant had no labels. These include:

• Hub sites.
• Communication sites.
• System sites (such as the one used to manage Viva Topics).

Knowing that teams created using templates didn’t ask team owners to assign a sensitivity label until Microsoft fixed the problem in October 2021 (MC281936, Microsoft 365 roadmap item 84232), I could account for some other unlabeled sites. However, stripping all the explainable sites from the 126 noted by SharePoint still left a bunch that I couldn’t explain except by concluding that at some points in the past, the synchronization of sensitivity labels didn’t work as well as it should between SharePoint Online and the other workloads. This is an important thing to fix because if SharePoint Online doesn’t know about a sensitivity label assigned to a site, it can’t apply the management controls defined in that label.

For the record, the synchronization of sensitivity labels for new groups works well. This might be the vestige of a long-solved problem.

Fixing Up Site Sensitivity Labels

To address the problem, I decided to write some PowerShell. The first stage was to find all the sites created for teams and Microsoft 365 Groups that didn’t have a label. To do this, the code:

• Runs the Get-SPOSite cmdlet to find all sites created using the team site template.
• Run Get-SPOSite against each site to find sites without a sensitivity label. You need to access each site to find if it has a label because Get-SPOSite doesn’t return this property when run against multiple sites.
• Store the unlabeled sites in a list.

Here’s the code I used:

[array]$Sites = Get-SPOSite -Limit All -Template Group#0
If (!($Sites)) { Write-Error "No sites for Microsoft 365 Groups found... exiting!" ; break}
   Else { Write-Host ("Processing {0} sites" -f $Sites.Count) }

$SitesNoLabels = [System.Collections.Generic.List[Object]]::new()
ForEach ($Site in $Sites) { #Check each site to see if it has a sensitivity label
        $SiteData = Get-SPOSite -Identity $Site.Url
        If ([string]::IsNullOrWhiteSpace(($SiteData.SensitivityLabel)) -eq $True) {
           Write-Host ("Site {0} has no label" -f $SiteData.Url) 
           $SiteInfo = [PSCustomObject][Ordered]@{  
              URL    = $SiteData.Url
              Title   = $SiteData.Title   }
           $SitesNoLabels.Add($SiteInfo) }
} #End ForEach Sites

The properties of a Microsoft 365 group store the GUID of the sensitivity label, if one is assigned to the group/team. The next step is to retrieve the sensitivity label information for all groups. It’s possible to match a group with a site because the group properties include the site URL. I therefore:

• Used the Get-UnifiedGroup cmdlet to find all Microsoft 365 Groups. This won’t be a fast operation in large tenants, but it’s acceptable because this is a one-time operation. In the largest tenants, consider replacing the Get-UnifiedGroup cmdlet with the Groups Graph API (see the call to fetch all Microsoft 365 groups in a tenant described in this article).
• Removed any group that didn’t have a SharePoint site URL in its properties (sometimes an error in the provisioning process leaves this property blank. Microsoft 365 will eventually synchronize the site URL from SharePoint Online to Exchange Online).
• Store the site URL and sensitivity label GUID in a hash table. A list would also do, but it’s much faster to lookup against a hash table.

Here’s the code for this segment:

Write-Host "Retrieving sensitivity label information for Microsoft 365 Groups"
[array]$Groups = Get-UnifiedGroup -ResultSize Unlimited 
$Groups = $Groups | ? {$_.SharePointSiteUrl -ne $Null}
$GroupsTable = @{}
$Groups.ForEach( {
       $GroupsTable.Add([String]$_.SharePointSiteUrl, $_.SensitivityLabel) } )

We now have a list of sites without labels and a table with the labels assigned to the underlying groups. The next step is to check each site against the groups table to see if we can find what label the site should have. If we find a match, we can update the site. The next code segment does the following:

• Loop to check each unlabeled site.
• Use the site URL as a lookup against the groups table.
• If the site URL matches, use the label GUID to update the site with the Set-SPOSite cmdlet.

This code applies sensitivity labels to sites using the information from Microsoft 365 Groups:

[int]$Updates = 0; [int]$NoUpdates = 0
ForEach ($Site in $SitesNoLabels) {
    $Label = $Null
    $Label = $GroupsTable.Item($Site.Url)
    If ($Label) { # Update the site with the label we find
       Write-Host ("Updating site {0} with label {1}" -f $Site.Url, $Label.Guid) 
       Set-SPOSite -Identity $Site.Url -SensitivityLabel $Label.Guid 
       $Updates++ }
    Else {
       Write-Host ("Can't find sensitivity label for site {0} - group might be deleted" -f $Site.Url)
       $NoUpdates++ }
} #End ForEach Sites

The complete script is available from GitHub.

A Better Card

Of the 126 unlabeled sites reported by SharePoint Online, 116 were team sites. The technique described above managed to apply sensitivity labels to 103 sites. The remaining 13 are deleted sites kept by SharePoint Online because of a retention policy (the associated Microsoft 365 group is gone). The card displayed in the SharePoint Online admin center looks better (Figure 2) and all the sites belonging to Microsoft 365 groups and teams have their correct labels. All is well.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Discovering Some Nuggets from Microsoft’s Coverage

It’s been a busy week for anyone following the Microsoft 365 ecosystem as Microsoft released a slew of blog posts and announcements to support keynotes and other sessions at the Microsoft Ignite Fall event. You could spend hours reading about new features and functionality and wonder when the code will appear in your Office 365 tenant and if any additional licenses are necessary.

This post captures notes about several features available now that I noticed as I perused Microsoft’s coverage. By themselves, each is not enough to warrant a separate post, but they’re interesting all the same. These changes are examples of the stuff we track to maintain the content of the Office 365 for IT Pros eBook. All our chapter authors have been busy this week.

SharePoint Online and OneDrive for Business

Sharing links show who you’ve shared a document with. This feature was announced in June but seems to have taken its time to roll out. The idea is simple. When you send a new sharing link, SharePoint Online and OneDrive for Business tell you who the document is already shared with (Figure 1), including a thumbnail of each person (if available in Azure AD). You can hover over a thumbnail to see who the person is. The number of active sharing links also appears. It’s a small but useful change.

Easy to overlook, the SharePoint Online admin center now displays connected channel sites when a site used by Teams creates private channels (Figure 2). If you can’t remember which sites have private channel sites, connect to SharePoint Online PowerShell and run:

Get-SPOSite -Limit All -Template TeamChannel#0 | ? {$_.TeamsChannelType -eq "PrivateChannel"}

If you click the channel sites link, the admin center displays details of those sites. Teams manages the settings for these sites, but it’s nice to be able to have easy access to the information. Shared channels, which are delayed until early 2022, also use channel sites.

OneDrive for Business supports Known Folder Move (KMF) and Files on Demand on MacOS, which is nice if you’ve invested in a brand-new M1-powered Mac.

If your tenant uses sensitivity labels and has SharePoint Syntex, you can apply sensitivity labels to protect the document understanding models. The application of a label in this manner flows through to protect individual documents identified by models. It’s another way of automatically applying labels to sensitive content.

Sensitivity label control over sharing capabilities of SharePoint Online sites is now generally available. In addition, co-authoring and autosave of protected documents is generally available in the Microsoft 365 apps for enterprise (Word, Excel, and PowerPoint). We use protected documents heavily to store chapter files for the Office 365 for IT Pros eBook, so this is a welcome advance.

Exchange Online

Microsoft Scheduler can now dynamically adjust the scheduling of recurring meetings. This is message center notification MC295855 (November 2) and it’s a great idea. Static recurring meetings are all too often cancelled or rescheduled because someone is sick or otherwise unavailable. After a recurring meeting finishes, Scheduler looks for the best time slot for the next instance and books that time.

Everyone’s probably familiar with the Exchange Online campaign to remove basic authentication for email connection protocols (that October 2022 date is getting nearer!). PowerShell is on the list of protocols to be blocked for basic authentication, but the Exchange Online management PowerShell module still uses basic authentication to communicate with WinRM on a local workstation. Work is under way to remove the need to use WinRM. Microsoft has released a preview version (2.0.6-3preview) of the module to demonstrate how they will remove the dependency by using a REST API in the background. Exchange Online has many cmdlets, not all of which have been converted to use the new mechanism, but you can test the preview now.

On the downside, Microsoft didn’t say anything at Ignite about the next version of on-premises Exchange. This is strange given the September 2020 announcement said the next version of Exchange Server would be available in the second half of 2021.

Microsoft 365

Microsoft says that Visio web app is rolling out to Microsoft 365 commercial tenants (all tenants with Office 365 enterprise plans). The rollout goes through to the end of January 2022, so keep an eye on the app launcher to see when Visio web app (aka Visio in Microsoft 365) shows up in your tenant.

Microsoft Cloud App Security (MCAS) is now Microsoft Defender for Cloud Apps (surely MDCA?). The app governance add-on is now generally available. It’s a good way to chase down apps registered in Azure AD that are over-permissioned or not being used. If you don’t have MDCA or don’t want to pay for the add-on, use our DIY audit method for Azure AD apps.

Access to the knowledge available in topic cards created by Viva Topics has been restricted to some lesser-used applications up to now. Things will change when topic cards appear in OWA and Teams. Apparently, this will happen soon and should be a game changer for the organizations who have invested in the work needed to harvest organizational knowledge through Viva Topics.

Teams

Microsoft prioritized Teams at Ignite as the center of a new way to work (see my practical365.com article), so there were lots of Teams-related developments discussed, most of which can be left until they appear in a tenant near you. One snippet in a blog post about improving meeting quality is that noise suppression in Teams meetings will be available for iOS soon. Microsoft claims that they saw a “31% decline in comments about background noise distractions” after the launch of noise suppression. This sounds like a good thing, but a single statistic provided without any further context or detail is worthless. We don’t know the sample size, whether the clients were Windows or Mac. What kind of meetings, and what is meant by “comments” (good, bad, or indifferent). Like many Microsoft statistics, there’s plenty of room for fudging an issue.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what’s happening.

Delving Into SharePoint’s Custom Properties

I’ve used SharePoint since the initial release of SharePoint Portal Server 2001, but I would never regard myself as being a SharePoint expert. I am perfectly happy to perform site management using the SharePoint Online PowerShell module or the admin center, but admit that the finer points of the client-side object module (CSOM) and the Patterns and Practice (PnP) library often surpass the limits of my knowledge. Given that much of SharePoint Online usage is generated by the sites used by Microsoft 365 Groups and Teams, less need exists to get down and dirty with CSOM or PnP than appears to be the case for SharePoint Server.

The Site Property Bag

However, sometimes no other option exists but to interact with SharePoint using PnP, which brings me neatly to the subject of the site property bag. This is a feature allowing the assignment of custom values to sites. If you come from the Exchange world, it’s analogous to being able to set custom properties for mailboxes. And just like custom properties are often used in Exchange as filters to identify specific mailboxes, the site property bag can refine searches by marking sites with custom values.

Custom values written into the site property bag are simple name/value pairs. For instance, the name might be “Test” and the value “Tony.” The idea is that users can then search for sites by looking for those where “Tony” is present in the “Test” property. Being able to find sites using a filter is important for functionality like adaptive scopes for Microsoft 365 retention policies. Custom values end up as crawled properties in the SharePoint Online search schema. The crawled properties can be linked to refinable strings to become searchable, which is how the property bag values can be used in filters.

Updating Values in the Site Property Bag

The standard Set-SPOSite cmdlet in the SharePoint Online management module doesn’t update the property bag, but cmdlets from the PnP PowerShell module do. To begin, I downloaded and installed V1.8.0 from the PowerShell gallery. The developers issue frequent updates for the module, so it’s wise to make sure that you use the latest (non-preview) version.

Before attempting to update the property bag for a site, you must disable the site’s DenyAddAndCustomizePages setting. By default, SharePoint Online blocks custom scripts, and to update the property bag, we need to lift the restriction temporarily. To do this, run the Set-SPOSite cmdlet to set DenyAddAndCustomizePages to 0 (zero). Before proceeding, make sure that the value of DenyAddAndCustomizePages is Disabled (the default is Enabled).

$Site = "https://office365itpros.sharepoint.com/sites/BallyconneelyBuglers"
Set-SPOSite -Identity $Site -DenyAddAndCustomizePages 0
Get-SPOSite -Identity $Site | Select DenyAddAndCustomizePages

DenyAddAndCustomizePages
------------------------
                Disabled

The updated setting is effective immediately. The next step is to connect to the site using the Connect-PnPOnline cmdlet. An account can connect to a site only if it has access to the site. In this instance, I used a global tenant administrator account.

Connect-PnPOnline -Url $Site -Credentials $O365Cred
Set-PnPPropertyBagValue -Key "OrgPrivacy" -Value "Restricted" -Indexed
Set-PnPPropertyBagValue : Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) Site might have NoScriptenabled, this prevents setting some property bag values. At line:1 char:1

You’d imagine that a global tenant administrator can update site properties. After all, we’ve just used the same account to update the site customization setting with the Set-SPOSite cmdlet. However, the PnP module imposes its own rules. Everything looked good, but the error surfaced each time I attempted to write a new value into the site property bag.

After some debugging, I discovered that it is possible to update the site property bag only if you connect with a site administrator account. After adding the global administrator account as a site administrator, the Set-PnpPropertyBagValue cmdlet ran without a problem. If we examine the contents of the site property bag with the Get-PnPPropertyBag cmdlet, the custom value is present.

Get-PnpPropertyBag

Key                              Value
---                              -----
GroupId                          ff168380-8f71-4419-980c-7f1e8e6ea83a
vti_sitemasterid                 e2ea95e2-b7be-484f-bb63-e2b0fd4b38b6
vti_categories                   Travel Expense\ Report Business Competition Goals/Objectives Ideas Miscellaneous Waiting VIP In...
vti_createdassociategroups       3;4;5
vti_defaultlanguage              en-us
HomepageProvisioned              1
contenttypessynctimestampversion 1
vti_approvallevels               Approved Rejected Pending\ Review
taxonomyhiddenlist               73396654-2d02-47d9-a078-6f0ffe401097
vti_associategroups              5;4;3
profileschemaversion             6
GroupDocumentsListId             2825b7cc-43f3-4eef-b970-f9789082f70d
disabledhelpcollections
SiteNotebookGuid                 ddb569bc-70b8-4eae-8e02-cd221f11d5d2
GroupType                        Public
contenttypesusagebackfillversion 3
vti_associatevisitorgroup        4
vti_extenderversion              16.0.0.21409
OrgPrivacy                       Restricted
GroupAlias                       BallyconneelyBuglers
LastGroupSitePrivacyUpdated      637612064800877337
vti_associateownergroup          3
enabledhelpcollections           VGSEndUser
ProvCorrelationId                9462025b-ebf9-468c-bbde-3729d938bdbf
FollowLinkEnabled                TRUE
vti_associatemembergroup         5
GroupDocumentsUrl                Shared Documents
vti_indexedpropertykeys          TwByAGcAUAByAGkAdgBhAGMAeQA=|

After writing the custom values into the site property bag, make sure that you replace the block on custom scripts for the site:

Set-SPOSite -Identity $Site -DenyAddAndCustomizePages 1
Get-SPOSite -Identity $Site | Select DenyAddAndCustomizePages

DenyAddAndCustomizePages
------------------------
                Enabled

Checking Custom Scripting Status for Sites

Some blogs say that the DenyAddAndCustomizePages setting reverts to the default setting after a period. I have not seen this happen, but this could be simply a case of not waiting long enough for a SharePoint Online background Some blogs report that the DenyAddAndCustomizePages setting reverts to the default setting after a period. I have not seen this happen, but this could be simply a case of not waiting long enough for a SharePoint Online background process to work. In any case, it’s best to be proactive and leave sites in the correct state. A quick check with PowerShell will reveal any sites which need to be updated and correct the situation. In this example, we check only for group-enabled sites:

$ScriptingSites = 0
[array]$Sites = Get-SpoSite -Limit All -Template Group#0 | Sort Url
ForEach ($Site in $Sites)  {
   If ($Site.DenyAddAndCustomizePages -ne "Enabled") {
      $ScriptingSites++
      Write-Host ("Site {0} has scripting enabled, so now disabling scripting..." -f $Site.Url)
      Set-SPOSite -Identity $Site.Url -DenyAddAndCustomizePages 1 }
}
If ($ScriptingSites -gt 0) { Write-Host ("{0} sites found with scripting enabled - now disabled." -f $ScriptingSites) }

If you’ve added a tenant administrator account as a site administrator to update the property bag, make sure that you remove the account afterwards. It’s not good to allow access to site contents to tenant administrator accounts unless this is intended.

Moving Forward

As it turns out, updating SharePoint Online site property bags isn’t difficult. That is, if you satisfy all the requirements. In this case, making sure that you use a site administrator account is the important point. It’s something that I didn’t see covered in any of the blogs which describe how to update the property bag (I’m sure this is documented somewhere). Now that I know how to assign custom values to SharePoint sites, the road is clear to use these properties in adaptive scopes.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Making Compliance Work Better

As discussed last week, Microsoft is simplifying how retention processing works for SharePoint Online and OneDrive for Business. It’s a good initiative because this topic is like a black box for many tenant administrators. The latest step comes in MC289965 (7 October – roadmap item 82063) to align how the SharePoint Online and OneDrive for Business browser interfaces deal with user requests to delete a file assigned a retention label configured to retain items for a specific period. For instance, a file might have a retention label with a retain action for seven years. (A retention label can be set to don’t delete or retain items, which makes it a visual marker).

Deleting Files in SharePoint Online and OneDrive for Business

Up to now, the following happens:

• OneDrive for Business: User deletes file with retention label. OneDrive for Business moves the file into the Recycle bin and captures a copy in the preservation hold library for the user’s account. A OneDrive account is a personal space and it’s reasonable to allow the account user to delete files if they wish. Note that you can’t delete a file assigned a record label. To create a retention label as a record, you need to use the Records Management solution in the Microsoft 365 compliance center (requires E5).
• SharePoint Online: User attempts to delete file with retention label but is blocked because of the presence of the retention label (Figure 1).

You can argue a case that SharePoint Online does the right thing. By not allowing the deletion to happen and keeping the file in place until its retention period expires, SharePoint Online demonstrates that the file has some importance.

The Problem for Compliance

However, the problem is that the current Microsoft 365 group model allows group members full control over most items in the SharePoint Online team sites used by Teams and Groups. Therefore, if SharePoint Online blocks a user from deleting a file because of a retention label, they can simply remove the label and then delete the file (unless the retention label is a record label). Although most users might not realize that they can remove a retention label to delete a file, the fact that they can is a big problem in terms of compliance. In that light, it’s better to allow the deletion to proceed. SharePoint Online will capture the file in the preservation hold library to ensure that its content remains indexed and discoverable for retention purposes.

Earlier Attempt to Change Ran into Problems

Last June, Microsoft published MC264360 to notify tenants that they planned to change the way the SharePoint Online browser interface worked to bring it in line with OneDrive for Business. In other words, users would be able to delete files even if a retention label with a retention period was present.

After pushback from customers, Microsoft withdrew the proposed change to do some additional work. The result of that work will roll out in early November for completion by the end of the month. SharePoint Online users will be able to delete labelled files like they can in OneDrive for Business unless the organization decides that this is a bad idea and updates the SharePoint Online configuration to retain the existing behavior. SharePoint Online will continue to block deletion of Items labelled as records.

Update January 11, 2022: The controls over deletion behavior are available in the Records management section of the Microsoft 365 compliance center (Figure 2).

Changing Things Back

If an organization decides that they’d like to keep things as they are, administrators will have to crack open the SharePoint Client Object Model (CSOM) and use the SetAllowFilesWithKeepLabelToBeDeletedSPO function in the SPPolicyStoreProxy class to set the value to False. Quite why Microsoft didn’t add a new parameter to the SPO-Tenant cmdlet to update this setting like all the other SharePoint Online organizational settings is beyond me. Microsoft says that when the feature rolls out, the ”configuration will be available within the Records Management solution settings.” That’s all fine and dandy, but Records management requires Office 365 E5 or Microsoft 365 Compliance E5 licenses, so many administrators might avoid it. This setting should be in the SharePoint Online admin center and settable through PowerShell.

No doubt someone who knows their way around CSOM will create and publish the code necessary to update the setting with PowerShell so that people without deep knowledge of SharePoint object models don’t have to, but I think it is unacceptable for Microsoft to push a change out that cannot be easily controlled by tenant administrators. On the bright side, I think most tenants will like the new delete behavior for files with retention labels and can therefore ignore grappling with CSOM.

Change Based on Experience

Changing the way SharePoint Online works when deleting files with retention labels with retention periods is the right thing to do. It will make compliance work better and is more logical for users. It’s just a pity that the opt-out control is hidden.

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new development as they happen.

Making Retention More Efficient

Message center notification MC288633 (1 October) covers the topic of optimized behavior of file versions preserved in SharePoint Online and OneDrive for Business. It’s a title guaranteed to turn off most Office 365 administrators unless they’re interested in compliance. As it happens, I am, so I read the notification.

My reading of the situation is that Microsoft is replacing an old-fashioned implementation of the preservation hold library with a more modern approach. As you might know, the preservation hold library is the location used by SharePoint Online to keep information needed for retention purposes. It’s the equivalent of Exchange Online’s Recoverable Items structure, a place where updated and removed content stays until the retention period expires.

The Preservation Hold Library

Up to now, SharePoint Online has used the preservation hold library to retain multiple versions of changes made to documents and list items. If someone edits a document which comes within the scope of a retention policy, SharePoint captures a pre-change copy of the document in the library. If someone deletes a document that must be retained, it goes into the preservation hold library. The actual processing is more complicated, but that description is sufficient here.

The net effect is that a preservation hold library for a busy site can accumulate a bunch of items (Figure 1). Although users cannot access the preservation hold library, its content is indexed and discoverable and available for searching, which means that eDiscovery investigators can recover the full change record for documents and list items. Administrators can also recover files from the preservation hold library, so there’s lots of goodness available.

The Downsides of Retention

Except that a downside exists. Or rather, two significant downsides. The first is that capturing edits and deletions for a busy SharePoint Online site can consume a large percentage of the storage quota used for the site. The amount differs from site to site depending on the characteristics of site usage and the type of file stored. For instance, the site which I use to store the Word documents for blog posts has thousands of relatively small files (usually in the range of 1-5 pages), most of which are never edited after publication. The preservation hold library for the site holds 924 items of 292.6 MB, or 5.92% of the site storage.

The site used for the Office 365 for IT Pros book has completely different characteristics. The Word documents (and some Excel spreadsheets) are larger (some chapters are over 100 pages) and they receive frequent revisions. For example, according to its version history, the chapter covering Teams architecture and structure in the 2021 edition has 330 versions, most generated using the Office AutoSave feature. The combination of large files and multiple revisions drives storage consumption to 15.3 GB, or 21.8% of the site (Figure 2).

The problem is that SharePoint Online regards the storage consumed by the preservation hold library in the same manner as it treats other libraries. Everything counts against the tenant’s overall SharePoint storage quota, which seems a little unfair given that Exchange Online provides additional free storage per mailbox to handle retention. It’s easy to run a report to find the storage consumed by each site, but you’ll need to access the site to discover how much is consumed by the preservation hold library.

The second issue is that content searches find multiple copies of files stored in SharePoint Online sites. This might be what you want, but usually it’s confusing (Figure 3).

The Change

The change rolling out in mid-Novembers means that files with multiple versions deleted from a SharePoint Online site or OneDrive for Business account which must be retained will be preserved as a single file instead of multiple versions. Storing fewer versions should reduce the demand for storage, but I shall wait and see how things work before making a definitive statement on that point. Reducing the number of versions held for a file will also speed up deletions and eliminate errors caused when retained files had more than a hundred versions in the preservation hold library.

Existing files in the preservation hold library are not updated and behave as before. Eventually, after the retention period for items expire, the weekly background job to check and remove obsolete material from the preservation hold library will remove the older files and release storage.

The new approach applies to any file which ends up in the preservation hold library because of a retention policy or in-place eDiscovery hold.

Given the number of files now stored in SharePoint Online due to increased use by apps like Teams, the effect of AutoSave in generating multiple file versions, and the impact on tenant storage quota that retention can have, this is a good change. It also simplifies administration and might even make it easier for backup and restore scenarios (fewer files to deal with). Time will tell!

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Personal and Organization Document Management for Microsoft 365

I don’t know why Microsoft ever thought that it was wise or desirable to consider SharePoint Online and OneDrive for Business as two separate workloads. The decision might have made sense years ago, when Microsoft began to extract itself from the legacy of its on-premises servers and wanted to demonstrate that it had multiple services to offer within Office 365. It makes none in the context of today’s cloud services.

The simple fact is that OneDrive for Business is no longer an optional extra for Office 365 users. Teams uses OneDrive for Business to share files, including the components built using the Fluid framework, in chats. Recordings of Teams personal meetings also go into OneDrive for Business, and Whiteboard is about to make the transition to OneDrive storage too. If you save an email attachment from Outlook, OneDrive is the preferred target. Users are encouraged to move their files stored in well-known folders from local workstations to OneDrive for Business to take advantage of features like Autosave and differential synchronization.

Increasing Importance of OneDrive for Business

Microsoft makes large amounts of storage available to OneDrive for Business users to make it possible to store data online. All signs indicate that Microsoft will continue to move application and personal data to OneDrive for Business storage whenever possible because it makes it easier to index and search files, including eDiscovery support. In a nutshell, the central importance of OneDrive for Business to cloud users increases as time passes.

The Demise of the OneDrive Admin Center

Which brings me to the elimination of the OneDrive for Business admin center. Or at least, the move of OneDrive settings into the SharePoint Online admin center (Figure 1), which removes the need for the OneDrive admin center. The SharePoint Online admin center has always had settings which affected OneDrive for Business, like sharing controls. Now we have a single place to manage system and personal document and file management for Microsoft 365, which is what these products deliver.

Microsoft covered the move of the OneDrive settings in a July 2021 blog post. With so many blog posts, announcements, updates, and other information about different aspects of Microsoft 365 appearing each week, you might not have noticed the transition. If you go to the Settings section of the SharePoint Online admin center (Figure 2), you’ll find the OneDrive for Business controls.

Checking Sensitivity Labels and Sites

Another topic featured in Microsoft’s July blog is the new insight card to report the number of unlabeled sites. These are sites that don’t have an assigned sensitivity label. As you might notice from Figure 1, my tenant reports 128 of these sites. Given that I’ve invested lots of time working to implement sensitivity labels for container management, this seemed like a high number.

After checking the list of sites, I discovered that the set includes:

• Sites retained by a compliance policy after removal of the original Microsoft 365 group.
• System sites like the App Catalog site and the home site and its predecessor.
• Sites created for Yammer communities before the switch of the Yammer network to Microsoft 365 native mode.
• Teams created from a template (to close the gap, MC281936 describes an update rolling out soon to allow team owners to assign a sensitivity label when creating a new team from a template).
• The Viva Topics center site.
• The site created for the group used to control who can create custom templates for the Teams Approvals app.

In short, a bunch of sites turned up, some of which could do with a sensitivity label and others which don’t. In other words, a list that’s well worth reviewing.

Simplification is Goodness

I strongly approve of Microsoft’s move to incorporate OneDrive for Business management into the SharePoint Online admin center. There are still too many administrative consoles across Microsoft 365 and this step simplifies the tenant management landscape.

With the introduction of the new Exchange Online admin center and the transition of the old Security and Compliance Center to the Microsoft 365 compliance center, we’re also seeing rationalization of user interfaces. On the downside, the switchover from old to new consoles seems to be taking forever. Maybe it’s because it people need time to absorb change, but sometimes you’d wonder if it wouldn’t be better if Microsoft pulled the plaster off quickly and launched a family of new fully-functional administrative tools.

Make sure that you’re not surprised about changes which appear inside Office 365 applications (like updates to admin portals) by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Search for What Participants Say During Teams Meetings

Microsoft message center notification MC260749 (last updated August 12) titled Microsoft Search: Find a meeting recording based on what was said is both technically interesting and important. Described in Microsoft 365 roadmap item 82003, the roll-out was delayed several times, but the way is now clear for Office 365 tenants to be able to search videos using spoken text along with a bunch of other changes to make Teams meeting recordings more accessible and useful. While it’s hard to say exactly when individual tenants will have all the functionality described here, I expect worldwide deployment to be complete by the end of October 2021.

Everything in OneDrive

Exposing the content of meeting recordings for search is important because it starts the process to close a major compliance gap. Up to now, transcripts for online meetings have not been searchable. The problem first surfaced when Teams stored its recordings in Stream. When the meeting finished, Stream processed the recording and created the transcript. However, the transcript remained in the Stream Azure service and was inaccessible to Microsoft Search. If something can’t be indexed by Microsoft Search, its content cannot be found by a search.

Microsoft completed the migration the storage of Teams meeting recordings from Stream to OneDrive for Business or SharePoint Online (ODSP) on August 16, 2021. All new meeting recordings from that date are in ODSP with the migration of older content from Stream to ODSP happening later. Microsoft is busy building out the rest of the Stream 2.0 platform to handle videos which don’t come from Teams. For instance, they’ve released a preview of the new Stream browser interface which supports access to videos stored in both ODSP and the original Stream store.

The move to ODSP removed the ability to create and replay transcripts for meeting recordings which exists in Stream classic. Starting September 20, Microsoft plans to remove some of the automatically-generated transcripts from older videos in Stream classic to prepare for the migration to Stream 2.0.

To fill the functionality gap, Microsoft introduced a transcription capability for Teams meeting recordings (a recent update means that if you record a Teams meeting now, you generate a transcript automatically). However, the issue of searchability remained. Because ODSP stores the recording files, Search could index file metadata like the name of the recording, but that’s about all.

The gap in indexing and searchability is now closed. Teams stores the spoken text captured during a meeting (including speaker attribution so you know who said what) and meeting metadata in the Exchange Online mailbox of the meeting organizer. Capturing the spoken text in mailboxes allows Microsoft Search to index the data and therefore makes it possible for searches to find this information. And as we’ll see, ODSP also holds a copy of the transcript to allow the words in the transcript to connect with segments in a meeting recording.

Exchange Mailbox Storage for Transcript Information

Teams stores transcript information in a folder called ApplicationDataRoot/93c8660e-1330-4e40-8fda-fd27f9eafe10/MeetingTranscriptCollection in the non-IPM part of the mailbox. Hidden means that the folder isn’t available to users through clients like Outlook, but its contents are available to administrative interfaces like Microsoft Search and programs like MFCMAPI.

Transcripts are captured as mail items. Examining the captured items with MFCMAPI, it looks like two properties for are most interesting:

• TranscriptJsonBlob: stores the spoken text captured during the meeting. In Figure 1, you can see some captured text, including the name of the speaker. When users view the transcript in Teams, the information is displayed in a nicer format. It’s also possible to download transcripts in VTT or Word (DOCX) format.
• TranscriptMetadataJsonBlob: stores metadata about the call.

Linking Words to Videos

The original implementation for Teams meeting recordings stored in Stream classic supported transcription, including the ability to edit the transcript to correct obvious errors. To allow Microsoft Search to find the MP4 file for a meeting recording based on words spoken during a meeting, a background process copies the transcript data captured in Exchange Online and indexes it against the recording to match segments of the video with the spoken words.

Replication of transcript data from Exchange Online to ODSP can take anything from 15 minutes to a day after the meeting ends. Once the process completes, you can search for text spoken in meetings and find recordings using the transcript (Figure 2).

Transcript Playback

Matching words in the transcript with meeting recordings (and eventually, any video stored in ODSP) allows concurrent playback for the two elements. Microsoft 365 roadmap item 82057, rolling out in September 2021, delivers a transcript pane for video playback (Figure 3). No ability is yet available for a video owner to edit the transcript.

Curiously, closed captions are available for only 60 days from the date of recording. In addition, Microsoft says that “Closed captions aren’t fully supported” if you move or copy a recording from its original location. Presumably, this is because the move might affect the link to the transcript data.

Making Transcription Available to More Teams Users

The option to transcribe meetings used to be restricted to accounts with enterprise E3/E5 and Business Premium/Standard. In early July, Microsoft made live transcription available (MC260564) for other licenses, including the E1, F1, academic, and Business basic SKUs, noting that this step improves the accessibility of Teams and makes meetings more inclusive for those who are hard of hearing. Microsoft followed up with MC280258 (August 24), to announce support for transcripts and captions in 27 additional languages (Figure 4) to join the previous support for U.S. English.

Another Compliance Gap Nearly Closed

All the information shared during Teams meetings is gradually coming within the scope of compliance policies. eDiscovery can already find chat, presentations, and documents, and the advent of indexed speech means that spoken comments should soon come within the scope of eDiscovery searches. This hasn’t happened yet, probably because of the work needed to export transcripts and videos in eDiscovery cases, but I am sure this capability is high on Microsoft’s agenda.

Although the captured text is sometimes inaccurate, capturing any record of spoken comments is better than nothing. As time goes by, the artificial intelligence technology used to analyze speech to create the transcript will improve in terms of accuracy and ability to handle accents.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what’s happening.

Fixes a Very Old Bug First Reported in 2016

Updated: December 19, 2021

Every channel in a team has a folder in the default document library of the SharePoint Online team site associated with the team. When a new channel is created in Teams, SharePoint Online creates a new folder with the same name as the channel. The channel and folder continue to share the same name until you rename the channel, in which case the names of the channel and the folder diverge. Microsoft acknowledges that losing the naming connection between Teams and SharePoint is a problem.

The issue has existed since the earliest days of Teams. The first user voice request for Microsoft to remove confusion by making sure that the channel and folder continued to share the same name following a rename appeared on November 3, 2016. I wrote about the issue in June 2019, saying that renaming channels could be messy.

Microsoft says they will fix the problem (MC280294) and clean up the mess with an update in mid-September (Microsoft 365 roadmap item 72211). The heading for Microsoft’s is “Pairing naming convention between Teams channels and corresponding SharePoint folders,” which I think is a poor attempt at conveying the impression that the change is something good. It’s not. Instead, it’s a long (very long) overdue fix for something that Microsoft should have addressed in 2017.

Update: In message center notification MC306666 (December 18), Microsoft says that they will roll out the fix in “late February through late April (2022).” The delay in the roll-out is likely due to the need to fix bugs which came to light during testing.

The delay means that any channel renamed before the deployment of the fix will remain unpaired, unconnected, unsynchronized, and seemingly unrelated to its SharePoint folder. Given the massive growth in Teams to 250 million monthly active users and the consequent growth in SharePoint Online usage, one can only guess at how many disconnected channels exist.

One of Those Complex Software Engineering Problems

No one denies that Teams is a complex product. Teams has dependencies on and consumes many different Microsoft 365 services from Azure AD to Exchange Online. The Teams development group has done a terrific job in growing the feature set in the product and expanding its capabilities into areas like multi-geo support. As Teams development VP, Rish Tandon, explained to me last May, the engineers have faced and solved many challenging problems as they developed the product from initial idea to world-class service.

But from time to time, the Teams development group just doesn’t deliver detail as well as it should. The failure to fix the channel rename problem is a classic example. Rolling out version 2.5.0 of the Teams PowerShell module with a broken version of the New-Team cmdlet is another. Neither seem to appear to be one of the complex software engineering problems that slow products down.

The New Channel Rename

After Microsoft deploys the update to Office 365 tenants, performing channel name (Figure 1) updates the value in both Teams and SharePoint Online.

As you can see in Figure 2, the synchronization with SharePoint Online means that the channel folder has the same name as used in Teams. In the past, the folder would still be “Projects” instead of the new “Projects 2021” name.

The General channel is an exception because it cannot be renamed. This is because the General channel represents the team. In fact, because the General channel exists in every team, the Teams clients translate its name to show translated values. For instance, it’s called Général in French and Allgemein in German. The names given to other channels are not translated and keep whatever name is given when created or renamed.

Rename synchronization for channels and folders applies for standard, shared, and private channels. Channels renamed prior to the update are not adjusted. If you want the names of these channels to synchronize with SharePoint Online, you’ll need to rename them again in Teams.

Microsoft notes that the new channel name will not be used by the OneDrive sync client until the client fully processes the channel following the rename. This usually doesn’t take long.

The Long-Awaited Fix

It’s good that Teams and SharePoint are now on the same page when it comes to channel renaming. It’s taken too long to happen, but it’s better later than never.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.