Table of Contents
Trimming to the 500 Version Limit Works Well to a Point
A reader asked about intelligent versioning, the new SharePoint Online method of controlling the amount of storage consumed by file versions created for Office documents. Intelligent versioning uses algorithms to decide what versions must be kept for file recoverability and discards (trims) unnecessary versions. The issue raised was how SharePoint Online deals with versions created past the limit of 500 versions set for sites when intelligent versioning is used.
I’ve already covered the question of how SharePoint Online removes the versions deemed to be unnecessary to recover content. If Purview data lifecycle management (retention policies) are not in force, SharePoint Online can trim versions back to the set needed to recover content. However, if retention is in force for items in a site via a retention policy or eDiscovery hold, the need to retain all versions trumps trimming and SharePoint Online cannot remove versions. According to Microsoft documentation, version trimming can occur when individual files have retention labels and aren’t under the control of a retention policy. I have not observed this to be the case because retention policies apply to all the sites in my tenant.
Retention Requirement Trumps Version Trimming
The same applies to file versions created past the 500 limit. SharePoint Online cannot remove any versions to stay within the limit when retention is in force. For instance, if a file reaches version 501, SharePoint Online will normally remove version 1 to trim the set back to 500. But if retention is in force, version 1 and all other versions must be kept so that eDiscovery processes work.
Figure 1 shows the version history for the source document for the Automating Microsoft 365 with PowerShell eBook (part of the Office 365 for IT Pros bundle). The document is updated very frequently to add new code examples and explanations or to refine existing text and has accumulated 519 versions since the creation of the original file on 10 April 2024. At the date of writing, that’s roughly two versions created each day since.
Finding that the number of versions for a file exceeds 500 is unsurprising. Given the way that the Office applications auto save automatically, several versions can be created during an editing session. For instance, creating the Word document for this article generated ten file versions. Generally speaking, the more changes are made to a file, the more versions are created, especially when new text or other elements are added to the file.
The Influence of Retention
The net result is that the current implementation of intelligent versioning does not contribute to any reduction of storage consumption when data lifecycle management is used. This is disappointing but understandable. If a tenant chooses to deploy retention policies, they obviously have a need to retain content. Being able to retrieve the current version of a document is interesting for eDiscovery investigators, but being able to retrieve earlier versions is often even more valuable.
Searching for a Solution
Whether Microsoft can do anything to resolve the conflict between storage consumption and retention remains to be seen. On the surface, it seems like this is an intractable problem. However, if algorithms can be found to discard file versions on the basis that they are not required to recover content, the ingenuity of software engineering knows no boundaries.
Perhaps the key is to offer tenants a choice between conserving storage by removing unnecessary file versions or maximum retention by keeping every available version. After all, if a version is deemed unnecessary for recovery purposes, it’s might not be of much use for eDiscovery because the differences between the preceding and following versions probably aren’t large. Of course, no self-respecting eDiscovery specialist will countenance the thought of losing any data that might possibly be of interest during an investigation, but sometimes practicality has to come first.
So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.
Thanks for the article Tony – I’ve been looking at this version trimming vs retention issue for some time. The article below says that versioning limits are ignored when a retention policy is applied, but they are honored when only retention labels are applied (see the “Version storage behavior” section in https://learn.microsoft.com/en-us/sharepoint/document-library-version-history-limits says). Let me know if you think I’m reading this wrong.
However this is not what I’ve found in my testing. I need to remove both retention policies AND retention labels before intelligent versioning will delete versions. I tried to log a ticket with MS last year to understand why but I was unable to get anyone useful to assist. I’ll log another ticket soon and let you know if I get any useful info.
Thanks Matt. The documentation does say that version trimming can happen for files with retention labels that are not under the control of retention policies. I guess I have not observed this behavior because all the sites in my tenant are covered by a tenant-wide retention policy. Items have retention labels too, but the policy trumps the label. Also, relatively few files get to a point where sufficient versions accrue to force trimming to happen, so it’s hard to test. Let me know what Microsoft says and I will also check with my contacts.
My testing has shown that “Longest Retention Wins” still holds true. As both retention labels and policies are used for regulatory compliance, they’re designed to override system or library defaults, ensuring that all versions of content under retention are indeed, retained.
That’s always likely to have been the case. I still have an open query with Microsoft as to why the documentation differentiates between retention policies and labels. So far it’s taken them quite a while to contemplate the issue.