Inconvenient authorization management in 'classic' MS Stream

In the corporate usage of Microsoft Stream as Enterprise Video Portal (EVP), authorization to watch videos is also applied on 'need-to-known' base. In current Stream, it turns out a bit inconvenient to execute effective permission management.

First issue is that it is made complex to nearly impossible to efficient configure permission management on a collection of videos. The root cause of this is in how Microsoft Stream handles the authorization and organization of the video store: "In Microsoft Stream, you can use channels and groups to organize and grant permission to your videos" [Source: https://docs.microsoft.com/en-us/stream/groups-channels-overview]. This is actually not a valid statement. Indeed Stream (aka Azure AD) groups "are both a way to organize videos and to control access to videos", but channels merely "are an organization method for videos, but not a permission method". Limiting for efficient permission management is that Stream portal does not include a capability to logical associate video(s) within either a Group or Channel, this is only supported initial on the moment of adding / uploading video(s) into Microsoft Stream. Once already stored in Microsoft Stream, the only possible way to associate video(s) with additional groups or channels it to do this per video, via the manual Add to group/channel action in the video-edit menu. When this must be done for larger collection of videos, this is a cumbersome and time-consuming effort.

Second issue is that Stream suffers from a delay before the indirect authorization assignment via Stream groups is actual applied (becomes active). In situation that authorization to watch a certain video is managed by one or more Stream groups, authorizing another person for access by adding her/his Office 365 account to an authorized group (e.g. via Azure Portal), does not immediate grant the person allowed access to the video. In reality it can take up to even an hour before the group based authorization within Stream context is updated to incorporate the new added account. Until then, the person remains confronted with Stream access denial on the video.

Even worse, similar effect occurs when revoking the access by removing from Stream group. This is neither immediate effectuated. Luckily the elapse time period is in this situation shorter, max 1 to 2 minutes; not a full hour. Still, immediate access revocation on unjustified granted video is not possible.

Perhaps within new Stream, in which the video storage moves to SharePoint Online, the authorization management improves. Not 100% confident yet, as Modern SharePoint also uses Azure AD groups for permission management. However, experiences within regular SharePoint Online usage are that any change in the Azure AD group(s) are almost immediate applied for access control, both on access assignment as revocation.

Needed Api Permission for Service Principal to invoke Get-AzureADUser

This week I extended our custom B2B invitation handling build as Azure Automation Runbook to check upfront whether guest account is not present yet. The way to check is by quering for potential presence of requested Guest account via the UPN:

if ($guestAccRequests.length -gt 0) {
    # Get the Service Principal connection details for the Connection name
    $servicePrincipalConnection = Get-AutomationConnection -Name $servicePrincipalConnName   

    # Logging in to Azure AD with Service Principal
    Connect-AzureAD -TenantId $servicePrincipalConnection.TenantId -ApplicationId $servicePrincipalConnection.ApplicationId -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint

    ...

            # 1. Check whether already present as guest account
            $guestUpn = ConvertExternalEmailToUpn -guestEmail $guestEmail
            $existingAccount =  Get-AzureADUser -Filter "userPrincipalName eq '$guestUpn'" | Select UserType, UserState
            if ($existingAccount) {
                ... 
            if ($continueToInvite) {
                try {
                    $inviteResult = New-AzureADMSInvitation -InvitedUserEmailAddress $guestEmail -InvitedUserDisplayName $guestDisplayNameWithOrg -InvitedUserMessageInfo $messageInfo -SendInvitationMessage $False -InviteRedirectUrl "http://www.wvstrien.com" -ErrorAction Stop
                    $inviteurl = $inviteResult.InviteRedeemUrl

The PowerShell coding part was quickly up-and-running. More time-consuming was identifying the proper Api Permission to grant to the Service Connection that connects into Azure AD. We already assigned the 'User.Invite.All' Api Permission from the Microsoft Graph Api, and one would expect that also permission for Get-AzureADUser is within the Graph Api permissions. However, granting the likely permissions 'User.Read.All', 'Directory.Read.All', 'Domain.Read.All'; none of these authorizes the Service Connection.

The information provided by Microsoft in identifying the needed Api Permission is not very good, or at least not easy to find. The StackOverflow post Graph API - Insufficient privileges to complete the operation put me on the right track. Turns out that for this cmdlet you still need to assign permission from the deprecated Azure Active Directory Api.

Beware with Unique Permissions on PublishingPages, AddAndCustomizePages and Embed Code in a page

Interesting puzzle solved: page authors complained that some of them face an issue when they try to insert a Microsoft Stream video via embed code in the page body:

The message clearly points to missing of 'AddAndCustomizePages' permission for the logged-on page author. However, on checking the permissions, all the authors / editors do have that permission via an assigned 'Content Management' permission level. After some trial-and-error, I discovered the SharePoint internal cause. The below error message on trying to insert a ScriptEditor webpart in the page content, led me to the Eureka insight:

This site says... Aha: In the site the permission inheritance is broken, to achieve that page authors can only create and modify page (= news) items, but are restricted from modifying anything else in the site. The 'Content Management' permission level is for that purpose not assigned to them on site level, but on the Pages library. On required permission for their author task this assignment is sufficient. But a flaw in SharePoint is that it checks in the 'Insert Embed-code' handling whether the 'AddAndCustomizePages' permission is granted to the logged-on author on site level.

With this insight / understanding on SharePoint level, the fix clear: make sure that all the page authors are also granted a permission level on site level that includes the 'AddAndCustomizePages' permission. And with that fix, all page authors are enabled / authorized to user-friendly apply 'Embed Code' in page edit mode:

Confusing SharePoint UI to add guest to Modern Group-connected Site

Once in Azure AD a guest account is created for an external, site owners can use that guest account in same way as for regular accounts to grant access in a SharePoint Site: 'Share' the site, and lookup the guest account. However, within Modern / Office 365 Group-connected site, this similarity is broken. The cause is that in the SharePoint UI of a Group-connected site, the Membership in the top-level UI is actually referring to the Office 365 Group; not that of the connected SharePoint Site. Yet external / guest accounts cannot be added via SharePoint to an Office 365 Group, this needs to be done via Outlook on Web as membership management interface (Manage guest access in Office 365 Groups).

Not possible to add guests to Office 365 Groups via SharePoint top-level UI

The proper way to add guest accounts to the B2B shared site, remains to authorize them on level of SharePoint site. Preferable utilizing SharePoint Groups, instead of direct individual assigned authorization. The path in the UI to this is via 'Settings' menu, 'Site permissions', 'Invite people'. And in this step, select 'Share site only':

How-to authorize guests to access Modern Site via SharePoint UI

The above experience is a smoothened version for the old-fashioned SharePoint membership management: making one aware on the existance and concrete membership management of the SharePoint Groups in the site. This 'old-way' is also still available, via 'Advanced permissions settings'. Or for those knowledgable SharePoint insiders...; direct via the known URL (<site-URL>/_layouts/15/user.aspx).

(old-way) How-to authorize guests to access Modern Site via SharePoint UI

'Check Permissions' displays 'Deny' for Azure AD B2B guest of domains that are not whitelisted

Individual authorized guests of non-whitelisted domains rightful denied access

SharePoint external sharing via Azure AD B2B adds an additional access-control layer. On top of the authorizations granted in the SharePoint site collection, you can explicitly control for guests whether their domain is allowed. Although I personally consider it a design flaw that business users can share their site with guests of non-whitelisted domains without warning that they are not actually allowed access, what eventually matters is that these invited guests will be blocked from accessing the site as long as it is not allowed on the higher authorisation-governance level of site collection sharing. The invited guests of non-whitelisted domains will get an Access Denied on trying to access the site for which they received a SharePoint invitation. Also when you as site owner check the permissions of such a guest, you will see both an allowed part for the authorizations granted via the site permissions, as well as a long list of 'Deny' due the lack of their guest domain within the list of whitelisted domains.

Domain of guest account not whitelisted in site collection sharing settings

Guest account can be invited to the site although its external domain not whitelisted

"Check Permissions" displays 'Deny'-permissions for guest accounts of non-whitelisted domain

"Check Permissions" displays for guest accounts of whitelisted domain only granted site-permissions

Noteworthy also is that the observed behavior is different depending on whether the guest invitation is redeemed or not. The above is for an invited Azure AD B2B guest account which has been activated already, and invited guest tries to access the SharePoint site. For non-redeemed guest account it is still possible to authorize to SharePoint site despite the domain not white-listed. But for non-redeemed, 'Check Permissions' displays "None", independent on whether the external domain is whitelisted.

Guest account can be invited to the site although its external domain not whitelisted

Beware: governance of SharePoint site underneath MS Teams largely gone

In most organizations that employ SharePoint for collaboration and content handling, governance is (tried to be) applied to give some structure and consistency in the SharePoint usage. Some typical elements of such SharePoint governance are:

• The powerful authorizations via Site Collection Administrator (SCA) role is reserved to IT support only; and business users are authorized via SharePoint Groups + Permission Levels (see e.g. Site Owner vs Site Collection Administrator)
• Pre-defined site structures (in the old days via Site Definitions; nowadays via Site Templates, provisioning code (e.g. PnP provisioning))
• Organization consistent branding of the sites: logo, site classification, layout, ...
• Naming conventions for site titles and URLs
• Version Control + Content Approval policies
• Metadata (Managed + Folksonomy)
• Controlled availability of SharePoint Designer, enabling the business power-users to self-create workflows, customize views, create structure, ...
• Prerequisites imposed on the site requestor, checked upon by the helpdesk handling site provision process
• Lifecycle model
• ...

Microsoft itself acknowledges the importance of SharePoint Governance, and delivers support to its customers on this topic via guidance, trainings and templates (e.g. Overview: best practices for managing how people use your team site).

And then there was the new concept of MS Teams....; highly promoted by Microsoft and moreover highly appreciated and valued by the business users. A.o. the business values the concept of self-service creation of MS Teams, in particular when comparing it with in some organizations the (perceived) cumbersome governance process on SharePoint site provisioning. And some of the business users are even more pleasant surprised when they detect that as part of the MS Teams instance creation, also a.o. a SharePoint Site Collection is created 'underneath'. In the MS Teams concept, Microsoft makes pragmatic reuse of the availability of SharePoint Online as part of Office 365 suite for the Files handling capability. It is even a generic applied pattern by Microsoft to position and use SharePoint more and more as 'backend' underneath other of its products and services. Pre-online this already is done with MS Dynamics, MS Project; and now thus with MS Teams, Office 365 Groups, ...

But although Microsoft intended within MS Teams context the created SharePoint Site to deliver the Files capability, knowledgable and curious business users quickly discover that the connected SharePoint Site is, well a full-blown SharePoint site. Including all the SharePoint capabilities that they are familiar with when working with a standalone provisioned Site Collection. Plus the (default) governance on SharePoint site collections created underneath MS Teams instance is much less restrictive. One of the most significant is that all the MS Teams owners, automatically and outside IT control get assigned in the Site Collection Admin role! Other on permission handling are that the MS Teams owner(s), by deriviation thus also the SCA's of the MS Teams SharePoint site, can share access on the SharePoint level; deviating from the permission management executed on the level of the enclosing MS Teams. Yet another to configure on the SharePoint level sharing with externals, while that might not be configured (even allowed) at the MS Teams level.

How should any Office 365 customer deal with this differences in the governance and support on SharePoint sites created standalone, versus the ones created as element of other Office 365 services? I have not a conclusive view on this yet, but moreover I also have not yet identified such view brought to us by Microsoft. The impression is that Microsoft just kinda 'let it go', and have their customers themselves come to a positioning and best-practice(s). In my opinion this is a shortcoming from Microsoft in support and guidance perspectives: we know that one of the biggest hurdles with IT and Collaboration Tools is the user adoption and confusion about what tool to use when. Delivering the same SharePoint collaboration tool via different creation processes, and with different governance + support models as result, does not help bring a clear collaboration story. I understand and applaud the pragmatic decision by Microsoft to utilize the SharePoint capabilities within other products, but I fail to understand their considerations to make that same SharePoint site on itself accessible; outside the simplified interface via MS Teams.

How-to resolve peculiarity with .aspx file upload from automated client context

The capabilities (powers) of SharePoint as underlying business applications platform can be utilized in multiple ways. Example of a pragmatic one is to utilize SharePoint as authorized web-distribution platform for content created elsewhere. The added value it brings here are that the origin of the content itself does not need to be (made) accessible for the readers, no need to (web) serve content, the permission handling of SharePoint can be utilized to only make the content available for authorized persons.

This simple application usage is for instance applied to continuously publish and distribute system monitoring dashboard report on infra level from Linux servers to the monitoring people. They do not / are not allowed access to the Linux servers in the datacenter, but are granted access to SharePoint as application platform. This worked perfectly, until we recently migrated the hosting site from SharePoint on-prem to SharePoint Online.

The problem symptom is that the uploaded .aspx file on selecting it in the SharePoint Online UI, does not open in browser, but instead starts the ‘Download / Save As’ behavior. Which clearly obstructs the SharePoint role as host of the published infra dashboard. Other .aspx files in the same library that were migrated from the source site on-prem, all do open in the browser. That rules out document library settings. So it must be directly tied to the upload of the file. The particular upload is via Curl – which gave us some challenges to authenticate against SharePoint Online, but I will post on that separately -, but once uploaded nothing can be identified what clarifies why this file behaves different from the other .aspx files in the library. Inspected the document item properties, even up to detailed level via SharePoint Designer: all the same. The only noticeable difference is when trying to resolve in SharePoint Designer via file item properties the url to document: for the troublesome document this returns in ‘file not found’.

Strange, as the file is clearly present; and as such accessible both in the browser via the SharePoint listview UI, as when opening the library in Windows Explorer via ‘Open with Explorer’. Heck, even with sync via OneDrive, the file is included in the synced library content.

So this really kept us puzzled. Until business user self-remembered an action we did on restoring the upload via Curl: as good SharePoint citizen, I reduced the permission level of the automated client account from ‘Full Control’ to ‘Contributor’. This turned out to be the key to explaining and next resolving the issue. On SharePoint level, also uploaded .aspx files are treated as (content) page. And for completed upload + administration, the account uploading an .aspx file must have ”Add and Customize Pages - Add, change, or delete HTML pages or Web Part Pages, and edit the Web site using a Microsoft SharePoint Foundation-compatible editor”. And that permission is missing from “Contributor” permission level. It does have "Add items to library", and therefore the upload itself succeeds from the automated client context. But the next processing on SharePoint (Online) side after the file upload to convert it into a browsable page context is not allowed when only 'Contributor'. The needed permission is included in ‘Full Control’, but that gives away too much control to the automated client account. Applying ‘Least Privilege’security principle, I therefore configured a new Permission Level “Upload ASPX page”, included the needed permission, and assigned this permission level to the automated client account.

(Risky) Approach to invoke SharePoint Online API on whatever site from external automation client

For an external automation client to access SharePoint Online via the API, same holds as for you as human visitor: it first needs to authenticate itself against the accessed SharePoint Online tenant, and next it must have the needed authorization in the specific accessed SharePoint content entity (site collection, site or list). This can be achieved by registrating an Add-In, and assign it the needed permissions. Then that Add-In can be used as authentication + authorization endpoint for the external client. Post Access SharePoint Online using Postman includes a good reference of the steps that you should automate in the external client. Drawback of the described approach is that it is on scope of individual site collection or even site. The Add-In registration is not utilizable as generic gateway to all sites in the SharePoint Online tenat.

To extend the scope to full tenant, the Add-In Only permissions must be requested on the 'tenant' level'. This can only be done by a tenant administrator; and must be done via the tenant-admin site (How to provide add-in app only tenant administrative permissions in SharePoint Online). But be very careful and considerated before going this approach: the implication is that whoever knows the client id and secret of the registrated Add-In, is now enabled to access via an external client data from whatever site in your tenant. Making site-specific permission management pretty useless / even a laughter. But your information security will certainly not consider it a good + acceptable joke...

With Add-In Only permission on 'tenant' level; external client that knows the Add-In's client id + secret can request an accesstoken; and then use that same single token to access whatever site in the SharePoint tenant:

Peculiar but Explained: Access Denied on page in search result preview

This morning I noticed that a page present in search result, displayed 'Sorry, you don't have access to this page' in the preview pane. Peculiar as an authorization principle of SharePoint Enteprise Search is that the search result is security trimmed, and only returns results for which the logged-on user has authorization to see. Brainstorming with a colleague we came up with the explanation. The search result actually included the url of a SharePoint subweb, for which I do have read-access. The page configured as 'Welcome Page' (landing page) in this subweb is not [yet] checked in, and therefore not available for me. And as SharePoint Search previewing applies the SharePoint publishing 'Welcome Page' redirection when hovering over (sub)web url, this explains the 'Access Denied' experience in the preview pane. Above is confirmed: after that landing page has been checked in, now in preview I see that page impression instead of the 'Access Denied'.