Showing posts with label performance. Show all posts
Showing posts with label performance. Show all posts

Saturday, February 2, 2019

Alternative to detach initial page loadtime from retrieving structural navigation

In the post 'Optimize for bad-performing navigation in SharePoint Online' I outlined an approach to release the initial pageload from the server-side execution time to retrieve the global site navigation. This approach serves its purpose to optimize the pageload time. However, it also comes with some aspects that complicate the usage. In particular in the perception of the site owner (maintainer): one thinks to configure the global navigation via "Site Settings \ Navigation", but next experience that it does not have immediate effect due the caching, nor is it complete in case one explicitly adds entries in the navigation that are not tied to the site structure itself. To address these drawbacks I designed + developed an alternative approach:
  1. Use webservice call /_api/navigation/menustate?mapprovidername='GlobalNavigationSwitchableProvider' to client-side request the navigation structure, exact as how it is configured by the site owner via "Site Settings \ Navigation'
  2. Cache the retrieved structure in the property bag available for all users
  3. Each time the Site Settings page is succesful loaded in browser, which implies it must be by someone with site maintenance authorization, redetermine via the webservice the global navigation structure, compare it with the one cached in property bag, and if changed replace the value in the property bag
Main advantage of this alternative compared to the 'search-driven' is that this restores for site owners the capability to manage the displayed global navigation via the out-of-the-box "Site Settings \ Navigation" configuration. And this includes the option to add specific entries also direct in that same configuration experience, no need to maintain a separate 'additional links' administration. Another advantage is that changes in the Navigation are now immediate applied, and for all users. No need to wait on client cache expiration. Which likely occurs on different moments for different visitors, and colleague X then can see a different menu from his/her colleague Y sitting on next desk. Also on code- and execution-level some improvements: the code is severly reduced + simplified, and there is no longer a dependency on Linq.js to runtime derive tree-model data structure from the flat web-nodes data structure determined via search.
Disadvantage is that the retrieved + cached global navigation is based on the authorizations of the person visiting the site settings page. In case some elements in the global navigation structure are restricted for other site visitors, they will now see those items in the global navigation. SharePoint authorization model prevents them from actually accessing the non-authorized content, however at the expense of a dissatisfying user experience: click on a menu item, and then get a 404 Access Denied. Therefore in case you have restrictive areas in the global navigation, then this approach might not be the one to use. A 2nd alternative is then to combine the 2: in essence, let each individual visitor request via the webservice the global navigation structure that (s)he is authorized to access, and cache this clientside for reuse on next visits. The drawback of this approach is that it suffers from risk of outdated navigation: if site owner makes changes to the navigation via "Site Settings \ Navigate", there is no trigger to enforce flush of all the individual site-visitors cached representations. But that can still be softened by putting a maximum on the cache duration, e.g. after 3 hours re-invoke the webservice to potential renew the navigation structure for you.

Sunday, March 25, 2018

Optimize for bad-performing navigation in SharePoint Online

Good navigation capability + support is essential for the user adoption of any website. And thus also for (business) sites delivered through SharePoint. The default model for this is and always has been via Structural Navigation, which is rather self-explaining for site owners to set up. However, for performance reasons, Structural Navigation is a bad-practice in SharePoint Online when it concerns site collections with a deeper nested site hierarchy. In short, the cause of this is due the dependency on Object Cache, which has value in on-prem farms with a limited number of WFEs, but is useless in Office 365 where large numbers of WFEs are involved to serve the site collections. See a.o. So, why is Structural Navigation so slow on SharePoint Online? in Caching, You Ain’t No Friend Of Mine.
Microsoft itself recommends to switch to search-driven navigation, and even has some 'working' code to set this alternative up in the context of a SharePoint Online site collection. Noteworthy is that the switch to search-driven navigation requires you to customize the masterpage; something which Microsoft otherwise warns us against. But in the classic experience there is no other option to replace the structural navigation by search-based, and it is an weighed decision to risk modifying the masterpage. Risk which to my opinion is small, as I do not foreseen that Microsoft will make big change changes to the standard 'seattle.master', if any changes at all. Reason is that Microsoft is fully focussing on the modern experience, and little innovation is to be expected anymore in the classic experience. This is underlined by the fact that 'seattle.master' has been stable for years, without any change brought by Microsoft to it.
The 'working' code-snippet that Microsoft provides as part of their advise how-to improve the performance of navigation is included in the Microsoft support article Navigation options for SharePoint Online. Although this is a good first resource, on deeper sight the code has some flaws. Some of them are disclosed in the helpful post SharePoint Search Driven Navigation Start to Finish Configuration. On top of this, in my implementation I included some additional improvements in the ViewModel code:
  1. Encapsulate all the code in it's own module + namespace, to isolate and separate from the anonymous global namespace
  2. On the fly load both jQuery and knockout.js libraries, if not yet loaded in the page context
  3. Made the script generic, so that it can directly be reused on multiple sites without need for code duplication (spread) and site-specific code changes; this also enables to distribute and load the script code from an own Content Delivery Network (CDN)
  4. Cache per site, and per user; so that the client-cache can be used on the same device for both multiple sites, as well as by different logged-on accounts (no need to switch between browsers, e.g. for testing)
  5. Display the 'selected' state in the navigation, also nested up to the root navigation node
  6. Display the actual name of of the rootweb of the sitecollection, iso the phrase 'Root'
  7. Extend the navigation with navigation nodes that are additional to the site hierarchy; and include them also in the navigation cache to avoid the need to retrieve again from SharePoint list per each page visit
  8. Hide from the navigation any navigation nodes that are identified as 'Hidden' (same as possible with the standard structural navigation)
  9. Execute the asynchronous 'get' retrievals parallel via 'Promise.all', to shorten the wait() time, and also for cleaner structured code
  10. Extend with one additional level in the navigation menu (this is accompanied with required change in the masterpage snippet)
  11. Include a capability to control via querystring to explicit refresh the navigation and bypass the browser cache; convenience in particular during development + first validation
Also made some changes to the suggested snippet for the masterpage:
  1. Extend with one additional level in the navigation menu (see above, this is accompanied by required change in the ViewModel code)
  2. Preserve the standard 'PlaceHolderTopNavBar', as some layout pages (e.g. Site Settings, SharePoint Designer Settings,...) expect that to be present, and give exception when missing from masterpage
  3. Optional: Restore the 'NavigateUp' functionality; via standard control that is still included in the standard 'seattle.master' (a source for this: Restore Navigate Up on SharePoint 2013 [and beyond])

Saturday, July 15, 2017

Approach to performant display from a SPList with multiple Lookups + Person fields

Issue: business users complain that the page loading of a SharePoint listview takes a long time, up to a minute. During page loading + rendering, the browser is totally unresponsive (Chrome even pops up a dialog about page unresponsive).
In the analysis for the root cause of this structural slow performance I observe that the list contains multiple Lookup fields to other lists in the site, and also a 'Person or Group' field - which is a specific type of Lookup, to the hidden UserInformationList.
This multitude of Lookups in the list is the most significant cause for the slow performance. However, also on user experience / functional level, it is wrong designed: the usage of the page is to first load + display all the items from the list, and next the end-user must filter to select the relevant items.
I therefore first aligned with business owner on another functional approach: let the end-user start with specifying the relevant filter(s) [can be a combination of filters for multiple columns], and then select + retrieve only the items from the list that satisfy the filter-conditions. For the technical design, the Search tool must be setup future-proof, aka cloud ready. Thus interoperating with SharePoint either via CSOM or via REST services. I decided to utilize REST, so that data is returned in JSON data-format, and can be directly data-binded in client-side UI (I used Knockout.js; but same holds for other clientside UI frameworks as Angular, Ember).
In SharePoint REST one uses the $expand parameter to include referred lookup values in the result set. But similar as for XsltListView, this quickly destroys any performance:
So I needed a way to avoid the Lookup-expands, while still being able to filter on and select the values of the Lookup fields. Totally getting rid of the Lookup columns is not an option: for consistent data-management it is a Must-Have that one can select only from the values maintained in the Lookup list. And it holds even more for the Person field: selecting via the PeoplePicker ensures a user-friendly selection + validation.
The approach I decided to is to 'flatten' the lookup values in additional columns. Functional management can still manage the data-items using the lookup functionality, and on data-selection + retrieval I avoid the need to $expand.
The elements of the approach
  1. Per lookup value that is needed in the selection and/or display, add a single-line of text column to the list. Set them to hidden in the Item content type, so that their existense is invisible for functional management in the New/Edit/View list-forms;
      For the 'Person or Group' column, include 4 new fields, for
    1. Person Name,
    2. Photo,
    3. Department,
    4. and the UserId in the UserInformationList
  2. Create a SharePoint Designer Workflow on the list that activates on ItemCreated + ItemModified events. Design the workflow to propagate ('flatten') the lookup values to their respective flattened counter-field;
  3. Realize the bulk flattening of the existing list items through the same workflow, via javascript start the workflow on every item.
Impression of the solution setup + result

Saturday, April 22, 2017

Adding tenant-url to trusted sites improves performance + user experience

We employ Application Performance Management to structural monitor the performance of company applications, including SharePoint Online. Via APM we observed a severe effect of initial login to Office 365: it structural adds up to 8 seconds to the actual SPO activity, for all of our worldwide locations:
And via the same structural APM measuring this week we noticed that the effect of initial login was largely improved, to a mere 2 seconds. We analyzed what changes were done on our side that caused this performance improvement. Turns out that a general policy update was applied in which multiple Office 365 URLs, including that of our SharePoint tenant, were added to IE trusted sites. This workplace change has a significant positive effect on the Office 365 network access times, and it also results in situation with federated login that IE automatically logs the user in.

Friday, December 23, 2016

Shortcircuit launch sequence of provider-hosted Add-Ins

The launch of provider-hosted Add-Ins consists of 2 recurring steps:
  1. First, go to the SharePoint hostweb, and request AppRedirect.aspx
  2. The returned AppRedirect.aspx response contains a FORM structure, with as Action parameter the start-url of Add-In; via javascript embedded in the AppRedirect.aspx response the browser POST submits to request the Add-In start page.
See Source: Chris Johnson, Launching SharePoint Apps, launch sequence, debugging and AppRedirect.aspx
An essential role and value of AppRedirect.aspx is to pass the authenticated identity of current user as context token from SharePoint WFE to the remote web. However, this only applies for OAuth based, low-trust authentication via Azure Access Control Service (ACS). In case of Server-2-Server, High-Trust authentication, SharePoint host(web) environment does not pass a context token at all. And thus there is no concrete added value nor necessity for the AppRedirect call before the launching of the provider-hosted Add-In.
The most important FORM parameter is SPAppToken. SPAppToken is specific to low-trust configurations. If you are using a high-trust configuration the SPAppToken token is missing. Managing Identity and Context in Low-Trust Hybrid SharePoint 2013 Apps

The recurring invocation of AppRedirect.aspx per Add-In launch has some drawbacks:
  1. it involves an extra http request/response handling; in particular with longer network-distance, this costs noticable latency
  2. AppRedirect.aspx itself can take processing and elapse time. In IIS logs I observed that although not always, multiple times the server-side execution of this requests takes > 0.5 seconds to 1 second or more
  3. with higher load, I've earlier (Load testing SharePoint Add-in (former App) Model) noticed that the AppRedirect.aspx handling eventually becomes the bottleneck within the SharePoint WFEs
In the Add-In launch sequence, there is optimization possible as the AppRedirect.aspx step is avoidable in an High-Trust authentication situation. The added-value is in S2S authentication: zero. All the requested information to launch the Provider-Hosted Add-In is already included by SharePoint in the response of the host page that contains the Add-In(s): the 'src' value of provider-hosted Add-In iFrame element(s) exists of the pattern '<host>/_layouts/15/AppRedirect.aspx' and 'redirect_uri=<encoded uri of the Add-In startpage>'. Thus it is possible to shortcircuit the Add-In launch sequence by aborting the AppRedirect.aspx request, and replace it to instead immediate request as iFrame src the url-decoded 'redirect_uri' value.
Options to shortcircuit AppRedirect are:
  • Server-side: via an HTTP-module, to intercept the HTTP response before it reaches the browser, and for every iFrame element replace the value of the src parameter from AppRedirect.aspx into it's redirect_uri value. Disadvantages of this approach are that it is non-discrimating: all responses returned will be parsed, thus also all those that do not even contain an Add-In structure; it requires custom server-side deployment, something that is nowadays not always allowed as companies aim for cloud-ready / future-proof deployments
  • Client-side: on-the-fly change in browser / javascript-engine context, for every iFrame element replace the value of the src parameter from AppRedirect.aspx into it's redirect_uri querystring value. Disadvantage of this approach is that it is late, the browser typically already has requested the AppRedirect.aspx call. It still benefits to on-the-fly change the frame-src, and as such abort the waiting on AppRedirect.aspx response: in particular with longer network-distance, e.g. in case of a global company, it pays out to not wait for AppRedirect.aspx response to return, but immediate request the launch of the provider-hosted Add-In. And also it shortens in the scenario in which the processing time of AppRedirect.aspx would otherwise be [a] noticable bottleneck.
Snippet of the client-side approach:
<lSharePointWebControls:ScriptBlock runat="server"> function ShortCircuitLauchProviderHostedAppsInWPZone(webpartZone) { var frmRedirectSnippet = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"' + '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">' + '<html dir="ltr" lang="en-US">' + '<head><meta http-equiv="Expires" content="0" /><meta http-equiv="X-UA-Compatible" content="IE=8"/><title>Working on it...</title>' + '<link rel="shortcut icon" href="/_layouts/15/images/favicon.ico?rev=23" type="image/vnd.microsoft.icon" />' + '</head>' + '<body>' + '<div style="text-align:center;margin-top:40px;">' + '<img src=\'data:image/gif;base64,R0lGODlh...'>' + '<span style="color:#0072c6;font-size:36px;font-family:\'Segoe UI Light\', \'Segoe UI\', Tahoma, Helvetica, Arial, sans-serif;">' + 'Working on it...' + '</span>' + '</div>' + '</body>' + '</html>'; var iframesProviderHostedApps = webpartZone.querySelectorAll('iframe[src*="appredirect.aspx"]'); var i; for (i = 0; i < iframesProviderHostedApps.length; i++) { var src= iframesProviderHostedApps[i].getAttribute("src"); var redirect_uri = src.substring(src.indexOf("redirect_uri=") + "redirect_uri=".length); var addInUrl = decodeURIComponent(redirect_uri); iframesProviderHostedApps[i].contentWindow.document.write(frmRedirectSnippet); iframesProviderHostedApps[i].setAttribute('src', addInUrl); } } ShortCircuitLauchProviderHostedAppsInWPZone(document.getElementById("app_1")); </SharePointWebControls:ScriptBlock>

The client-script approach enables yet another performance and resource usage optimization: delayed loading per Add-In until and only if it is in the visible part of the browser window. The user experiences the effect one is familair with on mobile devices: swipe the screen, and the application's content is on-the-fly retrieved + screen estate filled (Twitter, Facebook, ...).
Slightly modified snippet for lazy loading Provider-Hosted Add-Ins:
<lSharePointWebControls:ScriptBlock runat="server"> function isWithinVisibleViewport(element) { var $w = jQuery(window); var windowWidth = $w.width(), windowHeight = $w.height(), viewTop = $w.scrollTop(), viewBottom = viewTop + windowHeight, viewLeft = $w.scrollLeft(), viewRight = viewLeft + windowWidth, offset = element.offset(), _top = offset.top, _bottom = _top + element.height(), _left = offset.left, _right = _left + element.width(), compareTop = _bottom, compareBottom = _top, compareLeft = _right, compareRight = _left; return ((compareBottom <= viewBottom) && (compareTop >= viewTop)) && ((compareRight <= viewRight) && (compareLeft >= viewLeft)); } function ShortCircuitLauchProviderHostedApps() { var nonVisibleInViewport = false; var frmRedirectSnippet = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"' + ..... '</html>'; var iframesProviderHostedApps = document.querySelectorAll('iframe[src*="appredirect.aspx"]'); var i; for (i = 0; i < iframesProviderHostedApps.length; i++) { var delayedFrame = iframesProviderHostedApps[i]; var src = jQuery(delayedFrame).attr('src'); if (src != '') { var redirect_uri = src.substring(src.indexOf("redirect_uri=") + "redirect_uri=".length); var addInUrl = decodeURIComponent(redirect_uri); } else { addInUrl = jQuery(delayedFrame).attr('DelayedSrc'); } if (isWithinVisibleViewport(jQuery(delayedFrame))) { jQuery(delayedFrame).contents().find('html').html(frmRedirectSnippet); jQuery(delayedFrame).setAttribute('src', addInUrl); } else { jQuery(delayedFrame).setAttribute('src', ''); jQuery(delayedFrame).setAttribute('DelayedSrc', addInUrl); nonVisibleInViewport = true;' } } jQuery('#s4-workspace').off('scroll', ShortCircuitLauchProviderHostedApps); if (nonVisibleInViewport) { jQuery('#s4-workspace').on('scroll', ShortCircuitLauchProviderHostedApps); } } </SharePointWebControls:ScriptBlock>

Sunday, November 27, 2016

Large time-taken value in IISlogs do not per se mean an issue on IIS / SharePoint server level

The time-taken field measures the length of time that it takes for a request to be processed. The client-request time stamp is initialized when HTTP.sys receives the first byte of the request. HTTP.sys is the kernel-mode component that is responsible for HTTP logging for IIS activity. The client-request time stamp is initialized before HTTP.sys begins parsing the request. The client-request time stamp is stopped when the last IIS response send completion occurs.
In the IIS logs of our SharePoint WFEs I observed large time-taken values for the responses of several GET requests, with times above 2500 (!!) seconds.
LogParser.exe "SELECT TOP 10 date,time,c-ip,cs-uri-stem,sc-status,sc-substatus,time-taken from u_ex*.log WHERE time-taken > 2500000 ORDER BY time-taken DESC"
We investigated whether this points to an actual performance problem on SharePoint server side, or is merely to be considered a false positive wrt the performance of the SharePoint environment. An indication of the latter is that our end-users have not reported (complained) on reoccuring long wait times for these requests. After some lengthy investigation - the observed symptom is not well documented -, we conclude to the last.
The justification for this conclusion is a) that we did not identify any performance issue due these specific requests on the SharePoint WFEs, and b) that beginning in IIS 6.0, the time-taken field typically includes “all the network time” being spent when transferring all the bytes to/from client:
HTTP.sys waits for the client to acknowledge the last response packet send operation or HTTP.sys waits for the client to reset the underlying TCP connection, before it logs the value for process duration in the time-taken field in the IIS logs.
This correlates with our observation that the longer time-taken responses were all for the download of large static files, typical powerpoint files (.pptx), that are rendered in browser via Office Web Apps (OWA). It appears that the browser on occassion takes long time before it completely processed the received file response, and acknowledges the last packet.

Wednesday, June 1, 2016

Performance repercussions of the Add-In model

The Add-In model is great to deliver self-contained functionalities, isolated from (harming) the SharePoint farm health - runtime and wrt physical installation. But be aware that there is a price: performance. In general, the overall performance will be negatively impacted due the Add-In model. Let me clarify:
  1. Conceptual every Add-In is an own and independent (mini) webapplication. Requesting a SharePoint page that hosts <X> Add-Ins, effectively means that the browser is visiting 1+<X> webapplications to load and render the page: first the SharePoint hostweb for the SharePoint hostpage, and next the individual AppWebs for the contained Add-Ins (SharePoint-hosted or Provider-Hosted).
  2. Each Provider-Hosted Add-In (re)enters in an own Add-In launch cycle, starting with App-authentication via the _layouts/AppRedirect.aspx page.
    Our renewed SharePoint based intranet heavily applies the Add-In model, with personalized homepage containing between 2-10 Add-Ins (mixture of SharePoint-hosted and Provider-hosted). In preparation of the Go-Live date, I validated performance and scalability through Visual Studio loadtesting. Due the multiplication of Add-In instances on the homepage there are multiple AppRedirect.aspx calls per homepage visit. On increasing the load the SharePoint server execution of that request became the scalability bottleneck, resulting in CPU to reach 100%. Investigation by Microsoft Premier Support confirmed that the high number (higher as what Microsoft architects had foreseen) of AppRedirect.aspx requests caused the peak in CPU:
    • The high CPU is caused by triggering of Garbage Collector
    • Garbage Collector is triggered due large managed heap, with int64 array datastructures
    • The int64 array datastructures are allocated by App Authentication, to encrypt and decrypt SharePoint internal certificates

    Note: As this (CPU reaching 100%) was for load well beyond both the expected / typical as the peak load, it was not needed to qualify as a No-Go for our Go-Live.
  3. Each Add-In is included in the page-html via iframe element. Due iframe boundaries, there is no runtime sharing possible of client-side objects and resources: every iframe must load its own required objects in own runtime browser context (can be from browser cache for static resources, if that is enabled at client and server-side).
  4. Be careful with building (or buying) too much self-contained Add-Ins, that all store the resources used by the Add-In in own AppWeb. Typical examples is sp.js. If that is administrated per AppWeb, then this heavy library will be retrieved multiple times: typical from the hostweb for standard SharePoint handling, and next from AppWeb(s) due Add-In usage. Due different urls, the browser does not recognize them as same resource, that is already retrieved. On each 'first' visit scenario this will result that the browser will retrieve the same resource multiple times, despite that it is already retrieved before and cached in browser.
  5. Resources that are provisioned as Add-In rootcontent are not included in the blobcache. The effect of this is twosome: [1] SharePoint does not cache the content in blobcache, and on every request must get it again from SharePoint content database; and [2] retrieved resources do not include the ‘max-age’ cache-control header setting (see Max-age cache-control setting for SharePoint content), and therefore the browser must every time ask the webserver whether client-cached resource is still valid or changed on the webserver side. Which in case of static resources (e.g., jQuery library) typically will not have changed, always responding with HTTP 304, and therefore wasting requests and network bandwidth + time.
    Via Yammer Office 365 Network I’ve inquired how-to reduce the number of 304 responses for an Add-In, but apparent this answer is not known as no answer is given - also not by any of the connected Microsoft Office 365 architects and engineers.
  6. Due own appdomain's, no sharing of http-connections across hostweb and the Add-Ins, and as result also no sharing of http-authentication context (see SharePoint App-Model + NTLM results in more 401’s).
  7. Client-side altering of the page-DOM results in a reload of all Add-Ins. This is standard iFrame behavior, implemented as such in all current browsers (E.g. iFrame reloading when moving it in DOM).
All of the above enumerated performance-degradation aspects are inevitable when utilization of Add-In model. Our end-users on average are confronted with one or more 'Working on it...' spinners when they open up their homepage filled with Add-Ins. We managed to mitigate for a large part by combination of some asynchronous behaviour, lazy loading, and enforce the Add-Ins to retrieve resources from shared location (hostweb, but can also be a CDN - internal or SharePoint static).
Once we're enabled to utilize the new SharePoint Framework, we will evaluate which of the Add-Ins can be migrated to that new model and thus break out of the iFrame boundaries + constraints. At minimal the SharePoint-hosted Add-Ins are good candidates for migration to SharePoint Framework based solutions.

Tuesday, December 29, 2015

The long quest to reinstate SharePoint Explorer View performance

In our company, business end-users like SharePoint Explorer View for its usage familiarity with shared folders access. Our business end-users reported non-performance of the SharePoint Explorer View. It used to perform satisfactory, but since a while the responsiveness structural degraded. Taking 30 seconds or even minutes to open the Explorer View initial. On immediate retry, it typically opens direct. Wait a while, and the problem repeats itself.
It proofed problematic to establish the root cause of this performance degradation. SharePoint Explorer View functionality depends on a stack of IT components, local, network and SharePoint server side.
We identified – based on literature study (internet search), and own common sense – the following list of potential causes:
  1. Slow SharePoint processing (IIS, SharePoint code)
  2. Slow SharePoint content retrieval (SQL, storage)
  3. Slow or blocking network
  4. Slow authentication handling (NT Explorer as web client to SharePoint server, e.g. see Prompt for Credentials When Accessing FQDN Sites From a Windows system)
  5. Slow Web Proxy Auto Detection (see Explorer View very poor performance, Slow response working with WebDAV resources on Windows systems)
  6. SMB protocol blocked (Source: Microsoft Whitepaper "Understanding and Troubleshooting SharePoint Explorer View")
  7. Interference with IIS WebDAV Publication Service at SharePoint server (SharePoint – Open with Windows Explorer – problems, 5. WebDAV Publishing, and Explorer view does not work in some scenarios when the SharePoint farm is on Windows Server 2008 R2)
  8. Conquer with network requests from other local running programs
  9. Network requests blocked or delayed by anti-virus processing
  10. Slowness in local WebDAV client processing (Windows Explorer as WebDAV client)
  11. Outdated local Windows binaries (Explorer binaries (Shell32.dll), WebDAV binaries (Webclnt.dll, Davclnt.dll, Mrxdav.sys) & SMB binaries (Mrxsmb.sys, Mrxsmb20.sys, Rdbss.sys))
  12. Interference with Internet Explorer AddOn's
It was hard to identify whether any of the above really had the negative impact. This is in particular due that it is difficult to get the overall view on the processing of all the involved IT components: local, virus scanner, network, firewall, load-balancer, server side, SQL, storage, … In reality it boils down on inspecting the behaviour of each single component, and then try to correlate it with the behaviour of the other components to gain insight in the overall picture. We inspected IIS logs to detect whether the long times originated on SharePoint processing of the WebDAV protocol, but this did not result in rootcause identification. Then I applied Fiddler to detect what is going on over the wire. And as Fiddler is limited to http protocol, I also used Wireshark to dig deeper in on the wire level, and also including other protocols. None of these exercises did result in cause identification. If any investigation outcome, my cautious conclusion was that the delay is not on the wire nor SharePoint processing, but rather originates on the local client. To investigate that, I used ProcessMonitor; benchmarking scenarios with and without opening Explorer View. I did see some noticeable differences and thus suspects for the delay: AntiVirus processing, extensive additional operation of svchost.exe; but could not make either a final confirmation or refute of this as symptom or cause.
As the progress in problem investigation was stalled, we involved Microsoft Premier Support. The engineers started with investigating our captures on problem occurrences – Wireshark, Fiddler, Process Monitor, Netsh. Their initial analysis confirmed my own finding that it was not due network delay. Next suspect on the Microsoft list was local interference with AntiVirus filters. To conform or exclude this as cause, we uninstalled the AntiVirus on a test client. Afterwards, the problem still manifested on that test client. Via a renewed scan of the network captures, Microsoft identified a 3rd suspect: delay caused on local Webclient due provider priority handling with SMB protocol above WebDAV. I actually though of this myself a few weeks before, inspired by an old (2006!) whitepaper of Microsoft Services Support. The symptoms of our issue namely matched with what is described in that whitepaper. However, for unclear reason – lack of understanding the complexity of the WebClient handling, no trust in whitepaper given its age? – my suggestion on this as problem area was not in-depth investigated by our operations service provider. As now Microsoft suggested basically the same, I convinced our operations to conduct a pragmatic validation to determine or exclude it as the problem cause. The simple first test is to block ICMP traffic on the SharePoint farm, and then re-test the performance of SharePoint Explorer View. But also for this the results were negative: no performance improvement in Explorer View observed after blocking ICMP traffic.
We finally had a breakthrough when we noticed that the Explorer View slowness did not occur in infra scenario in which the laptop via VPN connects to our SharePoint farm. It was then a matter of identifying the differences in the IT stack of components of on-premisse access versus VPN. Also I made Wireshark captures of the 2 different infra scenarios. In the Wireshark captures we noticed that in the on-premisse scenario, the retrieval of the browser Proxy Auto Configuration (PAC) file repeatedly timed-out. In the VPN scenario, this effect did not occur. The explanation of this is the different network path to retrieve the PAC file, and that of the on-premisse situation included a blocking IT node (which actually is a cloud-hosted solution).
As it turned out, this was indeed the root cause. We resolved the blocking of the PAC file through caching it on the network perimeter, and immediate we regained the performance of SharePoint Explorer View.
Bonus: the last mile for accelerating Explorer View performance ...
With this infra correction, performance of using ‘SharePoint Explorer View’ is greatly improved: opens up almost immediately, < 1 second. That is, except for the very 1st time: that takes 4-6 seconds.
The cause of this is on Windows OS level: Windows Explorer utilizes WebClient component to connect via WebDav protocol. And the very 1st time this WebClient component must be instantiated / brought to live, what takes the additional 3-4 seconds.
If you also want to get rid of those seconds, you can set the startup mode of WebClient to ‘automatic’:

Friday, November 20, 2015

Chrome the better performant browser for App/AddIn-model based applications

In the SharePoint App/AddIn model, each AddIn operates in it's own isolated runtime security context. The separation is achieved by app-individual DNS domains. Beyond security isolation, the individual DNS domains also has performance ramifications. Good and bad (see SharePoint App-Model + NTLM results in more 401’s). A positive effect on performance is that the individual DNS app-domains, enables the utilization of more parallel http connections by browsers. Modern browsers support parallel http connections, but limit this on host level (maximum http connections per host). In SharePoint App/AddIn context, the browser maximizes the number of http connections multiplied for the separate DNS domains.
The Chrome browser has an additional performance advantage. As only browser, it predicts the DNS destinations that will be requested in the handling of a request - DNS Prefetching & TCP Preconnect:
A unique and important optimization in Chrome is the ability to remember the set of domains used by all subresources referenced by a page. Upon the next visit to the page, Chrome can preemptively perform DNS resolution and even establish a TCP connection to these domains.
This prediction is based on previous visit(s). Upon subsequent visit, Chrome initializes http connections per dnshost, and up to the maximum of 6 per dnshost. For a SharePoint page with one or more AddIns on it, Chrome immediate sets up 1 to maximum 6 http connections for the SharePoint hostweb domain, and per AddIn app-domain also up to maximum of 6 http connections. Upon receiving and processing the SharePoint page, for the dependent urls for the contained AddIn(s), Chrome can immediate send the request(s) over the already instantiated http connections. IE, Firefox, Safari, Edge (?) all apply a more 'just-in-time' approach. Delay with opening (new) http connection only when and until needed.

Monday, August 17, 2015

Project-level Best Practices for delivering a performant Add-in based application

Indeed, all of the below are open doors!!
But yet in practice often either overlooked, or on purpose but due wrong reasoning (e.g. save time now in the project) bypassed in development project(s) to deliver a SharePoint Add-in based application:
  1. Performance and Capacity Management must be applied as an integral subject during the application development project
  2. In the requirements, agree with the application owner on the performance aspects: page download times considered acceptable, applicaton parallel usage
  3. Include performance Best Practices in the development guidelines. And make sure that all involved developers know of the guidelines, and that they apply them in their individual developed Add-Ins
  4. Do not only develop + test SharePoint Add-ins ('Apps') in isolement; also conduct integration test with multiple Add-ins on a page as the user is likely to use them, and monitor the page payload
  5. Thorough intake of any external Add-ins before before purchase; intake on architecture, functionality, capacity management and maintenance aspects
  6. Structural monitor and proof the application performance during the project, to detect in early stage when something is introduced with a severe performance decrease impact. Load testing is a good means to implement this in the project, for performance quality assurance

Some architectural and technical tips

  • Cache where appropriate (but be aware that caching costs resources itself)
  • Reduce the number of network roundtrips from client to application server - batch requests
  • Retrieve resources that are used in multiple Add-ins from a shared location - e.g. root site of the HostWeb, or a CDN (external or internal)
  • Reduce the impact of (NTLM) Authentication by retrieving non-authorized resources from anonymous accessible location
  • Utilize the sprites-concept for images
  • And the same for custom javascripts; for maintenance it is good to separate responsibilities in different libraries, but for performance it is better to collapse in a single resource file
  • Apply minimalization in resource files: javascript and CSS resource files
  • Apply lazy loading where appropriate (e.g. avoid processing and retrieval impact for Add-in functionality that is initial non-visible, and/or only rare used; in favor of delayed execution if and only if the user intends to use the Add-in)

Thursday, August 6, 2015

Load testing SharePoint Add-in (former App) Model

Validate healthy application performance behaviour

Essential for any Enterprise Application is that it can performant and scalable handle the varying usage load by the users. Nothing as embarrasing as a new application that soon after Go-Live, breaks by the enthiousastic usage of the users. To prevent such, you must build trust in the scalability of the application, and establish before Go-Live that the application – application software + system infra - can handle the expected load. Introduce load testing.
This also holds for modern SharePoint application that is composed with the Add-in model, former SharePoint App-model. But loadtesting of AddIn model does bring some extra peculiarities to loadtesting. I enumerated below the ones I encountered.
And note: our loadtesting proofed both valuable and successful: initial the loadtest revealed some performance and scalability problems. We then made some essential changes in the application code (in particular in the applied custom Add-ins / Apps), until we achieved our usage load target goal. And at the crucial moment of Go-Live, the application did not give a blinch, and perfectly handled the usage load of > 14.000 users.

Application Performance health factors

2 health factors monitored:
  1. Responsiveness of the application for the user, measured as Page Download Time
  2. Scalability of the SharePoint infra, measured as CPU, Memory and I/O utilization on the servers

Application Performance validation approach

  1. Identify target goals for application utilization
  2. "Green zone"
  3. Proof the health factors at the target-utilization goals via load testing, to simulate the real usage
  4. Identify the ‘breaking’ point via increased load/stress testing
  5. "Red zone" - performance issues monitored
  6. Determine the rootcause of the issue; this can be non-optimal code, insufficient infra parts (CPU, memory, network throughput, database IOPS)
  7. Fix the issue(s)
  8. Repeat the validation, at step 2

Loadtest execution

loadtest preparation

  1. Identify the usage/application scenarios you will use to build trust. You should select scenarios for which you expect these will be used during typical usage. An heavy transaction that in the normal operation will only be rare executed, will have a neglectable effect on the application load.
  2. Establish the target load. This is the application load for average usage. For web applications, this is typically stated in ‘Page Visits per Second’. Note that this is different from Requests per Second / RPS. In nowadays modern apps, a single page visit encompasses multiple http requests: for the page itself, dependent resources as javascript and css, and javascript calls to execute service calls for data retrieval and application functions.
    The determination/specification of the concrete target value is a challenge on itself. One easily is tempted to overstress the target value - we have 'X' users, so the parallel application usage will be 'X * Y'... However, in reality those 'X' users do not continuously all hit the application: they log on at different times, stay on pages, use other applications, go to the coffee machine, ... In our setup we identified the target value twofold:
    1. Fact: as we were introducing a renewed intranet, we could reuse the application usage statistics of the current intranet;
    2. Prediction: determine the target value via Microsoft (Bill Baer) Capacity Management Formula, an unofficial best practice recommendation
    And in our situation, the 2 values determined via the different paths delivered about the same target value, which confirmed us that we determined a realistic value.
  3. Establish the heavy load: this is abnormal but still foreseenable application usage, in special circumstances.
  4. Determine how-to build trust: manual load testing, custom test software, or utilize a load test tool – e.g. HP LoadRunner, Visual Studio LoadTest.
  5. Get sufficient test accounts to simulate different users. This is also required to prevent cache effect during load test execution. E.g. continuous retrieving user profile values of the same user.
  6. Prepare the test context for the test accounts. E.g. if the application makes use of SharePoint user profile, then the user profile must be provisioned for the test accounts to ensure reasonable load behavior.

Specialities with setting up testscripts for Add-ins / Apps

  1. The load test scenario must join in App authentication flow. In essence, this means that SPAppToken value must be set as FORM POST parameter in submit request to appredirect.aspx. The value is runtime determined in the App launcher, and returned in the initial AppRedirect.aspx response.
    In the Visual Studio webtest recording, the reference is made to this hidden field in the response.
    We encountered that the SPAppToken value is not successful runtime retrieved. This can in some circumstances be corrected by monitoring the traffic via Fiddler, and set SPAppToken to a fixed value that you get from the Fiddler trace.
  2. FormDigest value returned in JSON response from contextinfo call instead of hidden FORM parameter in response body.
    Resolution is augment the Visual Studio loadtest: Add a Text Extraction Rule to extract the value from the /_api/contextinfo JSON response.
  3. Default, Visual Studio LoadTest execution does not mimic browser-cache, resulting that each dependent resource is requested over and over. You can change/fix this by configuring the loadtest script to ‘parseDependentRequests = false’.
  4. Visual Studio LoadTest does not include the execution of javascript in the browser. If required, the activity of the javascript code must be simulated in the test scripts.
  5. With multiple provider-hosted Apps in the load test scenario, the Visual Studio loadtest scenario can make error in the runtime construction of the load test recording and assign a wrong {app_?} value. In such case, you must manually add a '<ContextParameter Name="AppId_1" Value="<APP domain value>" />', and correct the relevant Requests in the script to send request to the correct app-domain:
  6. Visual Studio LoadTest recording misses to set header variable ‘Origin’, which hinders CORS protocol handling.
  7. You can easily overwhelm the usage load by setting the ‘concurrent user’ configuration value. The use of this configuration parameter is misleading: it does not really simulate actual users. It merely sets the threads in the loadtest execution from which to continuously execute the webtest(s) in the loadtest scenario. Per thread, after finishing the webtest, the execution halts for the thinktime value; and then repeats. If you set the thinktime to zero – which is what Microsoft advices on Technet, "Don't use think times…" -, the effect is that continuously requests are fired against your application. The load on the application is then much higher as the value configured in ‘concurrent users’.
  8. Visual Studio loadagent itself can become the limiting factor. If you want to simulate a larger concurrent usage, this results in equal large set of threads in the Visual Studio execution, all of which busy to execute and monitor a webtest instance. The cpu on the load agent grows to 100%, and the load does not linear increase with the number of ‘concurrent users’ aka threads.

Load test monitoring

  • CPU, memory and disk IO per server: WFE, SharePoint backend, AppHost
  • State of the IIS queue on WFE and AppHost
  • Page download times
  • Slowest pages

Interpretation of load test output

  1. The (average) Page Response Time is the summation of the download time for that request, AND augmented with including the download times of all dependent requests beneath that main request.
  2. The RPS / Requests per Seconds output is not fit to determine whether the application + infrastructure can handle the foreseen application usage. The application usage translates in Page Visits per Second, in which each page visit typically encompasses multiple (http) requests: the .aspx request, requests for javascript and css resources. In the App execution model, each App launch on the page is in effect an own page visit. As result, the RPS factor is of little use. You must measure the ‘Page Visits per Second’ factor. Pragmatic way to monitor this is to set the thinktime for webtest on 1 minute; so that each minute the webtest is executed. The ‘Page Visits per Second’ factor then equals the Visual Studio reported 'Test per Second'.

Tuesday, May 19, 2015

Beware: BLOB cache may miss modifications via SPD

There are 2 alternative approaches to manually (not via a SharePoint solution) update content resources (.css, .js, page layouts, masterpage) that are provisioned in a SharePoint library. Via the browser: you can download the content resource/file, make the changes in the downloaded file, and upload, checkin and publish the modified file into the SharePoint library. Alternative approach is to open the SharePoint site in SharePoint Designer, open + edit the content resource direct from within SPD, and afterwards checkin + publish the modified file from SPD.
Although both approaches work to modify the content file administrated in a SharePoint library, the SPD alternative has a caveat. This week we noticed that the SharePoint BLOB cache may miss the modification trigger when the modification is done through SPD. One of our developers changed a CSS file via SPD, but when browsing the site did not see the effect of his modifications. I immediately had my suspicion towards the BLOB cache. To verify, I inspected the BLOB cache folder on the WFEs, and noticed that on all 3 of the WFEs the date of the file in cache was earlier as the published date of the file with modifications. All other files in the cache appeared up-to-date, so it was not a situation of complete BLOB cache corruption. Merely the file changed via SPD was outdated in the cache. Pragmatic resolution here is to delete the outdated file from the BLOB cache folder on all WFEs.

Sunday, May 3, 2015

Beware of script-dependencies with AMD loading

Asynchronous Module Definition is very useful to manage the loading of (larger) sets of libraries in the JavaScript runtime engine. Instead of explicit in own code take the responsibility for loading each needed javascript library one by one, it is more managable to delegate this to one of the AMD implementations. Require.js is by my knowledge most applied currently, but there are alternatives.
Our developers have implemented AMD loading in multiple of our custom build SharePoint Apps, utilizing Require.js. On inspecting (F12, Fiddler) the request/response traffic of our App-Model based intranet, we observed that sometimes a specific script file is requested twice, and the 2nd time from wrong url and not in minimized version. When I asked the App developer about it, he could not explain: from his code, he explicitly specified to load the minimized library. Also weird that the re-request does not occur always.
I decided to inspect runtime behavior plus the App code myself, and try to analyze (or rather, puzzle) what caused the behavior.

Code inspection

require(['../Scripts/AppHelpers'], function () { var spHostUrl = decodeURIComponent(getQueryStringParameter('SPHostUrl')); var hostProtocol = spHostUrl.split("//")[0]; var hostRoot = spHostUrl.split("//")[1].split("/")[0]; spHostUrl = hostProtocol + "//" + hostRoot; require([spHostUrl + '/Style%20Library/Scripts/jquery-1.11.1.min.js'], function () { require([spHostUrl + '/_layouts/15/MicrosoftAjax.js', spHostUrl + '/_layouts/15/init.js', spHostUrl + '/_layouts/15/sp.runtime.js', spHostUrl + '/_layouts/15/sp.js', spHostUrl + '/_layouts/15/sp.requestexecutor.js', spHostUrl + '/_layouts/15/sp.core.js', spHostUrl + '/_layouts/15/sp.init.js', spHostUrl + '/_layouts/15/ScriptResx.ashx?culture=en%2Dus&name=SP%2ERes', spHostUrl + '/_layouts/15/sp.ui.dialog.js', "../Scripts/jquery.rotate.js", "../Scripts/moment.min.js", "../Scripts/moment-timezone.min.js", "../Scripts/ListController.js", "../Scripts/UserSettings.js", "../Scripts/sp.communica.js", "../Scripts/App.js"], function () { jQuery(document).ready(function () { initialize(function () { }); }); }); });
Basically, the above code instructs require.js to first load the library ‘/Scripts/AppHelpers.js’, once that is loaded to load jQuery library, and once that is loaded, load a bunch of other libraries that are a.o. dependent on jQuery. And when all libraries loaded, invoke a custom initialization function (not displayed here, as not relevant for the issue).

Runtime analysis, via Fiddler and F12

In Fiddler, often however not always, the following sequence of requests is visible.
So first ‘/Scripts/moment.min.js’ is succesful requested, followed by unsuccesful (HTTP 404) request for ‘/Pages/moment.js’. The initiator of the http-request is setting the ‘src’ property of a <Script> element. Likely this is initiated from require.js handling.
I also inspected the runtime DOM. Herein it becomes clear why the browser requests the library a second time. And also it is indeed inserted in the DOM by require.js handling, as visible via the require.js properties.

Explanation: asynchronous loading + library-dependency

In the above displayed App HTML code, you see that in 3rd require.js load handling, a set of libraries are requested for load on the same level. Crucial here is that:
RequireJS uses Asynchronous Module Loading (AMD) for loading files. Each dependent module will start loading through asynchronous requests in the given order. Even though the file order is considered, we cannot guarantee that the first file is loaded before the second file due to the asynchronous nature
In the App code, moment.min.js and moment-timezone.min.js are specified required at same level:
"../Scripts/moment.min.js", "../Scripts/moment-timezone.min.js",
But AMD thus does not guarantee that moment.min.js is loaded BEFORE moment-timezone.min.js; And as moment-timezone.min.js on its turns includes “define(["moment"],b)”, require.js resolves this to load the moment.js library in case not yet loaded. This explains why it does not always occur: sometimes moment.min.js is already loaded, sometimes not…

Solution

There are 2 alternative approaches to resolve the behavior. Essence of both is to make sure that moment.min.js is loaded before the dependent library moment-timezone.min.js:
  1. Extend on the above code-pattern of explicit separating the load of libraries: still retrieve moment.min.js in the 3rd level, and move the load of moment-timezone.min.js to a 4th level:
    require(['../Scripts/AppHelpers'], function () { var spHostUrl = decodeURIComponent(getQueryStringParameter('SPHostUrl')); var hostProtocol = spHostUrl.split("//")[0]; var hostRoot = spHostUrl.split("//")[1].split("/")[0]; spHostUrl = hostProtocol + "//" + hostRoot; require([spHostUrl + '/Style%20Library/Scripts/jquery-1.11.1.min.js'], function () { require([spHostUrl + '/_layouts/15/MicrosoftAjax.js', spHostUrl + '/_layouts/15/init.js', spHostUrl + '/_layouts/15/sp.runtime.js', spHostUrl + '/_layouts/15/sp.js', spHostUrl + '/_layouts/15/sp.requestexecutor.js', spHostUrl + '/_layouts/15/sp.core.js', spHostUrl + '/_layouts/15/sp.init.js', spHostUrl + '/_layouts/15/ScriptResx.ashx?culture=en%2Dus&name=SP%2ERes', spHostUrl + '/_layouts/15/sp.ui.dialog.js', "../Scripts/jquery.rotate.js", "../Scripts/moment.min.js", function () { require(["../Scripts/moment-timezone.min.js", "../Scripts/ListController.js", "../Scripts/UserSettings.js", "../Scripts/sp.communica.js", "../Scripts/App.js"], function () { jQuery(document).ready(function () { initialize(function () { }); }); }); });
  2. Configure Require.js to be aware of the Module dependency
    requirejs.config({ shim: { 'moment-timezone.min': ['moment.min'] } });

Friday, April 17, 2015

CQWP, PictureLibrary and Blobcache

ContentByQueryWebPart is a very usable ‘tool’ to display all types of SharePoint data in a page. Listdata, document libraries and also picture library. But be aware that CQWP has some strange default behaviour wrt picture library. CQWP by default displays the preview images, instead of the actual images.
This also has several performance ramifications:
  1. On server side: actual images can be cached in blobcache, but preview images and also thumbnails are by SharePoint design not cached in blobcache. The reasoning is that blobcache is for content that is multiple times (often) retrieved, while preview images are typically only of interest for content manager upon functional management of the picture library contents. As result, each request for a preview image means that SharePoint needs to retrieve it from the content database.
  2. On client, network + server side: browser cache can be applied to avoid the browser over and over requesting same image. But the browser then still needs to query the server whether the cached resource is unmodified at the server (response 304). For images / static resources that do not change (often), even this request can be avoided: minimizing request/response handling between client (browser), server, and the network transfer. Browsers support this via ‘max-age’ setting. SharePoint supports this ‘max-age’ setting for SharePoint content, via… Blobcache. For SharePoint content retrieved outside blobcache, as thus preview images, the max-age value is not set in the http response. As result, the browser will query the remote SharePoint server whether the image is unmodified, and the server responds with '304 NotModified'. And this can end-up in some noticeable latency, dependent on how busy the SharePoint server is:
Solution is to modify the CQWP configuration to retrieve and display the actual image. This comprises of 2 parts:
  1. In the ItemStyle.xsl, change the rendering specification to display ‘EncodedAbsUrl’ iso ‘ImageUrl’;
  1. And you need to modify the ‘CommonViewFields’ specification of CQWP instance, to also include ‘EncodedAbsUrl’.

Friday, April 10, 2015

ScriptEditor reloads script-links

On the homepage of our SharePoint 2013 intranet we allow employees to install apps. One of them is for internal company ‘fun and facts’: Did you know? The global design of this app is a SharePoint list with the ‘fun and facts’, and jQuery script to get selected listitems via listdata.svc and bind the returned JSON data to html elements. The jQuery script is deployed to the Style Library in the site collection, and the link to this script file is provisioned to the page via a ScriptEditor webpart:
<div id="DW-DYK-Container"></div> <script type="text/javascript" src="/Style Library/Scripts/DidYouKnow.js" ></script>
For performance tuning I regularly monitor via Fiddler the http requests that the browser sends for a page visit. With the modern web applications constructed partly as html+css+javascript, the number of http requests can grow to a larger set as one is aware. In the Fiddler trace, I spotted the browser request for the script file ‘/Style Library/Scripts/DidYouKnow.js’, on which SharePoint responds with ‘304 NonModified. But to my surprise I also noticed a second Http Get request for the same script file, and this url is made unique to force renewed retrieval from the server.
I analyzed what causes this second request for the script file. My finding is that it is caused by the ScriptEditor class. The above ScriptEditor content is rendered to the following html:
SharePoint 2013 includes Embed Code handling that detects ‘orphan’ client script code in the page html, and does some magic with that. Part of that magic apparently is to force the (re)loading of script-links that are within the ‘ms-rte-embedcode’ div-element, via an equivalent of jQuery’s getScript() method (that also adds an unique part to the url to force script file reload from the server). However, from performance and in particular latency perspective, I dislike this behaviour: I do not want the extra request and certainly not the everytime renewed retrieval of the script-file of which I know it remains stable. So I came up with an approach to break out of the ‘magic’ of ScriptEditor: move the ‘script’ element outside of the ‘ms-rte-embedcode’ div-elements:
<div id="DW-DYK-Container"></div> <script language='javascript' type='text/javascript'> var head = document.getElementsByTagName('head').item(0); var script = document.createElement('script'); script.setAttribute('type', 'text/javascript'); script.setAttribute('src', '/Style Library/Scripts/DidYouKnow.js'); head.appendChild(script); </script>
I admit: the code is more complex as the initial one. But the result is what I want: no more duplicate and repetitive forced retrieval of the script file.

Friday, April 3, 2015

Max-age cache-control setting for SharePoint content

Browser-cache is often utilized to improve the performance and in particular latency of webapplications. Default, the browser will still go back to the remote web server on each request for cached resources to ask whether it can be reused from cache (HTTP 304), or has changed on the server since retrieved. For resources that rarely or never change (e.g. a specific version of jQuery library), it is beneficial to avoid the browser even send out the ‘has it changed’ requests. The http protocol supports this via ‘max-age’ cache-control header.
SharePoint supports the ‘max-age’ header setting. Application pages retrieved from the file system (12/14/15 hive) follow the IIS Response Headers setting. However, the IIS setting is not applied to content that is retrieved from SharePoint lists and libraries. If you also want to have the ‘max-age’ setting in the HTTP response for these items, you need to enable the SharePoint blobcache and specify ‘max-age’ value in the web.config: <BlobCache location="C:\blobCache" path="\.(gif|jpg|png|css|js)$" maxSize="10" max-age="86400" enabled="true"/>
Something to be aware of is that SharePoint only applies the blobcache for published items. The version of a file stored in a SharePoint library (e.g. Style Library) that has not been published, will not be cached in the blobcache. The valid reasoning here is that caching is contra-productive, as non-published SharePoint content typically is accessible to only a few persons instead of a large(r) audience. A consequence of this is that the ‘max-age’ cache-control setting will not be set in the http response for the retrieved item if the logged-on user has SharePoint permission to see the non-published version:
Publish the version, and the 'max-age' cache-control setting is included in the http response.