Evaluating SharePoint Forum products

At one of the organizations I consult, there is a business demand for forum functionality in their external facing websites. The organization has selected SharePoint as target architecture for webapplications, including public websites. SharePoint as platform itself contains DiscussionBoard as a (kind of) forum functionality, but this is not qualified for usage on external facing websites. Among its criticisms are the look & feel which is very ‘SharePoint-like', and not what endusers expect and typically are familiar with on public websites. Also DiscussionBoard lacks forum functionalities of moderation, sticky posts, avatars, lock a post, rich text editing, tagging, rating, vote as answer. And an important restriction for the usability on public websites: a SharePoint DiscussionBoard in practice requires authenticated users (ok, you can allow anonymous access, but as posting topics and answers occurs via SharePoint forms you then will have to give anonymous users access to layouts folder. Not a wise decision to make from security perspective).

One option would be to custom develop forum functionality that satisfies the extended business requirements. But because we regard forum as commoditiy functionality, this is not something we want to develop and maintain ourselves. Therefore instead I did a market analysis and evaluation for available SharePoint forum products. It appears a very tiny market, with only 5 market products found:

• TOZIT SharePoint Discussion Forum
• Bamboo Solutions Discussion Board Plus for SharePoint
• KWizCom Discussion Board feature
• LightningTools Storm Forums
• LightningTools Social Squared

In each product evaluation I addressed the following aspects:

1. Product positioning by supplier (wide-scale internet usage, scaleability?)
2. Installation of the product
3. Effect of the installation on the SharePoint farm (assemblies, features, application pages, databases, …)
4. Product documentation: installation manual, user / usage manual
5. Functional Management: actions involved how-to provision a new forum
6. Functional Usage: using a forum, in the roles of moderator and forum user
7. User Experience and internet-ready
8. Branding capabilities (CSS, clean HTML)
9. Product support

For the requirement set of this customer, Social Squared turned out as the best fit. It has a rich forum feature set, on the same level as what you see in other (non-SharePoint) established internet forums. Also considered a strong point is that the forum administration is done in own SQL database(s), outside the SharePoint (content database). This allows to easily sharing a forum between multiple brands of the customer, each with their public presence in an own host-named site collection. And LightningTools is an well-known software products supplier, which gives the customer (IT) trust that the product will be adequatly and timely supported. As a demonstration of this: during my evaluation I already had extensive contact with their product support, answerring on questions and issues that I encountered, and also give me insight on the Social Squared roadmap.

HowTo secure enable anonymous access of SharePoint external facing website

SharePoint can be used to provision public facing websites, accessible for the anonymous audience. Naturally it is then required that the contents (publishing pages, images, stylesheets) on the SharePoint website are anonymous accessible. In SharePoint 2007 the only option was to completely open up the SharePoint site collection, including all of its contents. Starting with SharePoint 2010 you have more fine-grained options. You can still open up the entire SharePoint context, which is the easy approach. Or you can decide to explicitly determine which of the content in the SharePoint site collection is anonymous accessible.

Enable Anonymous Access on site-collection level

As said, this is the easy, and as result also the most common applied approach. Basically you have to set one switch (ok, it takes some more, you can find the steps outlined here), and you're good to go.

What you must realize is that with this approach, basically all your content is anonymous accessible. Not only the publishing pages, but all SharePoint lists and libraries in the site collection are open for data retrieval from outside your domain. The data can be retrieved via the "old" Lists.asmx service (e.g. using CodePlex SPServices.js within jQuery context; or by invoking Lists.asmx from a .NET console application), or via ListData.svc. Whether this is a concrete problem depends: if the lists and libraries do not contain any sensitive data, what's the harm? But if they do, then you do not want that data to be retrievable outside your control.

Noteworthy: This is the applied approach when you host your public facing website in SharePoint Online / Office 365.

Enable Anonymous Access explicitly only for the required SharePoint artefacts

As can be seen from the picture, you also have the option to enable anonymous access on 'Lists and Libraries' level. If you select that option, default still all your SharePoint content is prohibited for anonymous access. You have to explicitly enable per List / Library if their contents must be anonymous accessible. In practice this means that for public facing websites, you must enable Anonymous Access on the Pages library in each Publishing Site in the site collection. But also on the Style Library, so that .css and .js resources are accessible. If you display images stored in a PictureLibrary on website pages, then you must also enable that library. Same for documents. In case you need to display on a page data from a List that is deliberate not enabled, you can still do. But you will have to wrap the data retrieval in an ElevatedPrivilege context; and thus requires custom code.

So this approach implies much more work as simple enable on 'Total website'. And also you'll have an extra maintenance responsibility: when the website is augmented with another Publishing Site, you have to [remember to] enable the Pages library in that site also explicitly for Anonymous Access.

Sounds like a drag? Well, the [very] good side is that you are in explicit control of which content you allow for anonymous access. And the default mode is Not Accessible, which is a security best practice.

Lock _vti_bin usage

There is a third option. Go the easy route of enabling Anonymous Access for total website; and avoid uncontrolled data retrieval via the SharePoint webservices by disable on DNS level the usage of '_vti_bin'. This approach has however some drawbacks. You cannot use SharePoint Designer anymore to open the site. And you can no longer use the SharePoint REST services in anonymous context. In the current time of Rich HTML5 Apps, it is becoming de-facto reference architecture to connect the front-end to your data and logic via REST services. So this poses a real limitation. Which in my book should not be taken too lightly.

No silver bullet for multilingual SharePoint websites

In 2007 I wrote an advice on how-to deliver multilingual user experience in a SharePoint 2007 based public facing website. Althought the SharePoint Variations concept promised to fulfill this need, our experiences turned out differently. As result my advice was that we better just made a localized copy of the website per language.

This month I repeated the analysis on multilingual solution directions for SharePoint websites; now for SharePoint 2010 (and looking forward to SharePoint 2013). Sadly, the findings have not really improved compared to the 2007 support.

Aspects of multilingual in SharePoint sites

The information architecture of SharePoint based websites differs considerable from plain asp.net, php websites. SharePoint artifacts are typically administrated within the SharePoint content database, instead of physical files on the webserver. The SharePoint platform is also very rich with out-of-the-box capabilities and components that are used to construct a website. And for public facing websites it enables content managers to logical extend the websites with new content and composition pages.

In a SharePoint website the text displayed can come from any of the following places:

1. Contained within physical .aspx and .ascx files in the layouts folder; typically via asp.net Resources mechanism
2. Contained within code of SharePoint controls: standard and custom
3. SharePoint content database

On deeper level, the following multilingual aspects can be distinguished in SharePoint public facing websites

1. Multilingual of the SharePoint standard UI (MUI: Multilingual User Interface)
   • menu’s
   • list and libraries column names
2. Multilingual of custom developed SharePoint components
3. Multilingual of the SharePoint content
   • Publishing pages content
   • Static html in Masterpages, PageLayouts and SitePages
   • List and libraries items
   • Localized images
4. Multilingual of SharePoint navigation: Global, Current, Mega-menu
5. Multilingual of SharePoint search-discovery items: website title, friendly-urls

Solution directions

Manual website-copy per language

This is the most basic and also most flexibel approach. But it also implies that all multilingual aspects must be performed manual and repeated per language website-copy: at moment of the initial website provisioning, and later for each change to its information architecture and content. The latter also beholds a risk: site content manager forgets to propagate a change on the source site to one or more of the language copy-sites.

Site-copy per language via SharePoint Variations

Variations is an out-of-the-box capability of the SharePoint platform since 2007 version. The concept is that you declare one site in your sitecollection as the Variation root, and create + declare sites for other languages as so-called Label sites. Each change to site structure and publishing pages in the Variation root is propagated as draft to all Label sites. Per Label site, the content manager must next translate the propagated page(s), and publish them. Visitors do not see the content as it is propagated from the root site.

The most important limation of Variations is that it does not cover all the multilingual aspects. It only applies to publishing page content. SharePoint list and libraries are not in scope, and still need to be handled manual.

Another major limitation is that Variations cannot be applied as an afterthought to an already existing SharePoint site. It must be applied at start of provisioning the information architecture of the source SharePoint site.

In SharePoint 2007, the Variations functionality suffers from multiple issues that in general gave it a bad press; and there are very limited organizations that actually (trust to) apply it. In SharePoint 2010 the robustness of Variations behaviour should be a bit improved; but there are still only limited known practisioners in real world. Also noteworthy is that the STSADM VariationsFixUpTool is reported by Microsoft self to be still needed for fixing corruption in Variations Relationships list.

Multilingual in-place in the SharePoint site

The idea of this is that every localizable artefact (text + images) in the SharePoint site is inherent multilingual itself; and it server-side renders in the language of choice of the website visitor. Major benefit of such approach is that you avoid the need to maintain individual language site-copies consistent with each other. Most important limitation however is that it alone is not sufficent. You cannot apply this concept to out-of-the-box SharePoint menu’s, webparts and application pages, and also not without modification in SharePoint Web Content Management (Publishing Pages).

Multilingual via SharePoint Language Store

Basically a similar concept as ‘Resources’, and also somewhat comparable to the ‘in-place’ approach. Essence is that all text-parts are administrated in a Language Store that includes per text-part its translations in the multiple languages. The Language Store can be downloaded from CodePlex, however only for SharePoint 2007. It is not upgraded to SharePoint 2010. This approach has same limitations as the ‘in-place’ approach: does not work for SharePoint out-the-box ui-controls, menu’s, application pages, nor does it combine with SharePoint publishing.

Multilingual in SharePoint site via off-the-shelf products

Well, this in itself is a clear proof that multilingual is a complex matter in SharePoint context… The only 2 products found are from smaller and unknown companies (Oceanik and IceFire). Short evaluation of both propositions already demonstrates multiple shortcoming and functional-robustness issues – examples: mix language content on page, translation support not applied to standard SharePoint aspects (e.g. Reusable Content).

Conclusions

1. SharePoint multilingual is a complex matter.
2. Multilingual always implies extra work for SharePoint content manager → automatic text translation is yet immature to trust upon for public facing website
3. There is no 100% automatic solution for supporting all the SharePoint multilingual aspects

Selective enable ScriptManager presence in SharePoint Publishing Site

It is a well-known symptom of standard SharePoint Publishing that it renders bloathed HTML pages. The cause is that SharePoint publishing pages serve 2 modes: the end-user view, and the editorial view. To serve the latter, considerable extra html and SharePoint javascript files are included that are of no use for the end-user [view].

Approach to mitigate this effect is by separating the editorial from the end-user view. Examples of this approach are Macaw Dual Layout and Mavention Flex Layout. At high level the working is to have a distinct masterpage for end-user view, and another for the content editor. Only the latter must have the means to enable SharePoint WCM functionality; the former can be designed ‘clean’: no Form runat=”server”, ScriptManager, and include of SharePoint javascript files (init.js, core.js, …). Besides loading faster, additional advantage is that pure HTML sites are also more SEO-friendly.

However, there typically remain situations in a SharePoint publishing site in which you do need the presence of ScriptManager and the standard SharePoint client-side scripts. Multiple of the standard SharePoint controls are dependent on ScriptManager: CalendarView, TaskList, External List (XstlListView), Business Data / BCS webparts, to name some. Also custom developed controls might depend on presence of ScriptManager, e.g. rotating banner control using jQuery libraries for client-side user interface behaviour.

You have basically 2 options to enable ScriptManager et al for Publishing Pages on with controls are placed that need the ScriptManager:

1. Duplicate the ‘clean’ end-user masterpage; and augment it with the required entities.
2. Create an additional Publishing PageLayout; and augment that the required entities.

Option 1 suffers from the following disadvantages:

• As consequence that a SharePoint SPWeb can only have 1 masterpage, ‘clean’ content publishing pages and WebControls pages cannot be within the same SharePoint SPWeb. In fact, the type of pages now influences the site structure. This is visible for the end-user. And for the content manager it implies that (s)he is forced to locate new page a priori within the correct SPWeb given the intended type of page: content or control. Also, as standard SharePoint navigation is based on the site structure, it will require custom development to repair the logical site navigation.
• Double maintenance of the site look&feel in the 2 public masterpages.

As in the 2nd option both page types use the same public masterpage, all pages can be within the same SharePoint SPWeb. So with this approach it is non-visible for the end-user, and also the editing experience is far more transparent for the content editor. Not entirely, as (s)he must apply the correct PageLayout given the type of page. But the applied PageLayout is something that in SharePoint publishing can be modified / corrected after initial creation.

From SharePoint technical perspective, the 2nd option is more difficult / challenging to realize as the 1st (this is simple: just copy the public masterpage and add ScriptManager et al). Issues here are following:

• The extended PageLayout must function both in combination with the clean masterpage as with the content-editor enabled masterpage. But the latter also as Form runat=”server” and ScriptManager included; resulting in 2 occurences of them. And this is not allowed in ASP.NET plus SharePoint.
• SharePoint scriptmodel imposes hidden dependencies on the order of including + evaluating/loading SharePoint javascript files (through SOD framework).

You can deal with these issues via following approach and architecture:

• Include Form runat=”server” in the extended PageLayout; but default not visible.
• A server control included in the extended PageLayout to dynamically insert the ScriptManager plus ScriptLink for core.js; but only if no ScriptManager present yet (included from the content-editor masterpage). When runtime determined that the ScriptManager must be inserted, also the invisible server-side Form is made visible; and all siblings of the Form control are moved within the server-side HtmlForm control.
• A server control included in the public masterpage to dynamically insert WpzEmitControl (explanation why needed) if needed; the condition for this is whether the InsertScriptManager control is include in the page context.

Result

Regular content pages still render ‘clean’ html in public view, without the unneeded SharePoint html and javascript. Control pages render in public view including the required ScriptManager and Form runat=”server”; and in editing view also with only occurrence of ScriptManager and server-side HtmlForm.

Data architecture considerations and guidelines for retrieving SAP data via Duet Enterprise into SharePoint

In Duet Enterprise FP1 it is relatively easy to generate a Gateway Model, as long as your SAP data sources live up to the required constraints. The challenge and intellectual work transfers to the data architecture, for deciding on and defining the proper data service interfaces given the application context.

This blog is earlier published on SAP Community Network Blogs

Real-time retrieval versus Search indexing

Bringing SAP data into SharePoint based front-ends can take different forms. You can unlock the SAP data real-time via ExternalList, to display and even edit the SAP data via the familiar SharePoint List UI metaphor. If the row-based List format doesn’t fit, you can develop a custom SharePoint webpart. You can also access the SAP data via the extensive SharePoint Search architecture and capabilities. With the improved Enterprise Search in SharePoint 2010 [or FAST if also purchased by customer organization], more and more new applications will be architected as search-driven. In context of SAP data think of searching from SharePoint context for a certain Supplier administrated in PLM, a Customer in CRM, et cetera.

Duet Enterprise can facilitate all of the above scenarios, in combination with the strengths of the SAP and SharePoint landscapes. However, the different nature of these scenarios may well result into different approaches for the data integration architecture. In the case of real-time retrieval of SAP data to render into SharePoint, it is advisable to limit the amount of data retrieved per SharePoint - Gateway - SAP backend interoperability flow. Don’t put unnecessary SAP data on the line that is not being used in the front-end: only include the fields of the SAP data that will be displayed in the UI and are relevant in this application context for the end-user. Search however has a different context. It is feasible to search on any of the field data of the SAP entity. This requires that all that SAP field data must be indexed upon SharePoint Search crawling time. So here it is advisable to not limit the amount of SAP data retrieved, but instead retrieve as much of the SAP data that expresses some functional value.

What if the same SAP data is to be unlocked both via SharePoint ExternalList, as via SharePoint Enterprise Search? The data integration architectures of these are thus contra dictionary: reduce the retrieved SAP data versus give me all. Well, nothing prohibits you from constructing multiple data integration pipelines, tuned for the different scenarios. For Search, it’s Gateway Model returns in the Query method all fields that contain functional value. And for the real-time ExternalList, that Gateway Model returns in the Query method only those fields that will be rendered as list columns.

In Duet Enterprise 1.0, constructing the Gateway Model takes considerable time. This is an obstacle for constructing multiple Gateway Models with different data representation / signature, as it can easily double your Gateway mapping effort and time. Luckily in Duet Enterprise FP1 it is relatively easy to generate a Gateway Model, as long as your SAP landscape data sources live up to the required constraints.

Complex SAP data source structure

If the SAP data is conceptually a flat structure, it is sufficient to have a single Gateway Model to retrieve the data into SharePoint context. However, we all know that SAP (or, business) data is typically of a more complex structure: hierarchical with multiple child entities. When you want to retrieve such a structure into SharePoint, you basically have 2 options.

The first is to flatten the structure. This approach suffers from some disadvantages. In case of multiple occurrences of the same child type (e.g. Ordered Item); how many of them should you include in the flattened representation? All, or an arbitrary limited number? Also, for the end-user a flattened structure can be conceptually wrong and counter-intuitive: Ordered Item information is of different functional level as the Purchase Order information.

The second option is to maintain the hierarchical SAP structure within the Gateway Model architecture. For this you need to generate a Gateway Model for the parent SAP data entity, as for each type of its child data entities. At SharePoint client-side you associate the 2 resulting External ContentTypes, to establish the parent-child relation. SharePoint BCS respects the association between the External ContentTypes when it operates on the data. In case of SharePoint search, the child-associations are crawled in the context of the parent data, and either the parent or child SAP data entity will appear in search results. The BCS Profile page of a parent data entity renders also the data of its child entities.

In case of real-time retrieval, you can apply the BCS Business Data webparts: add to the SharePoint page both a Business Data WebPart for the parent entity and a Business Data Related List WebPart for the child data entities. The latter will display the child SAP data entities of the parent SAP data entity that is selected in the Business Data List web part.

Mind you, the user experience of this setup is not always intuitive. In such case, it can prove better to build an own custom presentation. An example of this is outlined in Working with complex SAP business entities in Duet Enterprise.

More [functional] robustness issues with OOTB InfoPath Forms Services

Earlier I blogged about some robustness issues I encountered with the application of InfoPath Forms Services in an external facing SharePoint site. The basis of the framework developed in that project is re-used in context of the company's intranet. Hereby I ran into some new robustness issues. I'll describe them here, and also the outline of the applied solutions.

1. Placing XmlFormView webpart on a SharePoint page conflicts with inline server codeblocks in MasterPage
   Immediately upon adding the out-of-the-box Microsoft.Office.InfoPath.Server.Controls.XmlFormView webpart on a publishing page in the company's intranet, SharePoint throwed an exception:

   Source: System.Web
   Message: The Controls collection cannot be modified because the control contains code blocks (i.e. ).
   Stacktrace:
   at System.Web.UI.ControlCollection.Add(Control child)
   at Microsoft.Office.InfoPath.Server.SolutionLifetime.ScriptIncludes.AddCssLinkToHeader(HtmlHead head, String href)
   at Microsoft.Office.InfoPath.Server.Controls.XmlFormView.TryToAddCssLinksToHeader()
   at Microsoft.Office.InfoPath.Server.Controls.XmlFormView.OnDataBindHelper()
   at Microsoft.Office.InfoPath.Server.Controls.XmlFormView.OnPreRender(EventArgs e)
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.WebControls.WebParts.WebPart.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint

   
   

   First analysis learned that the problem was related to the applied SharePoint MasterPage: it manifested itself with 2 of our custom MasterPages, but not on other custom nor the standard (Publishing) Masterpages. Inspection of the particular custom MasterPages disclosed the following server-side code:

   <% Response.Write("<script>var Collaboration_SiteRelativeUrl='" + SPContext.Current.Site.Url + "';</script>"); %> 

   
   

   The inclusion of this inline server codeblocks resulted in the conflict with the server-side operation of the XmlFormView webpart. The solution was to replace the inline server codeblocks-tags '<%' and '%>' approach by a HtmlHead (which is a webcontrol) databinding approach:

   
   

   <script>var Collaboration_SiteRelativeUrl="<%# SPContext.Current.Site.Url %>";</script> 

   
   

   And in the MasterPage codebehind:

   
   

   protected override void OnLoad (EventArgs e)
   {
       base.OnLoad(e);
       Page.Header.DataBind();
   } 

   
   

   
   

   
   


2. IE 6 gives a runtime Javascript error in case of RichTextBox control on the form
   While demonstrating the potential of the self-service WebForms functionality to internal stakeholders, I was asked to modify an earlier published InfoPath form with the placement of a RichTextBox control. After re-publishing the InfoPath form to the SharePoint target site, the form still correctly displayed in the content page; including the added RichTextBox control. However, when positioning the mouse in the RichTextBox control, the IE6 browser (Red: still the company's standard) reported a runtime Javascript error. Also the RichTextBox toolbox was not displayed, leaving the control pretty useless wrt a plain TextBox control.
   
   With these problem symptons, I kinda got a 'Deja-Vu' feeling. It looked very much like the problem earlier encountered with displaying InfoPath forms in IE 6. And indeed: via Javascript debugging I detected that the client-side runtime error manifested itself in the same OOTB InfoPath Inc/Core.js method as before: ErrorVisualization.ComputeAbsoluteTop(). Was then the problem caused by the usage of relative CSS-positioning applied in the custom branding, this time it proved completely InfoPath related. I could easily reproduce the same problem within a standard SharePoint WebPart page, with a standard MasterPage. The explanation of the IE 6 Javascript problem is in the HTML that InfoPath Forms Services renders for a RichTextBox control: an embedded iframe element within the DOM document. Due to the fault behaviour of the IE 6 offsetParent method implementation in combination with the non-robust/error-prone implementation of the OOTB InfoPath method, this results in the runtime Javascript error.The solution was to runtime at browser/client-side expand that OOTB InfoPath method to make it robust for the IE 6 faulty CSS offsetParent implementation.
   

   
   

   
   

   
   

   
   


3. InfoPath re-publishing adds duplicate site columns to the ContentType
   
   Upon re-publishing an InfoPath formtemplate during the demonstration, I noticed another issue. It looked as if all promoted fields in the ContentType were now duplicate present. A quick internet search learned that this is a known InfoPath feature/behaviour, just something I myself was not aware of upto now. And neither something I was particular happy with. I think it is pretty ridiculous to add a complete other set of fields to the InfoPath ContentType each time you re-publish the associated formtemplate. You have some control-options in the InfoPath publishing wizard to prevent the creation of another set of promoted fields, but this is manual-dependent and thus error-prone; let alone it is also very cumbersone and clumsy for the functional managers (the first-class users of the self-service WebForms functionality in the company portal). Therefore I sought for a manner to afterwards correct the modified ContentType, by removing all duplicate FieldLinks. Ideally did would be done fully automatic in the scope of the InfoPath publishing, and thus transparent and invisible to the WebForms functional managers. Sadly, SharePoint does not have an EventReceiver signalling the creation or update of a ContentType. The solution is therefore to deliver a custom SharePoint application page via which the WebForms functional manager can afterwards fix the InfoPath modified ContentType. He/she selects in the dropdown the ContentType that [potentially] needs a cleanup. The custom application page then inspects
   the FieldLinks in the ContentType for duplicate occurences, and removes each one found. The changes (deletions) in the SiteCollection ContentType are propagated to all childs. Also each duplicate detected FieldLink is removed from the Fields collection of each SPDocumentLibrary that is associated with the selected InfoPath ContentType (code snippet).
   

   
   

Handling save conflict within SPList EventReceiver

Background

Publishing an InfoPath form to a SharePoint site creates a new contenttype in the site. Via the InfoPath publishing wizard you have some control over the structure of the created contenttype. But in essence the control is very limited when compared with explicitly provision a contenttype yourself [see also previous post Administrate data from InfoPath form in a self-provisioned ContentType for further explanation why I prefer this above the automatic created information architecture entities]. However, it is not always possible or viable to explicit provision the InfoPath utilized IA entities. The big selling point of InfoPath is that it enables self-service (web)form-definition by functional management. A key cornerstone in this self-services proces is the InfoPath publishing functionality to allow functional managers to themselves distribute the InfoPath forms to a SharePoint site.
One of the aspects missing with the InfoPath publishing managed contenttypes is including the InfoPath promoted fields in the display form. Displaying all InfoPath form fields is in particular handy when quickly browsing the data submitted to a SharePoint forms library.

Solution approach

To fix the functional shortcoming requires to afterwards make the promoted fields displayed. This is not something you want to burden the functional managers with. The aim is therefore to execute it automatically in the context of the target SharePoint site. The ideal moment would be when the InfoPath managed contenttype is created or updated. SharePoint however does not provide an eventreceiver for these events. The next best moment is when the contenttype is associated with the forms library to which the InfoPath form data is submitted. Here you neither have a direct event notifying the contentype association. But you can receive indirect notification via SPListEventReceiver::FieldAdded. The idea is then to check each added field whether it is an InfoPath promoted field, and if so to alter its definition so that the field will appear in the standard SharePoint display form.

Issues encountered

So, easily said and done. Associate the custom ListTemplate with a SPListEventReceiver, and fill in the FieldAdded method. In my local development image it worked like a charm: after you associate via the SharePoint GUI a document library with an InfoPath contenttype, all promoted fields of that contenttype appear in the document library display form. But after deploying to the central SharePoint development farm, I ran into the first issue. Upon associating a library with a contenttype, SharePoint faulted with a weird message: The specified program requires a newer version of Windows.

Despite this misleading error message, it was immediate clear that the issue was caused by the custom FieldAdded eventhandler:

Apparently, SharePoint does not allow to update a list field in the runtime context of that field being added to a list.

Solution outline

Solution is to isolate the execution of the field-update from the field-addition runtime context. Instead of direct updating the field, schedule this work via a worker thread. One extra aspect to take into account is that the scheduled worker threads can still run into a parallel update conflict:

Save Conflict

Your changes conflict with those made concurrently by another user. If you want your changes to be applied, click Back in your Web browser, refresh the page, and resubmit your changes.

  at Microsoft.SharePoint.Library.SPRequest.UpdateField(
        String bstrUrl,String bstrListName, String bstrXML)
  at Microsoft.SharePoint.SPField.UpdateCore(Boolean bToggleSealed)
  at Microsoft.SharePoint.SPField.Update()
  at WebformulierFieldsHandler.Worker.b__1()
 

The likelihood of these save conflicts increases with the number of promoted fields in the contenttype, and can already manifest itself when that number is 4 to 5. It can be made functional robust by implementing a retry mechanism in the worker thread.

Tip: use Lookup iso Choice field for (semi)fixed set of defined values

Often, in your SharePoint information architecture there are some fields identified with a set of defined allowed values. A common approach is then to apply the Choice fieldtype herefore. For real fixed set of possible values this is a very sensible approach. For instance, for datatype sex; we'll have 'male', 'female', and well 'unknown' in case of doubts. However, more than often the set is (semi)fixed: departments within a company, the customer base of an IT consultancy, etc. In such situation, usage of Choice is not flexible nor user-friendly towards the functional managers of the application. For each modification, it is required to change the definition of the Choice-based field. And one must then also not forget to propagate this change to the lists on which the field / site column has been applied (direct, or via contenttype). All this is more technically doing, than SharePoint functional management. A better approach is to utilize the Lookup fieldtype, and refer to another list with in it the currently known set of defined values (masterdata values actually). Whenever a modification to this set is in order, functional management can suffice with adjusting this masterdata list. Actually, this is just sane data model normalisation. Like in such context, in addition to defining the set of allowed values, it is also possible to augment them with more details. For instance, department name (allowed value), and in another list column more details of the department.

Something you'll have to take into account when applying this datamodelling, is a peculiarity upon provisioning the SiteColumns. The way Lookup SiteColumn are administered in the SharePoint content database, is with the Guid of the referential SharePoint list. This results in a problem when provisioning the SiteColumns the SharePoint standard way via Feature, with the field specifications in XML. The ID of the referential list is typically unknown at coding/specification time, and will be different per environment provisioning. This is a known issue, and so there is also a known resolution. I'm not going to describe it here, of even try to take the credits for it. Frankly, I've used a blog-entry of Chris O'Brien, Creating Lookup columns as a feature, as start information source for hinting how to solve this sequential provision issue.