Avoid corrupted site columns due feature re-activation

One of our clients reported an issue that provisioned site columns get corrupted after deinstall of the provisioning feature. The feature deinstallation is a step within the repair of erroneous situation in which sandbox solution per accident is also deployed as global farm-based solution. This incorrect deployment requires a fixture because an effect is that features now appear as duplicate entries in the list of site(collection) features: one installed via the sandbox solution (correct, administrated in the sitecollection's content database), and one installed via the farm solution (incorrect, administrated in the farm configuration database).

On first thought the simple fix is to deactivate the provisioning feature that originates from the farm deployment, remove the feature, then retract the global farm-solution, and remove it from the farm solution store. Next activate the feature deployed via sandbox-solution to arrive at the correct deployment situation. However, in a test execution we experienced that this approach gives an error upon the re-provisioning of the site columns: The local device name is already in use.

Error details in ULS log:

Unable to locate the xml-definition for FieldName with FieldId '<GUID>', exception: Microsoft.SharePoint.SPException: Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED)) ---> System.Runtime.InteropServices.COMException (0x8000FFFF): Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED)) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetGlobalContentTypeXml(String bstrUrl, Int32 type, UInt32 lcid, Object varIdBytes) at Microsoft.SharePoint.Library.SPRequest.GetGlobalContentTypeXml(String bstrUrl, Int32 type, UInt32 lcid, Object varIdBytes)

To come with a solution, I started with a root-cause analysis. Why are the provisioned site columns not completely deleted from the site upon its feature deactivation? The explanation is that the provisioning feature also performs a contenttype binding to the Pages library, and that in our testcase a page was created based on that contenttype. This effectually results in the contenttype being kept ‘in usage’ by the Pages library. On feature deactivation the site columns can still be removed at sitecollection level, but the contenttype not anymore due the descendant sibling binded to the Pages library.

The real problem however lies in the 'removed' site columns. They are deleted from sitecollection level, but due the preserved contenttype (Pages library) their definition has remained in the sitecollection's content database, with the same ID as on sitecollection level (note this is different for a contenttype binded to a list, that gets a new ID based on the ID of the source/parent contenttype at sitecollection). SharePoint administrates per provisioned artifact whether the origin is a feature, and if so effectively couples the artifact to that feature. SharePoint disallows these artifacts by automatically deleted or modified by another feature. As result the feature re-activation halts with an error when trying to (re)provision the sitecolumns that are still present deep down in the content database, coupled to the Pages library.

With this SharePoint-internal insight, I was able to come up with a faultproof approach to fix the 'duplicate features' issue. The trick is to initial leave the feature definition that originated from the erroneous farm solution in the configuration database, deploy the sandbox-solution (stores the feature definition in the sitecollection's content database), activate the feature with the same feature id from that sandbox solution. Now the feature activation proceeds completely without errors, and restores the site columns. Ultimately the farm-based solution can then be retracted from the farm solution store.

Note: I came to this insight by inspecting on SQL level. We all know it is not allowed to perform changes on SharePoint content database level (or loose your Microsoft support), but it is perfectly ‘SharePoint’-legal to review and monitor on SharePoint content database level.

Used/useful SQL statements:

SELECT * FROM ( SELECT *, Convert(varchar(512), CONVERT(varbinary(512), ContentTypeId), 2) As key FROM [ContentTypeUsage]) as T where key like '%< contenttypeid >%'

SELECT tp_Title FROM ( SELECT *, Convert(varchar(512), CONVERT(varbinary(512), ContentTypeId), 2) As key FROM [ContentTypeUsage] Join AllLists on ContentTypeUsage.ListId = AllLists.tp_ID ) as T where key like '%< contenttypeid >%'

More [functional] robustness issues with OOTB InfoPath Forms Services

Earlier I blogged about some robustness issues I encountered with the application of InfoPath Forms Services in an external facing SharePoint site. The basis of the framework developed in that project is re-used in context of the company's intranet. Hereby I ran into some new robustness issues. I'll describe them here, and also the outline of the applied solutions.

1. Placing XmlFormView webpart on a SharePoint page conflicts with inline server codeblocks in MasterPage
   Immediately upon adding the out-of-the-box Microsoft.Office.InfoPath.Server.Controls.XmlFormView webpart on a publishing page in the company's intranet, SharePoint throwed an exception:

   Source: System.Web
   Message: The Controls collection cannot be modified because the control contains code blocks (i.e. ).
   Stacktrace:
   at System.Web.UI.ControlCollection.Add(Control child)
   at Microsoft.Office.InfoPath.Server.SolutionLifetime.ScriptIncludes.AddCssLinkToHeader(HtmlHead head, String href)
   at Microsoft.Office.InfoPath.Server.Controls.XmlFormView.TryToAddCssLinksToHeader()
   at Microsoft.Office.InfoPath.Server.Controls.XmlFormView.OnDataBindHelper()
   at Microsoft.Office.InfoPath.Server.Controls.XmlFormView.OnPreRender(EventArgs e)
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.WebControls.WebParts.WebPart.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Control.PreRenderRecursiveInternal()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint

   
   

   First analysis learned that the problem was related to the applied SharePoint MasterPage: it manifested itself with 2 of our custom MasterPages, but not on other custom nor the standard (Publishing) Masterpages. Inspection of the particular custom MasterPages disclosed the following server-side code:

   <% Response.Write("<script>var Collaboration_SiteRelativeUrl='" + SPContext.Current.Site.Url + "';</script>"); %> 

   
   

   The inclusion of this inline server codeblocks resulted in the conflict with the server-side operation of the XmlFormView webpart. The solution was to replace the inline server codeblocks-tags '<%' and '%>' approach by a HtmlHead (which is a webcontrol) databinding approach:

   
   

   <script>var Collaboration_SiteRelativeUrl="<%# SPContext.Current.Site.Url %>";</script> 

   
   

   And in the MasterPage codebehind:

   
   

   protected override void OnLoad (EventArgs e)
   {
       base.OnLoad(e);
       Page.Header.DataBind();
   } 

   
   

   
   

   
   


2. IE 6 gives a runtime Javascript error in case of RichTextBox control on the form
   While demonstrating the potential of the self-service WebForms functionality to internal stakeholders, I was asked to modify an earlier published InfoPath form with the placement of a RichTextBox control. After re-publishing the InfoPath form to the SharePoint target site, the form still correctly displayed in the content page; including the added RichTextBox control. However, when positioning the mouse in the RichTextBox control, the IE6 browser (Red: still the company's standard) reported a runtime Javascript error. Also the RichTextBox toolbox was not displayed, leaving the control pretty useless wrt a plain TextBox control.
   
   With these problem symptons, I kinda got a 'Deja-Vu' feeling. It looked very much like the problem earlier encountered with displaying InfoPath forms in IE 6. And indeed: via Javascript debugging I detected that the client-side runtime error manifested itself in the same OOTB InfoPath Inc/Core.js method as before: ErrorVisualization.ComputeAbsoluteTop(). Was then the problem caused by the usage of relative CSS-positioning applied in the custom branding, this time it proved completely InfoPath related. I could easily reproduce the same problem within a standard SharePoint WebPart page, with a standard MasterPage. The explanation of the IE 6 Javascript problem is in the HTML that InfoPath Forms Services renders for a RichTextBox control: an embedded iframe element within the DOM document. Due to the fault behaviour of the IE 6 offsetParent method implementation in combination with the non-robust/error-prone implementation of the OOTB InfoPath method, this results in the runtime Javascript error.The solution was to runtime at browser/client-side expand that OOTB InfoPath method to make it robust for the IE 6 faulty CSS offsetParent implementation.
   

   
   

   
   

   
   

   
   


3. InfoPath re-publishing adds duplicate site columns to the ContentType
   
   Upon re-publishing an InfoPath formtemplate during the demonstration, I noticed another issue. It looked as if all promoted fields in the ContentType were now duplicate present. A quick internet search learned that this is a known InfoPath feature/behaviour, just something I myself was not aware of upto now. And neither something I was particular happy with. I think it is pretty ridiculous to add a complete other set of fields to the InfoPath ContentType each time you re-publish the associated formtemplate. You have some control-options in the InfoPath publishing wizard to prevent the creation of another set of promoted fields, but this is manual-dependent and thus error-prone; let alone it is also very cumbersone and clumsy for the functional managers (the first-class users of the self-service WebForms functionality in the company portal). Therefore I sought for a manner to afterwards correct the modified ContentType, by removing all duplicate FieldLinks. Ideally did would be done fully automatic in the scope of the InfoPath publishing, and thus transparent and invisible to the WebForms functional managers. Sadly, SharePoint does not have an EventReceiver signalling the creation or update of a ContentType. The solution is therefore to deliver a custom SharePoint application page via which the WebForms functional manager can afterwards fix the InfoPath modified ContentType. He/she selects in the dropdown the ContentType that [potentially] needs a cleanup. The custom application page then inspects
   the FieldLinks in the ContentType for duplicate occurences, and removes each one found. The changes (deletions) in the SiteCollection ContentType are propagated to all childs. Also each duplicate detected FieldLink is removed from the Fields collection of each SPDocumentLibrary that is associated with the selected InfoPath ContentType (code snippet).
   

   
   

Handling save conflict within SPList EventReceiver

Background

Publishing an InfoPath form to a SharePoint site creates a new contenttype in the site. Via the InfoPath publishing wizard you have some control over the structure of the created contenttype. But in essence the control is very limited when compared with explicitly provision a contenttype yourself [see also previous post Administrate data from InfoPath form in a self-provisioned ContentType for further explanation why I prefer this above the automatic created information architecture entities]. However, it is not always possible or viable to explicit provision the InfoPath utilized IA entities. The big selling point of InfoPath is that it enables self-service (web)form-definition by functional management. A key cornerstone in this self-services proces is the InfoPath publishing functionality to allow functional managers to themselves distribute the InfoPath forms to a SharePoint site.
One of the aspects missing with the InfoPath publishing managed contenttypes is including the InfoPath promoted fields in the display form. Displaying all InfoPath form fields is in particular handy when quickly browsing the data submitted to a SharePoint forms library.

Solution approach

To fix the functional shortcoming requires to afterwards make the promoted fields displayed. This is not something you want to burden the functional managers with. The aim is therefore to execute it automatically in the context of the target SharePoint site. The ideal moment would be when the InfoPath managed contenttype is created or updated. SharePoint however does not provide an eventreceiver for these events. The next best moment is when the contenttype is associated with the forms library to which the InfoPath form data is submitted. Here you neither have a direct event notifying the contentype association. But you can receive indirect notification via SPListEventReceiver::FieldAdded. The idea is then to check each added field whether it is an InfoPath promoted field, and if so to alter its definition so that the field will appear in the standard SharePoint display form.

Issues encountered

So, easily said and done. Associate the custom ListTemplate with a SPListEventReceiver, and fill in the FieldAdded method. In my local development image it worked like a charm: after you associate via the SharePoint GUI a document library with an InfoPath contenttype, all promoted fields of that contenttype appear in the document library display form. But after deploying to the central SharePoint development farm, I ran into the first issue. Upon associating a library with a contenttype, SharePoint faulted with a weird message: The specified program requires a newer version of Windows.

Despite this misleading error message, it was immediate clear that the issue was caused by the custom FieldAdded eventhandler:

Apparently, SharePoint does not allow to update a list field in the runtime context of that field being added to a list.

Solution outline

Solution is to isolate the execution of the field-update from the field-addition runtime context. Instead of direct updating the field, schedule this work via a worker thread. One extra aspect to take into account is that the scheduled worker threads can still run into a parallel update conflict:

Save Conflict

Your changes conflict with those made concurrently by another user. If you want your changes to be applied, click Back in your Web browser, refresh the page, and resubmit your changes.

  at Microsoft.SharePoint.Library.SPRequest.UpdateField(
        String bstrUrl,String bstrListName, String bstrXML)
  at Microsoft.SharePoint.SPField.UpdateCore(Boolean bToggleSealed)
  at Microsoft.SharePoint.SPField.Update()
  at WebformulierFieldsHandler.Worker.b__1()
 

The likelihood of these save conflicts increases with the number of promoted fields in the contenttype, and can already manifest itself when that number is 4 to 5. It can be made functional robust by implementing a retry mechanism in the worker thread.