ListTemplate cached in Sandboxed process

Sandboxed solution are a great addition to SharePoint 2010 capabilities whenever it is required to deploy own customizations. However, be aware of the restrictions, and some oddities. Today I encountered such an unexpected behaviour. ListTemplates are one of the SharePoint artefacts that allow themselves to be deployed via a Sandboxed solution. In the initial version of a custom ListTemplate, some errors and omissions were made with respect to schema.xml and some forms. I corrected these, and deployed (= uploaded + activated) the Sandboxed solution. To discover next that my changes were not present; not within the Feature-provisioned ListInstance based on the ListTemplate, nor within a manually created FormsLibrary instance based on the custom ListTemplate. It took an AppPool recycle to have the latest deployed ListTemplate definition become effective in the sitecollection. Tip: always recycle the AppPool before re-deploying a Sandboxed solution.

HowTo omit the standard xml-namespaces in WCF MQ requests

Earlier I reported that we encountered an issue with communication via IBM MQ WCF Channel over a clustered IBM WebSphere MQ-queue. The problem was already known within IBM and the fixed software ready on the shelves, and naturally we were very willing to be a first beta-tester. With this IBM fix, the EndPointNotFound problem was indeed solved. Sadly, we next ran into another issue. This time it manifested itself in our own service implementation at the receiving side. Upon receiving a MQ-request, the service responded with a technical error. Due the holiday season it took some elapse time before the right people were available and able to investigate the cause. The problem analyse led to the suspicion that the presence of xml-namespaces in the webrequest was unexpected, and therefore the service refused the request:

<ServiceRequestName xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"> ... </ServiceRequestName>

Both namespaces are default included when serializing a System.ServiceModel.Channels.Message instance from inside WCF client processing. You can however prevent this by implementing an subclass of System.ServiceModel.Channel.Body.Writer, and override the OnWriteBodyContents method. Next inject this method within the WCF Channel:

    System.ServiceModel.Channels.Message message =
        Message.CreateMessage(
            MessageVersion.None,
            string.Empty,
            new XmlBodyWriter<TRrequest>(request));

    var responseMessage = webmqclient.Request(message);

...

public class XmlBodyWriter<TBody> : BodyWriter
{
    private readonly TBody request;

    public XmlBodyWriter(TBody request) : base(true)
    {
        this.request = request;
    }

    protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
    {
        WriteBodyContents(SerializeRequest(), writer);
    }

    private static void WriteBodyContents(StringBuilder output, XmlWriter writer)
    {
        using (var reader = new StringReader(output.ToString()))
        {
           using (XmlReader xmlReader = XmlReader.Create(reader))
            {
                writer.WriteNode(xmlReader, true);
            }
        }
     }

    private StringBuilder SerializeRequest()
    {
        var output = new StringBuilder();

        using (var xmlWriter = XmlWriter.Create(output,
            new XmlWriterSettings { OmitXmlDeclaration = true, Encoding = Encoding.UTF8 }))
        {

            var serializer = new XmlSerializer(typeof(TBody));

            // Service does not expect xml-namespacing; including the standard
            // xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            // xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            // Omit these from the serialized request
            XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
            ns.Add("", "");
            serializer.Serialize(xmlWriter, request, ns);
        }

        return output;
    }
}

With this modification to the serialization processing of the WCF MQ request, the service at the receiving WebSphere MQ side now replies with a correct functional response.

Deleted SPWeb in RecycleBin can obstruct deactivation of Sandboxed Solution

The other day, upon deactivating a Sandboxed Solution in our test farm, SharePoint aborted on it with the message:

Cannot access web-scoped feature {GUID} because it has references to a site with id {GUID}.
System.ArgumentException: Value does not fall within the expected range.
at Microsoft.SharePoint.SPWebCollection.get_Item(Guid id)
at Microsoft.SharePoint.SPFeatureEnumeratorBase.GetCachedWeb(SPSite site, Guid webId, Guid featureId)

We earlier encountered this problem in the development farm. Thorough investigation by SharePoint operations together with development led to the conclusion that the SharePoint content database had reached a corrupt state due deletion of a SPWeb. On that SPWeb a Feature provisioned via the Sandboxed Solution had been activated. And now after deletion of that SPWeb, apparently a lock internal in the SharePoint content database was present obstructing disablement of that Feature. And since it is not recommended - and certainly unsupported - to manually alter a SharePoint content database, there seemed no other realistic approach to get out of this erroneous situation as by recreating and reprovisioning the entire sitecollection.

For the development instance, this was an acceptable pragmatic solution. However, in our test environment a lot of content is already created by end-users. Just throwing away the SPSite and replacing it by a brand new, is not viable. The only remaining solution seemed to afterwards restore the backed-up content into the new SPSite. Not undoable, but it will take time [to setup and execute the site content restore; and next to validate the correctness and completeness of it]. And moreover, it will result in a loss of trust by our end-users and customer on the robustness of SharePoint 2010 as application platform. If it happens once [actually twice], what guarantee is there it will not happen again?

Luckily, then I had a smart thought while discussing the problem symptons with a co-developer. If the problem appeared to be caused by the deletion of a SPWeb, would it then help to restore this SPWeb instance from the RecycleBin? Worth a try. And guess what: it did!! After the earlier deleted SPWeb had been restored from the recyclebin, the earlier activated Sandboxed Solution could next successfully be deactivated.

Misleading SecureStore message with InfoPath Forms Services

In current project we publish an InfoPath 2010 formtemplate to SharePoint 2010 sitecollection. In the formtemplate, multiple controls are populated with data retrieved from a single SharePoint list via application of owssvr.dll and filtering views on that list. The SharePoint WebApplication is set up as Claims-Based. A consequence is that from within InfoPath Forms Services context it is not possible to authenticate via owssvr.dll webservice to the SharePoint list. The proper solution for this is to retrieve the data via Universal Data Connections, and let each UDCX authenticate itself to SharePoint. Either by explicit authentication; that is including the credentials in all UDCX files. But preferably by using the SecureStore; so that the credentials are maintained in a single location and not readable included in the UDCX files.

So far for the theory. In real practice, we encountered a problem with this setup: our formtemplate failed to retrieve the filtered data. In the ULS following message was logged per owssvr.dll/XmlQuery data connection upon opening the formtemplate in browser:

InfoPath Forms Services Maintenance 82lm Information Delegation was attempted for Secure Store application APPL_InfoPathService. (User: 0#.w|domain\useraccount, IP: , Request: http://appl.dev.hosting.corp/Pages/orderform.aspx)

This seems to indicate as if the individual logged-in user has no permission to access/use the SecureStore ApplicationID. Inquiry with Operations refuted that suspicion: the SSS ApplicationID was configured for 'All Authenticated Users'. The actual cause appeared that the service account configured in the SSS ApplicationID no longer had permission/read access to the SharePoint site collection. Instead of above message, I would have preferred a direct 401 or AccessDenied cause-indication in the ULS...

Publication on 'Succesful disclosure of SAP data and processes in SharePoint'

In the May-edition of the Dutch Software Development Network (SDN) magazine I published an article co-authored with Marcel Kempers on the why, and approach for how succesful disclose SAP data + processing within a SharePoint based solution. As of this month, the publication is also online available. Mind you; it is in Dutch...

Personalization approach / settings administrated outside content database

Personalization is an important functional aspect in my current SharePoint 2010 project. Personalization on itself is a phrase that can have multiple meanings and appearances. Examples:

• Customize a page for your own preferences; via personal settings of webpart properties
• Content targeting; on basis of the visitors profile

SharePoint enables page personalization via the Provider model, inherited from ASP.net. Implication of the standard SharePoint personalization approach is that each authenticated user gets an own copy per personalized page in the content database. The personalized settings are thus administrated in the content database, similar as in case of shared webpart properties. A disadvantage of this approach is that this makes it very difficult to (web content) manage the page. A content manager can update the (template) page, but none of the content changes are automatically propagated to the individual personalization page copies in the content database.

In our application scenario, we have the additional functional requirement that it must be viable to extract management information reports of the personalized settings: what type of user (profiles) typically select personalized setting X, choose to close webpart Y etcetera. Although not impossible, it is rather impractical to extract this management information when the personalized data is stored in the content database. The schema of the SharePoint content database must be treated as internal black box, and cannot be relied on. The schema also does not support ad-hoc querying for answering varying management info requests.

We also have User Experience related requirements: the users must be able to personalize, without being aware or be directly confronted with SharePoint. It should be intuitive and natural, without explicit [noticable] personalization-setting modus. Examples of requested personalizations are tuning the webpart behavior via settings, and dynamically reposition the webparts in the page.

These combined end-user and management requirements effectively rank out the utilization of standard SharePoint personalization. So what’s a viable alternative? The one we came up consists of the following elements:

1. [Standard] shared webpart properties for enabling the content managers to set the initial and default values for personalizable settings. The shared value is regularly stored in the SharePoint content database, and applied for each visitor which has not [yet] explicitly personalized the shared setting.
2. Storage of personalization settings in an own SQL Server database
3. The iGoogle-like reposition behavior via jQuery and webpart zones tagged as droppable zones
4. An abstract base PersonalizationWebPart; that handles both the server-side aspects of personalization (retrieving + applying personalization settings, as well as saving them), as the client-side (webpart movement); and that implements virtual methods for concrete subclasses to hook into.
5. Custom AudienceProvider to derive on-the-fly whether visitor is within a certain audience by checking setting stored in the own SQL Server database [thus no need for User Profiles, nor compiling of Audiences]

Extra and a major advantage of this personalization approach is that there no longer originates a copy of the page being personalized. If the content manager updates the page, by adding or removing a webpart, modifying content on a publishing page; these changes are automatic and immediate effective for all the visitors, whether one has personalized the page or not.

Make AudienceFilter webpart setting visible without User Profile Service Application started

In current project we aim to apply SharePoint's audiencing mechanisme for content targetting. As audience filters we'll use SharePoint group membership, we will not derive the audience from User Profile properties. In our current state of SharePoint farm, the User Profile service application is even not available .

One of the advantages of SharePoint audiencing is that it is out-of-the-box available for every webpart; standard SharePoint and custom webparts. However, when we intended to apply an audience filter to a standard ContentEditorWebPart in our test-environment, we were confronted with a missing Audiencing setting in the webpart toolpart. Search on the web points to prerequiste of the User Profile service application being activated. However, this is neither possible in our farm infrastructure planning, nor needed for the manner in which our application will use audiencing, that is on basis of SPGroup memberships.

Via reverse engineering SharePoint code [using .NET Reflector] I found out what directly determines the visiblity of the AudienceFilter webpart property. The responsible AdvancedToolpart class does a check on the web.config for config property "SharePoint/RuntimeFilter": if the property is not present, or the referred assembly is not valid, none of the webparts in this SharePoint webapplication will display the AudienceFilter webpart property in their toolpart. I validated this by outcommenting in web.config the property; for instance the toolpart of a ContentEditorWebPart then misses the AudienceFilter setting. After reinstating the web.config property, and reopening the toolpart; the webpart property is visible again. And thus available for usage for filters on basis of SharePoint groups, or a custom AudienceProvider.