1st lessons learned from applying Content Enrichment web service callout

In my previous post I reported how I successfully applied the new capability within SharePoint 2013 Enterprise Search to enrich the crawled content. That is immediately my first lesson: SharePoint 2013 Content Enrichment web service callout is far more easy to utilize compared to the Item Processing capability of FAST Search Server for SharePoint 2010. You can setup a working scenario in matter of hours, while FAST Item Processing typically costs you days. Moreover as it is badly documented.

However, I also do have some lessons learned to keep in mind when utilizing Content Enrichment Service callout:

• Be aware that if you leave out a trigger in the ContentEnrichmentConfiguration object, the callout will occur for each crawled data element. In a production scenario this can seriously increase the total crawling time.
• Be aware that the Content Processing pipeline invokes the Content Enrichment web service via anonymous service calls. This means it is not viable to deploy as web service in a SharePoint webapplication, as this typically is an authenticated web resource. Best is to deploy the Content Enrichment web service in an own anonymous IIS website.
• You can only configure one single ContentEnrichmentConfiguration object per SharePoint Search Application (SSA). The consequence is also that you can only use one single Content Enrichment web service to enrich the crawled content in this SSA. This is a serious limitation!! I found a post with a creative solution to workaround the single endpoint limitation: Using Multiple Endpoints as a Content Enrichment Web Service in SharePoint 2013 Search. But a real solid approach has to come from Microsoft itself, by removing the limitation of only 1 ContentEnrichmentConfiguration per SSA.

Content Enrichment callout to properly map crawled data format

In combination with SharePoint BCS, SharePoint Enterprise Search can crawl external business systems as datasources. Starting SharePoint 2013, BCS can now also consume REST/OData services. In an OData service response, the type of all data fields is by nature text. The values of SharePoint Search crawled properties are all also of raw text. In the Enterprise Search content processing pipeline this text-based value can be mapped to a managed property of specific datatype, e.g. text, integer, decimal, date and time. In order to successful and meaningful map from text dataformat to specific datatype, the text format must be parsable via current localization / culture into that specific datatype. In case not, the mapping will fail and the value of managed property will be null.

When crawling an external data source, you typically do not have control over the dataformat. In case the dataformat does not match the localization format, the mapping thus fails.

Example: OData returns date-information in format ‘YYYYMMDD’; the default localization datetime formats do not support this and the mapped managed property of type Date and Time contains null value:

In such situation, you can utilize SharePoint 2013 content enrichment capability to explicit parse the crawled non-localized dataformat.

Source: Custom content processing with the Content Enrichment web service callout

The approach is as follows:

1. Remove the mapping from the Managed Property of type Date and Time
2. Create a new Managed Property of type text, and map this to crawled property
   
3. Create an implementation of IcontentProcessingEnrichmentService; with the ProcessItem method set to parse ‘YYYYMMDD’ into a Property<DateTime>
4. Configure SharePoint Search Application to map the new managed property (input) to the datetime managed property (output)
5. Issue a full crawl on the business data content source.

The result in SharePoint index:

Tip for ‘Secure Store Service application is not accessible’

This week I was forced to restart our SharePoint 2013 single-server system. After the server had restarted, Duet Enterprise 2.0 Single Sign-On gave an error: The Secure Store Service application Secure Store Service is not accessible. The full exception text is: An error occurred while making the HTTP request... Opening Secure Store Service Application in Central Management displayed the same error. I checked IIS Manager: the applications pools of the SharePoint Web Services were all running. Retried to recycle, but problem stayed. Even after a brute IISReset.

I already was preparing myself for inspecting the full identity trail of the app pool account to detect whether comprimized, and/or potentially even repair Secure Store Service application. But in a bright moment, I thought of inspecting ‘Manage Services on Server’. Secure Store Service showed in the overview as ‘Started’. As quick trial-and-error attempt, I stopped and next (re)started the Secure Store Service. And guess what: this resolved the issue! Easy infra fix after all, when you think of it.