HowTo send JSON data from WCF service with special characters

The manner to return special characters within the JSON response of a (SharePoint-based) WCF REST service, consists of 3 essential elements:

1. Define the method response as System.IO.Stream;
2. Set the charset of the JSON response to 'ISO-8859-1';
3. Apply UniCode encoding to the JSON response string.

Code example

 [ServiceContract(Namespace = "...")]
 public interface IRESTService
 {
    [WebInvoke(Method = "GET", 
     UriTemplate = "/RESTMethod?$parameter1={parameter1}&...",
     RequestFormat = WebMessageFormat.Json,
     ResponseFormat = WebMessageFormat.Json,
     BodyStyle = WebMessageBodyStyle.Bare)]
    System.IO.Stream RESTMethod(string parameter1, ...);
}

[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)]
public class RESTService : IRESTService
{
   public Stream RESTMethod(string parameter1, …)
   {
       OutgoingWebResponseContext context = 
          WebOperationContext.Current.OutgoingResponse;
       context.ContentType = "application/json; charset=ISO-8859-1";
  
       string jsonData = ...;
       return new MemoryStream(UnicodeEncoding.Default.GetBytes(jsonData));
   }
}

Augment GWPAM AddIn project to consume JSON dataformat

GWPAM consumption of efficient JSON format only possible when Gateway service supports OData V3

Standard the GWPAM generated AddIns consume the Gateway REST OData service via the AtomPub dataformat, and not via the more data-efficient JSON format. The reason is that the used .Net WCF Data Services Client library is not capable of consuming JSON dataformat. Because I prefer to consume REST services via the mobile-friendly JSON dataformat, I set out to augment a default generated GWPAM Outlook AddIn project to consume via OData JSON. As the service consumption is within .Net code, this is mostly a Microsoft.Net tail. However, as it turns out that to be even able to consume JSON in the Microsoft WCF client library, an important prerequisite is imposed on the consumed REST service; it also extends to there.

JSON consumption via WCF Data Services Client Library

Initially, WCF Data Services Client library only supported consumption of REST services via AtomPub dataformat. As of release 5.1 it is also possible to consume JSON, but with the limitation that the JSON format must be OData V3. Prerequisite for the consumed service is thus that it must be able to provide its data in OData JSON V3 dataformat. For the older OData versions (V1, V2), it is not [yet] possible to consume JSON in WCF Data Services Client library.

Steps to augment to GWPAM project to consume JSON dataformat

First prepare your project to be able to handle the JSON consumption:

1. Install the latest WCF Data Services Client Library, at present this is 5.6.0. Installation is done via NuGet Package Manager, and must be applied to each GWPAM project in which you want to consume via JSON dataformat. The effect per project is that reference ‘Microsoft.Data.Services.Client’, version 5.6.0 is added; and that the (via GWPAM project template) already existing references ‘Microsoft.Data.EDM’, ‘Microsoft.Data.OData’, ‘System.Spatial’ are also upgraded to version 5.6.0.
2. If present, remove the reference ‘System.Data.Services.Client’ from the GWPAM project(s). Note: Visual Studio 2010 default installs with that older WCF Data Services Client library.
3. Install WCF Data Services 5.3.0 RTM Tools Installer in Visual Studio; also via NuGet.

Next setup the consuming code to consume the REST service via OData. That is, if the service supports it

4. Validate that the consumed service is able to return data in the required JSON dataformat, OData V3. Query for this the $metadata of the service, and validate that it contains “m:MaxDataServiceVersion='3.0'”
5. Generate an EDMModel for the service, via ‘Add Service Reference’ (iso ‘Add SAP Service Reference’). Open the generated service proxy, and change the accessibility of ‘GeneratedEdmModel’ to public
6. Edit the earlier generated SAP data services client code to consume the service via JSON dataformat
1. In the constructor, set to initialize for ‘System.Data.Services.Common.DataServiceProtocolVersion.V3’
2. Set the Format to link to the generated EDM Model: this.Format.LoadServiceModel = GeneratedEdmModel.GetInstance;
3. Set the Format to useJson: serviceContext.Format.UseJson()
4. Copy the implementation + usage of methods ‘ResolveTypeFromName’ and ‘ResolveNameFromType’ from the generated service proxy in step 5. Modify the code to correspond to the full name of the SAP service proxy.

5. Query the consumed REST service for JSON format; either via querystring param ‘$format=JSON’, or via Http Request Header ‘accept
   = application/json’

That's it

And that’s it. Now the GWPAM based Office AddIn is able to consume the REST Service via the more efficient JSON dataformat. Note that for the consumption itself it does not even has to a Gateway REST Service; it is also possible to consume REST services from other webservice platforms. But the GWPAM project templates and code generated are focussed on standard SAP NetWeaver Gateway services + data models; WFService as one evident example.

Architecture of a cloud-enabled SAP/SharePoint document viewer

End 2012 IntelliDocx LLC approached us to design the global architecture for their new DocSet.ECM for SAP and SharePoint product suite, and build DocSet.ECM for Plant Maintenance (PM) as first member. The DocSet.ECM functionality is enable end-users to search, navigate and access documents linked with SAP entities, directly within SharePoint based workplaces. The SAP entities come from any of the known SAP Business Suites - PM, Sales, HCM, SRM. IntelliDocx had an ambitieus deadline, as they aimed to release the PM solution to the public and market analysts at the European SharePoint Conference in Copenhague (Februari 2013).

The base functional requirement for the product suite is that it renders a view in SharePoint on documents linked in SAP. But to make it a winning solution, they have more requirements. The products must be very user-intuitive and friendly; enabling arbitrair end-users to utilize it without requiring inside SAP knowledge. Product deployment must be fit for both on-premise as cloud. More and more organizations nowadays have decided, or are looking into, SharePoint Online utilization iso maintaining an own on-premise SharePoint farm. And of course, the product should support both SharePoint 2010 and 2013; or at least be prepared for 2013 ahead.

System Architecture

The diagram below illustrates the architecture we proposed to address the requirement set. (And which we indeed applied to successfully deliver the DocSet.ECM for PM product to market, within the tight timeframe [News release]).

Essentials

DocSet Viewer

• Deployed via Sandboxed Solution
• UserControl iso aspx page
  • allows to add the DocSet viewer to regular SharePoint site and content pages / intuïtieve integration within customer’s SharePoint based portals
• Server-side only rendering the html
  • No server postback handling
  • No interactive dependency on SharePoint server (may be cloud-hosting)
• Client-side handles communication with the DocSet webservice, and dynamic UI-rendering
  • By-pass SharePoint server
  • Use jQuery:
    • Invoke DocSet JSON webservice
    • Render tree-view of hierarchical document folders
    • Drag-and-drop of documents (for Move + Add)
• Configuration:
  • Endpoint of DocSet webservice (may be cross-domain)

DocSet webservice

• WCF JSON webservice
• Enable CrossDomain invocation via JSONP
  • CORS is not yet widely supported; and in particular not within the enterprise domain
• Utilize ERP-Link RFC Connector for SAP / Microsoft.NET interoperability with DocSet.ECM RFCs
  • The architecture allows usage of other SAP / Microsoft connectors, e.g. Duet Enterprise, but we decided to initially use the ERP-Link connector as IntelliDocx told us a lot of their customers already use the Gimmal ERP-Link Suite.
• Configuration:
  • Connection settings to DocSet RFC
  • Address of SharePoint server hosting the DocSet viewer, to allow cross-site requests from this possible remote / other domain server

DocSetViewer App - SharePoint 2010 + 2013

The DocSet.ECM for SAP and SharePoint product suite is fit to deploy and run within both SharePoint 2010 and 2013, without any code-change; and hosting supported on-premise and within the Office365 cloud.
  

Related info: IntelliDocx