Friday, June 6, 2014

Winner of SAP Microsoft Unite Partner Connection Customer Impact and Value Award with VIEW solution

I am very proud that The Next View has won the 2014 edition of the SAP Microsoft Unite Partner Connection Award for Customer Impact and Value with our VIEW solution!!
VIEW stands for Virtual Integrated Enterprise Workplace. For us, the VIEW concept is not new, I for instance already defined and published on this blog parts of the Conceptual Solution Architecture back in 2009. But it is only up to now with the advent of the modern integration technologies SAP NetWeaver Gateway, Duet Enterprise, Gateway for Microsoft (GWM), plus the availability of standard functional products from our partner Cordis Solutions, that we are enabled to actually realize the VIEW concept in a cost-effective manner.
So, what does VIEW stand for? VIEW is a new operating concept in which the central concept is that of an employee-centric mindset. In VIEW, we strive to optimally enable organization’s employees to perform their daily work-related activities. In current reality, this work execution often means that one must operate in (and switch to) multiple applications and systems, monitor multiple tasklists in different environments (SAP, SharePoint, Oracle, Outlook, ...), remember login credentials of the diverse systems. With VIEW, we relieve the employees from all this ‘IT landscape’ hashle. Instead that the employees must explicit go to all the different applications, in VIEW we collect all the work execution in a central place: the VIEW landing page.
And this VIEW landing page has multiple appearances: desktop and mobile, to fit in with the nowadays reality that employees are [willing to be] always connected to the business systems, to at minimum monitor and act on urgent matters.
Although the VIEW concept does not mandates this, the typical platform for the desktop appearance is SharePoint; as this is in majority of the organizations the declared [by Information Management, Enterprise Architecture] business webplatform. The VIEW landing page is merely added as new employee business application within the already present SharePoint-based intranet.
Also for the mobile appearance, SharePoint can be the platform [certainly SharePoint 2013 has made some big steps on enabling us to provide a proper mobile appearance, a.o. taking into account the diversity in mobile devices]. But the mobile landing page can also be hosted outside SharePoint, e.g. via SAP Mobile Platform [SMP], an hybrid App [HTML5/CSS/PhoneGap], and other alternatives. Again, the VIEW concept does not put strict restrictions on this.
If you want to learn more about the VIEW solution, check out the SAP Microsoft Unite Partner Collection solution brief.

Sunday, May 4, 2014

GWPAM renamed into SAP Gateway for Microsoft, short GWM

‘SAP NetWeaver Gateway Productivity Accelerator voor Microsoft’, that is a mouthful. This long name arose in several variations of how the product is actually being called in the market and press, resulting in confusion and making it difficult to find product information. SAP product management has acknowledged this drawback effect, and they now have a more catchy product name assigned: SAP Gateway for Microsoft, abbreviated as GWM.
Other GWM news besides this name change is the availability of GWM Service Pack 02. Key parts of SP2 are alignment with Visual Studio 2012 [modern UI] look and feel, support for Microsoft Office 2013, support of SAP Fiori services consumption, and a project template for building your own Excel Add-In. Earlier versions of GWM already include the capability to link an Excel sheet to a Gateway REST/OData service for (mass) data management. This feature is aimed for business people who arrange their own (master)data management via Excel. The new Excel Add-in template is designed specifically for the developer, to build your own innovative solutions with Excel UI platform.
See also: SAP CodeTalk: GWPAM Update / Interview with Holger Bruchelt

Saturday, May 3, 2014

Corrupt SharePoint account breaks Duet Enterprise workflow publishing

One of our Duet Enterprise customers where I had a.o. configured the workflow capability, requested my support as workflow tasks where no longer published from SAP workflow backend into SharePoint. I performed cause analysis in the combined SAP + SharePoint landscape, starting with inspecting the logs – SAP backend, SAP Gateway and SharePoint systems. The SAP Gateway log contained a recurring error log “logical port not found for routing url”.
I checked in SIMGH on the Gateway system, and found a valid routing url that referred to ‘LOGICALPORTFORWORKFLOW’; and in SOAMANAGER an active logical port with this name. But when I tried to ping the /IWTNG/CO_TASKFLOW_WEB_SERVICE consumer proxy, http error 405 was returned. The actual cause of the malfunctioning Workflow publishing from Gateway into SharePoint was the SharePoint service account that Gateway uses to authenticate against the Duet Enterprise webservice '/_vti_bin/OBAWorkflowService.asmx' on the SharePoint server. It had somehow become corrupted, and lacked the authorization to access the WSDL document of the Duet Enterprise service. After recreating/resetting the new service, account workflow publishing is working fine again.
Note: the logged message ‘No logical port found’ can be misleading; in the ABAP code it is the generic catch of whatever problem is encountered upon trying to propagate SAP task notifications from SAP Gateway to SharePoint.

Sunday, April 27, 2014

HowTo send JSON data from WCF service with special characters

The manner to return special characters within the JSON response of a (SharePoint-based) WCF REST service, consists of 3 essential elements:
  1. Define the method response as System.IO.Stream;
  2. Set the charset of the JSON response to 'ISO-8859-1';
  3. Apply UniCode encoding to the JSON response string.
Code example
[ServiceContract(Namespace = "...")] public interface IRESTService { [WebInvoke(Method = "GET", UriTemplate = "/RESTMethod?$parameter1={parameter1}&...", RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)] System.IO.Stream RESTMethod(string parameter1, ...); } [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)] public class RESTService : IRESTService { public Stream RESTMethod(string parameter1, …) { OutgoingWebResponseContext context = WebOperationContext.Current.OutgoingResponse; context.ContentType = "application/json; charset=ISO-8859-1"; string jsonData = ...; return new MemoryStream(UnicodeEncoding.Default.GetBytes(jsonData)); } }

Monday, April 7, 2014

Service Application Proxies runtime reported with locale-dependent TypeName

The SharePoint architecture enables shared usage of service applications across farms. A typical setup is a Shared Services farm that hosts the service applications, and multiple consumer / front-end farms that host the webapplications. In the consumer farm(s), each individual webapplication is associated with the service applications it requires. For instance, webapplication A is associated with Secure Store and Business Connectivity Services, and webapplication B is associated with Secure Store and Secure Token service applications.
In a distributed SharePoint architecture you cannot programmatically access the service applications in the local farm. Instead you must access via the service application proxy that is associated with the webapplication. The retrieval model of this is weakly-typed, you retrieve the desired service application proxy by string-comparision (!) on the proxy TypeName. This weakly-typed usage model is errorprone; you can easily make a typo error that goes unnoticed at compile time. But you will be immediately aware upon the first runtime test of the code.
However, this weakly-typed model incorporates another strange behavior: the reported TypeName is locale dependent! In my local SharePoint image, I tested against an EN-US sitecollection to retrieve the BCS service proxy, filtering on TypeName:
BdcServiceApplicationProxy proxy = webApplication.ServiceApplicationProxyGroup.Proxies.SingleOrDefault( p => p.TypeName == "Business Data Connectivity Service Application Proxy" ) as BdcServiceApplicationProxy;  
With above code, I successfully retrieve the BCS service application proxy.
But the same code running against a webapplication in the integration-test farm does not select the BCS service application proxy. In Central Admin I verified that the webapplication is associated with the BCS service application. So what is the problem here? The cause rather surprised me: the sitecollection in the integration-test environment is provisioned via a Dutch-locale site definition. And as unexpected side-effect the TypeName of the associated service application proxies are now all reported in their Dutch localization name:
Fix for the above code is to make it locale-independent. For BCS this is possible by filtering on TypeName pattern ‘Business Connectivity Service’;
BdcServiceApplicationProxy proxy = webApplication.ServiceApplicationProxyGroup.Proxies.SingleOrDefault( p => p.TypeName.Contains("Business Data Connectivity") ) as BdcServiceApplicationProxy;  
For other service application proxies it might be required to compare the TypeName against the established Resources value:
private static string _BCSApplicationProxyTypeName = null; // Derived from Microsoft.SharePoint.CoreResource static string BCSApplicationProxyTypeName { get { if (String.IsNullOrEmpty(_BCSApplicationProxyTypeName)) { Assembly _aIntl = Assembly.Load("Microsoft.SharePoint.intl, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"); ResourceManager _BusinessDataRM = new ResourceManager("Microsoft.BusinessData.strings", _aIntl); _BCSApplicationProxyTypeName = _BusinessDataRM.GetString( "ApplicationRegistry_BdcServiceApplicationProxy_TypeName"); } return _BCSApplicationProxyTypeName; } } BdcServiceApplicationProxy proxy = webApplication.ServiceApplicationProxyGroup.Proxies.SingleOrDefault( p => p.TypeName == BCSUtility.BCSApplicationProxyTypeName ) as BdcServiceApplicationProxy;  

Sunday, April 6, 2014

Asynchronous invocation of BCS

Return early from an elapse-time-expense BCS handling

Functional scenario: from UI context, check the user input via serverside business validation, and if functional correct propagate the same user input for processing in the business backend.
In this scenario, the user needs only to wait for the period of the validation step; if the outcome reports functional errors these must be reported to the user so (s)he is able to correct. In case of no errors, the subsequent business processing can be performed transparent to the user, no need to explicitly wait / block until this (potential time-expensive) processing is completed in the backend.
Technical context: HTML5/JavaScript client, that interopates with SAP business backend via a SharePoint RESTful services, which internally utilizes SharePoint BCS to invoke Duet Enterprise / Gateway services.

Inconvenient operating BCS from background thread

Initially, I thought it would be simple to realize the sketched execution context: handle the first step on the main thread of the SharePoint WCF REST service; if errors detected report these back; if no errors noticed, delegate the elapse-time expense propagation step to a background thread, and directly return from the main thread to unlock the client.
However, SharePoint / BCS reality proofed less trivial…
In my first attempt, I directly interoperate against BCS on the background thread:
main thread:
PerformCreation job = new PerformCreation() { _createMethod = createMethod, _jsonBody = jsonBody, _siteId = SPContext.Current.Site.ID, _userToken = SPContext.Current.Site.UserToken }; Thread backgroundThread = new Thread(new ThreadStart(job.PerformCreate)); backgroundThread.Start();  
background thread:
public void PerformCreate() { using (SPSite site = new SPSite(_siteId, _userToken)) { BCSModuleCall.Create(site, _createMethod, _jsonBody); } }  
When I executed this, the BCS Create invocation faults with a ‘Cannot complete this action’ exception. This orginates from the BCS internal method call ‘CalculatePermissionsForCurrentThread’, that checks whether in the context of the executing thread, the user has the rights to perform the BCS operation. Via code reverse engineering, I detected that the 'Microsoft.SharePoint.BusinessData.Infrastructure. BdcAccessControlList.AccessCheck(BdcRights rights)' method requires an active SPRequest context. And that only lives on the main / SharePoint thread.
To arrange that the BCS Create method is executed within the context of an active SPRequest, in my second attempt I changed the code to serverside issue from the background thread a webrequest to the SharePoint REST service. Although this technically works (that is, the ‘CalculatePermissionsForCurrentThread’ succeeds and the BCS Create operation is successful performed), a drawback is that you loose the credentials of the logged-on SharePoint user. It is not possible to propagate the claims-authenticated identity of the logged-on user to the webrequest invoked via WebClient class. It always authenticates and thus executes with the Application Pool identity.
public void PerformCreate() { var claimIdentity = (Microsoft.IdentityModel.Claims.ClaimsIdentity) Thread.CurrentPrincipal.Identity; var upnClaim = claimIdentity.Claims.FirstOrDefault( c => c.ClaimType.Equals(Microsoft.IdentityModel.Claims.ClaimTypes.Upn, StringComparison.InvariantCultureIgnoreCase)); if (upnClaim != null) { string upn = upnClaim.Value; WindowsIdentity windowsIdentity = Microsoft.IdentityModel.WindowsTokenService.S4UClient.UpnLogon(upn); var wic = windowsIdentity.Impersonate(); try { // SharePoint BCS API requires a current SPContext. In this thread, // there is no SPContext. To make sure the create is executed within // a current SPContext, create the entities by issuing REST request. WebClient webClient = new WebClient() { UseDefaultCredentials = true, BaseAddress = _baseAddress }; webClient.Headers.Add(HttpRequestHeader.ContentType, "application/json;charset=utf-8"); webClient.UploadDataAsync( (new Uri(String.Format("{0}/CreateItems", _baseAddress))), "POST", Encoding.UTF8.GetBytes(_jsonBody)); } finally { wic.Undo(); } } }  
In our scenario, it is a necessity that all data updates are authorized and audited for the issueing user. So although technical it is possible to delegate in this manner the BCS data update to a serverside background thread, from the perspective of data governance it is not allowed.

Solution: client-side operate BCS in asynchronous mode

Ultimately, I skipped the approach of serverside splitting up the BCS invocation in a synchronous blocking part, and an asynchronous non-blocking part, and instead decided to manage this from client-side. At first, issue a REST JSON request for the data validation; and let the user wait (block) on the response. If no validation errors detected, issue from the browser with the same JSON object the second update request in a fire-and-forget behaviour.
var checkUrl = bindingContext.$root.siteUrl + "/_vti_bin/WebApi/BCSAccessService.svc/CheckItems"; var checkCall = jQuery.ajax({ url: checkUrl, type: "POST", data: JSON.stringify(json), success: function (data) { if (Utilities.CheckNoErrorInResponse(data.BODY.ET_MESSAGES)) { var submitUrl = bindingContext.$root.siteUrl + "/_vti_bin/WebApi/BCSAccessService.svc/CreateItems"; // Data is already validated: //Fire-and-forget ajax async request and ignore response. var submitCall = jQuery.ajax({ url: submitUrl, type: "POST", data:JSON.stringify(json)}); alert("User input is validated and submitted."); window.parent.jQuery('#dialogForm').dialog('close'); } else { Utilities.DisplayResponseMessages(data.BODY.ET_MESSAGES); } } }).fail(Utilities.ErrorHandler);  
And this works out fine: the end-user earlier regains the input-control and responsiveness in the front-end application, while the update in the back-end is still processed in the background with the credentials of the logged-on user.

Friday, March 14, 2014

Use JSOM from a plain webpage

With the richness of available javascript libraries (knockout.js, angular.js, …) combined with SharePoint JSOM, you can nowadays build some real nice functionality without need of server side code. Drawback however seemed that to be able to use JSOM, the script needs to be included on a SharePoint page. In some cases this is fine, when you utilize JSOM/Knockout/Angular to enrich a SharePoint page (sitepage, publishing page) with dynamic clientside behavior. But there are also scenarios that the clientside behavior is on its own (e.g. functionality exposed to the user via a dialog), and then it is undesired to include the SharePoint overhead: bloated HTML and an heaver page payload.
So the question is then, can you JSOM outside the context of a SharePoint page? Seems evident that it must be possible, after all it boils down to identifying the right resources to send to the browser, and that then does the work. However, it appeared to be a little less easy and logical to setup such a context. And the Microsoft provided information is insufficient and unclear.
After some trial-and-error, I now have it working. The required elements in the setup are the following:
  1. The page must be rendered by SharePoint; can be from the layouts directory (note: not as an application page), or from a SharePoint Document Library
  2. The page cannot be a plain .html; as it is required to utilize SharePoint server rendering pipeline
  3. Include a SharePoint.ScriptLink to deferred load 'sp.js'
  4. First trick: include '<SharePoint:FormDigest runat="server"></SharePoint:FormDigest>' in your page
  5. Second trick: encapsulate the javascript code that invokes JSOM, within a '<form runat="server"> -- </form>'. Especially this aspect took me some trial/discovery time.
With these 5 aspects applied, SharePoint JSOM can be used from a lean and lightweight webpage, without the SharePoint master page overhead.
Example:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<%@ Page Language="C#" %>
<%@ Register Tagprefix="SharePoint" Namespace="Microsoft.SharePoint.WebControls"
  Assembly="Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
<html dir="ltr" xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
  <meta name="WebPartPageExpansion" content="full" />
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <title>Plain JSOM usage</title>
  <SharePoint:ScriptLink language="javascript" name="sp.js" OnDemand="false" runat="server" Localizable="false" LoadAfterUI="true"/>
  <SharePoint:FormDigest ID="FormDigest1" runat="server"></SharePoint:FormDigest>
 </head>
<body> 
  <form id="form1" runat="server">
     <script language="ecmascript" type="text/ecmascript">                                    
         $(document).ready(function () {
             EnsureScriptFunc("sp.js", "SP.ClientContext", function () {
                 var ctx = new SP.ClientContext();
                 var site = ctx.get_site();
                 ctx.load(site, 'Url');
                 ctx.executeQueryAsync(function (s, a) {
                     ko.applyBindings(new Demo.AppViewModel.ProcessesVM(site.get_url()), document.getElementById('DemoUsage'));
                 });
              });
         });
    </script>
 </form>
</body>