The Quest for searching thorough advice on best-practices SAP / SharePoint interoperability

What started out as a rather simple inquiry question on the SDN forum SAP NetWeaver .NET Technologies, is now transforming in a more in-breadth discussion on getting best approaches to apply within SAP / SharePoint interoperability. André Fischer, SAP employee, posted a direction to the whitepaper Interoperability between SAP NetWeaver Portal and Microsoft SharePoint Technologies. Although a good overview document of the possible approaches, that is also its shortcoming: it's an overview, but it does not give the reader solid advice on best approaches to apply in specific situations. Not strange, that is not the purpose of this whitepaper. And also, the answer on such questions is for a major part influenced by the particular enterprise situation, thus difficult to generalize. But when involved in such a concrete enterprise situation, you are in need of such guidelines and advice.

So we try to 'challenge' André to come up with more concrete advice. If you're interested, follow the forum-thread to see whether he accepts and responds to this...

Tags: SAP NetWeaver Microsoft SharePoint integration interoperability architecture guidelines roadmap

A reflection on 'Enterprise Portal'

Wikipedia contains the following definition for the concept of an Enterprise portal: An enterprise portal, also known as an enterprise information portal (EIP) or corporate portal, is a framework for integrating information, people and processes across organizational boundaries. It provides a secure unified access point, often in the form of a web-based user interface, and is designed to aggregate and personalize information through application-specific portlets. One hallmark of enterprise portals is the de-centralized content contribution and content management, which keeps the information always updated..

I agree with this definition on the conceptual level. However, I want to supplement it with a fundamental notion on a more architectural level: Enterprise Application Integration. Especially the concept of business process integration is achieved by combining the functionalities of multiple business applications within one single employee workplace. The High Performance Workplace.

Patterns in Enterprise Portals

Industry analysts and portals vendors identify the following set of enterprise portals patterns / types:

1. Content portals
   Aggregates enterprise content from many sources throughout the enterprise. Typically augmented with search functionality to allow employees to find the requested information quick and easily. Authorization applied at content level to protect sensitive information.
2. Transaction portals
   Extend content portals by not only displaying documents and textual information (unstructured data), but also provide the employee with the means to access and operate individual application functionality direct from within the portal. The applications are provisioned via individual portlets in the portal, and provide on-the-fly / realtime business data (structured data), and allow users to interact with the application via the portlet.
3. Collaboration portals
   Support a team in collaborating on a mutual task. This can be a project, an assignment, or the performance of the daily and regular team responsibilities. The collaboration features enable the employee to self-organize the ad-hoc work, and share + work on the same information through a dedicated team workspace.
4. Process portals
   Compared with the individual and non-connected silos in a transaction portal, the process portal provides the employees with an integrated business process that spans multiple back-end systems. The employee is not aware of single applications, but can rather enlist in the total business process.

Enterprise Portals platforms

An enterprise portal is not a shrinked-wrapped product, but requires configuration, customization and provisioning to be of added value for an individual enterprise situation. Custom development is often required to achieve enterprise specific functionalities and integration. Portals vendors position their enterprise portal product more and more as an Application Integration Platform. A.o. SAP uses the phrase "Applistructure" to point out the combination of out-of-the-box enterprise applications and functionalities, together with application infrastructure. Applistructure capabilities which are expected in an enterprise portal platform offering include the following:

• Presentation services
• Content aggregation
• Data management
• Enterprise search support
• Webservices / interoperability support
• Team collaboration support
• Document handling
• Enterprise Content Management
• Web Content Management
• Workflow support and integration
• Business Process Management
• Business Intelligence functions
• Personalization
• Site design and structure
• Branding
• Authentication and Identity Management
• Authorization, roles management
• Operational management support
• Multi-channeling
• Functional management support

Commercial and Open Source

The enterprise portal product space consists currently of numerous parties. Some are the well-known traditional technology-stack vendors, but more and more also (commercial) Open Source platform offerings come up. Here is a list of some of the major current players in this field:

• IBM - WebSphere Portal
• Microsoft - SharePoint
• SAP - NetWeaver / Enterprise Portal
• Oracle - BEA WebLogic
• Sun Microsystems - GlassFish Web Space Server
• Vignette Corporation - Vignette Portal
• Tibco - PortalBuilder
• Liferay - Liferay Enterprise Portal
• Red Hat - JBoss Portal
• Jahia
• Alfresco
• Covisint
• DotNetNuke

Trends and new capabilities

Incorporate Web 2.0 capabilities (Wiki's, blogs, social computing), Mashups, Rich User Experience, Mobile usage, Cloud Services, incorporation of Self Service applications

Related sources

• Introduction to Enterprise Portals - Why they Benefit IT and the Business
• Gartner about (Enterprise) Portal Scenarios
• Gartner - Magic Quadrant for Horizontal Portal Products, 2008
• ZDNet - 12 predictions for Enterprise Web 2.0 in 2008
• ZDNet - 8 predictions for Enterprise Web 2.0 in 2009

Schematic overview of SAP enabling technologies for SSO

I came accross a figure displaying a graphical overview of the diverse paths and approaches for achieving Single Sign-On within SAP. This nicely summarizes at high-level the available approaches as discussed in the SAP/Microsoft Alliance whitepaper on SSO (see posting Studying Single Sign-On approaches for SAP and Microsoft):

Tags: SAP NetWeaver Microsoft SharePoint integration interoperability SSO identity management IDM IAM

Silverlight and WCF Faults

The WCF FaultContract behaviour is a nice functionality to report functional faults to the invoking client. The WCF service implements this by throwing a FaultException<T> instance. The proxy side catches this, and extracts the functional error.

Well, that's the general idea. With a standard .NET WCF proxy client this works perfectly. However, in Silverlight you ran into some problems. At first, the generic version of FaultException is not available. So you cannot catch and distinguish at WCF client side a FaultException<T>, but must resort to catching generic FaultException. But the real problem is that Silverlight treats all WCF Fault HttpResponse as general CommunicationException iso SOAP FaultResponse. The underlying cause of this is that the WCF service side default sets the HTTP status code to 500 in case of an unhandled exception, even in case of the SOAP FaultException. At Silverlight side (and also Flash) any non-200 HTTP response is generally treated as a communication error. Thereby ignoring/loosing the functional fault information that is responded by the WCF service:

Several posts suggest to prevent this situation by modifying the WCF service side to not return HTTP 500 errorcode for a WCF Fault. This will work if you yourself also have the control over the WCF service. Still, personally I feel this is the wrong approach. The problem is namely at Silverlight side, so you should correct it there. Additional benefit is that you are so also prepared for SOAP faults of external webservices (WCF or other).

My solution would be to implement a custom channel for Silverlight. I thankfully used the example code on this blog for the initial setup. Next I wanted to hook into the proper place in the custom channel to convert 500 HTTP status codes into 200 codes.

However, I've to admit I failed to correct the problem at the Silverlight client side. The reason is that the HTTP transfer is handled up in the stack via inaccessible Silverlight internal clients (System.Net.BrowserHttpRequest, InternalWebRequest, ...), and that all information of the HTTP Response is lost when reaching the context of the channel. Even Reflection cannot give an helping hand here, since the responsestream is closed already when returning to the channel context.

So for now I have to satisfy with letting the own WCF service also return HTTP status code OK (200) when a functional fault is thrown. The bigger drag is the inconsistency in all of this between Silverlight versus standard .NET CLR; in the latter the SOAP response for s:fault is always given priority above the HTTP status code. I think this is as it should be. Moreover given the ongoing discussion what is the correct and standard HTTP status code in case of SOAP FAULT (SOAP and HTTP status codes (200 vs 500 etc), Issue #12: HTTP Status Codes 500 v 200).

Tip - solve DateTimeControl's datepickerframe positioning fault for IE8

The SharePoint DateTimeControl uses clientside javascript to runtime position the datepickerframe nearby the imagebutton. When browsing via IE8, the frame is however positioned incorrect, at a visually unrelated position.

I inspected the SharePoint datepicker.js client library from the 12hive\layouts folder to locate the cause of this. At first I had the impression that it was due to the usage of dynamic properties (pixelTop, pixelLeft, pixelRight). As of IE8, DynamicProperties are namely no longer supported to comply with CSS standards.

The real cause however is the usage of the offsetParent method to calculcate the display position for the popped-up datepickerframe. It appears that IE8 returns another value for this method than earlier IE versions for hidden objects: they all return the document.body object as their offsetParent. This causes the pickerdateframe to be displayed at an unexpected place.

A quick fix is to force the browser in IE7 emulation mode in the HTML header:

<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7" />

A more elegant fix is correctly position the frame by circumventing the offsetParent malfunctioning. I solved the problem by runtime extending the standard 'clickDatePicker' method, and so make sure that for IE8 the datepickerframe.style.display is unequeal 'none' upon requesting it's offsetParent.

Studying Single Sign-On approaches for SAP and Microsoft

In todays typical heterogenous enterprise IT environment, an end-user works with multiple IT products and applications which have varying authentication mechanisms. In such situations a well-desired functionality is to have Single Sign-On (SSO) enabled. The last days I (re)did some research into this area.

The concept of SSO has a direct and narrow relationship with Identity Management. At the area of IM several interesting new developments are lately either done or announced. Sun (almost Oracle nowadays) has its OpenSSO Enterprise flagship, Microsoft recently gave indepth insight into it's forthcoming "Geneva" initiative. Both technologies/products are representatives of the so-called Claims-Based Identity model. And the beauty is that both IM technologies conform to the Security Assertion Markup Language (SAML) federation standard as a basis for interoperability and ease of collaboration. In a joint co-operation, Sun and Microsoft have produced a whitepaper that discusses the interoperability possibilities between the 2 Identity Federation approaches, and gives answers on typical questions. Microsoft and Sun also have some working showcases to demonstrate the interoperability.

For the specific area of SSO in an heterogenous SAP / Microsoft IT landscape, the SAP-Microsoft alliance has an excellent overview whitepaper available: Unleash the Power of Single Sign-On with Microsoft and SAP. A sidemark is that this document appeared in september 2007. Later developments as the mentioned OpenSSO and "Geneva" are therefore not included. Still, to my opinion the whitepaper is not outdated, but simple no longer complete. The description and positioning of the included SSO approaches is still valid and useful. And you can easily complement yourself the 2007-temporal overview of SSO alternatives with the new technologies and products.

Tags: SAP NetWeaver Microsoft SharePoint integration interoperability SSO Identity Management IDM IAM SPNego OpenSSO

Tip - Silverlight's AsyncCallbackException on invoking WCF service

A Silverlight application deployed to the testserver gave a browser error upon invoking a WCF service. This WCF service is running on the same testserver. The error message is:

Error: Unhandled exception in Silverlight2 application [AsyncCallbackException]

According to the callstack, the error occurred somewhere in the completion of the asynchronous WCF call. This makes the bug analysis somewhat trickier, since you cannot simple use System.Windows.Browser.HtmlPage.Window.Alert then. The call completion namely does not run at the UI thread. However, searching around gave me an idea of the problem cause: I forgot to deploy a clientaccesspolicy.xml file within the folder of the deployed WCF service. Copied it there, and the call from the Silverlight client executes successfully.

Tip - jQuery setExpression and IE8

I ran today into a jQuery exception when testing a WebPart on a system with IE8 installed. In the custom WebPart I apply jQuery a.o. to round buttons and grid-headers. I'm utilizing jquery.corner.js for this. It appears that of IE8, Internet Explorer no longer supports dynamic properties. To prevent the Javascript exception, I made a small modification into 'jquery.corner.js' to check on the IE-version:

if (($.browser.msie) && ($.browser.version < 8.0))
    ds.setExpression('width', 'this.parentNode.offsetWidth');

Silverlight app errors when WCF Contract Has Sync Operations

For a WCF service, it does not matter what type of client is invoking it. .NET, Java, PHP or whatever based, as long has the consumer conforms to the W3* standards. For a WCF proxy however, it does matter what type of .NET client is using it to invoke WCF operations. Although a regular CLR-based client supports both synchronous and asynchronous invocations of the WCF service, a Silverlight client only allows asynchronous operations. In a Proof-of-Concept setup in which I have both a regular WebPart (server side operation) and a Silverlight application (client side operation) hosted in a SharePoint portal, I dealt with this by letting the Silverlight strictly use the asynchronous methods. This worked perfectly in my development environment. However, running at the remote test-server, I get an error indication in the browser when trying to invoke from the Silverligth client an asynchronous method on the WCF proxy. Even although the Silverlight code only uses the asynchronous methods, the Silverlight runtime checks at forehand whether the proxy contract also supports synchronous operations, and if so errors out:

Error: Unhandled Error in Silverlight2 Application [ContractHasSyncOperations]
Arguments:IExpensesHandlingAsync

The quick fix in the PoC is to exclude the synchronous methods when compiling the interface/contract for Silverlight client usage, and included in the .xap package. This can be done through a check on the conditional symbol SILVERLIGHT. This way, it is still possible to code-reuse and share a single interface sourcefile, both for non-Silverlight and Silverlight clients.

Tip - Generic robust approach for disposing both normal object as WCF proxy

Situation: a consumer object invokes a provider object to access functionality. The consumer operates against an interface to decouple it at compile-time from the actual implementation(s) of the provider side. The provider is runtime injected via Dependency Injection. After usage, the provider instance may have to be disposed, typical to release native resources (e.g. network or database connections). However, not all provider types need to have the IDisposable interface implemented. To prevent the consumer class from being cluttered with checks on IDisposable being implemented, I delegate this to an utility class in our framework. An example of the consumer usage is now:

In the implementation of the DisposableDecorator class the check is done whether the decorated/encapsulated object implements the IDisposable interface, and if so the Dispose method is propagated on to the encapsulated object. Simple, and it keeps the consumer code clean and focussed. However, when the encapsulated object is actually a WCF proxy, something special needs to be accounted for. The standard application of the using block may then result in an undesired effect. This occurs when an exception is thrown on the channel, putting the proxy in the Faulted state. When next trying to dispose the proxy, the closing of the proxy results in throwing a new CommunicationObjectFaultedException. The latter exception hides the original exception, which has put the proxy in the Faulted state. The solution is to first check whether the encapsulated object is a WCF proxy, and if so if it's communication state is Faulted. If that's the case, then invoke the Abort method on the proxy. Otherwise just maintain the Dispose propagation.

Tip - howto retrieve non-required Lookup column via List webservice

When running my deployed Silverlight application on a remote test server, I ran into a runtime problem. Debugging (since I deployed to a remote test server without Visual Studio support, I needed to resort using 'System.Windows.Browser.HtmlPage.Window.Alert'...), I discovered that the cause was a missing column in the returned XElement result of the GetListItemsAsync List webservice call. The missing column is a lookup to information in another list, and non-required. And moreover, this column is not included in the default view (it is in my local development environment, therefore I did not experience the problem earlier). So there's the explanation why it is not included. The solution is to explicitly specify via the ViewFields parameter to include this field in the result. However, even then I did not get the column included (?) The cause of this is that not all items in the list have a value for this non-required field. And for that you have to cope by specifying that the viewfield may be nullable:

Proud - Top 1 Contributor on SAP / .NET Interoperability Expert Forum

I really don't like to brag on my achievements. When however you receive as a 'Microsoftie' the positive acknowledgement from the SAP community on your contributions, this simple deserves to be proud of. So with humble modesty, I include a screenshot of today's status on the SAP Community Network Expert Forum on SAP NetWeaver .NET Technologies:

Tip – Provision custom publishing masterpage, pagelayouts and webparts always via a Feature with Site-scope, not Web-scope

The masterpage and webpart galleries for publishing sites live at the top level site (rootWeb) in a sitecollection, not at individual sites. This can be misleading, since every subsite get its own ‘_catalog/masterpage’ and ‘_catalog/wp’ folders. The Publishing functionality however only goes to the gallery at the top level site; for the masterpages (Site and System), pagelayouts as well as for building up the list of selectable webparts.
A best practice is to provision your custom masterpage, pagelayouts and .webpart definition files via a Feature. Thereby you must specify the level at which the feature is applied. For MasterPage, PageLayouts and .webpart files the appropriate level appears thus to be ‘Scope=Site’. However, it is also allowed to set ‘Scope=Web’, as long as you’re activating the web-scoped feature in the rootweb. But when you activate such a feature in a subsite, the results are less. It appears as if the SharePoint feature mechanism simple ignores the request to provision masterpage and pagelayouts into ‘_catalog/masterpage’, and .webpart files into ‘_catalog/wp’. They are simple not added anywhere. Not at the indicated subsite level, which would also be useless given the publishing behaviour. But also not at the rootweb level. Annoying is that no signal of this is given, no error reporting but simple ignored to provision the SharePoint files at the incorrect subsite level. Ok that stsadm doesn’t signal it, but I would have expected an error indication when activating the feature via the GUI. And at the least I expected to find some information about it in the extensive SharePoint logs in the 12hive. In reality none, instead the feature framework just silently ignores the feature specifications which are logically incorrect when activated at subsite level.

So, what then to do when you what to provision content to a publishing site at subsite level ? I recommend the following 2-steps approach:

1. Provision your masterpage, pagelayouts, and .webpart files to the catalog gallery at rootlevel; apply a Feature with scope = Site.
2. Provision your site structure (libraries, lists) and content via a Web-scoped Feature; and activate this against your specific site, the root or a child subsite. E.g. http:<root-entry>/Intranet

Workaround shortcoming of SPWebConfigModification behavior wrt explicit positioning childNode

I structurally apply the SPWebConfigModification class upon all my SharePoint deployments to propagate the needed web.config modifications to the target environment (be it my local VM or in test/staging/production environments, typically in a farm). Benefits of using this functionality are:

• All web.config modifications are done automatically. No need to have someone (yourself or in production, IT operations) do this manually by editing the web.config file(s);
• Addition of the modifications becomes part of your ALM process; they can be tested and validated before going to production, and repeatedly fully automatic executed;
• The same web.config modifications are propagated throughout the entire farm; and even when later on a new WFE or a new extension is added;
• Added modifications can also be neatly removed upon uninstalling your SharePoint application.

Although I’m a big fan of this functionality, I’m also aware of some of its shortcomings. One of this is that upon adding a new childNode, it will always be appended as the last childNode in its parent webconfig node. Sometimes this is not good enough, and you need to be able to have it inserted at a specific position. In my case it was needed for enabling an ExceptionHandler HttpModule. To have this correctly operating, it must be the first active httpModule in the pipeline. Typically however the httpModules section in the web.config already has multiple httpModule nodes added:

Initial HttpModules section for a new provisioned publishing portal

The standard behavior of SPWebConfigModification places my ExceptionHandler behind them, effectively making it non-function. I could resort to manually correct my / all the web.configs; yeah right, you gotta be kidding…

Instead I decided to devote some time to investigate how to make a full-automatic correction for this. Via Reflector I inspected the runtime operation of the SharePoint ApplyWebConfigModifications method. Then I did an attempt to ‘break in’ on this functionality by a combination of first removing the already present httpModule childnodes, next add my exceptionHandler so it be the first, and then add the original httpModule nodes again. However, this was unsuccessful; and the outcome unpredictable. I suspect that an explanation for this is that my application is not the ‘owner’ of the (re)moved nodes. But that is hard to determine, since that requires a deep insight in the inner internals of this SharePoint behavior. After this I came up with a solution to correct the sequence in the httpModules afterwards the invocation of SharePoint’s ApplyWebConfigModifications. And this works satisfactory!! I must admit it has one weakness in that this afterwards correction will not be applied when later the farm is extended with new servers or web app zones. But honestly, seeing the SPWebConfigModification code I do not have a clue to make that happen. Another issue to watch out for is that SharePoint administrates in its config database all commanded web.config modifications, and reapplies them all upon each ApplyWebConfigModifications. I solved that by administrating in SharePoint’s PropertyBag the name of the ExceptionHandler, and then after each invocation of ApplyWebConfigModifications invoke the correct positioning of this HttpModule.

Code extracts:

1. Detect whether an ExceptionHandler HttpModule is provisioned

2. Set required ordering of childNode in each web.config

And ultimately, this gives me the desired system/debug-enabling functionality in the deployed application:

Oops, apparently I've forgotten to enable an implementation for the IExpensesHandling interface...

I've earned my first SAP SDN forum points for providing a Solution..

The integration and interoperability of SAP and .NET environments is a subject which receives attention both from the SAP as from the .NET developer's community. It's for example interesting to see that both the Microsoft (MSDN) as SAP Developer Network (SDN) have a forum dedicated to this subject. Since the subject is all about integration, I as a .NET developer also like to keep up with the SAP SDN forum. And even contribute my bit when I'm able to help out someone in the SAP/.NET community who is struggling with questions or problems. It's gets even beter when your response is appreciated. So I've earned as a .NET developer my very first 'Forum Points' on the SAP NetWeaver .NET Technologies forum; on answering the question Calling a function module using C#. Especially motivating when you consider that this was my first contribution to this forum; I hope that more will follow, and also to learn myself from solutions of others on this forum.

Difficult upto impossible to derive the typename of error-causing webparts in migrated site

I migrated a SharePoint sitecollection into my local SharePoint development environment. As source material I had a backup of the sitecollection’s content database, and a couple of assemblies to go into the local webapplication bin. Missing is the web.config, and also the knowledge whether or not there is a dependency on other assemblies: be it within the local bin or within the GAC. I restored the database backup within my local SQL installation, created a new SharePoint webapplication, and attached the restored db to it. This made the site locally available and browsable.

Runtime it then appeared that I am indeed still missing some of the site resources. Browsing of several pages results in the infamous message: “Web Part Error: A Web Part or Web Form Control on this Page cannot be displayed or imported. The type could not be found or it is not registered as safe.”

The trick is now to detect the type of the error-causing webpart(s). Then I can verify whether or not I have them in the assemblies, and if so add an SafeControl entry for it in the web.config. I tried to have a look at the inner content of the webpage by opening it in SharePoint designer. This also gives the error message. Then I tried to find out the typenames by putting the page in webpart maintenance mode. If you have the appropriate rights (given my local environment, I naturally do), you can easily enter this mode by attaching ‘?Contents=1’ to the page-url. This gave the strange effect that 3 out of 5 webparts are displayed as ‘ErrorWebPart’.

This looked to me a display issue within webpart maintenance. Therefore I went explicitly self at a deeper level by inspecting the webpart info via the SharePoint Object API. However, even then the error-causing webparts have runtime the type ‘Microsoft.SharePoint.WebPartPages.ErrorWebPart’:

Surfing around I found out that this is a known SharePoint behavior. Any webpart that has gone bad, appears runtime as ‘ErrorWebPart’ instead.

So I decided to take it to yet another deeper level, and get into directly at the SQL content db. I queried the WebParts table for retrieving information of the webparts on the specific page:

This gives the 5 entries for the contained webparts, but sadly not a readable description of their types. The displayed tp_WebPartTypeId is not a cross-reference to an entry in another table. This guid is actually a MD5 calculated hash of a string composed of <the fullname of webpart’s assembly> + ‘|’ + <full classname of webpart>. MD5 hashing is irreversible, so it is not possible to convert these guids back into a human-readable format. Apparently, SharePoint itself builds up runtime an in-memory administration of the calculated hash of all the allowed webparts in a sitecollection. When requesting a webpart page, SharePoint compares it’s tp_WebPartTypeId values against this administration to derive the concrete webpart type. In case a WebPartTypeId is not found, SharePoint reports this via the errormessage “Web Part Error: A Web Part or Web Form Control on this Page cannot be displayed or imported. The type could not be found or it is not registered as safe.” Because of the irreversibility of the MD5 algorithm, even SharePoint code itself cannot derive the typename of these error-causing webparts.

To my humble opinion, this really sucks! How are you ever able to determine the typename of an error-causing webpart in this approach? And for a best practice: document whatever non-SharePoint out-of-the-box webpart is used in your site, and document for each webpart the fullname (including version number!) of its assembly.