SPWebConfigModification playing tricks in farm

At customer premisses we apply a.o. the following deployment guidelines:

1. In case of Farm-based deployment, then deploy to virtual bin (WebApplication), unless...
2. Apply SharePoint support for all required web.config modifications; instead of manual action (which is error prone in the farm).

Both guidelines are valid from SharePoint operations AND development perspective, to ensure a consistent and controlled SharePoint farm situation.

The SharePoint platform provides multiple deployment functionalities to comply with the 2nd guideline. Via the SharePoint solution manifest, you can specify modifications that typically have effect on the runtime operation of an assembly: SafeControls, CodeAccessSecurity. And via the SPWebConfigModification class you can modify in a controlled manner the web.config for other changes; e.g. for custom application settings, navigation providers. As from SharePoint 2007, I'm a big fan of applying SPWebConfigModification - it gives you as developer full control to have the needed web.config modifications executed upon application deployment and/or provision time. And to make it even better: the same set of changes are applied to all the individual web.config files of the SharePoint webapplication in the total farm; accross zones and accross servers. Even when at a later time another server is added to the farm, the same set of web.config modifications are applied (rather, repeated) also on that new server.

However, the SPWebConfigModification functionality does have it's peculiarities. Lately we encountered one I was not yet aware of, and which caused us both some headaches as well as cost some elapsed time.
The situation is as follows: SharePoint assembly that uses EntLib 5.0 for accessing an external SQL database. Comform the guidelines, this custom assembly is deployed to the virtual bin; and it is therefore required to set custom CAS-policies for this assembly. One of the required permission is the SqlClientPermission. That permissions is not present in the default WSS_Minimal trustlevel, but it is within WSS_Medium. To have the total set of custom CAS-policies based on the Medium trustlevel, a 2-steps approach is applied:

1. First change the trustlevel in the web.config from WSS_Minimal to WSS_Medium; a task performed via SPWebConfigModification class
2. Next, deploy the SharePoint solution with in its manifest the CodeAccessSecurity element for the assembly. The resulting custom CAS-policy file is now based on the Medium trust level, thus inheriting a.o. the SqlClientPermission SecurityClass setting. The SharePoint solution framework takes care of the required modifications in the web.config: link to the generated custom policy file, and set the trustlevel to WSS_Custom.

The setup worked perfectly, both local as in the shared test-farm.

That is, initially. From time to time our application appeared broken. Root cause analysis exhibited that in those situations our web.config was modified; but without any deployment activity on our own webapplication (???) However, another SharePoint webapplication in the same farm had been redeployed. Each time that application was redeployed, our web.config was modified; and the trustlevel reset from WSS_Custom to WSS_Medium.

It appears that this is standard behaviour of the SPWebConfigModification class: each time that it is requested to apply administrated SPWebConfigModifications entries in the context of a single SPWebApplication, it effectively re-applies the administrated SPWebConfigModifications of ALL the SharePoint web-applications in the farm. At minimal, the result of this is that all the web.config files in the farm are touched; and have their timestamp updated. But in our case, the administrated SPWebConfigModification for setting the trustlevel to WSS_Medium was reapplied; which broke our application!

So I learned 2 things here:

1. SPWebConfigModification is not a 'decent' SharePoint citizen; I regard it as ultimately incorrect that the intented application of SPWebConfigModification administrated entries on 1 SharePoint web-application, also effects all other SharePoint web-applications web.configs.
2. That as result of this behaviour the considered 2-steps approach for setting the correct medium level of a.o. SQLClientPermission cannot be maintained; due the inherent risk that the administrated SPWebConfigModification can be re-applied at any moment; out of the control and knowledge of the administrators of our webapplication. So I modified this to set the CAS-policies in a single step; directly steered via the Solution manifest file.
   
   
   Thus instead of the below CodeAccessSecurity specification, which is relative and relies on the presence of a.o. SqlClientPermission SecurityClass in the base trust-level (WSS_Medium):
   

   
   <PermissionSet class="NamedPermissionSet" version="1">
   <IPermission class="SecurityPermission" version="1" Flags="Execution" />
   <IPermission class="AspNetHostingPermission" version="1" Level="Medium" />
   <IPermission class="SqlClientPermissionversion" version="1" Unrestricted="true" />
   <IPermission class="SqlClientPermission" version="1" Unrestricted="true" />
   <IPermission class="Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, Version=14.0.0.0, Culture=neutral,
   PublicKeyToken=71e9bce111e9429c" version="1" ObjectModel="True" />
   <IPermission class="Microsoft.Office.SecureStoreService.Server.Security.SecureStorePermission, Microsoft.Office.SecureStoreService.Server.Security,
   Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" Unrestricted="true" />
   <IPermission class="System.Security.Permissions.ReflectionPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1"
   Unrestricted="true" />
   <IPermission class="System.Diagnostics.EventLogPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1"
   Unrestricted="true" />
   <IPermission class="WebPartPermission" version="1" Connections="True" />
   </PermissionSet>
   
   
   
   specify all the required permissions explicit:
   

   
   <PermissionSet class="NamedPermissionSet" version="1">
   <IPermission class="Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" ObjectModel="true" Unrestricted="true" />
   <IPermission class="SecurityPermission" version="1" Flags="Execution" Unrestricted="true" />
   <IPermission class="System.Web.AspNetHostingPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Level="Medium"/>
   <IPermission class="System.Data.SqlClient.SqlClientPermission, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Unrestricted="true"/>
   <IPermission class="Microsoft.Office.SecureStoreService.Server.Security.SecureStorePermission, Microsoft.Office.SecureStoreService.Server.Security, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" Unrestricted="true" />
   <IPermission class="System.Security.Permissions.ReflectionPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Unrestricted="true" />
   <IPermission class="System.Diagnostics.EventLogPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Unrestricted="true" />
   <IPermission class="Microsoft.SharePoint.Security.WebPartPermission, Microsoft.SharePoint.Security, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" Connections="True"/>
   </PermissionSet>
   
   
   

Utilizing Maps services for runtime determining nearest relations

In a former SharePoint project – an external facing public website -, a required functionality was to display the nearest X insurance advisors given the submitted postal code of user. We weighed several alternatives to realize this:

1. Re-use the distance-calculation algorithm used by the predecessor of the website (note: the essence of the project was a technical migration from a non-strategic proprietary WCM platform to the enterprise architecture target platform: SharePoint)
2. Utilize service of postal company: a data repository of all postal code tuple combinations, with the distance between them administrated.
3. Utilize Geo-coordinates to runtime calculate the distance between 2 locations, via the Haversine formula.

The first would have the disadvantage to still depend on the non-strategic platform and supplier. Also it had a rather complex programming API, without documentation and support.

The second would require us to host an external database besides the SharePoint content database in which to administrate the millions of postal code / distance records. This database would have to be updated each year with the management update from the postal company. Also, security constraints would not allow to directly access from the SharePoint WFE in the DMZ, the external database hosted in the internal network. The access should be regulated from DMZ to inner network via a secure webservice interface; which of course implies extra work and additional hosting costs.

So, the third alternative appeared the best fitted. Issue here is how to translate the postal code (submitted by the website user) into geo-coordinates (required for the Haversine formula). Luckily, both GoogleMaps and BingMaps provide services for this. Rough outline of the applied approach:

Preparation
1. Store in a SharePoint List the relations (less than 5000) data, including their geo-coordinates

Runtime
2. Invoke Maps service to derive the geo-coordinates (Latitude + Longitude) given the postal code submitted by user
3. Calculate the distance to each applicable relation via the Haversine formula
4. Ascending sort on calculated distance
5. Display the X nearest advisors

An hosting constraint for step 2 is that it is not allowed to directly invoke from the SharePoint farm, external webservices. To circumvent that restriction we decided to call the Maps service in the security context of the browser, via jQuery/JSON. The returned geo-coordinated are next via a postback with arguments transmitted to the server side; where then the distance calculations are done.

Architecture sketch:

Automatically publishing multiple files fails with error '0x81020015'

In my previous post I wrote about the well-known artefact of sandbox solutions that provisioned files are by default still checked-out. This can be fixed via a FeatureReceiver that checks-in all files and approves the ones that require moderation, so that all provisioned SharePoint artefacts are direct available for the end user.

Initially this worked like a charm, and the problem seemed resolved. However when our project progressed and the number of provisioned files increased, we structurally encountered the following error upon feature activation:

Microsoft.SharePoint.SPException occurred
  Message=The file _catalogs/masterpage/ApplicationX/ApplXPageLayoutXX.aspx
    is modified by domain\WvStrien on May 11 2011 17:06:12 +0200.
  Source=Microsoft.SharePoint
  ErrorCode=-2130575305
  NativeErrorMessage=FAILED hr detected (hr = 0x81020015)

In the root cause analysis of the problem I discovered that the problem only manifests itself when more than 1 file requires approval. This also explained why initially we did not encounter the problem: we started out with only a single PageLayout. With this insight I did an internet search for '0x81020015'. Although no direct related pointers to our situation, this led me in the direction of the problem cause. Apparently it is a race condition within SharePoint internally when first checking in a file and immediately approving it in the same execution context. The internal SharePoint administration is typically not ready yet to handle the second state change on the file.

If this is the cause, then the solution is to break up the execution context: first do the checkin(), and in another execution context the approve() invocation. Normally you would do this by delegating it to a background thread. However, this is not possible in a Sandbox context: the SharePoint ObjectModel is only supported on the main thread of the SPUserWorker process. The best next and pragmatic approach is then to break the execution flow by introducing a delay between the invocation of the 2 SPFile methods:

private static void PublishFile(SPFile checkedOutFile, bool approvalNeeded)
{
  checkedOutFile.CheckIn(String.Empty, SPCheckinType.MajorCheckIn);
  if (approvalNeeded)
  {
    int nrOfWaits = 0;
    while (!checkedOutFile.Exists && nrOfWaits++ < 5)Thread.Sleep(1000);
    if (checkedOutFile.Exists)
     checkedOutFile.Web.GetFile(checkedOutFile.UniqueId).Approve(COMMENT);
  }
}

Error “Unable to load assembly group” upon redeploying a sandbox solution with a feature receiver

IT operations set up a fresh new SharePoint webapplication for my current project on the development farm, and provisioned it with a root site collection based on the publishing template. Next I took over the deployment control by following the sandbox route; in this preliminary phase merely to provision the first version of the branding: master page, CSS files, images and page layouts. A known artifact of sandbox solutions is that the provisioned files are by default still checked-out in the content database by the person who activated the solution, and thus not visible for anyone else. [It is a mysterie to me why Microsoft implemented it such; and moreover why they didn’t delivered us a setting to configure this provisioning behavior to automatic checkin and publish/activate the provisioned SharePoint artifacts.] Instead of manually via the browser or SharePoint Designer having to lookup in the site collection all the provisioned SharePoint artifacts to check them in and publish them, I more favor the approach to have this done automatically in the context of the feature activation. This can be easily accomplished via a feature receiver (Waldek Mastykarz has a good start/example for this in his blogpost Automatically publishing files provisioned with Sandboxed Solutions). So I added a feature receiver to the provisioning feature in the sandboxed solution, and in the FeatureActivated event let all the provisioned SharePoint artifacts be located in the content database and automatically checked in + published if applicable. Tested it locally, works like a charm. Deactivated and deleted the previous uploaded version of the sandboxed solution, and uploaded the new version with the feature receiver inside. However, upon activating this sandbox solution version, I consistently got the following error:

Error occurred in deployment step 'Add Solution': Unable to load assembly group. The user assembly group provider threw an exception while trying to provide user assemblies for the specified assembly group.

And in the ULS logging the following detail info:

Unable to load assembly group. The user assembly group provider threw an exception
Assembly Group Id: GroupId = "7C00665459714EEC9BDA8727AD711EC7-FES7hxwCdYldvtgExF/L+b259Rh3T1thWDeijNWTrSU="" - Inner Exception: System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
at Microsoft.SharePoint.SPListItemCollection.get_Item(Int32 iIndex)
at Microsoft.SharePoint.Administration.SPSolutionLanguagePack.
GetSolutionInfoFromGallery(Guid siteId, Guid solutionId, String solutionHash, String& fileName, String& hash, Byte[]& fileBytes)
at Microsoft.SharePoint.UserCode.SPUserCodeLightweightSolutionAssemblyGroupProvider.
GetAssembliesInGroup(Guid siteId, String assemblyGroupId)
at Microsoft.SharePoint.UserCode.SPUserCodeAssemblyCacheManager.
EnsureUserCodeAssemblyGroupIsCached(Guid siteId, SPUserCodeAssemblyGroupId userCodeAssemblyGroupId)

An internet search did not give many nor useful hits; and neither could the SharePoint development gurus in my neighborhood help me. So it left me rather puzzled at first how to analyze and solve this. A good clue however was that a sandboxed solution from another SharePoint (web)application with also a feature receiver inside, had the same problem trying to activate it in our site collection, while it successfully activated in the own site collection. So it must be something at SharePoint infra level, not caused by code development. With this insight I went back to my operations companion, and provided him with a sandboxed solution to monitor what events occurred at system level upon trying to activate the solution. This quickly let to a conclusive result: somehow the permission state of the User Code service account on the C:\ProgramData\Microsoft\SharePoint\UCCache subfolder got compromized, which disallowed it to re-copy the new version of the sandboxed solution with code inside to the UCCache subfolder. The problem was not even resolved by restarting the User Code service; instead it was needed to manually reset the User Code service account, so that upon the next invocation the ACL on the UCCache subfolder was again built up.

Our operations guy was very thoughful to make a note of this behavior in the internal SharePoint operations Knowledge Base, and document it in a work instruction.

Duet Enterprise lessons learned for custom application scenario

The Duet Enterprise proposition is multi-faceted. It is a product with ready-to-use functionalities. It is a composition environment for constructing your own solution by composing delivered functional and technical building blocks, e.g. SharePoint Site templates for SAP business objects, SAP reports scheduling and publication into SharePoint, expose SAP workflow decision points as SharePoint tasks. And it is an Integration Foundation for building your own custom application scenario.

In case of that last context, a Duet Enterprise project is a software development project. However, it is not just like any other development project. The application of the Duet Enterprise Integration Foundation in custom SAP / SharePoint application scenarios imposes specific prerequisites to be successful. From experiences with applying Duet Enterprise in custom application scenario, we have learned several lessons.

Starting with Duet Enterprise

Before you are ready to apply Duet Enterprise in a custom application scenario, the project team needs to invest time to get familiar with the product. Get acquainted with the Duet Enterprise concepts, architecture, capabilities, configuration, development approach, constraints and its restrictions. It proofs a steep learning curve to become familiarized with Duet Enterprise and the diverse aspects. This learning experience is made more difficult by the lack yet of concise study material and available reference cases in the open to learn from.

Also you need to install Duet Enterprise in your combined SAP and Microsoft IT landscape. It’s common that this task is performed by the IT operations in your company. Be aware that installation and configuration of Duet Enterprise is more than a simple ‘next-next-OK’ click exercise. Especially on the SAP side, the installation of the Duet Enterprise SAP Add-on consists of configuration steps in diverse parts of the SAP landscape (e.g. SE80, IMG, SOA Manager, …). The total elapse time for installing and configuring Duet Enterprise can easily extend to 5 or more workdays. For the major part the SAP and SharePoint installation and configuration can be done isolated and in parallel. However, there are also multiple moments where it is essential to work together, and exchange system information. From SAP imported into the SharePoint farm, and later on SharePoint security settings required at the SAP NetWeaver / Service Consumption Layer (SCL) side.

Starting within a custom SAP / Microsoft application scenario

This is no different from any SAP / MS integration scenario. Don’t start with the tool, start with the business question! Make sure you understand which problem(s) you are trying to solve. Invest proper time for performing the business and information analysis. Agree with the customer what kind of user experience is asked for. With these demand aspects sufficiently clear, continue with deriving the integration and the software architecture design. In this process both SAP and Microsoft solution architects should be involved. They bring in the insights, knowledge of best and bad practices from the experiences on their own turf. When sketching the integration architecture, continually evaluate whether and where Duet Enterprise can add value, given the Foundation capabilities and the imposed constraints. Don’t make the application of Duet Enterprise your goal; but determine for the application context if and where Duet Enterprise adds value.

Functional system architecture

Input here are the business process, the requested functionalities, the end-user audience, and the workplace characteristics. Decompose the system functionality in use cases. Often not all of the system use cases have the same set of actors. The functional and process architect(s) must carefully decide for which of the use cases there is a business case to expose outside of the SAP boundaries, and for which not. Example of the last is an approval task by a manager or the HR department. They are SAP power users, spending a large part of the day within the SAP GUI. The GUI has no secrets for them, it’s their familiar workplace. The larger workforce in the company is instead a non-casual user of the SAP environment. They are not used to the SAP GUI, and loose valuable time finding out how to operate within the GUI. The financial business case here is reducing the amount of required SAP licenses, and reducing the time spend by this larger set of employees due the unfamiliarity with the SAP UI. A soft business case is that the employee satisfaction is improved by integrating the SAP related work within their common and familiar workplace.

Starting with Development approach

Again, this is no different from other enterprise application integration scenarios. A proven best practice is to start with establishing and defining the contracts via which the application counterparts will communicate. Initially these contracts are defined in abstract format, focusing on the identification and separation of system responsibilities. Microsoft architects recognize this as applying the ‘Contract-First’ approach; SAP architects are more familiar with the term ‘Outside-In’. Do a design time validation on paper or whiteboard that all of the identified functional use cases can be realized by the set of identified interfaces. Augment and tune the interfaces until this design check leaves no more gaps.

Next, model the identified abstract service interfaces in the Duet Enterprise SAP Add-on. The tool for this is the ES Builder, a Java-based modeling environment delivered either as part of SAP PI, or of SAP NetWeaver Composite Environment (CE). In this development step the technology-agnostic service interfaces must be translated into specific SAP service-technology format: data structures, message types, interface operations. Here you also augment the abstract service interfaces with the specific Duet Enterprise requisites: Correlation ID and Business Object Instance Key . These 2 specific parameters are required to enable the Duet Enterprise system monitoring and routing capabilities. Also add here SOAP standards-based fault handling to each service operation. The ES Builder directly supports this via the SAP ESA SOAPFaultException datatype. The SharePoint consumer of the Duet Enterprise service interface can handle the received SOAP Faults in a standard manner through WCF, transparent to any SAP specifics in the fault notification. When all the identified service interfaces are modeled in ES Builder, generate in the ESB the WSDL service description. This W3* standards-based and platform independent service description is utilized and needed on both the SAP service provider and the SharePoint service consumer sides for building the integrated application.

Building the SAP / SharePoint integrated application

With the WSDL service description available, the SAP and SharePoint development teams can each go ahead with the realization of their respective application responsibilities.

At the SAP side the identified service interfaces must be implemented by concrete service realizations. In a Duet Enterprise landscape the term Service Provider refers to the SCL system, and SAP Backend refers to the system that hosts the Business Functionality. SAP Duet Enterprise service implementations are thus positioned in the SCL. The SCL services connect to the SAP Backend to consume the business functionalities and data. Since this SCL-SAP Backend connection is strictly within the SAP environment, it is possible to rely on SAP proprietary connection technology. In fact, between the SCL and the SAP backend it does not add anything additional to connect via web services (BAPI or Enterprise Services). Using web services to connect to the Backend brings some additional performance overhead so RFC-based connection is faster. The development work inside the SCL further consists of mapping the external faced service signatures to the internal SAP processing and data structures. This involves the technical mapping from the W3* standards-based integration handling to the SAP internal invocation model and data structures. And potentially at business process level the composition of multiple SAP actions in the scope of a single Duet Enterprise service operation.

The required work at the SharePoint Duet Enterprise consumption side is largely dependent on the operation and data signature of the provided SCL services. At SharePoint side, Duet Enterprise builds on Business Connectivity Services. Implication of this is that the SCL services must exhibit a CRUD+Q operation signature. If not, consumption via BCS is not even possible. If BCS-based consumption of the external SCL services is possible, another aspect is the data structure. SharePoint 2010 out-of-the-box delivers the concept of the External List UI to interoperate with external data. A practical limitation of the External List UI metaphor is that it requires a flat data structure. If not flat, it is not possible to display the external data in a row-based manner. Non-flattened / hierarchical data can still be consumed via BCS in SharePoint, however not via the External List concept. The project will have to build a custom UI to interoperate with hierarchical SAP data structures. The custom UI must connect to the BCS Object Model to interoperate with the external SCL services. Current drawback is that the BCS API only supports a weakly-typed programming model. You have to program per exchanged hierarchical SAP data structure an ORM-based mapping from strong-typed data representation at the SharePoint front-end to the BCS weakly-typed representation.

Debugging Duet Enterprise connection problems

Here is where the application of Duet Enterprise clearly adds additional value. If you obey to the Duet Enterprise constraints; that is if you have per service operation included the Correlation ID parameter; the Duet Enterprise landscape supports the problem analysis via an integral audit trail through the combined SharePoint + SAP systems chain. You look up the Correlation ID value in the SharePoint ULS logs for where the BCS-based connection starts, and follow it through the SCL layer via the SAP system monitoring. Problems can be of infrastructure nature, e.g. SSL certificate at SAP and SharePoint not in sync; SharePoint BCS-SCL service connection level as result of a wrongly chosen SAP proxy endpoint; SAP authentication and/or authorization due incomplete Duet Enterprise user mapping. The Correlation ID based audit trails proofs a big help in tracing where in the integrated SharePoint / SCL / SAP Backend landscape the actual problem cause manifests itself.

Duet Enterprise development work plus applicability

At the global level, following development activity is involved in building a custom Duet Enterprise scenario.

SAP side

Plain and simple: Provide SCL-ready services, that are SharePoint BCS consumable.
There is no other option here, this work is required to deliver SAP data + processing services that are enabled for SCL-publishing. At minimum this involves ABAP custom development for mapping the W3*-standards based SCL data representation onto the native SAP data representation, and vice versa; and invoking from within the SCL runtime context the inner SAP processing at either RFC, BAPI web services or SAP Enterprise Services level.

SharePoint 2010 side

Here the amount of development work depends, namely on the signature of the SCL-published services. Ideally the services provide a CRUD+Q signature (required) + a flattened data structure (optional). If both aspects are present in the SCL-services, these can be directly consumed and displayed in SharePoint External List. Then there is no development work required at SharePoint side, it is all configuration done in SharePoint Designer. You get the maximum added Duet Enterprise value in case of flattened data: SAP/SharePoint connection, authorization, system monitoring; and out-of-the-box presentation of the SAP data.

In case of non-flattened, complex/hierarchical data structures, the External List UI metaphor is not viable. Duet Enterprise can still have its value, on the connection, authorization + system monitor aspects (plumping). What you loose here is the out-of-the-box UI presentation. You will have to build a custom UI instead, and program directly against the BCS Object Model. Thus more work, but Duet Enterprise still has standards-based value and role for SAP/SharePoint interoperability.

Connection requirements / BCS implication for Duet Enterprise

The Duet Enterprise SharePoint Add-on builds and relies upon BCS. Consequence is that BCS prerequisites are imposed on the SAP / SharePoint service connection, provided by the SCL-publishing and BCS-consumption. The SCL provided SAP-services must expose a CRUD+Q interface, with at minimal a ‘ReadList’ (= Query) and ‘ReadItem’ (= Read) type of operation. If the SAP processing behavior cannot be mapped on the CRUD+Q signature, you cannot reasonable apply Duet Enterprise. Point. The SAP .NET Connector 3.0 may be a alternative for such connections.

Schematic overview of the Duet Enterprise applicability

Positioning of .NET Connector 3.0 versus Duet Enterprise

In a timeframe of a couple of weeks, SAP independently released 2 products for .NET interoperability: first [December 2010] the next version of the .NET Connector – NCo 3.0; followed a few weeks [Februari 2011] later by the public launch of Duet Enterprise – a combined effort of SAP and Microsoft. These near overlapping release moments may raise questions and uncertainty on the positioning of the 2 .NET interoperability products. Are they competing? Are the successive release moments a symptom of independent products groups within SAP, and will market acceptance determine which one will ‘win’? And what about the SAP ES Explorer? Earlier, SAP spokesman declared the .NET connector outdated in favor of the ES Explorer.

In my thoughts and analysis, the story is differently. NCo3.0 and ES Explorer on one side, and Duet Enterprise all have a distinct positioning. The 3 products are neither competing nor exclusive for .NET interoperability. Each serves a specific and dedicated purpose.

NCo3.0 and ES Explorer

Both NCo3.0 and ES Explorer are in essence .NET Interoperability technologies. Neither of them is positioned as a product. You can not purchase or license it, but instead can download them from SAP Marketplace if you have a valid S-User ID that acknowledges you as a SAP developer. The release of .NET Connector 3.0 passed rather silently, with only an announcement on SDN. It has not received attention from any of the influencing IT business magazines, nor IT Research Analysts. There was a Ramp-Up with selected beta-testers, but with minimal noise ahead and attention during.

• NCO 3.0 is intended as a general purpose technology tool for low level integration plumping. Basically it enables bi-directional interoperability between .NET custom code and SAP RFCs and BAPI Function Modules. Nothing more, nothing less.
• The role of SAP ES Explorer is in essence the same. In the earlier SAP statement, Rima Rudnik-Sirich characterized it as 'It succeeds SAP .NET Connector 2.0 for .NET'. The difference is within the interoperability manner. SAP ES Explorer works at the level of SAP Enterprise Services, W3*-compliant. You apply it as Visual Studio Add-In, to search through all the SAP Enterprise Services available in your landscape; standards ones from SAP, from third parties deployed in your landscape, and your customer-build Enterprise Services. Runtime invocation of the selected Enterprise Services occurs via a generated WCF service proxy.

Duet Enterprise

Duet Enterprise is a whole other story. Both SAP and Microsoft position it as a product that will directly provide business value. The Duet Enterprise release was done at large: a Virtual Launch Event, press releases and articles within the important IT business magazines, analysis reports by Forrester and CITO Research. Application of Duet Enterprise requires a license, the product can be purchased via SAP and Microsoft. (Note: at moment of writing there is no information disclosed on pricing and licensing model). Before reaching General Availability, Duet Enterprise has been evaluated in a combined Ramp-Up / Rapid Deployment Program hosted by SAP and Microsoft together. Participating in the RDP was given substantial noise ahead, and intensive attention during the program course self. (Note: TopForce participated together with a large Dutch insurance company in this RDP).

The role of Duet Enterprise is twofold. It comes out of the box with direct usable functionalities, and capabilities to compose customer-specific solutions. Its second face is what distinguished Duet Enterprise from the previous Duet versions: integration Foundation to build integration + interoperability that is specific for your situation. Also, SAP and Microsoft believe and want ISVs (the ecosystem) to deliver add-ons on the Duet Enterprise products, e.g. for specific vertical markets. To stimulate this, SAP and Microsoft also launched a partner program: Unite Partner Connection Program.

The Duet Enterprise Foundation is also interoperability plumping, but of higher level as NCo 3.0 and ES Explorer. SAP / .NET interoperability occurs via standard W3* services; there is out-of-the-box support for SSO, Authorization, landscape monitoring. All aspects that earlier you typically had to handcraft yourself, and thus also maintain.

Duet Enterprise is bounded to usage from within a SharePoint 2010 context, and via SharePoint 2010 as intermediary layer in Office 2010 clients. Duet Enterprise has no role in other .NET contexts, e.g. SharePoint 2007 or 2003, Silverlight, WinForms or WPF apps, WCF handling, BizTalk.

Implications

The added value of Duet Enterprise [product, foundation] wrt NCo 3.0 and SAP Explorer [basic interoperability technologies] is that it raises the level of SAP - .NET interoperability. In addition to the bi-directional SAP - .NET runtime communication, it also provides interoperability concepts as SSO, Authorization, System Monitoring. And on top, it comes with direct usable functionalities, and capabilities to compose solutions [building blocks].

The limitation is that it only works from a SharePoint 2010 context (and via SharePoint in Office 2010 clients). If you need .NET interoperability from different .NET context, you have to resort to another approach. SAP provides for this both the NCo 3.0 and SAP ES Explorer; which one is usable is dependent on the level of SAP back-end consumption. If SAP Enterprise Services are available; apply SAP ES Explorer; for RFCs and FMs you can use NCo 3.0.

Note: there are also several technologies and products delivered outside SAP to enable .NET interoperability. For instance Microsoft provides the BizTalk WCF LOB Adapter SDK, BizTalk itself; third parties provide product like Sitrion, Ometa, ERP Connect. For the scope of this article they are however not considered.