Duet Enterprise lessons learned for custom application scenario

The Duet Enterprise proposition is multi-faceted. It is a product with ready-to-use functionalities. It is a composition environment for constructing your own solution by composing delivered functional and technical building blocks, e.g. SharePoint Site templates for SAP business objects, SAP reports scheduling and publication into SharePoint, expose SAP workflow decision points as SharePoint tasks. And it is an Integration Foundation for building your own custom application scenario.

In case of that last context, a Duet Enterprise project is a software development project. However, it is not just like any other development project. The application of the Duet Enterprise Integration Foundation in custom SAP / SharePoint application scenarios imposes specific prerequisites to be successful. From experiences with applying Duet Enterprise in custom application scenario, we have learned several lessons.

Starting with Duet Enterprise

Before you are ready to apply Duet Enterprise in a custom application scenario, the project team needs to invest time to get familiar with the product. Get acquainted with the Duet Enterprise concepts, architecture, capabilities, configuration, development approach, constraints and its restrictions. It proofs a steep learning curve to become familiarized with Duet Enterprise and the diverse aspects. This learning experience is made more difficult by the lack yet of concise study material and available reference cases in the open to learn from.

Also you need to install Duet Enterprise in your combined SAP and Microsoft IT landscape. It’s common that this task is performed by the IT operations in your company. Be aware that installation and configuration of Duet Enterprise is more than a simple ‘next-next-OK’ click exercise. Especially on the SAP side, the installation of the Duet Enterprise SAP Add-on consists of configuration steps in diverse parts of the SAP landscape (e.g. SE80, IMG, SOA Manager, …). The total elapse time for installing and configuring Duet Enterprise can easily extend to 5 or more workdays. For the major part the SAP and SharePoint installation and configuration can be done isolated and in parallel. However, there are also multiple moments where it is essential to work together, and exchange system information. From SAP imported into the SharePoint farm, and later on SharePoint security settings required at the SAP NetWeaver / Service Consumption Layer (SCL) side.

Starting within a custom SAP / Microsoft application scenario

This is no different from any SAP / MS integration scenario. Don’t start with the tool, start with the business question! Make sure you understand which problem(s) you are trying to solve. Invest proper time for performing the business and information analysis. Agree with the customer what kind of user experience is asked for. With these demand aspects sufficiently clear, continue with deriving the integration and the software architecture design. In this process both SAP and Microsoft solution architects should be involved. They bring in the insights, knowledge of best and bad practices from the experiences on their own turf. When sketching the integration architecture, continually evaluate whether and where Duet Enterprise can add value, given the Foundation capabilities and the imposed constraints. Don’t make the application of Duet Enterprise your goal; but determine for the application context if and where Duet Enterprise adds value.

Functional system architecture

Input here are the business process, the requested functionalities, the end-user audience, and the workplace characteristics. Decompose the system functionality in use cases. Often not all of the system use cases have the same set of actors. The functional and process architect(s) must carefully decide for which of the use cases there is a business case to expose outside of the SAP boundaries, and for which not. Example of the last is an approval task by a manager or the HR department. They are SAP power users, spending a large part of the day within the SAP GUI. The GUI has no secrets for them, it’s their familiar workplace. The larger workforce in the company is instead a non-casual user of the SAP environment. They are not used to the SAP GUI, and loose valuable time finding out how to operate within the GUI. The financial business case here is reducing the amount of required SAP licenses, and reducing the time spend by this larger set of employees due the unfamiliarity with the SAP UI. A soft business case is that the employee satisfaction is improved by integrating the SAP related work within their common and familiar workplace.

Starting with Development approach

Again, this is no different from other enterprise application integration scenarios. A proven best practice is to start with establishing and defining the contracts via which the application counterparts will communicate. Initially these contracts are defined in abstract format, focusing on the identification and separation of system responsibilities. Microsoft architects recognize this as applying the ‘Contract-First’ approach; SAP architects are more familiar with the term ‘Outside-In’. Do a design time validation on paper or whiteboard that all of the identified functional use cases can be realized by the set of identified interfaces. Augment and tune the interfaces until this design check leaves no more gaps.

Next, model the identified abstract service interfaces in the Duet Enterprise SAP Add-on. The tool for this is the ES Builder, a Java-based modeling environment delivered either as part of SAP PI, or of SAP NetWeaver Composite Environment (CE). In this development step the technology-agnostic service interfaces must be translated into specific SAP service-technology format: data structures, message types, interface operations. Here you also augment the abstract service interfaces with the specific Duet Enterprise requisites: Correlation ID and Business Object Instance Key . These 2 specific parameters are required to enable the Duet Enterprise system monitoring and routing capabilities. Also add here SOAP standards-based fault handling to each service operation. The ES Builder directly supports this via the SAP ESA SOAPFaultException datatype. The SharePoint consumer of the Duet Enterprise service interface can handle the received SOAP Faults in a standard manner through WCF, transparent to any SAP specifics in the fault notification. When all the identified service interfaces are modeled in ES Builder, generate in the ESB the WSDL service description. This W3* standards-based and platform independent service description is utilized and needed on both the SAP service provider and the SharePoint service consumer sides for building the integrated application.

Building the SAP / SharePoint integrated application

With the WSDL service description available, the SAP and SharePoint development teams can each go ahead with the realization of their respective application responsibilities.

At the SAP side the identified service interfaces must be implemented by concrete service realizations. In a Duet Enterprise landscape the term Service Provider refers to the SCL system, and SAP Backend refers to the system that hosts the Business Functionality. SAP Duet Enterprise service implementations are thus positioned in the SCL. The SCL services connect to the SAP Backend to consume the business functionalities and data. Since this SCL-SAP Backend connection is strictly within the SAP environment, it is possible to rely on SAP proprietary connection technology. In fact, between the SCL and the SAP backend it does not add anything additional to connect via web services (BAPI or Enterprise Services). Using web services to connect to the Backend brings some additional performance overhead so RFC-based connection is faster. The development work inside the SCL further consists of mapping the external faced service signatures to the internal SAP processing and data structures. This involves the technical mapping from the W3* standards-based integration handling to the SAP internal invocation model and data structures. And potentially at business process level the composition of multiple SAP actions in the scope of a single Duet Enterprise service operation.

The required work at the SharePoint Duet Enterprise consumption side is largely dependent on the operation and data signature of the provided SCL services. At SharePoint side, Duet Enterprise builds on Business Connectivity Services. Implication of this is that the SCL services must exhibit a CRUD+Q operation signature. If not, consumption via BCS is not even possible. If BCS-based consumption of the external SCL services is possible, another aspect is the data structure. SharePoint 2010 out-of-the-box delivers the concept of the External List UI to interoperate with external data. A practical limitation of the External List UI metaphor is that it requires a flat data structure. If not flat, it is not possible to display the external data in a row-based manner. Non-flattened / hierarchical data can still be consumed via BCS in SharePoint, however not via the External List concept. The project will have to build a custom UI to interoperate with hierarchical SAP data structures. The custom UI must connect to the BCS Object Model to interoperate with the external SCL services. Current drawback is that the BCS API only supports a weakly-typed programming model. You have to program per exchanged hierarchical SAP data structure an ORM-based mapping from strong-typed data representation at the SharePoint front-end to the BCS weakly-typed representation.

Debugging Duet Enterprise connection problems

Here is where the application of Duet Enterprise clearly adds additional value. If you obey to the Duet Enterprise constraints; that is if you have per service operation included the Correlation ID parameter; the Duet Enterprise landscape supports the problem analysis via an integral audit trail through the combined SharePoint + SAP systems chain. You look up the Correlation ID value in the SharePoint ULS logs for where the BCS-based connection starts, and follow it through the SCL layer via the SAP system monitoring. Problems can be of infrastructure nature, e.g. SSL certificate at SAP and SharePoint not in sync; SharePoint BCS-SCL service connection level as result of a wrongly chosen SAP proxy endpoint; SAP authentication and/or authorization due incomplete Duet Enterprise user mapping. The Correlation ID based audit trails proofs a big help in tracing where in the integrated SharePoint / SCL / SAP Backend landscape the actual problem cause manifests itself.