Tuesday, January 5, 2010

Automatic publish InfoPath forms via Feature-based ALM deployment

Fixed the Feature-based deployment of InfoPath forms to SharePoint site while treating them as first-class source artifacts in VS + TFS environment

I’m a supporter of applying Application Lifecycle Management (ALM) principles within application development projects. This also goes for SharePoint-based custom applications. Basically this means:
  1. designing and developing the SharePoint artifacts within a team-based source repository environment (aka, Team Foundation Server);
  2. applying version control to the individual application/building blocks;
  3. daily build;
  4. and full automatic and repeatable deployments
With InfoPath development, this gives some challenges. First of all, the InfoPath forms are setup and maintained via its own designer-tool, namely the InfoPath Designer client; and not directly from within the scope of Visual Studio. However, the most significant issue is the manner of deployment: the InfoPath client contains functionality to publish the InfoPath form to the destination environment, with SharePoint site as one of the available options. Given my ALM aspirations, I want to copy this InfoPath client deployment-experience to within the regular SharePoint deployments, via the Solution framework and SharePoint Features. In essence, I want to achieve the following:
  1. Automatically provision the required Information Architecture artifacts:
    • SiteColumns
    • Masterdata for LookupFields
    • ContentTypes
    • Forms Libraries, associated with the ContentTypes, for administration of the filled in InfoPath forms
  2. Automatically provision and activate the required InfoPath infrastructure
    • InfoPath forms
    • DataConnections
The first category, the Information Architecture artifacts, is well-known SharePoint provisioning. It can be done via a Site Definition, Features, or a combination. I favor the Feature-based approach, because it is modular, and allows me to turn it off and on.
The second category gave me more challenges. On internet search, I came across some posts that mentioned the SharePoint OOTB XSNFeatureReceiver that aids in here. However, it quickly appeared to me that it only takes care of one part, namely registrating the InfoPath form templates within the target SharePoint site. To successfully achieve this, XSNFeatureReceiver has constraints on the deployable InfoPath forms. And when even a single one of them does not comply to these constraints, XSNFeatureReceiver rather silently fails to correctly install the forms. Another missing part is for the DataConnections. The upload is not the issue here, being standard SharePoint Module functionality. But rather adjusting them to the specific target environment. In my ALM principles, I want to hold on to 1 single instance per DataConnection file, and not be forced to maintain versions for each of them per target environment (development, test, staging, production).
Luckily, Microsoft has not made XSNReceiverFeature sealed. So I decided to overload it to augment the standard deployment functionality with the above described aspects. Adjusting the DataConnections appeared rather simple, with thanks to this blog (Building InfoPath Form Services Solutions using Visual Studio 2008), including a link to demo provision code. I took this as basis to realize a generic approach, feeded by an XML configuration file delivered within the feature scope
The correct deployment of the InfoPath forms appeared to be more of a challenge. Eventually, I found out that the direct cause of the failed deployment was that I earlier on published the forms during my development and testing of the forms. As soon as you publish an InfoPath form, the InfoPath client administrates this within the form itself. And this appeared to result in the malfunctioning XSNFeatureReceiver based InfoPath forms registration. This can be mitiligated by correcting the InfoPath forms. An alternative is to modify the original InfoPath form artifacts. However, this would then be a corrective action to be repeated every time you’d published an InfoPath form just for an InfoPath designer (programmer) test. I therefore went for the alternative to do the correction as part of the Feature installation procedure. This way during the development phase the team can just go one and (re)publish the forms, without worrying about the effect on deployment later on.


Another thing to take into account when doing Feature-based deployment of InfoPath forms, is that the Forms must be contained at the Feature root-directory. I like to structure Feature contents within subfolders for different deployable types. ContentTypes, SiteColumns, DataConnections, and also InfoPath forms, each within their own subfolders within the feature. However, in case of InfoPath forms deployed via XSNFeatureReceiver, this is not working. Without a concrete error message, the registrating will then just not be performed. So keep your Forms at the Feature root-level.

Deployment steps, and thus functionality of InfoPathInfrastructure feature

Upon Feature installation event:

  1. Fix the forms files, for successful automatic publishing via XSNFeatureReceiver
    • Extract the forms; this on itself presented a challenge, since .NET does not provide standard cabinet (.cab) handling
    • Inspect the manifest.xsf file on the presence of publishUrl and trustLevel; remove any of them present
    • Repack the .xsn container / file

Within the Feature Activation:

  1. upload forms files to FormsServerTemplate library
  2. Registrate the forms
  3. upload data connections to DataConnections library
  4. provision site columns
  5. provision master data lists
  6. fix the lookup references
  7. provision content types; including reference to the uploaded forms
  8. provision contents/data libraries
  9. assign content types to the contents libraries
  10. fix retrieve data connections --> associate with the master data list in the deploy environment (iso your local / development source)
  11. fix submit data connection --> associate with the content data list in the deploy environment (iso your local / development source)

15 comments:

  1. Hello
    Nice post!

    I have a question here: You mention "ACTIVATE Infopath infrastructure" and "upload forms files to FORMSSERVERTEMPLATE library", which are bits of the FORMS SERVICES in Sharepoint ENTERPRISE Site Collection features. On the other hand, I am sure you imply no ALM over "Forms Services", since there is a ton of info on the net about automating that(such as this: http://www.sharepointdevwiki.com/display/public/Developing+a+InfoPath+Solution+using+a+Solution+Package ) that you would have noticed!
    That brings some confusion, isnt't it?! To be more presize - I am investigating automation in deployment plain(non-browser enabled) IP forms, and have difficulties using the XsnFixup.js from Microsoft. In fact, I investigated on the pragmatics of both(the "Forms Services"-involved and plain(contennt-type attached) IP publishing methods) and will be happy to get your point about these.

    ReplyDelete
  2. Hi,

    Yes, there are other sources of information on the web; and I've thankfully read and applied them. However, it was not sufficient for my ALM aspirations. I want to treat all the InfoPath related artifacts as first-class source artifacts. This includes the InfoPath forms, DataConnections, ContentTypes + SiteColumns, Forms Libraries. I aimed to include all of them within the safe context of Team Foundation Server + Visual Studio. Also I wanted to setup a generic approach, which I can reuse for arbitrair number and types of InfoPath infrastructure.
    Given this [source]repository point-of-view, I don't want it to constraint my manner of defining and editing the IP forms self. A significant charm of IP forms (above other kinds of webforms) is just the ease of setting them up in the rich InfoPath designer client. As I explained, doing the combination gave my direct deployment issues.

    Note that browser-enabled IP forms don't preclude the usage of them in a rich InfoPath client context. This also means that a deployment for Forms Service context, will also serve for non-browser context (and dual).

    I'm a bit puzzled by what you see as difference between FS and CT publishing methods. In my experience, the combination is very well possible. Even more, I favour that: http://williamvanstrien.blogspot.com/2010/01/administrate-data-from-infopath-form-in.html

    ReplyDelete
  3. Forgive me, I put that in badly! Here is what I expect you to agree with: when a user publishes a form to a content type or a library, he or she doesn't use the FORMS SERVICES in Sharepoint ENTERPRISE Site Collection features. What happens is the template gets altered to let its form instances know where in Sharepoint it is located and the ContentType alters its formtemplate location. And maybe some dataconnections stuff happens. But again, that does not involve any "FORMS SERVICES in Sharepoint ENTERPRISE Site Collection features".
    That is what I aim to Automate and that is what I wrongly meant by CT publishing method, since CTs are also used in the other scenario!

    If I understand you right, what I am trying to do can be done through Enterprise Features.With my little knowlege on licensing, I assume that no additional charge would be set on neither the company, nor the users, when I automate deployment(publishment) of non-browser-enabled forms. That is because the Enterprise SP Server license is just a logical addition to activate, without a license to pay for it. And when none of the actual partial functionalities are needed(I can develop only non-browser-enabled forms), only the admins would have to see more options in CentralAdministration and across the sites?! Am I right?!

    ReplyDelete
  4. It's possible to apply XSNFeatureReceiver to automate the 'functional user' experience (via the InfoPath client) of publishing InfoPath strict non-browser enabled forms.

    You can publish them to any document library on your site, possible but not necessarily the Forms Services provided FormsServerTemplates document library. You then provide the end users with a link to the template, and if they have InfoPath installed they can open the form template in their local IP client. Via data connections in the form template, the data is submitted to a document/forms library (or to a webservice, or emailed, ...).

    Instead of specifying a central forms library as destination for your Feature provisioned form templates, you can also associate each form template with the document library to which the data is submitted. You can then apply XSNFeatureReceiver to upload the form templates to their document library. Via custom code you should then make sure the documenttemplate of the library is set to the form template. The end-user can then start a form via the 'new' menu of the document/forms library.

    ReplyDelete
  5. Hello, again

    You say:
    "It's possible to apply XSNFeatureReceiver to automate the 'functional user' experience (via the InfoPath client) of publishing InfoPath strict non-browser enabled forms.

    You can publish them to any document library on your site, possible but not necessarily the Forms Services provided FormsServerTemplates document library."

    However, I suspect the XsnFetureReceiver does only the "Sharepoint-land" job: alters the libraries and CTs, so that they know where is their template located. What about the actual XSN - it has "publishurl" attribute inside its definition and it has other attributes and even nodes, that need to be altered/added?!
    However, I couldn't find a single meaningful example on the web, showing neither "Sharepoint-land" part, nor an "Infopath-land" part of a programatic(automatic) USER-like publish done by the XsnFeatureReceiver. What is that class meant to do?! All of the examples involve FORMSSERVICES-like publish: "FormServerTemplates", activation in CentralAdmin, etc - Forms Services infrastructure, and only refer to the class.
    I am confident that I can point my libraries and/or CTs to the correct location of the template(they have DocumentTemplate and DocumentTemplateUrl), I guess I can also upload the template to the correct library(or Forms folder of a directly set form template if need). But nobody wrote anywhere - is that the whole "shrapoint-land" part of publishing?! In the end, assuming XsnFeatureReceiver can do no useful job for me, I am desperate to do the "Infopath-part" by publishing a form template on two sites, downloading the XSNs, and after comparing them(by downloading the xsns, and saving as source files), to do a programatic alter of the differences, on site-provisioning!
    One thing has just come to my mind - since XsnFeatureReceiver is not obfuscated I can see in Reflector what it does and wheather it can help me, or should I just go without it. I see some privata methods, two obfuscated(FeatureActivated and FeatureDeactivating) and two overridable(FeatureInstalled and FeatureUninstalling). Which is the right way do you think - using the XsnFeatureReceiver or not?!

    And a few words about this:
    "Instead of specifying a central forms library as destination for your Feature provisioned form templates, you can also associate each form template with the document library to which the data is submitted. You can then apply XSNFeatureReceiver to upload the form templates to their document library. Via custom code you should then make sure the documenttemplate of the library is set to the form template. The end-user can then start a form via the 'new' menu of the document/forms library."

    Adhering to the principle of looking at the form template as a first class object, which can be put under source control, and be part of my deployment project, it is reasonable to isolate the users from accessing it from a centralized library by as you say "associating form template directly with the document library to which the data is submitted". I won't experience unwanted overriding of the template when redeploying, if it has been chagned by users. This would result in the form template being uploaded into the Forms folder of the coresponding document library, and not associated with any ContentType.
    That way, I would lose, the flexibility of content types, which if I wanted to use, should think of enabling users to dinamically set data connections on creating new libraries based on Infopath-Form-Templated content type. But even if I don't use the goodies of Content Types(we both know what are they), I at least will have a one determined working way to provision IP form templates - each one associated with a single doc library and doing its work fare well. I can then leverage to content type usage, which will be awesome! Is that what you wanted to say?!

    Sorry for the big comment! :)
    And huge regards from me!

    ReplyDelete
  6. You can associate your form template with a content type, and still allow your end-users to directly create new form instances via a document library. To enable that, you have to specify the form in the CT definition as edit form. There are examples of that on the web. Or as alternative: publish your form template via InfoPath client to a content type, and next view the definition of the created CT in your site. I use the FieldsExplorer tool for doing that.

    If you (which is very justifiable) prefer the CT approach, there is no reason to not upload your form templates to the central FormsServerTemplates library. It will also enable you to associate the CT, and thus the associated InfoPath form, with more than 1 forms library; without need to duplicate the form-deployment per library. And you can thus apply XSNFeatureReceiver for the deployment.

    One final word: 'publishUrl' should not be set in deployed forms. In my overload of XSNFeatureReceiver amongst other this property is emptied in the manifest.xsf to allow the form template to be filled in by and end-user.

    ReplyDelete
  7. Hi William,
    Thank you very much for this usefull publication is it possible to have the code of checkOnPresenceandremoveAttribute method?
    Thanks

    ReplyDelete
  8. Hi William,
    Thank you for this usefull post is it possible to get the code of this feature it would be very helpfull.
    Regards

    ReplyDelete
  9. Hi William,

    Thank you for your great post. Are you willing to make any of you code available? More specifically , the checkonpresenseandremoveattribute method?

    Kind Regards,
    Tarek

    ReplyDelete
  10. Hi,

    that particular helper-method is actually not that exiting and complex :-)
    Added the code for your convenience.

    ReplyDelete
  11. Have you discovered a mechanism to mimic the binding of IP form fields to existing site columns with publishing the IP form as a content type. The inability to do this results in site columns with similar names that have different scopes library vs site (for columns reference within the content types)...

    ReplyDelete
  12. No, but this was not my intent. From Information Architecture view I want to deploy own site columns + content type. And since you thus control the provisioning of this information architecture artifacts, you can also decide on the scope; individually deployed to site, or central deployed to content type hub.
    One issue to account for is that the default behavior of InfoPath designer re-creates site columns for all the fields in the form at each (re)publishing. You have some control-options in the InfoPath publishing wizard to prevent the creation of another set of promoted fields, but this is manual-dependent and thus error-prone; let alone it is also very cumbersone and clumsy for the functional managers. To cope for situations in which one has forgot to apply this control, I delivered an 'afterthought correction tool' to programmatically remove the unneeded duplicated site columns (http://williamvanstrien.blogspot.nl/2010/08/more-functional-robustness-issues-with.html)

    ReplyDelete
  13. Thanks William can you please advise how can we go for upgrade scenario meaning once it has been deployed to Prod.do i need to full retract and activate again feature

    Ronak

    ReplyDelete
    Replies
    1. Hi Ronak,

      I assume with upgrade you indicate on an update in one or more of the feature-published InfoPath Form Templates. You can achieve an update by deactivating + deinstalling the InfoPath feature; and next install + activate the feature again with as content the update form template(s), and new site columns, update content type.
      But as you are now already in production with the forms, no longer in project developed; it may be more pragmatic to publish an updated form directly from InfoPath Designer. This is especially true if the changes in form template are small; e.g. merely cosmetic, layout change. Also in case of adding a limited set of fields / and thus site columns; this may still be wise. You must then also via SharePoint GUI extend your feature provisioned content type with site columns created for the new fields.

      Delete
  14. I found your this post while searching for information about blog-related research ... It's a good post .. keep posting and updating information.
    car mechanic brandon

    ReplyDelete