Beware: timerjob processes invoked via Central Admin deploy solution do not reload FeatureReceiver assembly

Recent we ran into an exception when deploying an external facing website to the production SharePoint farm. The problem originated in the FeatureInstalled() method of a custom FeatureReceiver class. We did not encounter this particular deployment problem in the integration nor QA environment. Since on a production environment it is not viable to debug, we had to resort to inserting logging statements in the code, and then redeploy the assembly. But strangely, no logging was written leaving us still blank with respect to the exact code location and cause of the deployment problem.

After several deployment attempts together with operations, we came up with the explanation why the logging was not done. The timerjob process that executes the FeatureReceiver FeatureInstalled() method still has the old code loaded!! To force that the assembly is reloaded from either GAC or virtual bin after deploySolution and before execution of FeatureInstalled, you need to recycle the timerjobs processes. The right time for this is after the retractSolution of the previous version of the deployed SharePoint solution package.

NB: we didn't notice this 'faulty' deploy/timerjobs behaviour before because it's not very common to actually utilize the FeatureInstalled() method. In this particular case it was necessary to timely correct something before the SharePoint feature framework performs the OOTB feature activation work.