Martin Hatch – Tales of Tech

So in Part 1 we talked about some of the upgrade mechanisms available in SharePoint 2010, including Assembly level and Feature level versioning.

In Part 2 we are going to look at 2 distinct versioning strategies:

• In Place Upgrade (replace old with new)
• Side-by-Side Release (allow both version to run at the same time)

In-Place Upgrade

This is going to be most common scenario for bug fixing, hotfixes and minor improvements to the solution. So you might be adding some help files, or tweaking a user interface, or perhaps fixing some JavaScript or CSS bugs in your code.

Typically you would not use this approach for major versions of products, as a major change to interface or functionality is likely to confuse existing users, and might causing existing configurations to break! There are exceptions however, your client might not want to manually swap out 500 web-parts on their portal and might want a “big bang”.. provided it has been thoroughly tested!

So what did we discuss in Part 1 that can be used in SharePoint 2010 to achieve an In-Place Upgrade?

$SharePoint.Project.AssemblyFullName$

This can allow you to easily update all of your ASCX / ASPX / CAML references with the new assembly.

Next time your Page / Web Control / Feature receiver tries to execute then it will be running the new code, seamlessly.

Assembly Binding Redirects

This allows you to pick up any code that is attempting to execute in the old version, and forces it to run through the new assembly methods instead.

This is a blanket, assembly-wide approach, typically deployed at the Web Application level (although you can go server-wide using the machine.config!)

Warning – This obviously only works from code that runs through the web.config and IIS / ASP.Net. Any other code (such as Timer Jobs and Workflows) will be completely unaware of these redirects and will happily continue running the old code!

Workflows especially should not use binding redirects this way. If you drop in a redirect for a workflow that is currently running then you will break it next time it tries to serialize itself to the database!

Feature Versions

The feature framework allows you to do incremental upgrade steps to modify existing features, upgrading and “replacing” the functionality for existing items (such as Content Types, site columns, custom actions … anything that you would provision through a Feature.xml).

You might have to be careful with custom code (and the CustomUpgradeAction element) but all in all, Feature versions are designed to do an “in place upgrade” of your features.

Side-by-Side Release

This is likely to be the most common scenario for Major Versions, functionality changes, new templates. It is important to recognise that certain elements in the SharePoint schema MUST be upgraded in this way (such as Site Definitions, List Definitions and Workflows).  For this scenario “upgrade” is probably the wrong word to use, because what we really have to do is create a copy (and in some scenarios hide the old one!).

So how would you go about creating a side-by-side upgrade? Well, the first thing to consider is that you need to leave your existing assembly in place.

Solution Package (WSP) Considerations

If you “retract” the solution package then typically it will remove your original DLLs. This will cause existing functionality to break (this is bad).

One workaround is to create a new WSP. SharePoint will treat this as a separate solution, allowing you to install it while leaving the existing one in place.

Assembly (DLL) and Package (WSP) Considerations

The assembly needs careful consideration, depending on where your assembly is going to live;

If you have a full trust package which deploys to the GAC, then you really don’t have too much of a problem. You can create your new assembly with a different version number and deploy that DLL to the GAC. The GAC has been designed that the same assembly with multiple version numbers can happily sit side-by-side.

If you have a minimal trust package which deploys to the web application BIN folder then you have more of a problem. Releasing the same assembly name is going to overwrite the file (deleting the original DLL, which will break existing functionality … also bad). Other than moving it to the GAC (which breaks the whole point of deploying for minimal trust in the first place!) there is only one practical workaround here; rename your assembly (so you have 2 different DLL file names)

So now that you have dealt with your assembly, what about other assets?

Workflows (WWF)

Workflows are a nasty one. One they are running the class gets serialized into the database, so you HAVE to leave existing code on the system until all instances of that workflow have stopped running! (you can set a workflow to “no more instances” from the remove screen). The problem is that in a typical environment you are probably going to have NO IDEA how many workflow instances are running (and even if you did, you probably won’t be able, or willing, to stop them!).

The only scenario is to provide a duplicate workflow. Copy the workflow in Visual Studio, make your changes, and provision it into SharePoint as a DIFFERENT workflow.

Again, you would hide the original (so people can’t add them to new lists) and you would probably want to perform some kind of exercise to remove any existing workflows (once all the running instances have stopped) and setup the new workflow as a new instance on the list / content type.

As you can imagine, this is NOT going to be an easy task and it’s highly likely that this will be a manual governance effort, and not something that can be easily automated.

(note – even if you are leaving the existing workflow class in place you still can’t change the assembly version … the serialization will fail and existing workflow instances will break!)

Site Definitions (onet.xml)

Microsoft very clearly states that modifying a Site Definition while it is use is not supported.

So how DO you “upgrade” them? Well, you have 2 options:

Feature Stapling

The easy solution is to use a feature stapler to add new functionality to an existing site.

This allows you to modify NEW sites that are created, although you won’t be able to impact on everything (features activate before lists and pages are created, so you won’t be able to modify those!)

This is discussed in detail on MSDN: https://msdn.microsoft.com/en-us/library/bb861862(office.12).aspx

Copy and Hide the Original

This is pretty straightforward. You copy your Site Template folder and create a new version.

You then modify your WEBTEMP*.xml file to add in your new template (using the same name / description as the original).

You then set the original template in the WEBTEMP*.xml as Hidden=TRUE.

Deploying this, your users still “see” exactly the same when they create new sites, but when they do create the site it is now provisioning using your new template!

List Definitions (schema.xml)

This is a very similar story to Site Definitions. You can change “some” things but not everything (and some of the things you change may break existing lists!).

The most reliable way is to copy your Feature (giving it a new GUID) and hide the original ListTemplate. This will stop users from using the old template to create new lists, but if you have existing sites that are provisioning them automatically then you’ll have to look at the Site Definition upgrade (see above!).

That should give you a pretty good grounding (at the very least a start!) at how to upgrade your projects, either in-place upgrade or side-by-side upgrade.

A lot of it is going to be head-scratching and sitting down to think .. don’t rush into it, take your time, and make sure you truly understand what you are trying to achieve, and what the impact is going to be on the end users!

This post was prompted by needing to explain the versioning and upgrade strategy in SharePoint 2010 to some partners we are working with at Content and Code. It ended up a very length conversation so I decided to jot it all down and split it up into 2 parts.

• Part 1 – Feature and Assembly Versioning
• Part 2 – Upgrading your projects for In-Place Upgrade or Side-by-Side release.

Regarding upgrading SharePoint 2010 projects there are 2 specific areas of new functionality that will key for SharePoint 2010 Application Lifecycle Management:

• Assembly Versions
• Feature Versions

Assembly Versions

It is typically recommended that the Assembly Version is incremented when you are upgrading / fixing / changing any of your managed code. This will however have an implication for web parts, web controls and pages across the system, but SharePoint 2010 has provisioned for that.

$SharePoint.Project.AssemblyFullName$

This is a new placeholder value that you will find all over SharePoint 2010. This includes both ASPX / ASCX  files (such as Page Layouts or Visual Web Parts, as an <% @Assembly %> tag) as well as Feature Receivers (in the Feature XML). Basically, when you package your WSP SharePoint will automatically swap out this value for the fully-qualified Assembly Reference for the current project state. This means it will swap out the full Assembly name, version number, culture and Public Key Token.

Assembly BindingRedirect element (solution manifest.xml)

Another new feature in SharePoint 2010 is the ability to specify an AssemblyBindingRedirect from the WSP.

This is part of the Manifest.xml schema (and has intellisense support in Visual Studio 2010) to allow you to automatically add new assembling binding information when a new WSP is rolled out.

Example: binding redirect to redirect version “1.0.0.0” to the current assembly version in the package:

<Assemblies>

    <Assembly Location="MyCustomAssembly.dll" DeploymentTarget="GlobalAssemblyCache">

      <BindingRedirects>

        <BindingRedirect OldVersion="1.0.0.0"/>

      </BindingRedirects>

    </Assembly>

</Assemblies>

With these two functions combined, we should have no problem upgrading an existing assembly to a new version.

You should be aware though that the Binding Redirects will only apply to the Web Application’s web.config, and therefore will NOT work for anything operating outside of the web context (such as Timer Jobs and Workflows!). This is discussed in more detail in Part 2.

Feature Versions

There are a number of new version-related parts to the Feature framework. Each feature.xml can now be given a specific Version number property (if you don’t specify then it will be given a value of 0.0.0.0). This property was listed in the WSS 3.0 SDK, but it was reserved for internal use at the time (now it finally gets some legs!)

This all stems from a new set of XML elements that you can add to the Feature.xml, starting with <UpgradeActions> element.

<VersionRange>

The first element is the Version Range .. which allows you to specify the BeginVersion and EndVersion. The actions specified in the upgrade will only be processed if the if the current version of the feature is between that range!

Typically you would wrap this element around the actions that you wish to be executed when the upgrade is performed.

<AddContentTypeField>

Pretty much as it says on the tin, this allows you to add a field to an existing Content Type.

<ApplyElementManifests>

This allows you to apply a specific element manifest when the upgrade process kicks off.

This is good because if you are adding a new element manifest (say to provision a new site column?) then you can process that specific manifest in isolation as part of your upgrade process without going off and executing all of the other manifests in the feature.

<MapFile>

According to the SDK this allows you to re-map a file in the content database to a different “ghosted” location. I suppose this is a supposed to be a quick, memory efficient way of gracefully modifying files, although I’m not sure what value this really has? (could you not just overwrite the file on the hard disk?)

<CustomUpgradeAction>

This element ties in with your Feature Receivers. There is a new method in the SPFeatureReceiver class. This method is called: FeatureUpgrading() and it allows you to execute custom code as part of your upgrade process.

(the only problem is that .. if you deploy your feature to a brand new farm and execute it “first time” from the new version .. this method does not get executed! You should really consider using the Feature Activated event to make sure it the behaviour is consistent!)

Example:

Below is an example snippet.

Here, if we are upgrading from Version 1 to Version 2 then a new field will be added to a content type, and those changes pushed down to all child content types and lists.

If we are upgrading from Version 2 to Version 3 then a new element manifest will be processed.

<UpgradeActions>

    <VersionRange BeginVersion="1.0.0.0" EndVersion="2.0.0.0">

      <AddContentTypeField

ContentTypeId="0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39"

FieldId="{C6885FF1-4F70-425F-AB32-41767AB1B8CC}"

PushDown="TRUE"/>

    </VersionRange>

    <VersionRange

        BeginVersion="2.0.0.0"

        EndVersion="3.0.0.0">

      <ApplyElementManifests>

        <ElementManifest Location="UpgradeManifest.xml"/>

      </ApplyElementManifests>

    </VersionRange>

  </UpgradeActions>

With the feature upgrade framework, this should allow you to do almost anything when managing how your features are “upgraded” in a graceful way.

In Part 2 we will discuss how we can leverage these for two different upgrade scenarios:

• In-Place Upgrade (replacing existing functionality)
• Side-by-Side Installation (allowing two versions to co-exist)

What a way to lose 4 hours of your life.

I was in a glorious mood this morning having upgraded my main development environment to RTM (both SharePoint 2010 and Visual Studio 2010 (Premium)).

The main code base we’d been working on (a brand new SP2010 web site, due to go live next month) compiled, the packaged could be created and I manually deployed the WSP and all was good … that is until I tried the “Deploy” button in Visual Studio…

We’re mid-way through building our first commercial SharePoint 2010 system, We’ve been at it for a few months, but are starting to shift all our software over to RTM, and there are some quite lovely gotchas to be aware of.

Visual Studio 2010 is a fantastic product, and a great improvement over VS2008, but if you are not careful it can lead to lazy development.

The main lesson of the day is do not rely on Visual Studio 2010 “Deploy” to test your packages!

There are a number of reasons why this method is unreliable from a testing perspective:

1. Deploys to existing site collection
2. Automatically activates your features
3. Automatically resolves conflicts

Challenge #1 Deploys to existing site collection
This is quite critical (and in many ways, rather annoying) that Visual Studio 2010 will deploy to a Site Collection. This of course means that if you have deleted your site collection then you have to deploy the old fashioned way (typically using STSADM or PowerShell).

But how many developers in the coming months are going to end up shipping WSPs to their test teams with broken site collections?

I know it sounds stupid, but I’ve even done it myself .. deleted features from the Package, done a VS2010 deploy and everything works fine. Until you re-create your site collection and there is a Feature ID in the onet.xml that doesn’t exist anymore.

All good common sense stuff .. but make sure you are regularly re-creating your site collections!

Challenge #2 Automatically activates your features
This is another great feature, but can seriously break your builds (or lead you to produce incomplete builds if you aren’t careful!)

Central Administration is a great one here. If you are buidling features for Central Admin (say new admin pages or custom action features) is quite common (nay, useful) to set your Features to AutoActivateInCentralAdmin=”TRUE”. Yeh .. if you have your VS project pointing at the Central Admin URL then this will break your deploy function!

You see .. VS will “deploy” your WSP (which auto-activates the features in Central Admin) .. then VS tries to activate them AGAIN. This causes the deployment process to crash, so you have to kick VS back into non-activation mode.

This even creates challenges in your Site Collection (don’t forget Challenge #1 up above). VS2010 kindly activates all of your features for you … yes .. ALL of them. You don’t get to choose which ones get auto-activated, and unless you order them correctly in the package then you can’t explicitly control which order they are activated in either.

So you have another problem where your dev build is happy as larry, but when you hand the WSP to your testers they moan that activation dependencies are breaking, or certain features aren’t showing up!

Challenge #3 Automatically Resolves Conflicts
Now .. don’t get me wrong .. for a dev environment this is GREAT … it means we can roll things in, out and in again all day long without having orphaned files or worrying about overwrites.

But again, it creates that “non-realistic deployment” scenario where developers roll out their WSP for the 20th time .. not realising that when you do the same thing in your test environment you don’t have Visual Studio sitting in the background doing a load of resolution conflicts.

I suppose the lesson of the day is be careful!

Think about where your WSP is going to end up, who is going to be deploying it, and what environment it’s going to live in.

So .. there I was in SharePoint 2010, working on a new corporate website in Visual Studio 2010.

We decided we needed some image assets to be provisioned, so I decided to make use of some of the new features of VS2010 and created a “Module” SharePoint Project Item.

I added my images, telling to to deploy to RootWebOnly and setting the path as PublishingImages, everything a good SP dev should do…  or so I thought.

Deployed my solution .. blam .. Site Collection dead. I fired up the (quite excellent) ULS Viewer which told me the following error messages:

‘Failed to create the ‘Images’ library.’. Exception was: ‘Microsoft.SharePoint.SPException: A list, survey, discussion board, or document library with the specified title already exists in this Web site.

Publishing Feature activation failed. Exception: Microsoft.SharePoint.SPException: Provisioning did not succeed. Details: Failed to create the ‘Images’ library.

Now this was instant brown trousers time … the Publishing Feature was broken!!! (or so I thought).

What had actually happened was brain stunningly simple. You see .. Visual Studio had kindly placed my new Module item into a Site Scoped feature…

The Publishing Feature is Web Scoped…

Now .. hopefully the sharper reader will already have realised the predicament. As I explained in an earlier blog post (onet.xml order of execution) Site Scoped features will activate before web scoped features … more accurately my Modules feature is trying to place my images into /PublishingImages/ before the document library has been created!!

What it actually did was create a folder in the SPWeb (yes .. you can have SPFolder objects directly within an SPWeb .. you don’t need a document library) so when the Publishing feature came along the URL “/PublishingImages” was already in use!

I created a new Web scoped feature for my images module and re-deployed … everything is now working. (the publishing feature activates first .. creating the image library … and my images are placed inside it).

*sigh*

Another one to put large and clear on the blackboard of experience!

(Note – this behaviour was observed in VS 2010 RC .. I haven’t tested yet in RTM)

One of the most challenging things I’ve had to deal with in a long time is trying to work out how to get Visual Studio 2010 to activate my Fields feature before it tries to activate my Content Type feature 🙂

First I was (obviously) getting errors that the field didn’t exist (because it couldn’t find the Field that the FieldRef was pointing at).

Next I tried adding a Feature Activation Dependency, hoping that Visual Studio would intelligently activate them in the order of the dependencies that I setup … WRONG … I am now getting errors saying it cannot activate because the dependency is not active (doh!).

Finally struck upon something .. going to the Package .. the features were being activated in the order they appeared in my VS 2010 Package item.

Unfortunately there is no “move-up”, “move-down” or drag and drop re-ordering, so I had to remove all of the features from the package and then put them back in the correct order of activation.

UPDATE
It seems there ARE move-up / move-down buttons, but they don’t stick to the scrollbar so if you scroll down (to the bottom of the package) you can’t see them anymore. Anyway .. yes you can re-order them .. but you’ll maybe find this slightly more difficult if your features are at the bottom of the package list.

Slightly frustrating, but glad it’s all working now!