Our Sitecore Setup, Environment and Best Practices Part 1

I am going to talk about our Sitecore environment including the different servers involved, the process of building new functionality and fixing bugs. This post is the product of a lot of blood, sweat, tears + money. It hopefully might give you some direction in how to set up your environment. The project has gone from simple to complex, back to simple and probably ended up somewhere in between. Our set up is probably not in its final state yet as we have found that things don’t last long in our Agile environment. Constant tweaks here and there in aid of improvement.

To start with, we wanted a robust, tested system with as many regression tests as possible. To support this we chose to use continuous integration (CI). The annoying thing is that Sitecore and CI don’t really fit hand in glove. The main pain is the fact that a working environment is a mix of code and data. We found it nice and easy to add code into Subversion and for that code to be automatically checked out, built and tested on a CI box. However how do you handle data changes? For example you could delete a bunch of nodes in Sitecore and break the site, change the template types, change names, change permissions. How do you tie this in with the versioned code.

We decided that trying to directly tie data into the CI process would be super hard. You could have some event serialize the items in the tree and do something with them but we decided it was a time sink for the project.

Moving forward we have stuck with two real areas for testing for our CI, firstly using WatiN for the front end of the web site as well as unit testing and integration tests for core business logic.

Development Environment a.k.a. “Dev Dirty”

This is where the ball starts rolling. This is the developers local machine. The developers local machine is connected to a shared database, shared amongst the other developers. (Caveat – We try not to work on the same task at the same time as we find caching can cause an issue (usually resolved with an IE cache clear and IIS reset)).

We have 2 main branches in our Subversion repository. One called “current” which is the current code we are working on in our development environments and one called “test” for our test environment.

When we pull a feature task off the Scrum board, we would create a branch off the “current” and call it something like Feature – [Seb's New Feature]. This means I can go off and work on this new feature. When I am done with the code I create WatiN tests that test the frond end UI and any business logic testing. We use a Model View Presenter pattern to break open the larger/more complex stuff so that we can test it.

When I am done and happy, I will check this back into to my Feature – [Seb's New Feature] branch. I will then jump back on to the current branch and merge my feature back in. (I am told that this is the easiest way of doing it).

Cruise Control Environment

Upon me checking code back into the current branch, a seperate box running Cruise Control picks up on the fact that something in the code has changed in Subversion. It then runs a custom NAnt build file that pulls out the latest version of code from Subversion, compiles it, tests all our of Nunit and Watin tests and then alerts us to whether or not the build worked. You can at this point make it even more swanky and have it check code coverage (NCover), quality of the code (FXCop) etc…

Development Environment a.k.a. “Dev Clean”

We have experimented with pushing information between environments via packages as well as full blown system publishes (add the destination as a publishing target and it will overwrite it for you). We are currently trying to stick to piecemeal packages for each task/bug we work on. We can then commit those into Subversion under our branch so we can see which code was related to which piece of Sitecore Data.

To give an example, say I changed 2 templates in Sitecore as part of a new feature, I would create a package containing them, and save it under a folder called “Uninstalled Packages” under my feature branch.

This process gives us the ability to logically group and version code + data.

Any way back on track – The Development environment is the next environment upstream of our Local development machines. It is still a development environment, however this is the make or break point to test to see whether our packages work.

We will do an update on the codebase using subversion behind the scenes and install the package using the package installer in Sitecore. If it doesn’t work we go back and re-create the package. Repeat until it works, and re-commit to Subversion.

We are now ready to push onto a Quality Assurance testing environment. The environments upstream of Dev will be covered over the next posts Tune back in for more.

Our Scrum Board

I thought I’d post about what constitutes our Scrum stand-up board. I thought about doing this as I constantly tweak our board, changing its appearance as our current project progresses (our current project is a large website being built in ASP.Net on Sitecore). I’d appreciate any suggestions any of you may have.

Current setup

We currently have 3 mid sized white boards showing information. The first two boards are side by side, containing swim lanes (an alternative name we use for a column on the board) for the following:

• Epic – The name we use to group a bunch of stories together. Example: an epic could be make a sandwich and the story would be butter bread, or apply jam.
• Story Backlog – The swim lane containing all the stories that are to be done, but have not yet been started.
• In Development – Cards are placed in here when they are being worked upon.
• On Dev Box – Cards that have been finished and placed on the development environment.
• On Test Box – Cards that have been finished and placed in the test environment.

Notes from the above:

• We use sticky tags with our names on when cards are in development swim lane. This is done to show who is working on them.
• UA testing is done once cards land in the test swim lane.

We also have a board underneath our main swim lane board. This board contains other project related areas aimed at improving processes.

Firstly we have a story backlog area, where mid-sprint if we stumble upon a story that we hadn’t thought out, we add the story to this area. During the IPM for the next sprint we include these added stories as potential candidates for getting done this coming sprint.

Next we have an area for questions for the project team. This is a good place to throw questions you want to ask the group but might forget about. It is visual so everyone can see it and have a think about the question in hand if they walk past the board.

We also have an area for questions for the business. This is where bigger questions that need business feedback from a business reprasentative go. We usually get a BA to fire these off and get us some answers on.

We have created an area for Standards and Practices on the board that gives people an area to put ideas up that require dedicated time to talking about. For example on this project we might have an idea of talking about how to make our XSLT creation more uniform or how to improve a process. If we create a card and place it in the Standards and Practices area, when it gets to the allocated hour we have bi-weekly we can talk about it.

Finally we have an area for blocked cards. Cards are moved off the swim lanes and placed under the blocked section to visually indicate a dead end with work.

We also have a numerical indicator of the number of bugs to kill and an excel spreadsheet print out graphing our burn down.

Moving forward

I recently had the of adding yet more information to our current setup – but then thought that it might be information overload. What I was thinking of was to add a BIG TEXT, visual calendar so that we could keep an eye on sprint and iterations as well as milestones/deadlines and public holidays. I was also considering a free-to-scribble are of whiteboard where people could write down their greatest concerns for the project. I’m considering recommending that we should regularly touch upon personal concerns in a sprint or iteration technical retrospective.

I believe keeping concerns fresh in our mind will be highly beneficial as we have seen quite a few times when stale, forgotten about concerns have been re-raised, tackled and really helped out the project.