Our Sitecore Setup, Environment and Best Practices Part 1

I am going to talk about our Sitecore environment including the different servers involved, the process of building new functionality and fixing bugs. This post is the product of a lot of blood, sweat, tears + money. It hopefully might give you some direction in how to set up your environment. The project has gone from simple to complex, back to simple and probably ended up somewhere in between. Our set up is probably not in its final state yet as we have found that things don’t last long in our Agile environment. Constant tweaks here and there in aid of improvement.

To start with, we wanted a robust, tested system with as many regression tests as possible. To support this we chose to use continuous integration (CI). The annoying thing is that Sitecore and CI don’t really fit hand in glove. The main pain is the fact that a working environment is a mix of code and data. We found it nice and easy to add code into Subversion and for that code to be automatically checked out, built and tested on a CI box. However how do you handle data changes? For example you could delete a bunch of nodes in Sitecore and break the site, change the template types, change names, change permissions. How do you tie this in with the versioned code.

We decided that trying to directly tie data into the CI process would be super hard. You could have some event serialize the items in the tree and do something with them but we decided it was a time sink for the project.

Moving forward we have stuck with two real areas for testing for our CI, firstly using WatiN for the front end of the web site as well as unit testing and integration tests for core business logic.

Development Environment a.k.a. “Dev Dirty”

This is where the ball starts rolling. This is the developers local machine. The developers local machine is connected to a shared database, shared amongst the other developers. (Caveat – We try not to work on the same task at the same time as we find caching can cause an issue (usually resolved with an IE cache clear and IIS reset)).

We have 2 main branches in our Subversion repository. One called “current” which is the current code we are working on in our development environments and one called “test” for our test environment.

When we pull a feature task off the Scrum board, we would create a branch off the “current” and call it something like Feature – [Seb's New Feature]. This means I can go off and work on this new feature. When I am done with the code I create WatiN tests that test the frond end UI and any business logic testing. We use a Model View Presenter pattern to break open the larger/more complex stuff so that we can test it.

When I am done and happy, I will check this back into to my Feature – [Seb's New Feature] branch. I will then jump back on to the current branch and merge my feature back in. (I am told that this is the easiest way of doing it).

Cruise Control Environment

Upon me checking code back into the current branch, a seperate box running Cruise Control picks up on the fact that something in the code has changed in Subversion. It then runs a custom NAnt build file that pulls out the latest version of code from Subversion, compiles it, tests all our of Nunit and Watin tests and then alerts us to whether or not the build worked. You can at this point make it even more swanky and have it check code coverage (NCover), quality of the code (FXCop) etc…

Development Environment a.k.a. “Dev Clean”

We have experimented with pushing information between environments via packages as well as full blown system publishes (add the destination as a publishing target and it will overwrite it for you). We are currently trying to stick to piecemeal packages for each task/bug we work on. We can then commit those into Subversion under our branch so we can see which code was related to which piece of Sitecore Data.

To give an example, say I changed 2 templates in Sitecore as part of a new feature, I would create a package containing them, and save it under a folder called “Uninstalled Packages” under my feature branch.

This process gives us the ability to logically group and version code + data.

Any way back on track – The Development environment is the next environment upstream of our Local development machines. It is still a development environment, however this is the make or break point to test to see whether our packages work.

We will do an update on the codebase using subversion behind the scenes and install the package using the package installer in Sitecore. If it doesn’t work we go back and re-create the package. Repeat until it works, and re-commit to Subversion.

We are now ready to push onto a Quality Assurance testing environment. The environments upstream of Dev will be covered over the next posts Tune back in for more.

I forgot to renew my domain name :(

Update – I’ve got it back again *rejoice*

I forgot to renew my domain name of www.sebastianpatten.com – sigh. Oh well, I’ll try to get it back when its up for sale in a years time Bloody domain name parkers.

Our Scrum Board

I thought I’d post about what constitutes our Scrum stand-up board. I thought about doing this as I constantly tweak our board, changing its appearance as our current project progresses (our current project is a large website being built in ASP.Net on Sitecore). I’d appreciate any suggestions any of you may have.

Current setup

We currently have 3 mid sized white boards showing information. The first two boards are side by side, containing swim lanes (an alternative name we use for a column on the board) for the following:

• Epic – The name we use to group a bunch of stories together. Example: an epic could be make a sandwich and the story would be butter bread, or apply jam.
• Story Backlog – The swim lane containing all the stories that are to be done, but have not yet been started.
• In Development – Cards are placed in here when they are being worked upon.
• On Dev Box – Cards that have been finished and placed on the development environment.
• On Test Box – Cards that have been finished and placed in the test environment.

Notes from the above:

• We use sticky tags with our names on when cards are in development swim lane. This is done to show who is working on them.
• UA testing is done once cards land in the test swim lane.

We also have a board underneath our main swim lane board. This board contains other project related areas aimed at improving processes.

Firstly we have a story backlog area, where mid-sprint if we stumble upon a story that we hadn’t thought out, we add the story to this area. During the IPM for the next sprint we include these added stories as potential candidates for getting done this coming sprint.

Next we have an area for questions for the project team. This is a good place to throw questions you want to ask the group but might forget about. It is visual so everyone can see it and have a think about the question in hand if they walk past the board.

We also have an area for questions for the business. This is where bigger questions that need business feedback from a business reprasentative go. We usually get a BA to fire these off and get us some answers on.

We have created an area for Standards and Practices on the board that gives people an area to put ideas up that require dedicated time to talking about. For example on this project we might have an idea of talking about how to make our XSLT creation more uniform or how to improve a process. If we create a card and place it in the Standards and Practices area, when it gets to the allocated hour we have bi-weekly we can talk about it.

Finally we have an area for blocked cards. Cards are moved off the swim lanes and placed under the blocked section to visually indicate a dead end with work.

We also have a numerical indicator of the number of bugs to kill and an excel spreadsheet print out graphing our burn down.

Moving forward

I recently had the of adding yet more information to our current setup – but then thought that it might be information overload. What I was thinking of was to add a BIG TEXT, visual calendar so that we could keep an eye on sprint and iterations as well as milestones/deadlines and public holidays. I was also considering a free-to-scribble are of whiteboard where people could write down their greatest concerns for the project. I’m considering recommending that we should regularly touch upon personal concerns in a sprint or iteration technical retrospective.

I believe keeping concerns fresh in our mind will be highly beneficial as we have seen quite a few times when stale, forgotten about concerns have been re-raised, tackled and really helped out the project.

Differences between Javascript NULL and Undefined

This one is pretty slippy.

The main differences are that:

• Undefined, as its name suggests, is the value when a variable’s value has not been defined.
• Null is a value of “no value”.
• Undefined can have the value of null
• Null can not have the value of Undefined
• The == equality operator will return true for a comparison of an undefined variable and null variable
• The === identity operator will return false for a comparison of an undefined variable and a null variable
• Undefined is its own type
• Null is typeof object

Useful tools I use as a software developer

UPDATED – 22/05/09: Added Reflector

Beyond Compare

Website: Beyond Compare
What it is: A file diff tool.

Beyond Compare is probably my favourite day to day tool and has given me an astonishing reduction in breakages when deploying code across systems. There are other free diff tools out there, however after evaluating most of them this is my favourite due to its intuitive design, colourful, easy to distinguish interface.

There is a small cost for the software but I think it is worth it.

See also: (Free) WinMerge

Notepad++

Website: Notepad++
What it is: A free text/source code editor

I like Notepad++ because it is the complete opposite to most of the IDE’s and editors that I use, in that the application is not sucking up all of your RAM and grinding to a halt, when I quickly want to modify a text or HTML file. It makes a nice alternative to the usual Windows Notepad application as it has syntax highlighting for example and useful features such as regular expression based finds. I’ve recently taken to using Notepad++ for a recent PHP client project.

Color Pic

Website: Color Pic
What it is: A free colour picker, and palette

How many times have you needed to open Photoshop to get at the colour picker tool? For me – lots. I found this program helpful as, again, it does not gobble up RAM like Photoshop and is useful for storing a palette of colours indefinitely. I use this tool a lot for finding out what colour something is on a page or for Web Development work.

FreeMind

Website: FreeMind
What it is: An open source Mind Mapping software

I find it very useful at a begging of a personal project to brainstorm everything I want to see in an application. It often becomes quite hard to document and visualize due to the excess of information. Mind Maps can be a really good canvas to paint your ideas on to.

I love how fluid the interaction with this application is. You can draft personalized maps very quickly and relatively intuitively. You can also save them in pdf or image format so others can read.

It’s not something you use lots but can really help when starting off on a project.

Subversion & Tortoise SVN

Website: Subversion, Tortoise SVN
What it is: Version control

I’ve used 3 version control systems in my life and this is my favourite by far. In fact when I look back to the days of Visual Source Safe – I shudder, it was that bad. I like Subversion because of its really well thought out logic and relatively low learning curve.

Subversion itself is the version control system, which comes with natively with a command line interface. I, like many people also install Tortoise SVN which integrates into windows so you can see the state of each file. Version control then becomes a right click, command operation. Easy!

See also: My article on free Subversion hosting

Firefox & Firebug

Website: Firefox, FireBug
What it is: A tool for analysing web page structure, CSS and Javascript debugging

Firebug is a MUST HAVE for web development. Firebug is a Firefox plugin that allows you to visually inspect the web page you are looking at. You can look through the DOM, inspect elements, change HTML and CSS on the fly, watch HTTP headers and debug Javascript. It makes website debugging a lot easier having all these powerful tools in one place. It also saves you having to make changes to your underlying code, building and then refreshing.

7-Zip

Website: 7-Zip
What it is: A file compression + extraction tool

Throw away WinZip and WinRar and replace with this open source solution.

IIS Admin

Website: IIS Admin
What it is: A tool for Windows 2000/XP that allows you to create multiple sites under IIS

I still use this tool a lot as it allows me to create multiple seperate website entities in IIS. This tool is not useful for any one using IIS 7 as it comes natively with this missing functionality.

Twihrl

Website: Twihrl, Twitter
What it is: Twihrl is a client for Twitter, based upon the Adobe Air platform. In case you are behind the times and don’t know what Twitter is – it is a social networking platform where users can post short messages or statuses. A bit like Facebook’s status message.

Why do I think it is a useful tool? Purely because it is a good educational/information tool. If you subscribe to people that “Tweet” (updating of their message), you can passively learn useful information.

Reflector

Website: Reflector
What it is: Reflector is a tool that will disassemble an assembly so that you can read through the source code (as long as it is not obfuscated). I use it a lot for looking through third party code, or perhaps an old .dll that I may have built, but lost the source code for.

Whether to reference by GUID or Path in Sitecore

Here is an interesting debate – whether to use GUIDs or Paths to reference Items in Sitecore.

There are advantages and disadvantages to both routes:

By GUID: If you reference an Item by GUID it will mean that you can move it around the Sitecore tree and even rename it without your code breaking. The caveat with this method is that a GUID is not as representative as a path in giving an indication of what the Item is.

To find its path, place the GUID in the Sitecore search, then mouse over to find the path.

By Path: If you reference an Item by path it means you can read the Item’s location easily, however, if you move it around in the tree or rename it, the code will break.

 

My suggestions for referencing an Item are in the following order:

1. Try to add the Item you want to reference as a link from a field in another important, stationary Item in Sitecore. For example, say you wanted to link to a Meta Data folder in your tree, you could create a field in your Site root called “Meta Data” which was a link field to your Meta Data folder.If this doesn’t apply…
    
2. Use a GUID to reference your item, and some sort of comment in the code to give an idea as to what the GUID relates to.Next best…
    
3.  Use a path to reference your item. It would be beneficial to accompany the code with some sort of NUnit Integration Test which would alert you if the path is not there.

I did come up with an alternative which offers slight benefit to the second and third options above and decided to throw it up onto my blog incase any one else found it interesting. It works in a way similar to ConfigurationManager.AppSettings dictionary, in that you can pass it a key name and it will return you a value. However in this case, using Sitecore Templates, we create a GUID Mapping template that will model a name value pair mapping. Where the mapping is a link to a Sitecore Item. So effectively we would say to GUIDManager that we wanted the GUID for the “Meta Data” item and if we had mapped this in Sitecore, it would return us the Meta Data GUID. (We could even make this return an object if we really wanted).

Ultimately what we get from this class is the flexibility of the GUID in that the underlying link can move round the tree and be renamed and not break our code, as well as having an easy to read, meaningful name.

I’ll post up some Sitecore configuration pictures next week, to explain the GUID Mapping template.

    public static class GUIDManager
    {
        /// <summary>
        /// This method will get the GUID for the given name
        /// </summary>
        /// <param name="name">The name to get</param>
        /// <returns>The associated GUID</returns>
        public static string GetGUID(string name)
        {
            if(string.IsNullOrEmpty(name))
            {
                return null;
            }

            try
            {
                string guidStorageLocation = ConfigurationManager.AppSettings["GUIDStorageLocation"];

                // Check to see whether the storage location has been set
                if(!string.IsNullOrEmpty(guidStorageLocation))
                {
                    // Check to see whether the storage item exists
                    Database database = Sitecore.Data.Database.GetDatabase("master");

                    Item guidStorageItem = database.GetItem(new ID(guidStorageLocation));

                    if(guidStorageItem != null)
                    {
                        Item guidMapping = guidStorageItem.Children[name];

                        // Check to see whether the specified mapping exists
                        if(guidMapping != null)
                        {
                            // Return the GUID from the mapping
                            return guidMapping["GUID"];
                        }
                    }
                }

                return null;
            }
            catch (Exception)
            {
                return null;
            }
        }
    }

Screenshot of an example GUID Mapping

Screenshot of a GUID Mapping template

Sitecore and Item transactions

As I have been doing a lot of work with bulk data insertion and manipulation within Sitecore, coupled with the belief that I don’t think Sitecore offers you the ability to use transactions, I have created my own stand in, simple transaction class.

It utilizes generics and .Net’s disposable interface http://msdn.microsoft.com/en-us/library/system.idisposable.aspx to achieve an easy to read transaction.

If you look at the following code, the first segment shows you how to use the Transaction.cs, the second segment is Transaction.cs. You can see we wrap the transaction in a using statement, which will call the rollback delegate if an exception is thrown at any point. Or alternatively we can call transaction.dispose() which will roll back the transaction also.

How to use Transaction.cs

    public class UtilityMethods
    {
        private Transaction<Item> transaction;

        public void AddChildItems(Item parentItem, string newChildName, TemplateID childTemplateID, int numberOfChildrenToAdd)
        {
            using(transaction = new Transaction<Item>(Rollback))
            {
                for (int i = 0; i < numberOfChildrenToAdd; i++)
                {
                    Item insertedChild = parentItem.Add(newChildName, childTemplateID);
                    transaction.TransactionableItems.Add(insertedChild);
                }
            }
        }

        public void Rollback()
        {
            using (new SecurityDisabler())
            {
                if (transaction != null && transaction.TransactionableItems.Count > 0)
                {
                    foreach (Item item in transaction.TransactionableItems)
                    {
                        try
                        {
                            item.Delete();
                        }
                        catch (Exception)
                        {
                            continue;
                        }
                    }
                }
            }
        }
    }

Transaction.cs

    /// <summary>
    /// This class models a transaction
    /// </summary>
    /// <example>
    /// using(Transaction transaction = new Transaction(// Provide delegate to handle rollback here))
    /// {
    ///     // Sitecore insert
    ///     // Sitecore update
    ///     // Sitecore insert
    ///     transaction.Commit();
    /// }
    /// </example>
    public class Transaction<T> : IDisposable
    {
        public delegate void RollBack();

        private readonly RollBack rollBack;
        private bool isCommited;
        private readonly List<T> transactionItems = new List<T>();

        /// <summary>
        /// Public constructor
        /// </summary>
        ///
<param name="rollBack">A method to use to rollback with</param>
        public Transaction(RollBack rollBack)
        {
            this.rollBack = rollBack;
        }

        /// <summary>
        /// Handles the dispose call
        /// </summary>
        public void Dispose()
        {
            // If the transaction is not commited
            // and object is disposing, rollback
            if (!isCommited && rollBack != null)
            {
                rollBack();
            }
        }

        /// <summary>
        /// Sets the commited flag to true,
        /// rollback cannot happen after commit is called
        /// </summary>
        public void Commit()
        {
            isCommited = true;

            transactionItems.Clear();
        }

        /// <summary>
        /// Gets the list of transactionable Items
        /// </summary>
        public List<T> TransactionableItems
        {
            get { return transactionItems; }
        }

        /// <summary>
        /// Returns whether or not the transaction was comitted
        /// </summary>
        public bool WasCommited
        {
            get { return isCommited; }
        }
    }

Up to date reading list

I’ve added a page listing all the books and decent articles I have read since mid-2008. It might be of interest to some one.

Free hosted Subversion

I was recently looking for some free Subversion hosting when I stumbled upon Unfuddle http://unfuddle.com/.

It is a great FREE service that offers Subversion and Bug Tracking. The free account comes with 200MB of storage, one user account and the ability to have 1 active project. It works great for me being a single developer and I manage to get past the issue of being capped at one project max by using branches. I’m pretty sure I’ll burn up that 200MB space quick enough mind you.

Oh – you can also browse your repository via the web and get updates on changes to the repository.

Running Internet Explorer 6, 7 and 8 simultaneously

UPDATED – Added My DebugBar (15/07/2009)
UPDATED – Added information on Microsoft’s Super Preview (06/05/2009)

Working as a Web Developer, it’s important for me to test my code on multiple web browsers, to make sure that the pages look correct accross browsers (thanks IE6 grr).

My usual testing routine will be to test new web pages in:

• Internet Explorer 6
• Internet Explorer 7
• Internet Explorer 8
• Firefox 3
• Chrome

The problem is getting Internet Explorer 6, 7 and 8 installed on the same box at the same time. Below are my opinions on the different options for installing IE 6,7 and 8.

Best Solution –

1. My DebugBarI like this solution as it is a small download (around 25MB) and manages to run all the IEs in a stand alone application. At the time of testing however Windows 7 was not supported.
2. As an alternative, there is always Xeno Code’s on demand browsers. These guys offer a unique technology that allows you to run any web browser on demand, so that you don’t ever have to install it. It’s nice for some one like me who doesn’t want to install Safari http://www.xenocode.com/browsers
   Taken from their site:

   Xenocode® is an advanced application virtualization and streaming microkernel that emulates core OS subsystems entirely within user-mode space. Virtualized applications reside in isolated “sandboxes”, allowing software to run side-by-side without conflicts or modifications to the host device.

   Our unique, patent-pending application virtualization and XStream™ delivery technology provide users with a highly reliable, low-latency product experience on the web, intranet, or USB storage devices.

Good Solution – Web Super Preview, an Expression based Internet Explorer only preview application. A bit large if you ask me – its around 250+ MB! What I like about it is the ability to put different versions of internet explorer side by side. http://www.microsoft.com/downloads/details.aspx?FamilyID=8e6ac106-525d-45d0-84db-dccff3fae677&displaylang=en

Poor Solution – http://tredosoft.com/Multiple_IE gives you IE 5, 6 and 7 in one bundle. It works well, but the software now seems a bit out dated (no IE8).

Untested – Internet Explorer Collection http://www.softpedia.com/get/Internet/Browsers/Internet-Explorer-Collection.shtml, which includes IE8.