Tridion Content Delivery and Caching

I recently had to assist a customer increasing the performance of their (Tridion-powered) site. The initial claim was that Tridion caching was not performing as expected and their database was being hit too many times.

I have used Tridion cache in all my projects, and especially when coupled with Dynamic Component Presentations I see drastic performance improvements. I don't remember ever having performance issues related to this, so I was a bit astonished as to the claim.

A quick overview of the code for this "static" site quickly revealed the reason why cache was not performing as desired - and I'll get to this in a second.

The Tridion content delivery cache uses a fairly complex dependency algorithm and something we call the "Cache Channel Service" to notify all delivery nodes when a given item has been published/republished/unpublished and therefore invalidate its cached version. When all is operating normally you will NEVER get an out-of-date page, even on a large (100+ servers) delivery environment. (You can read more details about the Cache Channel Service on this article at SDLTridionWorld).

However, what very often developers seem to not understand is what is cached, and what is a "dynamic" page in the context of Tridion. So let's do some definitions here.

Tridion Dynamic Page.
We call a page "Dynamic" not when it responds to a user's behavior, preferences or profile. We consider a page as dynamic if the system doesn't necessarily "know" what components will be displayed at the time the page was published. A good example of such a page is a "press release index" page, where the component presentations displayed are dynamically retrieved from the Content Broker at run time.

A page that includes ASCX controls to perform taxonomy lookups and determine which component presentations to display is not what we would call static - even if the page contains no logic to interact with the user's profile (which raises some more questions about why the taxonomy queries are being performed, but that's an issue for another day).

Tridion Object Cache
As the name implies, the Tridion object cache contains all the recently used Tridion objects loaded by your application - this typically includes Page Metadata, Component Metadata, Component Presentations, Dynamic Links, Keywords, Taxonomy objects, etc.

It allows us to minimize the number of times a given object is loaded from the content repository (file system or database) by storing a cached version of it - which gets invalidated only when 1) it is republished/unpublished or 2) the cache runs out of usable memory (configurable setting, by default set to 16 MB).

OK, so what does "object" cache mean?
It means that queries to the content broker (for instance "all press releases sorted by last published") will result on a set of objects being received by your application - and all those objects will probably be in memory already. So when you call ComponentPresentation.GetContent() you  will have an average wait time of 0 milliseconds or thereabouts.

However, your query results are not cached. In context of:

• How cache invalidation occurs;
• What we consider dynamic;
• How often your content may be updated,

it makes sense that database query results don't get cached - otherwise any publishing action would invalidate all query result caching, and on a site that is updated thousands of times a day, this would result in more cache invalidation messages than anything else.

We could automatically make the results cached for a duration of, let's say, 5 minutes - but then you don't get the latest results, etc.

So Tridion (as we expect) took the "hands-off" approach of letting you control the cache through your application.

OK, so back at the issue at hand. I'm trying to ensure that the page I currently load, which may display up to 10 Component Presentations based on keywords attached to these components, improves its average load time from 1 second to as quick as possible. Since all these queries are going to the database, even if database cache is performing correctly, you will have to cope with additional db connections, network latency and what-not - multiplied by 10-12 queries per page, and here's a recipe for "slowness".

This site doesn't get updated very often, and has a very high number of average sessions, so I decided to test it with a 5 minute ASP.NET cache object for all my queries. Each individual query will be cached for a maximum of 5 minutes, which is an acceptable value for content editors/publishers.

What did I find?
Average load time went from 800 milliseconds to 17 milliseconds with an extremely simple change to my code.

For many reasons, I will not post the code as is currently being used at the customer, but here's the basics of it.
A wrapper method was created for Query (Tridion.ContentDelivery.DynamicContent.Query) which takes 2 parameters: the query object and a unique (per query) cache key. This cache key is nothing more than the string concatenation of the query parameters.

private String[] ExecuteTridionQuery(Query query, 
      String CacheKey)
{
  String[] results;
  if (Cache.Get(CacheKey) != null)
   results = (String[])Cache.Get(CacheKey);
  else
  {
  results = query.ExecuteQuery();
  Cache.Add(CacheKey, results, null, 
    DateTime.Now.AddMinutes(5), 
    System.Web.Caching.Cache.NoSlidingExpiration, 
    System.Web.Caching.CacheItemPriority.Normal, null);
  }
  return results;
}

And voilà, instant 60x boost to your "static" page.

N

#upintheair

I get frequently asked "where I've been lately", and been finding it really hard to tell people where I've been, simply because it mostly feels like I only briefly "was" anywhere, never really got to be there - but I do remember going there and coming back.

So, here's a list of where I've been to this year for work, that I can refer people to if I don't feel like telling them... :)

• San José, CA (
• Laguna Beach, CA
• Newport, CA
• San Francisco, CA
• Oakland, CA
• San Ramon, CA
• Las Vegas, NV
• Washington, DC
• Phoenix, AZ
• Columbus, OH
• Amsterdam, The Netherlands
• Helsinki, Finland
• Lisbon, Portugal

MVP Nomination stats

Got ten nominations in less than a week, so it's looking pretty good for the 2011 MVP awards.

With still nearly 3 months to go (nominations are valid if received before January 1st 2011) I'm hoping to have many more candidates! See how you can nominate someone for an MVP award here.

An interesting fact is that 85% of people nominating others are based in the Netherlands, the UK has not nominated anyone yet, and only one person in the USA has nominated someone.

There's clearly something wrong with those numbers, last time I checked the US had overtaken the Netherlands and the UK in number of visits to the forum and SDLTridionWorld, however the community is still very much based in and around Amsterdam.

A second interesting fact - which I hadn't noticed until someone mentioned it in the MVP summit last week - is that we have several (4) MVPs based in the Netherlands, but none of them is dutch. Actually, we don't have any dutch MVPs, which is quite odd if you look at the company's history. Maybe it will change this year?

Well, keep 'em coming, there is no maximum number of MVPs, all you need to do is to be valuable to the community, and share your knowledge.

MVP Summit 2010, Fortaleza do Guincho, Portugal

Judging from all the activity on twitter, one would think that the MVPs locked themselves in a room and did nothing else but code.

Indeed, I think we got something very useful off the ground with the Hello World GUI extension sample, and the Content Interaction extension (aka MVP Fondue Kit), but what I really liked the most was the opportunity to spend some time with the Tridion MVPs and pick their brain about the Tridion community, what to improve, what to expect from 2011, the whole MVP nomination process, etc.

And, of course, the location!

Surrounded by the cliffs of Guincho and the imposing Serra de Sintra, the scenery has to be one of the prettiest I've seen.

I'm obviously biased, having lived for 11 years pretty close to this location, but I had never stayed in the Fortaleza do Guincho hotel before, and I must say it was quite a good choice for the event, as we couldn't have asked for more.

So, rather than bore you with more details about what the MVP did or didn't while in Portugal, I'll just leave you with a couple more pictures...

Phoenix, Arizona

Just added one more city to my list of "places I've been to for work". It's 101 degrees outside, at 9:36 PM (that's 38 degrees celsius).

Pretty long week ahead, being involved in a rather large Proof of Concept for an amazing reference if we get the deal.

Time will tell, I hope to announce this company as a customer soon. Now I think I'm going to pass out in bed.

Amazing how you can fly for 5 hours and 1) never leave the same country and 2) land in a place that feels just like the place you left from - except for the temperature, that is.

This is cool... Tridion 2011 from my iPhone

After all the talk about Tridion on the iPad I thought that if it runs on the iPad, it should run on the iPhone, right?

Right indeed.

Not that I think it's a very useful client, but it does work...

N

Preparing for 2011 MVPs

Last year we launched the SDL Tridion MVP program, and being the first time it took a rather "Career Achievement" look to it, and (perhaps unfortunately) many of the nominations (over 60%) were for SDL employees.

No matter what people think of it, we decided to create an independent Selection Panel that would select who deserved the MVP title, and awarded it to those that did make an exceptional job of assisting people in the community.

In preparation for the 2010 MVPs I can confirm that we will keep the independent selection panel (with a couple of adjustments as to the members of the panel to adapt to people's changes in careers), but this year we will not allow for SDL employees to be nominated.

This means that great Tridion resources like Jeremy, Julian, Jeff and Peter will not be 2011 MVPs (and, of course, yours truly).

Now that this is out of the way, here's the 2nd part of my message for today...

Why would you want to be an MVP? 

Granted, being the first time we were doing it, there was a lot of guess work as to what people would want. First - and perhaps foremost - you get to have your name in a rather exclusive list of Tridion professionals. I know of at least 2 people that got contacted for work directly from being on that list, and there's probably more.

Second, you get a shiny award to show off to your friends (who already considered you a geek beforehand, and will now have confirmation of your uber-geek status worldwide).

Third, you get access to a MVP-only Linked-In group of very dubious benefit at the moment.

Fourth, you get to spend a "work" weekend with all your fellow MVPs on a nice sunny location, 1-star Michelin restaurant, and breakfast by the beach (aha!).

Fifth, and professionally the most important, you get access to anything related to Tridion. You want to participate in "internal-only" bootcamps? No problem. You want some version of some software? No problem. Want to play around with Tridion 2011 Beta? No problem. Want to escalate an issue with Customer Support? No problem (well, depending on the issue). Need access to a license, information on some upcoming product, you-name-it? No problem.

So, question for the day: What are you doing now that makes you a great candidate for the MVP award next January?

N

MVP

I was asked to provide a picture of me with my shiny MVP award... so here goes :)

Outputting a keyword hierarchy in XML

With Tridion 2009 we saw the introduction of hierarchical keywords - aka Taxonomy, aka Intelligent Navigation - and with it a whole new world of possibilities for content classification.

However, in some cases, you really don't want all the intelligence around it and would much rather have just an "old style" xml hierarchy. If you tried this yourself, you probably figured out by now that the keyword hierarchy doesn't look so... hierarchical when viewed through the API. Not at all.

It looks rather flat.

I had to write a template recently with some information about every keyword in the hierarchy (including its metadata) for a site navigation, and keeping the keyword "parent/child'' relationships.

Here's how can you do it.

Category Navigation = engine.GetObject(NAVIGATION_CATEGORY) as Category;
using (MemoryStream ms = new MemoryStream())
{
    XmlTextWriter w = new XmlTextWriter(ms, new System.Text.UTF8Encoding(false));
    w.Indentation = 4;
    w.Formatting = Formatting.Indented;

    w.WriteStartDocument();
    w.WriteStartElement("Navigation");
    log.Debug("/Navigation created");

    Filter filter = new Filter();
    filter.Conditions.Add("Recursive", false);
    foreach (XmlNode RootChildren in Navigation.GetListKeywords(filter).SelectNodes("//*[@IsRoot='true']"))
    {
        Keyword RootKeyword = engine.GetObject(RootChildren.Attributes["ID"].Value) as Keyword;
        log.Debug("Generating XML for keyword " + RootKeyword.Title);
        w.WriteStartElement("Item");
        w.WriteAttributeString("ID", RootKeyword.Id);
        w.WriteAttributeString("Title", RootKeyword.Title);
        if (RootKeyword.Metadata != null)
        {
            WriteKeywordMeta(RootKeyword, w);
        }
        WriteChildrenXml(RootKeyword.Id.ToString(), w);
        w.WriteEndElement();
    }
    w.WriteEndElement();
    w.WriteEndDocument();
    w.Flush();
    w.Close();

    package.PushItem(Package.OutputName, package.CreateStringItem(ContentType.Xml, Encoding.UTF8.GetString(ms.ToArray())));
}

And the recursive "WriteChildrenXml" method would look more or less like this:

private void WriteChildrenXml(String RootKeywordId, XmlWriter writer)
{
    Keyword ParentKeyword = _engine.GetObject(RootKeywordId) as Keyword;
    Filter filter = new Filter();
    filter.Conditions.Add("Recursive", false);

    foreach (Keyword ChildKeyword in ParentKeyword.GetChildKeywords(filter))
    {
        writer.WriteStartElement("Item");
        writer.WriteAttributeString("ID", ChildKeyword.Id);
        writer.WriteAttributeString("Title", ChildKeyword.Title);
        if (ChildKeyword.Metadata != null)
        {
            WriteKeywordMeta(ChildKeyword, writer);
        }
        WriteChildrenXml(ChildKeyword.Id.ToString(), writer);
        writer.WriteEndElement();
    }
}

Nuno

SDL Tridion MVP Awards - online now

It's official, it's out there, and here's the location statistics:

Excluding SDL:
Netherlands: 2
UK: 2
US: 1

Including SDL:
Netherlands: 4
UK: 3
US: 3

Check it out, you may know someone from that list: http://www.sdltridionworld.com/community/mvp_award/mvp-award-2010.aspx

PS - Before people start shouting that giving the award to SDL employees is wrong, I think it's worth re-reading the MVP Program rules at http://www.sdltridionworld.com/community/mvp_award/index.aspx, specifically the part about the MVP Selection Panel. This panel is NOT controlled by SDL and most of the nominations received were for internal SDL people (and many didn't get the award).
PPS - If you use the forum, you will certainly know that the SDL MVPs really do deserve it.

Tridion User Group?

I've been toying with the idea of organizing a Tridion user group in NYC/North East Corridor for a while, but just didn't find those 60 minutes needed to get into it. There are quite enough professionals working with Tridion in this area of the world to make it sustainable, however I don't have the faintest clue into what would be needed from an organization stand point.

And to put it bluntly, I really don't have the time to bother with it.

But I still want to do it.

So, let's itemize what we need to make it happen, shall we?

#1 User community - it does exist, and I can get everyone's contact via SDL Customer Support
#2 A reason to meet - Meeting for a couple of drinks with no other excuse used to be a valid reason in the past, but I think that in today's my-company-is-nimbler-than-that-and-all-I-do-contributes-to-the-bottom-line economic environment we will need more than this. So, first hurdle to go through: get speakers/presenters.
#3 A venue - This being NYC, it's rather easy to get us a bar/lounge where we can meet. If we want to do more corporate, it's going to cost a pretty penny. It becomes hurdle nr 2, thus.
#4 Corporate Sponsorship? #3 is really a candidate for Corporate Sponsorship... but then #2 becomes even more important, since with sponsorship you really must provide value

So, ideas thrown in the air. I would like to see it happen, but I can't manage it, I don't have the time for it.

Anyone willing to pick it up? I'll help where I can, and can certainly get some buy-in from SDL (Marketing, PS, Support, etc)

SDL Tridion MVP Awards

A few months ago our CEO asked me to launch a MVP initiative, intending to raise awareness about our community, and giving this people a way to differentiate themselves from the rest of the "general population" of Tridion consultants.

Thinking this was a great idea, I put my hands into it and about a month later we had a budget, a process and a web page. We wanted to be different from other MVP programs out there, so we decided that rather than having someone within Tridion selecting the MVPs, we would instead have a "Community Selection Panel" to do this. Idea is simple: 5 people, nominations by email, come January 1st just yay or nay per candidate.

And that's when you realize that not everything is black or white.

Anyway, all this to say that the Community Selection Panel has reached an agreement today about who is getting Tridion MVP status for 2010, and there's only 10 of them. Hopefully others will realize that this could be valuable for them too and start contributing more actively to the SDL Tridion community.

Many people think that this is an award for knowledge, where the most knowledgeable people get the award. It's not the case at all. This is an award for those that - for no financial reason - share their knowledge freely and openly to the whole community.

More to follow.