E-business Briefing: September 2008

Lou Rosenfeld is the founder of Rosenfeld Media, a publishing house focused on user experience design, and the co-author of Information Architecture for the World Wide Web.

He also co-founded the Information Architecture Institute and UXnet, the User Experience Network, and as an information architecture consultant, has helped numerous Fortune 500s and other large, messy, political enterprises make their information easier to find.

Here, we asked Lou a few questions about site search analytics (SSA) - it's importance in site design and development, and how it is being used by different types of businesses online.

In a nutshell, why is it important to analyse site search usage on your website?

Every time someone uses your site's search engine, he is telling you what he wants from you, in his own words - his query. We analyse query data to find interesting patterns and anomalies. These can help us identify and diagnose problems with our site's content, search system, navigation, metadata, and other aspects of its design. Think of SSA as a user research tool that helps all kinds of designers make more informed, data-driven design decisions.

We in the user experience community often tend to be a bit uncomfortable when analysing quantitative data; that's not the sort of thing that we learned in design school. But dipping a toe in SSA helps us understand the 'what' aspects of our site and its users. It also helps us generate better 'why' questions to explore further. So SSA helps to counter-balance the qualitative tilt of many designers' research methodology, and it helps us use our old standby qualitative methods more effectively.

How many visitors typically use site search and how does that vary between sectors?

Search rates are highly variable, and have as much to with users' information needs and the nature of the indexed content as much as it does with sectors. For example, it's probably fair to assume that the information-hungry users of Amazon or your intranet's staff directory search far more than they browse. And that the converse would be true of users who seek social encounters at Facebook or entertainment at game sites.

Some suggest that as many as half of all users are search-dominant - they automatically search before considering other options - but again, it really depends on the nature of the information need and the information itself.

How many sites are using the SSA functionality in their analytics tools, as well as making changes to their business based on that data?

Although many analytics tools and search engines now support at least a minimal degree of SSA functionality - and there are excellent free tools like Google Analytics that offer SSA - few organisations tap their query data. This is due to a variety of fairly predictable reasons; general ignorance about SSA, technical roadblocks that may have been thrown up by IT, political landmines thrown up by managers, and legal barriers due to overly-paranoid legal departments.

But the good news is that more organisations are figuring out that they can benefit from SSA, and not just the e-commerce players that live and die by analytics. We've talked with people at such organisations as the BBC, Vanguard, Reed Elsevier, Netflix, WW Norton, and Sandia Labs who are taking full advantage of what they learn from SSA.

And some have gone beyond using SSA simply as a diagnostic tool. For example, the Financial Times monitors its site's recent search activity for spikes in queries for personal and company names. They check to see if these suddenly popular queries correlate with their recent editorial coverage; if not, a spike might indicate a breaking story, and the paper's editors are informed. So, in effect, the Financial Times is using SSA as a predictive tool.

How should you start off with site search analytics?

Start with some query data and some basic questions. The following questions are generic enough to pertain to just about any situation, and your analytics tool or search engine might already provide you with reports to help answer them:

• What are the most frequent unique queries?
  
• Are frequent queries retrieving quality results?
  
• What are the click-through rates per frequent query?
  
• What are the most frequently clicked results per query?
  
• Which frequent queries retrieve zero results?
  
• What are the referrer pages for frequent queries?
  
• Which queries retrieve popular documents?
  
• What interesting patterns emerge in general?

If you spent an hour a week poking around the data to answer these questions while equipped with even an inexpensive, rudimentary analysis tool (say, Microsoft Excel), you'll start benefiting immediately.

You'll quickly notice problems - like a very frequent query that's not retrieving any results - that you can fix after a little follow-up (for example, do we not have the appropriate content? Is it there but not indexed, tagged, or titled properly?). And by demonstrating to your colleagues the benefits of just a smidgen of SSA, you can go a long way toward getting it permanently established in your organisation.

Can you use site search findings to improve your search marketing strategy?

It's not so obvious. There are many differences between the queries people use to find your site and the ones they use to search within your site. For example, your most frequent web search queries will likely include many variants of your organisation's name; naturally, these are unlikely to occur among your top site search queries.

That said, there probably is a useful connection between web searches and site searches. Here's a theory that I'd like to test: Let's assume that, because searchers have more context, site search queries are more specific than web queries. Let's also assume that as the web becomes a more crowded and competitive arena, bidding for more general keywords becomes more expensive. You'd rather market your content using more specific, less expensive keywords. So, you might find that your site search queries are the ideal place to look for candidates.

If you have a less than optimal site search tool and a tight budget, is it better to invest in a better search product or in SSA?

It's much better to invest in analytics, as it will help you diagnose problems with your current search engine that might not be that difficult or expensive to fix. So you'd avoid ripping out the old and installing the new.

Really, any organisation considering investing in an expensive new search engine (and when is a new search engine not expensive?) should do their analytics homework first. It'll help profile what users need from the search experience, leading to a set of functional requirements that is far more comprehensive and balanced than the ones your IT department will write on their own.

How has Google's launch of site search boxes within its organic results impacted SSA? Is it making site search analysts' job harder?

I'm not sure it's all that common at this point; I encounter it mostly with very large sites. That said, it clearly does channel traffic away from site search engines, thereby reducing the quantity of data we have to play with.

Does that give analysts the right to get mad? I don't think so: As they usually do, Google finds great opportunities to improve its service, its customer loyalty, and its bottom line where others have ignored the search experience. In other words, years of horrible site search implementations are coming home to roost.

What else are you up to at the moment?

For years I've made my living as an information architecture consultant for Fortune 500s and other large organisations. That work has increasingly led me to SSA; I now teach day-long workshops on the topic in five or six cities annually, and I'm also finishing up a book on site search analytics, co-authored by Marko Hurst. Our book will be available in 2009 from my publishing house, Rosenfeld Media, which produces short, practical books on user experience design. 

Many thanks for the opportunity to talk site search analytics, Richard!

The three most redundant search debates
MySpace Music sings a disappointing tune
Happy staff = happy customers. Right?
Technorati sheds light on the business of blogging
Unity in search is crucial to success
CAPTCHA tools
Online reputation management avoided by PR people
The Web Week in Review
Betfair provides a lesson in customer alienation
Q&A: Gareth Williams, CEO of Skyscanner
BusinessWeek's Business Exchange: putting Web 2.0 in its proper place
Dealing with the financial meltdown - tips for businesses
Site review: social shopping with Tribesmart
Blogosophere continues to grow - Technorati
Financial industry woes hit display ad market but there is hope

Send your client-side interview requests to editor@e-consultancy.com.

70,000 internet professionals use E-consultancy. We provide low cost access to heaps of practical advice on internet marketing and e-commerce. Our sole aim is to help you deal with the day to day challenges you face in your role.

We're highly independent and we really understand what it takes do well online, and about business more broadly.

So isn't it about time you or your company signed up?

Get the detail on our subscription packages , training and events.

To start or stop receiving the newsletter, or to change your email settings, please visit: http://www.e-consultancy.com/account/email.asp.

If you have forgotten your log in details please email help@e-consultancy.com with 'Password AWOL' in the subject line.

Telephone: +44 (0)207 681 4052.

***NOTE TO READERS - E-consultancy never sends attachments in automated emails, apart from the initial invoice when you subscribe or renew a subscription. Do not open emails that have attachments unless you are absolutely sure they are from a trusted (non-spoofed) source.