Website Content Management System...

Measuring for Effectiveness…

How is your web site doing? This is a tough question to answer but it’s one you should be asking every day. 

• The cost of creating and maintaining an effective web site has been going up steadily since the inception of the World Wide Web and shows no signs of letting up. 
• If your site is not effective your competitors are increasing their competitive advantage. 
• Building a site is no longer a guarantee that you will gain an audience. More importantly, it does not guarantee that the site will add to your profitability. 

With all of this in mind, knowing how your web site is doing is worth the effort. Starting with the most basic statistics, we will work through some of the statistics, ratios, charts and graphs you should keep. There is no one right way to keep these records. There are, however, a few guidelines that do apply to just about every situation. 

• Don’t measure for the sake of measuring. Tracking a specific statistic or trend should lead you to a goal. Set the goals for the site and track the numbers that will help you achieve your goals.
• Don’t measure things that you have no control over. This is a real waste of time and will, as often as not, keep you focused on the wrong things.
• As the goals of your site change the things you measure should also change. Just because you choose a measure now does not mean that you will keep it forever. Change is the only constant. 
• Build a base line. Before you start making massive changes build a base line of measures. This will insure that when you do start making changes you will have a frame of reference to determine the relative effectiveness of those changes.
• Don’t wait forever. You want to have a base line but you don’t want to wait for months to build your base line before you make any changes.
• Building your site is an iterative process. After developing short and long term goals and defining your improvement cycle, expect change. Don’t be afraid of changing your methodology, your content or the entire site… if it makes sense.

Most of this can be boiled down to a single sentence. Set your goals, keep your measures and make changes as needed. Sounds simple but in reality fewer than one in a hundred small business sites use any kind of process for tracking and measuring the success of their site or changes to the site. Many look at their site stats less than once a month. Very few use those numbers to make marketing decisions and fewer use them to determine the success of their marketing decisions. 

After reading this, you are going to be one of the very, very few.

Most (but not all) of the numbers you use to measure the success of your site will come from your web site log files. Log files are generally text files that hold information about every request to your site. In web-speak, everything is a request. When some one enters your URL in their browser, it goes to your site and requests a page located at that address. The HTML that makes up the page makes other requests for images, sound files or other elements that make up the page. Each request is completely separate from all of the other requests. When you look at your log files you will see that each line in the log is a request form some asset on your site; pages and images. This is referred to as a stateless connection. In laymen terms, your web server does not know that the page requested was requested by the same person that made the request for an image on that page. 

To maintain state web sites save little files to the browsers’ computer. These files are called cookies. Cookies are normally no more than a small file with a piece or two of information that allows the web program to know that the same person is requesting a number of different pages. 

There are two types of cookies. Session cookies store a session ID that is checked by some web pages and lets the program know that it is the same person looking at the page. It contains no personal information and can only be read by the site that writes the cookie. Data cookies can contain personal data. Like session cookies, data cookies can only be read by the site that writes it. These cookies generally contain data that helps the web site program provide information that suits the user’s needs. For instance, LetMeShop, our catalog software, writes a user ID to the data cookie that lets the user add items to a shopping cart and come back as much as a month later and still have the items in the shopping cart.

Back to log files. The following is a sample of an entire log entry from a real log file. The at (@) indicates the beginning and end of a log entry. There are tens of thousands of entries like this in your log files. It should be easy to see why just having access to your log files is useless without a log analysis program to sort through all of the information.

@ 2001-12-03 22:17:02 208.25.247.180 - W3SVC1 GET /_admin/ - 302 304 555 0 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT) newscorePartnerID=newscore;+newscoreZPI=newscore;+EZNewsUSERID=ZPI%3Dnewscore%26USERID%3Dwebtrans%26;+OptInEmailUserID=USERID%3D1013%26USEREMAIL%3Dbarryb%40webtransitions%2Ecom%26;+WebTransSupportID=webtrans -@

The first piece of information you see is the date and time the request was made. (2001-12-03 22:17:02)
Next comes the requesting IP address. (208.25.247.180)

The next important piece of information is about the HTTP request (the item being requested) and the requesting browser. (GET /_admin/ - 302 304 555 0 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT))
In this example the last piece of information is info pulled from the site cookie. (newscorePartnerID=newscore;+newscoreZPI=newscore;+EZNewsUSERID=ZPI%3Dnewscore%26USERID%3Dwebtrans%26;+OptInEmailUserID=USERID%3D1013%26USEREMAIL%3Dbarryb%40webtransitions%2Ecom%26;+WebTransSupportID=webtrans)

If you really want to get into the details of the log files, contact your hosting company. While basic log analysis report packages are generally available through every hosting company, most will charge an additional fee for more detailed reports or reports generated more often than once a month. If you are REAL detail oriented, the raw log files should be available for download from your site. You can then use your own analysis program to process the information to your hearts content.

Log analysis programs can be a little difficult to master. To measure the effect of your marketing program you will need to understand at least a little about the statistics. Trying to look at all of the information available is generally overkill. It is also a waste of time you could be spending on more productive activities (like marketing). 

Most hosting companies will provide you with the statistics you need to do most of the basic tracking for your site. There will be some cases, however where the statistics offered by your hosting company will not do. For instance, most statistics programs will show you the top 10 or 20 pages with the most hits. If you are running an ad that is linked to a special page on your site, you will need to know the number of hits to that page regardless of where it falls in the rankings. If you need these types of statistics you will almost certainly need to upgrade the statistics reports you generally get from your hosting company.

The image below is a view of the basic statistics information you might see from your hosting company. The chart at the top gives you a graphic view of visitor sessions for the month of December, 2001. A visitor session is a visit to your site by one person. If the person looks at a hundred pages while they are on your site, it is still counted as a single visitor session.

The numbers under the chart indicate some of the basic statistics of the site through the month of December 2001.

• Hit – a request for any file. If a page has ten images on it a request for that page will result in 11 hits. One for the page and one for each image or other asset on the page.
• Page View – a request for a single page. This is counted as a single view regardless of how many images are on the page.
• Visitor Session - A visitor session is a visit to your site by one person. If the person looks at a hundred pages while they are on your site, it is still counted as a single visitor session.
• Visitor – A person that visits the site.
• Unique Visitors – the number of individuals who visited the site. If a person comes more than once to the site they are still only counted as one unique visitor.

For many sites, this level of reporting is all you will need. However, having your own analysis program and access to your log files will allow you to look at your site statistics based on your specific needs. As an alternative, ask your hosting company if they can supply a more detailed analysis program. Let’s look at some of the more common statistics available through better web analysis programs

Your site statistics probably contains terms that you are not familiar with. Let’s take a look at a few and what they mean.

• Hits: This is a pretty useless statistic. Unless otherwise defined a hit is any call to any resource on your web site. For instance if one of your web pages has 10 images on it and 10 buttons the log file will contain 21 hits for each time that page is loaded. One hit for each resource called. Web servers are stateless. That basically means that each request is an isolated incident. The server does not know that the request for each of the 10 images came from the same request. It only knows that the request was made and it sent the image.
• Page Impressions or Page Hits: These are important hits. Only calls to web pages are recorded here.
• Visitor Sessions: Another important statistic. A visitor session is one visitor to the site regardless of the number of pages he/she looked at. 
• Unique Visitors: This indicates the number of unique visitors to the site. You do need to remember that any visitor to the site is included in the Unique Visitor list. This includes search engine robots that index your site.
• Visitors who visited more than once: This number indicates the number of visitors that have more than one Visitor Session. It means that the visitor came to your site, looked around and left. Then, at a later time (generally more than 20 minutes) came back to your site again.
• Visitors who only visited once: Just as indicated, these visitors only visited your site once within the date ranges of your report. It does not mean that they did not visit you last month.
• Most Requested Pages: This gives you a list of the most visited pages in your site. While this is not generally an indication of preference, it can tell you if a promotion you are running is working. For instance, if you add a page to your site on a specific topic and use a link to this site in your advertising, the number of hits to this page will help determine the success of your advertising.
• Least Requested Pages: Again this is not necessarily an indication of preference; you should look at these pages. If some of your more important information is on these pages you may do well to look at making the links to the pages more obvious.
• Top Entry Pages: Entry pages are the first page a person goes to at the beginning of a Visitor Session. These are important pages because they tell you what pages have been book-marked or where search engines are sending people. Keep an eye on these pages. If you have one page that is particular high, you may consider changing the content to include the information you want everyone to see.
• Visitors by number of visits: This is a good indication of how many people are coming back to your site again and again. Generally you will want to increase the number of visits per person.
• Top Visitors: While this sounds like a great bit of information it is generally not worth much. A lot of the visitors to your site will be cache.something. This is because most of the bigger ISP’s don’t send a surfers request straight to your server. Instead the requests are sent to a “cache” machine that makes the request, pulls the page down and stores it inside of the ISP’s network. This “cache” machine then serves the page to the surfer and other surfers from the same network. 
• Visitor by hour, day, week: While these statistics may be useful for very active sites, most small businesses will not have to worry too much about them. They may be interesting but will not help you much until you are getting hundreds of thousands of hits and are running your own servers.
• Page not found errors: This list gives you a cross section of calls to your web server that did not go through. There are a great number of errors that don’t mean a lot except to the administrator of the web server. This is the one area where it is easy to spot certain types of hackers. The page errors you are interested in are errors calling specific pages and/or images on your site. All of these errors will begin with no slash or a forward slash and will include the names of files in your site. You may also see a number of errors with forward and back slashes and calls to the operating system root. You can disregard most of these unless you are responsible for the security of the web server.
• Top Referring sites: This is a good list to look at. It will show you who is linking to your site and how many people are using the link. The most common referring sites are your own site and No Referrer. The rest will probably be search engines.
• Top Search Engines: This list gets rid of all referrers except search engines and will give you a quick view of which search engines are doing the best for you. 
• Top Keywords: LOOK AT THIS LIST EVERY MONTH. Keywords give you a glimpse into what people are using to find your site. If you are actively looking for more traffic these keywords will help you determine what keywords to register with search engines or use in pay-per-click engines.
• Visiting Spiders: This is a list of the spiders that have visited your site. Spiders are search engines that come to your site and follow each link, indexing the words on each page. If you are actively registering your site you will be able to tell when a specific search engine has spidered your site. This lets you know when to start checking to see how the registration worked.

Now that we know a little more about what statistics are available through the analysis of log files we can start to make some decisions about what measures we want to keep on our site. Some of these measurements can be taken straight from the analysis program. Total visitors or visitor sessions can be pulled straight from the stats program and plotted on a simple bar or line graph.

At this point most people ask me why they need to graph the stats since they already have the numbers. After all, aren’t the numbers what we need to look at? The simple answer is NO. Generally the individual numbers mean much less than the numbers over time. That’s what a graph does for you. It displays numbers over time, giving you an easy to read graphical reference to the numbers. Sure, knowing that you had 10,000 unique visitors is important. Knowing whether the number is increasing or decreasing over time is more important.

Graphing numbers…

Creating a graph can be a real headache. If you have ever tried to create a graph in Excel, you know exactly what I mean. Given time everyone can learn to do it but there is a learning curve. But I’m going to show you a real world way of graphing that requires no sophisticated software and you don’t have to be a computer wizard. 

Use a pencil and a piece of paper.

The geometry of a graph is simple. Before we draw our first graph, however, we need to go over a little terminology so you will understand what is being described.

Axis – An axis is a “side” of a graph. When you think of graphs as two-dimensional objects (there are three and more dimensional graphs but we will stay away from them in the beginning) each has a height and a width. These are the two “axis” of a graph. Each axis indicates a measurement. In the following example the “height” axis holds the number of visitors to a web site. The “width” axis indicates the flow of time.

A quick look at this chart tells us a couple of things. First, we can see that the number of visitors goes up and down. We can also see that, over time, the number is increasing. If your goal is to increase visitors, your plan is working. To create this graph by hand, start with a blank piece of paper and draw a vertical line about an inch from the left side. Now draw a horizontal line across the page about an inch above the bottom of the sheet. 

Now, look at all of the monthly visitor numbers. Choose a starting number for the left axis of your graph of 0 (zero). Choose a top number of 4 times the highest number on your list. Now divide the top number by 10 (you can use any number you choose once you see how it works) and use this as your increment. For instance is your top number is 20,000, you would divide this by 10. 20,000 divided by 10 = 2,000. Your increment for the left “visitors” axis is 2000. That means that starting at 0 you would count by 2,000 up the left “visitors” axis. Make sure that increments are evenly spaced up the sheet.

The bottom axis in this example is easier because we are getting our reports monthly so we simply write in the names (or dates) of the months equally across the bottom. For a chart to give you a good, easily readable indication of the data the increments for both axes must be equally spaced.

You now have a graph. All you have to do is plot the numbers. To do this find the point directly above the month and directly to the right of the number you want to plot and draw a point. Repeat this for each month and then connect the dots with lines. You just created your first graph. Now isn’t that easy.

Now that you have a graph I’ll show you another element of a graph that really helps illustrate what a graph can do for you. This additional piece of information is called an “intervention” line. An intervention line is a point on the time axis (the bottom axis in this case) where a change was made in your processes.

In this example we will assume that the first red line on the left is the “intervention” point where we started our magazine advertising. The red line on the right is when we started our direct mail campaign. As you can see, the addition of intervention lines gives you a quick picture of what happened when those changes were made. Looking at this chart, what can we tell about what happened when we started our magazine advertising? 

Initially, the number of visitors drops but then takes a sudden upturn. This may indicate that the response from the magazine ads took a while to materialize or that they were not effective at all. However, something caused a pretty dramatic jump two months later. Unfortunately the second intervention point comes to play right after the sudden jump so we can’t tell for sure if the magazines resulted in the growth. 

The second intervention line tells us that if the goal of our direct mail campaign was to bring more visitors to the site, it may not have worked as well as we would like. Right after the campaign visits were flat, then fell and only slowly increased. But the increase was not as dramatic as the increase during the first part of the year. This chart pretty much tells us that the things that we are doing to increase visits to the site do not seem to be working any better than the natural growth. 

It’s important to remember that graphs not only tell us what worked, they tell us what did not work. This illustration also tells us that you need to choose the information you are charting very carefully. In this particular case, using a unique URL for our magazine advertising and direct mail campaign would have given us a better understanding of their effectiveness.

For instance if the direct mail campaign went to a www.yourdomain.com and the magazine advertisement went to www.yourdomain.net and your main site is www.yourdomain.org it would be very easy to see how effective each was because we can track exactly how many visitors each generated.

Time to rethink our marketing and to rethink our graphing. 

The whole purpose of tracking is to help us understand what is going on with the site and the effect of our decisions. If this were a real world graph, it would point out a couple of important issues. The first is that our current graphing system is not giving us the information we need. The other is that our marketing doesn’t seem to be working as we planed. We need a different graph to help us see what is really happening.

For more information on how to choose what to measure, see our article “Creating Business Alignment”. This article discusses how to make sure that everything you do in your business (including your web site) is aligned with business goals. It also shows you how to determine what to measure.

The next graph is a double line graph. Both lines are using the numbers to the left but each represents something different. The blue line represents the number of visitors; the purple line represents gross sales. The left intervention line indicates where the opening page of the site was changed to promote a contest. This was intended to increase visitors. The right intervention line indicates when the opening page reverted back to a storefront.

This graph tells us that changing the opening page seemed to have the desired effect of increasing site traffic. It also seems to have had a negative effect on gross sales. After the opening page reverted back to a sales page we see a continued rise in site visitors and in sales. Again, it’s important to choose the correct numbers to track. If you are looking to increase sales, tracking visitors will not tell you what you need to know. It’s possible to get hundreds of thousands of visitors and sell nothing if you are getting the wrong kind of visitors.

Remember that these examples are to illustrate the tracking of site statistics and are not real life situations. They are meant only to give you an idea of how to graph your site stats and what you can do with the information once you have it.

Below is another way to represent the two-line graph. Numbers are plotted on both sides of the chart. The left “height” axis is Visitors, the right “height” axis is Revenue and the bottom “width” axis is Months. All of these are graphs that you can create with a pencil, paper and a ruler. They don’t have to be fancy graphs to do their job.

Well, we’ve spent a good bit of time looking at single and multiple line graphs, let’s take a look at a few graphs you may want to keep for your site.

• Total Visitors – Generally available with all analysis programs, this number indicates the total number of visitors to your site. It includes visitors that come to your site more than once. The raw number will tell you if the traffic to your site is increasing, decreasing or staying the same. A line chart or bar chart is the best way to track this number. In the sample below, the left column of numbers indicates the number of monthly visitors. The bottom line indicates the months. The three red lines indicate changes in your marketing plan. For instance, 1 may be when you started your link exchange program. A graph like this will give you an instant picture of the effect of your marketing on total visitors. This same type of chart can be used for most measures. It is always a good idea to include intervention lines. These lines (on the time axis) indicate when changes were made. They give you a quick reference between changes and their effect.
• Unique Visitors – Another standard measure in most analysis programs, this number indicates the total number of individuals that visited your site. If repeat purchases are a substantial part of your profit, you will also want to track your repeat visitor ratio. The repeat visitor ratio is generated by dividing the total number of visitors by the total number of unique visitors. This number tells you if the number of repeat visitors is going up or down as a percentage of total visitors regardless of how many visitors you have.
• Average time spent in the site – This number gives you the average number of minutes that your visitors stayed within your site. Generally, the longer they are in your site the better. You may also want to track the effect of average time on the site to sales. One method of doing this is to develop a ratio of dollar sales per visitor or some block of visitors (divide total sales by total visitors). Then you can use a double line chart to compare the two numbers. The left column of numbers is the average revenue per visitor (or visitor block – for instance you can use the average revenue per 100 visitors). The right column is the average length of stay on the site. The bottom shows the tracking months. The thick line indicates the average revenue per visitor. The thin line indicates average stay on the site in minutes. A quick look at this chart will tell you if length of stay has any relationship to the average revenue per visitor. If you are aggressively working your site and making changes, this graph will help tell you if the changes you have made positively affect the performance of the site. If the change causes the average stay to go up and the revenue per visitor to go down, you may want to reconsider the changes. For instance, if a change in your order entry system causes people to take five additional minutes to make a purchase it may well increase the average time spent on the site. It may also decrease the average revenue per visitor because some visitors get fed up with the additional time required and leave prior to finishing the order.
• Referrer – A referrer is a site that refers (provided the link) to your site. This can be a search engine or another site. If you are doing a lot of search engine registration work (or paying someone to do it) this is a measure you need to keep. Most registration companies measure their success by “getting your site in the top 10 or 20 results on one or more search engines with one or more keyword phrases”. This sounds pretty good, but don’t let it fool you. Your real reason for paying someone to get your site higher in the rankings is to pull more traffic to the site and probably increase your bottom line.
  Tracking referrers is one way of telling if the search engine registration is working. I suggest using a multi-line graph or a bar chart to represent the referrers. 
  The following graph shows visitors by referrer over a six-month period. The number of hits (per month) is on the left and the number of months on the bottom. Adding intervention points are necessary for this graph to be of any real use.
• Key Words – In the case of a reference from a search engine the keywords used in the search are kept. You want to track which keywords people are using to find your site. In the case of keywords, you want to keep track of all the keywords used and not just the top ones. To track keyword use the best bet would be a bar chart. See the example below.

In addition to these numbers which are available through the log files you also want to track visitors based on the purpose of the site. This list is not so easy to define. Every site is a little different.

If your site is a sales site you may want to track:

• Total sales
• Average sale per purchase
• Average sale per visitor
• Average profit per purchase
• Average profit per visitor

If your site is a customer service/support site you will also want to keep:

• Total number of support/service users
• Total number of unique support/service users
• Total number of support/service calls from other sources (phone, fax, email, etc.)
• Satisfaction level of customers using the site. (Requiring some type of satisfaction survey)

Once you are tracking these (or other) numbers there are a couple of ratios you may also want to keep. One method of determining the amount you can afford to spend to build traffic is the profit per visitor ratio. This is simple to figure. 

Divide the profit you make from one sale by the average number of visitors per sale.

For instance if you sell widgets for $100.00 and you make $50.00 profit on each widget sold. If you sell one widget for every 1000 visitors to your site the maximum you can spend to get a new visitor and still break even is 5 cents.

$50.00 (profit) divided by 1000 (visitors) = $.05

Assuming that you can expect repeat sales from each customer, you can afford to spend up to 5 cents to get a new visitor to your site. If you want 5000 new visitors you can spend up to $250.00 to get them.

Each business will have a different profit per visitor ratio. It is very important for you to understand YOUR profit per visitor before you start spending money on increasing traffic to your site. There is nothing more frustrating than to find out that for every new customer you get you loose more money.

Another interesting measure is the sales by length of stay. This too, is simple to track. You can easily tell if there is a correlation between length of stay on the site and total sales. If there is, work to increase time on the site. Then continue to track to see if the relationship holds. Correlations are wonderful things. When looking for correlations you are really looking for cause and effect. In other words, if A increases then B increases; or A causes B. For example, is visitors goes up sales goes up; If length of stay goes up sales goes down.

When tracking you want to always look for relationships and correlations between your primary goal(s) and your other measures. Some are obvious; move visitors = more sales. If more visitors does not cause more sales there is a problem with the kind of visitors you are getting. Others are not so obvious. But if you can find correlations between numbers you are tracking and your primary goal it will let you know that changing one will cause a change in the other.

So there you have it. We have the numbers, what do we do with them?

The simple answer is make changes. Make a change, measure, adjust, measure, adjust, measure… and do it forever. Specifically it depends upon the goals for your site. Your web site should never be viewed as a cost center. As a cost center, the easiest way to reduce the cost of the site is close it. Cut your hassles, time spent updating it and the cost to keep it updated and online. If you view your site as a cost center, just shut it down and spend your money somewhere else.
Instead, you must view your web site as a tool to help you achieve specific business goals. And each of these goals should be tied directly to your profitability. Each must provide higher customer satisfaction, lower operational costs, improved cash flow or profitability, increased sales, higher productivity, etc. When your web site is properly designed, maintained, marketed and managed, it will be a benefit to you, your company and your customer. If it is not tied directly to particular business goals you will have no incentive to improve it or it’s usefulness.

And measurement is the only real way to determine if your site is meeting the goals set for it. So, jump in there and give it a try.