Deep Jive Interests

So, I got an email about about 4 days ago now saying that Crazy Egg was finally open to those who had signed up many many months ago (that includes yours truly). Crazy Egg has been given the preliminary look over at TechCrunch a few months ago, and reading about heat maps, quite frankly got me stoked. I am always interested in statistics and analytics. After all, if you don’t know what your visitors are doing, you’ll never be able to improve your blog.

The final release to the public, however, I feel has been less than thrilling.

How It Works:
Crazy Egg is a remotely hosted analytics solution. You embed a piece of javascript to your site, and it starts crunching visitor data for you. Their free option allows for 10 000 free visits per month, and unlimited testing sessions. What’s a testing session, you ask? Well, Crazy Egg has structured itself so that it centers around testing, which, in and of itself is a worthy goal. One of the best things about e-commerce is that you don’t have to guess about your marketing and conversion techniques. Long copy or short copy? Does color matter? How about placement of the ads? Adequate testing with an adequate sample size should answer all of your questions. With the right software a great deal of information can be gleaned quickly and cheaply.

And that’s something that should be remembered. It is not a ‘total’ statistics or analytics package in the same way that Google Analytics or Mint or Wp-Slimstat is. You won’t be able to tell where your visitors are coming from, how long they’re staying on your site, a time line of when they’re arriving. You won’t be able to tell which pages they’re going to either. If you need a more complete package then you’ll have to look elsewhere.

After you set up a “session”, a given time that you’re measuring or a number of visits, and a URL of a site that you’re measuring, you’re ready to go. Crazy Egg starts tracking pretty much right away after that. To check your results, you log back on to the website, and hit the “view results” button.

The results gives you a few options. You can see where people are clicking, principally. The “overlay” option gives you a schematic overlay of where clicks are happening, by putting up color coded “plus” signs where links are. Different colors gives you an idea of how many clicks were done, but as well, clicking on the “plus” sign gives you a nice graphical representation of what percentage of clicks were done on that particular link.

The heat map and list are the other options. The list allows you to see in descending order, the links that were clicked and how many clicks you got. The heat map is the option where you there is a graphical representation of where people literally clicked on your page.

The good:

1. Its easy enough to implement: From a wordpress point of view, you just add the piece of javascript to the theme of your choice and presto-chango! Its done.
2. Its cool: Checking out its vaunted heat map overlay your own site is cool. You can see where people are literally clicking on your site. And the way that the percentage of clicks is represented by a changing blob of color is also pretty cool.
3. Its simple and focused: testing is the only thing this cat does and the interface is very clean and uncluttered.

The bad:

1. Limited stats: Did I mention how limited the stats were? Over a given testing period, sure, you can see which links people clicked. But you can’t tell when during the day those clicks were made. You also can’t tell the sequence in which people have clicked which links (if they’ve done that). If you’re testing ad placement, would it be userful to know WHEN people were clicking? Probably. Maybe there’s a difference in AM vs. PM ad placement. Or, after a major story hits the news, and you got a flood of traffic — vs. your normal traffic. You’d never know.
2. No real split-testing: Ideally, if you’re considering a test, you ought to be able to split the page that over a specified period of time half of the people see one page, while the others see another after they’ve both arrived at the “same” page. The way that Crazy Egg has arranged this thus far is that you can test things sequentially only. Put up one version of the page. Test. Put up another version of the page. Test. Statistically this isn’t as robust, and all sorts of biases can challenge the veracity of your results.
3. Poor correlation with other testing packages: So over 4 days, CrazyEgg says I’ve had about 150 visitors. Well, that doesn’t so far agree with Wp-SlimStat, or Google Analytics. In fact, its probably off by a factor of 4-5. That is, I’ve probably had on average 200 uniques per day. CrazyEgg doesn’t really disclose how they measure their uniques, so I don’t know how accurate their numbers are. But this could be the most serious problem about CrazyEgg.

My own first impressions of it were some what underwhelmed, but that’s probably because I was expecting more of a full featured statistics package. Crazy Egg clearly isn’t that piece of software, and doesn’t bill itself as such. Its testing software, really.

It is difficult to know whether or not CrazyEgg will be successful in enticing heavy-weight clients or not; the stats they do offer, such as they are, are lightweight to say the least. The other canard, of course, is the numbers themselves. So far, they’ve under estimated the amount of traffic significantly on this site. Since I don’t know how they crunch their numbers, it is hard to know what sort of fudge factor I should/ could be using.

At the end of the day, I’d have to say that the hype outweighs the hump for CrazyEgg. I’ll agree there aren’t that many testing packages quite as slick or pleasing to the eye, but its dubious numbers and limited statistics create a lightweight package that is best for hobbyist or small business blogs.

Other Resources
SolutionWatch seems to like CrazyEgg as a tool for usability
VibeTechnology, however, agrees that CrazyEgg may be more flash than substance