SplitTester - Great Free Tool for Validating Test Results

A co-worker recently shared this free tidbit with me, and it's been so helpful that I wanted to share it here. It's called SplitTester, and it was developed by Brian Teasley and Perry Marshall. The way it works is simple: you enter clicks and click-through rate for two test cells, and it spits back the likelihood of the test having the same long-term results, along with the confidence level.
If you're a stats genius, you probably won't need this tool. But for the rest of us, it's a great way to quickly make sure that you're making statistically significant decisions. Not only can it be used for click-through prediction, but I've used it for conversion rate prediction as well - which is actually more relevant, anyway.

Check it out, and let me know what you think!

More On The Ranking Report Debate

The debate continues on SEM forums and blogs over whether ranking reports serve a useful purpose in SEO or not. Ian McAnerin has a great post in his McAnerin Muse blog in defense (or defence, if you're Canadian like Ian) of ranking reports. He rightly points out that "rankings don't matter - it's visitors and sales. How very convenient. It sounds great. Very forward-thinking and modern. Except it's wrong. Why? Because there is no context."

My sentiments exactly. I absolutely agree, as I've said before, that rankings in and of themselves are unimportant. What good is a top ranking if you're not getting any visitors or conversions from it? And what good is ranking well for a super-long-tail term that no one is searching for?

However, if you're doing SEO for a client, the hope is that you've done your research and identified key high-volume terms for which your client is not currently ranking well and is not getting conversions on that term. Logically, you'll focus SEO efforts on increasing rankings for those terms, with the goal of getting converting traffic.

With skill, and a bit of luck, you succeed - and your client will start getting traffic and conversions. Now, you need to demonstrate that hey, I did my job - you went from ranking #51 to ranking #3 - and look at all the sales you've gotten as a result! But how do you prove that the rankings increased?

You either check every phrase manually (the thought of which gives me a stomachache), or you use a rank checker. And, as Ian points out, "The next step is to blend your rankings and other external data with your analytics." This is exactly what we're doing at our agency - using ranking reports in combination with visitor and conversion data to demonstrate the full effect of a successful SEO effort. Without this context, you can't prove that the improvement is due to SEO. And if a client is paying you to do SEO, you'd better be proving to them that they're getting their money's worth.

So what are the options in the absence of Web Position et al? Are you manually checking rankings? Have you found another tool that works? Comment and let me know!

Are Search Rankings Important?

Ever hear the saying "A day late and a dollar short?" (Silly me, of course you have.) I'm a day late and a post short. I had a post partially drafted on the hot topic of the blocking of rank checking software by Google, but ran out of time to finish it. Well, Stoney deGeyter at Search Engine Guide just saved me the trouble by taking the words (and links to other relevant posts) right out of my virtual mouth with his post, An End To Ranking Reports Is An End to Analysis.

Seriously, it's a great post, and really does sum up what I was going to say on the topic, especially his final sentence: "But to not know where you're ranking for your keywords leaves you without an important tool for assessing how, when, where, and why your sales figures may have suddenly changed." This is why I'm concerned, as our clients have come to depend on us for this information.

An aside: In reading the comments in the article, I see I'm not the only one Stoney beat to the punch! Oh well - good for you, Stoney!

The Disney Marketing Machine

Last week, my family and I made our first (and possibly last) pilgrimage to Walt Disney World in Orlando, FL. It was a great trip and a lot of fun, but the entire time I was struck by what a monster the Disney marketing machine really is.

Anyone who doesn't live under a rock knows that Disney marketing is everywhere - on TV, at the movies, in the toy aisle, etc. But nowhere is it more apparent than at the Disney theme parks.

We started planning our trip relatively late - we began considering it in late April. The first thing I did was send away for their free vacation planning DVD. It's billed as "personalized," but really the only thing that's personalized is the cover letter that accompanies the DVD. Nonetheless, the DVD is brilliant - with information for everyone from small children to adults. Surprisingly, my 11-year-old twins had a lukewarm reaction when we told them we were taking this trip, saying it was for "little kids" - until we showed them the clip on the DVD that's geared to tweens. The clip features 6 tween-age kids touring all the Disney parks and commenting on their experience. It's interesting and well-done, and after watching it, my kids were sold and couldn't wait to go.

Disney's email marketing program is another brilliant point of contact. My DVD included a special link where I could view "personalized" information online, including "special" vacation package rates - so of course I went and checked it out. Email signup was prominent on the web site, so I registered for the mailing list. I wasn't disappointed - during the 3 months leading up to our trip, I received several very relevant emails with helpful links for trip planning. Since I'd told Disney the ages of our children, we received emails with info and links geared to "big kids," and were spared all the toddler stuff that we weren't interested in. I found myself referring to these emails over and over again. A week before we left for our trip, we got an email with all kinds of great info, including a PDF for packing lists and meal reservations, as well as detailed information about our Disney resort, transportation, and theme park tickets. It was great.

The marketing magic really shone at the parks. Since we stayed at a Disney resort, we were surrounded by messages about Disney - in our room, on the Disney buses, everywhere. The parks are absolutely brilliant - we got there just before opening, and the way they open the parks is really exciting, with music and big announcements about the "magic" and all.

Then there are the major attractions. We rode most of the "big" rides at the parks, and while I thought most of the roller coasters were pretty tame, the marketing at the end of the rides is amazing. As you exit the ride, you're plunged into a store filled with themed merchandise relating to the ride. You can get a t-shirt for every ride at the place, and purchase toys and trinkets too. You can't leave the ride without walking past all this stuff, and of course your children will beg for every little thing, saying "that was the best ride ever - can I get a t-shirt?" We were able to resist and say no, but I saw hundreds of guests succumbing to the pressure!

There are a few instances where Disney stumbles, though. Their web site is awful - and I mean awful. I couldn't even get several sections of the site to function at all in IE, and when I emailed their support staff, I received a reply saying that they have "known issues with IE7,"and I should try a different browser. OK, that's fine for me as an SEM, since I'm used to working in multiple browsers. But the average layperson, I'm sure, has no clue what that even means. And hello, how about fixing the "known issues" instead of letting them fester? The site worked better in Firefox, but it's so graphic- and code-heavy that I got tired of waiting for pages to load and finally switched to their "low-bandwidth" site, even though I wasn't using a low-bandwidth connection. There were so many usability issues with their web site that it would take me far too long to detail them here. Suffice it to say that if I, an online marketing professional, found their site frustrating, I'd bet their average visitor gives up pretty quickly - translating into increased customer service costs and probably a lot of folks spending their travel dollars elsewhere.

The other big missed opportunity: no email follow-up after we got home. My entire family was so high on the whole trip experience when we got back - if there had been an email in my inbox with a survey, an invitation to share our experience in online Disney forums, or even to book my next vacation, I would have jumped on it (well, maybe not the "book your next vacation" part, but definitely the survey and stuff!). What better way to grab customer testimonials and feedback than right when they get home from their trip? What better way to keep the social network conversation going? But there's been nothing but crickets from Disney in my inbox since our return. I guess maybe Disney doesn't need any more testimonials?

All in all, it was a fun trip and a great lesson in superior customer experience. Rounding out email and online marketing would really put Disney over the top.