Stephen Foskett, Pack Rat

Measuring the Importance of Google’s First Page

Posted on 15. Jan, 2009 by Stephen in Personal

Google's first page accounts for more than two thirds of my web site traffic!

I’m not blogging to get traffic; I’m blogging because I have something to say. But, being a curious person, I do measure my blog traffic, and I’ve become interested in how the search engine optimization gurus ply their trade. So a recent tip on using a custom Google Analytics filter to determine “front page” placement on that search engine piqued my curiosity.

After just one week of monitoring, the results speak for themselves: Google’s first page of results accounts for more than two thirds of my web site traffic, and less than a quarter of my visits come from a source other than Google! It should come as no surprise that the search giant’s first page has come to dominate Internet traffic, but it was amazing for me to see it demonstrated so graphically using data from my own blog.

Since the first of the year, I’ve noticed that my traffic has been up by about 50%, too. I was puzzled, since the uptick comes across all topics and posts. My friend Louis Gray suggests that Google might be responsible for this as well: Google updated their Page Rank results, potentially moving many of my pages into the first page of results.

Of course, I also recently switched to a full-text RSS feed, so perhaps fewer people are visiting my site to read longer articles. But my examination of direct and referring traffic in Google Analytics is inconclusive on this matter.

Using Google Analytics

Google is notoriously cagey on releasing information about their ranking system. They don’t want those SEO experts gaming the system to get their pages on top. Of course this hasn’t stopped anyone, but it can be difficult to determine where exactly one’s site is ranked.

The best information I’ve found is in Google’s Webmaster Tools. Here, one can view the top search queries for one’s own (verified) site, including the absolute position for both impressions (views in the search results) and traffic generated. Although this information is useful, it is not as quantitative as one might desire: I can see that a search for “iphone exchange setup” puts my iPhone ActiveSync setup page on top, but I can’t compare this to Yahoo or direct visits.

Google’s Analytics service is another great resource, with excellent recording and reporting of web site metrics over time. But the company seems reluctant to link this information to their search data, perhaps worried that they’ll make life too easy for those who are trying to game the system to get more traffic.

The Google Analytics SEO Filter

The SEO filter proposed by André Scholten certainly is clever: Although Google doesn’t include absolute ranking data in Analytics, their own URLs do include an indication of which page of results a user was viewing when they clicked to visit a site. When one is not satisfied with the results on the first page of Google’s search, they click “Next” and are taken to another page of results. Google tracks which page to display using an element in the URL called “start”. For example, the second page of results is called by including “&start=10″ in the URL, while the 10th page includes “&start=90″.

Since Google Analytics normally records the full referring URL, we can use an advanced feature called filtering to tag Google referrals with an indication of which page the user was looking at when they clicked. André doesn’t include the whole recipe, but it’s not difficult for a novice to implement:

1. Get Google Analytics up and running for your site. I’m not going to go into how to do this here.
2. In the main Analytics page listing Website Profiles for your user, click “+ Add new profile” in the gray bar. It is important that anything you do is done in a new profile, since this is a filter, and will remove (filter out) things that don’t match.
3. Select “Add a Profile for an existing domain” (since you already set this domain up in step 1) and give it a Profile Name like “blog SEO”.
4. Click Finish, and now we have a profile to work with without messing up our main Analytics results.
5. Find this new profile in the Website Profiles list and click “Edit” under the “Actions” heading.
6. The third box is “Filters” – this is where the magic happens. We will create three filters to only collect information on Google searches, and once you create them you can re-apply them in other profiles for other sites without reinventing the wheel.
7. Click “+ Add Filter” and add a new one called “Ranking 1″. “Filter Type” will be “Custom filter”, and you will select “Include” under this. Then use “Campaign Source” as the “Filter Field” and enter “Google” as the “Filter Pattern”. This will filter out any traffic that didn’t come from a Google referral.
8. Click “+ Add Filter” and add a new one called “Ranking 2″. “Filter Type” will be “Custom filter”, and you will select “Include” under this. Then use “Campaign Medium” as the “Filter Field” and enter “organic” as the “Filter Pattern”. This will filter out any paid traffic from Google ads.


The magic Google Analytics SEO filter

9. Now comes the big one. Click “+ Add Filter” and add a new one called “Ranking 3″. “Filter Type” will be “Custom filter”, and you will select “Advanced” under this.
10. For “Field A -> Extract A”, select “Referral” and enter “(\?|&)q=([^&]*)” in the box (no quotes!) This matches the key/value sets in the URL.
11. Next, for “Field B -> Extract B”, select “Referral” again and enter “(\?|&)start=([^&]*)” in the box (no quotes again!) This extracts the value for the “start” key, if present.
12. Finally, for “Output To -> Constructor”, select “User Defined” and enter “$A2 (page: $B2)” in the box (think you include quotes? Think again!) This adds a custom tag with the full URL and the value of the “start” key.
13. Finish it off by checking “Yes” for “Field A Required”, “No” for “Field B Required”, and “Yes” for “Override Output Field”.
14. Now give it some time – at least a day for Google to start adding data to this profile. Assuming you did everything correctly, you shouldn’t have any trouble with the statistics recording in your main profile.

Watching the Results

Assuming it worked, you can now add the correct chart to Analytics in the new profile:

Google's first page dominates their referrals

1. Enter the new profile’s Dashboard by clicking “View report”.
2. Select “Traffic Sources” on the left side.
3. Click “google (organic)” in the “Top Traffic Sources” table – it should be the only item there!
4. In the “Dimension” drop-down box, select “User Defined Value”. This is the payoff for all this work!
5. Add this to the dashboard by clicking the “Add to Dashboard” button at the top.
6. Now use your creativity to examine the data! For example, enter “page: 10″ in the “Find User Defined Value” box. These are search terms that showed up on the second page for some reason, so you might try some SEO kung fu to improve their ranking…

You will notice a number of visits coming up as “(not set)”. This means that the Google referral didn’t include the long URL with keys and values, so you don’t know where the user was when they clicked.

With some creativity, you could adapt this filter to grab Yahoo, Microsoft, or other SEO information. But Google is definitely the 800 lb gorilla right now, and page 1 is the target – it accounts for 90 to 95% of Google referrals to my site!

Tags: blogging, Google, Google Analytics, Google Webmaster Tools, Microsoft, SEO, Yahoo, Yoast