When you diagnose a problem with your website, first make sure you actually have a real problem. The most obvious indication of a problem is a traffic drop. But before you start making modifications and wasting a lot of precious time, look at your analytics to confirm that you have a real problem.
If you do have a real issue, Vanessa Fox suggests that you benchmark. Look at the top ten queries that give you the most traffic and find out what pages rank for them and what the position is. By doing this, you can better determine where the problem is.
To verify a crawling issue, Vanessa advises that you run a script over the server logs to look at search engine bot activity. This step allows you to see how well your site is crawled, how often it is crawled, and also helps to categorize your pages.
For more information on these issues, Vanessa has several checklists listed on Jane and Robot, which include: Discoverability Checklist, Accessibility Checklist, and User Conversion Checklist. Also, Jane and Robot is hosting a Developer Summit Friday, June 12 and will examine these very issues. You can access the registration information here.

Pingback: SEO Checklist with Vanessa Fox - News: Everything-e
Pingback: SEO Checklist with Vanessa Fox : virtual gambling
Is that so?
Great video. Thanks.
Good points! On a similar note, checkout and add your comments to:
http://www.metropoliscreative.com/2009/06/avoid-website-embarrasment.html
would something in a website hinder google bot of crawling it even if it knows about?
So say you submit a sitemap but even though google know about it, it will not crawl or index that page because you are using certain scripts there?
The reason I ask is because my website shoptimesaver.com has been getting crawled and indexed on Brand Pages but only Brand Pages that do not use the ajax script for our left navigation.
Your help is greatly appreciated.
Thanks Mike !
Pingback: seo-blog-traffic-tips
Pingback: SEO Checklist with Vanessa Fox | Internet Marketing Superstar
Pingback: SEO Checklist with Vanessa Fox | GoSocial.us : Building Brands in Social Media and Online Marketing
“The most obvious indication of a problem is a traffic drop.”
Of course, this requires some traffic to begin with. Diagnosing issues with new sites (Why isn’t it performing?) or old sites that long ago lost their visitors follows the same path as diagnosing active sites, by using multivariate or simple A/B testing, detailed logging and skilled analysis. These have almost nothing to do with what is discussed in the video.
This interview (Google’s Vanessa Fox or JaneAndRobot’s Vanessa Fox … conflict of interest, anyone?) discusses only spider and indexing issues. The title of the article would be better if it reflected that (i.e. “Diagnosing Negative Search Result Changes”)
Please, Web “Pro” News, remember that there are web “pros” out there who want more than superficial “get a ranking report” and “set up keyword tracking” advice, and who are far more knowledgeable about the nuances of search than you are giving them credit for. Where’s the beef?
Thanks for the great info, particularly the resource page!
Pingback: Google Changes to No-Follow on the Horizon? | Search Engine Optimization News, Website Promotion Tips
Very informative article. I hv experienced google webmaster tools and found really helpful to see if the pages are crawled by google bot.
Pingback: SEO Checklist with Vanessa Fox | Blog Free Now
I think sometimes, sites can just be boring. Just a lack of excitement and plenty of boredom. For instance this site is flooded with video, which adds to the excitement and the professionalism at the same time. I wish I could find more quality information about marketing with video and how it’s most effective on websites. For starters I’ve signed up for a free acount at http://www.adwido.com
They specialize in video marketing and it’s free.
Pingback: Google Changes to No-Follow on the Horizon? | Blog Free Now
Micheal. I think you have answered your own question, most defiantly menus can make or break a website, even if the bots have found the page before, it doesn’t mean they will find them again regularly.
We just had a site redesign, and the search engines give us errors on any page that ends with / a back slash. I have searched, and can’t find if this is a problem. My apologies if this is not the right place. An example, is this page: http://www.generousgems.com/p/nautical-jewelry/shells-sand-dollars-and-star-fish/sand-dollars/
When we do crawl errors, most pages that end that way seem to have errors. Any suggestions?
I have found when changing shortcuts on pages- like http://www.ezhangdoor.com/benefits-for-homeowners-how-to-hang-and-install-doors can effect rankings, if the page is no longer availible.
Good, relevant information. Thanks to Vanesssa Fox from Google for helping us out in understanding how to best do a rescue mission when the site is dissappearing from the search engines. Jane and Robot is a great ressource.
No doubt i will be working on all the useful tips Vanessa has spoken about, among many things she talked about, it seems that creating several sitemaps for different categories might be quite useful especially when you have a large site.
Fantastic infor that will give me some interest in the start of the SEO process. Liked the sitemap info. great.
Pingback: SEO Checklist with Vanessa Fox | Blog Tips, blogging Blog
What a gem! It’s a real shame more people haven’t heard about this site, it covered what I needed this morning
Pingback: Top Ten WebProNews Videos of 2009 - Top Affiliate Programs Review
Great post. There is associated with good data right, although Used to want to allow you know something — I am running Mac pc Os the actual latest beta associated with Firefox, and the design associated with the blog is kind of funky for me. I can read the articles, nevertheless the navigation doesn’t function therefore great.
Alton are you serious in this website? Or maybe i misunderstood something?
Thanks for sharing these kind of wonderful threads. In addition, the perfect travel and medical insurance approach can often relieve those issues that come with vacationing abroad. The medical crisis can quickly become extremely expensive and that’s guaranteed to quickly put a financial problem on the family’s finances. Putting in place the best travel insurance package prior to leaving is well worth the time and effort. Thanks
I am running Mac pc Os the actual latest beta associated with Firefox,great!Your help is greatly appreciated.
I have been browsing online greater than 3 hours lately, but I never found any fascinating article like yours. It is pretty value enough for me. Personally, if all webmasters and bloggers made good content material as you probably did, the internet might be much more useful than ever before.