When you diagnose a problem with your website, first make sure you actually have a real problem. The most obvious indication of a problem is a traffic drop. But before you start making modifications and wasting a lot of precious time, look at your analytics to confirm that you have a real problem.
If you do have a real issue, Vanessa Fox suggests that you benchmark. Look at the top ten queries that give you the most traffic and find out what pages rank for them and what the position is. By doing this, you can better determine where the problem is.
To verify a crawling issue, Vanessa advises that you run a script over the server logs to look at search engine bot activity. This step allows you to see how well your site is crawled, how often it is crawled, and also helps to categorize your pages.
For more information on these issues, Vanessa has several checklists listed on Jane and Robot, which include: Discoverability Checklist, Accessibility Checklist, and User Conversion Checklist. Also, Jane and Robot is hosting a Developer Summit Friday, June 12 and will examine these very issues. You can access the registration information here.
iEntry 10th Anniversary
RSS
Feedback

RSS
Bookmark
Twitter
Facebook
Digg

Stumble
Del.icio.us
Reddit
Furl
Google
Yahoo!


(6 votes, average: 3.83 out of 5)
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]
Fantastic infor that will give me some interest in the start of the SEO process. Liked the sitemap info. great.
No doubt i will be working on all the useful tips Vanessa has spoken about, among many things she talked about, it seems that creating several sitemaps for different categories might be quite useful especially when you have a large site.
Good, relevant information. Thanks to Vanesssa Fox from Google for helping us out in understanding how to best do a rescue mission when the site is dissappearing from the search engines. Jane and Robot is a great ressource.
I have found when changing shortcuts on pages- like http://www.ezhangdoor.com/benefits-for-homeowners-how-to-hang-and-install-doors can effect rankings, if the page is no longer availible.
We just had a site redesign, and the search engines give us errors on any page that ends with / a back slash. I have searched, and can’t find if this is a problem. My apologies if this is not the right place. An example, is this page: http://www.generousgems.com/p/nautical-jewelry/shells-sand-dollars-and-star-fish/sand-dollars/
When we do crawl errors, most pages that end that way seem to have errors. Any suggestions?
Micheal. I think you have answered your own question, most defiantly menus can make or break a website, even if the bots have found the page before, it doesn’t mean they will find them again regularly.
[...] by the search engine accepted and approved concept of Crawl Efficiency (see the Vanessa Fox video or article for more on that). Search engines aren’t going to spend forever crawling and [...]
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]
I think sometimes, sites can just be boring. Just a lack of excitement and plenty of boredom. For instance this site is flooded with video, which adds to the excitement and the professionalism at the same time. I wish I could find more quality information about marketing with video and how it’s most effective on websites. For starters I’ve signed up for a free acount at http://www.adwido.com
They specialize in video marketing and it’s free.
Very informative article. I hv experienced google webmaster tools and found really helpful to see if the pages are crawled by google bot.
[...] by the search engine accepted and approved concept of Crawl Efficiency (see the Vanessa Fox video or article for more on that). Search engines aren’t going to spend forever crawling and [...]
Thanks for the great info, particularly the resource page!
“The most obvious indication of a problem is a traffic drop.”
Of course, this requires some traffic to begin with. Diagnosing issues with new sites (Why isn’t it performing?) or old sites that long ago lost their visitors follows the same path as diagnosing active sites, by using multivariate or simple A/B testing, detailed logging and skilled analysis. These have almost nothing to do with what is discussed in the video.
This interview (Google’s Vanessa Fox or JaneAndRobot’s Vanessa Fox … conflict of interest, anyone?) discusses only spider and indexing issues. The title of the article would be better if it reflected that (i.e. “Diagnosing Negative Search Result Changes”)
Please, Web “Pro” News, remember that there are web “pros” out there who want more than superficial “get a ranking report” and “set up keyword tracking” advice, and who are far more knowledgeable about the nuances of search than you are giving them credit for. Where’s the beef?
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]
would something in a website hinder google bot of crawling it even if it knows about?
So say you submit a sitemap but even though google know about it, it will not crawl or index that page because you are using certain scripts there?
The reason I ask is because my website shoptimesaver.com has been getting crawled and indexed on Brand Pages but only Brand Pages that do not use the ajax script for our left navigation.
Your help is greatly appreciated.
Thanks Mike !
Good points! On a similar note, checkout and add your comments to:
http://www.metropoliscreative.com/2009/06/avoid-website-embarrasment.html
Great video. Thanks.
Is that so?
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]
[...] to keep in mind in terms of assessing the search viability/index status of a website. You can watch the video obviously to get the entire scoop, but I thought I might bullet point some of the more important [...]