Loading ...At the 2009 PubCon in Las Vegas, Mike McDonald talks with Ecordia’s Sean Jackson discusses some of the strategy and challenges involved with SEO for sites with massive amounts of content.
Ecordia had recently been working with PRNewswire, a site with over 5 million pages in the Google index. Optimizing a site with that kind of content volume can certainly be a daunting task.
Sean talks about the importance of coming into the project very early in the process. When dealing with sites that have this kind of volume the optimization project touches so many elements of the website. You have to be in sync with teams developing the site’s architecture, you have to understand the processes in place for the content management systems and how various other web teams within the organization will be managing both the creation of new content and the management of existing content.
Sean described one of the most primary considerations being that of looking at the site in terms of logical compartmentalized chunks. Sitemaps are limited to 50,000 URLs… so how do you handle a site with 5 million links? The solution is simple enough in concept, but requires a good deal of forethought and planning.
From our experience with a 2 million indexed pages website i want to add a few more tips for the important guidelines in handling such a website :
1.URL structure is one of the most delicated and dangerous components when handling big websites, we always have to make sure that if the URL changes we will use a 301 redirect when we move to the new address structure to make sure the content is transfered with all of its power.
2. internal link structures are possibly the greatest challenge in big websites.
as we go on and try to amke sure all areas of the site are reachable to the crawlers, cutting down the depth of levels it takes to reach the deepest content is important.
some strategies are combining the classic HTML sitemap, only in sites this big, HTML sitemaps wont do because the page wont be able to contain the ammount of links.
3. you should use an external crawler and crawl the entire site to make your own conclusions about duplicate content issues, URL looping and other errors.
youll be suprised to find out how many unsolved issues are founf the the google webmastertools HTML suggestions wont be able to track but still may be damaging the ability of the site to fully obtain its maximum value on the search engine results.
Interesting video, but he didn’t really say too much about what they decided to actually do.
Its interesting and very good information for me as i am also managing a news site with more than 100 thousands pages, I will make a new site map to get it better…
I am planning to also build a sitemap to get my mega websites indexed fast. Good post.