Loading ...Google, Yahoo, and Microsoft announced today a joint effort to help reduce duplicate content. The three major search engines came together to allow users to point out their preferred version of a url. As Matt Cutts explains in this video, this format offers users more control.
Duplicate content has been a challenging issue for a long time. Websites containing a lot of content such as a retail site, could end up with several urls for each page making it difficult for search engines to crawl.
Google gives the following example on the Webmaster Central blog:
<link rel=”canonical” href=”http://www.example.com/product.php?item=swedish-fish” />
Simply place this link tag in the head section of the duplicate content urls.
The tag can only be used on pages within a single site. Both absolute and relative links are acceptable, but the search engines recommend absolute links. Also, links to all urls will be directed to the one preferred url.
For more information, each of the search engines have explanations and examples in their own announcements: Google, Yahoo, and Microsoft.
Also, be sure to look for the rest of our video interview with Matt Cutts that includes his take on Google penalizing Google Japan, top security issues for 2009, and Google’s continued efforts with personalized search. The extended version will be available very soon.
Update: Matt further explains the canonical link tag here.
Excellent – this will make things a whole lot easier to manage & not have to do a lot of checking around, changing robots.txt files, .htaccess etc. Very cool.
This works great for large database web sites. It isn’t relavent for small, simple HTML web sites. The whole point is to simplify the web robot searches and cut down on clutter. I am catching the whole buzz around this. But you know this only important to the kinds of web sites Matt mentioned. People building simple microsites don’t have these worries.
awesome. wonder how long it will be until there is a nifty little wordpress plugin to automate the creation of these tags? what can say.. I’m lazy (it’s why i use wordpress… lol)
Looks like a realy great idea! Simple and effective.
Approximately 2 days .. lol … check out Joost De Valk’s website!
I can see potential problems here for SEO. One example would be if a webmaster chooses what is the most authorative, relevant page. But, the search engines might not like that page, and drop the site in the SERPS. If all pages on a site were viewed as equal, that would work, but unfortunately they’re not. The search engines might give preference to the duplicate url even though it displays the exact content, and when you redirect it to the other url, you might lose some rankings. Supposedly, the pagerank and other properties will be transferred to the preferred page, but we will see.
There will be other issue as well, such as when a product is unavailable, and the visitor is
temporarily redirected to a generic page in that category.
If this new feature is being introduced as a convenience for webmasters, it should be great. But, if it is another do-it-or-die edict from Google, here we go again.
Nifty… I will stick to 301′s and what I am others can see… If somebody ever bails on payment… I will know which tag I can use to fix their rankings…
Nice!
The biggest 3 search engines working in same team! Will see how this tag will work on my 15000+ pages domain ( not domain above), have a lot of problem whit the duplicate content.
Nice, a new standard, will definitely try this out.
At last, a simple solution to handle duplicate issues. The canonical wordpress plug in is pretty cool. Thanks.
Breaking news! – This is way overdue. However, this is designed for pages that have alternate versions, like printable pages etc.
Also this will set into stone where your permalink really is… Fantastic!
What a great Idea!
http://crackzsl.blogspot.com/
During the developmental process of web related technology, techies never envisioned these complicated routes will emerge. Today, we are trying to decipher and outguess the systems we created just a few years ago, which should not be the case. Why? Because almost everything we want to do on the intenet has been ditorted in some way. Do indexing, crawling, redesigning workds, etc. serve any purpose. We have created a jungle with no definite rules to adhere to.
All internet / website related activities should be transparent, simple, truthful, defined, and accessible. Room for outsmarting, fooling, outwitting, juggling, etc. should be minimized primarily because technology should make life easier and not increasingly complicated.
Best regards,
I think this is a good start to what is a rather large problem. As Matt explains there is a lot of “junk” out there, but the junk is usually posted from spammers and internet users who will never have heard of duplicate content or have no interest in trying to help solve the problem.
I do like how the big search engine players have worked together on this one though.
I would totally bone the interview chick
Yeah this is really useful. And moreover I was wondering for a long time while looking out in the analytics reporting pointing out / and /index.html. Now this solution is more useful.
That;s why no wonder the information of tagging in the web is so crowded and intensely duplicated!
this video make me clear more about this Canonical tag, thanks
Great. Solves a ton of problems for webmasters, makes a whole lot of things easier. Long overdue, well done : Just Insert tag and plan your friday night out …. bargain!
thanx for sharing
This sounds great, but has anyone done any testing, yet? I mean, are dup URLs actually being *removed* from indexes by this? How is this going to affect manipulative duplicate content alogos?
Canonicalization issues should be addressed in planning and development and are very easily avoided when you’ve structured your website appropriately. Keyword research, develop, deploy. I just don’t trust this *one* tag (anyone remember metas?) to resolve the issues, entirely; it’s up to programmers to program accordingly. Dynamic 404s and strict URL structuring is an extremely effective, preemptive technique that people aren’t using as it is. What happens when this tag gets abused or deployed incorrectly?
Will this tag actually have any effect on ‘big’ sites that *don’t* implement this technique?
I need to understand the reward and penalty structure of this tag, in direct reference to white hat, and black hat, policies; and what Search Engines have in mind for this consideration.
This will be interesting to watch unfold over the next several months…
Arow
This is just Great as far as eradicating the duplicate content on the web is concerned!
But does this help in how google sees the dynamic urls ?
With a not so adequate knowledge about how google sees the Dynamic URLs as not so google
friendly, I was looking for all the information that is available for changing the dynamic URLs to the Static ones.
I am not sure if this tag saves all the research that I was about to do starting from the .htaccess files to the MOD rewrites for the php applications. OR is this tag really a substitute for that , Anyone – Any comments on that would be much appreciated.
Thanks,
Arun – web developer,
ProximaSystemsIndia.
Excellent Information! Now we have only to test it.. great Post
Matt…I am in the process of implementing it on my sites where there are loads of duplicates and more than 90% of the pages have very poor ranks.
it is a good mesure tu reduce the duplicate content.
great idea! Simple and effective, Thanks for sharing a good information
My fellow on Orkut shared this link with me and I’m not dissapointed that I came to your blog.
Thanks for explaining canonical tags in detail.
Awesome stuff. Will have to read up on some of the technical things but I believe it to be highly valuable!
This really sounds like a great thing. Will dup URL’s be taken away or how does this work? But this must be mainly for larger sites right? I have a few smaller ones and hardly believe it to be relevant there. But if you have a webstore, it’s a must have I believe. Databases and all.
Awesome
Nice to see Google being so pally with Yahoo and Microsoft
What’s with the interviewer pointing the mike at Matt when he’s already wearing a clip-on microphone?!
Hmm, this could be useful, although need to see it in practise and get peoples feedback on it before I build in these links.
However, what about ASP, PHP sites etc which use a single “header” file for all pages, I guess this complicates setting this up!
Google tells us to use a certain code, they even provide us with that code syntax.
They specifically say “use this and you’ll be alright”,
Well I did “do that”, just as Google described, I tried it on Google’s own blog platform: Blogger.com
umm, one Google hand apparently doesn’t know what the other Google hand is doing?
http://advertisingsecretweapon.blogspot.com/2009/08/google-canonical-tag-doesnt-work.html
Some bloggers are not aware that we have to add a space and a forward slash immeadiatly after the closing tag or it wont work!
I’ll bet there are hundreds of frustrated bloggers out there muttering curses to the Google engineer who assumed we can code in PHP or XML
Bloggers claim to fame (and I love it, I have 60 of em) is that you don’t NEED to know how to code
Amazing! Very informative. “Both absolute and relative links are acceptable, but the search engines recommend absolute links.” now we can take the advantage.
Thanks guys, handy tip, now I just have to go through the 1000′s of pages on the site. Maybe someone could come up with automation for Drupal?
Hmmm… I expected as much about the site map influence on weighting but it is good to have confirmation. Now I would like to know Mr. Cutt’s views on what kind of changes on a site warrant resubmission of a site map. I have heard many varying opinions on this.
Nice job on the video discussing Canonical Tags.
A good idea to facility our lives, SEO consultors!!!
http://www.otimizacaoemsites.com.br
very clever sollution. I get angry myself, why I couldnt think it before
I’m author of software product and I’m interested about what will happen with the description of this product on my site (neshkov.com). The same product description will be sent to download-portals (Download.com, Simtel.Net, Softpedia…) via the PAD file.
Excellent info!
This will definitely help!
wow,,, thank you for this information
Very important information! Thank you!
Although it is useful to know this, I agree with Arow that canonicalization issues should be addressed in the planning stages of the website’s information heiracrchy. More specifically, how the page naming convention of a websites content pages are architected and implemented within the website itself.
I found your blog on google and read a few of your other posts. I just added you to my Google News Reader. Keep up the good work. Look forward to reading more from you in the future.
Thanks for the post! This clears a lot.
Can we now forget the 301 redirect if working with only one domain and it’s sub domains?
I can’t believe it took me this long to learn about canonical links, but now I’m very glad I did. This can be a useful tool for PageRank shaping. Thanks for making it so easy to understand.
Great tool. I need to read more about it though, as I want to use it to resolve duplicate content for the many smaller area notary subpages. I don’t want to have duplicate content cause a problem and I think this may do the trick.
Thanks Matt
Kurt
7/24 San Diego Mobile Notary
If a canonical tag is only a hint than a directive, its not as hot as its made out to be.
That is such an awesome resource that you’re offering and you give it away for free. I enjoy seeing websites that understand the value of providing a chief resource for free. I truly loved reading your post. Thanks!
Thanks Matt, that is reallllly helpful
Duplicate content has been a challenging issue for a long time. Websites containing a lot of content such as a retail site, could end up with several urls for each page making it difficult for search engines to crawl.
I don’t understand just how i discovered this blog because I have been searching information regarding about the problems in Afghanistan, but anyway I’m extremely pleased I came across your post since I had a good 10 minutes reading your thoughts. Keep it up! One final thing, can you advise about various other useful content in regards to the same subject matter?
I would totally bone the interview chick
Hello! I’ve been following your blog for some time now and finally got the courage to go ahead and give you a shout out from Humble Tx! Just wanted to say keep up the fantastic job!
MADENCILIK FORUM SITESI
Good stuff!!. It helps me to understand canonical tags usage and how can we use it. Really I appreciate your whole information. Though when I search about this term I don’t even understand about it. But thanks after ac crossing so many pages I found yours.
Realy good. Thank you for this Information. Greetings, Chris
Seyret, wacth in video
Thanks, very good article. Was useful for me. I’m guessing would be useful to others. Hope to see you write better articles.
Great video. Thbaks…
I like it, thx
Thank you for any other wonderful article. The place else could anyone get that kind of info in such a perfect manner of writing? I’ve a presentation next week, and I’m on the search for such information.
I do believe all the concepts you’ve offered for your post. They are very convincing and will certainly work. Nonetheless, the posts are very brief for newbies. May just you please lengthen them a bit from next time? Thank you for the post.
Unquestionably believe that which you stated. Your favorite justification appeared to be at the internet the simplest factor to take note of. I say to you, I definitely get annoyed at the same time as people consider worries that they plainly do not know about. You controlled to hit the nail upon the highest as smartly as outlined out the whole thing with no need side effect , other people could take a signal. Will likely be again to get more. Thanks
I am really impressed together with your writing abilities and also with the format to your weblog. Is this a paid subject or did you modify it your self? Either way stay up the nice high quality writing, it’s uncommon to look a great blog like this one nowadays..
It is appropriate time to make some plans for the longer term and it is time to be happy. I’ve read this post and if I may just I wish to suggest you some interesting issues or advice. Perhaps you can write next articles referring to this article. I wish to learn more issues approximately it!
You actually make it seem really easy with your presentation but I to find this topic to be actually something that I feel I’d never understand. It seems too complicated and very large for me. I am looking forward in your subsequent submit, I will try to get the hold of it!
Great video, thx
A person necessarily assist to make critically articles I would state. That is the first time I frequented your web page and so far? I amazed with the research you made to create this particular put up extraordinary. Excellent activity!