I've noticed in the logs for the blog that search engines are trying to access pages with "?advanced_search=1" in the URL. E.g., I've seen a lot of entries similar to the following:
5.10.83.52 - - [12/Mar/2014:00:32:23 -0400] "GET /blog/blosxom/<a%20href=/<a%20h ref=/<a%20href=/2008/05/01/2008/03/2008/05/05/network/email/clients/outlook/2008 /10/network/email/sendmail/2008/07/network/email/clients/outlook/2008/05/25/2008 /12/2008/05/18/2008/05/03/index.html?advanced_search=1 HTTP/1.1" 200 12080 "-" " Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)"
They seem to be getting erroneous URLs reflecting a directory structure
related to dates that doesn't exist on the system. The URLs appear to be
related to the find plugin, since its search option includes code for
"advanced_search=1", so I've edited the Perl code for that plugin to
include rel="nofollow" at the end of the URL generated for
the advanced search capability.
The orignal code was:
<a href="$blosxom::url/$path_withflavour?advanced_search=1">Advanced Search</a>
The line is now:
<a href="$blosxom::url/$path_withflavour?advanced_search=1" rel="nofollow">Advanced Search</a>
Adding rel="nofollow" to a URL tells search engines, such as
Google's search engine not to follow any link that includes the nofollow
parameter.
The following meta tag can be included in the head section of the HTML code for a page to tell search engines not to follow any links on a page.
<meta name="robots" content="nofollow">
But there may be instances, such as this case for me, where a webpage designer wants only some links on a page not to be followed to their destination by search engines.
The attribute can also be added to individual links if you don't want to vouch for the content of the page to which the link points. E.g., adding it to links placed in comments by those commenting on a page will allow visitors to go to the linked page, but search engines that adhere to the nofollow parameter won't use the link to increase their ranking of the page to which the link points, which may discourage some comment spammers.
The rel="nofollow" option for links was developed as a
way to combat link
spam. In January 2005, Google, Yahoo! and MSN announced that they would
support use of the "nofollow" tag as a way to deter link spam. Microsoft's
MSN Spaces and
Google's Blogger
blogging services joined the effort to utilize the tag to discourage
link spamming At that time a number of blog software providers, including
Six Apart,
WordPress,
Blosxom, and blojsom, also
joined the effort by supporting use of the tag.
References:
