While troubleshooting a problem with a website using wireshark, I was capturing HTTP traffic. I noticed a connection from 65.98.224.2 with the contents of the first packet received from that address showing the software accessing my support website identifying itself as shown below:
User-Agent: TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html)
Checking the URL listed, I found the following:
Chances are that you are reading this because you found a reference to this web page from your web server logs. This reference was left by Turnitin.com's web crawling robot, also known as TurnitinBot. This robot collects content from the Internet for the sole purpose of helping educational institutions prevent plagiarism. In particular, we compare student papers against the content we find on the Internet to see if we can find similarities. For more information on this service, please visit www.turnitin.com
The Wikipedia article on Turnitin states that it is as "an Internet-based plagiarism-detection service created by iParadigms, LLC. Institutions (typically universities and high schools) buy licenses to submit essays to the Turnitin website, which checks the document for plagiarism."
I had read that many schools now use such services to deter students from submitting plagiarized papers. I've seen services offerring pre-written papers for students to submit for classes, so I can see the need for teachers to use such detection services. I didn't realize this service crawled websites to index materials on the web as part of its detection efforts, but it makes sense to me that the service would do so. This is the first time I've noticed this particular web crawler
