Anatomy of a Search Engine | Development of the EveKnows.com adult search engine

Jun/07

18

Server Crash

So last weekend the eveknows.com server lost a hard drive. I had a recent backup, so not much was lost. It did, however, set us back about 20,000 porn galleries which had been indexed between the time of the backup and the moment the drive died. While I was getting it back online, I noticed a number of duplicate galleries had crept into the database, so I cleaned those out as well. That knocked us down another 30,000 or so. The good news is that the server is now back online with a faster drive system and no known duplicate galleries. The Caroline spider is back at work and has already brought us up to 270,000 pages of sexy photos and videos. She seems to be averaging about 8-10,000 new galleries each day, so by the end of June we should be nearing the half-million mark. I’m also looking into getting a mirrored RAID system for the drives, to prevent this sort of problem going forward.

One thing that I did lose in the crash was a blog post about a new feature, the EveKnows.com Search Plugin. Firefox 2 and IE 7 users can now add EveKnows to the list of search providers on their browser’s integrated search bar. Pretty cool, right? :)

RSS Feed

No comments yet.

Leave a comment!

«

»

Theme Design by devolux.org