Archive for June, 2007

EveKnows.com Roadmap

June 19th, 2007

I’m putting together a roadmap of ideas for the future development of EveKnows.com, and I want your help. Please feel free to comment with any feature suggestions. Currently, my thinking is as follows:

  • Continue to build up gallery database: I’m still shooting for 1,000,000+ indexed porn galleries by the end of summer
  • Quality control: I’d like to add an easy method for people to report broken/mislabeled galleries
  • Easier RSS access: There should be an easier way for people to get the RSS links for their favorite searches. Maybe I could also build in an RSS aggregator so that all of a user’s queries could be viewed from a single page.
  • Interface redesign: I’m obviously no graphics artist; the look of EveKnows needs a lot of work.

Well, that’s what I intend to be working on next. Let me know what else you’d like to see!

Server Crash

June 18th, 2007

So last weekend the eveknows.com server lost a hard drive. I had a recent backup, so not much was lost. It did, however, set us back about 20,000 porn galleries which had been indexed between the time of the backup and the moment the drive died. While I was getting it back online, I noticed a number of duplicate galleries had crept into the database, so I cleaned those out as well. That knocked us down another 30,000 or so. The good news is that the server is now back online with a faster drive system and no known duplicate galleries. The Caroline spider is back at work and has already brought us up to 270,000 pages of sexy photos and videos. She seems to be averaging about 8-10,000 new galleries each day, so by the end of June we should be nearing the half-million mark. I’m also looking into getting a mirrored RAID system for the drives, to prevent this sort of problem going forward.

One thing that I did lose in the crash was a blog post about a new feature, the EveKnows.com Search Plugin. Firefox 2 and IE 7 users can now add EveKnows to the list of search providers on their browser’s integrated search bar. Pretty cool, right? :)

mod_perl vs mod_cgi and HTML::Template

June 6th, 2007

Today I migrated EveKnows.com from mod_cgi to mod_perl and switched from using Perl to directly output HTML to using the HTML::Template module to handle formatting. The move to HTML::Template has been a long time in coming–I’m a big fan of separating logic from presentation, and this finally allows me to do that with EveKnows. Editing the HTML was getting messy since it was all embedded in the Perl code, and small changes on my staging server were breaking programming logic. Now the search engine is broken up into a Perl back-end and a simple HTML front-end, which should allow me to easily upgrade the interface as the engine matures.

While I was researching HTML::Template, I came across some benchmarks showing the benefits of using Apache’s mod_perl rather than running regular CGI processes, which is how EveKnows was originally designed. Besides cool advantages such as shared memory caching, mod_perl is generally an order of magnitude faster at rendering an HTML file from a Perl source script. EveKnows.com didn’t get *that* much of a performance improvement from the migration (most of its processing time is spent searching and sorting data), but it did increase from being able to serve ~2 requests/second to ~10/second, which is a pretty hefty increase. For anyone attempting a similar migration, I’d like to point out a couple of issues I ran into.

First, Apache needs to be told to use mod_perl rather than mod_cgi. This means editing your httpd.conf file (or virtual host file if you use those) and adding the following section:

SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
Options ExecCGI FollowSymLinks

Restart Apache and any scripts ending in .cgi will now be processed by mod_perl.

The second problem stemmed from CGI::Simple. For whatever reason, this module does not play nice with mod_perl. I had to switch to the regular CGI module to get things back to normal.

Anyway, once these fixes were in place, EveKnows.com was working faster and cleaner than ever!

EveKnows 1.0 Beta

June 5th, 2007

This week marks the release of EveKnows.com 1.0 Beta. The past three months have seen a crazy pace of development, and I believe the results speak for themselves:

Wicked-fast searching of porn galleries? Check.
Incredibly accurate sex search results? Check.
Custom RSS feeds for every set of search results? Check.
Predictive suggestions when entering search terms? Check.
Suggestions of more popular words for searches with few results? Check.
Integration with TGPs and MGPs for site searching? Check.
An awesome Web 2.0 tag cloud of popular search terms? Check!

Since the site interface seems to be under control, I’m focusing on building up a giant database of porn pictures and movies to search through. Its nearing 200,000 now and growing daily–my goal is to get 1,000,000 searchable galleries in the database in the next two weeks.

Anyway, check it out at http://eveknows.com and share the site with your friends :)

Update in Progress…

June 3rd, 2007

EveKnows.com is currently being upgraded to version 1.0-beta. The site may not be 100% available for the next few hours, but I’ll try to keep it working while the update is in progress.