Those of you testing out the initial release of EveKnows likely noticed the slow response times; the problem was not the web server but the SQL server. To keep costs low I was using a shared server but five to ten seconds to generate results for a query on only a few thousand galleries was unacceptable. This afternoon I purchased a dedicated machine and found a few ways to optimize the queries, resulting in searching in the half-second range. I imagine there is still room for improvement, but this should do for the testing period.

Last night I integrated records of incoming links into EveKnows. The spider now tracks the text of links to galleries (or alt tags in the case of images), but with reducing weighting and a character limit. This makes keyword spamming far less productive, while giving galleries without much embedded text more accurate rankings. Sadly, spamming seems rampant in the porn industry, perhaps more so than in other Internet subcultures. I’m attempting to program the spider so as to detect natural English phrases and rate them higher than lists of keywords. Feedback on the quality of search results is most welcome, as well as ideas on how to combat the constant problem of gallery spamming.

After a day of messing around with SQL queries, I’ve finally got a handle on doing a logical AND search against the reverse index. Previously the search terms were OR’ed together, so searching for Liz Vicious would return any gallery that matched the word Liz and then every gallery containing the word Vicious. Turns out that there’s a lot of stuff with the word Vicious in it that has absolutely nothing to do with the sexy goth redhead I was looking for, so I knew the searching algorithm needed work. The trick is in SQL’s HAVING clause; the Eve engine does a COUNT(*) on the returned results, which are grouped by URL. The result of the COUNT(*) function is the number of matching terms; a quick HAVING COUNT(*)=$n_terms line in the SQL SELECT statement cleaned up the mess.

I also made some changes to the spider so that it pulls search terms from incoming links; this should help improve search quality, but the change means the existing gallery database is worthless. I’ve scrapped it and started spidering from scratch. In a day or two I’ll take whatever’s been spidered and move it to the production server at EveKnows.com. Stay tuned for some quality porn searches!

The spider component of EveKnows.com has been progressing swiftly. It’s become very reliable at examining a webpage and determining whether the page is or is not a photo gallery. In the past 48 hours is indexed over 15,000 galleries, which is a decent amount to begin testing the searching and sorting algorithms on.

The user interface has been uploaded to http://eveknows.com, but the database will not be put online till its performance has been tested. You’ll notice that, unlike just about every other search engine on the Web, there is no Advertise link. There will be no paid-placement galleries on EveKnows.com. Rest assured, every search result will be a genuine, organic gallery linked to by popular TGP sites. Feel free to comment on the look of the interface and include any features you’d like to see added to the search engine.