Those of you testing out the initial release of EveKnows likely noticed the slow response times; the problem was not the web server but the SQL server. To keep costs low I was using a shared server but five to ten seconds to generate results for a query on only a few thousand galleries was unacceptable. This afternoon I purchased a dedicated machine and found a few ways to optimize the queries, resulting in searching in the half-second range. I imagine there is still room for improvement, but this should do for the testing period.
Last night I integrated records of incoming links into EveKnows. The spider now tracks the text of links to galleries (or alt tags in the case of images), but with reducing weighting and a character limit. This makes keyword spamming far less productive, while giving galleries without much embedded text more accurate rankings. Sadly, spamming seems rampant in the porn industry, perhaps more so than in other Internet subcultures. I’m attempting to program the spider so as to detect natural English phrases and rate them higher than lists of keywords. Feedback on the quality of search results is most welcome, as well as ideas on how to combat the constant problem of gallery spamming.
