The Volokh Conspiracy - Site Search:

We'd like to improve out site search feature. Right now, we use a free version of PicoSearch, but that doesn't really work well, since it limits the number of indexed pages and doesn't automatically reindex. We could probably solve those problems by upgrading to a pay version ($250 or $500 per year), and we're willing to pay for a system that works well, especially since we often want to search the site ourselves.

But we're just not sure what's the best solution for our particular setup:

(1) We have our post-May-2004 archives in pages under the http://volokh.com/archives/ directory and our 2002-to-May-2004 archives in pages linked to from http://volokh.com/volokh_archive.html, named things like http://volokh.com/2003_11_16_volokh_archive.html (though I'm sure we could move them to the archives directory if necessary). Each archive, though, has lots of posts.

(2) We have each individual post-May-2004 post in the http://volokh.com/posts/ directory, but together with comments.

(3) (a) We'd prefer to have our search facility find only words in the posts, not the comments (for searches including comments, a Google search for site:volokh.com ... should do the job), but (b) we don't want the search to find words across multiple posts on the same archive page. The trouble is that preferences (a) and (b) seem to be in some tension -- searching through the archive pages satisfies (a) but not (b), and searching through each post page satisfies (b) but not (a). Surely there's some technical way to do both, partly because the comments are clearly delimited from the post text in the HTML. But I'm just not sure which search tools take care of that.

So I'd love to get whatever advice you folks might have. Thanks!

	[Eugene Volokh, October 3, 2008 at 12:46pm] Trackbacks Site Search: We'd like to improve out site search feature. Right now, we use a free version of PicoSearch, but that doesn't really work well, since it limits the number of indexed pages and doesn't automatically reindex. We could probably solve those problems by upgrading to a pay version ($250 or $500 per year), and we're willing to pay for a system that works well, especially since we often want to search the site ourselves. But we're just not sure what's the best solution for our particular setup: (1) We have our post-May-2004 archives in pages under the http://volokh.com/archives/ directory and our 2002-to-May-2004 archives in pages linked to from http://volokh.com/volokh_archive.html, named things like http://volokh.com/2003_11_16_volokh_archive.html (though I'm sure we could move them to the archives directory if necessary). Each archive, though, has lots of posts. (2) We have each individual post-May-2004 post in the http://volokh.com/posts/ directory, but together with comments. (3) (a) We'd prefer to have our search facility find only words in the posts, not the comments (for searches including comments, a Google search for site:volokh.com ... should do the job), but (b) we don't want the search to find words across multiple posts on the same archive page. The trouble is that preferences (a) and (b) seem to be in some tension -- searching through the archive pages satisfies (a) but not (b), and searching through each post page satisfies (b) but not (a). Surely there's some technical way to do both, partly because the comments are clearly delimited from the post text in the HTML. But I'm just not sure which search tools take care of that. So I'd love to get whatever advice you folks might have. Thanks!