Yeah yeah, I know it is not Friday but in fact Wednesday, but this past weekend was a holiday so I've delayed my little nugget of geekdom until now. In this week's episode we'll discuss the comparison of four different indexers. This paper talks about picking one to package it with the Gnome desktop environment and how do use it for desktop search. So if you're into search engines and linux read on, if not I'll try to have something for a more general audience this coming Friday.
I think the authors of this paper did a good job when they went about testing the four different indexers. Obviously if you are putting an application onto a user's machine things like performance, usability, extensibility, and integratability are all important. The difficulty with testing the different indexers is you have to run the tests on the same hardware and software, you have to use the same data, and the tests have to be exactly the same. On top of that you have to have good tools to measure the things you are looking for which is usually CPU utilization, IO, and Memory used as well as search results accuracy in this case. Once you get all that it can be a painstaking process to run through all of your tests (I know this from running 30 hours worth of tests just like these last week). And usually when you've finished, or thought you've finished, the numbers tell you things that make you want to run even more tests.
Overall I thought their methodologies were solid and their results showed some serious problems with each of the four indexers. But like anything you pick the best tool for the job and go with it. Some new tools that I didn't already know about but will try out for my own testing include Exmap and heap-shot. One tool that I've been using a lot lately which does a pretty good job of creating pretty and colorful charts for the managers is nmon.
Posted by troutm8 at September 5, 2007 10:03 AM