We’ve been working as hard as we can to address this problem, and it has required some re-design of how our search works.
The biggest changes you should see with the new search tool are:
1) Speed. It should be fast now!
2) Search only applies to activities and documents now
3) Productivity and categories are no longer searchable in the same way as actitivies.
4) The current month is searchable already, and we are building the new index going back in time– so every day more time in the past will be added, for historical reporting.
In retrospect, while key words make sense for documents and activities, it doesn’t so much for categories or productivity scores where are known and belong to a short list. You can use a categories or productivity report from the links on the right to see those results, and actually do a search “inside” them. The ability to merge category results or multiple productivities will need to wait for new filter controls, which we are planning to introduce in the future.
Those of you have Custom Reports (which are really saved searches) may need to adjust your key words for best results. Likewise, those of you who use hints in Projects may also need to adjust your keywords.
You can think of the core issue this way: when you go to Google and you do a search, you usually care only about the top results– maybe you drill a few pages to find something. When you use “search” at RescueTime, the task is very different: you are actually doing something like “give me a report of all my time that has words like this in it”.
The key distinction here is that in the first case, you only care about retrieving a few, certainly less than 100, of the all the possible results– and if you want more, you have to make multiple requests back to the server. In our case, for your reports to be accurate with total time results, you always need every possible result to be returned. Search technology for documents is pretty well understood at this point, and there are excellent tools available for use– like Sphinx and Lucene. However, search tools are designed, for practical performance reasons among others, to operate well for the Google model of search results.
When applied to our challenge, however, we have to be much more clever about how are system is designed to allow for the “give me everything that matches” idea to work.