repository search seems to be impacted by presence of branches

Looking into it: the algorithm is prefectly correct, as far as it goes. When a branch or tag is created, then the new branch or tag is indexed.

A possible nice feature is if the Node getentries call accepted a filter so the return list could be shorted, but that's in the Subversion supplied python bindings...

The one remaining optimisation is to make re-indexing new tags/branches as cheap as creating them in subversion.

Unless creating from working copies, a branch will be identical to the previous location, so all that needs to be done is to duplicate the index references for the original location for the new location.

This assumes a lot about the storage of the index, and what APIs are available from the svn wrappers, but it would be great if tagging didn't end up penalising the repository search.

I noticed this through having a dozen or so branches/tags of several thousand files, and the indexing process was using a lot of memory when I tried the first index. I was actually worried about the process running out of address space!

If this enhancement were possible, this would make the algorithm about as optimal as I can see, from the perspective of repository queries and file indexes.

Thanks for the info, I hadn't really thought about it in detail but it makes sense. I'll have a look at the API and see if there is a more efficient way of handling copies.

(In [1881]) Not for the faint of heart, but if anybody wants to test the pyndexter branch it would be most useful to get some feedback.

You will need the refactoring branch of pyndexter, available from here.

More info in the README.

References #362, #371, #385 and #388.

I was having some pretty severe issues with the repository search when I added some tagged releases. I am trying out the pyndexter branch, I've got hype set up and my logs show the plugin is happily indexing my repository, however I get a 500 error when the searches go through. I've enabled debug logging in both Apache and Trac but can't get more information than this:

[Mon Mar 05 17:56:12 2007] [error] [client 127.0.0.1] malformed header from script. Bad header=test: trac.cgi, referer: http://localhost/trac.cgi/seas/search

Happy to debug further, if you can advise how to proceed.

Hi: I'm more and more convinced that to get a user-proof solution, any indexer that plugged in to the svn API would need to have some cheap way of identifying when a revision of a file is a "copy with history" and simply duplicate the search entries, or in a re-structured database, simply add links to the index entries from that file and revision.

There are several issues that exist with the brute: one is currently the ability of any user who branches and tags enough to blow the available memory or the cgi execution time-out by supplying an (apparently) huge amount of files to index.

The other is that the way Subversion uses deltas, even if the memory and time budget were not exhausted, you can see that there are file accesses to all the previous revisions contributing to a given revision, so it's very easy to generate a huge amount of file reads.

The only generalised solution I can see is to make the costs to the user and the indexer scale the same way for "copy with history" - this is going to be a challenge...

This plugin seems interesting but with this issue it;s very unlikely people will use it.