New Feature CORE-135
1 vote

Experiment with using a Trie to intern URIs to reduce memory usage

Created by Rob Vesse on 10/14/2011 5:46 PM Last Updated by Rob Vesse on 1/4/2012 6:04 PM
%
 (hrs)
Logged: 0  (hrs)

 Description

A Trie is a really neat data structure for mapping sequences of characters (or bits in the general case) to values and compresses data since it is a tree structure by nature.  Since URIs have many common characters e.g. http:// it should be relatively efficient to use a Trie to may raw strings to .Net Uri instances and thus cut down memory usage significantly - we hope!

 

There is a good implementation at http://code.google.com/p/typocalypse/source/browse/#hg%2FTrie which could be adapted and incorporated into dotNetRDF

    Rob Vesse (Wednesday, January 04, 2012 6:04 PM) #

Added the remaining code changes to the Trie implementation necessary to allow interned URIs to be cleared

    Rob Vesse (Monday, December 05, 2011 12:33 AM) #

Looks pretty good so far, working solidly in SVN revision 1996 for those who wish to test out the memory usage reductions, note later revisions of the 0.6.0 branch may be unusable due to ongoing refactoring work related to other issues

    Rob Vesse (Friday, December 02, 2011 7:51 AM) #

Began initial work on this in the 0.6.0 branch as of SVN revision 1993