2008-02-24
1. I keep a large collection of text data as a list in memory (lappend x {text...} etc)
2. I want to search this data.
Here's what happens.
set match [lsearch -regexp $x {needle}]
-> memory usage of the tcl process more than doubles (before: 80MB, after: 200MB)
foreach k $x { if {[regexp -nocase {needle} $k]} {puts "match"}
- >ditto
foreach k $x { if {[regexp -nocase {needle} [list $k]]} {puts "match"}
-> Heureka! total memory usage stays at 80MB.
I'm still not quite sure what's going on, it's about keeping lists 'pure' I guess. I'm now consulting these pages:
6am EDIT: I think I almost get it now. regexp treats $x as string and forces every element into a string representation AS WELL as a list representation. sigh. lsearch is forcing a string representation of all elements. seems unavoidable. I could also lappend items as strings not lists, but I'm saving my data as TCL source for various reasons, and of course { } is much cleaner in that case.
-hans