Collation

Richard Suchenwirth 2005-10-04 - Collation is "the logical ordering of character or wide-character strings according to defined precedence rules. These rules identify a collation sequence between the collating elements, and such additional rules that can be used to order strings consisting of multiple collating elements."

Tcl's lsort sorts according to numerical Unicode values, which may not be correct in some locales. For instance, in Portuguese, accented letters should sort as if they weren't, but in Unicode sequence come after "z".

The following oversimplified code takes a map in which collation differences can be listed as {from to from to...}, sorts the mapped items, and retrieves only the original elements:

 proc collatesort {list map} {
    set l2 {}
    foreach e $list {
       lappend l2 [list $e [string map $map $e]]
    }
    set res {}
    foreach e [lsort -index 1 $l2] {lappend res [lindex $e 0]}
    set res
 }

Testing, Portuguese:

 % collatesort {ab ãc ãd ae} {ã a}
 ab ãc ãd ae

Spanish (ll sorts after lz):

 % collatesort {llano luxación leche} {ll lzz}
 leche luxación llano

German (umlauts sorted as if "ä" was "ae"):

 % lsort {Bar Bär Bor}
 Bar Bor Bär
 % collatesort {Bar Bär Bor} {ä ae}
 Bär Bar Bor

jima:To be precise, in what is normally known outside Spain as Spanish language (don't want to mess things up with Catalán or any other tongue spoken there) there is no letter accentuated with the ` character. Our tilde (that is the term we use for the graphical notation of an accent) is ´. Therefore, it should be luxación.

If my precision is somewhat anoyying to anyone please, just delete it from this page. - RS: No, every correction is welcome. Fixed above - thanks!

RS: Even English data may have collation problems - if they contain ff, fi, fl, ffi, ffl ligatures. Then it might help to do

 set sorted [collatesort $input {\uFB00 ff \uFB01 fi \uFB02 fl \uFB03 ffi \uFB04 ffl}]