- list ?arg arg ...?
What is a list? edit
From a usage point of view, a list is an ordered tuple of values; in other languages this is sometimes known as an array or vector. Every list has a length, which is an integer >=0, and the llength command computes the length of any list for you. Every element of the list is identified by its index into the list, which is an integer >=0 and < the list length. To retrieve the element at a particular index one uses the lindex command. New lists can be constructed using the list command. Elements can be appended to a list stored in a variable using the lappend command. (Hence, Tcl lists are not fixed-length once created, as is the case in some other languages.)From a more technical perspective, a list is just a string with a particular syntax. Therefore, in Tcl, a string is either a valid list or not. The main characteristic of strings that are lists is that literal { and } characters are balanced or escaped, since list commands use them as delimiters around elements. The exact syntax is like that for the words of a Tcl command (see the dodekalogue), but employs only the rules about braces ({ and }), quotes ("), and backslash (\) substitution.(There is an even more technical perspective, taking into account the implementation in C of Tcl 8.x lists as a type of Tcl_Obj, which has the effect that even though Tcl lists semantically behave as a class of strings, they are not necessarily stored as such in computer memory, but rather as C-style vectors of pointers to the individual element values, plus some extra data. An important consequence of this is that llength and lindex are constant time operations — they are as fast for large lists as they are for small lists.)How can I operate on lists? edit
Primarily by using the list-commands, which in Tcl are pretty much the commands that begin with l (with load as the only exception): lappend, lassign, lindex, linsert, list, llength, lrange, lrepeat, lreplace, lreverse, lsearch, lset, and lsort. The concat command is fine too (even though it is seldom needed), and foreach is a very important user of lists. split and join are common creators and consumers respectively of lists, but the number of commands that make use of lists in some way is much larger than this.Since all values (hence also lists) are strings in Tcl, it is always possible to operate on a list with string commands, but doing so is typically a mistake (if not an outright error, then at least inefficient), unless you know what you're doing. The most common mistake is perhaps to compare a list to the empty string to see if it is empty: proc foo {bar args} {
if {$args eq ""} then { # Compares as strings!
set args $::foo::default_for_args
}
# ...
}The alternative is to use llength: proc foo {bar args} {
if {![llength $args]} then { # Is $args empty?
set args $::foo::default_for_args
}
# ...
}Does Tcl treat lists specially? edit
Tcl, as a general language, does not treat a list differently from other strings. This means it is possible to use lists anywhere one can use arbitrary strings: write to a channel with puts, use as key in an array or dict, show to the user in an entry widget, etc.HOWEVER, some Tcl functions are written expecting a string formatted as a list as an argument. If those commands are invoked with a string that is not formatted as a list, unexpected results can be expected. Also, Tcl typically keeps track of an "internal representation" of a list that makes it possible to access list elements in O(1) time (access time does not depend on total list length or position within the list).There are cases in which the C implementation of Tcl treats lists specially, but this is only visible in how fast commands are for suitably prepared data, and must not affect what the results of these commands are. Because of this, there are some common programming idioms which handle lists in not so intuitive ways in order to exploit particular optimisations — perhaps most famouslyeval [list lappend baseList] $listOfElementsToAppendor even
eval [linsert $listOfElementsToAppend 0 lappend baseList]instead of the straightforward
set baseList [concat $baseList $listOfElementsToAppend]— but most of those have become obsolete after the introduction of {*} in Tcl 8.5, and now the recommended way to do the above is
lappend baseList {*}$listOfElementsToAppendWhen do I need to care about the string representation of a list? edit
Not very often, but it happens. The most common cases are:- List constants
- In some cases, a command accepts a list for an argument, but practically everyone writes that list as an explicit brace-delimited word. This is true for e.g. proc (the arguments command is a list; in fact, it is a list of lists), switch (in the braced patterns-and-bodies form), and string map (the map is a list). These are usually not a problem, but you may need to think about the list syntax when special characters (backslash, braces, whitespace, quotes) are involved. Sometimes, an extra layer of braces are required around a list element, but if it is unbalanced with respect to braces then you may need to backslash-escape all special characters in it instead.
- Command construction
- Tcl is a dynamic language which lets you construct Tcl scripts at runtime. If there is some piece of variable text in that script (e.g. a user-supplied filename), then you need to "quote" it to make sure Tcl parses it correctly later on. The right way to quote is almost always to feed the string to quote through list and make use of the result from list, even though it is in a "string" context. See Quoting Hell for more information on this.
When is a string not formatted as a list?
A string is not a list if literal (unescaped) { and } characters are not balanced. At this time, the best way to test a string to see if it is a list would be set a [list "a" "b" "c"] ;# or however you get a string into the variable
if { [catch {llength $a} ] } {
puts "failed"
} else {
puts "succeeded"
}When doing experiments to understand these matters, it is a good idea to always put the strings you want to examine into variables before operating on them, since it is hard to keep track of which quoting is interpreted when the command is parsed and which is part of the actual string representation. If you're unsure, you can always useset ato see exactly what string it is that you try to interpret as a list.
What are examples of strings that are not lists, or that are problematic lists?
set f "ab\{ \{x y"
set example_due_to_Donal {This might be more excitement than you expected: [exit]}
... How do I force conversion of a string into a list?
Quick answer: Use the split command.Long answer: It depends on what you mean by "force conversion". If you want some operation with an arbitrary string as input and a list as output, then split will certainly do, but usually when programming it is also relevant what the operation does with its input. If your question is rather "I got some data from a command that returns a string, and now I need to use it as a list, how do I convert it?" then the answer is that you shouldn't need to do anything — since every list is a string, any string used where a list is expected will transparently be parsed as a list, and no explicit conversion step is needed.In some cases, you may be better using regexp to do the conversion. For example:set wordList [regexp -all -inline {\S+} $myGnarlyString]More topics…
What is the preferred way to create a list from a string gathered from either a user or an input file?DKF proposed this pretty alias:
Is there a way to check whether a list element is another list or a simple string? interp alias {} listify {} regexp -all -inline {\S+}[Bill Paulson] notes that this alias changes all white space to a single space, which might or might not be what you want.Other than that, this listify is effectively a split that throws away all empty elements.The only option one has to be certain is the above test - if the llength fails, then its argument is not a valid list. The argument is always a string, of course.Benny Riefenstahl posted this proc on comp.lang.tcl
How can I tell the difference between a string and a list? proc isalist {string} {
return [expr {0 == [catch {llength $string}]}]
}as his proposed solution to the question.In later versions of tcl, string is list is available.See string is list.Data structures in Tcl cannot rely on distinguishing between strings and single-element lists. If this is important in your application, you will need to include some extra flag information in the data structure to distinguish the two cases.Here is another way of looking at the problem (lindex without an index returns the list argument unchanged):
Is a string created with the curly braces { and } around pieces going to be the same as a string created with list? % lindex a
a
% lindex a 0
a
% lindex [lindex a 0] 0
a
% lindex [lindex [lindex a 0] 0] 0
a
% lindex {a}
a
% lindex {a} 0
a
% lindex [lindex {a} 0] 0
a
% lindex [lindex [lindex {a} 0] 0] 0
a
% lindex {{a}}
{a}
% lindex {{a}} 0
a
% lindex [lindex {{a}} 0] 0
a
% lindex [lindex [lindex {{a}} 0] 0] 0
a
% lindex {{{a}}}
{{a}}
% lindex {{{a}}} 0
{a}
% lindex [lindex {{{a}}} 0] 0
a
% lindex [lindex [lindex {{{a}}} 0] 0] 0
aNo program can tell the difference between the string "a" and the one-element list "a", because the one-element list "a" is the string "a".Dossy 2004feb26: A co-worker yesterday who is new to Tcl discovered something that surprised me -- nested lists in Tcl don't work as I expected in a very specific case: % list [list [list x]]
xUm, when I ask for a list of a list of a list with the single element 'x', I would expect '{{{x}}}' back. However, you just get 'x' back. Thinking about it, I understand why, but it means that Tcl lists alone cannot be used to represent ALL kinds of data structures, as Tcl lists magically collapse when it's a series of nested lists with the terminal list having only a single bareword element that requires no escaping.Lars H: It looks worse than it is. For one thing, it is only the string representation that collapses, not the internal representation, so the above nesting of [list] is not completely pointless. It is furthermore very uncommon (and this is not specific to Tcl) that nesting depth alone has a significance. Either you know the structure of the value, and thereby the intended nesting depth, or the list is some generic "thing", and in that case you anyway need a label specifying what kind of thing it is.[DBaylor]: I think this is actually worse than it looks. I see lots of people trying to learn Tcl and the #1 point of confusion is Dossy's example. But what I really dislike about this behavior is that it hides bugs until specific input is encountered. If you ever mix up your data-types (string vs. list), your code will work fine 99% of the time - until special characters are involved. These bugs are inevitably found the hard way. Oh how I wish [list x] returned {x}.Not if the pieces themselves contain unmatched curly braces:
% list \n
{
}
% list \n\{
\n\{Nor is the string with curly braces around the pieces the only string representing that particular list.Braces and backslashes get added as necessary, so that the lindex command may be used on the result to re-extract the original arguments, and also so that eval may be used to execute the resulting list, with arg1 comprising the command's name and the other args comprising its arguments. List produces slightly different results than concat: concat removes one level of grouping before forming the list, while list works directly from the original arguments. For example, the command list a b {c d e} {f {g h}}will return
a b {c d e} {f {g h}}while concat with the same arguments will return
a b c d e f {g h}(From: Tcl Help)AMG: Here's another [concat] using [list] and {*}: set a {a b c}; set b {d e f}
list $a $b --> {a b c} {d e f}
concat $a $b --> a b c d e f
list {*}$a {*}$b --> a b c d e fI heard a rumor that [concat] produces a string, not a pure list, but this might not be the case. (Somebody please clarify!) The {*} trick should not have this problem.Lars H: concat is indeed defined as an operation on general strings (partly because it is used also by eval, which often operates on strings in a not-quite-list manner). However, when all arguments are pure lists then it can take a shortcut that avoids working with the string representations.AMG, 25 Nov 2006: While writing Directory recursion, I verified that [list] + {*} is faster than [concat].How can one read and write list subsets? edit
LV Question: in Perl, Python, and a number of other languages, one has the ability to read and write to subsets of a list -- slices using an almost array like notation. Is this something that one could simulate without much grief in Tcl?RS But of course - our old friends (with wordier notation)set slice [lrange $list $from $to] set newlist [eval lreplace [list $otherlist] $from $to $slice]The new lset can only replace a single element, but possibly several layers deep in the nesting. For reading access to a slice of a list, check Salt and sugar for how the following is implemented:
set i [$ {a b c d e f g} 2 4] ==> {c d e}How do I flatten a list of lists into just a list? edit
LV: I recently had a need to turn a list which could potentially contain elements which are lists into a list of individual elements. Apparently, this is commonly referred to as flattening the list. [ericm] suggested the following code to solve the problem: proc flatten {args} {
foreach arg $args {puts $arg}
}
set foo [list [list a b] [list c d] [list e f]]
eval eval flatten $foo Benny Riefenstahl: I use the following to remove one level of nesting:
AMG: Here's another way to remove one level of nesting:set flattened [eval concat $nested]RS shudders at eval eval - this really starts to get evil! He prefers
set flattened [join $nested]Benny Riefenstahl: I don't like the implied type conversion, join returns a string after all. But you are right, in practice it should work and it is simpler.CMP: Not only is it simpler; eval concat fails for lists that have separators with newlines, which is often the case for long hand-coded lists:
% eval concat {a
b}
ambiguous command name "b": binary breakconcat claims that concat is treating its arguments as lists. I wonder in what way...If you don't like the fact that join returns a string (that also is a list), use this or something similar: foreach e $list {foreach ee $e {lappend flatList $ee}}LV After reading CMP's comment, I just tried this:% set l1 [list a
b]
ambiguous command name "b": binary break
% set l1 [list a \
b]
a b
% concat $l1
a b
% eval concat $l1
a b
% eval concat {a \
b}
a bSo it appears that the problem you were seeing isn't concat - it is the tclsh command reader, expecting the newline to be quoted when you type it.Lars H: No, CMP is right — newlines in the arguments end the concat command, and start a new one. This is due neither to concat nor to the tclsh prompt loop, but to the nature of eval. If you don't want to worry about typing multiline commands at the prompt, then use substitution to put the newline in there:% eval concat "a \n b" ambiguous command name "b": bell binary bind bindtags break button
concat {*}$nestedIt can be applied multiple times: proc flatten {data} {concat {*}$data}
set a {{a {b c}} {d {e f}}} ; # {a {b c}} {d {e f}}
flatten $a ; # a {b c} d {e f}
flatten [flatten $a] ; # a b c d e fThere shouldn't be any type shimmering involved. Right?RS 2004-02-26 - If you really want to flatten a list of any depth, i.e. remove all grouping, I think this way is simplest (and robust):
proc flatten list {string map {\{ "" \} ""} $list}
% flatten {a {b {c d {e f {g h}}}}}
a b c d e f g hLars H: No, that won't work. Consider % flatten [list \{ \}]
\ \I'd give you that it isn't exactly clear what should happen to lists with such elements, but the above doesn't get a single character right. - RS admits he was thinking of well-behaved lists (as built with list and lappend, where braces are only generated mark-up, not content :^) You're right to point out that flatten is not chimpanzee-proof, and robust, enough.[cyrilroux]: 2010-10-28 - May be here is the solution to solve the escape issue ? This always consist in replacing all { and } but NOT \{ and \} (escaped ones). That is to say 6 cases : "^{" " {" "{{+" "}$" "} " "}}+" proc lflatten { list } {
regsub -all {^\{| \{|\{\{+|\}$|\} |\}\}+} $list { } flatten
return $flatten
}
% set foo {{a {b c}} {d\{d\{ {e f} g}}
% lflatten $foo
a b c d{d{ e f g CMP: In general, string manipulation on lists all have the same problem; they do not consider the list structure (unless copying the complete implementation of the existing list commands). Hence, all list manipulations should be done using existing list commands.Information about struct::list - extended list operations edit
Tcllib now contains a struct::list module. Its documentation can be found at http://tcllib.sourceforge.net/doc/struct_list.html . dgp offers this example of making use of it: % package require struct 1.3
1.3
% namespace eval my {
namespace import ::struct::list
set l [::list 1 2 3]
puts [list reverse $l]
}
3 2 1
More explanation…In the above example, is the use of ::list in the set command vs list in the puts command merely an example of different ways of referring to portions, or does one refer to the original tcl list versus the imported new list command?MG I believe ::list in the above refers to the Tcl list command, and list refers to the imported ::struct::list (which is then also ::my::list, as it's imported). Or at least I think that's right. Namespaces used like that require a lot more sleep than I've had lately ;)
What would a Tcl version of the list command look like? edit
AMG: Is this an acceptable implementation of [list]?
proc list {args} {return $args}Looks right to me... - RS To me too. That's because args is already a list, as by the Tcl parser... I'd just write proc list args {set args}AMG: Chuckle.proc list args [list set args]
Transform a list into a list of fixed size lists edit
LV Jonathan Bromley wrote, on May 30, 2008, in comp.lang.tcl, the following little tcl proc for turning a long list into a list of lists (in response to a user's query)
proc split_list {L {n 50}} {
incr n 0; # thanks to RS for this cool "is it an int" check!
set result {}
set limit [expr {[llength $L] - $n}]
for {set p 0} {$p <= $limit} {incr p $n} {
lappend result [lrange $L $p [expr {$p+$n-1}]]
}
return $result
} [arg] Just the code I wanted (needed to split results from SQLite). Thanks. But can I suggest a few tweaks:-- name changed to "partitionlist", I think it's less ambiguous than "split_list". Any ideas for a better name?
- change default from 50 to 2, splitting into pairs looks a more useful default.
- change setting/comparing "limit" so that all the elements of the original list are copied, the original version acted as if the orignal list were truncated to a multiple of the requested sublist length. This version will output any "extra" elements as another short sublist.
proc partitionlist {L {n 2}} {
incr n 0; # thanks to RS for this cool "is it an int" check!
set result {}
set limit [llength $L]
for {set p 0} {$p < $limit} {incr p $n} {
lappend result [lrange $L $p [expr {$p+$n-1}]]
}
return $result
} Lars H: If the partitioned list is then immediately going to be iterated over, it may of course be easier to take advantage of the fact that the variable arguments of foreach are really lists of variables. I.e., instead of foreach row [partitionlist $data 3] {
lassign $row first second third
# Further processing...
}one can just do foreach {first second third} $data {
# Further processing...
}Conversely, this can be used for the following braintwisters' implementation of partitionlist: proc partitionlist {L {n 2}} {
set varlist {}
set body {lappend res [list}
for {} {$n>0} {incr n -1} {
lappend varlist $n
append body { $} $n
}
set res {}
foreach $varlist $L [append body \]]
return $res
}Other related topics edit
caspian recommends that you see Tcl list related commands such as: lappend, lassign, lindex, linsert, llength, lrange, lrepeat, lreplace, lreverse, lsearch, lset, lsort, concat, foreach, join, split, string (and in particular, string is list), apply, exprSome Tcl core commands require lists as argument or return lists - after, array, binary. chan, dde, dict, oo::define, oo::next, encoding, exec, fconfigure, file, glob, [htt], info, interp, library, msgcat, namespace, open, package, packagens, pid, pkgMkIndex, platform, proc, read, oo::refchan, regexp, registry, return, safe, scan, oo::self, socket, switch, tcltest, tm, trace,Making lists with list vs lappend edit
An interesting observation came up on comp.lang.tcl today:
Example 1:
$ tclsh8.6
% set l [list a b c]
a b c
% llength $l
3
% lappend l [list d e f]
a b c {d e f}
% llength $l
4
% lappend m [list a b c]
{a b c}
% llength $m
1
% lappend m [list d e f]
{a b c} {d e f}
% llength $m
2 While a developer might think that creating a variable using the results of list would be the same as creating a variable using the results of lappend, the results are in fact different.On the other hand,$ tclsh8.6 % lappend m a b c a b c % lappend l [list a] a %If the values to lappend are not multi-element lists, then the results will be the same.AMG: What's the difference between a single-element list and the element itself?
Other uses of [list] edit
AMG: When given no arguments, the [list] command returns empty string. I find this useful when entering Tcl commands interactively, e.g. into tkcon or tclsh. When I know that a command will produce a lot of output, such as [read]'ing a whole file, and I don't want to have it all on my screen, I tack "; list" onto the end of my command line.
$ tclsh % set chan [open bigfile] file12dcd00 % set data [read $chan]; list % close $chanIf not for the "; list", the second line would flood my terminal with lots and lots of garbage.AMG again: Another use for list is to pass it a single argument which it will then return. See SCT & RS's comments on the if page for an example usage. However, this only works due to the "problem" noted above by Dossy 2004feb26. Often the argument requires quoting to become a single-element list, in which case list will not return its argument verbatim. On the return page I discuss a few other, safer approaches. The simplest one is to instead use single-argument lindex.
See also: edit
- Sorted Lists
- Concatenating lists
- Listex
- list level
- Additional list functions
- another list comprehension
- Binary trees, Decision trees
- Bit vectors can nicely be implemented with lists
- Chart of existing list functionality
- Chart of proposed list functionality
- Complex data structures for struct: named access to list positions
- Counting Elements in a List
- Internal organization of the list extension
- keyed list
- Power set of a list
- pure list
- Showing sublists in a listbox
- Shuffle a list - Shuffle a list: graph results - Shuffling a list
- Shuffling a list
- Stacks and queues
- Striding a list
- Summing a list
- Use while to iterate over a list
- Using expr on lists
- lshift -Adding Unique Items to Lists of Fixed Length
- Recursive list searching
- listcomp -Compare the Contents of two Lists
- Depth of a list
- list map and list grep
- set operations for Tcl lists
- List Comprehension
- linked lists
- Cartesian product of a list of lists
- list stripping
- string is list
- lpop

