Updated 2014-08-17 16:41:52 by pooryorick

list, a built-in command, creates a list.

Synopsis  edit

list ?arg arg ...?

Summary  edit

Returns a list whose elements are the arguments of the command (in the same order). Returns an empty list if no arguments are specified.

Documentation  edit

man page
generic/tclUtil.c
documentation of the list format
Tip 407
Tip 148

See Also:  edit

Tcl Quoting
Is everything a list?
Unique Element List
Sorted Lists
Concatenating lists
Listex
list level
Additional list functions
another list comprehension
Binary trees
Decision trees
Bit vectors
can nicely be implemented with lists
Chart of existing list functionality
Chart of proposed list functionality
Complex data structures for struct
named access to list positions
Counting Elements in a List
Finding a sublist
Internal organization of the list extension
ldiff
lexpr
lsplit
creates two lists from a single list and a test expression
keyed list
Power set of a list
pure list
Scripted List
write a list using command and variable substitution, without having to escape the newline between the elements, and with the ability to make comments in between elements of the list and comment out some elements of the list
Showing sublists
in a listbox
Shuffle a list
Shuffle a list: graph results
Shuffling a list
Stacks and queues
Striding a list
Summing a list
Use while to iterate over a list
Using expr on lists
lshift -Adding Unique Items to Lists of Fixed Length
Recursive list searching
listcomp -Compare the Contents of two Lists
Depth of a list
list map and list grep
set operations for Tcl lists
List Comprehension
linked lists
Cartesian product of a list of lists
list stripping
string is list
lpop

List Syntax  edit

Syntactically, a list is a script as defined by the Dodekalogue, except that newlines are treated as regular whitespace, rather than as command terminators, and, the semicolon character has no special meaning. Since it is not operationally a script, the following rules rules of script interpretation do not apply to a list:

  • commands
  • Argument expansion
  • variable substitution
  • command substitution
  • backslash substitution
  • Comments

Looking at it from the inverse persepctive, a Tcl script is a well-formed list in which the semicolon character and newlines in word delimiters are additionally interpreted as command separators. If you're wondering whether the list you've attempted to construct manually is well-formed, make sure it looks like a valid command or script.

Enclosing a well-formed list in braces results in a list containing one item, the original list.

The body of a proc, being a Tcl script, is in fact a well-formed list, although this is only checked when the proc is invoked, not when it is created.

The format of an empty list is the empty string.

Standard List Operations  edit

append
lappend and lset
delete
lreplace
extend
concat, lappend, and list
insert
linsert, lreplace
length
llength
prepend
lreplace
search
lsearch
set
lset
retrieve
lassign, lindex and lrange
transform
join, lmap, lrepeat, lsort, lreverse, and split
validate
lappend, string is list

lappend, lassign, and lset are the only 3 list commands which use name of a variable containing a list rather than the list value itself.

Other common tasks:
validate a value as a well-formed list
lappend

Description  edit

list creates a new list, whose elements are each arg. If no args are specified, the list is empty.

A list is an ordered tuple of values. In other languages this is sometimes known as an array or vector. llength returns the length of the list, which is always 0 or greater.

To retrieve the element at a particular position, one uses lindex. The first element is at position 0.

Values can be added to a list using lappend, prepended to the list with linsert, and modified or deleted using lreplace.

A list is a string, and can be used wherever strings are used: as an argument to puts, a key in an array or dict, the value of and entry widget, etc, or as an argument to eval.

All Tcl lists are strings, but not all strings are valid lists. The various list functions expect their arguments to be properly formated. See below for examples of strings which are not valid lists.

One of the keys to succesfully working with lists in Tcl is to understand that although braces ({) are often used to delimit individual items in lists, braces do mean "list" - they are simply an escaping mechanism for some of Tcl's special characters, e.g., whitespace. As DGP put it, "Don't imagine that braces have magic list-ifying powers."

Many simple strings are also lists:
llength hello ;# -> 1

Some strings, however, are not lists:
string is list \" ;# -> 0
string is list \{ ;# -> 0
string is list \{}a ;# -> 0
string is list {{a b} {c}]} ;# -> 0
string is list "ab{ {x y" ;# -> 0

In the previous two examples, the values aren't lists because they violate the Braces and words rules, which state that a braced word is terminated by a matching closed brace, and that words are separated by whitespace

An empty string is an empty list:
set list [list]
llength $list ;# -> 0

set list {}
llength $list ;# -> 0

set list "" 
llength $list ;# -> 0

A string containing nothing but whitespace is also an empty list:
set list { }
llength $list ;# -> 0

set list \n
llength $list ;# -> 0

set list \t\n\t
llength $list ;# -> 0

A string that contains no whitespace is often, but not always a list with one item:
set list hello
llength $list ;# -> 1

set notalist \{hello 
llength $list ;# -> unmatched open brace in list

The following is a list that contains one element, which is an empty string:
set list {{}}
llength $list ;# -> 1

set list [list {}]
llength $list ;# -> 1

Whitespace separates items in a list:
set list "1 2"
llength $list ;# -> 2

set list {1 2}
llength $list ;# -> 2

Here is a list that contains two elements, both of which are empty strings:
set list {{} {}}
llength $list ;# -> 2

List commands parse strings into lists in much the same way Tcl parses strings into commands. In the following example, llength parses the string into a list, interpreting the backlash-space combination as whitepace:
set list {1 \ 2}
llength $list ;# -> 2

It is very common, and perfectly acceptable, to use braces instead of list when writing a list:
set list {one two three}
llength $list ;# -> 3

But in Tcl, braces are simply a means of escaping whitespace and other special characters in strings. They are a nice way to write out lists, but Tcl itself doesn't euqate braces with lists. Any quoting can be used to make a well-formed list, and there are an infinite number of ways to produce a string that is a well-formed list:
set list "one two three"
llength $list ;# -> 3

set list one\ two\ three
llength $list ;# -> 3

set list one\x20two\x20three
llength $list ;# -> 3

set list one\ttwo\tthree\t
llength $list ;# -> 3

#... and on and on ...

When formatting a list as a string, list will escape values where necessary:
list \n\{
;# -> \n\{

Canonical Representation  edit

A list may be represented using various combinations of double quotes, braces, and backslash. The following will return the canonical representation of $somelist:
list {*}$somelist

Using Braces to Write Lists  edit

In source code, braces are often used to write lists. One example is the arguments to proc:
proc move {item speed args} ...

It would be a bit awkward, and a bit slower, to use list for that:
proc move [list item speed args] ...

Likewise, literal braced strings are used with switch (in the braced patterns-and-bodies form), and string map (the map is a list). These braced strings are usually not a problem, but you may need to think about the list syntax when special characters (backslash, braces, whitespace, quotes) are involved. Sometimes, an extra layer of braces are required around a list element, but if it is unbalanced with respect to braces then you may need to backslash-escape all special characters in it instead.

Braces are also used to escape the body of a proc, but in that case, the body is not parsed as a list, but as a script (unescaped newlines take on special meaning):
proc myproc {} {
    this
    is not
    parsed as a list,
    but as a script
}

Generating Code  edit

Tcl is a dynamic language which allows for the construction of Tcl scripts at runtime. Since each command is a list, and a script is a sequence of lists, list operations are often used to build up commands and scripts that can then be interpreted using eval, apply and friends. When substituting some piece of information into the dynamically-generated script, it is usually necessary to subtitute in a list, even if the context requires a single value. string map is can be used for this purpose.

KBK writes on comp.lang.tcl (with some modifications):

If a string has been built up using the list commands, for instance list and lappend, then it is always well-formed as a command. The first element of the list (element number zero) will be the command name and the remaining elements will be what the command sees as its parameters. This method, in fact, is one of only a very few ways to construct command strings that have the desired parameters when one or more of the parameters contains user-supplied data, possibly including nasties like backslashes or unbalanced braces.

In particular, using double-quotes and the string commands such as append and subst is NOT safe for constructing command strings with arbitrary data.

Moreover, eval and its friends give you special support for the technique of using lists as commands. If a command being evaluated is a "pure" list, that is, one that was constructed using the list commands and has never acquired a string representation, then the evaluator is able to short-circuit the parsing process, knowing that all arguments have been substituted. It doesn't need to do the (fairly expensive) scan for $- g- and \- substitution, nor balance "" and {}, but can go directly to looking up the command name and invoking it.

Using List to Concatenate Lists  edit

list can be used with {*} to the same effect as concat:
set a {a b c}; set b {d e f}
list $a $b                    ;# -> {a b c} {d e f}
concat $a $b                  ;# -> a b c d e f
list {*}$a {*}$b              ;# -> a b c d e f

See Concatenating lists for a timing comparison of the various methods.

Concatenating Lists  edit

The following three methods for concatenating list are roughly equivalent in performance:
set list hello
concat $list $list
list {*}$list {*}$list
lappend list {*}$list

The difference is that concat does not make sure its arguments are valid lists, and lappend modifies $list

Before the advent of the {*} operator, the following syntax was used:
eval [list lappend baseList] $extraList

Newline-delimited Lists  edit

When writing a list of lists to a file, it's useful to represent it using the newline character to separate the words of the lists. Here's how to do that:
foreach list $tosave {
        puts $chan \{
        foreach item $list {
                puts $chan [list $item]
        }
        puts $chan \}
        puts $chan {}
}

The result is a list of lists, having the same length as $tosave.

Internal Structured Representation  edit

Internally, Tcl tracks the structure of the list, and the various list commands take advantage of this to improve performance to O(1) time (access time does not depend on the total list length or position within the list). A string representation of a list is not made until is is needed. Therefore, performing string operaions on a large list can incur a dramatic performance penalty, if a string representation has to be generated. The rule of of thumb is to use list-aware commands for lists, and avoid string commands.

concat is a string operation, but is smart enough not to incur the penalty if all its arguments are pure lists.

The internal representation of a list should be transparent at the script level, but for the curious:

In the C implementation of Tcl 8.x, a list is a type of Tcl_Obj, The elements of a list are stored as C-style vectors of pointers to the individual Tcl_Obj element values, plus some extra data. The consequence of this is that llength and lindex are constant-time operations — they are as fast for large lists as they are for small lists.

concat  edit

concat also operates on lists, but does not require that its arguments be valid lists. foreach operates on lists. split creates lists, and join consumes them. Various other commands also make use of lists.

Since all values, including lists, are strings at the Tcl scripting level in Tcl (but not all strings are lists!), it is possible to use string commands on lists, but performance can suffer and there are usually better ways to accomplish the task. Perhaps the most common example of such a mistake is to compare a list to the empty string to see if it is empty:
#example of bad coding practice
proc foo {bar args} {
    if {$args eq {}} then { #string comparison might force internal Tcl gyrations
        set args $::foo::default_for_args
    }
    # ...
}

The better way is to use llength:
proc foo {bar args} {
    if {![llength $args]} then { # $args is empty
        set args $::foo::default_for_args
    }
    # ...
}

Strings that are not Lists  edit

Because lists are so pervasive in Tcl, one of the common beginner misconceptions is that all values are lists. This is not the case. A string is not a list if literal (unescaped) { and } characters are not balanced. To check whether a string is a list:
string is list $somevariable
catch {llength $somevariable}

When doing experiments to understand lists, it is a good idea to first assign the values in question to variables before operating on them, since it is hard to keep track of which quoting is interpreted when the command is parsed and which is parsed by the command being invoked. This makes it possible to first inspect the value of the string before passing it to a command:
set somevariable 

Layers of Interpretation  edit

In the following example, the value of a_single_backslash is a single backslash:
set a_single_backslash [lindex \\\\ 0]

prior to evaluating the lindex command, Tcl performs backslash substitution on the four backslashes, so that the first argument becomes two backslashes. To convert the first argument to a list, the lindex command then performs backslash on substitution on the two backslashes, resulting in one backslash, which is then assigned to a.

Converting a String to a List  edit

split takes a string and returns a list, but the best choice depends on the task at hand. regexp is often handy:
set wordList [regexp -all -inline {\S+} $myGnarlyString]

DKF proposed this pretty alias:
interp alias {} listify {} regexp -all -inline {\S+}

[Bill Paulson] notes that this alias changes all white space to a single space, which might or might not be what you want.

Other than that, this listify is effectively a split that interprets adjacent delimiters as as a single delimiter rather than interpreting them as delimiting an empty string, like split does.

Validating a List  edit

Various ways to check whether a string is a list:
string is list $some_value
catch {lindex $some_value 0}
catch {llength $some_value}

In later versions of Tcl, string is list is available.

List Vs. List of Lists  edit

escargo 2003-03-16: How can you tell if a value is a string of words or a list of strings of words?

The practical application that I had for this was an error-printing proc. It could be passed a value that might be a single error message or a list of error messages. If it were a single error message, then I could print it on one line; if it were multiple messages, then I wanted to print each on its own line.

So, how could I distinguish between the cases?

I think I eventually made all sources of errors provide a list of errors, even if was always a list of 1 (instead of just the error message string).

But the question always stuck with me? Was there a way I could have easily distinguished between the two? Could I look at the representation and see an opening curly brace if it were a list?

RS: The (outer) curlies are not part of the list - they are added, or parsed away, when needed.

Tcl lists are not fundamentally different from strings, rather, I'd say they are a "view" on strings. Just as 42 can be viewed as string, or integer, it can also be viewed as a one-element list. Except if you introduce your own tagging convention, there is no way of telling whether a list is in reality a string - in the other direction, only strings that cannot be parsed as lists (unbalanced braces, quotes..) cannot be viewed as lists. But for your concrete error-printing problem: if you simplify the interface to "a list of one or more error messages", you can have the desired effect with
puts [join $errormessages \n]

Just make sure that the "one message" case is properly nested, e.g.
errorprint {{This is a one-liner}}
errorprint "{This too, with a $variable reference}" ;# braces in quoted strings allow substitution
errorprint [list "another $variable reference"]
errorprint {{Two messages:} {This is the second}}

Single-item Lists vs non-list value  edit

Many simple string values can also be interpreted as single-element lists. Programs should use additional data or rely on program logic to decide whether a value should be interpreted as a list or a string.

Here is another way of looking at the problem (lindex without an index returns the list argument unchanged):
% lindex a
a
% lindex a 0
a
% lindex [lindex a 0] 0
a
% lindex [lindex [lindex a 0] 0] 0
a
% lindex {a}
a
% lindex {a} 0
a
% lindex [lindex {a} 0] 0
a
% lindex [lindex [lindex {a} 0] 0] 0
a
% lindex {{a}}
{a}
% lindex {{a}} 0
a
% lindex [lindex {{a}} 0] 0
a
% lindex [lindex [lindex {{a}} 0] 0] 0
a
% lindex {{{a}}}
{{a}}
% lindex {{{a}}} 0
{a}
% lindex [lindex {{{a}}} 0] 0
a
% lindex [lindex [lindex {{{a}}} 0] 0] 0
a

No program can tell the difference between the string "a" and the one-element list "a", because the one-element list "a" is the string "a".

Dossy 2004-02-26: A co-worker yesterday who is new to Tcl discovered something that surprised me -- nested lists in Tcl don't work as I expected in a very specific case:
% list [list [list x]]
x

Um, when I ask for a list of a list of a list with the single element 'x', I would expect '{{{x}}}' back. However, you just get 'x' back. Thinking about it, I understand why, but it means that Tcl lists alone cannot be used to represent ALL kinds of data structures, as Tcl lists magically collapse when it's a series of nested lists with the terminal list having only a single bareword element that requires no escaping.

Lars H: It looks worse than it is. For one thing, it is only the string representation that collapses, not the internal representation, so the above nesting of list is not completely pointless. It is furthermore very uncommon (and this is not specific to Tcl) that nesting depth alone has a significance. Either you know the structure of the value, and thereby the intended nesting depth, or the list is some generic "thing", and in that case you anyway need a label specifying what kind of thing it is.

[DBaylor]: I think this is actually worse than it looks. I see lots of people trying to learn Tcl and the #1 point of confusion is Dossy's example. But what I really dislike about this behavior is that it hides bugs until specific input is encountered. If you ever mix up your data-types (string vs. list), your code will work fine 99% of the time - until special characters are involved. These bugs are inevitably found the hard way. Oh how I wish list x] returned {x}.

PYK 2013-10-27: What would lindex x 0 return, then? An error that x is not a list? All commands are lists, and x is potentially a valid command. Therefore, x must be a list. There is some subtlety to Tcl which can take a little time for beginners to wrap their heads around, but this subtlety is Tcl's strength, not its weakness. The gripe about hiding bugs until specific inputs are encountered is a gripe about dynamic languages in general, and not particular at all to Tcl.

Subsetting a List  edit

LV Question: in Perl, Python, and a number of other languages, one has the ability to read and write to subsets of a list -- slices -- using an almost array like notation. Is this something that one could simulate without much grief in Tcl?

RS: But of course - our old friends (with wordier notation)
set slice [lrange $list $from $to]
set newlist [eval lreplace [list $otherlist] $from $to $slice]

lset can only replace a single element, but possibly several layers deep in the nesting. For reading access to a slice of a list, check Salt and sugar for how the following is implemented:
set i [$ {a b c d e f g} 2 4] ==> {c d e}

Flattening a List  edit

To flatten a list:
concat {*}$nested

It can be applied multiple times:
proc flatten {data} {
    concat {*}$data
}
set a {{a {b c}} {d {e f}}}  ; # {a {b c}} {d {e f}}
flatten $a                   ; # a {b c} d {e f}
flatten [flatten $a]         ; # a b c d e f

alternatively:
set flattened [join $mylist]

Another possibility:
foreach e $list {
    foreach ee $e {
        lappend flatList $ee
    }
}

eval is not a good option because it chokes on newlines:
% eval concat {a
b}
ambiguous command name "b": binary break

Newlines in the arguments end the concat command, and start a new one. This is due neither to concat nor to the tclsh prompt loop, but to the nature of eval. If you don't want to worry about typing multiline commands at the prompt, then use substitution to put the newline in there:

To get around that issue, first pass the value through a command that parses strings into valid lists:
eval concat [lrange {a
b
} 0 end]

RS 2004-02-26: If you really want to flatten a list of any depth, i.e. remove all grouping, I think this way is simplest (and robust):
proc flatten list {
    string map {\{ "" \} ""} $list
}
% flatten {a {b {c d {e f {g h}}}}}
a b c d e f g h

Lars H: No, that won't work. Consider
% flatten [list \{ \}]
\ \

I'd give you that it isn't exactly clear what should happen to lists with such elements, but the above doesn't get a single character right.

RS admits he was thinking of well-behaved lists (as built with list and lappend, where braces are only generated mark-up, not content :^) You're right to point out that flatten is not chimpanzee-proof, and robust, enough.

[cyrilroux] 2010-10-28: Maybe here is the solution to solve the escape issue? This always consist in replacing all { and } but NOT \{ and \} (escaped ones). That is to say 6 cases:

"^{" " {" "{{+" "}$" "} " "}}+"
proc lflatten { list } {
    regsub -all {^\{| \{|\{\{+|\}$|\} |\}\}+} $list { } flatten
    return $flatten
}

% set foo {{a {b c}} {d\{d\{ {e f} g}}
% lflatten $foo
a b c  d{d{ e f g 

CMP: In general, string manipulation on lists all have the same problem; they do not consider the list structure (unless copying the complete implementation of the existing list commands). Hence, all list manipulations should be done using existing list commands.

Information about struct::list - extended list operations  edit

Tcllib now contains a struct::list module. Its documentation can be found at http://tcllib.sourceforge.net/doc/struct_list.html.

dgp offers this example of making use of it:
% package require struct 1.3
1.3
% namespace eval my {
    namespace import ::struct::list
    set l [::list 1 2 3]
    puts [list reverse $l]
}
3 2 1

More explanation…

In the above example, is the use of ::list in the set command vs list in the puts command merely an example of different ways of referring to portions, or does one refer to the original tcl list versus the imported new list command?

MG I believe ::list in the above refers to the Tcl list command, and list refers to the imported ::struct::list (which is then also ::my::list, as it's imported). Or at least I think that's right. Namespaces used like that require a lot more sleep than I've had lately ;)

What would a Tcl version of the list command look like?  edit

AMG: Is this an acceptable implementation of list?
proc list {args} {
    return $args
}

Looks right to me...

RS: To me too. That's because args is already a list, as by the Tcl parser... I'd just write
proc list args {set args}

AMG: Chuckle.
proc list args [list set args]

Transform a list into a list of fixed size lists  edit

LV: in response to a question, Jonathan Bromley, 2008-05-30, comp.lang.tcl, wrote the following Tcl proc for turning a long list into a list of lists:
proc split_list {L {n 50}} { 
    incr n 0; # thanks to RS for this cool "is it an int" check! 
    set result {} 
    set limit [expr {[llength $L] - $n}] 
    for {set p 0} {$p <= $limit} {incr p $n} { 
        lappend result [lrange $L $p [expr {$p+$n-1}]] 
    } 
    return $result 
} 

[arg]: Just the code I wanted (needed to split results from SQLite). Thanks. But can I suggest a few tweaks:-

  1. name changed to "partitionlist", I think it's less ambiguous than "split_list". Any ideas for a better name?
  2. change default from 50 to 2, splitting into pairs looks a more useful default.
  3. change setting/comparing "limit" so that all the elements of the original list are copied, the original version acted as if the orignal list were truncated to a multiple of the requested sublist length. This version will output any "extra" elements as another short sublist.
proc partitionlist {L {n 2}} { 
    incr n 0; # thanks to RS for this cool "is it an int" check! 
    set result {} 
    set limit [llength $L] 
    for {set p 0} {$p < $limit} {incr p $n} { 
        lappend result [lrange $L $p [expr {$p+$n-1}]] 
    } 
    return $result 
} 

Lars H: If the partitioned list is then immediately going to be iterated over, it may of course be easier to take advantage of the fact that the variable arguments of foreach are really lists of variables. I.e., instead of
foreach row [partitionlist $data 3] {
    lassign $row first second third
    # Further processing...
}

one can just do
foreach {first second third} $data {
    # Further processing...
}

Conversely, this can be used for the following braintwisters' implementation of partitionlist:
proc partitionlist {L {n 2}} {
    set varlist {}
    set body {lappend res [list}
    for {} {$n>0} {incr n -1} {
        lappend varlist $n
        append body { $} $n
    }
    set res {}
    foreach $varlist $L [append body \]]
    return $res
}

Some Tcl core commands require lists as argument or return lists
after
apply
array
binary
chan
dde
dict
oo::define
[oo::next]
encoding
exec
fconfigure
file
glob
http
info
interp
library
msgcat
namespace
open
package
packagens
pid
pkgMkIndex
platform
proc
read
[oo::refchan]
regexp
registry
return
safe
scan
[oo::self]
socket
switch
tcltest
tm
trace

Other uses of list  edit

AMG: When given no arguments, list returns empty string. I find this useful when entering Tcl commands interactively, e.g. into tkcon or tclsh. When I know that a command will produce a lot of output, such as read'ing a whole file, and I don't want to have it all on my screen, I tack "; list" onto the end of my command line.
$ tclsh
% set chan [open bigfile]
file12dcd00
% set data [read $chan]; list
% close $chan

If not for the ; list, the second line would flood my terminal with lots and lots of garbage.

AMG: Another use for list is to pass it a single argument which it will then return. For an example, see SCT & RS's comments on the page for if. However, this works due to the "problem" noted above by Dossy 2004-02-26. Often the argument requires quoting to become a single-element list, in which case list will not return its argument verbatim. On the return page I discuss a few other, safer approaches. The simplest one is to instead use single-argument lindex.