Updated 2012-12-23 17:54:31 by AMG

See http://www.purl.org/tcl/home/man/tcl8.5/TclCmd/scan.htm for the formal man page for the Tcl command scan.
scan string format ?varName ...?

If varNames are supplied, will store values parsed (from string according to format) in the named variables and return the number of variables written to. If no varNames are present, returns the list of values parsed.

One important use is to possibly strip off leading zeroes from decimals:
 scan $possiblyZeroedDecimal %d cleanInteger

Context: Leading zeroes induce octal parsing in the expr command and in if and similar commands that invoke expr. Use of scan to remove leading zeros can avoid this often-reported problem.

One very frequently-asked question is how one converts between display and numeric format for ASCII characters. Scan provides the usual answer:
    foreach character {a A b B c} {
        scan $character %c numeric
        puts "ASCII character '$numeric' displays as '$character'."

This makes the table
    ASCII character '97' displays as 'a'.
    ASCII character '65' displays as 'A'.
    ASCII character '98' displays as 'b'.
    ASCII character '66' displays as 'B'.
    ASCII character '99' displays as 'c'.

In more recent Tcl's (8.3 or so up) you also can call scan inline:
 set numeric [scan $character %c]

And the %c format is not limited to ASCII, but can handle any Unicode. (RS)

LV Can someone explain what this means - calling scan inline? Where is scan getting its input in this case? From stdin?

Lars H: It refers to the result of scan, not its input. Rather than setting some variables, it will return a list of the parsed values. (RS's example is rather ugly, since it really sets $numeric to a list with one element, but as quick character code look-up at a prompt, a scan char %c is hard to beat.)

Alastair Davies: When using in-line scan, note that the list that is returned will contain blank elements for format specifiers that are not satisfied. If there is only one format specifier, a list with one blank element is returned, and note especially that this is not the same as an empty string. For example, scan foo %d will return {}. Now, noting the statement on the man page:
 In the inline case, an empty string is returned when the end of the input string
 is reached before any conversions have been performed.

you might think that scan foo %d would return an empty string. But Don Porter carefully explains this special case (quoting from c.l.t.)

Back to the example:
 % scan foo %d

We're going to scan the string "foo", trying to parse a decimal integer from it, and when we're done, we're going to return the results as an inline list, since we provided no variables for field value storage.

While scanning, we see "f", and "f" can't be part of a decimal integer. In the non inline case, we would assign no value to the variable associated with this field. In the inline format, an empty string is stored in the corresponding list location to represent that situation.

Now we've exhausted the format spec string. But note that we haven't reached "the end of the input string." We're still sitting at the "f". So the "underflow" case where we run out of input before we run out of format spec does not apply, and we should not expect scan to return the empty string.

To see the underflow case in action, consider these examples:
 % scan a ab
 % scan {} %d
 % scan {   } %d

In all these cases we run out of string to parse before we run out of format spec string to guide our parsing. There are more cases where this happens, but they're difficult to construct. Consult the sources and the test suite for details and more examples.

Part of the reason this does tend to be confusing is that the detection of the "underflow" case and returning the special magic empty value when it is detected is just about 100% worthless in Tcl. It's a feature of sscanf() in C that was apparently slavishly copied over with full consideration of whether it had any value in Tcl. (A common problem in built-in commands and features from Tcl's earliest days. Octal, anyone?)

scan's rules are arcane and weird, but they are what they are.

To match specified characters use %[chars], to match un-specified characters use %[^chars]:
% scan 12a34 {%d%[abc]%d} x - y
% list $x $y
12 34

Notice that the above scan's format string says "some decimal number, followed by either an a, or a b or a c, followed by another decimal number". Then, the last 3 arguments are x, the name of the variable to be assigned the first decimal number, - , the name of the variable to be assigned the letter, and y, the name of the second decimal number. The name - was chosen with the intent to convey "oh, this value isn't going to be used".

Also see "Dump a file in hex and ASCII", as well as "u2x" in the "Bag of algorithms".

Binary has a scan which complements use of this scan.

AMG: Even a simple numeric scan always results in shimmering.
% set var [expr 123]; list
% tcl::unsupported::representation $var
value is a int with a refcount of 2, object pointer at 0317BDF8,
internal representation 0000007B:0317BD68, no string representation
% scan $var %d; list
% tcl::unsupported::representation $var
value is a int with a refcount of 2, object pointer at 0317BDF8,
internal representation 0000007B:0317BD68, string representation "123"

I'd think the single numeric conversions could be optimized to recognize when the input already has numeric representation, then simply return the input.