Updated 2013-02-20 15:13:46 by pooryorick

APW 2007-06-07: TemplaTcl: a Tcl Template engine is an example of a simple templating language where Tcl scripts are embedded into the template

jcw 2007-06-03: But I don't see the point in embedding an escape layer using XML. Here's a visually simpler approach
% if {$a != $b} {
    <h3> abc </h3>
    <li> [expr {$a != 1 ? $var1 : $var2}] </li>
% } else {
    <li> [myobj getResult] </li>
        <tr> Hello there </tr>
% }

What you get is the ability to use Tcl's [subst] where convenient, and an outer-level line-by-line escape for larger amounts of code. Just like in Tcl, where we use both if's and expr's.

The example is contrived and has relatively much flow control, so it doesn't come out as clearly as real-world examples would. The above isn't an arbitrary choice btw - it's what Mason uses. As it turns out, less than a dozen lines of Tcl are needed to convert the above into a normal Tcl script which consists mostly of subst calls. In fact, I use a second level of <%...%> escapes as well for structuring the entire files into "components". This is the sort of thing Mason has worked out long ago, driven by practical use, not mere preference of a specific technology or language construct.

Before taking this discussion further, my suggestion would be to take some notation and run with it for a while to see just well it works out. You can't discuss these design choices without using them in several decently-sized real-world website applications. Well, not meaningfully anyway - IMO.

APW: That approach looks very good to me and makes sense in that the final HTML layout is still visible in the template! Thanks to jcw for bringing that in.

jcw: Here's the code for that approach:
proc substify {in {var OUT}} {
    set pos 0  
    foreach pair [regexp -line -all -inline -indices {^%.*$} $in] {
        lassign $pair from to
        set s [string range $in $pos [expr {$from-2}]]
        append script "append $var \[" [list subst $s] "]\n" \
                                    [string range $in [expr {$from+1}] $to] "\n"
        set pos [expr {$to+2}]
    set s [string range $in $pos end]
    append script "append $var \[" [list subst $s] "]\n"

You give it a template, and it returns a script you can eval to apply that template. I turn such scripts into procs, but that's just one way to use this.

APW: Great work jcw, that's exactly, what I am looking for.

pyk: AMG highly recommends JCW's [substify], and uses a variant of it to drive his Web site [1], including a custom search engine made for a friend. See his code here: [2].

  • First look at template [3]
  • Then look at [emit_template] in site.tcl [4]
  • Which is called near the bottom of script [5].

see also: CGI script for directory listing, simple posts and templates

FF I noticed a little problem with switch:
% switch what {
% case1 {
% }
% what {
something other
% }
% }

It seems that two adjacent % lines generate an empty append line, and a construct like switch don't like that.

APW: Here is a slightly modified version of substify which should handle switch correctly:
proc substify {in {var OUT}} {
    set script ""
    set pos 0
    foreach pair [regexp -line -all -inline -indices {^%.*$} $in] {
        foreach {from to} $pair break
        set s [string range $in $pos [expr {$from-2}]]
        if {[string length $s] > 0} {
            append script "append $var \[" [list subst $s] "]\n"
        append script "[string range $in [expr {$from+1}] $to]\n"
        set pos [expr {$to+2}]
    set s [string range $in $pos end]
    if {[string length $s] > 0} {
        append script "append $var \[" [list subst $s] "]\n"
    return $script

jcw: Good catch, thanks!

FYI, fixed a missing init of $script in the above (when $in is empty).

merlin 2007-11-02 17:00 GMT: I think, it is simple to use a native tcl interpreter to parse templates. We have a simple template syntax, which is easy to convert ("compile" a template) into proper and correct tcl code. Doing subst on that code ("fetch" a template), we get a rendered page.

Compiling can be done without any variables set, which are used in the template. "Fetching" requires these variables to be set in the current scope (else we get a tcl error, about using unset variables).
# ====== Compiler ======
# compilng section block
proc parse_section {arg} {return "\[_section [lindex $arg 0] [parse [lindex $arg 1]] \{"}
proc parse_end_section {} {return "\}\]"}

# compiling if-else blocks
proc parse_if {arg} {return "\[if [parse $arg] \{_echo \""}
proc parse_else {arg} {return "\"\} else \{_echo \""}
proc parse_end_if {} {return "\"\}\]"}

# curly braces
proc parse_lbr {arg} {return "\\\{"}
proc parse_rbr {arg} {return "\\\}"}

# Determines block {...} type, passed to this proc as str parameter
# and calls a proc to change that content into complement tcl-code.
# Returns a new line to substitute {...} block
proc parse {str} {
    if [string equal -length 1 $str \$] {
        # Block starting with $ is a command to put variable value
        if [regexp {\$(.+)\[(\d+)\]} $str trash vh vi] {
            # If there are square brackets with an index in there, like [3],
            # replacing that recursively with call to list index
            set str "\[lindex [parse "$$vh"] $vi\]"
    } elseif [string equal -length 1 $str /] {
        # Block starting with / is an end for some block.
        # It can't contain any parameters
        set cmd [string range $str 1 end]
        set str [parse_end_$cmd]
    } else {
        if [regexp {^(\w+)(?: (.+))?$} $str trash cmd arg] {
            # If the block begins with the word, that word is considered as
            # command name, the rest is parameters of that command
            set str [parse_$cmd $arg]
        # !TODO!: allow "else" with expressions parsing
    return $str

# Find and replace any {...} to [parse ...], then subst the result
proc compile {tpl} {
    upvar template template

# --- Save ", ], [, $ between blocks ---
    # Find coordinates of that "betweens"
    set beg 0
    set escapes {}
    while {[regexp -start $beg -indices {\{([^\}]+)\}} $tpl res]} {
        set escapes [concat $beg [expr [lindex $res 0]-1] $escapes]
        set beg [expr [lindex $res 1]+1]
    set escapes [concat $beg [expr [string length $tpl]-1] $escapes]
    # $escapes now contain a reversed list of blocks to escape

    # Escape rules
    set escamap {\" \\\\\\\" \$ \\\\\\\$ \[ \\\\\\\[ \] \\\\\\\]}
    # " (extra \ for subst)

    foreach {i j} $escapes {
        set tpl [string replace $tpl $i $j [string map $escamap [string range $tpl $i $j]]]

    set template $tpl

    # Regexp below finds any {...} and replaces them to [parse ...].
    # then subst calls there parses, and strips some backslashes
    uplevel {return [subst [regsub -all {\{([^\}]+)\}} $template {[parse {\1}]}]]}
# !TODO! rewrite a compile to more clean code,
# allowing a nested {} in the {...} blocks,
# which will enable syntax like {section i {a b c}}<li>{$i}</li>{/section}

# ====== "Fetcher" ======
proc _echo {val} {return $val}
proc _section {name list block} {
    set result {}
    foreach $name $list {append result [subst $block]}
    return $result

proc fetch {ctpl} {
    upvar template template
    set template $ctpl
    uplevel {return [subst $template]}

Usage of this code is demonstrated below (assuming code is in the file cp.tcl):
source "cp.tcl"

# A template
set tpl {Some{if $e}<ul>{section i $cycle}<li>"{$i}"</li>{/section}</ul>{else}"{$cycle[1]}"{/if}tails}

# Data for that template
set e 1
set cycle {one two three}

# ====== Call of compiler and fetcher ======
puts "Source template: $tpl"
puts "Compiled template: [compile $tpl]"
puts "Filled-in block: [fetch [compile $tpl]]"

Everything (fetch, compile) is done with subst =)

AMG: [substify] is great stuff. Thanks, JCW! See Web Templating for more information on how I use it. (Actually, comments on how Andy is using this was moved to this page above, just after "AMG highly recommends...")

AMG again: Here is [substify] adapted to work as a simple Unix-style text filter.
package require Tcl 8.5
proc applytemplate {template} {
    set script ""
    set pos 0
    foreach pair [regexp -line -all -inline -indices {^%.*$} $template] {
        lassign $pair from to
        set str [string range $template $pos [expr {$from - 2}]]
        if {$str ne ""} {
            append script "puts \[" [list subst $str] \]\n
        append script [string range $template [expr {$from + 1}] $to]\n
        set pos [expr {$to + 2}]
    set str [string range $template $pos end]
    if {$str ne ""} {
        append script "puts -nonewline \[" [list subst $str] \]
    uplevel 1 $script
while {[lindex $argv 0] eq "-file"} {
    set chan [open [lindex $argv 1]]
    set argv [concat [lrange $argv 2 end] [read $chan]]
    close $chan
    unset chan
dict with argv {applytemplate [read stdin]}

Example usage:
$ cat enums.c.tmpl
#include <stdlib.h>
typedef enum {
% foreach enum $enums {
% }
} ${name}_t;
char *${name}_names\[\] = {
% foreach enum $enums {
% }
% # This is a comment.
$ ./template.tcl enums "a b c d e" name "letter" < enums.c.tmpl > enums.c
$ cat enums.c
#include <stdlib.h>
typedef enum {
} letter_t;
char *letter_names[] = {

Lordmundi: [substify] is great. I'm using it to serve web templates like I'm guessing many others. However, I ran into an issue where the pages I'm serving have javascript in them, and therefore, lots of dollar signs("$") in them, which are getting interpreted by subst. I can backslash them, but this tends to make it very messy. Is there an easy way to edit substify such that another char, possible "#" at the beginning denotes pure text much in the way that % marks pure tcl? I tried editing it a couple of times and am not having much success.

Bezoar : I ran in to same problem as Lordmundi above. I was able to just declare tcl variables to resubstitute themselves back in to the script. Thank goodness the {} is a valid array name. For example:
% array set {} { document "$(document)" "'#sqlresult'" "$('#sqlresult')" }
      $(document).ready(function() {
      } );
     $(document).ready( function () {
       $('#sqlresult').dataTable( {
        "bPaginate": false
       } );
      } );
     $(document).ready( function () {
     $('#sqlresult').dataTable( {
        "bLengthChange": true
       } );
      } );

Will result in the script block going through unchanged.