TemplaTcl: a Tcl template engine

FF 2007-30-05 - The discussion evolved in ToW Tcl on WebFreeWay has generated some interest, and here is an effort for making a minimal template engine. I would like to use this page for collecting ideas from people. Feel free to change everything, fix bugs and revolutionize the concept - it is the purpose of the page.

FF 2007-06-02 - I rewrote the engine in a OO-like way (still haven't used any existing OO system, neither build my OO system) (as a rule, methods beginning with _ are meant to be private), improving some features. Very important is that from now it could run in a recursive way, allowing one to design modular/nested templates. (Note that since the rewrite I changed all the examples.)

Description

Really, the engine works by converting a template (which is a string, or a file) into a Tcl script, and then running it. Nearly each line of text encountered will be returned as is. The exception is text between <% ... %> which is treated as Tcl code (the eval happens in a safe interp, see Safe Interps). This makes it very flexible and it leaves you freedom of doing what you want, dealing with any data and putting how much code you want in the template (although the template should possibly hold minimum code - according to MVC, just the presentation logic, no code related to controller logic, state persistance, or data-retrieval logic).

Also for convenience there is a <%= ... %> for spooling output (the proc _defaultSpool takes care of sending the text to the right place; you can change that: see Preprocessor Commands).

Usage

First, let's create a template file (templtest.tmpl):

 <table><%
   for {set i 0} {$i < 4} {incr i} { %>
   <tr>
     <td><%= $i %></td>
   </tr><%
   } %>
 </table>

 <ul>
   <% foreach item $cityList {
     %><li><%= $item %>
   <% } %>
 </ul>

The above template (in HTML language) generates a 4-row table, and an unordered list, taking values from a $cityList variable, that we're eventually going to define.

First, let's load the code:

 $ tclsh
 % source TemplaTcl

now create a parser instance:

 % TemplaTcl::create tt

and parse our template:

 % tt parseFile templtest.tmpl

set the variables used in our template:

 % tt setVar cityList {Ragusa Ravenna Rieti Rimini Rome Rovigo}

and render the template:

 % puts [tt render]

Here's the output that gets produced:

 <table>
   <tr>
     <td>1</td>
   </tr>
   <tr>
     <td>2</td>
   </tr>
   <tr>
     <td>3</td>
   </tr>
   <tr>
     <td>4</td>
   </tr>
 </table>

 <ul>
   <li>Ragusa
   <li>Ravenna
   <li>Rieti
   <li>Rimini
   <li>Rome
   <li>Rovigo
 </ul>

Preprocessor commands

Running in a safe interp, you cannot source your Tcl function library. There's a special markup command: <%@ ... %> that allows you to do special things, i.e., setting parser/interp options. That happens before the parser and before running the interpreter.

To source a file you will do:

 <%@ source = poo.tcl %>

Instead, for including a template:

 <%@ include = template.tmpl %>

Also for changing the default spooler (mentioned above) do:

 <%@ printCommand = puts -nonewline %>

(that replaces the _defaultSpool spooler, which allows you to get the content as the return value of TemplaTcl::render).


 #!/usr/bin/env tclsh

 package require Tcl 8.5
 # 8.5 required cause the {*} in proc create and proc method
 package require struct
 # tcllib required

 namespace eval ::TemplaTcl {
        variable obj

        proc method {name args body} {
                proc $name [list self {*}$args] "variable obj ; $body"
        }

        method create {} {
                # create and setup a safe interpreter
                # for running template's tcl code
                catch {if [interp exists $obj($self:interp)] {
                        interp delete $obj($self:interp)}}
                set obj($self:interp) [interp create -safe]
                interp share {} stdout $obj($self:interp)
                interp eval $obj($self:interp) {
                        proc _defaultSpool {txt {cmd {}}} {
                                global content
                                if {$cmd == "clear"} {set content {}; return}
                                if {$cmd == "get"} {return $content}
                                append content $txt
                        }
                }
                uplevel "proc $self {method args} {namespace eval ::TemplaTcl \[list \$method $self {*}\$args\]}"
                return $self
        }

        method parseFile {filename} {
                # read the template into $rawl - list of chars
                set fh [open $filename r]
                set raw [read $fh]
                close $fh
                return [$self parse $raw]
        }

        method parse {template} {
                $self _setMode raw
                #$self setOption printCommand "puts -nonewline"
                $self setOption printCommand "_defaultSpool"
                $self setVar * {}
                set q [::struct::queue]
                set rawl [split $template {}]
                foreach ch $rawl {
                        # we work char-by-char :|
                        $q put $ch
                        # max block to compare (<%=) is 3 chars long:
                        if {[$q size] >= 3} {
                                set s3 [join [$q peek 3] {}]
                                set s2 [join [$q peek 2] {}]
                                switch $obj($self:mode) {
                                raw {   if {$s3 == "<%="} {
                                                # <%= is a shorthand for puts ...
                                                $q get 3; $self _setMode code;
                                                append obj($self:buf:$obj($self:mode)) "$obj($self:options:printCommand) "
                                                continue
                                        } elseif {$s3 == "<%@"} {
                                                # <%@ is for setting preprocessor options
                                                $q get 3; $self _setMode opt; continue
                                        } elseif {$s2 == "<%"} {
                                                # <% indicates begin of a code block
                                                $q get 2; $self _setMode code; continue
                                        } }
                                code {  if {$s2 == "%>"} {
                                                # and %> is the end of code block
                                                $q get 2; $self _setMode raw; continue
                                        } }
                                opt {   if {$s2 == "%>"} {
                                                # option parser
                                                $q get 2;
                                                $self _parseOptions $obj($self:buf:opt)
                                                set obj($self:buf:opt) {}
                                                $self _setMode raw; continue
                                        } }
                                }
                                append obj($self:buf:$obj($self:mode)) [$q get]
                        }
                }
                # finish processing the queue:
                while {[$q size] > 0} {
                        append obj($self:buf:$obj($self:mode)) [$q get]
                }
                $self _setMode flush
                # cleanup:
                foreach v {buf:code buf:opt buf:raw mode modeprev} {catch {unset obj($self:$v)}}
        }

        method render {} {
                # run the template script
                set tclBuf ""
                foreach l $obj($self:data) {
                        set t [lindex $l 0]
                        set d [lindex $l 1]
                        switch $t {
                                raw {append tclBuf "$obj($self:options:printCommand) [list $d]\n"}
                                code {append tclBuf "$d\n"}
                        }
                }
                foreach {var val} $obj($self:variables) {$obj($self:interp) eval [list set $var $val]}
                #puts $tclBuf;return
                if {$obj($self:options:printCommand) == "_defaultSpool"} {
                        $obj($self:interp) eval {_defaultSpool {} clear}
                }
                $obj($self:interp) eval $tclBuf
                if {$obj($self:options:printCommand) == "_defaultSpool"} {
                        set x [$obj($self:interp) eval {_defaultSpool {} get}]
                        $obj($self:interp) eval {_defaultSpool {} clear}
                        return $x
                }
        }

        method setOption {opt value} {
                switch $opt {
                        printCommand {set obj($self:options:$opt) $value}
                        include {
                                set o inc$value
                                create $o
                                $o parseFile $value
                                set prevMode [$self _setMode raw]
                                append obj($self:buf:raw) [$o render]
                                $self _setMode $prevMode
                        }
                        source {$obj($self:interp) invokehidden source $value}
                        default {return -code error -errorinfo "Unknown option: $opt"}
                }
        }

        method setVar {var value} {
                if {$var == "*" && $value == ""} {set obj($self:variables) {}; return}
                lappend obj($self:variables) $var
                lappend obj($self:variables) $value
        }

        method _setMode {m} {
                set modenew {}
                switch $m {code - raw - opt {set modenew $m}}
                if {$modenew != {}} {
                        if [catch {set obj($self:mode)}] {set obj($self:mode) $modenew}
                        set obj($self:modeprev) $obj($self:mode)
                        set obj($self:mode) $modenew
                        set obj($self:buf:$obj($self:mode)) {}
                        if {$obj($self:modeprev) == {}} {
                                set obj($self:modeprev) $obj($self:mode)
                        }
                }
                if {$m == "flush"} {
                        set obj($self:modeprev) $obj($self:mode)
                        set obj($self:mode) _
                }
                if {$obj($self:mode) != $obj($self:modeprev)} {
                        lappend obj($self:data) [list $obj($self:modeprev) $obj($self:buf:$obj($self:modeprev))]
                        set obj($self:buf:$obj($self:modeprev)) {}
                        return $obj($self:modeprev)
                }
        }

        method _parseOptions {o} {
                set optlist [split $o "\n;"]
                foreach opt $optlist {
                        set pair [split $opt =]
                        set opt_ [string trim [lindex $pair 0]]
                        if {$opt_ == {}} continue
                        $self setOption $opt_ [string trim [lindex $pair 1]]
                }
        }

}


Discussion

escargo - I think it's a mistake to have parse read a file; it would be potentially more flexible to use if it just took a string to process as input, and let the caller determine where the string comes from. FF - good point! (changed)

APW Just found textutil::expander[L1 ] package in tcllib, which is included in ActiveTcl 8.4.14.0. Have the feeling it does something similar, maybe it's worth looking at it.

FF but is not much versatile. Just tried this:

 % package require textutil::expander
 % ::textutil::expander myexp
 % set a 64
 % myexp expand {[if {$a > 50} {] put some text [} {] put some other text [}]}
 Error in macro:
 [if {$a > 50} {] put some text [} {] put some other text [}]
 --> missing close-bracket

and I realized I feel well with TemplaTcl O:)

APW I think if you look closer to the manual you will see that you have to set the left and right bracket command i.e. "<%" and "%>". Next the token directly following the left bracket is interpreted as the command for handling the contents, so if you have "<%= hello %>" you will need a proc with name "=" for handling the code inside the macro.

  % package require textutil::expander
  % ::textutil::expander te
  % te lb "<%"
  % te rb "%>
  % proc = {args} { return $args }
  % te expand {this is a text which says <%= hello %> to you}
  this is a text which says hello to you

Additionally for constructs like a "if" clause it is possible to push and pop the context and at the end to evaluate the code in the macro, normally you will need some tokens for marking begin and end of macro, means a construct like:

  <% code if { ...} { %>
    # some other code here
  <% /code %>

  proc code {args} {
    te cpush ....
    # some code here
  }

  proc /code {args} {
    set str [te cpop ...]
    # evaluate the code of the macro and set result
    return result
  }

You can also define the number of passes to run for the same text to be able to do nested expansion if you use the expand.tcl directly from the author: William Duquette [L2 ]


escargo - I think you want to give some thought into what scope the template code executes at. Right now it seems to be run in proc dump. Two likely alternatives are to run it in the caller's scope or at global scope.

FF running in the caller scope might seem a convenient idea, but can potentially generate trouble, since the template is a tcl script that would run just between TemplaTcl::dump and the next command, potentially altering/damaging the caller's scope (if there are subsequent commands after the TemplaTcl::dump command); in order to make it more strict, I make it run in global scope (hoping that global is not polluted or has variable that can make the application unstable); but ideally it should run in a separate interp, copying the needed vars into that interpreter. Don't you think so?

escargo - Running in a safe interp would be the right thing, but the question might be what data can the code executing as part of template processing access (or access easily). One could reasonable argue that it should only access values provided by a supplied data object. I could see wanting to supply maybe an environment data object as well (for access to info like date and time, host name, user name, and other info that would not be part of the data coming out of a table to be published).

FF added safe interp, setVar command, and also a new markup command for special use (see above).

escargo 1 Jun 2007 - See also subst. In some respects, subst is a better model than eval, since it assumes text with variable references and commands embedded in it, instead of valid programs with text embedded in the commands. The ability to intercept variable references and command executions that you don't know ahead of time would simplify some things that you want to do, but those aren't available yet. (Maybe what's needed is a subst that has been extended to take a -variables and/or -commands arguments, which are the names of commands to call when there are variable references or command references respectively.)

FF 2007-06-02 - changed much things (almost all). I implemented a pseudo OO system. This allows running recursive/nested templates. If you were interested in this project, maybe you might be interested in re-reading this page from the beginning.

CMcC 2Jun07 just doesn't get it - why create a language interpreter to substitute variables and function calls when tcl has a full-function command to do that, in subst? What does the new little language for templating buy you?

APW Would that also work for the following example? I don't think so!

  <%if {$a != $b} { %>
    <h3> abc </h3>
    <%if {$a != 1} {%>
      <li> <%= $var1 %> </li>
    <%} else {%>
      <li> <%= $var2 %> </li>
    <% } %>
  <% } else { %>
      <li> <% myobj getResult %> </li>
      <table>
         <tr> Hello there </tr>
      </table>
  <% } %>

CMcC no, this would work better:

 expr {
   $a != $b
    ? "[<h3> abc][<li> [expr {$a != 1 ? $var1: $var2}]]"
    : "[<li> [myobj getResult]] [<table> [<tr> "Hello there"]]"
  }

This would also work:

 if {$a != $b} {
    subst {<h3> abc </h3>
        [expr {$a != 1
               ? "<li> $var1 </li>"
               : "<li> $var2 </li>"}]
    }
 } else {
    subst {
        <li> [myobj getResult] </li> <table> <tr> Hello there </tr> </table>
    }
 }

(note: if returns a value!)

But nothing would work as well as refactoring it so it wasn't quite as messy in the first place.

APW from the discussion on ToW Tcl on WebFreeWay I have the feeling that people would like to have a template that looks like a HTML page (maybe also XML), so they could see what goes to the client and to have a little bit (or some more) code spread in to drive some output. That's also what Ruby on Rails (short RoR) does. So having Tcl code to produce HTML is not what is wanted in that case (my interpretation). Also for a Tcl programmer that might be more readable I agree. But I think the audience should be more people that do not know much about Tcl but are able tho enter some variables and simple constructs in Tcl to get input from a data source, for example a database.

CMcC This is getting silly. Here's a fragment of the stuff that generates this very page you're reading - it's done using subst:

    # page template for standard page decoration
    variable pageT {title: $name

        [div container {
            [div header {<h1 class='title'>$Title</h1>}]
            [div {wrapper content} {<p>$C</p>}]
            <hr noshade />
            [div footer "<p>[join $menu { - }]</p>
                <p><form action='/_search' method='get'>
                <input name='S' type='text' value='Search'>
                </form></p>"]
        }]
    }

If you really wanted to, you could replace the [div] calls with straight <div>...</div> text.

If you feel that subst can't be used on text that resembles an HTML page, you really should have a careful look at what subst does.

APW That works if you have defined a proc named div when using subst, but if you want to parse a XML template where you don't know all the tags and you only want to handle specially the part included in <% ... %> and handle all the rest as a text/string? Could you give me an example for that using subst?

  <company> my company </mycompany>
    <% if {[string first $mycompany "DE"]} { %>
       <cname class="DE"> <%= [getmycompany DE] %> </cname>
       <department> my department </department>
    <% } else { %>
       <cname class="OTHERS"> <%= [getmycompany OTHERS] %> </cname>
    <% } %>
  </company>

This is just a very simple example, which could be built differently to produce the same output, I know. How would the template file look like for being able to use subst? The output shall be again XML being that conversion the first pass of multiple passes, conversion to HTML being done in a later pass.

CMcC Since you asked for subst usage:

 [subst {
   <company> my company </company>
   [expr {[string first $mycompany "DE"]
     ? "<cname class='DE'>[getmycompany DE]</cname>
        <department> my department </department>"
     : "<cname class='OTHERS'>[getmycompany OTHERS]</cname>"
   }]
  }]

But I think I'd do this instead in that case:

  "<company> my company </company>
   [expr {[string first $mycompany "DE"]
     ? "<cname class='DE'>[getmycompany DE]</cname>
        <department> my department </department>"
     : "<cname class='OTHERS'>[getmycompany OTHERS]</cname>"
   }]"

Or, by preference, a bit of refactoring:

  set cclass [expr {[string first $mycompany "DE"]?DE:OTHERS}]
  "[<company> {my company}]
   [<cname> [list class $cclass] {[getmycompany $cclass]}]
   [if {$cclass eq "DE"} {<department> {my department}}]"

APW that's ok, but it doesn't look like an XML page template any more, and that was what some people including me would like to have! The last example you have can also be coded using Tcl code without any subst (and then it might be even more readable :-)) ), but that was not the intention.

CMcC Ok, so using subst-if:

 [subst {
   <company> my company </company>
   [If {[string first $mycompany "DE"]} {
        <cname class='DE'>[getmycompany DE]</cname>
        <department> my department </department>
     } else {
        <cname class='OTHERS'>[getmycompany OTHERS]</cname>
   }]
  }]

will now work. Does that look enough like an XML template for the purpose?

APW Meanwhile jcw has suggested an interesting solution for what I would like to have see below.


escargo - subst does not have a notion of iteration to expand things inside the string being operated on, for one thing.

CMcC Tcl has the concept of iteration, though. Use map, or it's easy to write a function that iterates (think foreach, which returns its iterated results). For example, the thing above to turn a list into an HTML list could be simply represented as <ul><li>[join $list </li><li>]</li></ul>, which is cleaner (I think) ... or even define a proc (such as is already in the tcllib html package) to perform the wrapping.

The page HTML generator provides some light-weight facilities along those lines.

FF the above mentioned systems sure are good for some things. But my target is a very specific case, where those systems maybe lack some things.

TemplaTcl works with templates rich in static text, but also rich in interruptions due to code or putting variables (the typical case is generating markup for a web page). Dealing with that using other systems can possibly result being counter-productive (the TemplaTcl examples found in this page are too simple to demonstrate that).

CMcC Do you mean that you can stop the template expansion midway? subst can also do that.

FF Even if right now it does its job nicely - just now I added template inclusion (<%@include = ..%>), and Tcl source-ing (<%@source = ...%>) - I'll keep adding missing features to satisfy the case of code generation, with a particular regard to X(HT)ML page templating, so don't expect it to be just that tiny set of features.

CMcC What I'm interested in is examples of some useful things TemplateTcl does better than Tcl, because that's a sign of something that could be improved in Tcl.

FF What I mean is: your example is more Tcl code than markup, the markup is hidden and split across the Tcl commands. A template designer (or an X(HT)ML editor) shouldn't care about that. If you don't see advantages in that, maybe you don't need this engine. Note: I'm not stating Tcl has some lack; I'm simply doing an application in Tcl. It is NOT doing "something better than Tcl"! It's doing something "with Tcl" - that sounds more appropriate ;)

CMcC I was hoping there was some lack, so it could be repaired.

03jun07 jcw - While subst is indeed great for templating IMO, it is unfortunate that it is completely linear. When you use expr and self-defined [...] calls inside to control flow and repetition, you end up nesting things twice each time: first to embed the operation, and again to protect/return the body of that operation. Even Tcl has two modes for that because it becomes unwieldy: the command mode where each command starts on a newline (or after ";"), and embedded calls with [...].

CMcC I agree with you, jcw. So I think I'd tend to define an if command that evaluated the test using expr, and then evaluated the consequent or else using subst instead of eval, thus:

  [If {$fred} {
    <h1> header $fred</h1>
    <..>
  } else {
    <h1> header not $fred</h1>
    <more html>
  }]

That would be an hour's work, would fit nicely alongside a modified foreach and sit inside a namespace. It was 20 minutes' work, and it's on the HTML generator page. Thanks, jcw, for cutting to the nub of the problem. Subst-foreach is next.

What other facilities would be useful?

APW What about Subst-while?

CMcC Too easy :) It's right after Subst-if on the other page.


03jun07 jcw - But I don't see the point in embedding an escape layer using XML. Here's a visually simpler approach:

Templates and subst

APW 2007-06-07 - I have moved the suggestion from jcw to its own page, as there are some places where I would like to add a link to that page. Hope all agree on that.

escargo 3 Jun 2007 - I think one of the reasons for templates is that you can do some of the work of layout in WYSIWYG HTML tools, and then decide what parts to replace with parameters and control structures to get a more generalized form. Certainly you would want to be using CSS wherever possible to separate structure from presentation, but for abstracting certain content details, having a templating system allows decomposition into something that is closer to the original source than replacing final HTML with calls to the tcllib html package or with hand-coded Tcl to generate the HTML.

escargo 19 Jun 2007 - I've been looking at the tcllib package named html. It is the opposite of templating, that is a script of commands produces the HTML result. I think it's interesting that many of its commands return strings based on subst in the body.

  • ::html::eval
  • ::html::for
  • ::html::foreach
  • ::html::if
  • ::html::while

Sly: I believe my implementation is much more minimal. And it works for most common cases.

  proc parse {txt} {
        set code "set res {}\n"
        while {[set i [string first <% $txt]] != -1} {
                incr i -1
                append code "append res [list [string range $txt 0 $i]]\n"
                set txt [string range $txt [expr {$i + 3}] end]
                if {[string index $txt 0] eq "="} {
                        append code "append res "
                        set txt [string range $txt 1 end]
                }
                if {[set i [string first %> $txt]] == -1} {
                        error "No matching %>"
                }
                incr i -1
                append code "[string range $txt 0 $i] \n"
                set txt [string range $txt [expr {$i + 3}] end]
        }
        if {$txt ne ""} { append code "append res [list $txt]\n" }
        return $code
  }

Let's test it:

  set code [parse {
  asdfgh
  <%
  for {set i 5} {$i < 10} {incr i} {
  %>
        ZZZZ <%=$i%> XXXXX <%=[expr {$i * 5}]%>
  <%
  }
  %>
  zxcvb
  } ]

  puts "   THIS IS CODE:\n\n$code\n\n"
  eval $code
  puts "   THIS IS RESULT:\n\n$res"

Output:

     THIS IS CODE:

  set res {}
  append res {
  asdfgh
  }

  for {set i 5} {$i < 10} {incr i} {

  append res {
          ZZZZ }
  append res $i
  append res { XXXXX }
  append res [expr {$i * 5}]
  append res {
  }

  }

  append res {
  zxcvb
  }



     THIS IS RESULT:


  asdfgh

          ZZZZ 5 XXXXX 25

          ZZZZ 6 XXXXX 30

          ZZZZ 7 XXXXX 35

          ZZZZ 8 XXXXX 40

          ZZZZ 9 XXXXX 45

  zxcvb

CAU See also, THP it's currently driving some dashboard tools for my employer. I think it's very similar to what you want.

AMG: Also, see the template system in Wibble. Look for "wibble::template" and "wibble::applytemplate".

jbr : Subst template expansion helpers


kRob - 2013-09-09 21:30:37

Hey Guys,

I'm trying to run this script and keep getting an error for the following line of script:

$obj($self:interp) eval $tclBuf

this is line 117 in the render method in TemplaTcl and the error gives " invalid command name "}"

I'm not familiar with TemplaTcl at all and cant' seem to figure it out.

Anyone else getting this?


milarepa - 2014-07-13 21:10:15

I got the same problem as kRob.


ray2501 - 2017-11-15 14:25:00

I got the same problem when I use Tcl 8.6.1. However, it works at Tcl 8.6.3 and 8.6.7.

And if you want to parse and render more than 1 time, before method rendor execute return $x yon maybe need clear data, add set obj($self:data) {}


FireTcl - 2015-02-15 18:13

This is my own version. It uses TclOO. It has no other dependency. Possibily to uses filters.

    namespace eval ::TemplaTcl {

        namespace eval filters {
            proc trim {txt} {
                return [string trim $txt]
            }
            proc title {txt} {
                return [string totitle  $txt]
            }
            proc lower {txt} {
                return [string tolower $txt]
            }
            proc upper {txt} {
                return [string toupper $txt]
            }
        }

        proc filter {filter_name text} {
            return [filters::$filter_name $text]
        }

        array set RE [list \
                    start_code {\A\n?[ \t]*<%\s*} \
                    finnish_code {\A\s*%>[ \t]*\n?} \
                    start_variable {\A<%=\s*} \
                    finnish_variable {\A\s*%>} \
                    start_options {\A\n?[ \t]*<%@\s*} \
                    finnish_options {\A\s*%>[ \t]*\n?}]

        oo::class create TemplaTcl {
            variable interp_name mode modeprev options parsed_template list_of_user_variables buffer_options buffer_raw buffer_code buffer_variable output RE

            constructor {{filename ""}} {

                set ns [namespace qualifiers [info object class [self]]]
                namespace eval [self namespace] "namespace upvar $ns RE RE"

                # create and setup a safe interpreter for running template's tcl code
                set interp_name [interp create -safe]
                interp alias $interp_name _filter {} ${ns}::filter
                interp alias $interp_name print {} puts 

                interp eval $interp_name {
                    set _output ""

                    proc ::puts {args} {
                        global _output

                        set list_of_filters [list]
                        set end_char "\n"

                        set i 0 

                        set args_length [llength $args]

                        while {$i < $args_length} {
                            set arg [lindex $args $i]
                            if {$arg eq "-varname"} {
                                incr i
                                set varname [lindex $args $i]
                                upvar $varname variable_value
                                set msg $variable_value
                            } elseif {$arg eq "-nonewline"} {
                                set end_char "" 
                            } elseif {$arg eq "-filters"} {
                                incr i
                                set list_of_filters [lindex $args $i]
                            } else {
                                set msg $arg
                            }
                            incr i
                        }

                        set msg $msg$end_char

                        foreach filter_name $list_of_filters {
                            set msg [_filter $filter_name $msg]
                        }


                        append _output $msg
                    }
                }
                array set options [list]

                set list_of_user_variables [list]

                set buffer_code ""
                set buffer_options ""
                set buffer_raw ""
                set buffer_variable ""

                if {$filename ne ""} {
                    my set_template_from_file $filename
                }
            }

            method set_template_from_file {filename} {
                set fh [open $filename r]
                set template [read $fh]
                close $fh
                my set_template $template
            }

            method set_template {template} {

                my Initialize_mode

                set template_length [string length $template]
                set index 0

                while {$index < $template_length} {
                    switch $mode {
                        raw {   
                            set found_mode 0
                            foreach possible_mode [list options variable code] {
                                set startToken start_$possible_mode
                                if {[regexp -indices -start $index -- $RE($startToken) $template matched_indices]} {
                                    my Set_mode $possible_mode
                                    set index [lindex $matched_indices 1]

                                    set found_mode 1
                                    break
                                }
                            }
                            if $found_mode continue
                        }
                        code -
                        variable -
                        options {  
                            if {[regexp -indices -start $index -- $RE(finnish_$mode) $template matched_indices]} {
                                my Set_mode raw
                                set index [lindex $matched_indices 1]

                                continue
                            }
                        }
                    }
                    append buffer_$mode [string index $template $index]
                    incr index
                }

                lappend parsed_template [list $mode [set buffer_$mode]]
                set buffer_$mode {}

                set template {}
            }

            method Initialize_mode {} {
                set mode raw
                set modeprev ""
            }

            method render {} {
                # run the template script

                set output ""
                set tclCode ""
                foreach parsed_item $parsed_template {
                    lassign $parsed_item item_type item_content 

                    switch $item_type {
                        raw {
                            append tclCode "puts -nonewline [list $item_content]\n"
                        }
                        code {
                            append tclCode "$item_content\n"
                        }
                        variable {
                            append tclCode "puts -nonewline -varname $item_content\n"
                        }
                        options {
                            my Parse_options $item_content tclCode
                        }
                    }

                }

                foreach {varname value} $list_of_user_variables {
                    interp eval $interp_name [list set $varname $value]
                }

                interp eval $interp_name $tclCode

                set output [interp eval $interp_name {set _output}]
                interp eval $interp_name {set ::_output ""}

                return $output
            }

            method set_vars {args} {
                lappend list_of_user_variables {*}$args
            }

            method get_vars {} {
                return $list_of_user_variables
            }

            method delete_vars {} {
                set list_of_user_variables [list]
            }

            method Set_mode {m} {
                set modeprev $mode
                set mode $m

                if {$mode != $modeprev} {
                    lappend parsed_template [list $modeprev [set buffer_$modeprev]]

                    set buffer_$modeprev {}
                }
            }

            method Parse_options {o tclCode_var} {
                upvar $tclCode_var tclCode

                set optlist [split $o "\n;"]
                foreach opt $optlist {
                    set pair [split $opt =]
                    set opt_ [string trim [lindex $pair 0]]
                    if {$opt_ == {}} continue
                    my Set_option $opt_ [string trim [lindex $pair 1]] tclCode
                }
            }

            method Set_option {opt value tclCode_var} {
                upvar $tclCode_var tclCode

                switch $opt {
                    include {
                        set o [[info object class [self]] new $value]
                        $o set_vars {*}[my get_vars]

                        append tclCode [$o render]
                    }
                    source {interp invokehidden $interp_name source $value}
                    default {return -code error -errorinfo "Unknown option: $opt"}
                }
            }

        }

    }

    # Example:
    set p [::TemplaTcl::TemplaTcl new] 
    $p set_vars cityList [list {New york} Madrid London]
    $p set_template {
        <table>
            <% for {set i 0} {$i < 4} {incr i} { %>
                <tr><td><%= i %></td></tr>
            <% } %>
        </table>
        <ul>

        <% foreach item $cityList { %>
            <li><%= item %></li>
        <% } %>
     </ul>}
    puts [$p render]