Tuplespaces are the generic term for what was called at Yale a Lindaspace. It's essentially a very simple, powerful and elegant technique for networking clients together where no one talks directly to each other and communication is accomplished via a publish/subscribe paradigm. You can do cool things like chat systems, load balancers and fault tolerance through Tuplespaces. See also http://c2.com/cgi/wiki?TupleSpace and http://c2.com/cgi/wiki?LindaTupleSpaces This is just a simple implementation in Tcl. An answer to the [Street Programming] Tuplespace that I have in production for my employer. -- [Todd Coram] [[The proprietary variant apparently beefs up memory management and access control capabilities.]] ''Yes, the memory manager flushed tuples from memory if they hadn't been accessed within N hours. The tuples would continue to exist in the [metakit] db, but didn't waste precious memory if they weren't being actively accessed. Since that tuplespace was used to hold hundreds of thousands of email contacts, inactive email accounts didn't take up memory... Access control allowed for readonly accounts -- some users could peruse the tuplespace (and even get callbacks) but couldn't "take" (remove) items...'' package require md5 2 # A very simple implementation of a Linda-like Tuplespace. # Tuples take the form {field1 ... fieldN}. # namespace eval tuplespace { # This array defines the tuplespace. Each column contains a list # of matching tuple ids (tids). The hash is a combination of # column number, tuple field. (.e.g - tspace(0,hello) references any # tuple (via its id) that has 'hello' in its first column. # variable tspace # Tuples are referenced in tspace by tuple ids (tids). Tids are # md5 hashes of the actual tuple and are used as an index into the # array that holds the tuples. # variable tid_arr # Put a tuple out into tuplespace. # proc out {tuple} { variable tspace variable tspace set tid [new_tid $tuple]; # create a new tuple id. set tid [new_tid $tuple]; # create a new tuple id. # For each field in the tuple, asign it to a column in tuplespace. # set fn 0 foreach tpf $tuple { lappend tspace($fn,$tpf) $tid;# this column now references tid. incr fn } return $tid } # Read (non-destructively) one (default) or more tuples from the space # matched by 'tpat'. Tpat is a tuple specification that may have a # "don't care" placeholder (?) in place of any tuple field. At least # one matching tuple field must be supplied. If rd_multiple is 1, then # return all matching tuples. # proc rd {tpat {rd_multiple 0}} { set res {} foreach tid [_rd $tpat $rd_multiple] { if {!$rd_multiple} { return [tid.tuple $tid] } lappend res [tid.tuple $tid] } return $res } # Take (by removing) a tuple from the space that matches 'tpat'. # proc in {tpat} { set res {} set tid [_rd $tpat] if {$tid != {}} { set res [tid.tuple $tid] _remove $tid } return $res } # Return a directory listing of all tuples that match 'tpat'. # proc dir {tpat} { return [rd $tpat 1] } # Do all of the work of reading a tuple. # proc _rd {tpat {rd_multiple 0}} { variable tspace set match {} # Look at all of the supplied pattern fields and collect # a list of matching tuples (based on each field). If a field # fails to match (and the field is not ?, return an empty tuple). # set fn 0 foreach tpf $tpat { if {[string equal $tpf "?"]} { incr fn continue } if {![info exists tspace($fn,$tpf)]} { return {} } lappend match $tspace($fn,$tpf) incr fn } return [_intersection $match $tpat $rd_multiple] } # Find the intersection between all of the matched (candidate) tuples. # proc _intersection {match tpat rd_multiple} { _intersection {match tpat rd_multiple} { # Calculate the number of fields actually supplied in tpat. # This is the number intersections in the matches we must find # in order to return a tuple. # set tpat_len [llength [lsearch -not -all -exact $tpat ?]] set tpat_len [llength [lsearch -not -all -exact $tpat ?]] # Find the intersect of these collected matches. The first # intersect count that equal $tpat_len is our matched # tuple (unless all matches are requested). # set result {} foreach tid [join $match] { set tuple [tid.tuple $tid] if {[llength $tuple] != [llength $tpat]} { continue } if {![info exists intersect($tuple)]} { set intersect($tuple) 1 } else { incr intersect($tuple) } if {$intersect($tuple) == $tpat_len} { lappend result $tid if {!$rd_multiple} { break } } } return $result } # Remove a tuple from the space based on the tuple id. # proc _remove {tid} { variable tspace set tuple [tid.tuple $tid] set fn 0 foreach tpf $tuple { if {[info exists tspace($fn,$tpf)]} { set tspace($fn,$tpf) [lsearch -all -not -inline \ $tspace($fn,$tpf) $tid] } incr fn } tid.delete $tid } # Create a new tid to hold a tuple. # proc new_tid {tuple} { variable tid_arr set id [tid $tuple] set tid_arr($id) $tuple return $id } # Calculate the tid from a give tuple. # proc tid {tuple} { return "\#Tuple[::md5::md5 -hex $tuple]" } # Return the tuple from the given tid. # proc tid.tuple {tid} { variable tid_arr return $tid_arr($tid) } # Destroy a tuple named by tid. # proc tid.delete {tid} { variable tid_arr unset tid_arr($tid) } # Flush a tuple from memory, but keep the tspace references around. # DANGEROUS! proc tid.flush_tuple {tid} { variable tid_arr set tid_arr($tid) {} } } # Examples: puts [tuplespace::out {hello there Todd Coram}] puts [tuplespace::out {hello there Maroc Ddot}] puts [tuplespace::out {linda space}] puts [tuplespace::dir {hello there ? ?}] puts [tuplespace::rd {hello there ? ?}] puts [tuplespace::in {hello there ? ?}] puts [tuplespace::dir {hello there ? ?}] puts [tuplespace::in {hello there ? ?}] puts [tuplespace::dir {hello there ? ?}] ---- [jmn] 2004-02-01 The above example seems to work just fine, but fails when I try: puts [tuplespace::out {hello there test etc}] I don't know what's magic about that string, but the tid returned doesn't begin with #Tuple like all the others. Something fishy with the md5 implementation I'm using I guess. Using tcllib1.5, md5 2.0.0, Tcl8.4.4 on [Windows]. Further experimentation seems to suggest there's some sort of carriage return or backspace type character being returned by [md5 {hello there test etc}] Seems the 1st 9 chars on the line are getting eaten! % puts "123456789[md5 {hello there test etc}]" ììÆ?¼ê[3#ü~²ºq % puts "123456789xxx[md5 {hello there test etc}]" ììÆ?¼ê[3#xxxü~²ºq Beats me what this means or whether it's a problem in the md5 implementation itself, or just in how it's being used here. ''Argh. The md5 package appears to have changed its API between versions. The prior version returned hex strings by default. Adding -hex to the tid generator proc's md5 call should fix the problem -- [Todd Coram]'' Either that or package require version 1 of the md5 package. [schlenk] ---- [Ruby], incidentally, has a nice, standard Tuple implementation as part of "Distributed Ruby" (drb). ---- 09/11/02 -- Incorporated Tcl'ish improvements made by [Michael Schlenker] into the code above (replacing a few clumsy for loops with foreach loops). Also, Each tid is now uniquely identified by an md5 hash from the [tcllib] [md5] package. And a few more tid procs were added. Why? Well it makes it easier to add persistence (via [metakit]!) --- [Todd Coram] : # Meta-kit persistence for the tuplespace. # namespace eval tuplespace-db { proc open {dbpath} { mk::file open db $dbpath mk::view layout db.tspace "id tuple" _load } proc close {} { mk::file close db } proc _load {} { trace variable ::tuplespace::tid_arr ruw {} mk::loop row db.tspace { set tuple [mk::get $row tuple] set tid [tuplespace::out $tuple] tuplespace::tid.flush_tuple $tid } trace variable ::tuplespace::tid_arr ruw ::tuplespace-db::getset } proc getset {name tid op} { switch -- $op { w { _write $tid } r { _read $tid } u { _unset $tid } } } proc _unset {tid} { set r [mk::select db.tspace -count 1 \ -exact id $tid] if {$r != {}} { mk::row delete db.tspace!$r mk::file commit db.tspace } } proc _read {tid} { if {[tuplespace::tid.tuple $tid] == {}} { # reload from database # set r [mk::select db.tspace -count 1 -exact id $tid] tuplespace::new_tid [mk::get db.tspace!$r tuple] } } proc _write {tid} { set tuple [tuplespace::tid.tuple $tid] # See if the tuple already exists in the database. # set r [mk::select db.tspace -count 1 -exact id $tid] if {$r != {}} { # Replace the tuple. # mk::set db.tspace!$r id $tid tuple $tuple } else { # Nope, add a new row. # mk::row append db.tspace id $tid tuple $tuple } mk::file commit db.tspace } } -------- 15/Oct/03 [schlenk]: I noticed that the [Metakit] persistence for this tuplespace does not work reliably, if md5 hashes are used. I base64 encoded things and everything works well: # Calculate the tid from a give tuple. # package require base64 proc tid {tuple} { return "\#Tuple[::base64::encode [::md5::md5 $tuple]]" } Is the problem really with Metakit itself, or with Mk4Tcl ? [schlenk] Bad phrasing on my side: It did not work with Metakit if only MD5 was used for this, but it worked when base64 encoded MD5 was used, probably due to embedded nulls or something like it...; have to take a look more closely tomorrow. [jcw] - Yep, nulls are trouble. Change: mk::view layout db.tspace "id tuple" to mk::view layout db.tspace "id:B tuple" ''Are nulls trouble for Metakit, or just for Mk4Tcl?'' [jcw] - Neither. Nulls terminate strings in C. That's why MK has type S (default in Mk4tcl) and type B properties. If you store a string with embedded null bytes in a S property, MK's C api will ignore everything past the first one, just like every other C function taking char*'s. Hmm...now you have me wondering: in a Tcl_Obj* string rep, embedded nulls are not possible, right? I wonder where the problem lies in this case. ''Wrong. Tcl_Obj string reps are counted strings precisely so they can include embedded NULLs. Since 8.1 established (modified) UTF-8 as the preferred internal string encoding, embedded NULLs are no longer needed and are frowned upon, but for compatibility with extensions written for 8.0, Tcl_Obj's still accept and preserve embedded NULLS in their string reps.'' [jcw] - Thanks, Don, for setting this straight. In that case, the conclusion is: if your strings can have null bytes, don't store them in an S property - use B instead. The frown carries over to Mk4tcl, MK, C, and C++. It would be nice for Mk4tcl to catch such cases and throw an exception - right now (MK 2.4.9.2), it doesn't - it truncates. [Jacob Levy] JCW, are the costs of storing B and S items basically the same? If yes, why not make the default property type for mk4tcl be B? ------ 3nov2003 [Todd Coram] A first cut at a Tuplespace server: namespace eval tuplespace_server { array set cb {} proc register_cb {chan tpat} { variable cb lappend cb($chan) $tpat return "ok" } proc deregister_cb {chan} { variable cb unset cb($chan) return "ok" } proc read {chan cmd tpat} { set res [::tuplespace::$cmd $tpat] return "ok $res" } proc write {chan tuple} { variable cb set tid [::tuplespace::out $tuple] foreach cbchan [array names cb] { foreach tpat $cb($cbchan) { if {[::tuplespace::rd $tpat] != {}} { puts $cbchan "cb $tpat" } } } return "ok $tid" } proc dispatch {chan cmd tuple} { switch -- $cmd { cb { set res [register_cb $chan $tuple] } rd - dir - in { set res [read $chan $cmd $tuple] } out { set res [write $chan $tuple] } default { set res "error invalid command!" } } puts $chan $res } } proc register_client {chan addr port} { fconfigure $chan -blocking 0 -buffering line fileevent $chan readable [list handle_input $chan] } proc handle_input {chan} { if {![eof $chan]} { if {[gets $chan data] == -1} { return; # only handle complete lines } } else { catch {close $chan} ::tuplespace_server::deregister_cb $chan return } set l [regexp -inline -- {(\w+)\s+(.*)} $data] if {[llength $l] != 3} { puts $chan "error usage: command {tuple}" } else { foreach {dummy cmd tuple} $l break puts stderr "cmd=($cmd), tuple=($tuple)" ::tuplespace_server::dispatch $chan $cmd $tuple } } socket -server [list register_client] 6667 vwait ::forever Connect to the server via telnet and try the following commands: out tom baker 56 out bob baker 54 dir ? baker ? cb ? baker ? out ginger baker ---------------------- [Todd Coram] Ugh. The above tuplespace server commits the sin of not practicing what it preaches. Here are revised procs that use the tuplespace itself to store callback information (not a Tcl array!): proc register_cb {chan tpat} { ::tuplespace::out [list cb_clients $chan $tpat] return "ok" } proc deregister_cb {chan {tpat ?}} { foreach cb [::tuplespace::dir [list cb_clients $chan $tpat]] { ::tuplespace::in [list cb_clients $chan [lindex $cb 2]] } return "ok" } proc tuple_match {tpat tid} { foreach tuple [::tuplespace::dir $tpat] { if {[::tuplespace::tid $tuple] == $tid} { return 1 } } return 0 } proc write {chan tuple} { set tid [::tuplespace::out $tuple] foreach client [::tuplespace::dir [list cb_clients ? ?]] { foreach {cb_clients chan tpat} $client break if {[tuple_match $tpat $tid]} { if {[catch {puts $chan "cb $tpat"}] != 0} { deregister_cb $chan } } } return "ok $tid" } -------------- 1/31/2004 Yikes! A long standing bug in the tuple callback code above has been fixed. The addition of tuple_match means that you don't get a callback EVERYTIME anything is written to the space (and you had a previous match that hadn't yet been read). -- [Todd Coram] ''(Yes this code is getting too unwieldy for the wiki)'' ----- [schlenk] 01/Feb/2004. I have a variant of the above code (without the notification part at the moment) running under [Tclhttpd] exposed as a [SOAP] webservice. Ask me if you are interested, it's not really polished yet. ----- [AK]: While storing the callback information in the tuplespace itself is interesting it also opens the possibility of an application screwing with the server by modifying and/or removing the relevant tuples. That is IMHO a bad thing, from a security point of view. I.e., keeping the server management information (callbacks) and the application data (tuples) separate is IMHO the better approach. And nothing prevents us from storing the data in a second tuplespace, if we wish to :). Of course, that requires rephrasing the code above as a class, so that we can have multiple independent t-spaces. ... I would also use [comm] for the communication part. Its hooks allow the restricted protocol we are running here as well. Possibly even the cross-linkage of many tuple-servers into one space. [Todd Coram]: Or... reserve any tuple beginning with cb_clients as ''system use only'' by disallowing any modifications via the ''write'' proc. A feature of my [Street Programming] Tuplespace was to reserve any tuple with a first element beginning with a '#' as a system tuple that couldn't be modified without a ''privileged'' connection. The benefit of storing tuplespace ''meta'' information as tuples themselves is that you can delegate system facilities to privileged clients outside of the tuplespace server. Maybe you would want a ''callback'' managager that could track who was getting what and displayed the frequency of queries in a graph. That would bloat the tuplespace server, but would make a nice external client. Of course, you now need some sort of password facility to make this useful. [AK]: Right. Access-Control, authentication, secrecy, the works. Because these system tuples should not be '''seen''' by a regular user either. Or otherwise system data leaks out which can help in attacks in other ways. ''Who was getting what'' ... That type of manager I would place on top of a tuplespace actually. Otherwise the tuplespace has an hardwired assumption that requests come from a network and that there is an id to be had identifying the requester. Note that this type of tracking can be of general interest beyond statistics. If you are linking several tuplespaces and each space can query others if he cannot answer a query on its own. That goes into the realm of P2P systems, or HA through replication and redundancy. Quite a lot of fun can be had. [CMcC] - I have a few questions: 1. Take only grabs one matching tuple, right? ''(Yes - [Todd Coram])'' 2. Do tuple IDs have to be unique for all time as well as at any given time for all tuples? ''(The ID value is directly related to the contents of the tuple. {Hello World} will always have the same unique ID, regardless when it was put into the space. Or, at least as unique as MD5 hashing allows - [Todd Coram] ;-)'' 3. Are duplicates allowed (duplicates for all but tuple ID, I guess) ''(No. A exact duplicate tuple will produce the same ID and squash the old one. - [Todd Coram])'' ''[escargo] 3 Apr 2005 - Wouldn't an exact duplicate not need to be written, since it's already in the data base?'' [NEM] ''1 April 2005'' - From what I can make out from this page, and what I've heard in the past, a tuplespace is a distributed set of tuples -- i.e., a (distributed) relation. Could someone summarise what the differences are between a tuplespace and a relational database? Can a tuplespace contain more than one relation? Can you do joins across tuplespaces? With the discussion above about access-control, authentication, scalability etc, this seems to be entering the territory of RDBMS's. [AK]: The tuples in a tuplespace are completely unorganized. Consider it as a bag of tuples. Any type of relational operations would have to be done by a client. The server has no idea at all about that. It might try to put tuples of identical stucture into tables, but that would be an internal optimization only, semantically there are no tables or higher structures. [NEM]: Hmm... from the no duplicates note above, it would appear to be a set of tuples, rather than a bag. A set of tuples is the mathematical definition of a relation (strictly, a graph which, together with some domain sets, forms a relation), although relations are usually "typed" in some way (hence the domain sets), which doesn't seem to be the case here. But, I see what you are saying: compared to a relational db, the tuplespace imposes much less structure on the data, and does less with it. Would it be fair to say that the primary aim of tuplespaces is distribution of data, rather than data processing? Would a comparison to [Tequila] be more appropriate? [AK]: Right, when no dups are allowed a set, not a bag. However I am usually operating under the assumption that duplicates of a tuple are allowed. IIRC this is what the other spaces allow. Re primary use, distribution is secondary I think. At least tuplespaces evolved out of parallel processing, a method to describe parallel algorithms without having to use low-level threads, and communication primitives. I.e. they are a method of synchronization. Comparison to tequila ... Possibly more appropriate. [Todd Coram]: How are duplicate tuples handled in a traditional Tuplespace? If I take {a b c} from the space does that mean that there can be another {a b c} left for the taking? Interesting. I've handled ''duplicate'' tuples in the past by putting {a b c 1} and {a b c 2} and taking {a b c ?}. I am afraid that the above implementation isn't geared toward doing true duplicates(although I am tempted to modify it to support duplicates -- actually... duplicates ARE allowed to exist in the above code, its just that once you remove one instance of the tuple, the duplicate is deleted too). ---- One of the reasons [Erlang] is so cool is that it builds in tuplespace concepts, although, to my astonishment, I have yet to come across a presentation of that line of descent. Erlang is ''all'' about sending messages between processes [[explain status of "cloud" and comparison to Erlang semantics]]; receipt is pattern-matched. ---- [NEM] ''21 Aug 2006'': A tuplespace also seems to be quite related to [Blackboard Systems]. ---- [MJ] - YATS (Yet Another TupleServer) that supports: * duplicate tuples * persistence with sqlite (create in memory db if persistence is not needed) * asynchronous and synchronous destructive and non destructive reads * running as a server * Unicode and binary tuples (thanks to sqlite3) '''Missing functionality''' * Selecting tuples with matches on anything but the tuplename (fixed in version below) tuples can now be matched on any part. String matching is used .e.g: {* test? 4} * Error checking on sqlite operations (error checking is now added). * protection from SQL injection attacks e.g. (fixed in version below) only variable bindings are being used * The Gelernter Linda paper also allows underspecified tuples to be written in the tuplespace, not only read from. For instance one should be able to output {test * 2} which could be read by a client requesting the tuple {test 1 2} as well as one requesting {test 2 2} (added by letting ''tuple_match'' do a two way match) '''Possible improvements''' * Currently the tuples are stored in one field in the database, this will probably make searching for tuples slow if the tuplespace contains a lot of tuples. Possible improvement would be to create a seperate table with the tuples. e.g. tuplespace contains tuid extra table contains: tuid field value id 0 id 1 . . id N [MJ] - This is not really useful because selecting matching tuples is non-trivial in a DB design like this. '''Tupleserver commands''' The following commands are supported by the server: * ''out tuplespace tuple'': write ''tuple'' to ''tuplespace''. * ''dir tuplespacepattern ?pattern?'': list the tuples matching ''pattern'' in tuplespaces matching ''tuplespacepattern'' if the pattern is not provided list all tuples. * ''dir'': list all tuplespaces. * ''purge tuplespace'': remove all tuples from ''tuplespace''. * ''exit'': disconnect session. * ''read space pattern'': read a tuple matching ''pattern'' from tuplespace ''space''. The tuple will remain in the space. * ''in space pattern'': read and remove a tuple matching ''pattern'' from tuplespace ''space''. * ''ain'', ''aread'': non-blocking versions of ''read'' and ''in''. Will return an empty result if no matching tuple is available. * ''ping'': check connection with server. * ''help'': returns all available server commands The server will respond with a 2 element list. The first element will contain the status of the command (ok or error). The second element will be the actual result. '''Code for the Tupleserver''' package require Tk catch { package require Iocpsock rename socket socketo rename socket2 socket } proc main {} { ts::init set server_port 4040 while { [catch {socket -server handle_socket $server_port}] } {after 1000} label .l -text "Server listening on port $server_port, F5 for console , F6 for restart" pack .l bind . {console show} bind . {exec wish $argv0 &; exit} # aliases that can be used from console interp alias {} ts_out {} ::ts::out stdout interp alias {} ts_ain {} ::ts::ain stdout interp alias {} ts_in {} ::ts::in stdout interp alias {} ts_aread {} ::ts::aread stdout interp alias {} ts_read {} ::ts::read stdout interp alias {} ts_dir {} ::ts::dir stdout interp alias {} ts_purge {} ::ts::purge stdout wm protocol . WM_DELETE_WINDOW {exit} } proc handle_socket {socket host port} { puts "Incoming connection from $host ($socket)" fconfigure $socket -buffering line -blocking no -encoding utf-8 fileevent $socket readable [list ::ts::handle_request $socket {}] } # namespace for server commands namespace eval ts { # commands supported by the tupleserver set commands {purge dir ain in aread read out exit ping help} proc init {} { # open persistent database package require sqlite3 sqlite3 db ~/ts.sqlite # sqlite3 db {} db eval {PRAGMA synchronous=OFF;} # create schema db eval {CREATE TABLE IF NOT EXISTS tuples(id INTEGER PRIMARY KEY,space,value)} db eval {DROP TABLE IF EXISTS callbacks} db eval {CREATE TABLE callbacks(id INTEGER PRIMARY KEY,type,space,pattern,socket)} # register the tuple_match function db function tuple_match [namespace code tuple_match] } # check if argument is a list. TODO replace with [string is list] in 8.5 beta proc islist {string} { # check if a string is a valid list before doing list ops if {[catch {lindex $string 0}]} { return false } else { return true } } # match a tuple with a pattern proc tuple_match {tuple pattern} { if {[llength $tuple] ne [llength $pattern]} { return 0 } foreach tup $tuple pat $pattern { if {![string match $pat $tup] && ![string match $tup $pat]} { return 0 } } return 1 } proc add_callback {socket type tuplespace pattern} { # puts "$socket - $type callback registered" db eval {INSERT INTO callbacks VALUES(NULL,$type,$tuplespace,$pattern,$socket)} } proc remove_callback {socket} { db eval {delete from callbacks where socket=$socket} } # handle incoming request on the tupleserver proc handle_request {socket data} { variable commands if {[catch {set rc [gets $socket line]}] } return if {[chan eof $socket]} { puts "connection with $socket closed" chan close $socket # remove pending callbacks for this socket remove_callback $socket return } if {$rc == -1} { # not a full line yet return } lappend data $line if {![info complete [join $data \n]]} { fileevent $socket readable [list ::ts::handle_request $socket $data] return } else { fileevent $socket readable [list ::ts::handle_request $socket {}] } set data [join $data \n] set cmd [lindex $data 0] set params [lrange $data 1 end] # puts "$socket -> $data" if {[lsearch -all $commands $cmd] eq {}} { catch {puts $socket [list error "unknown server command $cmd"]} return } set status [catch {$cmd $socket {*}$params} result] switch $status { 2 { # command resulted in a registered callback return } 1 { set result [list error $result] } 0 { set result [list ok $result] } } # puts "$socket <- $result" catch {puts $socket $result} } proc purge {_ args } { if {[llength $args] != 1} { error "wrong # of args, should be \"purge tuplespace\"" } set tuplespace [lindex $args 0] db eval {DELETE FROM tuples where space=$tuplespace} return } proc ping {_ args } { return } proc help {_ args} { variable commands return $commands } proc exit {socket} { chan close $socket # remove pending callbacks for this socket remove_callback $socket return -code return } proc dir {_ args } { set tuplespace [lindex $args 0] set pattern [lindex $args 1] switch -- [llength $args] { default { error "wrong # of args, should be \"dir ?tuplespace? ?pattern?\"" } 0 { return [db eval {SELECT DISTINCT space FROM tuples}] } 1 { set tuples [db eval {SELECT value FROM tuples WHERE space GLOB $tuplespace}] return $tuples } 2 { if {![islist $pattern]} { error "pattern should be a valid list" } return [db eval {SELECT value FROM tuples WHERE space GLOB $tuplespace AND tuple_match(value,$pattern)}] } } } proc out {_ args} { if {[llength $args] != 2} { error "wrong # of args, should be \"out tuplespace tuple\"" } set space [lindex $args 0] set tuple [lindex $args 1] if {![islist $tuple]} { error "tuple should be a valid list" } # check for pending callbacks db eval {select id,type,socket from callbacks where space=$space and tuple_match($tuple,pattern)} { # puts "$socket (callback) <- ok {$tuple}" set err [catch {puts $socket [list ok $tuple]}] db eval {delete from callbacks where id=$id} if {$type eq "in" && !$err} { # do not add the tuple when a in callback is registered and the tuple was succefully delivered return } } return [db eval {INSERT INTO tuples VALUES(NULL,$space,$tuple)}] } proc aread {socket args} { areq read $socket {*}$args } proc ain {socket args} { areq in $socket {*}$args } proc read {socket args} { req read $socket {*}$args } proc in {socket args} { req in $socket {*}$args } proc areq {type socket args} { if {[llength $args] != 2} { error "wrong # of args, should be \"a$type tuplespace pattern\"" } set tuplespace [lindex $args 0] set pattern [lindex $args 1] set id {} set value {} db eval {SELECT id,value FROM tuples where space=$tuplespace AND tuple_match(value,$pattern) LIMIT 1} { if {$type eq "in"} { db eval {DELETE FROM tuples WHERE id=$id} } } return $value } proc req {type socket args} { if {[llength $args] != 2} { error "wrong # of args, should be \"$type tuplespace pattern\"" } set tuplespace [lindex $args 0] set pattern [lindex $args 1] set resp [a$type $socket $tuplespace $pattern] if {$resp eq {}} { add_callback $socket $type $tuplespace $pattern return -level 2 -code return } { return $resp } } } main vwait forever And a client using the Tupleserver: proc connect_tuplespace {name namespace} { set s [socket localhost 4040] fconfigure $s -buffering line fileevent $s readable [list handle_command $s $name $namespace] puts $s "in $name {* * *}" } # connect to server connect_tuplespace LOGsrv logsrv puts "Client connected to server" # define commands understood by logsrv namespace eval logsrv { proc log {severity description} { puts "logging $severity: $description" } } proc handle_command {s ts ns} { gets $s resp # get next lines if response has embedded new lines while {![info complete $resp]} { append resp \n[gets $s] } set status [lindex $resp 0] set resp [lindex $resp end] if {$status eq "error"} { puts stderr $resp return } set cmd [lindex $resp 0] set args [lrange $resp 1 end] if {[catch {::${ns}::$cmd {*}$args} error]} { puts stderr $error } puts $s "in $ts {* * *}" } vwait forever If you now connect to the server running the tupleserver on port 4040 the following command: out LOGsrv {log INFO {testing info message}} will cause one (of possibly many clients listening to the LOGsrv tuplespace) to pick up the message and display the log line. [MJ] - Below an initial version (using Snit) to simplify writing of TS clients. package require snit snit::type client { variable connection {} variable active_tuplespace {} method connect {server port} { if {$connection ne {}} { return -code error "already connected close current connection first" } set connection [socket $server $port] fconfigure $connection -buffering line -encoding utf-8 return } method close {} { if {$connection eq {}} { return -code error "not connected" } close $connection set connection {} } method send {cmd args} { puts $connection [list $cmd {*}$args] return [$self response] } method response {} { gets $connection resp while {![info complete $resp]} { append resp \n[gets $connection] } return $resp } method open {tuplespace} { set active_tuplespace $tuplespace } method out {args} { if {[llength $args]==2} { $self send out {*}$args } elseif {[llength $args]==1} { $self send out $active_tuplespace {*}$args } else { return -code error "wrong # of args: should be \"$self out ?tuplespace? tuple\"" } } method in {args} { if {[llength $args]==2} { $self send in {*}$args } elseif {[llength $args]==1} { $self send in $active_tuplespace {*}$args } else { return -code error "wrong # of args: should be \"$self in ?tuplespace? tuple\"" } } } # testing set c [client ts%AUTO%] $c connect localhost 4040 $c open Test $c out {1 2 3 4} ; # takes the open TS $c out Test {4 5 6 7} ; # explicitly specify it puts [$c in {* * * *}] puts [$c in {* * * *}] console show ---- [LV] 2007 Jan 19 - Across this wiki, various styles of managing the page are practiced. In some cases, one ''authority'' code is edited and improved. In other cases, people add corrections and improvements as follow on comments. There are likely other methods in use as well. I just wonder if, for this page for instance, the original author is monitoring the page and updating some authority copy of the code, so that someone could come along and not have to make an attempt to ''merge'' changes and proc replacements into a copy of the code and then try to get it to work.... ''[escargo] 19 Jan 2007'' - This is also an issue for programs such as [wish-reaper] that extract code from the pages. Where there are multiple versions, or even code intended for separate files on the same page, grabbing all the code in one (somewhat undifferentiated) block doesn't work very well. It's a problem without a consensus solution. [LV] Good point - if I had remembered wish-reaper, I would have mentioned that problem. Thanks! ---- [AM] I am interested in distributed computing - that is: let different (instances of) programs (or threads) cooperate with each other to get the job done. I do not see, however, from the examples how you could create such a system using tuplespaces (more the practical implementation, I guess, than the principle). Anything that is easier than mutexes and their fiendish friends is welcome :). ---- [MJ] - Without knowing the details of your exact problem, it is difficult to be precise. Maybe a small example will help. Consider a simulation you want to run with a large number of parameters a possible use of tuplespaces could be: * Create 3 types of programs: 1. A program generating the simulation parameters, paramgen 1. A program performing the simulation, simu 1. A program analyzing the results, ana * Create a tuplespace. * Let the paramgen program post parameters in the tuplesspace for instance in the form {simulate parameters} * The (possibly multiple) simu instances running will check for tuples matching {simulate ....} take them from the tuplespace and start simulation * When the simulation is done, the simu program will put a tuple {result results} in the tuplespace. * The ana instance running will check for tuples matching {result ....} take them from the tuplespace and analyze the result. This will allow you to start additional simu instances on different nodes without any changes to the code (the Tuplespace will handle the coordination). If analyzing takes a long time, multiple instances could be started as well. To easily monitor progress one could image a Log process (as in the example above) that checks for tuples in the form {log src severity msg} and writes them to a central log file. All other processes can then do logging using the tuplespace. For instance the simu processes could post for instance {log simu info {simulation finished}}. Does this help? [AM] I think it does. I wll try and see how this fits into the sort of systems I am thinking about (domain decomposition for the solution of PDEs for instance where values on the boundaries of the various domains are exchanged and the processes need to wait for the results of their neighbours to become available), but this may be a small modification of the above. [AM] (Mostly as a reminder for myself) Alternative applications of tuplespaces: * Online visualisation * Computational steering * Cooperating agents (Well, these are examples I have been thinking about from time to time, nothing very original, but they could be well implemented using tuplespaces instead of via the "observer" pattern) Oh, I did find a nice tutorial on the subject, with a rather lame elaborated example (lame, because the technique used for solving the number crunching is awkward to say the least, but perhaps shows the principle better): http://www2.java.no/web/files/moter/tuplespace.pdf [MJ] - The original paper "Generative communication in Linda" by David Gelernter contains some more examples and provides some higher abstractions(like P2P communication) based on the basic primitives. Unfortunately I cannot find a free download, but the article is available from the ACM http://portal.acm.org/citation.cfm?id=2433. [MJ] - 2007-11-16 Added updated version that: * uses the ''db function'' functionality to add ''tuple_match'' as an SQLite function. This makes selecting tuples much easier (and probably faster) * changed callbacks to be stored in a table * removed open and close. Just ''out'' a tuple to a tuplespace and it will be created. ---- %|[Category Concept] | [Category interprocess communication]|%