Xector

(Started by TV)

A Xector is an associative vector or list, coming from the Connection Machine, which is a highly parallel and advanced machine from the 80s, which is based on a great number of computing nodes (e.g. 64k) which are list processing elements with high-throughput, low-latency connections, and which can be used to run LISP in parallel.

The Xector is probably the most important structure on the machine, using it for dynamic indirections.

I would be an interesting idea to have Xectors in Tcl, since Tcl is related to LISP; that shouldn't be too hard, and maybe fun to try some parallel implementation (I think the CM was used for social research purposes with large datasets), let's say on Nvidia Cuda or so. Anyway if I'd work in a university and be in charge, I guess that I'd think that fun and decent research.

Google books has this interesting quote: [L1 ]

I'll think of an implementation in Tcl or maybe a C one for efficiency comparison, and a good working example.

Lars H: So, an Xector is essentially an array meant to support loop-level parallelisation? (One could also envision it as a dict, but generating string representations for data distributed over 2**16 processors would be rather awkward.) I suspect existing Tcl idioms for working with arrays cannot be parallelised to that level (threads are huge compared to the tasks that would be handed out in this kind of setting), so there's a lot of work on designing APIs ahead if you undertake this.

One might also wonder how different from a relational database the end result is likely to be. I suspect a database table is pretty close to an Xector — at least a RDBMS is probably the kind of common software that is best posed to take advantage of the kind of parallelisation that xectors are meant to harness.

TV Well, the connection machine would allow indeed stuff like database associations I think. The interesting part for me was primarily what you dubbed in other words attention for fine grained parallelism, but it's also like when I do block execution schedule computations in BWise: when I take a certainly not astronomic number of blocks (16x16 or so) in some not too far out connection constellation there are probably great gains possible using dedicated C code to speed up computations. Of course this has to do with how a language works, and how caching constructs work, etc.

I don't know for sure whether the dict idea is similar. I'm sure a list of tuples (except for special cases defined in the Connection Machine LISP definition) is a comparison. Indeed I too find that a comparison with a database with no fixed record structure. I suppose discussions like Performance of Various Stack Implementations are often possibly and even justified in computer science. A parallel implementation of tcl in some form at least would seem logical enough.

I think the idea of the associativeness and not too course grained parallelism comes from my idea that computer hardware knowledge in combination with the subjects leads to finding xectors as they existed and the subject for a language like tcl (and how I historically know it) an interesting idea, possibly leading to actually new directions.

TV I thought I'd make this a page by itself: Parallel Tcl .