A possible tclvfs - CMcC 20041029
Storage of metadata (data about data) is an important and undersupported facility of file systems.
Tcl supports the [file attributes] command to store some system-defined metadata, and tclvfs permits this to be redefined and extended.
What this allows is vfs which intercepts the [file attributes] command and loads/stores attribute (name,value) pairs in some parallel store. Such a vfs would be stackable over other filesystems, so as to provide generic metadata for any filesystem. This would allow applications to simply assume that they could store arbitrary metadata.
The question is what form of parallel store would be best?
- parallel hierarchy - metadata is stored in a file hierarchy which parallels the structure of the original, so there's a mapping from ${path} -> ${path}.metadata which contains an array dump of name->value pairs. CONS: expensive/slow, PROS: persistence for free.
- hidden directory - metadata stored in special files hidden in a per-directory .metadata directory. CONS: expensive/slow, invasive (less so than invasive-interpolation, below), permissions problems, PROS: faster than parallel-hierarchy, metadata is joined with with data (less so than invasive-interpolation)
- persistent array - metadata is stored in an array $metadata($path) as an [array get] form per file, to be loaded/stored once, then accessed from memory. Could use tie for persistence. CONS: doesn't scale well, persistence needs work PROS: fast.
- metakit - metadata stored in a metakit file, loaded/stored as required. CONS: not pure tcl, slower than persistent-array, PROS: scales, faster than parallel-FS.
- invasive interpolation - metadata stored at the head of each file in (say) RFC822-style name:value lines. Similar to Mac resource forks. PROS: some data (e.g. emails) are already of this form, some data (e.g. caches) can be coerced to this form CONS: invasive, wrecks general files for general (non-tcl) use.
- (add more here)
[Category VFS]