[geeks] Versioning FIlesystem

[vance at neurotica.com]

On Fri, 23 May 2003, David A de Gruyl wrote:

> >Doesn't all the automation go out the window if you decide to mount the
> >filesystem on another OS?  I mean, if the versioning is in the spec for
> >the filesystem, then wouldn't a driver for that filesystem in any new
> >OS would have to support it?
>
> If a new filesystem type was created, then the kernel level junk would
> have to be introduced into any OS that intended to read said filesystem.
> But, if it is just UFS or similar with additional cruft, then another OS
> could read it, without the automatic versioning taking place.

What I'm saying is that if the support for versioning is *specified as
required* for the new filesystem, then the kernel level stuff would just
have to be introduced in the new filesystem driver.

> I am sort of torn on which way I would prefer.  I think that the fs
> support should be included in the target OSes, on the one hand.  But the
> ability to read the files on another (unmodified) OS is also valuable.

There's just too much that could go wrong if you allow a user to edit
stuff willy-nilly with no concern about maintaining version integrity.

> If ffs/ext3/or something is used as a base, and the additional
> information is housed in files with the name <name>:0 or something, then
> the userland programs and the system calls related to files would have
> to change, but versioning could be done on a per file basis.  somehow.

I'm not saying that an existing filesystem shouldn't be used as a base.
I'm just saying that it should be intentionally rendered incompatible to
keep people from using, say, an ffs driver to munge versioned files.

> I think that it might be better to make it a mount option for
> filesystem, and if that option is set, then the oddball stuff happens.
> If it is not, then the extra files would look like extra files.  I would
> still like to have extra information in the file attributes for the
> maximum number of revisions to keep, etc.  And to make the revisions be
> kept in the smallest portable diff format rather than as full copies.
> But editing an older version should be editing the applied diff, not the
> diff itself.  Blech.

That's how VMS does it.  It stores complete revisions of files.  Not just
diffs.  I don't agree with making it a mount option, however.  It would
make it far too easy to circumvent version integrity.

> Well, this idea came up from "The Unix Hater's Handbook".  And I noticed
> that it was still fairly unaddressed.  I am starting to understand why.
> Even blessed rm would have to have a new option to actually remove
> files, and not just send them in to old version stasis.  And all the
> fileutils, and find would have to be able to read the version
> information.

What version information specifically?  You would only have a bunch of
files.  find would only need to be modified to match all versions of a
file.  rm would only have to be modified to only remove the latest
version.  Plus, a purge command would probably be necessary.

> In order to make this an integrated part of the operating system, a new
> paradigm for the unix filesystem operations would have to develop.  You
> would have to remember that rm does not work the same on every platform,
> and that you can't always recover old versions.  I think that I (for
> one) could wrap my head around this one.

It's simple.  It's only necessary to have rm remove the last version.
That's simple enough to do with several different approaches.

> And I would want it to work over NFS so that for a first pass the local
> machine would provide the most recent version, but would still provide
> versioning on the local disk.  Maybe a "newnfs" could be set up to
> provide a versioning nfs mount option.  Allowing the remote machine
> access to the less recent version.

That's probably also not bad to do.

> Is it worth the effort to even look into?  Which OS should I look at --
> limited choice of Open/Net/FreeBSD and Linux.  I do not have source code
> licenses for non-free OS.  I am leaning towards OpenBSD, because I
> rarely have spaghetti code problems there.

I would use NetBSD, simply because of the wide hardware support.  You'll
reach a very wide audience very quickly.  It's definitely worth doing, and
I'll help in any way I can.

Peace...  Sridhar