userspace filesystems

Once a year I do a "filemanager" poll: I check on the status of two-pane file managers, looking for something that resembles Total Commander, only in Unix.

This year I ran into gnome-commander while looking for a file-manager that would support libgnomevfs (with its exotic sshfs, smbfs and so on).

It strikes me odd, the fact that there is so much code duplication between user-space filesystem projects. There are three levels of userspace filesystem implementations on Linux software right now:
  • Virtual filesystems implemented within the file-manager (such as a virtual filesystem for tar.gz contents)
  • Userspace filesystems exported via libraries (and object broker services), to applications (such as gnomevfs sshfs module, or samba kio-slave)
  • Kernel-assisted userspace filesystems, accessible by all applications, but requiring manual setup (see FUSE-related projects)
Is there a way to unite these projects so that people don't have to write the same code over and over again? Shouldn't a newly mounted userspace filesystem, be available to all applications and not just the one that generated it?

More on this later.


vvas said...

I think that, in order to understand this phenomenon, one first has to look at the cause. In my opinion, the cause for all this is that the classic model for mounting filesystems in Linux, where only root can mount a fixed set of compiled-in filesystems, is fundamentally broken. It's just too inflexible for modern desktop needs. Of course, the Linux kernel developers have always been reluctant to change this, because they like to follow the old UNIX ways. Well fuck that. The Hurd did it right when it allowed "translators" to be attached pretty much anywhere on the file namespace, but yeah, prod me when it's finished.

So what were userspace developers to do? Well, as usual, if the OS doesn't provide a feature, you write it yourself. Thus the different levels of implementations you describe were different takes for tackling this same problem. Now, the reason all these exist is because they didn't all come at once (and you list them in the correct chronological order), but emerged through a process of increasing sophistication:

- At first file managers such as the Midnight Commander needed to handle things like .tar.gz files transparently, as you mention, so they implemented some support internally. Not to mention things like emacs's ftp support that go even farther in the past.

- Then the desktop environments came, with the attitude that a feature-rich framework should be provided to all the applications of the DE. And so the relevant support was ripped from the file managers and became gnome-vfs / kioslaves / etc. (By the way, Miguel De Icaza was responsible for a large part of both MC and GNOME, so I wouldn't be surprised if it's the MC codebase that led to gnome-vfs).

- Finally people became more ambitious, and wanted to provide such features to all applications, without additional layers. And thus was born FUSE, which kills the Linux limitations in a roundabout way. IIRC it was quite an uphill battle for its developers to get it merged in the mainline kernel.

It should be obvious here which one I see as the best solution. :^) BTW you blame FUSE of requiring manual setup, but I think that as it gets integrated with distributions / DEs / etc. this won't be a problem eventually.

modal_echoes said...

I'm not saying 'FUSE requires manual setup' as a negative comment torwards FUSE. FUSE is the way to go for monolithic kernels. What we need now is FUSE to announce mounts, umounts, mountpoints, session-info & permissions through HAL->DBUS and listen for incoming mount requests through DBUS.

Hence some sort of middleware is needed and I have the feeling this should be a freedesktop project.