[SGVLUG] Distributed filesystems

[Chris Smith]

On 6/23/05, Terry Hancock <hancock at anansispaceworks.com> wrote:
> > If Speed is your problem, then you need to work on your configuration,
> > or your network. I have gotten more the 10MB per second, from a NFS
> > mount.
> I'm having slow data access because I'm sucking the data through a
> network and tying up two CPUs instead of one, and you're saying I
> need to work on my network speed?  Come on, why am I using the
> network for this at all?

Because you want to have your storage accessible on the network.
That's the usual reason people talk about distributed filesystems....

It sounds like you don't want a distributed filesystem at all, but
rather a synchronization tool like rsync or maybe something more user
friendly like union.
 
> I suspect it's slow because I'm using antiquated equipment, but so what?

It would have to be quite rediculously antiquated. Even a rediculously
high quality mp3 wouldn't require even 1mbps of bandwidth to play
smoothly. Unless you're using something as ancient as ARCnet, you
shouldn't have problems.

> NFS will *always* be slower than a local drive, no matter how I tweak
> my network.

Not at all true. For example at my work NFS is much faster in almost
all cases. A lot of times even at home NFS has advantages (the NFS
server can do caching that I might not have free memory for on the
desktop), paritcularly with my latop (damn those drives tend to be
slow).

>  And if I tweak it so that MP3s work fine, then something
> else like full-motion video is going to trash it.

This isn't a case of tweaking. Something fundamentally *broken*. The
kind of problems you are suggesting would imply NFS underforming by at
least an order of magnitude.

Yes, eventually you can push enough data through to kill the pipe, but
it's worth noting that even HDTV with multiple channels at 1920x1080
only needs 20mbps. So if you're using fast ethernet (and really if you
are using 10baseT so long as we are talking normal TV resolutiosn),
you are unlikely to hit any bandwidth barriers with even full-motion
video unless for whatever reason you need to work with uncompressed
video.

> In fact, I'm not even talking about a specific NFS setup, I'm talking about
> my general experience with NFS.  And while you just might be right about
> MP3 (after all, they are compressed), the problem is worse with WAV data
> and other formats, since they are bulkier.

Even WAV files shouldn't be a problem. If you're sampling at say
44KHz, each sample would have to be huge to cause a problem even on
dirty ol' 10baseT. You'd need like 1KB samples to come close to maxing
out 100baseT.

> I can see this kind of reaction if I'd asked the question on an NFS help
> list, but my question wasn't "I'm having trouble with NFS, how can I fix it?".

I think list members are quite legitamitely concerned that they don't
properly understand the nature of your problem. You described playing
mp3's over NFS as being a bad experience, and that doesn't make any
sense. So, we need to understand what the actual problem is, otherwise
we're going to likely recommend something that sucks just as badly or
even worse.

> I don't doubt that I could *make* NFS do it if I were determined to use
> NFS, but why should I be?

NFS is probably more established/standard than just about any other
network filesystem. AFS would be your next in line and does have a
number of advantages over NFS, but actually can be *worse* for
streaming a file over network (which is what you are describing as
being problematic) depending on circumstances.

It seems likely that your problem isn't with the protocol/filesystem,
but rather something else. Use NFS or don't use NFS, but whatever you
use is going to continue to annoy you until the problem is addressed.
 
> My point is that NFS's purpose does not seem to match well with my
> goals anymore:  I no longer need to share drives because of the need to
> conserve on storage space. The main motivation for it now is because
> of the need to make certain data available to several machines on the
> network.

Yup. Just use rsync. Forget filesystems.

-- 
Chris