[SGVLUG] Distributed filesystems

[Emerson, Tom]

> -----Original Message-----
> Behalf Of Terry Hancock
> On Wednesday 22 June 2005 05:29 pm, John Riehl wrote:
> >
> > ... is only a 100 mbit line,  and I am competing with a large 
> > quatity of other traffic.
> 
> "only a 100 mbit line" he says.  :-P
> 
> > what is your network like?  [...]
> 
> It's a mess and it's antiquated, but I am not motivated to 
> fix that if this is the only use case (and it is).

five pages in on pricewatches search results for GIGABIT ETHERNET gets you to actual networking cards (vs. mainly cables up to that point) at a price point of $15 or less.  A couple of pages later gets a "complete kit" at $40 [though only two NICs and I presume a "crossover" cable -- no switch or hub] and the lowest priced gigabit SWITCH is on the next page at $54 (netgear, no less!) so price shouldn't be a barrier to "upgrading" [though admittedly you never said it was...]

OTOH, I imagine that those bargain-basement cards will actually be WORSE for your environment (driver issues notwithstanding) as I suspect these will be (effectively) "winmodem" style NICS, i.e., they make heavy use of your own CPU to do the work that you bought them for in the first place [think of it this way: if you were do hook up the interrupt line to the clock line, you'd overclock your CPU... ;) ]

> Copying files from one system to another
> is not inordinately slow, but anything that requires fast 
> disk performance obviously suffers from use over NFS.

I'd look into your interrupt rates, cpu load, and other overhead tasks -- while NFS imposes it's own penalty, "simple copying" doesn't do much more than read and write the disk, but you may be running at 99% of your CPU(s) [one, the other, or both...] -- in other words, you've got a little room to spare.  Once you start decoding streams ON TOP OF the rest of the tasks, you may be trying to hit 103%, and hence you start stuttering (or whatever it is that makes playing music "uninspiring" by your standards)

> No doubt this has to do with using
> an ancient 10 mbit hub and driver problems on at least one 
> ethernet card,

Hmmm... "driver problems"... could it be that you have network cards "new enough" that they are little more than glorified serial buffers that really hog your CPU usage?  You might actually benefit from an OLDER network card that "does more of the work" [i.e., one that would have fetched a premium when new -- I might have some like this kicking around; remind me just before the next meeting and I'll see what I can scrape up]

> MP3 is actually a bad example.  WAV files are more likely to 
> cause problems.
> Certainly full-motion video will (which I don't currently 
> use, but I can imagine I will).

MP3s are compressed, so while there is less traffic "on the wire", there is more work for the CPU playing the file.  WAV files are essentially raw data, so this is the opposite: more bits going through the pipe, but once they arrive, they can practically be routed straight to the audio card [hmmm... a thought arose: what card and how are the drivers for the card?  what's your CPU usage like when playing "locally" stored files?  If playing a file from a local drive chews up a bit of your CPU, then your comment on "copying files is reasonably quick, but mp3s still suck" makes more sense -- when copying files, you probably aren't playing anything on your sound card...]

As to data rates: when I was doing some video editing, "raw" video ended up being 6 megabytes/second (full uncompressed frames at 30fps) like mp3's, avi's and similar file formats "are compressed", so bits-on-wire are reduced at the cost of CPU to decode -- you might be looking at 1 to 3 megabytes per second.  "on a good day", a 10megabit network with no other congestion or traffic /should/ be able to pump out a megabyte per second (theoretically as much as 1.2 megabytes, but reality rarely ever approaches theory here...)  By reducing frame rates or sizes, that number drops significantly (which is how you can get "acceptable" video over DSL, which is still an order of magnatude slower than even your "antiquated bottlenecked mess")

> I would hope it is clear that using *ANY* system which accesses data
> on a remote system's hard drive is going to be slow compared 
> to getting it from a local drive.

Oh, I'll bet I can dig up a hard drive that is slower than your network connection [after all, I've got "stuff" I've collected for 20+ years... ;) mind you, the capacity might not be large enough to support a swap file, much less a file you might actually "use"...)  OTOH, if you can get full use of a gigabit pipe, you may find that your "local" drive ends up being the tortoise.

final thought: the folks that produce Cinelarra (a.k.a. "broadcast2000") also maintain something they call the firehose (or something like that...) which is a method of connecting multiple USB-2.0 and/or FIREWIRE ports between two computers to get an outrageous "combined" network speed -- i.e., 5 usb2.0 ports connected between machines should get you 2gigabits or more (isn't usb2 rated at 400mbs, or is that firewire?)