==> On Wed, 16 May 2007 16:26:53 +0000, brad noyes <maitre@ccs.neu.edu> said: brad> On Wed, May 16, 2007 at 11:38:09AM -0400, Jeff Moyer wrote: brad> > ==> On Wed, 16 May 2007 14:53:16 +0000, brad noyes <maitre@ccs.neu.edu> said: brad> > brad> > brad> Hello All, brad> > brad> I am seeing some really slow performance regarding large files on linux. I brad> > brad> write a lot of data points from a light sensor. The stream is about 53 Mb/s and brad> > brad> i need to keep this rate for 7 minutes, that's a total of about 22Gb. I brad> > brad> can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then brad> > brad> things hit the wall and the writes to the filesystem can't keep up. The writes brad> > brad> go from 20ms in duration to 500ms. I assume the filesystem/operating system brad> > brad> is caching writes. Do you have any suggestions on how to speed up performance brad> > brad> on these writes, filesystem options, kernel options, other strategies, etc? brad> > brad> > Of course. Your data set is larger than the page cache, so when you brad> > hit the low watermark, it starts write-back. You can deal with this a brad> > few different ways, and I'll throw out the easiest ways first: brad> > 1) Get more memory brad> > 2) Get a faster disk brad> > brad> Ha :). I have 12GB of memory. Which actually brings me to another question. brad> How do i alter the per-process memory limit? I can only allocate a memory brad> buffer that is 3GB. I'd like to make use of the other 8GB left in the machine. brad> If i can double my buffer size i think i could sustain the 53MB/s for 7 brad> minutes that i need. I'm guessing that you're using a 32 bit system, is that right? 32 bit systems have the 3/1 memory split, unless you're using Ingo's 4/4 split patch (which Ubuntu doesn't carry, I think). brad> i'm using the 2.6.20 kernel from the ubuntu source tree. I recompiled it to get brad> the large memory support, up to 64GB. OK, yes, 32 bit system. brad> I was looking for some tunable vm options in sysctl, but i didn't see much that brad> made sense to me. If nothing else helps perhaps i will ask about the vm brad> options. Look under /proc/sys/vm. Documentation for these variables might be in Documentation/filesystems/proc.txt (it's not always up-to-date). But, as I said, I don't think this is the right avenue to explore. You can get more predictable results by using AIO+O_DIRECT (or maybe even O_SYNC as another mentioned). -Jeff