--- brad noyes <maitre@ccs.neu.edu> wrote:
Do you have any suggestions on how to speed up performance on these writes, filesystem options, kernel options, other strategies, etc?
How are you doing the writes, using fwrite(3), write(2), or mmap(2)? I've seen dramatic speedups reading large files using memory-mapped I/O.
- I have also tried spooling over several files (a la multiple volumes) but i see no difference in performance. In fact, i think this actually hinders performance a bit.
Do you mean multiple physical spindles or drives here? (I'd expect a slow down using multiple partitions on the same physical drive.)
- I keep my own giant memory buffer where all the data is stored and then it is written to disk in a background thread. This helps, but i run out of space in the buffer before i finish taking data.
With memory-mapped I/O you could just maintain the buffer and have the kernel page it out to disk for you as needed; no need to shuffle stuff around with a background thread. --Andre ____________________________________________________________________________________Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting