- WLUG - wlug.mailman3.com

Re: [Wlug] large file performance on linux

by Andre Lehovich

--- brad noyes <maitre(a)ccs.neu.edu> wrote: > Do you have any suggestions on how to speed up performance > on these writes, filesystem options, kernel options, other strategies, etc? How are you doing the writes, using fwrite(3), write(2), or mmap(2)? I've seen dramatic speedups reading large files using memory-mapped I/O. > - I have also tried spooling over several files (a la multiple volumes) > but i see no difference in performance. In fact, i think this actually > hinders performance a bit. Do you mean multiple physical spindles or drives here? (I'd expect a slow down using multiple partitions on the same physical drive.) > - I keep my own giant memory buffer where all the data is stored and > then it is written to disk in a background thread. This helps, but > i run out of space in the buffer before i finish taking data. With memory-mapped I/O you could just maintain the buffer and have the kernel page it out to disk for you as needed; no need to shuffle stuff around with a background thread. --Andre ____________________________________________________________________________________Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting

May 16, 2007

Re: [Wlug] large file performance on linux

by Jeff Moyer

==> On Wed, 16 May 2007 19:19:45 +0000, brad noyes <maitre(a)ccs.neu.edu> said: brad> On Wed, May 16, 2007 at 02:40:38PM -0400, Jeff Moyer wrote: brad> > ==> On Wed, 16 May 2007 12:59:54 -0400, Jeff Moyer <jmoyer(a)redhat.com> said: brad> > brad> > Jeff> Look under /proc/sys/vm. Documentation for these variables might be brad> > Jeff> in Documentation/filesystems/proc.txt (it's not always up-to-date). brad> > Jeff> But, as I said, I don't think this is the right avenue to explore. brad> > Jeff> You can get more predictable results by using AIO+O_DIRECT (or maybe brad> > Jeff> even O_SYNC as another mentioned). brad> > brad> > One other thing worth mentioning is that you should be doing I/O in brad> > large block sizes. What size are you currently using for your write brad> > buffers? brad> > brad> i'm writing in 16777216 byte chunks. That happens to be evenly divisible by 512 brad> for the O_DIRECT flag. However every time i try to use that flag the file gets brad> created, but nothing gets written. I've been looking online for an example. Are you aligning your buffers on a 512 byte boundary? Man posix_memalign. brad> I don't know if this means anything, but i ran brad> hdparm -T /dev/sdb1 brad> Timing cached reads: 1369MB in 2.00 seconds == 698.14MB/sec brad> hdparm -T --direct /dev/sdb1 brad> Timing O_DIRECT cached reads: 136 in 2.00 seconds == 66.54MB/sec brad> It really seems that 53MB/s shouldn't be hard. I have fairly heavy hardware, brad> scsi320 in a raid1 configuration. So long as nothing else is going on at the time. =) -Jeff

May 16, 2007

Re: [Wlug] large file performance on linux

by Karl Hiramoto

brad noyes wrote: > On Wed, May 16, 2007 at 02:40:38PM -0400, Jeff Moyer wrote: > >> ==> On Wed, 16 May 2007 12:59:54 -0400, Jeff Moyer <jmoyer(a)redhat.com> said: >> >> Jeff> Look under /proc/sys/vm. Documentation for these variables might be >> Jeff> in Documentation/filesystems/proc.txt (it's not always up-to-date). >> Jeff> But, as I said, I don't think this is the right avenue to explore. >> Jeff> You can get more predictable results by using AIO+O_DIRECT (or maybe >> Jeff> even O_SYNC as another mentioned). >> >> One other thing worth mentioning is that you should be doing I/O in >> large block sizes. What size are you currently using for your write >> buffers? >> >> > i'm writing in 16777216 byte chunks. That happens to be evenly divisible by 512 > for the O_DIRECT flag. However every time i try to use that flag the file gets > created, but nothing gets written. I've been looking online for an example. > > I don't know if this means anything, but i ran > hdparm -T /dev/sdb1 > Timing cached reads: 1369MB in 2.00 seconds == 698.14MB/sec > hdparm -T --direct /dev/sdb1 > Timing O_DIRECT cached reads: 136 in 2.00 seconds == 66.54MB/sec > > > It really seems that 53MB/s shouldn't be hard. I have fairly heavy hardware, > scsi320 in a raid1 configuration. > > -- Brad > _______________________________________________ > That is CACHED read.. your uncached write will be slower.. run a bonnie++ benchmark on your disks. You might want to switch to raid 0 or raid 1+0. And use a 64 bit kernel to be able to allocate more than 3G of ram. -- -- Karl Hiramoto http://karl.hiramoto.org/ US VOIP: (1) 603.966.4448 Spain Casa (34) 951.273.347 Spain Mobil (34) 617.463.826 Yahoo_IM = karl_hiramoto GTalk= karl.hiramoto [at] gmail [d0t] com -- Everyone complains of his memory, no one of his judgement.

May 16, 2007

Re: [Wlug] large file performance on linux

by brad noyes

On Wed, May 16, 2007 at 03:19:17PM -0400, John Stoffel wrote: > >>>>> "brad" == brad noyes <maitre(a)ccs.neu.edu> writes: > > brad> I am seeing some really slow performance regarding large files > brad> on linux. I write a lot of data points from a light sensor. The > brad> stream is about 53 Mb/s and i need to keep this rate for 7 > brad> minutes, that's a total of about 22Gb. I can sustain 53Mb/s > brad> pretty well until the file grows to over 1Gb or so, then things > brad> hit the wall and the writes to the filesystem can't keep up. The > brad> writes go from 20ms in duration to 500ms. I assume the > brad> filesystem/operating system is caching writes. Do you have any > brad> suggestions on how to speed up performance on these writes, > brad> filesystem options, kernel options, other strategies, etc? > > You've already had a good bunch of suggestions, but I've got some > questions on your hardware. > i don't mind answering more questions :). > - cpu? dual xeons (not sure if they are hyper threaded or dual core) cpu MHz: 3067.044 cache size: 512 KB > - memory - 12gb I know > - disk(s) SCSI hard drives, i believe they are SCSI 320. I have tried in a raid1 as well as stand alone. > - RAID setup at all? tried using RAID1. I'm afraid to try RAID0 b/c the data is pretty vital, but i may try it. > > One way to get more performance would be to add another disk or two > and to stripe your data between them. This assumes you have enough > PCI bus bandwidth available as well. You don't say how you're > capturing the light sensor data, but it's obviosly not over a serial > port or some other slow device. Network? So if you've got 53 > Mbyte/second comming into the system, and another 53Mbytes/second > writing out to disk, then you're starting to get close to the > 132Mbytes/sec bandwidth of the PCI bus. > Good point. Its not network, but it is still on the PCI bus. It doesn't seem that bandwidth is really a problem b/c it works great for the first minute. Once the file grows past 1GB the writes are extremely slow. Like i said above i might try the RAID0 configuration. > Finding a motherboard with two or more PCI busses would help. Or > something with PCI-E busses. It all depends on your budget and the > data acquisition tool you're using. > If i were to redesign this system i would really like to use PXI, which was really meant for this sort of thing. I'm really retrofitting my design to to something different than intended. Thanks -- Brad

May 16, 2007

Re: [Wlug] large file performance on linux

by brad noyes

On Wed, May 16, 2007 at 02:40:38PM -0400, Jeff Moyer wrote: > ==> On Wed, 16 May 2007 12:59:54 -0400, Jeff Moyer <jmoyer(a)redhat.com> said: > > Jeff> Look under /proc/sys/vm. Documentation for these variables might be > Jeff> in Documentation/filesystems/proc.txt (it's not always up-to-date). > Jeff> But, as I said, I don't think this is the right avenue to explore. > Jeff> You can get more predictable results by using AIO+O_DIRECT (or maybe > Jeff> even O_SYNC as another mentioned). > > One other thing worth mentioning is that you should be doing I/O in > large block sizes. What size are you currently using for your write > buffers? > i'm writing in 16777216 byte chunks. That happens to be evenly divisible by 512 for the O_DIRECT flag. However every time i try to use that flag the file gets created, but nothing gets written. I've been looking online for an example. I don't know if this means anything, but i ran hdparm -T /dev/sdb1 Timing cached reads: 1369MB in 2.00 seconds == 698.14MB/sec hdparm -T --direct /dev/sdb1 Timing O_DIRECT cached reads: 136 in 2.00 seconds == 66.54MB/sec It really seems that 53MB/s shouldn't be hard. I have fairly heavy hardware, scsi320 in a raid1 configuration. -- Brad

May 16, 2007

Re: [Wlug] large file performance on linux

by John Stoffel

>>>>> "brad" == brad noyes <maitre(a)ccs.neu.edu> writes: brad> I am seeing some really slow performance regarding large files brad> on linux. I write a lot of data points from a light sensor. The brad> stream is about 53 Mb/s and i need to keep this rate for 7 brad> minutes, that's a total of about 22Gb. I can sustain 53Mb/s brad> pretty well until the file grows to over 1Gb or so, then things brad> hit the wall and the writes to the filesystem can't keep up. The brad> writes go from 20ms in duration to 500ms. I assume the brad> filesystem/operating system is caching writes. Do you have any brad> suggestions on how to speed up performance on these writes, brad> filesystem options, kernel options, other strategies, etc? You've already had a good bunch of suggestions, but I've got some questions on your hardware. - cpu? - memory - 12gb I know - disk(s) - RAID setup at all? One way to get more performance would be to add another disk or two and to stripe your data between them. This assumes you have enough PCI bus bandwidth available as well. You don't say how you're capturing the light sensor data, but it's obviosly not over a serial port or some other slow device. Network? So if you've got 53 Mbyte/second comming into the system, and another 53Mbytes/second writing out to disk, then you're starting to get close to the 132Mbytes/sec bandwidth of the PCI bus. Finding a motherboard with two or more PCI busses would help. Or something with PCI-E busses. It all depends on your budget and the data acquisition tool you're using. John

May 16, 2007

Re: [Wlug] large file performance on linux

by Jeff Moyer

==> On Wed, 16 May 2007 12:59:54 -0400, Jeff Moyer <jmoyer(a)redhat.com> said: Jeff> Look under /proc/sys/vm. Documentation for these variables might be Jeff> in Documentation/filesystems/proc.txt (it's not always up-to-date). Jeff> But, as I said, I don't think this is the right avenue to explore. Jeff> You can get more predictable results by using AIO+O_DIRECT (or maybe Jeff> even O_SYNC as another mentioned). One other thing worth mentioning is that you should be doing I/O in large block sizes. What size are you currently using for your write buffers? -Jeff

May 16, 2007

Re: [Wlug] large file performance on linux

by Jeff Moyer

==> On Wed, 16 May 2007 16:26:53 +0000, brad noyes <maitre(a)ccs.neu.edu> said: brad> On Wed, May 16, 2007 at 11:38:09AM -0400, Jeff Moyer wrote: brad> > ==> On Wed, 16 May 2007 14:53:16 +0000, brad noyes <maitre(a)ccs.neu.edu> said: brad> > brad> > brad> Hello All, brad> > brad> I am seeing some really slow performance regarding large files on linux. I brad> > brad> write a lot of data points from a light sensor. The stream is about 53 Mb/s and brad> > brad> i need to keep this rate for 7 minutes, that's a total of about 22Gb. I brad> > brad> can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then brad> > brad> things hit the wall and the writes to the filesystem can't keep up. The writes brad> > brad> go from 20ms in duration to 500ms. I assume the filesystem/operating system brad> > brad> is caching writes. Do you have any suggestions on how to speed up performance brad> > brad> on these writes, filesystem options, kernel options, other strategies, etc? brad> > brad> > Of course. Your data set is larger than the page cache, so when you brad> > hit the low watermark, it starts write-back. You can deal with this a brad> > few different ways, and I'll throw out the easiest ways first: brad> > 1) Get more memory brad> > 2) Get a faster disk brad> > brad> Ha :). I have 12GB of memory. Which actually brings me to another question. brad> How do i alter the per-process memory limit? I can only allocate a memory brad> buffer that is 3GB. I'd like to make use of the other 8GB left in the machine. brad> If i can double my buffer size i think i could sustain the 53MB/s for 7 brad> minutes that i need. I'm guessing that you're using a 32 bit system, is that right? 32 bit systems have the 3/1 memory split, unless you're using Ingo's 4/4 split patch (which Ubuntu doesn't carry, I think). brad> i'm using the 2.6.20 kernel from the ubuntu source tree. I recompiled it to get brad> the large memory support, up to 64GB. OK, yes, 32 bit system. brad> I was looking for some tunable vm options in sysctl, but i didn't see much that brad> made sense to me. If nothing else helps perhaps i will ask about the vm brad> options. Look under /proc/sys/vm. Documentation for these variables might be in Documentation/filesystems/proc.txt (it's not always up-to-date). But, as I said, I don't think this is the right avenue to explore. You can get more predictable results by using AIO+O_DIRECT (or maybe even O_SYNC as another mentioned). -Jeff

May 16, 2007

Re: [Wlug] Reminder: Upcoming PGP Key Signing Party

by Minkov, Ross

Eric, It's past the cutoff time as defined by Andy now. Can you share with us how many key fingerprints you have received so far? Thanks, Ross -----Original Message----- From: wlug-bounces(a)mail.wlug.org [mailto:wlug-bounces@mail.wlug.org] On Behalf Of Andy Stewart Sent: Wednesday, May 09, 2007 20:35 To: Worcester Linux Users Group Subject: [Wlug] Reminder: Upcoming PGP Key Signing Party -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 HI Everybody, At our next WLUG meeting on Wednesday, May 16th, we will have a PGP Key Signing Party hosted by Chuck Anderson and Eric Martin. Eric has previously sent out instructions, so please be sure to follow them, and direct any questions to Eric or Chuck. I have posted those instructions on the WLUG home page (see description for May WLUG Meeting). I suggest a cutoff of noon on Wednesday, May 16th, for sending your information to Eric. This will give Eric time to collect the necessary information prior to the meeting. Dust off those passports - you'll need them! Later, Andy - -- Andy Stewart, Founder Worcester Linux Users' Group (http://www.wlug.org) Chelmsford Linux Meetup Group (http://linux.meetup.com/393) Amateur Radio: KB1OIQ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGQmjGHl0iXDssISsRAr1rAJ91nX/DNXHjRie+dDYyVjrq8ePq2wCeN8DC Q+SoklLvcW6MfOaCGuvwKwU= =kUPV -----END PGP SIGNATURE----- _______________________________________________ Wlug mailing list Wlug(a)mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug

May 16, 2007

Re: [Wlug] large file performance on linux

by brad noyes

On Wed, May 16, 2007 at 11:38:09AM -0400, Jeff Moyer wrote: > ==> On Wed, 16 May 2007 14:53:16 +0000, brad noyes <maitre(a)ccs.neu.edu> said: > > brad> Hello All, > brad> I am seeing some really slow performance regarding large files on linux. I > brad> write a lot of data points from a light sensor. The stream is about 53 Mb/s and > brad> i need to keep this rate for 7 minutes, that's a total of about 22Gb. I > brad> can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then > brad> things hit the wall and the writes to the filesystem can't keep up. The writes > brad> go from 20ms in duration to 500ms. I assume the filesystem/operating system > brad> is caching writes. Do you have any suggestions on how to speed up performance > brad> on these writes, filesystem options, kernel options, other strategies, etc? > > Of course. Your data set is larger than the page cache, so when you > hit the low watermark, it starts write-back. You can deal with this a > few different ways, and I'll throw out the easiest ways first: > 1) Get more memory > 2) Get a faster disk > Ha :). I have 12GB of memory. Which actually brings me to another question. How do i alter the per-process memory limit? I can only allocate a memory buffer that is 3GB. I'd like to make use of the other 8GB left in the machine. If i can double my buffer size i think i could sustain the 53MB/s for 7 minutes that i need. > If those are not options, then you can tweak your application by using > AIO and O_DIRECT. This will allow you to drive your disk queue depths > a bit further and avoid the page cache. Check the man pages for > io_setup, io_submit, and io_getevents to get started. > I'll check out these options and man pages. > brad> Things I have tried: > brad> - I have tried this on a ext3 file system as well as an xfs filesystem > brad> with the same result. > > You may not want to use a journalled file system. If you must, > though, with ext3 you could try running with the data=writeback > option. > yup. I'll check this option out. > brad> - I have also tried spooling over several files (a la multiple volumes) > brad> but i see no difference in performance. In fact, i think this actually > brad> hinders performance a bit. > > I'm not sure I fully understand what you mean. Are you saying you > write to separate physical volumes, > Not physical volumes, but different files. By the end of the data acquisition i will end up with the files: data.01, data.02, data.03 ... etc. Each file is a 1GB in size or whatever i set the limit to be. The reason i did this is because i thought that as the file grows larger there are several layers of indirection in the inode to get to the actual data blocks on disk; and perhaps that might hinder performance. > and that you don't see any performance increase from doing so? > Correct. I don't see any improvement. At least no measurable performance improvement in the kind of rates i'm dealing with. > brad> - I keep my own giant memory buffer where all the data is stored and > brad> then it is written to disk in a background thread. This helps, but > brad> i run out of space in the buffer before i finish taking data. > > Right, this is exactly what happens in the OS. ;) Speaking of which, > you don't mention which kernel you are using. Could you please > provide that information? There are a few vm tunables that you could > try tweaking, but I really don't think they will help if your data set > is larger than memory. We can explore that option, though, if you > like. > i'm using the 2.6.20 kernel from the ubuntu source tree. I recompiled it to get the large memory support, up to 64GB. I was looking for some tunable vm options in sysctl, but i didn't see much that made sense to me. If nothing else helps perhaps i will ask about the vm options. > > p.s. In your head, is Mb Megabit or Megabyte? > the latter. Jamie already pointed this typo out to me :). Perhaps this time around my unit abbreviations are correct. Thanks for your input. I'll keep the list posted. -- Brad

May 16, 2007

Re: [Wlug] large file performance on linux

by Klein, Richard

> -----Original Message----- > From: Jeff Moyer > > p.s. In your head, is Mb Megabit or Megabyte? I always thought that lower-case b meant bit and upper-case B meant byte, but the rest of the world doesn't seem to care what I think. -- Rich

May 16, 2007

Re: [Wlug] large file performance on linux

by brad noyes

On Wed, May 16, 2007 at 11:44:13AM -0400, Jamie Guinan wrote: > On Wed, 16 May 2007, brad noyes wrote: > Hi, > > Do you mean MB/s as in MegaBytes/second? Lower-case "b" implies > "bits" to me, but I suspect you meant bytes. :) > Yes, your assumption is correct. I never quite understood the capitalization conventions when it came to computers. > Anyway, if you know your total data set is going to exceed your system > memory, which will largely get used as page cache, you might as well > open the output with O_SYNC and write straight out. Then the big > buffer in the writer thread can go away, and you can just queue chunks > between the input and writer. > > A good experiment would be to run the writer thread with dummy input > data and see what kind of throughput you get. > > You could try ext2 (no journalling), or tweaking some of the > journalling options in ext3 (data=journal/ordered/writeback, see "man > mount"). > great idea. I forgot about tune2fs and mount options. thanks. > If you have enough CPU bandwidth, and your input stream has enough > redundancy, you could gzip it before writing, which might reduce your > output bandwidth requirements. > There isn't enough redundancy in the data. its essentially random data so it won't compress well at all. > Hope this helps, keep us posted. > Thanks for the input. -- brad

May 16, 2007

Re: [Wlug] large file performance on linux

by Jamie Guinan

On Wed, 16 May 2007, brad noyes wrote: > Hello All, > > I am seeing some really slow performance regarding large files on linux. I > write a lot of data points from a light sensor. The stream is about 53 Mb/s and > i need to keep this rate for 7 minutes, that's a total of about 22Gb. I > can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then > things hit the wall and the writes to the filesystem can't keep up. The writes > go from 20ms in duration to 500ms. I assume the filesystem/operating system > is caching writes. Do you have any suggestions on how to speed up performance > on these writes, filesystem options, kernel options, other strategies, etc? > > Things I have tried: > - I have tried this on a ext3 file system as well as an xfs filesystem > with the same result. > > - I have also tried spooling over several files (a la multiple volumes) > but i see no difference in performance. In fact, i think this actually > hinders performance a bit. > > - I keep my own giant memory buffer where all the data is stored and > then it is written to disk in a background thread. This helps, but > i run out of space in the buffer before i finish taking data. Hi, Do you mean MB/s as in MegaBytes/second? Lower-case "b" implies "bits" to me, but I suspect you meant bytes. :) Anyway, if you know your total data set is going to exceed your system memory, which will largely get used as page cache, you might as well open the output with O_SYNC and write straight out. Then the big buffer in the writer thread can go away, and you can just queue chunks between the input and writer. A good experiment would be to run the writer thread with dummy input data and see what kind of throughput you get. You could try ext2 (no journalling), or tweaking some of the journalling options in ext3 (data=journal/ordered/writeback, see "man mount"). If you have enough CPU bandwidth, and your input stream has enough redundancy, you could gzip it before writing, which might reduce your output bandwidth requirements. Hope this helps, keep us posted. -Jamie > > Thanks, > -- Brad > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug >

May 16, 2007

Re: [Wlug] large file performance on linux

by Jeff Moyer

==> On Wed, 16 May 2007 14:53:16 +0000, brad noyes <maitre(a)ccs.neu.edu> said: brad> Hello All, brad> I am seeing some really slow performance regarding large files on linux. I brad> write a lot of data points from a light sensor. The stream is about 53 Mb/s and brad> i need to keep this rate for 7 minutes, that's a total of about 22Gb. I brad> can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then brad> things hit the wall and the writes to the filesystem can't keep up. The writes brad> go from 20ms in duration to 500ms. I assume the filesystem/operating system brad> is caching writes. Do you have any suggestions on how to speed up performance brad> on these writes, filesystem options, kernel options, other strategies, etc? Of course. Your data set is larger than the page cache, so when you hit the low watermark, it starts write-back. You can deal with this a few different ways, and I'll throw out the easiest ways first: 1) Get more memory 2) Get a faster disk If those are not options, then you can tweak your application by using AIO and O_DIRECT. This will allow you to drive your disk queue depths a bit further and avoid the page cache. Check the man pages for io_setup, io_submit, and io_getevents to get started. brad> Things I have tried: brad> - I have tried this on a ext3 file system as well as an xfs filesystem brad> with the same result. You may not want to use a journalled file system. If you must, though, with ext3 you could try running with the data=writeback option. brad> - I have also tried spooling over several files (a la multiple volumes) brad> but i see no difference in performance. In fact, i think this actually brad> hinders performance a bit. I'm not sure I fully understand what you mean. Are you saying you write to separate physical volumes, and that you don't see any performance increase from doing so? brad> - I keep my own giant memory buffer where all the data is stored and brad> then it is written to disk in a background thread. This helps, but brad> i run out of space in the buffer before i finish taking data. Right, this is exactly what happens in the OS. ;) Speaking of which, you don't mention which kernel you are using. Could you please provide that information? There are a few vm tunables that you could try tweaking, but I really don't think they will help if your data set is larger than memory. We can explore that option, though, if you like. For now, my suggestion is to try using AIO with the open flag O_DIRECT. This will require you to align your data on 512 byte boundaries (and the size of the I/Os has to be a multiple of 512 as well). If you need any help converting your app, feel free to contact me off-list. -Jeff p.s. In your head, is Mb Megabit or Megabyte?

May 16, 2007

large file performance on linux

by brad noyes

Hello All, I am seeing some really slow performance regarding large files on linux. I write a lot of data points from a light sensor. The stream is about 53 Mb/s and i need to keep this rate for 7 minutes, that's a total of about 22Gb. I can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then things hit the wall and the writes to the filesystem can't keep up. The writes go from 20ms in duration to 500ms. I assume the filesystem/operating system is caching writes. Do you have any suggestions on how to speed up performance on these writes, filesystem options, kernel options, other strategies, etc? Things I have tried: - I have tried this on a ext3 file system as well as an xfs filesystem with the same result. - I have also tried spooling over several files (a la multiple volumes) but i see no difference in performance. In fact, i think this actually hinders performance a bit. - I keep my own giant memory buffer where all the data is stored and then it is written to disk in a background thread. This helps, but i run out of space in the buffer before i finish taking data. Thanks, -- Brad

May 16, 2007

Re: [Wlug] PGP Key Signing Party

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Eric Martin wrote: > Just a reminder that we are having a PGP Key Signing Party tomorrow for > WLUG. If you're name isn't listed below, I do not have your info (and > you should get it to me if you want to be on the list). I know Andy > said noon tomorrow but I'll take stuff until 4pm. Don't forget your two > forms of ID and I'll see you tomorrow. > > Eric Martin > Gregory M. Avedissian > Walt Sawyer > Alexander Pennace > Doug Mildram > Keith Wright > Alex Weiss > Andrew M. Stewart > James Philip Gray > > -Eric HI Eric, Thanks for sending out the meeting reminder! I'll see you all on Wednesday evening with my shiny new passport. :-) Later, Andy - -- Andy Stewart, Founder Worcester Linux Users' Group (http://www.wlug.org) Chelmsford Linux Meetup Group (http://linux.meetup.com/393) Amateur Radio: KB1OIQ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGSkvGHl0iXDssISsRApWmAJ9HpIERvKHryJXnIeBo8ZfoJnqBygCeIF2I tQxc9xAb6JDHR8dbSTc3yNA= =v7Xx -----END PGP SIGNATURE-----

May 16, 2007

PGP Key Signing Party

by Eric Martin

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Just a reminder that we are having a PGP Key Signing Party tomorrow for WLUG. If you're name isn't listed below, I do not have your info (and you should get it to me if you want to be on the list). I know Andy said noon tomorrow but I'll take stuff until 4pm. Don't forget your two forms of ID and I'll see you tomorrow. Eric Martin Gregory M. Avedissian Walt Sawyer Alexander Pennace Doug Mildram Keith Wright Alex Weiss Andrew M. Stewart James Philip Gray - -Eric -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGSj+vaiVxdKlBO58RAmxgAJ0QaQlNCSrw58e80VeQjjGL/GKsOwCdErcn vMjPw9eiQD4zXEzjNWPBKa4= =KiW+ -----END PGP SIGNATURE-----

May 15, 2007

Re: [Wlug] Reminder: Upcoming PGP Key Signing Party

by Chuck Anderson

On Thu, May 10, 2007 at 08:33:31AM -0400, Mike Perry wrote: > Just an FYI. If I go to "wlug.org" I see April being the next meeting. > However, if I go to "www.wlug.org" I see the upcoming meeting. Clear your browser cache and reload the page.

May 10, 2007

Re: [Wlug] Reminder: Upcoming PGP Key Signing Party

by Mike Perry

Just an FYI. If I go to "wlug.org" I see April being the next meeting. However, if I go to "www.wlug.org" I see the upcoming meeting. ~Mike On Wed, 2007-05-09 at 20:35 -0400, Andy Stewart wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > HI Everybody, > > At our next WLUG meeting on Wednesday, May 16th, we will have a PGP Key > Signing Party hosted by Chuck Anderson and Eric Martin. Eric has > previously sent out instructions, so please be sure to follow them, and > direct any questions to Eric or Chuck. I have posted those instructions > on the WLUG home page (see description for May WLUG Meeting). > > I suggest a cutoff of noon on Wednesday, May 16th, for sending your > information to Eric. This will give Eric time to collect the necessary > information prior to the meeting. > > Dust off those passports - you'll need them! > > Later, > > Andy > > - -- > Andy Stewart, Founder > Worcester Linux Users' Group (http://www.wlug.org) > Chelmsford Linux Meetup Group (http://linux.meetup.com/393) > Amateur Radio: KB1OIQ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2 (GNU/Linux) > Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org > > iD8DBQFGQmjGHl0iXDssISsRAr1rAJ91nX/DNXHjRie+dDYyVjrq8ePq2wCeN8DC > Q+SoklLvcW6MfOaCGuvwKwU= > =kUPV > -----END PGP SIGNATURE----- > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug

May 10, 2007

Reminder: Upcoming PGP Key Signing Party

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 HI Everybody, At our next WLUG meeting on Wednesday, May 16th, we will have a PGP Key Signing Party hosted by Chuck Anderson and Eric Martin. Eric has previously sent out instructions, so please be sure to follow them, and direct any questions to Eric or Chuck. I have posted those instructions on the WLUG home page (see description for May WLUG Meeting). I suggest a cutoff of noon on Wednesday, May 16th, for sending your information to Eric. This will give Eric time to collect the necessary information prior to the meeting. Dust off those passports - you'll need them! Later, Andy - -- Andy Stewart, Founder Worcester Linux Users' Group (http://www.wlug.org) Chelmsford Linux Meetup Group (http://linux.meetup.com/393) Amateur Radio: KB1OIQ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGQmjGHl0iXDssISsRAr1rAJ91nX/DNXHjRie+dDYyVjrq8ePq2wCeN8DC Q+SoklLvcW6MfOaCGuvwKwU= =kUPV -----END PGP SIGNATURE-----

May 10, 2007

SUN Ultra1 booting (still, sorry)

by ken jones

I used genkernel to compile my 2.6.20-gentoo-r5 kernel. After much thrashing I got it to boot. All looked good until the very end when it claimed that it "Failed to set the user font" announced the name of my machine and gave me the login prompt. It would not respond to any keyboard input. After a minute or two it printed a line about "disabled for 5 minutes" and went permanently dead. See below. Any ideas? ---------------snip from almost successful boot--------------- * Starting lo [ !! * Bringing up lo * 127.0.0.1/8 * Adding routes [ ok * 127.0.0.0/8 ... * Initializing random number generator ... [ ok NIT: Entering runlevel: 3 [ ok * Setting user font ... putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented * Failed to set user font * Mounting network filesystems ... [ !! * Starting local ... [ ok [ ok This is kjones-sun2.unknown_domain (Linux sparc64 2.6.20-gentoo-r5) 21:42:42 kjones-sun2 login: INIT: Id "s0" respawning too fast: disabled for 5 minutes

May 8, 2007

[OT] ethics

by Mark Richards

Hi Matt. Matt Shields wrote: > I haven't checked out CPSR, so I don't know what they are, but you > can't enforce your beliefs on others. This comes down to personal > choice and where you want to work, or maybe working for the government > is the only available choice at the time. Some people have no problem I agree. What I was suggesting is the possibility for one or more people to make a statement as to where they choose to allow the fruit of their work to go, and/or how it is to be used. That's not about enforcing beliefs, but about sharing (and refining) them. The problem I have with open source licensing as it applies to this discussion I raised is that it does not discriminate as to use. I had once posed the possibility of creating a modified license that would prohibit (any) government from using a certain work, but this is both way too complicated and of course begs the question as to exactly what in a project can be legally restricted in this manner. After all, any project (including the one I am working on now) is comprised of a host of others, all bringing in license terms that do not have such a restriction. Attorneys will be needed to figure out if the idea of a more restrictive version of a GPL is even possible. > working for the government. If you want to change how the government > works because you don't like what they're doing, you elect officials > to change policy and petition those officials to change things. You Well, I think that the effectiveness of the system itself is a whole other debate that would go way off topic here. :) I do find the idea of a national referendum on important issues like blowing up other nations a pretty good idea. Has immediacy. But again, swaying way off here. > don't come up with a society that boycotts working for the government > because there's plenty of other people willing to take the job. There are plenty of people. But, as in organizations like Union of Concerned Scientists (the "concerned", I think, puts the intensity of the overall problem mildly), people have come together and made quite a difference. If they hadn't tried then the issues they raise may well have remained in obscurity while scientists of conscience masked their understanding and "concern" with lots of rolaids. /m

May 2, 2007

Re: [Wlug] JOBS: Seeking a summer intern in Linux administration

by Eric Martin

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Clint Moyer wrote: > At least that's only the job listing. If you _apply_ for a unix job with a MS Word resume, you get auto-deleted. > > That's why I always use a pdf. Anybody can read it and it comes out exactly the way I want it to. - -Eric -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGOLn2aiVxdKlBO58RAulcAJwOCEo+CSvMqwDLcqoKbrpG5NSigACfRYqC VxRNlqYJDZUTEI2zzqNBtaw= =F7md -----END PGP SIGNATURE-----

May 2, 2007

Re: [Wlug] JOBS: Seeking a summer intern in Linux administration

by Clint Moyer

At least that's only the job listing. If you _apply_ for a unix job with a MS Word resume, you get auto-deleted. ---- Jared Greenwald <greenwaldjared(a)gmail.com> wrote: ============= Nothing says Linux like a Word doc... On 5/1/07, Andy Stewart <andystewart(a)comcast.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > HI Everybody, > > If you have an interest in this position, please reply directly to the > recruiter: > > Mischke, Ali <Ali.Mischke(a)ironmountain.com> > > Please note: when I forward email such as this, I'll be putting "JOBS:" > or "JOB:" at the beginning of the subject. You may thus filter at will. > > Thanks, > > Andy > > === > > Does WLUG have a student forum? I am the Engineering Internship > Coordinator at Iron Mountain Digital in Southborough, MA, and am trying > to get the word out about a paid internship we have for this summer. We > are looking for a student (junior, senior or graduate student) with > strong Linux skills to help administer an Engineering lab of 125-175 > Linux servers. I have attached the job description and would appreciate > if you could help get the word out to any student members of your > organization. > > Thanks in advance! > > Ali Mischke > Senior Manager, Analysis and Design > Iron Mountain Digital > > 120 Turnpike Road > Southborough, MA 01772 > Phone: (508) 808-7347 > ali.mischke(a)ironmountain.com > > === > > > - -- > Andy Stewart, Founder > Worcester Linux Users' Group (http://www.wlug.org) > Chelmsford Linux Meetup Group (http://linux.meetup.com/393) > Amateur Radio: KB1OIQ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2 (GNU/Linux) > Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org > > iD8DBQFGN7l3Hl0iXDssISsRAiGVAJ9PcqnYps7gc+2SZLc9cfXAz6mhWwCcCW4Y > 7c062GdyqG5Rw5nDEwT009I= > =0Jq9 > -----END PGP SIGNATURE----- > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > > > _______________________________________________ Wlug mailing list Wlug(a)mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug

May 2, 2007

Re: [Wlug] JOBS: Seeking a summer intern in Linux administration

by Keith Wright

> From: "Scott Venier" <scott(a)scooter.cx> > > Andy Stewart wrote: > > > Jared Greenwald wrote: > > > > > Nothing says Linux like a Word doc... > > > > You noticed that too, eh?? :-) > > > Unfortunately, that's the sad reality of the difference between > Engineering and HR. As long as HR is doing the hiring, having a Word doc > version of the old resume is always a good idea. :) I forwarded that to a WPI Management professor who is always looking for MQP sponsors, and doesn't hate Word documents. Now that I have moved it elsewhere I can clean the abomination from my hard drive. -- Keith

May 2, 2007