- WLUG - wlug.mailman3.com

Re: [Wlug] large file performance on linux
by Jamie Guinan 16 May '07

16 May '07

On Wed, 16 May 2007, brad noyes wrote: > Hello All, > > I am seeing some really slow performance regarding large files on linux. I > write a lot of data points from a light sensor. The stream is about 53 Mb/s and > i need to keep this rate for 7 minutes, that's a total of about 22Gb. I > can sustain 53Mb/s pretty well until the file grows to over 1Gb or so, then > things hit the wall and the writes to the filesystem can't keep up. The writes > go from 20ms in duration to 500ms. I assume the filesystem/operating system > is caching writes. Do you have any suggestions on how to speed up performance > on these writes, filesystem options, kernel options, other strategies, etc? > > Things I have tried: > - I have tried this on a ext3 file system as well as an xfs filesystem > with the same result. > > - I have also tried spooling over several files (a la multiple volumes) > but i see no difference in performance. In fact, i think this actually > hinders performance a bit. > > - I keep my own giant memory buffer where all the data is stored and > then it is written to disk in a background thread. This helps, but > i run out of space in the buffer before i finish taking data. Hi, Do you mean MB/s as in MegaBytes/second? Lower-case "b" implies "bits" to me, but I suspect you meant bytes. :) Anyway, if you know your total data set is going to exceed your system memory, which will largely get used as page cache, you might as well open the output with O_SYNC and write straight out. Then the big buffer in the writer thread can go away, and you can just queue chunks between the input and writer. A good experiment would be to run the writer thread with dummy input data and see what kind of throughput you get. You could try ext2 (no journalling), or tweaking some of the journalling options in ext3 (data=journal/ordered/writeback, see "man mount"). If you have enough CPU bandwidth, and your input stream has enough redundancy, you could gzip it before writing, which might reduce your output bandwidth requirements. Hope this helps, keep us posted. -Jamie > > Thanks, > -- Brad > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug >

2 1

SUN Ultra1 booting (still, sorry)
by ken jones 08 May '07

08 May '07

I used genkernel to compile my 2.6.20-gentoo-r5 kernel. After much thrashing I got it to boot. All looked good until the very end when it claimed that it "Failed to set the user font" announced the name of my machine and gave me the login prompt. It would not respond to any keyboard input. After a minute or two it printed a line about "disabled for 5 minutes" and went permanently dead. See below. Any ideas? ---------------snip from almost successful boot--------------- * Starting lo [ !! * Bringing up lo * 127.0.0.1/8 * Adding routes [ ok * 127.0.0.0/8 ... * Initializing random number generator ... [ ok NIT: Entering runlevel: 3 [ ok * Setting user font ... putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented putfont: PIO_FONT: Function not implemented * Failed to set user font * Mounting network filesystems ... [ !! * Starting local ... [ ok [ ok This is kjones-sun2.unknown_domain (Linux sparc64 2.6.20-gentoo-r5) 21:42:42 kjones-sun2 login: INIT: Id "s0" respawning too fast: disabled for 5 minutes

1 0

[OT] ethics
by Mark Richards 02 May '07

02 May '07

Hi Matt. Matt Shields wrote: > I haven't checked out CPSR, so I don't know what they are, but you > can't enforce your beliefs on others. This comes down to personal > choice and where you want to work, or maybe working for the government > is the only available choice at the time. Some people have no problem I agree. What I was suggesting is the possibility for one or more people to make a statement as to where they choose to allow the fruit of their work to go, and/or how it is to be used. That's not about enforcing beliefs, but about sharing (and refining) them. The problem I have with open source licensing as it applies to this discussion I raised is that it does not discriminate as to use. I had once posed the possibility of creating a modified license that would prohibit (any) government from using a certain work, but this is both way too complicated and of course begs the question as to exactly what in a project can be legally restricted in this manner. After all, any project (including the one I am working on now) is comprised of a host of others, all bringing in license terms that do not have such a restriction. Attorneys will be needed to figure out if the idea of a more restrictive version of a GPL is even possible. > working for the government. If you want to change how the government > works because you don't like what they're doing, you elect officials > to change policy and petition those officials to change things. You Well, I think that the effectiveness of the system itself is a whole other debate that would go way off topic here. :) I do find the idea of a national referendum on important issues like blowing up other nations a pretty good idea. Has immediacy. But again, swaying way off here. > don't come up with a society that boycotts working for the government > because there's plenty of other people willing to take the job. There are plenty of people. But, as in organizations like Union of Concerned Scientists (the "concerned", I think, puts the intensity of the overall problem mildly), people have come together and made quite a difference. If they hadn't tried then the issues they raise may well have remained in obscurity while scientists of conscience masked their understanding and "concern" with lots of rolaids. /m

1 0

Re: [Wlug] JOBS: Seeking a summer intern in Linux administration
by Clint Moyer 02 May '07

02 May '07

At least that's only the job listing. If you _apply_ for a unix job with a MS Word resume, you get auto-deleted. ---- Jared Greenwald <greenwaldjared(a)gmail.com> wrote: ============= Nothing says Linux like a Word doc... On 5/1/07, Andy Stewart <andystewart(a)comcast.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > HI Everybody, > > If you have an interest in this position, please reply directly to the > recruiter: > > Mischke, Ali <Ali.Mischke(a)ironmountain.com> > > Please note: when I forward email such as this, I'll be putting "JOBS:" > or "JOB:" at the beginning of the subject. You may thus filter at will. > > Thanks, > > Andy > > === > > Does WLUG have a student forum? I am the Engineering Internship > Coordinator at Iron Mountain Digital in Southborough, MA, and am trying > to get the word out about a paid internship we have for this summer. We > are looking for a student (junior, senior or graduate student) with > strong Linux skills to help administer an Engineering lab of 125-175 > Linux servers. I have attached the job description and would appreciate > if you could help get the word out to any student members of your > organization. > > Thanks in advance! > > Ali Mischke > Senior Manager, Analysis and Design > Iron Mountain Digital > > 120 Turnpike Road > Southborough, MA 01772 > Phone: (508) 808-7347 > ali.mischke(a)ironmountain.com > > === > > > - -- > Andy Stewart, Founder > Worcester Linux Users' Group (http://www.wlug.org) > Chelmsford Linux Meetup Group (http://linux.meetup.com/393) > Amateur Radio: KB1OIQ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2 (GNU/Linux) > Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org > > iD8DBQFGN7l3Hl0iXDssISsRAiGVAJ9PcqnYps7gc+2SZLc9cfXAz6mhWwCcCW4Y > 7c062GdyqG5Rw5nDEwT009I= > =0Jq9 > -----END PGP SIGNATURE----- > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > > > _______________________________________________ Wlug mailing list Wlug(a)mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug

2 1

JOBS: Seeking a summer intern in Linux administration
by Andy Stewart 02 May '07

02 May '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 HI Everybody, If you have an interest in this position, please reply directly to the recruiter: Mischke, Ali <Ali.Mischke(a)ironmountain.com> Please note: when I forward email such as this, I'll be putting "JOBS:" or "JOB:" at the beginning of the subject. You may thus filter at will. Thanks, Andy === Does WLUG have a student forum? I am the Engineering Internship Coordinator at Iron Mountain Digital in Southborough, MA, and am trying to get the word out about a paid internship we have for this summer. We are looking for a student (junior, senior or graduate student) with strong Linux skills to help administer an Engineering lab of 125-175 Linux servers. I have attached the job description and would appreciate if you could help get the word out to any student members of your organization. Thanks in advance! Ali Mischke Senior Manager, Analysis and Design Iron Mountain Digital 120 Turnpike Road Southborough, MA 01772 Phone: (508) 808-7347 ali.mischke(a)ironmountain.com === - -- Andy Stewart, Founder Worcester Linux Users' Group (http://www.wlug.org) Chelmsford Linux Meetup Group (http://linux.meetup.com/393) Amateur Radio: KB1OIQ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGN7l3Hl0iXDssISsRAiGVAJ9PcqnYps7gc+2SZLc9cfXAz6mhWwCcCW4Y 7c062GdyqG5Rw5nDEwT009I= =0Jq9 -----END PGP SIGNATURE-----

4 4

MySQL Server Hardware
by John Westcott, IV 30 Apr '07

30 Apr '07

Howdy All, I work in a place that has traditionally used Sun Sparc hardware running Solaris to host a production MySQL environment. We are going to be replacing our database servers this year and I am trying to determine if I want to maintain the norm or if I want to try to change. I am interested to see what other people use for hardware/OS for a production MySQL installation that requires a high amount of uptime. Thanks in advance for any replies. -John Westcott

6 5

I'd like to see Windows do this....
by Tal Cohen 30 Apr '07

30 Apr '07

Hi Folks, I recently ran across an interesting situation that I wanted to share this group. A customer of ours called us, letting us know that our server in their data center was beeping. We tried to ssh into the box, but kept getting connection refused. I asked the customer to hook up a monitor and keyboard to the box, and try a simple TTY. Sure enough, he was unable to log in. It turns out that the hard drive that holds the OS (Crux Linux) failed. Now, that in and of itself is not very interesting. What is interesting is that our software kept running even without the OS drive functioning. Our software resides on a separate drive. Since it was already loaded into memory and almost never needs to touch the OS drive, it just kept going.and going..and going.. Tal

3 5

PGP Key Signing Party
by Eric Martin 29 Apr '07

29 Apr '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This upcoming meeting we are going to have a PGP Key Signing Party. For those of you who don't know what PGP is, allow me to briefly explain. This is all base off of the GPG Key Signing HOWTO (http://www.rubin.ch/pgp/kspa/gpg-party.en.html) which I suggest for further reading. Those who do know can feel free to skip down to the next paragraph. PGP stands for Pretty Good Privacy, and is a Public Key Infrastructure that allows us to communicate securely and deal with data in a secure fashion. There is a public key that one would typically upload to a server and a private key that you hold on to. With the private key you can decrypt / sign, and with the public key you can encrypt / verify emails and documents. Key signatures come into play when you validate that a specific key belongs to a specific person and extend the web of trust. Here's what needs to be done BEFORE the meeting on your computer, or on a secure terminal (e.g. not in a computer lab on your campus). I'll post directions both for GPG (Gnu Privacy Guard) and Kgpg (the KDE front - -end to gpg for all of those who like GUIs). 1. Generate a Key Pair / Verify your key won't expire soon gpg --gen-key (Default values of DSA/Elgamal is fine and the default of 2048 is good.) I recommend 3 - 5 years for a key lifespan (how long the key will be good for) 1a) for those of you wishing to use Kgpg, goto Keys -> Generate Key Pair and fill in the dialog box Once you create the New key pair, check the Save As box under _Revocation Certificate_ to save a Revocation Certificate (see 2) 1c) For those of you who already have a key, make sure yours isn't about to expire (like mine did). If it has expired, you can extend the life by $ gpg --edit-key <key-id> and typing expire at the command prompt next, generate a new key-pair and sign your new key with your old key so anyone who trusted your old one will trust the new one. 2) Generate a revocation certificate Revocation certificates are for situations where you forget your passphrase, the key becomes comprised, etc you can revoke your key. $ gpg --output revoke.asc --gen-revoke <key-id> Save this in multiple places, print a copy etc in case you need it. 2GUI) If you didn't create a revocation cert in Kgpg, just right click on your new key and select Revoke Key. when the box pops up only check Save As 3) Upload your key to the Designated server for this party (pgp.mit.edu) $ gpg --send-key --keyserver=pgp.mit.edu <key-id> 3GUI) Right Click on the Key, select Export Public Keys, select Default Key Server (make sure it's pgp.mit.edu under Settings) 4) Email me your key's fingerprint so I can add it to the list $ gpg --fingerprint <key-id> Copy and Paste that into an email to the coordinator (me, Eric Martin). 5) print out a copy of that same output and bring it to the meeting. I'll make up a check list and bring them to the meeting for everyone to use. Example output from Step 4: pub 1024D/A9413B9F 2007-04-09 [expires: 2010-04-08] Key fingerprint = D1C4 086E DBB5 C18E 6FDA B215 6A25 7174 A941 3B9F uid Eric J. Martin <eric.joshua.martin(a)gmail.com> uid Eric J. Martin <freak4uxxx(a)gmail.com> sub 2048g/01F81B8E 2007-04-09 [expires: 2010-04-08] my <key-id> is A9413B9F (which is always the same as my last 8 hex digits of my fingerprint). If like me you don't always remember your key-id you can use your email address and it will still work. My full key information is: Key ID: A9413B9F Key Type: DSA Hex Fingerprint: D1C4 086E DBB5 C18E 6FDA B215 6A25 7174 A941 3B9F Key Size: 1024 (The DSA key is always 1024 bits. The ElGamal Encryption key is 2048 bits in my case, but isn't needed for the keysigning party.) What you should bring to this party 1. Yourself, no exceptions 2. Two forms of positive photo ID - driver's license and passport are good 3. Key Id, Key Type, Hex Fingerprint and Key size info ***Please print out your own copy of your info to compare against mine when you arrive *** 4. Pen / Pencil What you Should Not Bring 1. Computer I want to thank Chuck Anderson for helping me out. If you have any questions please let me know. Eric -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGMBvKaiVxdKlBO58RAsCVAJ9cA488l+SybdxD+rgqNWGi/U9tUQCeMvH8 UAhMTFVRG5SbVVn5jqbVwqg= =BSzn -----END PGP SIGNATURE-----

2 1

MySQL Server Hardware
by joel d 27 Apr '07

27 Apr '07

I'll take a stab at this one. At work our MySQL database contains product info and indexing data. The reasoning for designing and taking this outside of the business application is the business app. doesn't do web communication well (limited functionality and at times spotty app reliability). The business app doesn't matter (and I bet no one has heard of it anyway). Our design spec was hearty at a HP DL 385 with dual AMD CPUs (dual core I think) 5GB of RAM and around 7 internal 15k drives in RAID 5. Every bit of hardware is HP/Compaq. Without starting a distro thread we picked Red Hat AS V4. This combo has served us well and I am sure *many* other distros would do this at least as well. For us our biggest corporate reason for R.H. was most of our application vendors suported R.H in one way or another (a vote for getting good at only one skillset/distro). During application & DB development we considered uptime and ordered a second identical box mainly for a total server failure of any sort. They run very well as MySQL master and slave. In the year it has been up we are very happy with this combination. Feel free to forward any specific questions (direct or via the group). Hope this helps, Joel ============================= Date: Tue, 24 Apr 2007 14:59:55 -0400 From: "John Westcott, IV" <John.Westcott(a)tufts.edu> Subject: [Wlug] MySQL Server Hardware To: wlug(a)mail.wlug.org Message-ID: <20070424145955.3hftuclhdwg0ck4s(a)webmail.tufts.edu> Content-Type: text/plain; charset=ISO-8859-1; format="flowed" Howdy All, I work in a place that has traditionally used Sun Sparc hardware running Solaris to host a production MySQL environment. We are going to be replacing our database servers this year and I am trying to determine if I want to maintain the norm or if I want to try to change. I am interested to see what other people use for hardware/OS for a production MySQL installation that requires a high amount of uptime. Thanks in advance for any replies. -John Westcott

1 0

2 things
by Mike Leo 24 Apr '07

24 Apr '07

We talked about this at one point: http://arstechnica.com/news.ars/post/20070422-child-porn-case-shows-that-an… also.....wlug.org is down?

1 0