June 2005 - WLUG - wlug.mailman3.com

Re: [Wlug] HOWTO debug hard lockups

by Jeff Moyer

==> Regarding Re: [Wlug] HOWTO debug hard lockups; Andy Stewart <andystewart(a)comcast.net> adds: andystewart> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 andystewart> Jeff Moyer wrote: >> ==> Regarding Re: [Wlug] HOWTO debug hard lockups; Andy Stewart >> <andystewart(a)comcast.net> adds: >> andystewart> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> andystewart> Eric Martin wrote: >>>> In reguards to seeing messages on minicom, what logger do you have >>>> installed? Can you just shovel kernel messages over to ttyS01? eg >>>> sudo tailf /var/log/kernel ttyS01 >> andystewart> I did that per your suggestion and it works just fine. >> Umm, it works just fine so long as the system is up and running. You >> really want the kernel to be printing directly to the console. >> Otherwise, when the system locks up and the NMI fires, you still won't >> get your messages over the serial console. >> >> Andy, it looked to me like you had things configured properly. Indeed, >> I use a very similar setup all of the time. The only difference is it >> works for me. (I hate that phrase). >> >> I'm not sure what to suggest at this point. If you do an alt-sysrq-h, >> does that at least print out over the serial console? >> >> -Jeff >> andystewart> HI Jeff, andystewart> If I do an alt-sysrq-h, I get the one line help on the serial andystewart> console. If I do alt-sysrq-m to show memory, I see "SysRq : andystewart> Show Memory" ont the serial console, but nothing more (unless andystewart> I use Eric's trick). If I go to runlevel 1, alt-sysrq-m has andystewart> the entire output on the serial console. You have all of the luck. How about trying netconsole? It's easy enough to setup. On the host which will receive the logs, do something like this: nc -u -l -p 6666 On the problematic system, load the netconsole module: modprobe netconsole netconsole=@<ip-of-this-machine>/<ethX>,6666@<ip-of-server>/<MAC-address-of-server> You can try this or not, depending on whether you still experience crashes. It sounds like the latest kernel fixed things right up for you, right? Oh well, good to have in your bag of tricks, I guess. -Jeff

June 21, 2005

Re: [Wlug] HOWTO debug hard lockups

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeff Moyer wrote: > ==> Regarding Re: [Wlug] HOWTO debug hard lockups; Andy Stewart <andystewart(a)comcast.net> adds: > > andystewart> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > > andystewart> Eric Martin wrote: >>>In reguards to seeing messages on minicom, what logger do you have >>>installed? Can you just shovel kernel messages over to ttyS01? eg sudo >>>tailf /var/log/kernel ttyS01 > > andystewart> I did that per your suggestion and it works just fine. > > Umm, it works just fine so long as the system is up and running. You > really want the kernel to be printing directly to the console. Otherwise, > when the system locks up and the NMI fires, you still won't get your > messages over the serial console. > > Andy, it looked to me like you had things configured properly. Indeed, I > use a very similar setup all of the time. The only difference is it works > for me. (I hate that phrase). > > I'm not sure what to suggest at this point. If you do an alt-sysrq-h, does > that at least print out over the serial console? > > -Jeff > HI Jeff, If I do an alt-sysrq-h, I get the one line help on the serial console. If I do alt-sysrq-m to show memory, I see "SysRq : Show Memory" ont the serial console, but nothing more (unless I use Eric's trick). If I go to runlevel 1, alt-sysrq-m has the entire output on the serial console. Later, Andy - -- Andy Stewart, Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCuI64Hl0iXDssISsRAgYoAJwO2tnZkw6ox+URA61Lh16lyUhE4wCeOedC wQJZTh9CV8pUFPfEHkDYjw8= =H2Yg -----END PGP SIGNATURE-----

June 21, 2005

Yellow Dog Linux

by Stephen Ketcham

I've just set up my Mac G4 Powerbook to boot Yellow Dog Linux 4.0.1. (It dual boots, so it also runs Mac OS X.) is there anybody else on the list running Yellow Dog? Thanks, Steve

June 21, 2005

Re: [Wlug] HOWTO debug hard lockups

by Jeff Moyer

==> Regarding Re: [Wlug] HOWTO debug hard lockups; Andy Stewart <andystewart(a)comcast.net> adds: andystewart> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 andystewart> Eric Martin wrote: >> In reguards to seeing messages on minicom, what logger do you have >> installed? Can you just shovel kernel messages over to ttyS01? eg sudo >> tailf /var/log/kernel ttyS01 andystewart> I did that per your suggestion and it works just fine. Umm, it works just fine so long as the system is up and running. You really want the kernel to be printing directly to the console. Otherwise, when the system locks up and the NMI fires, you still won't get your messages over the serial console. Andy, it looked to me like you had things configured properly. Indeed, I use a very similar setup all of the time. The only difference is it works for me. (I hate that phrase). I'm not sure what to suggest at this point. If you do an alt-sysrq-h, does that at least print out over the serial console? -Jeff

June 21, 2005

Re: [Wlug] HOWTO debug hard lockups

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Eric Martin wrote: > In reguards to seeing messages on minicom, what logger do you have > installed? Can you just shovel kernel messages over to ttyS01? eg sudo > tailf /var/log/kernel ttyS01 I did that per your suggestion and it works just fine. Thanks! Andy - -- Andy Stewart, Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD4DBQFCuIeiHl0iXDssISsRAvu0AJ9meIbihDK9rwqbJ0NFOpV/t/dNkACYqFTA T+XthIh4VSCYVDrFOjSzfg== =gsn3 -----END PGP SIGNATURE-----

June 21, 2005

Re: [Wlug] HOWTO debug hard lockups

by Eric Martin

In reguards to seeing messages on minicom, what logger do you have installed? Can you just shovel kernel messages over to ttyS01? eg sudo tailf /var/log/kernel ttyS01 On 6/20/05, Andy Stewart <andystewart(a)comcast.net> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > HI everybody, > > Perhaps I have found some light in this tunnel. > > I upgraded to the 2.6.12 kernel (yeah, I know, that just changed the > entire experiment) and turned off "CPU freq scaling" and "PCI hotplug > support" since I didn't think I needed either of those. I built the > 2.6.12 kernel and played beer_golf 5 times in a row and the system DID > NOT lockup. I'm not sure why those features were enabled in the first > place. (I should probably go back to 2.6.11.11 <http://2.6.11.11> and > disable these two > features and see if that has any positive effect.) > > I'm far too much of a pessimist to claim victory, but perhaps my luck is > turning for the better. My fingers are still crossed. I am (extremely) > cautiously optimistic. > > I thought I'd mention this in case somebody reads this thread in the > future and maybe, just maybe, this tidbit helps. Besides, I've read a > lot of threads where the problem is described in detail, but the answer > is never forthcoming(!). > > Later, > > Andy > > > - -- > Andy Stewart, Founder > Worcester Linux Users' Group > Worcester, MA, USA > http://www.wlug.org > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.5 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > > iD8DBQFCtkacHl0iXDssISsRAvlGAJ9eVRN8NVKF7sWQtvIfdp4sqLi7BgCfXquw > ZP3/O9spJsGk+B2aty2RMDg= > =BGSi > -----END PGP SIGNATURE----- > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug >

June 21, 2005

Re: [Wlug] Need Homes: 2U+,custom cases/psupp/fans

by Eric Martin

Do you still have any cases? On 6/14/05, Doug Mildram <doug.mildram(a)mindspeed.com> wrote: > > Yet another Westboro,MA giveaway: (Anyone teaching EE or Hobbies909?) > Free and basically new: empty/nonstandard cases w/powersupp+fans. > > A dead project has left homeless ~15 cases, 19" wide, 24" deep, > (19" wide == standard 18.5" distance between frontpanel mounting holes) > 4.5" high == including .5" high rubber feet. (2U == 3.5", these are > taller), > semi-empty inside (a 13"x14"x3.5" area inside plus a 6"x8"x3.5" along > psupply) > 19" rackmountable , BUT, ***** NOT ***** set up for PC-based motherboard: > does NOT include disk/floppy/usb brackets or any of THAT pc-ready stuff. > > http://webpages.charter.net/mquinty/case_top.JPG > > Each case has a 400W standard 12v/5v power supply, > and (4) 3.5" fans already mounted (side exhaust) inside; > a removable suitcase-strap-panel on the front as shown; > (see picture case_front.JPG URL below..but it hides the switches below) > also on front panel: 1 power switch; 2 LEDs, "Fan Fail" and "Power". > > http://webpages.charter.net/mquinty/case_front.JPG > > Even MORE Gory detail: > Each case has a custom (now useless) NONstandard screw-post > arrangement which you'd need to re-do > (re-drill new holes as needed, may need shorter posts). > > So it's good if you have something REALLY CUSTOM you want mounted+powered. > > If you ask for just the power supplies, you're missing my mantra: > these babies (CASES) need an inventive home or homes. -- doug mildram > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug >

June 21, 2005

Re: [Wlug] HOWTO debug hard lockups

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 HI everybody, Perhaps I have found some light in this tunnel. I upgraded to the 2.6.12 kernel (yeah, I know, that just changed the entire experiment) and turned off "CPU freq scaling" and "PCI hotplug support" since I didn't think I needed either of those. I built the 2.6.12 kernel and played beer_golf 5 times in a row and the system DID NOT lockup. I'm not sure why those features were enabled in the first place. (I should probably go back to 2.6.11.11 and disable these two features and see if that has any positive effect.) I'm far too much of a pessimist to claim victory, but perhaps my luck is turning for the better. My fingers are still crossed. I am (extremely) cautiously optimistic. I thought I'd mention this in case somebody reads this thread in the future and maybe, just maybe, this tidbit helps. Besides, I've read a lot of threads where the problem is described in detail, but the answer is never forthcoming(!). Later, Andy - -- Andy Stewart, Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCtkacHl0iXDssISsRAvlGAJ9eVRN8NVKF7sWQtvIfdp4sqLi7BgCfXquw ZP3/O9spJsGk+B2aty2RMDg= =BGSi -----END PGP SIGNATURE-----

June 20, 2005

Re: [Wlug] HOWTO debug hard lockups

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andy Stewart wrote: > > HI gang, > > I have another clue - I found a way to lockup the system *guaranteed* > within about 3 minutes. Go to this URL and play this game. It is mouse > only. > > http://www.whtmtnliving.net/games/sports/beer_golf.htm OK, here's more to the saga of the machine suffering from hard lockups. Here's what I tried, and in all cases, the system still locks up playing beer_golf (but I am getting really good at this game!). This is all based on suggestions from all of you - thank you!! 1) Turned off video 3D acceleration. No effect, so I turned it back on. 2) Connected the KB and mouse directly to system (no KVM), no effect, so now the KVM is back in the picture. 3) Turned off sound hardware in the BIOS - no effect, so I turned it back on. 4) Tried a different video card (and this one was PCI, the original video card is AGP) - same deal, no effect, put back the original. 5) Enabled PNP in the BIOS (that's still on 'cause I thought I should probably have had it on anyway). 6) Disabled network hardware in the BIOS, put in Linksys PCI NIC. Nope, so I pulled the Linksys and reenabled the MB NIC. 7) Tried a different PS/2 mouse and tried my original mouse on another system. All mice seem to work properly. 8) I tried to get a serial mouse to work, but I couldn't get the serial mouse to work. I can't set the baud rate down to 1200 on this MB. Weird...the lowest it will go is 9600 (verified with minicom). I might play with this again to see if I can isolate the PS/2 mouse hardware on the MB (and the software in Linux). I don't have a USB mouse and Staples didn't have any when I visited. Here is what else I did: 9) Ran memtest for 24 hours - no failures 10) Access the offending system from over the network and play the game - - no lockup! 11) The beer_golf game is a swf file, so I downloaded the file and played it locally - no lockup! 12) Only when I play beer_golf in konqueror, over the net, does the system lockup reliably. 13) CPU temps measured independently from the MB are in the low 40 degree C range. Now, with regard to the serial console, I did get that working at least to a certain point. I've connected the serial cable to another Linux system running minicom. I can see the boot and shutdown messages in two locations (minicom/remote and local). However, when the system is running, I don't see the kernel messages on the serial console. When booting from grub, I have "console=tty0 console=ttyS1,115200n8" in that order. I do see the kernel messages in /var/log/messages and in the xconsole window. - From minicom, if I sent the Break character when the system is running OK (before the lockup occurs), that acts like Alt-SysReq. So, Break-? gives me the SysRq help line. Break-T (show tasks) puts output only on the local system (can't see it on minicom). I am not running getty on this ttyS1 since it seems to interfere with my attempts to use the MagicSysReq key. When it was enabled, I did get successfully logged in. When I put the system at run level 1, then all of the Alt-SysReq output goes to the serial console (minicom can see them!). Now, here's the part that I think might be bad. When I run the program which makes the system lock up, the lockup occurs. I then try to use minicom, and I get no output whatsoever. I can't even get the SysRq help output. I fear this may mean I have a real hardware lockup and not a Linux kernel problem. I know this is a lot to digest, and I really do appreciate all of the feedback. Does anybody have any comments that might help to further isolate this problem? Is there anything that I should try which I have not tried? Thanks!! Andy - -- Andy Stewart, Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCtPa7Hl0iXDssISsRAuZaAJ4mTMd8eSJEPo5LCCiqA8/rqazAQQCfSYmb mHIxlOPglV7Cq42hNKejVxU= =GJzk -----END PGP SIGNATURE-----

June 19, 2005

Re: [Wlug] HOWTO debug hard lockups

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 HI gang, I have another clue - I found a way to lockup the system *guaranteed* within about 3 minutes. Go to this URL and play this game. It is mouse only. http://www.whtmtnliving.net/games/sports/beer_golf.htm I disconnected my keyboard and mouse from the KVM switch and connected them directly to my system - and it *still* locks up reliably when I play the golf game (above) with the 2.6.11.11 kernel. Note I'm also running X.org 6.8.1. Could this be some flakiness in my XF86Config file with the mouse protocol? I could isolate this further if I could find a way to lock up the system in console mode using gpm. Its time to go to CompUSA and spend $4.38 on that serial console cable... Later, Andy - -- Andy Stewart, Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCsiV8Hl0iXDssISsRAoutAJ4qauOItdJnPoSXS2MuP149kKuMSgCdEXZ8 xP/T9Okh2ngQRZqZresZCDg= =2tw5 -----END PGP SIGNATURE-----

June 17, 2005