- WLUG - wlug.mailman3.com

Re: [Wlug] awk help
by Mike Peckar March 15, 2017

March 15, 2017

Nathan’s solution worked, thanks to all for the replies. I do feel better this was not a solution that could fit into a tweet. Theo’s is elegant, but there are spaces in the data, and John’s would have work, but I can’t use perl in the environment I’m working in. Mike awk -F, '{linecount[$3]++;tmp=dup[$3];if(length(tmp)==0){dup[$3]=$0}else{dup[$3]=dup[$3]"\n"$0}} END {for (count in linecount){if(linecount[count]>1){print dup[count]}}}' Hopefully this is not a homework problem. On Mon, Mar 13, 2017 at 7:20 PM, Mike Peckar <fog(a)fognet.com> wrote: Hey now, folks, This seemed like it should be simple, but I’m at wits end. I simply want to find duplicates in the third column of a csv file, and output the duplicate line _and_ the original line that matched it. There’s a million examples out there that will output just the duplicate but not both. In the data below, I’m looking for lines that match in the 3rd column… Normal,Server,xldspntc02,,10.33.52.185, Normal,Server,xldspntc02,,10.33.52.186, Normal,Server,xldspntc04,,10.33.52.187, Normal,Server,xldspntcs01,10.33.16.198, Normal,Server,xldspntcs01,,10.33.16.199, Normal,Server,xldsps01,10.33.16.162, Normal,Server,xldsps02,10.33.16.163, My desired output would be: Normal,Server,xldspntc02,,10.33.52.185, Normal,Server,xldspntc02,,10.33.52.186, Normal,Server,xldspntcs01,10.33.16.198, Normal,Server,xldspntcs01,,10.33.16.199, $ awk -F, 'dup[$3]++' file.csv I played around with the prev variable, but could not pumb it out fully, e.g { print prev } Mike _______________________________________________ Wlug mailing list Wlug(a)mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug -- Nathan Panike

1 0

Re: [Wlug] awk help
by John Stoffel March 15, 2017

March 15, 2017

>>>>> "Theo" == Theo Van Dinter <felicity(a)kluge.net> writes: Theo> Assuming there's no spaces in the csv, you could do: Theo> awk -F, '{print $0,$3}' | sort -k2 | uniq -D -f 1 | awk '{print $1}' Nice use of 'uniq -f 1' to re-read the file and do the de-duping. Theo> $ cat 1 Theo> Normal,Server,xldspntcs01,10.33.16.198, Theo> Normal,Server,xldspntc02,,10.33.52.185, Theo> Normal,Server,xldsps01,10.33.16.162, Theo> Normal,Server,xldspntc04,,10.33.52.187, Theo> Normal,Server,xldspntcs01,,10.33.16.199, Theo> Normal,Server,xldsps02,10.33.16.163, Theo> Normal,Server,xldspntc02,,10.33.52.186, Theo> $ cat 1 | awk -F, '{print $0,$3}' | sort -k2 | uniq -D -f 1 | awk '{print $1}' Theo> Normal,Server,xldspntc02,,10.33.52.185, Theo> Normal,Server,xldspntc02,,10.33.52.186, Theo> Normal,Server,xldspntcs01,,10.33.16.199, Theo> Normal,Server,xldspntcs01,10.33.16.198, Theo> On Mon, Mar 13, 2017 at 7:20 PM, Mike Peckar <fog(a)fognet.com> wrote: Theo> Hey now, folks, Theo> Theo> This seemed like it should be simple, but I’m at wits end. I simply Theo> want to find duplicates in the third column of a csv file, and output Theo> the duplicate line _and_ the original line that matched it. There’s a Theo> million examples out there that will output just the duplicate but not Theo> both. Theo> Theo> In the data below, I’m looking for lines that match in the 3^rd column… Theo> Theo> Normal,Server,xldspntc02,,10.33.52.185, Theo> Normal,Server,xldspntc02,,10.33.52.186, Theo> Normal,Server,xldspntc04,,10.33.52.187, Theo> Normal,Server,xldspntcs01,10.33.16.198, Theo> Normal,Server,xldspntcs01,,10.33.16.199, Theo> Normal,Server,xldsps01,10.33.16.162, Theo> Normal,Server,xldsps02,10.33.16.163, Theo> Theo> My desired output would be: Theo> Theo> Normal,Server,xldspntc02,,10.33.52.185, Theo> Normal,Server,xldspntc02,,10.33.52.186, Theo> Normal,Server,xldspntcs01,10.33.16.198, Theo> Normal,Server,xldspntcs01,,10.33.16.199, Theo> Theo> $ awk -F, 'dup[$3]++' file.csv Theo> Theo> I played around with the prev variable, but could not pumb it out Theo> fully, e.g { print prev } Theo> Theo> Mike Theo> _______________________________________________ Theo> Wlug mailing list Theo> Wlug(a)mail.wlug.org Theo> http://mail.wlug.org/mailman/listinfo/wlug Theo> _______________________________________________ Theo> Wlug mailing list Theo> Wlug(a)mail.wlug.org Theo> http://mail.wlug.org/mailman/listinfo/wlug

1 0

Re: [Wlug] awk help
by Theo Van Dinter March 15, 2017

March 15, 2017

Assuming there's no spaces in the csv, you could do: awk -F, '{print $0,$3}' | sort -k2 | uniq -D -f 1 | awk '{print $1}' eg: $ cat 1 Normal,Server,xldspntcs01,10.33.16.198, Normal,Server,xldspntc02,,10.33.52.185, Normal,Server,xldsps01,10.33.16.162, Normal,Server,xldspntc04,,10.33.52.187, Normal,Server,xldspntcs01,,10.33.16.199, Normal,Server,xldsps02,10.33.16.163, Normal,Server,xldspntc02,,10.33.52.186, $ cat 1 | awk -F, '{print $0,$3}' | sort -k2 | uniq -D -f 1 | awk '{print $1}' Normal,Server,xldspntc02,,10.33.52.185, Normal,Server,xldspntc02,,10.33.52.186, Normal,Server,xldspntcs01,,10.33.16.199, Normal,Server,xldspntcs01,10.33.16.198, On Mon, Mar 13, 2017 at 7:20 PM, Mike Peckar <fog(a)fognet.com> wrote: > Hey now, folks, > > > > This seemed like it should be simple, but I’m at wits end. I simply want > to find duplicates in the third column of a csv file, and output the > duplicate line _*and*_ the original line that matched it. There’s a > million examples out there that will output just the duplicate but not both. > > > > In the data below, I’m looking for lines that match in the 3rd column… > > > > Normal,Server,xldspntc02,,10.33.52.185, > > Normal,Server,xldspntc02,,10.33.52.186, > > Normal,Server,xldspntc04,,10.33.52.187, > > Normal,Server,xldspntcs01,10.33.16.198, > > Normal,Server,xldspntcs01,,10.33.16.199, > > Normal,Server,xldsps01,10.33.16.162, > > Normal,Server,xldsps02,10.33.16.163, > > > > My desired output would be: > > > > Normal,Server,xldspntc02,,10.33.52.185, > > Normal,Server,xldspntc02,,10.33.52.186, > > Normal,Server,xldspntcs01,10.33.16.198, > > Normal,Server,xldspntcs01,,10.33.16.199, > > > > $ awk -F, 'dup[$3]++' file.csv > > > > I played around with the prev variable, but could not pumb it out fully, > e.g { print prev } > > > > Mike > > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > >

1 0

Re: [Wlug] awk help
by John Stoffel March 14, 2017

March 14, 2017

>>>>> "Mike" == Mike Peckar <fog(a)fognet.com> writes: Mike> This seemed like it should be simple, but I’m at wits end. I Mike> simply want to find duplicates in the third column of a csv Mike> file, and output the duplicate line _and_ the original line Mike> that matched it. There’s a million examples out there that Mike> will output just the duplicate but not both. Mike> In the data below, I’m looking for lines that match in the 3^rd column… The sorting part is easy... sort -k 3 -t "," <file> Now to find the duplicates... I'd probably jump to a perl script: perl -e '@a=(<>);foreach @a {@t=split(",",$_) {push @{$t{$t[2]}},$_; } foreach sort (keys %t) { if ($#{$t{$_}} > 0) { print @{$t{$_}}, "\n";}}' Should also do the right thing. First it splits into keys, stuffs the line into an assoc array of arrays. Then it sorts the assoc array and prints out those with more than one entry in it. Admittedly done off the top of my head, without any actual testing. :-) Mike> Normal,Server,xldspntc02,,10.33.52.185, Mike> Normal,Server,xldspntc02,,10.33.52.186, Mike> Normal,Server,xldspntc04,,10.33.52.187, Mike> Normal,Server,xldspntcs01,10.33.16.198, Mike> Normal,Server,xldspntcs01,,10.33.16.199, Mike> Normal,Server,xldsps01,10.33.16.162, Mike> Normal,Server,xldsps02,10.33.16.163, Mike> My desired output would be: Mike> Normal,Server,xldspntc02,,10.33.52.185, Mike> Normal,Server,xldspntc02,,10.33.52.186, Mike> Normal,Server,xldspntcs01,10.33.16.198, Mike> Normal,Server,xldspntcs01,,10.33.16.199, Mike> $ awk -F, 'dup[$3]++' file.csv Mike> I played around with the prev variable, but could not pumb it out fully, Mike> e.g { print prev } Mike> Mike Mike> _______________________________________________ Mike> Wlug mailing list Mike> Wlug(a)mail.wlug.org Mike> http://mail.wlug.org/mailman/listinfo/wlug

1 0

Re: [Wlug] awk help
by Nathan W. Panike March 14, 2017

March 14, 2017

This seems to work: awk -F, '{linecount[$3]++;tmp=dup[$3];if(length(tmp)==0){dup[$3]=$0}else{dup[$3]=dup[$3]"\n"$0}} END {for (count in linecount){if(linecount[count]>1){print dup[count]}}}' Hopefully this is not a homework problem. On Mon, Mar 13, 2017 at 7:20 PM, Mike Peckar <fog(a)fognet.com> wrote: > Hey now, folks, > > > > This seemed like it should be simple, but I’m at wits end. I simply want > to find duplicates in the third column of a csv file, and output the > duplicate line _*and*_ the original line that matched it. There’s a > million examples out there that will output just the duplicate but not both. > > > > In the data below, I’m looking for lines that match in the 3rd column… > > > > Normal,Server,xldspntc02,,10.33.52.185, > > Normal,Server,xldspntc02,,10.33.52.186, > > Normal,Server,xldspntc04,,10.33.52.187, > > Normal,Server,xldspntcs01,10.33.16.198, > > Normal,Server,xldspntcs01,,10.33.16.199, > > Normal,Server,xldsps01,10.33.16.162, > > Normal,Server,xldsps02,10.33.16.163, > > > > My desired output would be: > > > > Normal,Server,xldspntc02,,10.33.52.185, > > Normal,Server,xldspntc02,,10.33.52.186, > > Normal,Server,xldspntcs01,10.33.16.198, > > Normal,Server,xldspntcs01,,10.33.16.199, > > > > $ awk -F, 'dup[$3]++' file.csv > > > > I played around with the prev variable, but could not pumb it out fully, > e.g { print prev } > > > > Mike > > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > > -- Nathan Panike

1 0

awk help
by Mike Peckar March 13, 2017

March 13, 2017

Hey now, folks, This seemed like it should be simple, but I’m at wits end. I simply want to find duplicates in the third column of a csv file, and output the duplicate line _and_ the original line that matched it. There’s a million examples out there that will output just the duplicate but not both. In the data below, I’m looking for lines that match in the 3rd column… Normal,Server,xldspntc02,,10.33.52.185, Normal,Server,xldspntc02,,10.33.52.186, Normal,Server,xldspntc04,,10.33.52.187, Normal,Server,xldspntcs01,10.33.16.198, Normal,Server,xldspntcs01,,10.33.16.199, Normal,Server,xldsps01,10.33.16.162, Normal,Server,xldsps02,10.33.16.163, My desired output would be: Normal,Server,xldspntc02,,10.33.52.185, Normal,Server,xldspntc02,,10.33.52.186, Normal,Server,xldspntcs01,10.33.16.198, Normal,Server,xldspntcs01,,10.33.16.199, $ awk -F, 'dup[$3]++' file.csv I played around with the prev variable, but could not pumb it out fully, e.g { print prev } Mike

1 0

Re: [Wlug] (for meeting projectors) laptops,VGA-out,linux....
by Chuck Anderson March 10, 2017

March 10, 2017

On Fri, Mar 10, 2017 at 10:37:53AM -0500, Tim Keller wrote: > Thanks for doing some great digging on the projector stuff.. It's only > 2017... Projectors should just *work*. They do "just work" in some OSes/desktop environments.

1 0

Re: [Wlug] (for meeting projectors) laptops,VGA-out,linux....
by Tim Keller March 10, 2017

March 10, 2017

Playing with Cinnamon last night I'm going to take the dive and throw it on a RHEL7 VM and see if I like it better than Mate. If so, I'll swap over my desktop and see... Thanks for doing some great digging on the projector stuff.. It's only 2017... Projectors should just *work*. On Fri, Mar 10, 2017 at 9:12 AM, Doug Mildram <dmildram(a)gmail.com> wrote: > doug> Thanks Tim for bringing a functional laptop w/linux for last night's > meeting. Our VGA-out projector worked fine as he installed a linux Mint VM > (Mate Cinnamon) onto it. > > When that finished, I wanted to show/compare Mint Mate, which I had > already on my laptop. > > When I unplugged Tim's laptop from VGA-out and tried my own thinkpad > running Mint Mate (dualbooted, my alternate OS).....well,.....chirp. > > The projector stopped projecting. My Fn-F7 key seemed to be trying to > cycle thru the blind toggles/choices (Duplicate display? screen-only? > VGA-only might expect), but no video came out, I had to give up, not having > tried VGA-out ever while running linux on my laptop. We put Tim's back > onto VGA-out, and it also seemed dead, but we chose not to fight it, > meeting being close enough to stop caring about projector, and it "wasnt a > proud moment" for me. > > SO I WENT HOME and plugged my (same) laptop into my TV's VGA-in port. > Started by booting win7. I'd never noticed a detail : at WHICH POINT does > this start working....should I expect to see the dualboot/GRUB menu on TV? > Apparently not. Soon enough though, Win7 (the OS preinstalled by > Lenovo/thinkpad) came up and VGA-out/TV started working as usual. > (somewhere AFTER "Resuming Windows", it starts working). > To control VGA-out, Fn-F7 works ...I've got (4) choices: Duplicate, > Extend(laptop-screen=#1, TV-VGAout=screen2), Projector Only (turns off > laptop display), and Disconnect Projector (only laptop display ON). No > doubt (i surmise), thanks to windows drivers. > > Feels like I learned: OS controls VGA-out (somehow naively I was inclined > to believe this was more of an OS-independent hardware function). > > OK, now rebooted laptop/win7 to try Linux Mint from a win7 state with > "Duplicate" working. "Shutting down Windows" displayed....laptop goes thru > boot/BIOS?...TV is blank now, I've learned to expect this....Linux kernel > boots, and I see some of the minimal console text messages as usual ALSO ON > TV, laptop-screen AND TV quickly go to graphical mode, also displaying the > Mate background... > > and this is where I was the most surprised. > > TV/VGA-out appeared "frozen' with the Mate background, appearing much like > my laptop screen BUT MINUS anything else (no bottom launch/status/"panel" > bar. Whatever I launch on my laptop screen, does not appear on VGA-out. > Fascinating! > > Wed eve late, at my house, I could swear Fn-F7 still wasnt doing anything. > But this Thurs AM, it DOES start working a bit (same 4 choices, but blind, > and it's strange. TV/VGA-out doesnt show EVERYTHING). > I wonder if I'd rebooted at the meeting, would that have helped, probably. > Today I play with it some more (ALT-F2 helps when I'm in that semi-blind > VGA-out situation, getting SOME things to show on TV) even though I'm not > likely to use this feature much?ever? at home, unless I break a leg. > > Caveat, practice what you bring for meetings to project-OUT! > HDMI is also an option in our WPI meeting/conf rms if you have it. > > > ---trivia for Mate interest... > (Mate = lightweight alternate to Cinnamon. > Their start/launch menus are different, and I can find good stuff in > Mate's Control Center. Cinnamon's Control center looks like this: > https://docs.google.com/file/d/0B2V313X0LzmQclJCVkI5aFhIeDA/edit > > Couldnt find a good Mate comparison Control Center screenshot > (not sharing some high-ad google'd choices) > but I did like this decent page mostly cuz i love keyb shortcuts: > > http://securitronlinux.com/bejiitaswrath/some-useful- > tips-for-the-linux-mate-desktop/ > where I learned > CTL-ALT-d (lowercase "d" not DEL, which offers to reboot) > .....lowers-or-raises what you're doing, a great "quick hide from > over-shoulder" ninja move. (sorry to degrade tricks/skill(s) of a Real > Ninja) > > Stay warm! --doug > > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > > -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

1 0

(for meeting projectors) laptops,VGA-out,linux....
by Doug Mildram March 10, 2017

March 10, 2017

doug> Thanks Tim for bringing a functional laptop w/linux for last night's meeting. Our VGA-out projector worked fine as he installed a linux Mint VM (Mate Cinnamon) onto it. When that finished, I wanted to show/compare Mint Mate, which I had already on my laptop. When I unplugged Tim's laptop from VGA-out and tried my own thinkpad running Mint Mate (dualbooted, my alternate OS).....well,.....chirp. The projector stopped projecting. My Fn-F7 key seemed to be trying to cycle thru the blind toggles/choices (Duplicate display? screen-only? VGA-only might expect), but no video came out, I had to give up, not having tried VGA-out ever while running linux on my laptop. We put Tim's back onto VGA-out, and it also seemed dead, but we chose not to fight it, meeting being close enough to stop caring about projector, and it "wasnt a proud moment" for me. SO I WENT HOME and plugged my (same) laptop into my TV's VGA-in port. Started by booting win7. I'd never noticed a detail : at WHICH POINT does this start working....should I expect to see the dualboot/GRUB menu on TV? Apparently not. Soon enough though, Win7 (the OS preinstalled by Lenovo/thinkpad) came up and VGA-out/TV started working as usual. (somewhere AFTER "Resuming Windows", it starts working). To control VGA-out, Fn-F7 works ...I've got (4) choices: Duplicate, Extend(laptop-screen=#1, TV-VGAout=screen2), Projector Only (turns off laptop display), and Disconnect Projector (only laptop display ON). No doubt (i surmise), thanks to windows drivers. Feels like I learned: OS controls VGA-out (somehow naively I was inclined to believe this was more of an OS-independent hardware function). OK, now rebooted laptop/win7 to try Linux Mint from a win7 state with "Duplicate" working. "Shutting down Windows" displayed....laptop goes thru boot/BIOS?...TV is blank now, I've learned to expect this....Linux kernel boots, and I see some of the minimal console text messages as usual ALSO ON TV, laptop-screen AND TV quickly go to graphical mode, also displaying the Mate background... and this is where I was the most surprised. TV/VGA-out appeared "frozen' with the Mate background, appearing much like my laptop screen BUT MINUS anything else (no bottom launch/status/"panel" bar. Whatever I launch on my laptop screen, does not appear on VGA-out. Fascinating! Wed eve late, at my house, I could swear Fn-F7 still wasnt doing anything. But this Thurs AM, it DOES start working a bit (same 4 choices, but blind, and it's strange. TV/VGA-out doesnt show EVERYTHING). I wonder if I'd rebooted at the meeting, would that have helped, probably. Today I play with it some more (ALT-F2 helps when I'm in that semi-blind VGA-out situation, getting SOME things to show on TV) even though I'm not likely to use this feature much?ever? at home, unless I break a leg. Caveat, practice what you bring for meetings to project-OUT! HDMI is also an option in our WPI meeting/conf rms if you have it. ---trivia for Mate interest... (Mate = lightweight alternate to Cinnamon. Their start/launch menus are different, and I can find good stuff in Mate's Control Center. Cinnamon's Control center looks like this: https://docs.google.com/file/d/0B2V313X0LzmQclJCVkI5aFhIeDA/edit Couldnt find a good Mate comparison Control Center screenshot (not sharing some high-ad google'd choices) but I did like this decent page mostly cuz i love keyb shortcuts: http://securitronlinux.com/bejiitaswrath/some-useful-tips-for-the-linux-mat… where I learned CTL-ALT-d (lowercase "d" not DEL, which offers to reboot) .....lowers-or-raises what you're doing, a great "quick hide from over-shoulder" ninja move. (sorry to degrade tricks/skill(s) of a Real Ninja) Stay warm! --doug

1 0

Reminder! Meeting TONIGHT March 9th, 2017! Topics: KVM, Virtualized windows apps, Linux Mint!
by Tim Keller March 9, 2017

March 9, 2017

Hey Gang, First the important details! Location: Rm 331 (Hagglund Room) @ 7PM Topic: read below... Tonight we've got a meeting! After the last meeting a couple of people were asking me questions about KVM (Kernel based Virtual Machine). New member Al Reinhart asked me a question about the best ways to run windows apps on linux, either via wine or through KVM / Oracle Virtual Box. He also mentioned Linux.Encoder (the first linux ransomware) which I didn't even know exists. Long time member Doug Mildram also wrote me about Linux Mint. A distribution I know quite well! They'll be some conversation and we'll kick off an install and talk about it's merits as a greate distribution. As usual, there'll be refreshments and snacks and afterwards we'll head to Tech for pizza! Later, Tim. -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

1 0