- WLUG - wlug.mailman3.com

Reminder! Meeting April 13th @ 7PM

by Tim Keller

Hey Gang, This is reminder that we've got a meeting this week! Location: Campus Center Room 331 Time: 7PM Topic: To be decided! Let me know if there's something that you have a burning urge to talk about! Thanks, Tim. Per our last meeting, here were some of the topics... Media server bake off Kodi (Xbmc) PLEX media center /server Open media vault (runs PLEX) Open ELEC (runs Kodi) Linux Arcade machines Research Distros Home Backup Server (zfs for linux) Bake off of backup solutions Sysadmin automation tools Salt / Puppet / Chef / Ansible Nicholas Howgraham-Grave Talk about finding flaws in RSA SHA1 collison stuff Git development Methods Virtualization on linux KVM / Vmware / OVB / Xen / Qubes Privacy on Linux: What to do when you're convinced the goverment is watching. Encrypted FS 2FA Isolated applications What distro is snowden running? 3d Printing and linux Tools for linux -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

April 9, 2017

Re: [Wlug] video from Identity Management, FreeIPA, and Kerberos talk

by Chuck Anderson

Someone tried to download a "high" quality file. Sorry there is nothing but this "low" quality one available right now. I might be able to make a higher quality one, but it takes like 4-8 hours for each try, and I figured the video quality isn't that important--just use the PDF slides with this 278MB video file. The audio should be good quality. On Wed, Apr 05, 2017 at 04:38:24PM -0400, Tim Keller wrote: > Nice!!! > > On Wed, Apr 5, 2017 at 4:07 PM, Chuck Anderson <cra(a)wpi.edu> wrote: > > > It only took me a year and a half, but I finally was able to get the > > 27gig AVCHD files from the video camera converted to a usable video > > file format. Please find the video (and slides) here: > > > > http://www.wlug.org/ > > > > December 10, 2015 - Dmitri Pal, Red Hat, Inc. - Identity Management, > > FreeIPA, and Kerberos (slides) (video) > > > > Direct links: > > > > http://www.wlug.org/files/Identity%20Management%20for% > > 20Modern%20Infrastructure%20(WPI)%20low.mp4 > > > > http://www.wlug.org/files/Identity%20Management%20for% > > 20Modern%20Infrastructure%20(WPI).pdf

April 5, 2017

Re: [Wlug] video from Identity Management, FreeIPA, and Kerberos talk

by Tim Keller

Nice!!! On Wed, Apr 5, 2017 at 4:07 PM, Chuck Anderson <cra(a)wpi.edu> wrote: > It only took me a year and a half, but I finally was able to get the > 27gig AVCHD files from the video camera converted to a usable video > file format. Please find the video (and slides) here: > > http://www.wlug.org/ > > December 10, 2015 - Dmitri Pal, Red Hat, Inc. - Identity Management, > FreeIPA, and Kerberos (slides) (video) > > Direct links: > > http://www.wlug.org/files/Identity%20Management%20for% > 20Modern%20Infrastructure%20(WPI)%20low.mp4 > > http://www.wlug.org/files/Identity%20Management%20for% > 20Modern%20Infrastructure%20(WPI).pdf > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

April 5, 2017

video from Identity Management, FreeIPA, and Kerberos talk

by Chuck Anderson

It only took me a year and a half, but I finally was able to get the 27gig AVCHD files from the video camera converted to a usable video file format. Please find the video (and slides) here: http://www.wlug.org/ December 10, 2015 - Dmitri Pal, Red Hat, Inc. - Identity Management, FreeIPA, and Kerberos (slides) (video) Direct links: http://www.wlug.org/files/Identity%20Management%20for%20Modern%20Infrastruc… http://www.wlug.org/files/Identity%20Management%20for%20Modern%20Infrastruc…

April 5, 2017

Ideas for next meeting (April 13th)

by Tim Keller

Hey Gang, A couple of ideas have been bandied about for the next meeting. 1. A round table discussion regarding internet security. 2. glusterfs. This is a distributed clustering filesystem that lets you tie a bunch of systems into a single cohesive filesystem. 3. ? If people have ideas, I'd love to hear them. I made a list at the last meeting but I don't have it on me... Thanks, Tim. -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

April 4, 2017

Re: [Wlug] (no subject)

by Tim Keller

Lustre solves the problem by getting out of the way. The metadata server simply tells the client on which of the nodes the pieces of the file resides. Reading the documentation, when it comes to Lustre, split brain can't happen because the moment any of the meta controllers go offline, the whole thing stops. Definitely with ZFS if you don't start with a plan you're going to have a bad day. The current design for a node is a 1U HP DL360G7 server with ~144 or 288GB of ram. Internal storage is handled by the smart array controller. The next one I build I think I'm going to buy a 9200-8i and route the cables so that all the internal and external storage is JBOD. My current HBA card dejour for external storage is the LSI 9200-8e. As for external JBOD enclosures, there are LOTS of choices. Generally I've stuck with the promise J610s. It's essentially the "expansion" cabinet for a "smart" promise array. It's beauty is in it's abject stupidity. As for drives, currently all my stuff is running 4TB Seagate SAS drives. For "mainline" nodes, I configure them as raid10 pools. This gets me ~29TB of storage per pool For "backup" nodes, I configure them as raidz2 pools. All my backup nodes are 32 disk boxes so the pools are 116TB. Obviously this design is a balance between cost and performance. If I need more slots I'll use a 380 instead of 360 and I'll stick in an Intel X540-T2 10GigE card. It's fun to imagine if you scaled up the disks and the interconnects what a filesystem would look like. Tim. On Tue, Mar 28, 2017 at 8:45 PM, John Stoffel <john(a)stoffel.org> wrote: > > Tim> My first experiment was simply setting up a distributed volume > Tim> that needs to have each of the "bricks" as they refer to them as > Tim> being reliable members. > > Nice. > > Tim> However you can configure it to mirror and strip across the > Tim> "bricks" so they don't have to be. In fact you can setup > Tim> stripped mirrors, etc. > > That would be safer for sure. I can just see a network problem taking > down a bunch of bricks and leading to a split-brain situation if > you're not careful. > > Tim> I've now got a number of these storage nodes as I call them. It > Tim> generally consists of a 1U box, stuffed full of ram, connected to > Tim> a JBOD chassis full of disks. I then use ZFS to create storage > Tim> pools. > > So I used to like ZFS and think it was "da bomb" but after using it > for several years, and esp after using the ZFS appliances from > Sun/Oracle, I'm not really enamored of the entire design any more. > > I think they have some great ideas in terms of checksumming all the > files and metadata. But the layering (or lack of) disks/devices > inside zpools just drives me nuts. It's just really inflexible and > can get you in trouble if you're not careful. > > Tim> The problem with this is that your scale is limited to the size > Tim> of a single "node" and while you can play games with autofs it's > Tim> not a cohesive filesystem. > > Can you give some more details of your setup? > > Tim> This solves that problem. My only complaint is that it's fuse based. > > Yeah, not really high performance at the end of the day. > > Tim> Lustre is a kernel loaded filesystem that'll do the same thing as > Tim> gluster. However, it doesn't have any of the redunancy stuff, > Tim> it simply assumes your underlying storage is reliable. > > This is the key/kicker for all these systems. > > I'm waiting for someone to come up with an opensource sharding system > where you use erasure codes for low-level block storage so that you > have alot of the RAID6 advantages, but even more reliability AND > speed. But it's also a hard hard problem space to get right. > > Thanks for sharing! > John > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

March 29, 2017

Re: [Wlug] (no subject)

by John Stoffel

Tim> My first experiment was simply setting up a distributed volume Tim> that needs to have each of the "bricks" as they refer to them as Tim> being reliable members. Nice. Tim> However you can configure it to mirror and strip across the Tim> "bricks" so they don't have to be. In fact you can setup Tim> stripped mirrors, etc. That would be safer for sure. I can just see a network problem taking down a bunch of bricks and leading to a split-brain situation if you're not careful. Tim> I've now got a number of these storage nodes as I call them. It Tim> generally consists of a 1U box, stuffed full of ram, connected to Tim> a JBOD chassis full of disks. I then use ZFS to create storage Tim> pools. So I used to like ZFS and think it was "da bomb" but after using it for several years, and esp after using the ZFS appliances from Sun/Oracle, I'm not really enamored of the entire design any more. I think they have some great ideas in terms of checksumming all the files and metadata. But the layering (or lack of) disks/devices inside zpools just drives me nuts. It's just really inflexible and can get you in trouble if you're not careful. Tim> The problem with this is that your scale is limited to the size Tim> of a single "node" and while you can play games with autofs it's Tim> not a cohesive filesystem. Can you give some more details of your setup? Tim> This solves that problem. My only complaint is that it's fuse based. Yeah, not really high performance at the end of the day. Tim> Lustre is a kernel loaded filesystem that'll do the same thing as Tim> gluster. However, it doesn't have any of the redunancy stuff, Tim> it simply assumes your underlying storage is reliable. This is the key/kicker for all these systems. I'm waiting for someone to come up with an opensource sharding system where you use erasure codes for low-level block storage so that you have alot of the RAID6 advantages, but even more reliability AND speed. But it's also a hard hard problem space to get right. Thanks for sharing! John

March 29, 2017

Re: [Wlug] (no subject)

by Tim Keller

My first experiment was simply setting up a distributed volume that needs to have each of the "bricks" as they refer to them as being reliable members. However you can configure it to mirror and strip across the "bricks" so they don't have to be. In fact you can setup stripped mirrors, etc. I've now got a number of these storage nodes as I call them. It generally consists of a 1U box, stuffed full of ram, connected to a JBOD chassis full of disks. I then use ZFS to create storage pools. The problem with this is that your scale is limited to the size of a single "node" and while you can play games with autofs it's not a cohesive filesystem. This solves that problem. My only complaint is that it's fuse based. Lustre is a kernel loaded filesystem that'll do the same thing as gluster. However, it doesn't have any of the redunancy stuff, it simply assumes your underlying storage is reliable. Tim. On Tue, Mar 28, 2017 at 11:08 AM, John Stoffel <john(a)stoffel.org> wrote: > > Tim> The subject for that was supposed to be "Fun with glusterfs" > Tim> On Tue, Mar 28, 2017 at 10:51 AM, Tim Keller <turbofx(a)gmail.com> > wrote: > > Tim> I've been messing around with "glusterfs" and it's pretty cool. > > Tim> I build 3 machine running RHEL7.2 each with an extra 250GB disk. > > Tim> I then setup a filesystem that stripes across all three machines. > > Tim> So it looks like a single 750GB filesystem. Over gigE I'm seeing > okay > Tim> performance ~50MB/s. > > Tim> Cool stuff. Next on my list of filesystems to play with it > Lustre. > > Sounds neat. Does glusterfs support RAID5/6 or does it depend on the > underlying nodes to be reliable? > > _______________________________________________ > Wlug mailing list > Wlug(a)mail.wlug.org > http://mail.wlug.org/mailman/listinfo/wlug > -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

March 28, 2017

Re: [Wlug] (no subject)

by John Stoffel

Tim> The subject for that was supposed to be "Fun with glusterfs" Tim> On Tue, Mar 28, 2017 at 10:51 AM, Tim Keller <turbofx(a)gmail.com> wrote: Tim> I've been messing around with "glusterfs" and it's pretty cool. Tim> I build 3 machine running RHEL7.2 each with an extra 250GB disk. Tim> I then setup a filesystem that stripes across all three machines. Tim> So it looks like a single 750GB filesystem. Over gigE I'm seeing okay Tim> performance ~50MB/s. Tim> Cool stuff. Next on my list of filesystems to play with it Lustre. Sounds neat. Does glusterfs support RAID5/6 or does it depend on the underlying nodes to be reliable?

March 28, 2017

Re: [Wlug] (no subject)

by Tim Keller

The subject for that was supposed to be "Fun with glusterfs" On Tue, Mar 28, 2017 at 10:51 AM, Tim Keller <turbofx(a)gmail.com> wrote: > I've been messing around with "glusterfs" and it's pretty cool. > > I build 3 machine running RHEL7.2 each with an extra 250GB disk. > > I then setup a filesystem that stripes across all three machines. > > So it looks like a single 750GB filesystem. Over gigE I'm seeing okay > performance ~50MB/s. > > Cool stuff. Next on my list of filesystems to play with it Lustre. > > Tim. > > -- > I am leery of the allegiances of any politician who refers to their > constituents as "consumers". > -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

March 28, 2017