- WLUG - wlug.mailman3.com

Re: LLM's and graphical interfaces.

by John Stoffel

I'm on the jitsi.org meeting, but the only one right now. More are welcome to join Tim and I! Tim is running late...

June 13, 2024

Re: LLM's and graphical interfaces.

by John Stoffel

Thanks for sending out this info Jansen, now to find time to actually play with this stuff! > (can this email address be added to the mailing list?) > the self-hosted webui i use is here: > https://github.com/open-webui/open-webui > ollama! > https://github.com/ollama/ollama > here's the docker for ollama: > https://hub.docker.com/r/ollama/ollama > On Tue, May 14, 2024 at 5:32 PM Kevin Harrington <mad.hephaestus(a)gmail.com> wrote: > ---------- Forwarded message --------- > From: Tim Keller via WLUG <wlug(a)lists.wlug.org> > Date: Tue, May 14, 2024, 4:49 PM > Subject: [WLUG] LLM's and graphical interfaces. > To: Worcester Linux Users' Group General Discussion <wlug(a)lists.wlug.org> > Cc: Tim Keller <turbofx(a)gmail.com> > A couple meetings ago Jansen showed up a really cool graphical interface for LLM. > Does anybody remember what it was? > Thanks, > Tim. > -- > I am leery of the allegiances of any politician who refers to their constituents as > "consumers". > _______________________________________________ > WLUG mailing list -- wlug(a)lists.wlug.org > To unsubscribe send an email to wlug-leave(a)lists.wlug.org > Create Account: https://wlug.mailman3.com/accounts/signup/ > Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ > Web Forum/Archive: > https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/XVOQB… > _______________________________________________ > WLUG mailing list -- wlug(a)lists.wlug.org > To unsubscribe send an email to wlug-leave(a)lists.wlug.org > Create Account: https://wlug.mailman3.com/accounts/signup/ > Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ > Web Forum/Archive: https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/4ZQIW…

June 13, 2024

Reminder: Meeting June 13th 2024 at 7PM! Topic: Your own personal LLM!

by Tim Keller

Hey Gang! We've got a meeting tomorrow!! Date/Time: June 13th 2024 @ 7pm. Location: Technocopia, 44 Portland St, 6th floor, Worcester MA. Virtual Location: (back to the old haunt) https://meet.jit.si/WlugMA <https://meet.jit.si/WlugMA> Considering how much AI has been in the news, I thought it would be cool to do a talk/demo about setting up your own personal Large Language Model! I'll either use my laptop of scrounge up a piece of good enough hardware to show it off.. hell I could even spin up a droplet and run it somewhere.. we'll figure that out. Jansen sent me some links to the cool stuff he uses and I'll definitely be leaning on his expertise and others. Here's what he wrote: the self-hosted webui i use is here: https://github.com/open-webui/open-webui ollama! https://github.com/ollama/ollama here's the docker for ollama: https://hub.docker.com/r/ollama/ollama I've experimented with the ollama models and they're cool. As usual we'll nerd out, wrestle with some LLM stuff and have a great time. Afterwards we'll head off to the Boyton or somewhere else for some dinner and keep the geekery in full swing. I'll bring some snacks and refreshments as well. Later, Tim. -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

June 12, 2024

Re: Hey Everybody! WLUG Meeting this week!

by Gregory Boyce

On Mon, Jun 10, 2024 at 6:01 PM John Stoffel <john(a)stoffel.org> wrote: > >>>>> "Gregory" == Gregory Boyce <gregory.boyce(a)gmail.com> writes: > > > On Mon, Jun 10, 2024, 2:42 PM John Stoffel via WLUG <wlug(a)lists.wlug.org> > wrote: > >>>>>> "Tim" == Tim Keller <turbofx(a)gmail.com> writes: > > >> Currently we're having issues getting the Turing Pi 2 to boot, so > >> doing a show off of Proxmox I could definitely do as well. > > > I'ev got proxmox running at owkr with some basic VMs on it for > > testing. I might be able to show it off doing some basic stuff like > > live migrations. > > > I recently switched my home server to proxmox, and I've been > > spending time using terraform and packer to build and deploy VMs. > > Are you running a cluster or just one system? I've got a three node > cluster at $WORK, but thinking of looking at their SCALE product. But > of course while VMware is going to shaft us pricewise, the manglement > might not want to leave the devil they know. Just a single system at the moment. Another issue is deploying OVA files. I find that ProxMOX is a pain > in terms of .iso and .ova files, you need to first add them into it's > own repository, then you can deploy from them. VMware just lets you > browse for the files and deploy. > Yeah, that makes sense. The interface isn't especially user friendly either. It takes me a while to remember where the interface for uploading ISO files is whenever I need to use it. That's one thing that Scale Computing does well. The interface for uploading images is pretty straightforward, and it will convert images from various formats for you automatically. Other than the current lack of support for compressed images. > Though I also admit that .ova deployments can be annoyingly tough, but > I'm running VCenter 7.0.x and a mix of ESXi 6.7 and 7.0 on my > systems. Legacy hardware which I can't move up. Another reason to > hate vendor designed stuff, it locks you in. > > > I'd be happy to share code if anyone is interested. > > Please do! > This is very much a work in progress as I work on learning the tools. https://gitlab.com/techstoa/proxmox -- Greg

June 11, 2024

Re: Hey Everybody! WLUG Meeting this week!

by John Stoffel

>>>>> "Gregory" == Gregory Boyce <gregory.boyce(a)gmail.com> writes: > On Mon, Jun 10, 2024, 2:42 PM John Stoffel via WLUG <wlug(a)lists.wlug.org> wrote: >>>>>> "Tim" == Tim Keller <turbofx(a)gmail.com> writes: >> Currently we're having issues getting the Turing Pi 2 to boot, so >> doing a show off of Proxmox I could definitely do as well. > I'ev got proxmox running at owkr with some basic VMs on it for > testing. I might be able to show it off doing some basic stuff like > live migrations. > I recently switched my home server to proxmox, and I've been > spending time using terraform and packer to build and deploy VMs. Are you running a cluster or just one system? I've got a three node cluster at $WORK, but thinking of looking at their SCALE product. But of course while VMware is going to shaft us pricewise, the manglement might not want to leave the devil they know. Another issue is deploying OVA files. I find that ProxMOX is a pain in terms of .iso and .ova files, you need to first add them into it's own repository, then you can deploy from them. VMware just lets you browse for the files and deploy. Though I also admit that .ova deployments can be annoyingly tough, but I'm running VCenter 7.0.x and a mix of ESXi 6.7 and 7.0 on my systems. Legacy hardware which I can't move up. Another reason to hate vendor designed stuff, it locks you in. > I'd be happy to share code if anyone is interested. Please do!

June 10, 2024

Re: Hey Everybody! WLUG Meeting this week!

by Gregory Boyce

On Mon, Jun 10, 2024, 2:42 PM John Stoffel via WLUG <wlug(a)lists.wlug.org> wrote: > >>>>> "Tim" == Tim Keller <turbofx(a)gmail.com> writes: > > > Currently we're having issues getting the Turing Pi 2 to boot, so > > doing a show off of Proxmox I could definitely do as well. > > I'ev got proxmox running at owkr with some basic VMs on it for > testing. I might be able to show it off doing some basic stuff like > live migrations. > I recently switched my home server to proxmox, and I've been spending time using terraform and packer to build and deploy VMs. I'd be happy to share code if anyone is interested. >

June 10, 2024

Re: We've got a meeting next week on the 13th!

by John Stoffel

>>>>> "soup" == soup via WLUG <wlug(a)lists.wlug.org> writes: I suspect that now that Broadcom is releasing their new ARM CPU and MS is shipping windows on it, there will be alot more ARM based boards out there, and they should be pretty good for running linux. In my use case though, I'd be happy to run on ARM for my personal email hosting and simple web stuff, but it's the memory that's the killer. I want to get a Linode with more RAM (1gb just doesn't cut it when you try to run rspam...) so I can run postfix/dovecot/rspam and all the other stuff you need for doing email well these days. Stuff like setting up keys, DMARC, SPF, signing outgoing emails, re-signing forwarded emails, etc. It's honestly getting pretty damn tough to run your own mail server well these days. > Damn that's some metal. Pricing is up there for a workstation > though. It'll drop hopefully, especially with Win11 ARM on the > horizon. Much obliged for the info. > --- > soup > On Fri, Jun 7, 2024 at 4:19 PM Tim Keller <turbofx(a)gmail.com> wrote: > In my data center I've got a bunch of Ampere ARM servers and they're awesome. U.2 NVME, lots > of PCI slots, etc. > https://www.jeffgeerling.com/blog/2023/everything-ive-learned-building-fast… > Tim. > On Fri, Jun 7, 2024 at 1:51 PM soup <soupforare(a)gmail.com> wrote: > Are there any shipping ARM boards/chips for the consumer market? Quick look just shows a > few ATX server boards with, relatively, ancient ARMs bundled. > The stuff Apple is shipping has the same T2 "problem" the later intel machines have with > running linux, it's kind of a pain, delicate, and ruins the point of having the hardware > security in the first place. It'd be amazing if they'd work on 1st party support but > considering their business model I don't see that ever happening. > On the server side, I'm loving ARM. I saw the revolution coming back when AWS Graviton was > first announced. Pushed a lot of things at work over early. With go binaries it's as > simple as a compile flag. Graviton2 offerings were ~1/3 off the equivalent intel machines > for EC2, similar discount for lambda. For some of our workloads they were faster than > intel too. Haven't had occasion to test the grav3 stuff but I imagine it's better still. > If you've got things in interpreted language, go, or projects that have been cross-arch > and stable a while- mariadb, postgresql, nginx, et al., definitely would suggest seeing > how it'd work for you. > --- > soup > On Fri, Jun 7, 2024 at 12:20 PM Tim Keller via WLUG <wlug(a)lists.wlug.org> wrote: > Keith, > It wasn't just you. It seems that it maxes out at some low number of users.. why.. > worth investigating, but not at the expense of people being able to get into the > meeting. > Yeah, ARM for linux has been the "hotness" for a while. Pi definitely brought it to > the forefront, but now more and more there are viable arm desktops. > As for the AI, there are some FOSS solutions out there that are shockingly amazing. > "ollama" is just one of them. > No need to pay NVidia for AMD for the privilege. > Tim. > On Thu, Jun 6, 2024 at 1:58 PM Keith Wright <kwright(a)keithdiane.us> wrote: > Tim Keller via WLUG <wlug(a)lists.wlug.org> writes: >> We've got a meeting next week! Note, I won't be using our own Jitsi server >> but going to back to the old one since we've had so many issues. > I think part of the problem was with my micophone to webserver > interface. > At this point I am thinking of this more as a test than a meeting, > but I plan to do it. >> If there's a topic that people would like us to tackle, I'd be up for it! > I haven't been paying a lot of attention to the latest hardware, but > the radio said something about ARM processor that caught my attention. > ARM means rasberry pi means Linux, no? Wikipedia has an enormous > article (the Citations alone are 11 pages two column) but the last > thing on the page seems to say that Wine is no longer for playing > Windows games on Linux, but that MS needs it to prop up the ass end > of their sorry operating system. Is the monopoly gone? > The radio also speaks of Nvidia. They sell those Invidious graphics > cards. The ones that need special secret drivers that only work if > you have the right *.dll? It seems that everyone now needs them to > do Artifical Intelligence (all rise). Have they changed their attitude, > or is it like: "Here's your AI. Pay the monthly bill to retain your > licence to use it. Any attempt to try to find out what it does or > how it works will be prosecuted to the fullest extent of the law of > Connecticut."? > -- Keith > -- > I am leery of the allegiances of any politician who refers to their constituents as > "consumers". > _______________________________________________ > WLUG mailing list -- wlug(a)lists.wlug.org > To unsubscribe send an email to wlug-leave(a)lists.wlug.org > Create Account: https://wlug.mailman3.com/accounts/signup/ > Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ > Web Forum/Archive: > https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/K35PA… > -- > I am leery of the allegiances of any politician who refers to their constituents as > "consumers". > _______________________________________________ > WLUG mailing list -- wlug(a)lists.wlug.org > To unsubscribe send an email to wlug-leave(a)lists.wlug.org > Create Account: https://wlug.mailman3.com/accounts/signup/ > Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ > Web Forum/Archive: https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/BUVOG…

June 10, 2024

Re: Hey Everybody! WLUG Meeting this week!

by John Stoffel

>>>>> "Tim" == Tim Keller <turbofx(a)gmail.com> writes: > Currently we're having issues getting the Turing Pi 2 to boot, so > doing a show off of Proxmox I could definitely do as well. I'ev got proxmox running at owkr with some basic VMs on it for testing. I might be able to show it off doing some basic stuff like live migrations. > I've got a three node setup as well as a backup node running at > work, but I don't have anything on it that I couldn't show the > world. Mine is strictly testing without anything it use that anyone could care about. I haven't spent that much time playing with it.

June 10, 2024

Re: We've got a meeting next week on the 13th!

by soup

Damn that's some metal. Pricing is up there for a workstation though. It'll drop hopefully, especially with Win11 ARM on the horizon. Much obliged for the info. --- soup On Fri, Jun 7, 2024 at 4:19 PM Tim Keller <turbofx(a)gmail.com> wrote: > In my data center I've got a bunch of Ampere ARM servers and they're > awesome. U.2 NVME, lots of PCI slots, etc. > > > https://www.jeffgeerling.com/blog/2023/everything-ive-learned-building-fast… > > Tim. > > On Fri, Jun 7, 2024 at 1:51 PM soup <soupforare(a)gmail.com> wrote: > >> Are there any shipping ARM boards/chips for the consumer market? Quick >> look just shows a few ATX server boards with, relatively, ancient ARMs >> bundled. >> The stuff Apple is shipping has the same T2 "problem" the later intel >> machines have with running linux, it's kind of a pain, delicate, and ruins >> the point of having the hardware security in the first place. It'd be >> amazing if they'd work on 1st party support but considering their business >> model I don't see that ever happening. >> >> On the server side, I'm loving ARM. I saw the revolution coming back when >> AWS Graviton was first announced. Pushed a lot of things at work over >> early. With go binaries it's as simple as a compile flag. >> Graviton2 offerings were ~1/3 off the equivalent intel machines for EC2, >> similar discount for lambda. For some of our workloads they were faster >> than intel too. Haven't had occasion to test the grav3 stuff but I imagine >> it's better still. >> If you've got things in interpreted language, go, or projects that have >> been cross-arch and stable a while- mariadb, postgresql, nginx, et al., >> definitely would suggest seeing how it'd work for you. >> >> --- >> soup >> >> On Fri, Jun 7, 2024 at 12:20 PM Tim Keller via WLUG <wlug(a)lists.wlug.org> >> wrote: >> >>> Keith, >>> >>> It wasn't just you. It seems that it maxes out at some low number of >>> users.. why.. worth investigating, but not at the expense of people being >>> able to get into the meeting. >>> >>> Yeah, ARM for linux has been the "hotness" for a while. Pi definitely >>> brought it to the forefront, but now more and more there are viable arm >>> desktops. >>> >>> As for the AI, there are some FOSS solutions out there that are >>> shockingly amazing. "ollama" is just one of them. >>> No need to pay NVidia for AMD for the privilege. >>> >>> Tim. >>> >>> On Thu, Jun 6, 2024 at 1:58 PM Keith Wright <kwright(a)keithdiane.us> >>> wrote: >>> >>>> Tim Keller via WLUG <wlug(a)lists.wlug.org> writes: >>>> >>>> > We've got a meeting next week! Note, I won't be using our own Jitsi >>>> server >>>> > but going to back to the old one since we've had so many issues. >>>> >>>> I think part of the problem was with my micophone to webserver >>>> interface. >>>> >>>> At this point I am thinking of this more as a test than a meeting, >>>> but I plan to do it. >>>> >>>> > If there's a topic that people would like us to tackle, I'd be up for >>>> it! >>>> >>>> I haven't been paying a lot of attention to the latest hardware, but >>>> the radio said something about ARM processor that caught my attention. >>>> ARM means rasberry pi means Linux, no? Wikipedia has an enormous >>>> article (the Citations alone are 11 pages two column) but the last >>>> thing on the page seems to say that Wine is no longer for playing >>>> Windows games on Linux, but that MS needs it to prop up the ass end >>>> of their sorry operating system. Is the monopoly gone? >>>> >>>> The radio also speaks of Nvidia. They sell those Invidious graphics >>>> cards. The ones that need special secret drivers that only work if >>>> you have the right *.dll? It seems that everyone now needs them to >>>> do Artifical Intelligence (all rise). Have they changed their attitude, >>>> or is it like: "Here's your AI. Pay the monthly bill to retain your >>>> licence to use it. Any attempt to try to find out what it does or >>>> how it works will be prosecuted to the fullest extent of the law of >>>> Connecticut."? >>>> >>>> -- Keith >>>> >>>> >>> >>> -- >>> I am leery of the allegiances of any politician who refers to their >>> constituents as "consumers". >>> _______________________________________________ >>> WLUG mailing list -- wlug(a)lists.wlug.org >>> To unsubscribe send an email to wlug-leave(a)lists.wlug.org >>> Create Account: https://wlug.mailman3.com/accounts/signup/ >>> Change Settings: >>> https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ >>> Web Forum/Archive: >>> https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/K35PA… >>> >> > > -- > I am leery of the allegiances of any politician who refers to their > constituents as "consumers". >

June 9, 2024

Re: We've got a meeting next week on the 13th!

by Tim Keller

In my data center I've got a bunch of Ampere ARM servers and they're awesome. U.2 NVME, lots of PCI slots, etc. https://www.jeffgeerling.com/blog/2023/everything-ive-learned-building-fast… Tim. On Fri, Jun 7, 2024 at 1:51 PM soup <soupforare(a)gmail.com> wrote: > Are there any shipping ARM boards/chips for the consumer market? Quick > look just shows a few ATX server boards with, relatively, ancient ARMs > bundled. > The stuff Apple is shipping has the same T2 "problem" the later intel > machines have with running linux, it's kind of a pain, delicate, and ruins > the point of having the hardware security in the first place. It'd be > amazing if they'd work on 1st party support but considering their business > model I don't see that ever happening. > > On the server side, I'm loving ARM. I saw the revolution coming back when > AWS Graviton was first announced. Pushed a lot of things at work over > early. With go binaries it's as simple as a compile flag. > Graviton2 offerings were ~1/3 off the equivalent intel machines for EC2, > similar discount for lambda. For some of our workloads they were faster > than intel too. Haven't had occasion to test the grav3 stuff but I imagine > it's better still. > If you've got things in interpreted language, go, or projects that have > been cross-arch and stable a while- mariadb, postgresql, nginx, et al., > definitely would suggest seeing how it'd work for you. > > --- > soup > > On Fri, Jun 7, 2024 at 12:20 PM Tim Keller via WLUG <wlug(a)lists.wlug.org> > wrote: > >> Keith, >> >> It wasn't just you. It seems that it maxes out at some low number of >> users.. why.. worth investigating, but not at the expense of people being >> able to get into the meeting. >> >> Yeah, ARM for linux has been the "hotness" for a while. Pi definitely >> brought it to the forefront, but now more and more there are viable arm >> desktops. >> >> As for the AI, there are some FOSS solutions out there that are >> shockingly amazing. "ollama" is just one of them. >> No need to pay NVidia for AMD for the privilege. >> >> Tim. >> >> On Thu, Jun 6, 2024 at 1:58 PM Keith Wright <kwright(a)keithdiane.us> >> wrote: >> >>> Tim Keller via WLUG <wlug(a)lists.wlug.org> writes: >>> >>> > We've got a meeting next week! Note, I won't be using our own Jitsi >>> server >>> > but going to back to the old one since we've had so many issues. >>> >>> I think part of the problem was with my micophone to webserver >>> interface. >>> >>> At this point I am thinking of this more as a test than a meeting, >>> but I plan to do it. >>> >>> > If there's a topic that people would like us to tackle, I'd be up for >>> it! >>> >>> I haven't been paying a lot of attention to the latest hardware, but >>> the radio said something about ARM processor that caught my attention. >>> ARM means rasberry pi means Linux, no? Wikipedia has an enormous >>> article (the Citations alone are 11 pages two column) but the last >>> thing on the page seems to say that Wine is no longer for playing >>> Windows games on Linux, but that MS needs it to prop up the ass end >>> of their sorry operating system. Is the monopoly gone? >>> >>> The radio also speaks of Nvidia. They sell those Invidious graphics >>> cards. The ones that need special secret drivers that only work if >>> you have the right *.dll? It seems that everyone now needs them to >>> do Artifical Intelligence (all rise). Have they changed their attitude, >>> or is it like: "Here's your AI. Pay the monthly bill to retain your >>> licence to use it. Any attempt to try to find out what it does or >>> how it works will be prosecuted to the fullest extent of the law of >>> Connecticut."? >>> >>> -- Keith >>> >>> >> >> -- >> I am leery of the allegiances of any politician who refers to their >> constituents as "consumers". >> _______________________________________________ >> WLUG mailing list -- wlug(a)lists.wlug.org >> To unsubscribe send an email to wlug-leave(a)lists.wlug.org >> Create Account: https://wlug.mailman3.com/accounts/signup/ >> Change Settings: >> https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ >> Web Forum/Archive: >> https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/K35PA… >> > -- I am leery of the allegiances of any politician who refers to their constituents as "consumers".

June 7, 2024