September 2006 - WLUG - wlug.mailman3.com

CentOS?

by Pete Wason

Hiall, Anyone working with CentOS? I have a RH8 server that needs to join the party. Any remote possibility that CentOS can upgrade RH8, or do I need to backup everything and start from scratch? Yes, I know it's a wacky question. :) PS: And, DAMN AOL! PPS: if this message is HTML, let me know; I'm still learning Thunderbird... ------------------------------------------------------------------------ *Pete Wason* Hy Noom Publications /codevark(a)aim.com/ 508.865.5414

Sept. 6, 2006

Re: [Wlug] bacula

by Mike Leo

Ok...I have it all figured out in my head, now i just need to start configuring the jobs. I was able to get a 2nd 35/70 drive, now DLT1 and DLT2 are options for my backups. I can't figure out how to have 1 job write to both either concurrently, or sequentailly as the job only allows me to pick 1 or the other. I tried using: Storage = DLT1, DLT2 and by having 2 Storage lines in my bacula-dir.conf, but neither worked. Any idea's? John Stoffel <john(a)stoffel.org> wrote: Just for grins, here's my output of 'mt -f /dev/nst0 status' on my DLT7k driver: > mt -f /dev/st0 status SCSI 2 tape drive: File number=0, block number=0, partition=0. Tape block size 0 bytes. Density code 0x1b (DLT 35GB). Soft error count since last status=0 General status bits on (41010000): BOT ONLINE IM_REP_EN You can also run the bacula command 'btape' on there to make sure you've got things configured properly. John _______________________________________________ Wlug mailing list Wlug(a)mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug

Sept. 6, 2006

Re: [Wlug] bacula

by John Stoffel

Just for grins, here's my output of 'mt -f /dev/nst0 status' on my DLT7k driver: > mt -f /dev/st0 status SCSI 2 tape drive: File number=0, block number=0, partition=0. Tape block size 0 bytes. Density code 0x1b (DLT 35GB). Soft error count since last status=0 General status bits on (41010000): BOT ONLINE IM_REP_EN You can also run the bacula command 'btape' on there to make sure you've got things configured properly. John

Sept. 5, 2006

MEETING: Thurs, Sept 7th, 7:00 PM

by Andy Stewart

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 HI Everybody, We have a WLUG meeting coming up on Thursday, September 7th, 2006, at 7:00 PM in the Access Grid room at the WPI campus in Worcester, MA USA. Anybody who is interested in Linux is welcome to attend. At this meeting, David Dichmann from Sybase will be the guest speaker, talking about Sybase on Linux. Checkout this link for more information: http://www.sybase.com/linux In addition, I'll be bringing a box of old (but working) computer parts. Feel free to be a scavenger! Included are ATAPI CDROM drives (8-16x), 3.5" floppy drives, a couple of power supplies, and a couple of Pentium 200MHz motherboards. Anything not claimed at the meeting will go to electronics recycling. There will be quite a few other things available for a free giveaway: Sybase teeshirts and other goodies 2 x LaTeX User's Guide and Reference Manual (Leslie Lamport) Self-Service Linux - Mastering the Art of Problem Determination See ya there! Andy - -- Andy Stewart, Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFE/bJzHl0iXDssISsRAlk+AJ9kJtIGusw5IydkvFoz/I71yxcILwCghpG9 2wAXiRdWKJmQkVllmU6qqQo= =WE0i -----END PGP SIGNATURE-----

Sept. 5, 2006

Re: [Wlug] bacula

by John Stoffel

Mike> Thanks a bunch. I had found this out 5 minutes after my first Mike> email, but your examples will come in handy i'm sure! No problem. Bacula is good, the manual is excellent for what it is, but the config files are Baroque... Kern's style I guess. Mike> i have a 35/70Gb DLT drive with IV tapes, but I am only getting Mike> less than 30 gigs' per tape. How is the tape drive hooked up to the system, and how fast is the system you're running on? Can you share some details on a completed job maybe? Mike> my bacula-sd.conf lists " Media Type = DLT"....if this is not Mike> accurate, could bacula think I have only a 15/30Gb drive? Mike> Should it read DTL7000 or something? Nah, doesn't really matter. Mike> Also, does bacula compress naturally or do i have to tell it too? I'm not aware of Bacula compressing data from the client when sending it to the tape drive unless you explicitly enable it. And it's generally not worth doing. The key thing with DLT is to push data to the drive at a high enough rate to keep it from shoe-shining. That both wears down the drive, and kills performance/storage capacity. Since the native speed is 5MB/s, you really really really wnat to be able to push 10MB/s to the drive to let it compress the data well. Unfortunately, that can be hard to do, esp if you have lots of small files to backup. Also, check your dmesg output to see how the drive is detected and at what speed it's running. When you put a tape into the drive, does the compression light come on as well? There's a known limitation (performance bottleneck actually) where Bacula uses transactions for inserting records into the DB one at a time. If they could insert more records at a time, performance would go up since the transaction overhead would be amortized across more INSERTS. Also, which DB are you using? And which version of Bacula? The newest released version I think (and I'd have to check) now uses sqlite3 as the default DB, which has a big performance loss compared to older 1.36.x versions which defaulted to sqlite which had better performance. Now one of the things I did to speed up my backups was to put my DB onto another host, since my tape drive and data were all one one system, so adding mysql updates ontop of that really just brought it down to it's knees. Anyway, more data on your backups and how well they currently run would be good. On my system, I get around 24gb to 68gb per tape, so it's certainly possible to get more data on there, esp since I'm using a dual CPU Xeon 550mhz box to do my backups. Old and not very fast. John

Sept. 5, 2006

Re: [Wlug] bacula

by Frank Sweetser

On Mon, Sep 04, 2006 at 03:11:14PM -0700, Mike Leo wrote: > i have a 35/70Gb DLT drive with IV tapes, but I am only getting less than 30 > gigs' per tape. > > my bacula-sd.conf lists " Media Type = DLT"....if this is not accurate, could > bacula think I have only a 15/30Gb drive? Should it read DTL7000 or > something? Bacula doesn't actually parse the media type - it's nothing more than a unique identifier string that bacula uses to determine what volumes can be read by what drives. You could call file volumes "blue", DLT tapes "red", and DVDRs "green" and it would work fine. Bacula doesn't try to pre-determine the amount of space available on a tape. It just writes until the drive reports that it's hit the end of media, re-reads the last succesfully written block to make sure it got there okay, marks the volume full and asks for another one. The fact that you're getting about 30G of data per tape implies that you have hardware tape compression off. I believe the 'mt' command will tell you the compression status - the online manual has an example. The other 5G or so is probably taken up by volume metadata, such as the volume label and file attributes. > Also, does bacula compress naturally or do i have to tell it too? No, you have to explicitly enable gzip compression if you want to use it. It's typically reccomended to use drive hardware compression instead, though, as a) drive hardware compression is usually faster, especially on older machines, and b) it's possible to run into file daemon incompatabilities that can make it difficult to restore data, especially if you're trying to restore the data onto a different machine that the one it was backed up on. -- Frank Sweetser fs at wpi.edu | For every problem, there is a solution that WPI Network Engineer | is simple, elegant, and wrong. - HL Mencken GPG fingerprint = 6174 1257 129E 0D21 D8D4 E8A3 8E39 29E3 E2E8 8CEC

Sept. 4, 2006

Re: [Wlug] bacula

by Mike Leo

Thanks a bunch. I had found this out 5 minutes after my first email, but your examples will come in handy i'm sure! Now i just need to start building my own backup FileSet's etc. One question, however... i have a 35/70Gb DLT drive with IV tapes, but I am only getting less than 30 gigs' per tape. my bacula-sd.conf lists " Media Type = DLT"....if this is not accurate, could bacula think I have only a 15/30Gb drive? Should it read DTL7000 or something? Also, does bacula compress naturally or do i have to tell it too? thanks again mike John Stoffel <john(a)stoffel.org> wrote: Mike> Someone at the cook-out mentioned they were using bacula for Mike> backups. I have just installed it and I have a DLT tape drive. Same here, I've got Bacula running right now at home to a DLT7k drive. Mike> I have it confugered OK and i can connect to my fd (client) and Mike> my sd (storage device). Mike> To keep things simple, lets say that I am doing everything on Mike> one server. Yup. Mike> I can label the tapes and even backup to my "File" device. Mike> My question is, what/where do I build a list of jobs to run? This is one of the weaknesses of Bacula in my mind. You need to setup a JobDefs { ... } section in your /etc/bacula/bacula-dir.conf file. Mike> Lets say I want a "Full" backup of the system, and a seperate Mike> backup of just /home. Mike> Where do these jobs get defined, etc etc etc. This seems to be Mike> the only piece I am missing in the doc's. The docs are actually pretty good, it just sometimes takes a bit of re-reading to make it work. For example, here's what I have for one client. JobDefs { Name = "jack to tape" Type = Backup Level = Differential Client = jack-fd FileSet = "jack Full Set" Schedule = "MonthlyCycle" Storage = jfsnew-DLT Messages = Standard Pool = Backups Priority = 10 } Notice the Schedule and FileSet lines. Those are OTHER sections in the same file which define which schedule to use, and which filesets to run. Then you need to define a Job {...} section too: Job { Name = "jack-fd" JobDefs = "jack to tape" Write Bootstrap = "/var/lib/bacula/jack.bsr" } So here's most of what I've got defined for this one client. It's a pain, and I wish he would just dump this into the DB so it's easier to manage. Or maybe having a tool to write these files would be useful. :] FileSet { Name = "jack Full Set" #noatime = yes Ignore FileSet Changes = yes Include { Options { wilddir = \.thumbnails exclude = yes } File = @/etc/bacula/FileSet-jack } Exclude { File = /sys File = /proc File = /tmp File = ".journal" File = ".fsck" File = "/var/lib/mysql" File = ".thumbnails" } } Schedule { Name = "MonthlyCycle" Run = Level=Full 1st sun at 1:05 Run = Level=Differential 2nd-5th sun at 1:05 Run = Level=Incremental mon-sat at 1:05 } Client { Name = jack-fd Address = jack FDPort = 9102 Catalog = MyCatalog Password = "" # password for FileDaemon File Retention = 60 days # 30 days Job Retention = 12 months # six months AutoPrune = yes # Prune expired Jobs/Files } Let me know if this helps or not. John _______________________________________________ Wlug mailing list Wlug(a)mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug

Sept. 4, 2006

Re: [Wlug] bacula

by John Stoffel

Mike> Someone at the cook-out mentioned they were using bacula for Mike> backups. I have just installed it and I have a DLT tape drive. Same here, I've got Bacula running right now at home to a DLT7k drive. Mike> I have it confugered OK and i can connect to my fd (client) and Mike> my sd (storage device). Mike> To keep things simple, lets say that I am doing everything on Mike> one server. Yup. Mike> I can label the tapes and even backup to my "File" device. Mike> My question is, what/where do I build a list of jobs to run? This is one of the weaknesses of Bacula in my mind. You need to setup a JobDefs { ... } section in your /etc/bacula/bacula-dir.conf file. Mike> Lets say I want a "Full" backup of the system, and a seperate Mike> backup of just /home. Mike> Where do these jobs get defined, etc etc etc. This seems to be Mike> the only piece I am missing in the doc's. The docs are actually pretty good, it just sometimes takes a bit of re-reading to make it work. For example, here's what I have for one client. JobDefs { Name = "jack to tape" Type = Backup Level = Differential Client = jack-fd FileSet = "jack Full Set" Schedule = "MonthlyCycle" Storage = jfsnew-DLT Messages = Standard Pool = Backups Priority = 10 } Notice the Schedule and FileSet lines. Those are OTHER sections in the same file which define which schedule to use, and which filesets to run. Then you need to define a Job {...} section too: Job { Name = "jack-fd" JobDefs = "jack to tape" Write Bootstrap = "/var/lib/bacula/jack.bsr" } So here's most of what I've got defined for this one client. It's a pain, and I wish he would just dump this into the DB so it's easier to manage. Or maybe having a tool to write these files would be useful. :] FileSet { Name = "jack Full Set" #noatime = yes Ignore FileSet Changes = yes Include { Options { wilddir = \.thumbnails exclude = yes } File = @/etc/bacula/FileSet-jack } Exclude { File = /sys File = /proc File = /tmp File = ".journal" File = ".fsck" File = "/var/lib/mysql" File = ".thumbnails" } } Schedule { Name = "MonthlyCycle" Run = Level=Full 1st sun at 1:05 Run = Level=Differential 2nd-5th sun at 1:05 Run = Level=Incremental mon-sat at 1:05 } Client { Name = jack-fd Address = jack FDPort = 9102 Catalog = MyCatalog Password = "" # password for FileDaemon File Retention = 60 days # 30 days Job Retention = 12 months # six months AutoPrune = yes # Prune expired Jobs/Files } Let me know if this helps or not. John

Sept. 3, 2006

Re: [Wlug] bacula

by Frank Sweetser

On Sat, Sep 02, 2006 at 10:42:10PM -0700, Mike Leo wrote: > My question is, what/where do I build a list of jobs to run? > > Lets say I want a "Full" backup of the system, and a seperate backup of just > /home. All of that kind of policy based config gets defined in the director config file. You create filesets to define what files get backed up, and schedules to define when. -- Frank Sweetser fs at wpi.edu | For every problem, there is a solution that WPI Network Engineer | is simple, elegant, and wrong. - HL Mencken GPG fingerprint = 6174 1257 129E 0D21 D8D4 E8A3 8E39 29E3 E2E8 8CEC

Sept. 3, 2006

never mind

by Mike Leo

Figured it out. /etc/bacula/bacula-dir.conf has different "File Sets" that define that. thanks

Sept. 3, 2006