Hi all, I have noticed the lack of sufficient backup programs for linux. A friend an i have decided to write our own. We have come up with a good list of goals we want to hit. Before we started seriously working on the design, i wanted to ask the group of features they would like to see in a backup systems. Thanks, --Brad
HI Brad, By "lack of sufficient backup programs", do you mean GPL programs? There are a few commercial offerings available such as BRU and Arkeia. Last I knew, Arkeia was available for free download for home use, but is not GPL software. OK, to answer your question about features that I like to see in a backup solution: o fits on a bootable floppy for disaster recovery o it would be nice if I didn't have to run through the whole tape to get one file restored. o GUI and command line (keeps everybody happy) o store a directory of files on the tape o sufficient error detection and corrections schemes to account for imperfect media o damaged media shouldn't prevent me from getting back the entire backup, just a file or two (unless of course the tape is shredded or something) o backup program does a read/verify to prove that the backup was really written correctly o ability to write backups to media other than tape, such as DVD-RAM or perhaps a hard disk partition That's just a few thoughts...perhaps they will stimulate some discussion. Later, Andy -- Andy Stewart Founder Worcester Linux Users' Group Worcester, MA, USA http://www.wlug.org
What I would like to see in a backup program: o Simple configuration. o Simple operation -- insert bulk media and bootable media (floppy or CD to burn), and run the backup program with one parameter (the configuration name). Good logging, but ending with a clear "yes, this backup worked without any problems", or "no, you aren't safe yet". o Simple disaster recovery -- insert the bulk media and bootable media pair, turn on power, answer "yes" to "Is this a disaster recovery?" prompt, identify media, and everything gets restored. o Simple file restore -- to original location or parallel file tree. o Catalogs on tape and optionally on disk. The ability to copy take catalogs to disk. o Support for multiple operating systems/boots/file system types in one backup set. B.J. At 22:44 19-06-2001 -0400, you wrote:
Hi all, I have noticed the lack of sufficient backup programs for linux. A friend an i have decided to write our own. We have come up with a good list of goals we want to hit. Before we started seriously working on the design, i wanted to ask the group of features they would like to see in a backup systems.
Thanks, --Brad _______________________________________________ Wlug mailing list Wlug@mail.wlug.org http://mail.wlug.org/mailman/listinfo/wlug
-- B.J. Herbison / bj@herbison.com / http://www.herbison.com/herbison/bj.html The Next Asylum / 203 Long Hill Road / Bolton, MA 01740-1421 / +1 978 634-1061
Brad> I have noticed the lack of sufficient backup programs for Brad> linux. A friend an i have decided to write our own. We have come Brad> up with a good list of goals we want to hit. Before we started Brad> seriously working on the design, i wanted to ask the group of Brad> features they would like to see in a backup systems. It depends on what you're trying to do here. Are you looking at doing a backup of just one system? Multiple systems to a central server? Multiple systems to a central server, along with extra drives attached directly to other high volume hosts? I'm a user of Legato software, and while I have some major problems with the company and the pricing, they do some things very well. Here are my thoughts on what a good backup system should have: 1. It should scale well. 2. Good reporting, especially for outside programs. 3. Support multiplexing of data streams to backup media to increase performance. 4. Very modular, with good APIs between sections. 5. Support a wide variety of backup medias. 6. Support a large variety of jukeboxes and their robots. - including barcode labels - intelligent handling of moving tapes. Don't single thread like Legato. 7. Be able to generate disaster recovery media. 8. Online indexes of files on tape. 9. GUI interface uses the command line commands for all work. 10. Staging to disk before writing to backup media. Useful for slow remote clients. 11. Completely flexible scheduler for backup times and levels. Legato sucks in this regard. 12. Support for pre/post processing on each file system on each client to handle special cases you don't think of ahead of time. Andy and B.J. seem to be asking more for a much smaller scale system which is targeted for a home user who has a complete disaster. I'm asking for a more enterprise level system. And before you do any actual coding, you should look at the following references: Unix Backup & Recovery by W. Curtis Preston (O'Reilly) http://www.amanda.org http://www.backupcentral.com (Curtis' web site) And see what's out there. And again, I can't stress this enough, try to steal what's out there if you can, and then make sure it's modular and flexible. You cannot make it to flexible. John John Stoffel - Senior Unix Systems Administrator - Lucent Technologies stoffel@lucent.com - http://www.lucent.com - 978-952-7548
On Tue, Jun 19, 2001 at 10:44:57PM -0400, Brad Noyes wrote:
I have noticed the lack of sufficient backup programs for linux. A friend an i have decided to write our own. We have
Other people have mentioned various products, so here are the ones I know of/use on Linux: - BRU (I think I'm a rev or two behind, it's commercial and acts as a better tar -- complete with GUI) - arkeia (commercial, haven't used) - amanda (OSS, and highly recommended by folks I speak with) - tar (OSS, everyone's friend and stand-by product) - dump/restore (OSS, see comments for tar) - legato networker (commercial, pricey. it's a good package though, although I used it on everything except linux at a previous job) - veritas netbackup (commercial. I don't know if the server is available on linux, but it's definately available as a client. according to the sales critters who came by for a presentation last week, linux is second in their list of supported ports (ie: they program for solaris, then port to linux, then (not in order) hpux/aix, and somewhere near the bottom is tru64) I think it was already mentioned, but hit up Curtis Preston's website: www.backupcentral.com Curtis also has an O'Reilly book that you may find useful. I haven't done more than flip through it yet (I'm not typically a B&R guy), but it covers almost anything you'd want to know about B&R practices in the real world. -- Randomly generated tagline: "Robotic tape changer mistook operator's tie for a backup tape." - Today's BOFH Excuse
Hi All, i just wanted to thank you guys for the great responses. There were several ideas that we hadn't thought of yet. A question was asked whether we wanted to make an backup solution for a single machine or a whole network for an enterprise solution. Initially We were going to write it for a single system, but thoughts of backuping up a system remotely had come to mind. We will probably stick to a small scale for now (since we only have a test bed of a single machine), but hopefully, if we plan this right and write the APIs well, we'll be able to scale it. Either way we'll keep the enterprise solution in mind as we go. Many Thanks! --Brad
Brad> We will probably stick to a small scale for now (since we only Brad> have a test bed of a single machine), but hopefully, if we plan Brad> this right and write the APIs well, we'll be able to scale Brad> it. Either way we'll keep the enterprise solution in mind as we Brad> go. The architecture that I would suggest goes something like this. Though it would be easier on a blackboard to diagram. Do functional blocks for you high level layout. Be flexible. Think big. 64 bit pointers everywhere. Data is *not* getting smaller. - central server - talks to clients (which can be on the central server) - talks to device driver(s) both local and remote. - talks to index database server(s). - talks to media mover(s) - maintains backup schedules - controls indexes and their retention times. - labels tapes with control info. - device drivers - accept raw data from central server, spool to tape/disk/etc. - multiplexes data streams to device. - optimized for specific devices. - clients - runs on various OSes - reads various file systems and dumps data in intermediate form to central server. - index server - lets you keep online indexes off all files saved to which tape and when, for easy restores. - media mover - controls jukebox - can accept multiple commands at once and optimizes them. - requires funky per-jukebox with per-device command queues. Can you tell that this is something I've been thinking about? Look at the Amanda scheduler. It's an interesting idea, but it locks you into their way of doing things too much. Needs to be more general. John John Stoffel - Senior Unix Systems Administrator - Lucent Technologies stoffel@lucent.com - http://www.lucent.com - 978-952-7548
participants (5)
-
Andy Stewart -
B.J. Herbison -
Brad Noyes -
John Stoffel -
Theo Van Dinter