Backups with Bacula
Hi All, I just recently joined your group and was hoping for some insight or advise on building a backup solution. My question is related to Bacula on FreeNAS. I hope it's ok that I ask if not I understand. ==Background== We run a mixed environment (Windows and Linux). I've been slowly moving services that we run over to Linux wherever possible however I've run into an issue with our existing backup server. It is a Microsoft product. I'm not looking for help on it, instead I am looking for advise on using bacula. I've configured Bacula and have it working successfully and I like how I can deploy agents onto my servers to handle backups. I love the distributed achitecture. So my question: I'd like to use FreeNAS as our StorageDaemon (sd) going forward. I'd also like to do remote offsite backups by either rsyncing files over to a remote server or by getting FreeNAS to backup to an external HDD that I rotate weekly. ==Questions== Does anyone see any issues with this? What do you think about the HDD rotation? Thanks, Tony
On Thu, 11 Oct 2018 at 13:01, Tony Fernandez via talk <talk@gtalug.org> wrote:
Hi All,
I just recently joined your group and was hoping for some insight or advise on building a backup solution.
My question is related to Bacula on FreeNAS. I hope it's ok that I ask if not I understand.
==Background== We run a mixed environment (Windows and Linux). I've been slowly moving services that we run over to Linux wherever possible however I've run into an issue with our existing backup server. It is a Microsoft product. I'm not looking for help on it, instead I am looking for advise on using bacula.
I've configured Bacula and have it working successfully and I like how I can deploy agents onto my servers to handle backups. I love the distributed achitecture.
So my question: I'd like to use FreeNAS as our StorageDaemon (sd) going forward. I'd also like to do remote offsite backups by either rsyncing files over to a remote server or by getting FreeNAS to backup to an external HDD that I rotate weekly.
==Questions== Does anyone see any issues with this? What do you think about the HDD rotation?
Bacula has always seemed to be one of the good options out there, and running it on FreeNAS is certainly well supported. There's nothing obviously wrong with your approach to rsync to a remote place or copy to external HDD for rotation. Madison Kelly did a talk on something akin back in 2004; Madison was the first person I heard that particularly "championed" using USB-connected HDDs as a backup medium at the time that tape drives were only just starting to get supplanted as a backup medium. Since then, that direction has become somewhere in between "viable" and "preferable." And it now looks like tape drives are pretty rarely used anymore, as rarity has made it difficult for vendors to boost capacity as quickly as is the case for disk drives. *Everyone* wants bigger HDDs. (Well, we're starting to glimpse a place where solid state drives are getting sufficiently large and cheap that a lot of computer systems now prefer SSD, and we may see HDDs go somewhat down the road that tape drives have...) Rotating the HDDs so that they do get spun up fairly regularly is a good idea. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?"
On Thu, Oct 18, 2018 at 04:21:41PM -0400, Christopher Browne via talk wrote:
Bacula has always seemed to be one of the good options out there, and running it on FreeNAS is certainly well supported.
There's nothing obviously wrong with your approach to rsync to a remote place or copy to external HDD for rotation.
Madison Kelly did a talk on something akin back in 2004; Madison was the first person I heard that particularly "championed" using USB-connected HDDs as a backup medium at the time that tape drives were only just starting to get supplanted as a backup medium.
Since then, that direction has become somewhere in between "viable" and "preferable." And it now looks like tape drives are pretty rarely used anymore, as rarity has made it difficult for vendors to boost capacity as quickly as is the case for disk drives. *Everyone* wants bigger HDDs. (Well, we're starting to glimpse a place where solid state drives are getting sufficiently large and cheap that a lot of computer systems now prefer SSD, and we may see HDDs go somewhat down the road that tape drives have...)
Rotating the HDDs so that they do get spun up fairly regularly is a good idea.
My experience some years ago with 3 USB harddrives that were rotated weekly was that the disks didn't last long. 3.5" HDs do not like being moved a lot and frequently died. Moving to tape was way way more reliable but certainly had a higher cost in terms of getting a tape drive and for recovery you might need another tape drive while a USB drive works with anything. If your backup is pretty small though, USB attached SSD seems like it could be a very reliable solution. -- Len Sorensen
Lennart Sorensen via talk wrote:
My experience some years ago with 3 USB harddrives that were rotated weekly was that the disks didn't last long. 3.5" HDs do not like being moved a lot and frequently died. Moving to tape was way way more reliable but certainly had a higher cost in terms of getting a tape drive and for recovery you might need another tape drive while a USB drive works with anything.
You tend to only need spinning rust nowadays for the largest datasets, and I find that splitting archival from active filesystems helps. The archival stuff tends to be huge but write-only: media files and tarballs and such get written and then left as-is for the life of the filesystem. That requires a much-less-frequent backup! And that reduces the size of the active filesystem plus any new archival files to the point it has a chance of fitting onto flash. Another way the world has changed is that sites are no longer such islands connected by 56k modems and sneakernet; there's more chance of being able to push a backup over fibre somewhere to a harddrive that's already offsite and doesn't need to be moved further. Having another datacentre to rsync to is ideal, as I'm not sure how much you can trust third-party "cloud" solutions. -- Anthony de Boer
On Tue, 23 Oct 2018 at 15:52, Lennart Sorensen via talk <talk@gtalug.org> wrote:
On Thu, Oct 18, 2018 at 04:21:41PM -0400, Christopher Browne via talk wrote:
Bacula has always seemed to be one of the good options out there, and running it on FreeNAS is certainly well supported.
There's nothing obviously wrong with your approach to rsync to a remote place or copy to external HDD for rotation.
Madison Kelly did a talk on something akin back in 2004; Madison was the first person I heard that particularly "championed" using USB-connected HDDs as a backup medium at the time that tape drives were only just starting to get supplanted as a backup medium.
Since then, that direction has become somewhere in between "viable" and "preferable." And it now looks like tape drives are pretty rarely used anymore, as rarity has made it difficult for vendors to boost capacity as quickly as is the case for disk drives. *Everyone* wants bigger HDDs. (Well, we're starting to glimpse a place where solid state drives are getting sufficiently large and cheap that a lot of computer systems now prefer SSD, and we may see HDDs go somewhat down the road that tape drives have...)
Rotating the HDDs so that they do get spun up fairly regularly is a good idea.
My experience some years ago with 3 USB harddrives that were rotated weekly was that the disks didn't last long. 3.5" HDs do not like being moved a lot and frequently died. Moving to tape was way way more reliable but certainly had a higher cost in terms of getting a tape drive and for recovery you might need another tape drive while a USB drive works with anything.
If your backup is pretty small though, USB attached SSD seems like it could be a very reliable solution.
I used to use a rotating set of 2TB 2.5" external USB hard drives. None of them ever failed on me over about three years use, although three out of four they were generally only accessed every week or two. I've now switched to three 4TB 2.5" drives: the heavily used one of those is now stuttering (confirming my pre-existing bias against Seagate ... the previous set were WD). The 2.5" spinning drives are somewhat more expensive than the 3.5", but they're much smaller - and, important in this use case, more built for movement and frequent spin-up/spin-down. And I have to disagree with Anthony on this one: spinning disks are going to be in use for a while as external backups: sure, they're slow, but this is a BACKUP. Cost per terabyte is immensely lower and speed isn't usually the priority, and you can put multiple copies of backups (or diffs, or whatever suits you) on one large external. -- Giles https://www.gilesorr.com/ gilesorr@gmail.com
On Mon, 29 Oct 2018 15:35:20 -0400 Giles Orr via talk <talk@gtalug.org> wrote:
I used to use a rotating set of 2TB 2.5" external USB hard drives. None of them ever failed on me over about three years use, although three out of four they were generally only accessed every week or two. I've now switched to three 4TB 2.5" drives: the heavily used one of those is now stuttering (confirming my pre-existing bias against Seagate ... the previous set were WD). The 2.5" spinning drives are somewhat more expensive than the 3.5", but they're much smaller - and, important in this use case, more built for movement and frequent spin-up/spin-down.
I have been using an internal 4TB hard drive as my nightly backup. I am now on to my second hard drive. Periodically, I transfer my backup to a 50GB Blu-Ray disk. Obviously, this constrains my /home partion to an extent that may be unacceptable to you. On a couple of occasasion now, I have pulled out months old disks to recover files I unknowingly deleted. I like to permanently archive my backups. I am getting concerned about Blu-Ray burners. Is there another cheap, high capacity medium out there? -- Howard Gibson hgibson@eol.ca jhowardgibson@gmail.com http://home.eol.ca/~hgibson
Giles Orr via talk wrote:
And I have to disagree with Anthony on this one: spinning disks are going to be in use for a while as external backups: sure, they're slow, but this is a BACKUP. Cost per terabyte is immensely lower and speed isn't usually the priority, and you can put multiple copies of backups (or diffs, or whatever suits you) on one large external.
Certainly you're going to use spinning rust for multi-terabyte backup sets, but the point I was getting at is that if that's your archival set whose files are set in stone once written then you may only need to do backups of that monthly or less often, while a small active fileset (including any recent archival additions) may be in the tens or hundreds of gigabytes range so that your dailies are small and manageable and have a good chance fitting onto flash. -- Anthony de Boer
participants (6)
-
Anthony de Boer -
Christopher Browne -
Giles Orr -
Howard Gibson -
lsorense＠csclub.uwaterloo.ca -
Tony Fernandez