Greetings Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs. Looking for suggestions for programs, methods, and/or techniques. TIA Dee
You could split it into a multipart tar file: http://superuser.com/questions/198857/how-can-i-create-multipart-tar-file-in... On 3 November 2015 at 15:59, o1bigtenor <o1bigtenor@gmail.com> wrote:
Greetings
Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs.
Looking for suggestions for programs, methods, and/or techniques.
TIA
Dee --- Talk Mailing List talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
On 15-11-03 03:59 PM, o1bigtenor wrote:
Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs.
I don't trust CDs and DVDs for backup. I don't think I would trust Bluray either. I have had discs go bad on me over time. For that much data I would use an external hard drive. -- Cheers! Kevin. http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick
No question go for Harddrive. BUT don't put your trust on just one drive either. The power up and power down puts a lot of strain on the drive, you are just better to leave the drive on all the time. Better option would be a NAS with RAID setup. I have had a few drives gone belly up, lucky I had them setup as RAID and had a backup of the backup. On Thu, Nov 5, 2015 at 11:00 AM, Kevin Cozens <kevin@ve3syb.ca> wrote:
On 15-11-03 03:59 PM, o1bigtenor wrote:
Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs.
I don't trust CDs and DVDs for backup. I don't think I would trust Bluray either. I have had discs go bad on me over time. For that much data I would use an external hard drive.
-- Cheers!
Kevin.
http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick --- Talk Mailing List talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
On Thu, Nov 5, 2015 at 10:14 AM, Digiital aka David <digiital@gmail.com> wrote:
No question go for Harddrive. BUT don't put your trust on just one drive either. The power up and power down puts a lot of strain on the drive, you are just better to leave the drive on all the time. Better option would be a NAS with RAID setup.
I have had a few drives gone belly up, lucky I had them setup as RAID and had a backup of the backup.
Its a raid array I'm trying to back up. What this issue (failed array) has taught me is that hard discs aren't enough!!! A tip - - - some of the drives marketed as applicable to NAS raid arrays really aren't applicable. You need to be purchasing drives that have ERC or error recovery control. I was slapped upside the head because I had drives that didn't have that but when I bought the drives (early 2012) NOBODY was talking about that. So now starts the search for effective backup media!! Dee
| From: o1bigtenor <o1bigtenor@gmail.com> | A tip - - - some of the drives marketed as applicable to NAS raid arrays really | aren't applicable. You need to be purchasing drives that have ERC or error | recovery control. I was slapped upside the head because I had drives that didn't | have that but when I bought the drives (early 2012) NOBODY was talking about | that. I've heard that some RAID systems do know how to deal with such drives. I haven't researched which ones. Lots of us were talking about this (whining, actually). I usually called it TLER, Western Digital's term for it. ERC is Seagate's name. CCTL was used by Samsung and Hitachi. Earlier it was possible to tell a drive to limit error recovery time. Then the drive manufacturers locked this feature out on their cheap drives. Grrr. Disks with TLER / ERC / CCTL & LCC [Table of drives] (2011 March) <http://hardforum.com/showthread.php?t=1590200> I posted to this list: From: D. Hugh Redelmeier <hugh@mimosa.com> To: tlug@ss.org Date: Thu, 28 Jun 2012 21:58:59 -0400 (EDT) Subject: Re: [TLUG]: 3TB Harddisk sale | From: Anthony de Boer <adb@adb.ca> | I expect I'll be trying the WD RE4 Lennart mentions next. My understanding is that RE and non-RE are the same EXCEPT for "TLER" (a trivial firmware difference). Without TLER, RAID won't work. A drive will spend so much time recovering from a simple local error that the controller will declare the whole drive offline. That is a big failure. It generally requires the array to be rebuilt, possibly taking longer than the actual MTBF! That's how they do "market segmentation". Market segmentation is a vendor's dream: sell essentially the same product at two different price points. If the drive manufacturing industry were not an oligopoly, this price differentiation would disappear. In fact, I think Samsung's normal drives were capable of TLER; that's been fixed by Seagate taking them over.
On Thu, Nov 5, 2015 at 12:05 PM, D. Hugh Redelmeier <hugh@mimosa.com> wrote:
| From: o1bigtenor <o1bigtenor@gmail.com>
| A tip - - - some of the drives marketed as applicable to NAS raid arrays really | aren't applicable. You need to be purchasing drives that have ERC or error | recovery control. I was slapped upside the head because I had drives that didn't | have that but when I bought the drives (early 2012) NOBODY was talking about | that.
I've heard that some RAID systems do know how to deal with such drives. I haven't researched which ones.
Lots of us were talking about this (whining, actually). I usually called it TLER, Western Digital's term for it. ERC is Seagate's name. CCTL was used by Samsung and Hitachi.
Earlier it was possible to tell a drive to limit error recovery time. Then the drive manufacturers locked this feature out on their cheap drives. Grrr.
This sounds like a problem looking for a solution -- - - a software hack to fix a hardware problem - - - I know 0 about programming or I would be looking into it already!
Disks with TLER / ERC / CCTL & LCC [Table of drives] (2011 March) <http://hardforum.com/showthread.php?t=1590200>
I posted to this list: From: D. Hugh Redelmeier <hugh@mimosa.com> To: tlug@ss.org Date: Thu, 28 Jun 2012 21:58:59 -0400 (EDT) Subject: Re: [TLUG]: 3TB Harddisk sale
| From: Anthony de Boer <adb@adb.ca>
| I expect I'll be trying the WD RE4 Lennart mentions next.
My understanding is that RE and non-RE are the same EXCEPT for "TLER" (a trivial firmware difference).
Without TLER, RAID won't work. A drive will spend so much time recovering from a simple local error that the controller will declare the whole drive offline. That is a big failure. It generally requires the array to be rebuilt, possibly taking longer than the actual MTBF!
That's how they do "market segmentation". Market segmentation is a vendor's dream: sell essentially the same product at two different price points.
If the drive manufacturing industry were not an oligopoly, this price differentiation would disappear. In fact, I think Samsung's normal drives were capable of TLER; that's been fixed by Seagate taking them over. ---
Very interesting - - - time for some hacker ingenuity!!! Anyone that knows this stuff care to hand out some pointers? Dee
How about a few USB flash drives. They are relatively cheap and if your concerned about failure it should be possible to copy the data twice. There are also a few archive formats that support repair of missing data but I am not sure how well they would do with losing a significant portion of the data. I wonder if something like par2 would fit the requirements. On 11/05/2015 11:14 AM, Digiital aka David wrote:
No question go for Harddrive. BUT don't put your trust on just one drive either. The power up and power down puts a lot of strain on the drive, you are just better to leave the drive on all the time. Better option would be a NAS with RAID setup.
I have had a few drives gone belly up, lucky I had them setup as RAID and had a backup of the backup.
On Thu, Nov 5, 2015 at 11:00 AM, Kevin Cozens <kevin@ve3syb.ca <mailto:kevin@ve3syb.ca>> wrote:
On 15-11-03 03:59 PM, o1bigtenor wrote:
Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs.
I don't trust CDs and DVDs for backup. I don't think I would trust Bluray either. I have had discs go bad on me over time. For that much data I would use an external hard drive.
-- Cheers!
Kevin.
http://www.ve3syb.ca/ |"Nerds make the shiny things that distract Owner of Elecraft K2 #2172 | the mouth-breathers, and that's why we're | powerful!" #include <disclaimer/favourite> | --Chris Hardwick --- Talk Mailing List talk@gtalug.org <mailto:talk@gtalug.org> http://gtalug.org/mailman/listinfo/talk
--- Talk Mailing List talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
-- Alvin Starr || voice: (905)513-7688 Netvel Inc. || Cell: (416)806-0133 alvin@netvel.net ||
On 11/05/15 11:14, Digiital aka David wrote:
No question go for Harddrive. BUT don't put your trust on just one drive either. The power up and power down puts a lot of strain on the drive, you are just better to leave the drive on all the time. Better option would be a NAS with RAID setup.
I have had a few drives gone belly up, lucky I had them setup as RAID and had a backup of the backup.
While you're at it, I recommend you use a UPS, if you're not using one already. http://www.webopedia.com/TERM/U/UPS.html https://duckduckgo.com/?q=why+you+should+use+a+UPS Be prepared, see "Murphy's law." Daniel Villarreal http://www.youcanlinux.org/
Looking at backing up ... Want to do one copy to 25 GB Bluray discs.
I don't trust CDs and DVDs for backup. I don't think I would trust Bluray either[...]
On Thu, Nov 05, 2015 at 11:00:38AM -0500, Kevin Cozens wrote:
On 15-11-03 03:59 PM, o1bigtenor wrote:
Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs.
I don't trust CDs and DVDs for backup. I don't think I would trust Bluray either. I have had discs go bad on me over time. For that much data I would use an external hard drive.
I use rsnapshot to a remote location. -- Len Sorensen
I have done that before to my Google Drive. There is a application I used to MOUNT a google drive on my system and used rsync to make backups. http://www.tecmint.com/mount-google-drive-in-linux-using-google-drive-ocamlf... On Thu, Nov 5, 2015 at 2:20 PM, Lennart Sorensen < lsorense@csclub.uwaterloo.ca> wrote:
On Thu, Nov 05, 2015 at 11:00:38AM -0500, Kevin Cozens wrote:
On 15-11-03 03:59 PM, o1bigtenor wrote:
Looking at backing up a little less than 300 GB of files. Want to do one copy to 25 GB Bluray discs.
I don't trust CDs and DVDs for backup. I don't think I would trust Bluray either. I have had discs go bad on me over time. For that much data I would use an external hard drive.
I use rsnapshot to a remote location.
-- Len Sorensen --- Talk Mailing List talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
On Thu, Nov 05, 2015 at 03:13:28PM -0500, Digiital aka David wrote:
I have done that before to my Google Drive. There is a application I used to MOUNT a google drive on my system and used rsync to make backups.
http://www.tecmint.com/mount-google-drive-in-linux-using-google-drive-ocamlf...
rsnapshot has the advantage of keeping older revisions around of changed files. Not sure if that is easy to do with google drive (I haven't looked at it). I keep my stuff in places I control. -- Len Sorensen
Totally agree. I like to keep my stuff on a server I can control. I was just using gdrive as a option. As for rsnapshot it should work on gdrive, since it's mounted just like a normal drive. The only difference you will see is the speed of transferring files to it. It looks and acts just like a 'usb drive' . On Thu, Nov 5, 2015 at 3:37 PM, Lennart Sorensen < lsorense@csclub.uwaterloo.ca> wrote:
On Thu, Nov 05, 2015 at 03:13:28PM -0500, Digiital aka David wrote:
I have done that before to my Google Drive. There is a application I used to MOUNT a google drive on my system and used rsync to make backups.
http://www.tecmint.com/mount-google-drive-in-linux-using-google-drive-ocamlf...
rsnapshot has the advantage of keeping older revisions around of changed files. Not sure if that is easy to do with google drive (I haven't looked at it).
I keep my stuff in places I control.
-- Len Sorensen --- Talk Mailing List talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
On Thu, Nov 05, 2015 at 04:13:50PM -0500, Digiital aka David wrote:
Totally agree. I like to keep my stuff on a server I can control. I was just using gdrive as a option. As for rsnapshot it should work on gdrive, since it's mounted just like a normal drive. The only difference you will see is the speed of transferring files to it. It looks and acts just like a 'usb drive' .
rsnapshot takes advantage of a local copy being fast and then doing rsync of the delta, so I don't think it can run efficiently to a remote drive, only from a remote machine that can run rsync. -- Len Sorensen
| From: o1bigtenor <o1bigtenor@gmail.com> | Looking at backing up a little less than 300 GB of files. Want to do | one copy to 25 GB Bluray discs. I think about this but don't act on it. I don't know the perfect solution. I've had ALL kinds of media go bad or unreadable due to loss of technology. The best bet (and I don't do this) is to have multiple copies on multiple media. Actively backed up (i.e. in a rolling repeated process, onto new media as they becomes available). Physically separated (so one fire, flood, or war doesn't take it out). All media age in some ways, not all known ahead of time. I've had CD's physically destroyed by plasticizer from the window on the paper sleeve they were stored in. It took a number of years. Threats can come from out of left field. The lifetime of media seem to have a LIFO nature. Newer stuff is often more delicate (and made obsolete sooner). - stone tablets came first and outlast most of their successors ... - paper from before ~1850 last a long long time - later paper (from wood pulp) last a long time (but less than older paper). I have books that I bought new that are deteriorating from the acid in the paper. - punch cards and paper tape seem to last indefinitely (I have modest amounts) and I can read them by hand. I think I could build my own reader if I felt the urge. - 9-track tapes (I have some backups on them) aren't useful any longer because the drives are expensive and rare. Probably the recordings have decayed but I have no way of finding out. - 8", 5.25", 3.25" floppies are starting to fall off the edge. - LASER disks and magneto-optical disks appear to be gone. - MFM and RLL disks are no longer supported. - SCSI and (p)ATA disks are all but gone. But maybe your information will be unimportant before any of this kicks in. There are DVD's that claim to have very long lifetimes. They might be worth a shot. <https://en.wikipedia.org/wiki/M-DISC> Blu-rays seem to have little takeup for data. I don't even know if Linux has software to master them. I would avoid them. (I bought a Blu-ray burner but have never used it for Blu-rays. It can also burn M-Discs but I haven't used that feature yet.) USB flash-memory sticks are very convenient. I have no confidence that their lifetime will be long and reliable. Anyone know? For our most important records we still keep paper.
On Thu, Nov 5, 2015 at 10:47 AM, D. Hugh Redelmeier <hugh@mimosa.com> wrote:
| From: o1bigtenor <o1bigtenor@gmail.com>
| Looking at backing up a little less than 300 GB of files. Want to do | one copy to 25 GB Bluray discs.
I think about this but don't act on it.
I'm thinking that this is quite true of most of us!!!
I don't know the perfect solution. I've had ALL kinds of media go bad or unreadable due to loss of technology.
The best bet (and I don't do this) is to have multiple copies on multiple media. Actively backed up (i.e. in a rolling repeated process, onto new media as they becomes available). Physically separated (so one fire, flood, or war doesn't take it out).
All media age in some ways, not all known ahead of time.
I've had CD's physically destroyed by plasticizer from the window on the paper sleeve they were stored in. It took a number of years. Threats can come from out of left field.
The lifetime of media seem to have a LIFO nature. Newer stuff is often more delicate (and made obsolete sooner).
- stone tablets came first and outlast most of their successors
...
You forgot papyrus reed stuff then came parchment
- paper from before ~1850 last a long long time
- later paper (from wood pulp) last a long time (but less than older paper). I have books that I bought new that are deteriorating from the acid in the paper.
- punch cards and paper tape seem to last indefinitely (I have modest amounts) and I can read them by hand. I think I could build my own reader if I felt the urge.
- 9-track tapes (I have some backups on them) aren't useful any longer because the drives are expensive and rare. Probably the recordings have decayed but I have no way of finding out.
- 8", 5.25", 3.25" floppies are starting to fall off the edge.
- LASER disks and magneto-optical disks appear to be gone.
- MFM and RLL disks are no longer supported.
- SCSI and (p)ATA disks are all but gone.
But maybe your information will be unimportant before any of this kicks in.
There are DVD's that claim to have very long lifetimes. They might be worth a shot. <https://en.wikipedia.org/wiki/M-DISC>
Blu-rays seem to have little takeup for data. I don't even know if Linux has software to master them. I would avoid them. (I bought a Blu-ray burner but have never used it for Blu-rays. It can also burn M-Discs but I haven't used that feature yet.)
Have on and am using it but it takes about an hour to burn about 24 GB which makes it a slow process when you have say a TB of data!
USB flash-memory sticks are very convenient. I have no confidence that their lifetime will be long and reliable. Anyone know?
For our most important records we still keep paper.
The easiest to destroy, change and do - - - hmmmmmm so much for the paperless office that was promised way back in the 80s! Dee
On Thu, Nov 05, 2015 at 11:47:15AM -0500, D. Hugh Redelmeier wrote:
| From: o1bigtenor <o1bigtenor@gmail.com>
| Looking at backing up a little less than 300 GB of files. Want to do | one copy to 25 GB Bluray discs.
I think about this but don't act on it.
I don't know the perfect solution. I've had ALL kinds of media go bad or unreadable due to loss of technology.
The best bet (and I don't do this) is to have multiple copies on multiple media. Actively backed up (i.e. in a rolling repeated process, onto new media as they becomes available). Physically separated (so one fire, flood, or war doesn't take it out).
All media age in some ways, not all known ahead of time.
I've had CD's physically destroyed by plasticizer from the window on the paper sleeve they were stored in. It took a number of years. Threats can come from out of left field.
The lifetime of media seem to have a LIFO nature. Newer stuff is often more delicate (and made obsolete sooner).
- stone tablets came first and outlast most of their successors
...
- paper from before ~1850 last a long long time
- later paper (from wood pulp) last a long time (but less than older paper). I have books that I bought new that are deteriorating from the acid in the paper.
- punch cards and paper tape seem to last indefinitely (I have modest amounts) and I can read them by hand. I think I could build my own reader if I felt the urge.
- 9-track tapes (I have some backups on them) aren't useful any longer because the drives are expensive and rare. Probably the recordings have decayed but I have no way of finding out.
- 8", 5.25", 3.25" floppies are starting to fall off the edge.
- LASER disks and magneto-optical disks appear to be gone.
- MFM and RLL disks are no longer supported.
- SCSI and (p)ATA disks are all but gone.
But maybe your information will be unimportant before any of this kicks in.
There are DVD's that claim to have very long lifetimes. They might be worth a shot. <https://en.wikipedia.org/wiki/M-DISC>
Blu-rays seem to have little takeup for data. I don't even know if Linux has software to master them. I would avoid them. (I bought a Blu-ray burner but have never used it for Blu-rays. It can also burn M-Discs but I haven't used that feature yet.)
Mastering bluray videos that will work on a player with menus, I have no idea. But putting data on them is simple. I just generate a 23GB file, format it UDF and loop mount it, put my files in, then unmount and use growisofs to write the file as an image to the buray write. Mounts perfectly and reads everywhere. I suspect you could even use the udf write support in the kernel to write directly to the bluray disc but I haven't tried that method yet.
USB flash-memory sticks are very convenient. I have no confidence that their lifetime will be long and reliable. Anyone know?
For our most important records we still keep paper.
-- Len Sorensen
participants (8)
-
Alvin Starr -
D. Hugh Redelmeier -
Daniel Villarreal -
Digiital aka David -
Kevin Cozens -
Lennart Sorensen -
o1bigtenor -
Tim Tisdall