[geeks] Cheap/reliable backup?

[Mike Meredith]

On Mon, 2 Dec 2013 14:20:19 -0500 (EST), Mouse wrote:
> > (like an Amazon S3-based storage system).
>
> If you're considering that, don't forget that it includes giving a
> copy of your data to someone else - in many cases, to someone in
> another country with comparatively weak data privacy and protection
> laws.

Or their privacy laws are strong enough, but only apply to citizens of
that country - i.e. _my_ data may be safe in the UK, but yours might
not be. Plus of course that's before the TLAs start playing b& GCHQ
ain't allowed to leaf through my data, but it's perfectly free to take
a look at your data.

And don't forget to check whether you're doing this as an individual or
an organisation. That makes a big difference in some cases.

On Mon, 2 Dec 2013 19:41:38 +0000, microcode at zoho.com wrote:
> I have solved (1) in the past with period rsync-age. It actually got
> me out of two real screwups. I still think it's a good idea.

I use rsync myself (plus mirrored drives) b& specifically rsync various
filesystems across ssh to a ZFS storage pool at work with a snapshot
taken afterwards. Also takes care of the "what did it look like 6 months
ago?" problem ... amazing how often that comes in handy.

> I'm trying to solve (3). I've had mixed results with consumer drives.
> Some seem to work forever, others seem to die at the worst time

Tell me about it. Consumer grade drives have also been known to lie
about whether they're failing or not. If you want to make the effort,
the ZFS On Linux mailing list archives has various comments on disk
recommendations.

But mirroring (or RAIDing) does give you a window of opportunity to
deal with a failed disk b& for example I would run one more rsync backup
and then shut down before replacing. You would have to be _very_
unlucky to have two drives fail simultaneously although it has been
known to happen.

On Mon, 02 Dec 2013 15:33:42 -0500, Phil Stracchino wrote:
> on how long it would take me to do a full backup to S3 cloud storage;
> if memory serves, it was measured in weeks.

Preload the full backup and only ever do an incremental online. My
method usually takes between 10-15 minutes to update although there's
an occasional 10-18 hour session.

On Tue, 3 Dec 2013 06:59:57 +0000, microcode at zoho.com wrote:
> Does anybody have any particular knowledge of whether BluRay is more
> reliable than spinning HD and how long it's supposed to last? I'll
> look into that next.

No, but I've heard of people using older consumer grade optical media
who found it worked well enough to begin with but as the media became
cheaper and cheaper, it tended to live for shorter and shorter periods.
In some cases it died in less than a year[0].

In the last days of when I used to burn CDs of bootable 'stuff', the
failure rate was enough to be irritating[1].

If I were to put backups on anything other than disk, I'd opt for LTO
tape. I've heard that the data life expectancy is from 20-30
years probably depending on storage conditions and how frequently it
gets written to.

On Mon, 2 Dec 2013 17:12:54 -0500 (EST), Patrick Giagnocavo wrote:
> Would ZFS replication, with snapshots, solve all the aforementioned
> issues?

Yes. However I'd be ever so slightly nervous about using ZFS
replication. Not because it doesn't work, but if you're not careful the
home and remote server may get out of step in terms of ZFS features.
And of course using something like rsync means you aren't stuck with
ZFS ... my own 'backup server' has moved from btrfs to ZFS in the last
year.

0: Although an 8-year old DVD backup I just checked reads fine. I
   probably should go through the lot to find out the overall failure
   rate.

1: 25-50%

[demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]