Pick of the Week - Nov 10 [Show all picks]
Path Finder 5 - A feature-laden Finder replacement
Submit Hint Search The Forums LinksStatsPollsFAQHeadlinesRSS
12,000 hints and counting!

Use Amazon S3 to automatically back up your Mac Internet
Amazon's S3 is an online storage solution; you pay for only what you use ($0.15/GB/month, plus some transfer costs). I wrote a simple step-by-step guide to setting you a Mac to sync with Amazon S3; here's the executive summary version:
  1. You need an Amazon Web Services account. Once signed up, you'll need your Amazon access key and your secret key. These are what s3sync will use to authenticate you to Amazon.
  2. You'll need this zip file (1.2MB), which contains all the files you are going to need.
  3. You need to create a "bucket" at Amazon to store your files. To create the bucket, you need of the S3 GUI applications that exist. I have included in the zip file the one I have used called S3 Browser (latest version). When creating your bucket, remember that the name has to be globally unique.
  4. In the s3backup folder created from the zip file downloaded above, you need to edit backup.sh. Replace the placeholder access key, secret key, and bucket name with the ones you obtained in the previous steps.
  5. The application will run as root at the system level in order to prevent file access issues, therefore I recommend storing the entire s3backup folder in your /Library folder. If you store it elsewhere, some paths in the backup.sh file will need to be updated.
  6. Set up your Mac to automatically run the backup shell script at regular intervals via cron or launchd.
[robg adds: The above is a very short summary of the much-more-detailed guide linked in the first sentence; make sure you read the full version for more help with this hint. Note too that S3 charges both for storage as well as for transfer in and out. You can use Amazon's Simple Monthly Calculator to get a feel for how much you would spend, based on your estimates of storage and transfer. I haven't tested this hint.]
    •    
  • Currently 4.00 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (1 vote cast)
 
[27,399 views]  

Use Amazon S3 to automatically back up your Mac | 17 comments | Create New Account
Click here to return to the 'Use Amazon S3 to automatically back up your Mac' hint
The following comments are owned by whomever posted them. This site is not responsible for what they say.
Use Amazon S3 to automatically back up your Mac
Authored by: obscuriosity on Mon, Feb 11 2008 at 8:04AM PST
FYI, if you want to back up more than 33GB, it's cheaper to use Mozy. Not affiliated with them, just a happy user.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: Spliff on Mon, Feb 11 2008 at 8:41AM PST
I've tried Mozy a couple of times and gave up on it. The Mac client is buggy as hell and slow. It won't let me max out my upstream bandwidth. I'm lucky if I can get it to upload at 20 k/sec. And often it refuses to upload at all.


[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: monickels on Mon, Feb 11 2008 at 8:14AM PST
I second Mozy. The Mac client is still in beta, but the latest version seems to do exactly what it's supposed to. Also, it is specifically a backup service, so they will send you DVDs of your data, if you request them (at a cost, of course). A colleague lost her hard drive and was able to recover nearly everything.

---
Double-Tongued Word Wrester: a growing dictionary of old and new words from the fringes of English. http://www.doubletongued.org/

[ Reply to This | # ]
Mozy
Authored by: moxieboy on Mon, Feb 11 2008 at 8:40AM PST

Can anyone else comment on Mozy? Recent twitterings [tweetscan.com] have been mixed.

That said, Mozy seems like the most cost-effective option [oreilly.com] for home users.



[ Reply to This | # ]
Mozy
Authored by: legacyb4 on Thu, Feb 28 2008 at 11:05PM PST
Mozy had some issues with earlier versions of their Mac agent; I've found the recent build (0.9.0.0 26912) to be pretty consistent and no longer hangs doing the backups.

Of course, they offer a free 2GB option which will cover most non-media data within reason; sign up with my referral link below and we BOTH earn an extra 256MB of storage capacity! Follow the link, click on MozyHome then click on the MozyHome Free link to sign up.

Mhttps://mozy.com/?code=SVU4AA

[ Reply to This | # ]

Use Amazon S3 to automatically back up your Mac
Authored by: benlan on Mon, Feb 11 2008 at 8:52AM PST
I unsecond your second.

For larger amounts of data (>100 GB), Mozy is still too buggy to be useful. It gives all sorts of spurious error messages and never seems to complete a backup. Maybe once it's out of beta, who knows...but it's buggy as hell right now.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: ShawnLevasseur on Mon, Feb 11 2008 at 11:58AM PST
I add a caveat to the unseconding of the seconding.

You really shouldn't be backing up 100GB + of data online.

Media files, your operating system, and applications are better backed up via other forms of off-site backup. (Physically writing to another drive, or optical disk, and taking that media off site, would be the better method.)

Online backups are best for critical data that changes frequently.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: Rylin on Wed, Jan 14 2009 at 4:02AM PST
That's the stupidest thing I've ever heard.
In your bandwidth starved little country, online backup might not be ideal.

I have roughly 20Mbps upstream -- 25 on a good day.
There's absolutely no reason I shouldn't back up my 140GB photo library to an off-site service via the Internet.

Yes, the first backup takes a bit of time (16 hours in this case), but everything after that is a differential backup, and takes very little time, even in the event that I go shooting a few GB of photos.

Stop being ignorant, and stop trying to hinder progress.

[ Reply to This | # ]
S3 works better for me than Mozy.
Authored by: bjdraw on Mon, Feb 11 2008 at 10:37AM PST
I was using mozy and although it saved me once, what a pain to do a full system restore.

I was backing up about 40GB which took about a month to finally complete despite the fact that I have 5Mbps upload on my FiOS connection. I use S3 just as the tip suggests now and I constantly see 5Mbps throughput. (just to check I tried Mozy again and it was still way under 1Mbps)

Mozy was always backing up more data then I thought it needed to, meaning more data then had changed since the last backup and combined with the slow speeds caused backups to fail all the time.

But in the end the real reason I stopped using it was the restore process is ridiculous. I had to sign into their site and request the files. It took almost 24 hours for them to have my files ready. Then I had to download 100 dmg files, mount them one by one and copy my data back to where it was.
This took me a good part of a day.

So now, I use Time Machine for regular backups and I use s3 for weekly backups. I only keep one version on S3. I still have about 40GB and my cost is about the same. And with S3 I can copy one file at anytime using S3 Broswer or sync my entire directory structure back with one s3sync command. The ease of backup and restore combined with the S3's ability to max out my 20/5 FiOS connection makes it worth the hassle to setup and price.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: appleman_design on Mon, Feb 11 2008 at 12:00PM PST
you can do the same w/ Gmail and it doesn't cost you a ยข

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: luhmann on Mon, Feb 11 2008 at 3:39PM PST
I'm surprised nobody's mentioned JungleDisk which does exactly this (plus a whole lot more) and works great on the Mac.

[ Reply to This | # ]
JungleDisk is junk
Authored by: lincd0 on Mon, Feb 11 2008 at 6:15PM PST
It mounts your S3 bucket as a WebDAV volume, so every process on the system has read-write access to it. This is practically the same as running everything as root. Avoid, unless you don't care about the integrity of your data.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: badtz on Mon, Feb 11 2008 at 11:39PM PST
anyone try using CrashPlan? Seems like a very good program, and does the same thing as Mozy (and more!!!) but not at a flat rate.

The benefit is that you can send it to other computers (including other family/friends computers) all encrypted.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: nekomatic on Tue, Feb 12 2008 at 2:15AM PST
Last time I checked, the Mozy Mac client didn't back up the resource forks of files - to me that makes it worse than useless. I'm trying out Jungledisk at the moment but haven't yet got round to checking it with Backup Bouncer to see what metadata it does or doesn't store. One comment is that I've just had my first bill from Amazon and the biggest charge was not for the storage or transfer but for the list/put requests - it's still very cheap though.

Anyone have any comments on the read/write access criticism? Seems to me this is just the same as backing up to a normal disk. If you only mount Jungledisk while running the backup, and quit other apps first, this shouldn't be a problem should it?

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: devinknighton on Tue, Mar 4 2008 at 3:46PM PST
Actually, as of version 0.9, we support
resource forks, symbolic links and bundles correctly.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: JungleDave on Tue, Feb 12 2008 at 8:41AM PST
It's worth noting that as of the last few releases you can completely disable the WebDAV volume and just use Jungle Disk for automatic backup and restore.

Most users consider the WebDAV volume to be a very useful feature however, as it allows you to use pretty much any backup or sync software you want with Amazon S3.

[ Reply to This | # ]
Use Amazon S3 to automatically back up your Mac
Authored by: Dornquast on Tue, Feb 12 2008 at 10:07AM PST
We love S3. So much so that we will be adding S3 as a possible destination for CrashPlan customers this year. That said, there are a few points we've discovered when working with their APIs that affect using them as a backup service.

1. Guarenteed restore. The problem with S3 is once you send your data there, you have to TRUST it's still in working order. Want to verify it's good? You have to do a restore, and that costs money. CrashPlan intelligently tests your backup archive at the destination automatically to validate your backup archive is still in good working order. You can't do this with S3.

2. It's cheaper. Amazon charges you every time you move data to and from their data center. There are no monthly fees with CrashPlan if you back up to your own destination (a friends, computer, the office, a neighbor). Our central storage is a flat $0.10 / GB / month, no bandwidth charges, also cheaper than amazon.

3. No limits. Amazon has a 4GB file size limit. CrashPlan has none. We'll leverage our engine to circumvent this limit, but most (all?) products today currently have a 4GB limit I think.

4. Better performance. S3's limited API resulting in poor performance. CrashPlan has better compression, less bandwidth.

5. Better security. With CrashPlan you know where your data is. It's not in the "cloud" you trust to work. Some people are ok with this, for me personally, I like knowing my macbook pro is backed up to my old XP box in the basement of my house & in our rack at the office.

S3 is an awesome service, but it's limited API limits what you can accomplish in terms of backup & restore. I think they're a great addition to a backup strategy, but I personally would not make them my primary off-site strategy.

2nd or 3rd tier? Absolutely.

~Matthew




[ Reply to This | # ]