Don't rely on USB attached back-up; Need suggestion for good backup solution.

[Och]

A very frustrating thing just happened to me. I have two 10TB drives that I use for back up. I connect them through USB and copy whatever I want to back up to both of them.

Yesterday I decided to clean up and rearrange my whole back up. I plugged in one of the drives, rearranged, cleaned up, added extra stuff, etc. Then I plugged the other one, erased it completely, and copy/pasted everything from the first drive. Given that its 10TB, filled with about 7TB, the process was going to take many hours, so I left it to proceed overnight.

Today, when I came back and unlocked the computer, there was no copy process open. I though that the process completed successfully, but when I looked at the 2nd drive, only about 2TB of data was copied. The process somehow stopped in the middle, no error or anything. I try to use "safely remove hardware" icon, it says cant be removed. After several tries, I just reboot the PC.

When Windows boots up, it presents me with a message "You need to format the disk in drive before you can use it" for both of the drives. Somehow the partition on both of them became "RAW". Apparently it often happens to USB drives when they aren't removed properly or whatever other reasons. This is very frustrating for me right now, I had files on these drives as far back as 1997.

Right now I have data recovery software running, that will literally take days. Meanwhile, whether or not I am successful at recovering the data, I can no longer rely on USB. I am thinking about getting a Synology diskstation NAS.
https://www.newegg.com/synology-ds718/p/N82E16822108680

Is this a good choice?

[Trunks0]

OOooo ouch. Yea i lost 3Tb of data I had on an external 3Tb USB 3.1 drive and learned my lesson. Luckily the worst of that loss was about 2yrs of photo's , it hurt but I've stuck to internal drives every since and back up to a 2nd PC.

One little tip though. If you do go with a NAS, avoid doing raid. Recovery is a bitch if the raid goes wrong.

[Hapatingjaky]

Had my NAS die on me the other day, felt great.

Just the enclosure **** the bed, the dual 4TB WD Red's work no problem, just the QNAP TS251+ did a die on me. Have not replaced it yet.

I will probably go with another NAS since this one lasted me 4 - 5 years before dieing.

[Och]

Originally Posted by Trunks0

OOooo ouch. Yea i lost 3Tb of data I had on an external 3Tb USB 3.1 drive and learned my lesson. Luckily the worst of that loss was about 2yrs of photo's , it hurt but I've stuck to internal drives every since and back up to a 2nd PC.

One little tip though. If you do go with a NAS, avoid doing raid. Recovery is a bitch if the raid goes wrong.

That is a very good point actually. What would be a good method then - NAS but copy everything manually between two drives to duplicate?

[Nagorak]

1) Buy a NAS and run mirrored drives in RAID 1. Everything is duplicated automatically, so if one drive dies the other is still good. (You can also do RAID 5, but it requires more hard drives and isn't worth the hassle for most people's purposes.)

2) Back up everything important on the NAS to an online backup service.

This way, if one drive dies, you buy a replacement, rebuild the mirror, and just keep rolling. If something more serious happens, like your house burns down, or your NAS dies and fries both hard drives simultaneously, then you can rebuild from the online backup. If there's a nuclear war, then you might be **** out of luck, but if you survived you probably have bigger problems anyway.

[Och]

Say if I am running Raid 1 on NAS, and one of the drives dies, how easy would it be to recover? Are NAS devices smart enough that I just have to swap the dead drive, or is there a lot more jumping through hoops than that?

[Och]

Now that I am thinking about it - why don't I just build a backup PC instead of going with NAS. I already have a spare motherboard with i5-6500 CPU and 16 gigs of ram. I also have a ton of old 120-250gb SSDs - I can use one of them for OS. All I need is a case.

What OS should I go with? This is going to be purely for backup functions. I will have the computer connected to my house network, but I want to block it from internet access. The computer wont have a screen, so it needs to be easily accessible by remote desktop and file sharing.

[drumphil]

Originally Posted by Och

A very frustrating thing just happened to me. I have two 10TB drives that I use for back up. I connect them through USB and copy whatever I want to back up to both of them.

Yesterday I decided to clean up and rearrange my whole back up. I plugged in one of the drives, rearranged, cleaned up, added extra stuff, etc. Then I plugged the other one, erased it completely, and copy/pasted everything from the first drive. Given that its 10TB, filled with about 7TB, the process was going to take many hours, so I left it to proceed overnight.

Today, when I came back and unlocked the computer, there was no copy process open. I though that the process completed successfully, but when I looked at the 2nd drive, only about 2TB of data was copied. The process somehow stopped in the middle, no error or anything. I try to use "safely remove hardware" icon, it says cant be removed. After several tries, I just reboot the PC.

When Windows boots up, it presents me with a message "You need to format the disk in drive before you can use it" for both of the drives. Somehow the partition on both of them became "RAW". Apparently it often happens to USB drives when they aren't removed properly or whatever other reasons. This is very frustrating for me right now, I had files on these drives as far back as 1997.

Right now I have data recovery software running, that will literally take days. Meanwhile, whether or not I am successful at recovering the data, I can no longer rely on USB. I am thinking about getting a Synology diskstation NAS.
https://www.newegg.com/synology-ds718/p/N82E16822108680

Is this a good choice?

The issue there isn't USB.

It's that you let yourself get to a position where there was only one copy of something. After you formatted one drive you no longer had any backup of some of your stuff.

If you've ever at a point where a single failure can cause data loss, it's the process that is at fault.

Might be less likely to happen with an internal drive, but ALL drives fail eventually.

Yes, I know... "Thanks Captain Hindsight, that's very helpful now"

[Nagorak]

Originally Posted by Och

Say if I am running Raid 1 on NAS, and one of the drives dies, how easy would it be to recover? Are NAS devices smart enough that I just have to swap the dead drive, or is there a lot more jumping through hoops than that?

So far I've never had to do it, but my understanding is you just take the dead drive out, and then put a new one in, log into the NAS and it should prompt you to repair the array. That's literally the sort of thing they're designed to handle, so it should be fairly straightforward.

Originally Posted by Och

Now that I am thinking about it - why don't I just build a backup PC instead of going with NAS. I already have a spare motherboard with i5-6500 CPU and 16 gigs of ram. I also have a ton of old 120-250gb SSDs - I can use one of them for OS. All I need is a case.

What OS should I go with? This is going to be purely for backup functions. I will have the computer connected to my house network, but I want to block it from internet access. The computer wont have a screen, so it needs to be easily accessible by remote desktop and file sharing.

Honestly, I'd just eBay that extra equipment and use it to buy a purpose built NAS. It will be less of a hassle and will use less power. Just my personal opinion.

No doubt there's a way to build your own file server, but it's going to require a lot more time and effort on your part. You also don't need hardware anywhere near that powerful simply to serve as a file server.

If you want to do it, more power to you, though.

[drumphil]

but it's going to require a lot more time and effort on your part.

I dunno. Install windows. Share folder on backup drive. You're right about the using more power bit though, unless you get an ultra low power CPU, like an atom, or one of the ULP intel i-something CPU's.

Also means that unlike with a NAS, if the computer dies and the HDD is still fine, you can plug it into any windows computer and access the data.

For systems with data that is important, and where downtime matters, I use mirrored RAID on the workstations, an automatic backup that sends the data to another windows computer, and then backups from that to an external USB drive or yet another computer.

That way if any single HDD dies, the workstation can keep going, and there is at least two copies of the data still existing.

[Hardwood]

Originally Posted by Och

I can no longer rely on USB.

My 2TB external USB failed or it was just causing some issue, can't remember. Anyhow, took it out of it's enclosure noted it has typical sata connections so I put it in the computer as a normal Sata6 drive. No problems since.

[Munkus]

I put everything I can't live without onto OneDrive. I only have 1 TB cloud storage, so I have to be picky about what I put there, but it's usually enough.

[DigitalDemon]

Originally Posted by Trunks0

If you do go with a NAS, avoid doing raid. Recovery is a bitch if the raid goes wrong.

True. Raid is redundancy, not a backup. Also, data does not exist until it exists in 2 places. It is not backed up until it exists in at least 3. Personally, since he has the spare hardware, I would go with building a homeserver. Mine is storing almost 150TB now and I still don't use raid (for backups).

An alternative, if you have the bandwidth, is Amazon's newish S3 Glacier deep storage. Costs $1 per TB/mo. They will tickle your butthole if you ever want it back, so it's for really long term storage.

[Mahjik]

Just curious, what are you backing up that you cannot live without?


I archive photos and videos to DVD. Important documents, I use Google Drive. Anything else can be re-installed.

[Och]

I was able to restore pretty much everything, just waiting for the NAS to arrive now.

[Payne3d]

for $399, wouldn't it be better to build an Unraid box?
With Unraid you don't have to RAID the drives and you can still have redundancy.

[atomBomb]

Anyone using FreeNAS? Just curious how its doing these days. Wouldn't it be the OS to go for a NAS?

[A2597]

cheap option? Backblaze.
Unlimited backup of a single computer (including external drives).

Note that it's a 30 day backup of what you have on your drive. After 30 days, files you delete from your PC are also deleted from the backup. It's really more a disaster recovery solution.

[Greasy]

I have a file server, and use Stablebit drive pool to handle my local replication. You can tell it to replicate specific folders across as many drives as you want. I also have offsite backup for the important stuff as well.

[Nagorak]

Originally Posted by DigitalDemon

True. Raid is redundancy, not a backup. Also, data does not exist until it exists in 2 places. It is not backed up until it exists in at least 3. Personally, since he has the spare hardware, I would go with building a homeserver. Mine is storing almost 150TB now and I still don't use raid (for backups).

An alternative, if you have the bandwidth, is Amazon's newish S3 Glacier deep storage. Costs $1 per TB/mo. They will tickle your butthole if you ever want it back, so it's for really long term storage.

RAID 1 is just mirrored drives, so I don't see it being that hard to recover a lost drive in that case. Other types of RAID are more complicated, of course.

As far as Glacier goes, I found it a bit of a hassle to use. However, other online backup options are slow as hell, so you just have to pick your poison. "Unlimited backup" are basically limited by incredibly slow upload speeds. It literally took over a year to fully backup my data to Crashplan. Granted that included several TB of lossless video files that I haven't gotten around to processing yet.

[Akumajo]

If you are using Windows 10... don't know what to say... but if you are using Windows 7... always make sure the process is actually transferring and that you restart the computer and double check that things were copied... otherwise it could have had a transfer hang or some other stupid thing that Windows 10 will do.

[Seyiji]

Originally Posted by Akumajo

If you are using Windows 10... don't know what to say... but if you are using Windows 7... always make sure the process is actually transferring and that you restart the computer and double check that things were copied... otherwise it could have had a transfer hang or some other stupid thing that Windows 10 will do.

Watch out MS! Big bad rock star. Windows 10 gonna slash your ass with a chain saw.

[Och]

Bought the 718+ and I can't be happier. This thing is fast and super convenient. Have it tucked away inside the rack, no more messing with USB.

[Nagorak]

Good choice. I've used Synology NAS's for years now. Currently I have a 218+, and previously I had a 213j (I think, been a while). I've been pretty happy with them over the years.

[Och]

Originally Posted by Nagorak

Good choice. I've used Synology NAS's for years now. Currently I have a 218+, and previously I had a 213j (I think, been a while). I've been pretty happy with them over the years.

I had a 214 for my media server, I will replace it with the 718 as well. The CPU in the 214 is just dreadful, the 718 is blazing fast with the celeron.

[Babel-17]

I had that happen a few times to some spare drives I had mounted in an external enclosure. Yadda, yaddda, yadda, apparently I had a bum cable that came with the drive. I ordered a new one from Amazon, and that too was glitchy due to having been folded too tightly. I was using a USB 3 cable from a known good WD external drive, so I knew my enclosure could work well with a good cable. Amazon took my word for things, shipped me another cable for free, and that one has worked well.

Ironically, I no longer use the enclosure much at all these days, as I scored an 8 TB WD USB 3 external HD from Best Buy during a crazy good sale. Still, it felt very gratifying to have worked things out, and I have the enclosure, and some spare drives, for if/when I need them.

[kboye]

Out of Unraid, DrivePool, SnapRaid (combo with DrivePool), FreeNAS, LTO/LTFS, and Windows software RAID so far the last 10+ years of failures/issues like writes onto bad sectors (those are really dangerous), crashes etc the best experience has been NAS4Free at RAIDZ2 as iSCSI target using 2x1GBit NICs.

That thing was stable, quick (low seek times), fast (~200MB/s). No need for PrimoCache, can make use of L2ARC and ZIL. Easy recovery. Snapshots.

I even accidently formatted the whole "drive" in Windows by accident. Undid it 15 minutes afterwards.

So heres a tiny detail few will mention about DrivePool: every single file I/O is processed. Just doing a directory Properties at say 250.000 files will take at least twice as long compared to local non-DrivePool array. Do it twice cached, still the same speed. Think virus scans, picture thumbnails, searching, games. I have a Hyperspin setup on a DP array now, and compared to iSCSI it feels almost like comparing a 7200RPM with an SSD. Not bad, but could be so much better.

DP and Unraid does have the spindowns going for it, but even then I had a few issues with SAS drives, some normal SATA ones, Greens and Reds worked OK. And getting Windows with DP to behave these days is not easy: kill/disable BitLocker service to avoid spinning up drives, avoid open shares on another computer (they keep polling the server/drives for directory changes), Windows Update is a nightmare (1903 now with the 'anti-sleep' issue where it wakes up a sleeping computer=full spinup).

At some point just using ZFS makes life simple Sprinkle with Wake on LAN, manual shutdown or spindown using cron jobs if needed. Can't imagine how Optane does on ZFS.

Though nothing beats Windows shares like Windows itself I think? SMB3 using multiple NICs works almost exactly like multiple NICs with iSCSI. Large single files scale multiple NICs. Exampke: NAS4Free or Unraid CIFS has high write speeds, but slow random access/IO. Setting up a NAS4Free iSCSI extent, connecting and sharing this on Windows was (in my case) quicker than going directly with CIFS. Almost felt like acessing a local drive directory shared on a physical Windows 10 box.