PDA

View Full Version : Remember folks, Hard drive failure is a serious problem...



NikkiA
May 29th, 2010, 11:28 AM
Just reading yet another 'My hard drive failed" post on here, may I remind everyone to keep backups of their assets and routes that they create. Personally I recommend either DropBox or MozyHome (or better still, both, yay redundancy!).

For dropbox feel free to use my referral code, which will give both of us a little more space than you get by default:

https://www.dropbox.com/referrals/NTYxNjE2OQ

And likewise for Mozy:

https://mozy.com/?ref=EH5BE1


Keeping your asset backups on an online source ensures that even if the worst happens, and your house burns down, your hard work will still be there waiting for you once your insurance pays for a new house and computer :)

Don't be another of those 'Well, XXXX created that, but he/she lost the mesh in a hard drive failure, so it can't be fixed...' people

JCitron
May 29th, 2010, 01:09 PM
Already been through a stuffed hard disk myself!

The big thing is backups. I have multiple hard drives and backup to multiple places including my Solaris server. This ensures that my route is backed up.

The off-site backups are a bit sketchy, IMO. They're great for small stuff, but user data such as that from Trainz can be a bit overpowering to move around. Moving the data between hard disks takes awhile, I'd hate to wait for it go from an off-site service to my hard drive over the Internet, which is a fast connect, but it's still not as fast as a computer buss.

John

lewisner
May 29th, 2010, 01:20 PM
I can't really be bothered to save to CD's but I have had a totally reliable mini hard drive for about 4 years now and I have lost count of how many times it has saved my bacon. The Autosave which really saves in TRS2009 is a big boost too.

NikkiA
May 29th, 2010, 01:20 PM
Already been through a stuffed hard disk myself!

The big thing is backups. I have multiple hard drives and backup to multiple places including my Solaris server. This ensures that my route is backed up.

The off-site backups are a bit sketchy, IMO. They're great for small stuff, but user data such as that from Trainz can be a bit overpowering to move around. Moving the data between hard disks takes awhile, I'd hate to wait for it go from an off-site service to my hard drive over the Internet, which is a fast connect, but it's still not as fast as a computer buss.

John

You don't 'sit and wait' with either of those services, DropBox transfers in the background, and mozy is a 'backup service' that waits until your computer is idle (normally, you can force a backup obviously) at night before performing its transfer to their servers.

In both cases the data resides primarily on your hard disk, it's just backed up to the remote location when possible. Both also give you versioning support, meaning if I make a mistake and accidently delete something within 3ds max on my 3d model, I can look at it a week later, and think 'hmm, that's not right', go to dropbox or mozy, and select to recover the version from a week ago - in both cases such versions don't seem to count against your storage limits either.

I'm by no means suggesting that people keep their entire library of downloaded cdp's on such services, but if you create content, you should very certainly be keeping your 3d meshes and textures on a backup, and local backups are NOT a complete solution. Too many times I see 'well, he/she lost the mesh' used to describe why reskins can't be improved. And even many creators that have left trainz after losing all their work and feeling utterly gutted that their hard work is gone.

Both of the services I mentioned, and pretty much all of their competitors, are secure and trustable, in the case of Mozy you can provide the encryption key that the program uses to encrypt the data - so even they can't access the contents of the files.

JCitron
May 29th, 2010, 01:45 PM
You don't 'sit and wait' with either of those services, DropBox transfers in the background, and mozy is a 'backup service' that waits until your computer is idle (normally, you can force a backup obviously) at night before performing its transfer to their servers.

In both cases the data resides primarily on your hard disk, it's just backed up to the remote location when possible. Both also give you versioning support, meaning if I make a mistake and accidently delete something within 3ds max on my 3d model, I can look at it a week later, and think 'hmm, that's not right', go to dropbox or mozy, and select to recover the version from a week ago - in both cases such versions don't seem to count against your storage limits either.

I'm by no means suggesting that people keep their entire library of downloaded cdp's on such services, but if you create content, you should very certainly be keeping your 3d meshes and textures on a backup, and local backups are NOT a complete solution. Too many times I see 'well, he/she lost the mesh' used to describe why reskins can't be improved. And even many creators that have left trainz after losing all their work and feeling utterly gutted that their hard work is gone.

Both of the services I mentioned, and pretty much all of their competitors, are secure and trustable, in the case of Mozy you can provide the encryption key that the program uses to encrypt the data - so even they can't access the contents of the files.


I'm well aware of backups and data security, and the fact that Mozy and others are most likely using a changed-data backup sytem. In other words, your data is only backed up when it has changed from the last backup. This works well until you do a restore of everything from scratch. The archive bit is reset, and therefore according to the backup client, nothing has changed since the previous backup.

The fact that the software would be running its thing in the background would drag down the performance. My machine is never really idle, and when it is it's off. I've run services such as this before, and found the tasks running in the background to be a drag on performance.

For decades I was involved in datacenter backups, and relied on various media that was available at the time. This ranged from tape to worm drives. All of this media at the time proved very reliable, and by keeping the data off-site the company would retrieve it only when there was a need for a restore. This works well in a corporate environment. These off-machine storage systems have proven their worth in their time, but as data size has grown, the backup time window has shrunk.

As hard drives have grown more reliable with much lower costs, they can be used instead of tape, which can become unreliable with age; it's the nature of the beast. In fact Mozy and others most likely use SANS or iSCSI networks configured with drive arrays setup to emulate tape drives. This is the solution to tape, which wears out and limits the amount of space. Storage cartridges now are hard drives, and no longer DLT tapes.

Like many people here I have, as I said above, multiple hard drives. One of them is something I've mentioned before, and highly recommended. Thermaltake's BlackX, which sells for about $60.00 US, is a USB based unit that holds any SATA drive. Simply plug the unenclosed hard drive into the base and go. It's hot swap-capable, and works well.

http://www.thermaltakeusa.com/Products.aspx?C=1346

I've setup a simple routine, which is not unlike what your software is doing for you. I backup my data, which is mostly my route and documents to my internal backup drive. At the end of a week, I then offload this data to my externally mounted 1TB SATA drive. I catalog my data by date, and can simply find what I am looking for.

I suppose it's all about priorities, but personal data to me doesn't have the priority-level that corporate data does, which losing that can cause legal and financial inplications in a big way. The fact that you are a content creator, perhaps puts you in the former category, but again having yet again something else running on the machine along with the antivirus/firewall, and anything else that everyone puts on there, only adds to the performance drag on the hardware.

John

wholbr
May 29th, 2010, 02:48 PM
Hi Everybody.
You're perfectly right there John. The best and most practical way to backup is on to a removable hard drive. All I do is to open up the removable drive on the desktop and then drag the whole 2010 folder across to the removable drive (2010 being sole version I use now).

It only takes a few minutes to copy the whole folder from my computer and then if anything goes wrong you can either replace the whole folder in 2010 with the copied folder or just replace any subfolder which has gone corrupt. I find the foregoing simple, easy-to-use and as stated it can be done in a couple of minutes.

All you've got to do is to remember to backup regularly just in case something happens.

Bill
Nb: sorry I did not get back to replying on the other thread John, I was asked to do an accident investigation by my old employer which I have been doing over the last couple of days. Thoroughly enjoyed the thread though, it really was a good laugh.

johnwhelan
May 29th, 2010, 02:51 PM
If you do get a hard drive failure step one is check the cables on the drive are firmly on especially the power cable.

Cheerio John

HiBaller
May 29th, 2010, 03:14 PM
All of the above is definitely true, but here's a tale that will curl your hair:

Four years ago I bought 2 USB external 80Gb drives (Maxtor) and hooked them to two different computers. I had scheduled backups running on both computers that not only saved the data on the local machine, but also saved it remotely on the other machine. Thus I had redundant backup - or so I thought.

Six days ago, one of my external drives crashed with a "not initialized" error and started clanking like a tank gun turret. Scratch one set of data.

I was super busy writing some software for a client and didn't take the time to start another backup set on a new external drive.

Last night, the SECOND external drive rolled craps and did the same exact thing. When I took them apart, the serial numbers differed by only 573. This, of course, spelled the demise of every bit of Trainz content (paid and free) that I [used to] own. I also lost all the backups of my routes, routes in progress, and downloaded routes/content.

So now what should a person do? Have triple redundancy?

By the way; Microsoft has a wonderful free tool called SyncToy (found on their site) that can be scheduled to do three different types of backup. I have used it for about six months now and love it.

Bill

NikkiA
May 29th, 2010, 03:19 PM
HiBaller, not surprised, despite what everyone else seems to think, external drives are an absolutely awful choice for backups - consumer external enclosures are over cramped and overheat incredibly easily, on top of that they are usually fitted with undersized fans that will not last more than a few months of use. That two drives would go at around the same time isn't really surprising at all. On top of that, a drive sat unused is just as likely to die as a drive being used - more so, actually, since the principle cause of hard drive failure these days is the fluids in the spindle bearings drying up - keeping the spindle turning actually halts the fluid drying up, slightly.

I'd much rather trust my backups to an offsite source where their business model depends on them being reliable and using constantly checked and maintained professional quality drive arrays.

HiBaller
May 29th, 2010, 03:31 PM
Well, I certainly was surprised that two of them would go that closely together. I went down to Cincinnati (Microcenter) today and bought four 1TB drives and plan to make a small server farm that will be sort of off site (in my basement) and direct all my further backups down to it/them.

I find now that one of the crashed drives will initialize once in about ten tries. Each time I get it going, I drag off old data as fast as I can. I have about half of it now (total is around 27GB).

I suspect heat is what got to the two drives. When I opened the second failed drive it was right after it failed and the drive itself was almost too hot to hold comfortably in my hand. The fans were turning in the enclosure, but not doing their jobs apparently.

Bill

VinnyBarb
May 29th, 2010, 06:42 PM
The price of back up USB external hard drives is that cheap nowadays, there is no excuse not to have one for this. By the same token, hard disk cradles, where one sticks any ordinary SATA or even IDE hard disks in when one needs to, are from around $20 onwards. I have a hard disk cradle where I can either stick a common 2.5"" or a 3.5" hard disk in. As I upped the hard disk in my laptop from 160 GB to 500 GB, I also have this spare 2.5" HD for back ups. Plus I use my redundant older surplus smaller sized 3.5 hard disks (after upgrades) for just that purpose too.

The first thing I do after finishing a session on my PC, whether route building or content creating, is to back this up to a second internal hard disk (not to another partition!) in my PC. Once a week or less I also back this up on to an external USB hard disk, in effect I have 3 copies less than a week old in most cases at a given time.

As I had been there too in the past and had a couple of hard disks bite the dust on me and I never want to go through the same frustrations and tearing out of hair I did then.

Cheers

VinnyBarb

HiBaller
May 29th, 2010, 07:12 PM
I know what you mean. I have 14 spare disks ranging from 750Mb to 120Gb on the shelf that I use for all kinds of backups. Larger perhaps than a memory stick, but just as useful for storage. If kept on the shelf until needed they can last quite a while.

Bill

meatloaf747
May 29th, 2010, 10:40 PM
For backing up Trainz's an external USB 3 Hard Disk Drive is the way to go. (up to 10x faster data transfer rate of USB 2)... You'll need a motherboard that has a spare PCI slot for the USB 3 PCI card.
(NOTE; that's the old PCI slot - Not PCI Express video slot/s)...
Cheers, Mac...

RRSignal
May 30th, 2010, 12:01 AM
DIVERSIFICATION IS THE WAY TO GO!

By that, I mean using different types of media.

Years ago, I backed up onto 180 and then 360 kb floppy.

Then 720 and 1.44mb floppy. Relatively few of which survived...a hot mess.

CD? Great, but it depended on the maker of this discs.

Bernoulli? SyQuest turned out to have highly failure-prone removable HDs.

Tape? Good, but expensive and variable. And tapes, like any other magnetic media, are prone to degradation.

DVD - Good if you have the right media (authentic Taiyo Yuden, procured from Supermediastore, Meritline, or Rima.com) but media is widely counterfeited by sleazy players.

Offsite - Good if the service is reputable and diligent. Also a good idea that you check and make sure if a credit card payment bounces that they do not wipe out everything.

USB/FW HD - Great, but only in addition to the above.

Seconary HDD - Ditto.

What I'm saying here is that you need to diversify your backup solutions, especially over the long term. It's always a good idea to have more than one solution, in case one doesn't work out due to reliability issues (1.44mb, some QIC-40 formats, SyQuest, etc.) It helps that parity systems exist now (QuickPar, etc.) which help increase data redundancy on questionable media.

JCitron
May 30th, 2010, 12:05 AM
HiBaller, not surprised, despite what everyone else seems to think, external drives are an absolutely awful choice for backups - consumer external enclosures are over cramped and overheat incredibly easily, on top of that they are usually fitted with undersized fans that will not last more than a few months of use. That two drives would go at around the same time isn't really surprising at all. On top of that, a drive sat unused is just as likely to die as a drive being used - more so, actually, since the principle cause of hard drive failure these days is the fluids in the spindle bearings drying up - keeping the spindle turning actually halts the fluid drying up, slightly.

I'd much rather trust my backups to an offsite source where their business model depends on them being reliable and using constantly checked and maintained professional quality drive arrays.

Heat kills drives. I've seen this before with old Micropolis drives (I don't want to date myself, but Micropolis have been gone at least 10 years now). They would heat up and seize. Old Seagates used to have sticktion problems, which happened when the lubericant dried out inside the drive. The heads would stick to the platter surface, and the only way to jiggle them free was to keep power-cycling the drives. Eventually they would pop loose.

The Thermaltake BlackX I've posted about here is not an enclosure. The drive simply sits in the device and is exposed to the air. That's it, simple and easy. Need more space? Get another drive and plug it in. The system isn't perfect, but it beats toasting hard disks in little plastic boxes.

John

HiBaller
May 30th, 2010, 11:01 AM
You bet John. I've been working with computer hardware/software/program development since 1963 and heat was always the #1 killer. In the Navy, I tended the operating system for our HFDF program. The processors, I/O units, memory units, ran down one wall and across another. Each box was about 3 feet wide, 7 feet tall, and six feet deep. Each box had a huge, 12-inch air conditioning duct under the floor that roared chilled air into them. It was a Bunker-Ramo and Burroughs setup that filled the room with noise. I loved it.

I am working on an extruded aluminum plate that is 1/4 inch thick. It can hold 6 of my old IDE drives when they are bolted down to it. The whole thing slides into a box with two fans in it (one at each end). Each drive has a USB to IDE dongle and they all lead to a USB 2.0 hub with only one connection to the computer which will act as a server.

There will also be two DVD-writers in the computer with burning software for full backups. Once every two weeks I plan on burning the current backup to DVD. Backup redundancy.

@rrsignal:

I used to have an Iomega backup system also. One that used 1GB plug-in disk cartridges. My tape backup was one that hooked to the printer port. Not much good now because computers nowadays rarely have printer ports. Our backup plan in the Navy was to 12-inch reel-to-reel IBM tape consoles. We had a huge rack at the rear of the room for tapes. We backed up every 8-hour watch.

Bill

RRSignal
May 30th, 2010, 12:37 PM
Ah, the Zip drive. I still have one, and I think it still works! I had a Ditto...that was a good, reliable tape.

NikkiA
May 30th, 2010, 12:39 PM
Ah, the Zip drive. I still have one, and I think it still works! I had a Ditto...that was a good, reliable tape.

1GB carts would be the Jaz drive, rather than Zip (100MB, 250MB then 750MB), most likely...

See: http://en.wikipedia.org/wiki/Jaz_Drive

wholbr
May 30th, 2010, 02:22 PM
Hi Everybody
I think I must be missing something in this thread as there seems to be a lot being written about a problem that although serious is easily resolved.

As I have already stated I backup my entire Trainz 2010 folder to a separate hard drive other than the two enclosed in the PC. The backup one is connected via a USB cable to the PC and has its own power on/off switch. It resides on the top right-hand corner of the desk and is usually switched off even when the PC is running.

The only time it is powered up is when I wish to backup my Trainz folder or reports I have done for my work (I normally also use Google to remotely backup my work documents only so I can access them anywhere I go) . However this is not necessary with the Trainz folder so the only time the desktop hard drive is switched on is when a backup is going on. Straight after the backup is complete the desktop hard drive is switched off again.

The above action means that the desktop/backup drive is only ever powered up for a couple of minutes each time before it is switched off again. Therefore the chances of this hard drive breaking down through wear on the spindle, bearings etc must be absolutely minimal. There are probably far more chances of a breakdown on one of the remote servers than there would be on my backup drive.

To put it in a nutshell, there have been times when I have had trouble accessing Google due to problems on their servers, but in the three years I have had my desktop/backup drive I have never had any trouble accessing that. It's probably never had more than a few hours use in all that time.

Bill:)

NikkiA
May 30th, 2010, 03:40 PM
The above action means that the desktop/backup drive is only ever powered up for a couple of minutes each time before it is switched off again. Therefore the chances of this hard drive breaking down through wear on the spindle, bearings etc must be absolutely minimal. There are probably far more chances of a breakdown on one of the remote servers than there would be on my backup drive.

That would have been true for a hard drive made in the 1990s, however, as I stated, modern hard drives use fluid bearings, the main cause of hard drive failure is the bearings drying up. The fluid is, iirc, a form of liquid silica lubricant that is partially kept liquid by application of force from the movement of the drive - it is designed, essentially, to be kept liquid by frequent use of the drive.

It is entirely possible for this fluid to 'go dry' even when you're not using the drive, thus you can just come to use the drive one day and it will grunt and give up.

wholbr
May 30th, 2010, 04:12 PM
Hi everybody and Nikki.
Thanks for the advice Nikki I did not realize that could happen. I always enjoy your postings and suggestions you give to so many users. I will take your advice on board and get my oil can out on my desktop drive first thing in the morning.

Bill:hehe:
Nb:-have you received your prize yet for winning the earliest computer user thread the other night.