Hi, I have a question about Ghost.

I've been using Ghost 2003 for several years, but I've always wondering this question.  Here it is:

I know Ghost can restore image to an UNFORMATTED disk directly, and I have proven this to be true (somewhat...still suspicious)
My concern is about the safety of restoring an image to an unformatted disk.  
For instance, I can restore a 200MB image to a new 200GB unformatted partition/disk  in just 1 min or less (new disk).  While it seems to work and I can read the restored data fine in XP, I've always wondering whether this is safe or not.  
It's too difficult for me to believe when I imagine how DOS or GDisk or even Windows XP needs at least 15-20 minutes to format the whole 200GB space, while Ghost can simply restore the image in less than 1 min and the disk is suddenly seen as completely formatted.

Some people gave me several answers which I still doubt very much:

1.   that Ghost is formatting the space at the same time it writes data back to the disk.  This doesn't make sense to me.  If that is the case, that means Ghost only formats the space as much data as found in the imagefile (200MB).   What about the rest of the free space that accumulates to 199.8GB (assuming a 200GB partition/disk) ?   Meanwhile, I admit that I can still write or read the data from the rest of the space available without a problem in XP.    However, XP sometimes gets locked up for no reason everytime I do this (while still freshly ghosted) , which seems to get stabliized over several days.  Coincidence ???

2. Some people say that XP is automatically formatting the rest of the sectors everytime it needs to write the data to those (unformatted) sectors.  This takes place automatically behind the scene without the user knowing it.   This also doesn't make sense to me because I didn't notice any slowdown in writing performance,  nor I have seen XP is formatting those sectors in the background when the computer is idle. I can see this from the HD led which doesn't show any disk activity.

For experiment, I have even tried to ghost a small imagefile containing only 20MB (MEGABYTES) of data to a new 250GB unformatted partition.  Ghost completes the imaging in only 5 secs or less, and HELLOOO....the 250GB partition is suddenly seen as formatted by XP.  

What happens?   How safe is this approach ?

Thanks

i think it would quick format instead of normal long sector format.

Serville,

Your 20 MBs of data came from a formatted partition. The image of that partition was restored to unallocated space and the restored partition has the same formatting as the original partition. Ghost didn't format the unallocated space, it restored the image and resized the restored partition.

Here is an interesting thread about restoring to unallocated space.

http://radified.com/cgi-bin/yabb2/YaBB.pl?num=1155827177;start=0

Serville wrote on Jan 11^th, 2009 at 10:48am:

What happens?

The classic old-style kind of "formatting" of a disk - from the 1970's and 1980's when MFM and GCR disk controllers were in use - did two-and-a-half things.

On of them, which is still important now, is to lay down the filesystem structures which the operating system uses to lay out the disk and turn the big bag of sectors into structures like directories and files. The structure necessary for this isn't all that large, and doesn't take a lot of space, so this part is quick.

The other main aspect of formatting in the old days is something that is taken care of by the disk hardware these days (except for floppy disks). Every sector on a hard disk is wrapped in a bunch of extra data which is necessary for the disk controller to synchronize clocks with and decode the magnetic pulses coming from the disk heads and turn them into recognizable bits (fields which used to be called sync, address, gap, and so forth). In the old days the magnetic media tended to be completely blank, and so there were special processes needed to write all this initial data to the raw surface so that the normal read/write process could work.

[ As background to this, it's worth while noting that a recorded MFM magnetic signal consists of magnetic changes which needed to be in effect an AC signal, not a DC one, so the recording process for regular data consists of frequency-modulating a carrier signal; the reading process consists of locking on to the carrier for that signal and decoding the modulation. The disk controller hardware for MFM floppy and hard disks took care of discovering the carrier, matching the address field, and then reading or writing MFM-encoded data field. ]

Nowadays for floppy disks this process still exists, but it's long been extinct for hard disks; pretty much since the introduction of the IDE hard disk system in the early 90's, the hard disk read/write hardware is integrated in the disk you buy, and all the necessary formatting has been done. Software in the host machine isn't involved any more in this process, because the way the data is encoded on the disk is completely different to what it used to be.

So, classic "formatting" is now only used for floppy disks, since the technology for those hasn't advanced since the 1980's and which still use MFM recording.

The "half" thing that used to result from software doing the low-level format process is that the low-level format process would result in discovery of damaged, unreadable sections of the media, and the format code could then feed that on to the next stage (the layout of the blank initial filesystem) so that the operating system wouldn't try and use those damaged areas. This is now (except for floppy disks) pretty much a complete waste of time, since the way the integrated controllers in modern hard disks work they have in effect taken care of this too, below the level that operating systems need to be aware of.

So, a "quick" disk format just consists of the one part of this process that is still actually useful these days; laying down a blank filesystem structure.

When Ghost is restoring a filesystem (which involves re-laying it out to fit the disk that you are restoring it to) it recreates all that structure, either by taking the structure in the image and tweaking it for the new disk, or just making up something fresh and copying the files into it. The fine details of this process are different between restoring FAT and NTFS or EXT-type filesystems, but the end result is always the same; a perfectly valid filesystem.

Now, that is the answer I've been looking for.  It makes sense now why Ghost could format such a big partition so lightning-fast, while other format utilities need so much time to do the same task.

I wonder why everybody (even Microsoft) still persists on using the old techniques to format new modern harddrives.  It is clear from your explanation that such approach has become obsolete.   Why no format utility can make use of this technology to shorten the time required to format  a new disk.

With this new understanding,  I assume I can keep a small ghost image containing dummy file (say 5-10MB) and use that to "format" every new harddrive,  so I  can skip all the lengthy process of the normal formatting method.    Is this correct ?

If that is true, that will save a huge amount of time for me.  I have lots of friends asking me to build new computers for them.  And with several partitions on the new disk (mostly 500GB-750GB), I always spend a lot of time waiting for XP to finish formatting them.

Serville wrote on Jan 11^th, 2009 at 7:52pm:

I wonder why everybody (even Microsoft) still persists on using the old techniques to format new modern harddrives.

Actually they really don't. The majority of computer have been manufactured using disk imaging anyway for about a decade now, whether with Ghost or one of the competing products; OEMs in particular cottoned onto the performance benefits of this as soon as Ghost appeared on the market in the mid-90's.

Quick-formatting has also been an option in most OS's for a very very long time, and in effect it's the default for the installation process on most newer OS's from pretty much everyone. For Windows from the command line, use the /q option for the "format" command to get this effect or with the "quick" option to the format script when using the DISKPART command.


Serville wrote on Jan 11^th, 2009 at 7:52pm:

I assume I can keep a small ghost image containing dummy file (say 5-10MB) and use that to "format" every new harddrive,so Ican skip all the lengthy process of the normal formatting method.Is this correct ?

You could, bearing in mind that Ghost 2003 is strictly a home product, for non-commercial use. As a home user, using it to format the odd disk this isn't doing any harm and isn't a problem. However, it's a really bad idea to do this in a business setting without properly licensing one of the business editions of Ghost - there have always been different editions with totally different pricing structures for commercial as opposed to home use.

Thank  you for spending so much time explaning everything in such detail.  Finally I got a definitive answer...after maybe 4-5 years.   :)

About the use of Ghost 2003.  I understand completely it is only for non-commercial use.   I only did a favor to many friends, who needs to upgrade their computer but are not knowledgable enough to set up the hardwares.  So, I help them choose what to buy, set up the hardwares, prepare the disk, and install their OS.  And they pay me for lunch.........or nothing.   [smiley=grin.gif]
Well, I don't mind. They're my friends anyway.

Serville,
I do love Nigel and his in-depth explanations, although sometimes while reading them I have to get up and stretch, then get another cup of coffee. ;) ;D ;D ;D
(Sorry Nigel) :-*

But, I've been doing PC's for 28 years and I DO remember the olden days of the MFM hard drive and low level formatting them with "Debug".
I guess I'm a dinosaur, a throwback and even a cave man, because when I get a new hard drive I first FDISK it to set up my partitions.  A small one for the OS and a large one for Storage, like Suites and Ghost Image Files.
I use my Windows ME, boot/utilities floppy disk, CD or Flash Drive.

After partitioning, I DO run the DOS format on both partitions, to set up the tracks and sectors and ferret out any bad sectors.
That also exercises the drive and certifies that it's running correctly and not a DOA.

At the end of that (yes, lengthy) process, I'm absolutely sure that the drive is OK to install my OS and programs.
I don't like surprises! :P

Then when XP Setup asks me if I want it to reformat the HD, I tell it "NO".
XP runs like a scalded dog on a FAT-32 partition. ;)
And that gives me DOS control over every file on my HD.
( I told you that I was a dinosaur! )

*  Only Vista refuses to install to a FAT-32 partition. (NTFS only)

Revisiting the past, I recently dug out my old "Universal IDE Low Level Format" program that's been in my archives for years, and I used it to recover several (so called) BAD hard drives.

One drive, a Seagate 160 gig IDE drive, had a bad sector zero, which Spinrite could not fix.  The low level formatter fixed it.
After the low level format, Spinrite then reported the drive as 100% OK.

The second drive was an old Quantum Fireball with a dozen or more bad sectors.  The low level formatter fixed that too.
Since then, I've low leveled several other drives showing surface errors and the little program fixed them.

It may fly in the face of reason and I can't even get into the technical explanations, , , all I know is that it works.
I never try to argue with success !!! ::)

Cheers Mates!
The Shadow  8-)

Shadow,

What is this Universal IDE LLF utility you're talking about ?
Is it freeware ?   Sounds interesting  :)

Gee!   I don't know if it's freeware or not.
Maxtor put it out for FREE years ago.
Many of the old websites are gone now.
But I did find it here:

http://www.softpedia.com/get/System/Hard-Disk-Utils/HDD-Low-Level-Format-Tool.shtml

What the heck..... give it a try.

This program must be run under Win-32.  Not my favorite.
The possibility of formatting your OS drive is just too great.
I'll keep looking.

Rule #1 for me is to have NO other hard drive connected to the system you're using to do the low level format.  I keep one old computer for stuff like that.  There is NOTHING on there that I'd not be willing to loose.

Nothing is worse than starting a low level format and realizing that you're formatting your main drive.....BY MISTAKE!!!!   Don't do it!!!

good luck,
Shadow  8-)

Ok.....that's not so good, running a low level format program from Windows. (Shuddering violently!)

So here's what I've done.
I added the Maxtor Low Level Format tool to my own Custom Windows ME Utilities disk (floppy disk), then I made a self extracting image of it, and put that image on an internet server where you can download it.

You can download the floppy disk builder program here:
http://www.box.net/shared/hd19xooxvh

Just run the program to build your own copy of my Utilities disk, including the Maxtor Low level format program.  It's item #7 on the DOS menu.
You don't have a floppy disk drive?  Well, it's about time you got one! ;)
Real computer enthusiasts wouldn't be caught dead without one. ::)

I hope someone will give me some feedback on just how helpful this was to you.

As I type this, the LLF program is busy formatting a WD 200 gig IDE drive that I had laying around my shop. ;)

Good Luck,
The Shadow  8-)

Thanks so much for your efforts.   Downloading now as I speak.
I don't need it right now, but I guess one more tool in my arsenal of defense is good.    ;D

Serville wrote on Jan 13^th, 2009 at 9:49pm:

I don't need it right now

You really don't ever need it. In actual fact, there is no low-level format command in the ATA specification. A "universal" low-level IDE format utility cannot possibly exist, and the interesting this is that this doesn't matter, because one simply isn't ever necessary, full stop.

What utilities that claim to be "low-level this or that" are actually doing is completely different, and much more straightforward; "low-level" is simply a nonsense word to impress people. It doesn't mean they do nothing, per se, but their actual effect is more a matter of psychology than anything else.

Manufacturers sometimes provided specific utilities that may or may not have used their internal diagnostic command sets, but actually nowadays the format utilities they provide tend to just write zeros all over the entire OS-visible drive, which prods the drive firmware's remapping ability to remap the bad sectors from its reserve capacity when they are written. Because that's something the drives do automatically, the only real purpose in using such a utility is to game those old classic "format" commands which nowadays (because there is no real physical format needed) only read the sectors to see that the disk surface is fine, because in times of yore they needed to.

If there is a damaged sector somewhere that has an uncorrectable error, the drive can't move it because the data isn't recoverable ; the sector will be replaced next time it is written. So, some low-level "check" utilities will see the disk as damaged permanently because they are only reading, not writing.

So, the end result is that these "low-level" format utilities really exist in practice only to scribble on the disk, just to fake out the "high-level" format utilities which check it, and don't do anything more useful than that. It's all a colossal waste of time and effort, because if you didn't read the sectors to check they were OK, you wouldn't need to write them then and there either.

The psychology of all this - the appearance, and the consequent sense of reassurance - that these utilities provide is sort-of the computer equivalent of the placebo effect, from which point of view it could be said that they do provide a useful service of some kind, even if it's not what you might think.