Welcome, Guest. Please Login
 
  HomeHelpSearchLogin FAQ Radified Ghost.Classic Ghost.New Bootable CD Blog  
 
Page Index Toggle Pages: 1
Send Topic Print
"Hot Imaging":  Managing Disk Activity (Read 28767 times)
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
"Hot Imaging":  Managing Disk Activity
Mar 12th, 2006 at 4:46pm
 
Creating a recovery point of a system volume while the operating system is active is known as "hot imaging."  But, what is the 'temperature' of a "hot image" – i.e., to what extent are disk write operations occurring during the imaging process?

To answer this question, I ran and logged the output from Filemon for the system partition ('C:') for one hour under conditions of quiescence (i.e., no keyboard or mouse activity, launching no new programs, etc.) with Norton Internet Security 2006 active.  An examination of all disk writes revealed the following key observations.
  • All disk writes that occurred (N=116) were generated either by Symantec applications (N=52) or by Windows (N=64).
  • All disk writes by Symantec were limited to only N=5 files (SETTINGS.DAT, SNDCON.LOG, SNDFW.LOG, USAGE.DAT, SYMLCRST.DLL).
  • All disk writes by Windows were limited to only N=7 files (SOFTWARE.LOG, INDEX.BTR, INDEX.MAP, MAPPING.VER, MAPPING1.MAP, OBJECTS.DATA, OBJECTS.MAP).
Therefore, when an application such as Ghost 10 is creating a "hot image," the extent to which it has to manage disk write operations in order to ensure that the recovery point is consistent is indeed quite minimal.  A total of only N=12 files had disk write activity during the hour, and N=5 of those were files under the management of Symantec applications.  If the duration of the backup job is less than an hour, then these numbers would be correspondingly reduced.  If the user is actively running applications during the backup job (e.g., creating or editing documents, spreadsheets, etc.), then the numbers would increase.

Nonetheless, if a Ghost 10 backup job is started and if the user simply allows the job to run without intervention, then there appears to be little "hot activity" that needs to be monitored and managed by Ghost 10 in order to ensure the consistency of the recovery point.  This does not logically prove, of course, that the "hot imaging" process is as inherently reliable as "cold imaging" (e.g., using Ghost 8.2 or Ghost 2003), but it does suggest that the extent of the "hot imaging" problem may be considerably less than had otherwise been expected.

I encourage others to repeat the experiment, so that these results can be verified and augmented.
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 

John.
Übermensch
*****
Offline



Posts: 2072


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #1 - Mar 12th, 2006 at 6:19pm
 
I know that hot-imaging has been discussed quite a bit before. Two areas to consider with hot-imaging:

1. Physical i/o or changes that occur during the backup process.  You've covered that quite well, and as I understand it, Ghost 10 monitors that case and takes care of it.

2. Memory or buffer changes that may have not yet been written to disk.  For example, the messages when shutting down XP, "logging off", "saving changes".  Are those changes which presumably are in the memory or the registry on the backup or not?

I believe that within limits hot-imaging and Ghost 10  works fine.  I'm satisfied with it if used carefully.

 

Ghost4me  Ghost 9, 10, 12, 14, 15.  Windows XP, Vista, Windows 7
 
IP Logged
 
El_Pescador
Übermensch
*****
Offline


Thumbs Up!

Posts: 1605
Bayou Country, USA


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #2 - Mar 12th, 2006 at 6:52pm
 
Pleonasm wrote on Mar 12th, 2006 at 4:46pm:
"... I encourage others to repeat the experiment, so that these results can be verified and augmented..."

Long before he even became a
Forum member
(which required considerable cajoling by others, I might add), I repeatedly promoted
Moderator
status for the author of this thread.  I based my recommendation on his consistently constructive contributions to these boards over time.  I am of the opinion that the initial post above certainly vindicates my longstanding advocacy.

EP
Cry
 

...
WWW  
IP Logged
 
Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #3 - Mar 12th, 2006 at 7:57pm
 
downloading filemon now. i like utilities like that which let me see what the system is doing.
 
WWW  
IP Logged
 
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #4 - Mar 13th, 2006 at 12:47pm
 
Ghost4me, concerning changes to the registry, Sysinternals has a tool called Regmon that monitors real-time registry changes.  I did not use Regmon in my ‘experiment,’ only because the creation of a snapshot of the registry seems to be a straightforward process.  For example, the VBS command ‘createrestorepoint("Restore Point", 0, 100)’ does the job.  The mechanism that Ghost 10 uses to intercept disk writes and suspend their execution until after the recovery point is created would seem to apply equally well to registry changes, since the registry is a disk file, too.

I don’t know how Ghost 10 handles buffered disk writes, but as long as the buffer is not actually flushed to physical disk, then one would anticipate that it would have no impact on the generation of a consistent recovery point.  If the buffer is flushed to disk, then it is the same case as if the application did a disk write.

One could envision circumstances in which the creation of a “hot image” by Ghost 10 becomes considerably more challenging.  In theory, while running a Ghost 10 backup job, a user could launch a disk defragmentation, uninstall one application and install another, run Windows Update, and download files from the Internet.  One would expect that at some point (such as in this example) Ghost 10 would become overwhelmed with disk changes and exit with an error condition.  This, however, is probably an extreme and unusual case.  I suspect that most users launch the Ghost 10 backup job, and walk away from the PC until it is complete – or, schedule the job to run during the evening hours when no one is using the PC anyway.  As my experiment illustrated, under these more typical conditions, the creation of a "hot image" doesn’t seem to be considerably more complicated (and therefore more risky) than the case of a “cold image” in which no files on the volume are open.

Again, I encourage others to repeat the experiment I described in the initial post on this thread, in order to either confirm or correct the reported findings.

* * * * * * * * * *

El_Pescador, thank you for the kind comments!  I am honored to have received the designation of moderator.
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #5 - Jul 28th, 2006 at 1:41pm
 
The following description of how Acronis True Image creates a ‘hot image’ may be of interest.

Quote:
Typically, Windows applications cannot copy the system partition because they are unable to copy open files. In cases where the user's hard disk is partitioned only as one drive, that meant the user could not image their system at all from within Windows. If the disk is large enough, it could mean that the user is out of commission for hours at a time while the program chugged away at creating an image.

Leading-Edge Technology
Acronis overcomes this issue of copying open Windows files by effectively freezing a moment of time. A special filter driver layers between file system drivers and volume drivers, allowing the software to create and backup consistent views of all files, including open files. The driver is installed above the volume drivers so it can see all the read and write requests passing to a volume — or partition.

When the IT manager or user initiates imaging of the hard disk drive containing the system volume, the filter driver flushes the file system mounted to that volume; then all the operations on the system volume are temporarily frozen. Immediately thereafter, the filter driver creates a point-in-time view of the system volume and a bitmap describing the used sectors on this volume. Once the bitmap is created, the filter driver unfreezes the I/O operations on the system volume. It generally takes just several seconds to create a point-in-time view of the volume. After that, the operating system continues working as usual and the software can start imaging of the system volume.

Acronis True Image reads the sectors on the system volume according to the created bitmap. Once a sector is read, the appropriate bit in the bitmap is reset. In its turn, the filter driver continues working to hold the point-in-time view of the system volume. Whenever the filter driver sees a write operation directed at the system volume, it checks whether these sectors are not backed-up yet, if they are not, the filter driver reads a copy of the sectors that will be overwritten into a special buffer created by the software, then it allows the sectors to be overwritten. Acronis True Image backups the sectors from the special buffer so that all the sectors of the point-in-time view of the system volume will be backed-up intact. Meanwhile, the operating system continues working and the user will not notice anything unusual in their operating system functionality.

Source:  http://www.acronis.com/enterprise/company/inpress/2003/01-ctr-disk-imaging.html
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 

NightOwl
Radministrator
*****
Offline


"I tought I saw a puddy
tat..."

Posts: 5826
Olympia, WA--Puget Sound--USA


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #6 - Jul 29th, 2006 at 12:29pm
 
Pleonasm

I think you *hit the nail on the head* with this statement:

Quote:
One could envision circumstances in which the creation of a “hot image” by Ghost 10 becomes considerably more challenging.  In theory, while running a Ghost 10 backup job, a user could launch a disk defragmentation, uninstall one application and install another, run Windows Update, and download files from the Internet.  One would expect that at some point (such as in this example) Ghost 10 would become overwhelmed with disk changes and exit with an error condition.

I think anyone who has used prior versions of Windows, i.e. Win9x or ME, or even Win3.1, has had the experience where *multitasking* causes the system to either become unstable or freeze--I suspect this is where a lot of the concern/negative feelings about the concept of *hot imaging* comes from.

I have to say the WinXP has been much more stable in this regard than prior versions of Windows!

I think it should almost be a recommendation that one not attempt multitasking when doing a complex procedure as *hot imaging* and tempting the *fate gods* to cause your system to *crash and burn*.

As you clearly show--the complexity isn't *overwhelming* when the system is merely *idling*--but, I'm sure it gets increasingly complex, especially if a program has HDD intensive activity.
 

____________________________________________________________________________________________

No question is stupid ... but, possibly the answers are Wink !
 
IP Logged
 
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #7 - Jul 29th, 2006 at 12:49pm
 
NightOwl, both as a practical matter and simply to be cautious, I agree that doing "heavy multitasking" when creating a Ghost 10 image may be "tempting the Gods of Fate."  I can't speak for other users of Ghost 10, but I suspect that most would either schedule a backup job when the PC is idle (e.g., in the evening hours) or simply walk away after starting a backup job and then return when the task is complete.

We do not know that Ghost 10 uses the same approach as that described by Acronis, but it is probably quite similar.  I found the 'hot imaging' algorithm to be intellectually elegant – simple, straightforward without convolutions, and apparently bullet-proof.  Provided that all I/O by applications on the PC are using Windows APIs (rather than direct-to-disk writes), I can't see how the algorithm would fail to produce a consistent and complete image of a partition at a point in time.
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #8 - Jul 30th, 2006 at 10:31am
 
The question, "How do you know that a 'hot image' successfully captured the state of the partition at a single and consistent point-in-time?" is an important one, and it speaks to core issue of whether 'hot imaging' is reliable.  I have no reason to believe that Ghost 10 is performing the following process, but here is one solution that – if implemented – would answer the question:
  • When the partition is in a locked state before the imaging is about to begin, compute and retain a cryptographic hash of the partition (e.g., MD5/SHA-1).
  • When the image is complete, compute its cryptographic hash and compare to the expected value.  If they match, then the image must be a complete and identical reflection of the partition.
To the best of my recollection, there has not been a single instance of anyone of this forum reporting a problem with a Ghost 9 or Ghost 10 image failing to accurately and completely represent the state of the partition at the time the image was created, which is strong support in favor of the reliability of the approach.
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
Re: "Hot Imaging":  Managing Disk Activi
Reply #9 - Jan 30th, 2007 at 7:16am
 
Readers of this thread might find the following discussion to be of interest.

Quote:
Maybe it would be useful to the forum to describe exactly what "snapshot technology" is. The goal of snapshot technology is to instantaneously capture all of the data on a volume at a given point in time without interrupting I/O to that volume (without locking anything, stopping applications, etc). This is invariably implemented using an "upper volume class filter device driver" which intercepts I/O underneath the file system, but above the volume devices (see the storage stack diagram below). When the snapshot device driver is instructed by the backup application to establish a snapshot of a given volume, it immediately starts tracking changes that occur to that volume. Each time a write occurs on the volume the snapshot driver first copies off the old data that was already at the location into some scratch pad area and then allows the new write operation to continue on down to the device. Thanks to this copy-on-write mechanism, the snapshot driver is able to preserve all of the information necessary to expose to a backup application the data that was on a volume at a given point in time. Snapshot drivers usually expose the snapshot point-in-time by creating a virtual device similar to a volume device. The backup application will read the sectors on this virtual device in the same way that it would have read them from the real volume. The backup application *may* lock access to this virtual snapshot device, but keep in mind that this virtual device is generally only used by the backup application (it's not visible to normal applications). Locking the virtual snapshot device has no effect on the actual volume device that is being backed up - the real device is never locked and remains accessible to applications. Because snapshot device drivers sit within the storage stack itself, they have absolute control of the flow of I/O to the disk. When a snapshot driver is instructed to establish a snapshot, it takes only a few microseconds, if not less time, for it to halt I/O to the device and establish the necessary in-memory structures necessary to maintain mappings of which sectors have been changed (causing copy-on-write operations) since the establishment of the snapshot, and then to allow I/O to recommence. It's important to understand that there is a difference between the act of "taking a snapshot" and the act of "imaging the data on a snapshot". Taking a snapshot requires only a few microseconds, if not less time. Imaging the snapshot is the process of backing up all of the data that are exposed by the snapshot device driver for a particular point-in-time (usually exposed as I mentioned by a virtual volume device), which is all of the data on the drive at a given point in time, and this can be a lengthy process. When a plain-vanilla snapshot device driver creates a snapshot of a volume, it is instantaneously capturing the volume's state at a given time, and the state of the volume's data is very similar to the state of its data if you kill the power to the computer. This is called a "crash consistent" state. The reason for this is that, at any given time, many files are open and in use and also the file system itself can have structures which are write-cached and have not yet been flushed. Creating a plain-vanilla snapshot captures the data in this in-use state, so it's not a very clean snapshot (it's "crash consistent"). VSS was introduced with Windows XP, and one of its primary purposes is to facilitate the establishment of snapshots when the data is in a clean state. To do this, backup applications and snapshot drivers must be written to interact with VSS. VSS controls the snapshotting process. The backup application will ask VSS to take a snapshot, using a specified snapshot "provider" (snapshot device driver). VSS will then tell all VSS-aware "writers" (applications which generate data such as Exchange, Oracle, SQL, IIS, and many system services such as the registry, etc) to quiescence (meaning that these applications flush their files to disk in a state that is clean and then pause for a small moment until they are told to resume activity) and then VSS will send a special flush-and-hold message (IOCTL) to the file system on the volume on which the snapshot is being established and when the file system receives this flush-and-hold message it will flush all of its metadata to disk and it will then send this flush-and-hold message further down the stack and the file system will not send any more I/O down the stack until the flush-and-hold IOCTL is completed by the snapshot driver which resides below it (which establishes an I/O barrier at the file system level). When the snapshot driver receives the flush-and-hold IOCTL, it knows that all VSS-aware applications as well as the file system itself have flushed their data in a clean state, so it establishes the snapshot and then completes the flush-and-hold IOCTL, which completion event is then received by the file system driver above it at which point the file system driver releases I/O and passes the completed flush-and-hold IOCTL back to the VSS service, which then releases all of the VSS-aware applications. Through this mechanism, VSS-compliant backup software, which use VSS-compliant providers, can create backups of extremely high-load Exchange, Oracle, IIS, SQL, etc. servers on a regular basis without the need to stop any services or applications, without shutting down the machine, while also ensuring that the backups contain good clean data (databases will not need to be fixed/repaired if they are restored, because VSS ensures that they are captured in a good state).

On platforms older than XP, where VSS is not present, other techniques are used to obtain images which are better than crash consistent state images. For instance, some snapshot drivers implements their own I/O barrier at the file system level (using a file system filter driver), if necessary, and on platforms that lack VSS (Windows NT and 2000) the driver is capable of performing the file system flush-and-hold operation which is natively supported on XP+. This enables it to perform snapshots at a time when the file system's metadata has been flushed to disk. Coupled with this is a more manual quiescence process which is facilitated by the backup application itself, which enables users to specify scripts which they wish to be called before and after the establishment of the snapshot, which scripts can be used to stop/start services, etc (or in other words, the scripts are a way of quiescing important applications). While this is more tedious, it gives users of older platforms, that lack VSS, some of the advantages of VSS.

Windows Storage Stack

I/O within the Windows kernel flows from one driver to the next until it is finally passed directly to the hardware device. Each type of device typically has several layers of drivers that manage I/O as it flows to the device. For storage, I/O generally is initiated in user mode (from applications) and is passed to the kernel's I/O manager where it then forwarded to the appropriate file system driver, then to the volume class driver, then to the disk driver, and finally to the storage controller port driver. Filters can be added at any level in this stack.
    ----------------------
    |Win32 Application(s)|
    ----------------------
    |
    -----------------
    |Win32 Subsystem|
    -----------------
    |
    ---------------
    |Native NT API|
    ---------------
    |
    -------------
    |I/O Manager|
    -------------
    |
    --------------------
    |File System Filter|
    |Driver(s) such as |
    |AntiVirus Drivers |
    --------------------
    |
    --------------------
    |File System Driver|
    |such as NTFS.SYS |
    --------------------
    |
    --------------------
    |Volume Filter |
    |Driver(s) such as |
    |Snapshot Driver |
    --------------------
    |
    -----------------
    | Volume Driver |
    -----------------
    |
    -------------------
    | Filter Driver(s)|
    -------------------
    |
    ---------------
    | Disk Driver |
    ---------------
    |
    -------------------
    | Filter Driver(s)|
    -------------------
    |
    --------------------
    | Port and Miniport|
    --------------------
    |
    -------------------
    | Actual hardware |
    | device |
    -------------------
Source:  http://www.wilderssecurity.com/showthread.php?t=139716&page=12

The author of the above comments is self-described as a “senior engineer focused on enterprise solutions” at StorageCraft.

The same author also comments:  “StorageCraft's snapshot technology is used by many companies, and in several backup products. It's been around for several years, is deployed on tens of millions of computers, and has matured to {a} very robust state.”

I also learned that the driver used by Ghost 10 to perform its “hot imaging” was developed by StorageCraft and is licensed to Symantec.  You can verify this fact by examining the properties of C:\WINDOWS\system32\drivers\SymSnap.sys (note the copyright designation).
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 
Pleonasm
Übermensch
*****
Offline



Posts: 1619


Back to top
Re: "Hot Imaging":  Managing Disk Activity
Reply #10 - Apr 7th, 2007 at 1:54pm
 
Readers of this thread who may still have some reluctance to employ a "hot imaging" solution should take comfort in the comments of Nbree, a Ghost developer and Principal Software Engineer at Symantec.

Quote:
Question:  …with respect to reliability, is there any reason to believe that one approach ["cold imaging"] is any more or less reliable than the other ["hot imaging"]?

Answer:  In the abstract, no I don't think so, although it takes substantial engineering effort to make it so - especially, the cooperation of the filesystem to take on more transactional semantics - compared to offline work.  The database world proved all this stuff works three decades ago.

Hopefully, the opinion of a Ghost (2003/8.2/Ghost Solution Suite) expert will serve to ameliorate any residual concerns about the use of "hot imaging".
 

ple • o • nasm n. “The use of more words than are required to express an idea”
 
IP Logged
 
Page Index Toggle Pages: 1
Send Topic Print