Accessing File Systems on Disk Block Image Files

You probably know how to mount CD or DVD iso images on Linux using the loopback block device. The loopback device can be used for a whole lot more. As a programmer I find virtual machines very useful. I use them so much now that I often need to access files in them…even when they are not running. I also have huge files containing block level archives of retired physical disks. I sometimes need to recover files from the file systems on those too. Linux has many cool features and some of them can help you access files on the file systems in a file that contains the block by block contents of a disk.

It has to be said that the method I’m about to describe only works for disk images that are flat files representing the contents of the disk in logical block order (raw disk images in common VM terminology). So, you cannot immediately use this method with the special disk formats supported by some VM platforms, e.g. vmdk, qcow2 etc. Converters exist that will create raw images from other formats (see below).

It’s very convenient that the contents of block devices are exposed as files in Linux. It makes it easy to duplicate disks, create backup images of disks, etc without the overhead of having to re-create the entire directory structure of the source. Many of the command line tools for disks only expect a filename as the location of the disk to work with and will actually work with any file.

The normal way to access files on a disk is to mount the file system(s) on the disk. In the case of the mount program, the filename used must be that of a block device file. Regular files are not block devices but you can get the mount program to use them if you make them available via a loopback block device.

The loopback block driver allows you to access a regular file as if it was a block device. Now that’s a Cool Solution, it lets you do this:

# losetup -v -f /path/to/disk.img
Loop device is /dev/loop0
# mount -t ext3 /dev/loop0 /mount/point

In the form above or the shortened form shown below it’s a common way to mount CD/DVD iso images without having to burn them onto physical disks. The first command finds an unused loop device, attaches the /path/todisk.img file to it and tells you which loop device was selected in the /dev tree. The second one mounts an ext3 file system (if present) from that block device.

You can also specify which loopback device should be used:

# losetup -v /dev/loop5 /path/to/disk.img
Loop device is /dev/loop5

I’ve used the -v option to losetup to get verbose information on the action. In the case of losetup it only causes the selected loop device name to be announced. I tend to use verbose options when available and I’m not using the command in a script.

The mount program can do the losetup for you:

# mount -v -t ext3 /path/to/disk.img /mount/point -o loop
mount: going to use the loop device /dev/loop0
/dev/loop0 on /mount/point type ext3 (rw)

Once you are finished, you can unmount the file system in the usual way then detach the file from the loopback device:

# umount /mount/point
#losetup -d /dev/loop0

Or do it all in the umount command:

umount -d /mount/point

That works really well for CD/DVD iso files but you’ll run into problems with disk images. By default, the losetup command (and mount with -o loop) makes a loopback block device that maps to the entire source file, i.e. the whole disk image. The mount command looks for the file system to begin at block zero on the specified block device file. Most of the time, hard disks are partitioned and a file system does not begin at the start of the disk. There may be more than one file system on the disk anyway and they can’t both begin at block zero on the disk. That’s why partition block devices exist, e.g. sda has associated sda1, sda2, etc devices. Each of these maps to just the contents of the relevant partition within the whole disk and, on those, the file systems begin at block zero so mount can use them. You need to do the same for the file systems on partitions in a disk image.

Accessing partitions in a disk image

From experience with physical disks you might assume a partition device exists after using losetup and you can just add 1, 2, 3, etc to the loop device name:

# losetup -v -f /path/to/disk.img
Loop device is /dev/loop0
# mount -t ext3 /dev/loop01 /mount/point
mount: special device /dev/loop01 does not exist

losetup attached the image file to /dev/loop0 and I assumed there was a /dev/loop01 first partition device that could be used by mount. Apparently not. Instinct might lead you to suspect it just has another name. You might remember you can use fdisk -l to see the device names for partitions:

# fdisk -l /dev/loop0
Disk /dev/loop0: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xbf9bbf9b

      Device Boot      Start         End      Blocks   Id  System
/dev/loop0p1               1         262     2104483+  82  Linux swap / Solaris
/dev/loop0p2   *         263       49328   394122645   83  Linux
/dev/loop0p3           49329       60801    92156872+  83  Linux

In that case, with the losetup is already done above:

# mount -t ext3 /dev/loop0p1 /mount/point
mount: special device /dev/loop0p1 does not exist

Well, there are probably other things we could do but the losetup program has a feature that can help here. If you knew where a partition was located on a disk then you could use the offset option to change the mapping between locations in the loop device and locations in the image file.

You can dump the partition table as-is from the disk image, as in the example above by attaching the whole image file to a loop device and using fdisk -u -l /dev/loop0 (etc). Accessing the image via a loop device just to get the partition table so that you can set it up on another loop device is not necessary. fdisk does not require the target object to be a block device so you can use fdisk -u -l /path/to/disk.img fdisk will complain that you must tell it the number of cylinders on the device but it will dump the partition table anyway:

# fdisk -l /path/to/disk.img
You must set cylinders.
You can do this from the extra functions menu.

Disk /path/to/disk.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x64b99de9

           Device Boot      Start         End      Blocks   Id  System
/path/to/disk.img1             63     4209029     4208966+  82  Linux
/path/to/disk.img2        4209030   792454319   788245289+  83  Linux
/path/to/disk.img3      792454320   976768064   184313744+  83  Linux

The useful part is in the last Start and Blocks columns. The 63 in the Start column for the first partition is the logical block address of the first sector of the partition and the Blocks column is the number of logical blocks in the partition. Logical blocks are usually sectors and sectors are normally 512 bytes. You should confirm that for any disk image you work with, see below for an example where the sector size was not 512 and how to deal with it.

The first partition is for swap (type 82) so I’ll work with the second partition to demonstrate accessing a file system. It begins at 4209030 sectors times 512 bytes into the image file, that’s 2155023360 bytes. The partition is 788245289 sectors times 512 bytes long, that’s 403581587968 bytes. You can use losetup to create a block device that maps only to those locations in the disk image then mount the file system on that loop device:

# losetup -v -f -o 2155023360 –sizelimit 403581587968 /path/to/disk.img
Loop device is /dev/loop0
# mount -v -t ext3 /dev/loop0 /mount/point
/dev/loop0 on /mount/point type ext3 (rw)

You can use the offset and sizelimit options directly in mount and combine those into a single command line:

# mount -v -t ext3 /path/to/disk.img /mount/point -o loop,offset=2155023360,size-limit=403581587968
mount: going to use the loop device /dev/loop0
/dev/loop0 on /mount/point type ext3 (rw)

So, now you can just copy the file you need right out of the disk image:

# cp /mount/point/home/me/documents/letter /home/me/documents/

As shown, the mounted file system is read/write unless the image file is read-only. So, you can clean up that virus on a typical Windows C: disk without having to boot the OS on it:

# mount -v -t ntfs /path/to/disk.img /mount/point -o loop,offset=32256,size-limit=8587160064
mount: going to use the loop device /dev/loop0
/dev/loop0 on /mount/point type ntfs (rw)
# rm /mount/point/Windows/System32/bad.dll

When you umount the file system and re-use it on the system it came from then the bad.dll file will no longer exist. Regardless of the existence of the loop device, this ability to mount many different native and non-native file system on Linux from physical disks as well as disk image files is a really good reason to have a USB housing you can plug physical disks into, e.g. a USB to ATA or a USB to SATA adapter. Or, to carry a bootable Linux rescue system on a CD or even a USB thumb drive.

If all I want to do is recover one or more files from the disk image then I protect myself by mounting the file system read only using the -r mount option:

# mount -v -r -t ntfs /path/to/disk.img /mount/point -o offset=32256,size-limit=8587160064
mount: going to use the loop device /dev/loop0
/dev/loop0 on /mount/point type ext3 (ro)
# cp /mount/point/Documents\ and\ Settings/joe/My\ Documents/passwords.doc ./

That prevents me from doing anything destructive to the source. If you are using losetup directly then the -r option to losetup causes the loop block device to be read only.

Be careful to check that the loop device is deleted if you umount the file system and don’t need it any longer:

# losetup -a
/dev/loop0: [0802]:12296228 (/path/to/disk.img)
#losetup -d /dev/loop0

I said above that the umount command can, rather conveniently, remove the loop device for you:

# umount -d /mount/point

It’s particularly cool that you can use -d arbitrarily and, if the device is a loop device it will be deleted otherwise the -d will be ignored.

When sectors are not 512 bytes long

I said earlier that you should be careful not to assume sectors are always 512 bytes long. A colleague asked me for some help mounting a file system on a dd image of the disk from his smartphone. It’s easy to assume there will be a file system starting 32256 bytes into the image when you are so familiar with 512 byte sectors and PC partition table restrictions. When we tried to mount the file system, the mount command reported that the file system’s superblock was bad or in an unrecognized format. Of course, our first reaction was that retrying the same command it would work the second…third…fourth…etc…time. Finally I suggested we check the partition table carefully. Using the example from my hexdump article:

hexdump -s 446 -n 64 -v -e '1/1 "Partition:| %02x" 3/1 " | %3u" 1/1 " | %02x" 3/1 " | %3u" 2/4 " | %9u" "\n"' /path/to/disk.img
Partition:| 00 |   1 |   1 |   0 | 83 | 254 | 255 | 255 |        63 |   2096474
Partition:| 00 |   0 |   0 |   0 | 00 |   0 |   0 |   0 |         0 |         0
Partition:| 00 |   0 |   0 |   0 | 00 |   0 |   0 |   0 |         0 |         0
Partition:| 00 |   0 |   0 |   0 | 00 |   0 |   0 |   0 |         0 |         0

It’s an 8GB disk. The partition begins where I expected, on the 63rd sector but it’s only 1GB long…if I assume 512 byte sectors. I know from the phone’s own tools that there is 8GB of space on the disk. So there must be an 8:1 gearing between my 512 byte sector assumption and the sector information in the partition table. That would make the sectors 4096 bytes and, indeed:

# mount -v -r -t vfat /path/to/disk.img /mount/point -o offset=258048,size-limit=857747456
mount: going to use the loop device /dev/loop0
/dev/loop0 on /mount/point type vfat (ro)

This works and we were able to access the files in the file system on the disk image. So, if you come across a disk image with partitions that should mount but won’t then look at the partition table. Divide the expected disk/partition size by the partition length in sectors (end logical block address minus the start logical block address). If the value is different from 512, change the offset and size-limit mount options to the relevant sector number/count multiplied by the calculated sector size (rather than 512). Sector lengths are unlikely to be anything other than a power of two so, if there is some fractional part (e.g. 4097.23412 bytes) then it is safe to assume the nearest power of two is the likely size (e.g. 4096). If you are careful to always use the -r option then you are unlikely to do any damage to the disk image by experimenting.

You’ll note that I didn’t have to tell either losetup or mount that the image was of a disk with 4096 byte sectors. Neither of those drivers care. That’s because the file system driver (selected by mount) sees a disk as a linear array of single bytes, each with its own address. The file system driver uses its own block size and alignment for its own convenience but it doesn’t care about the underlying storage device’s concept of blocks or alignment. The storage driver stack (including the loop device) permits access to any individual byte on any device. To prove it, read one byte from some randomly large, odd numbered location on a physical disk (i.e. not a file on a file system and not on a sector boundary):

# dd if=/dev/sda of=/tmp/somebyte bs=1 count=1 skip=213985

That will copy the byte located 481 bytes into the 418th sector on a typical disk with 512 byte sectors. Use ls -la and hexdump to confirm the result is one byte long. The input file was a disk block device file so we can assume that the block drivers allow us to access any byte location on the disk which means the file systems don’t need to know about sectors (though they can) which means none of our loop examples need to care about the original sector size and alignment of the source disk. And that means we can use unusual examples like the 4096 byte sector smartphone disk image.

Converting VM platform native disk images to raw disk images

If you have qemu installed on a Linux host you can use one of its components, qemu-img, to create a raw disk from the native disk format of some other VM platform, e.g. Vmware. If you have any recent version of OpenSuSE Linux then you will have qemu installed by default. The SuSE Linux Enterprise versions don’t include it by default. I believe it is part of the the Xen virtualization environment but it probably isn’t wise to install that for the sole purpose of getting qemu-img, so you might want to build qemu or get a pre-built copy from somewhere.

Using qemu-img you can convert from qemu’s native qcow2 disk formats to a raw disk image using:

# qemu-img convert -f qcow2 original.img -O raw newraw.img

If you had a Vmware virtual disk:

# qemu-img convert -f vmdk original.vmdk -O raw newraw.img

An interesting case is where you have a vmdk in the “split into 2GB files” format. You can actually convert each vmdk file individually then cat the raw disks together into a final raw disk:

# qemu-img convert -f qcow2 original-1.vmdk -O raw newraw-1.img
# qemu-img convert -f qcow2 original-2.vmdk -O raw newraw-2.img
# qemu-img convert -f qcow2 original-3.vmdk -O raw newraw-3.img
# qemu-img convert -f qcow2 original-4.vmdk -O raw newraw-4.img
# cat newraw-1.img newraw-2.img  newraw-3.img newraw-4.img > finalraw.img

Crazy example – would you like a bunch of 1TB disks at no cost!

Recently, I was discussing solid state disks (SSD) with a colleague. He’s a file system developer and often receives meta-data images of a damaged file system. That’s a copy of just the file system structure data, it doesn’t include the data in files or the data in unallocated blocks. So, a meta-data image of a 1TB volume might be only 2GB in size. A meta-data image like that can be restored in 30 seconds to a SSD but takes 15 minutes on mechanical disks. You can imagine my colleague would prefer to restore to the SSD. Unfortunately, his SSD is only 100GB and he regularly receives meta-data images for file systems 1TB and larger. We discussed a few ideas involving special block device drivers until I realized the loop device could handle it if:

• The loop device can use a sparse file
• A sparse file can have a length larger than the file system it is stored on
• The meta-data for the original file system is smaller than the free space on the SSD

The loop device can use sparse files. A sparse file means a file where only the blocks that have actually been written to are committed storage on the disk. For example, you can use dd to create a 1TB sparse file that occupies only a few kilobytes of disk:

# dd if=/dev/zero of=/tmp/somefile bs=1024 count=1 seek=1073741824

That creates a file called /tmp/somefile by writing a single 1024 bytes block at the location 1073741824 times 1024 bytes into the file. Basically that writes the last 1kB of a 1TB file. The preceding blocks are not written at all and no disk space is committed for them. They can be read but the contents of them will be undefined, probably all zero bytes:

# l -h /tmp/somefile
-rw-r--r-- 1 root root 1.1T 2009-01-30 14:56 /tmp/somefile
# du -h /tmp/somefile
16K     /tmp/somefile

So, that’s a 1TBfile that occupies 16kB of disk, it’s a sparse file. It’s larger than the 1kB we actually wrote because of some unknown overheads in the driver for the file system the sparse file is hosted on.

We can now make a loop device from that sparse file:

# losetup -v -f /path/to/disk.img
Loop device is /dev/loop0

Check if the size of the sparse file has changed yet:

# du -h /tmp/somefile
16K     /tmp/somefile

Not so far. We can partition it and put a file system on one of the partitions. Partitioning will write some data to offset zero of the loop device and that will map to offset zero in the underlying file. That block doesn’t yet exist in the sparse file. But, it’s up to the driver that underlying file is hosted on to allocate a block. In my test here on an ext3 host file system, a 1.5TB sparse file actually occupied 16kB when created with a 4kB block at the end. After attaching it to a loop device and partitioning it, the file grew by another 4kB – to 20kB. That left me with a 1.5TB “disk” to play with. My file system developer colleague could use that to work with a 1.5TB file system on a host with only a few gigabytes of free disk space. After restoring the 2GB of file system structure and accounting for overhead in the host file system driver for the file backing the loop device he would be using around 3GB of host disk space. It easily fits on his SSD. He can examine the faulty file system with various editors because reading the loop device won’t change the size of the underlying sparse file. He can even run automated recovery tools to rebuild the file system and the sparse file should grow by less than the 2GB of originally restored structure data. When he is done, he can delete the sparse file and move on to the next task and create a new giant “disk” whenever he needs it.

How cool is that!