TotalView  
Since 1994: The Original Monthly Magazine of the Linux Community    
Home

How a Corrupted USB Drive Was Saved by GNU/Linux

By Collin Park on Tue, 2005-06-14 01:00. HOWTOs
Would SUSE and fsck be able to recover the data in a usable way?

My friend's brother had a 512MB Lexar Media Jumpdrive Pro USB drive that became corrupted after using it with Windows 2000. His IT department was able to get back some but not all of the file contents, but without any file names. On his own, he tried some recovery utilities, but all failed. Using a typical Linux distro--in this case SuSE 8.0--however, it wasn't hard to recover almost all of the data from the drive along with the filenames and to burn a CD-ROM of the contents.

USB Drive Ruined by Windows

Here's what I heard about the data loss:


    Date: Sun,  1 Aug 2004 17:06:03 -0700
    Subject: USB
                                   ...  My USB drive is a
    Lexar Media USB Jumpdrive Pro 2.0 (512 MB).  I was working
    on it in a computer with Windows 2000 and logged off before
    ejecting the drive.  Next time when I tried to use it,
    it showed up as a Removable drive rather than the usual
    Lexar Media drive and when I tried to open it, it said the
    drive was not formatted; and under Properties, 0 bytes free
    and used space and file system "RAW"
    According to Lexar tech support, there is a bug with
    Windows 2000 (that MS never bothered to fix) and can corrupt
    the drive when it is removed without proper eject. They
    recommend EasyRecovery Pro for data recovery which did
    allow me to recover some files (> 500) using their RAW data
    recovery program (all other tool failed because usually
    said "no recognizable file on disc").  Unfortunately,
    all the file names are lost and some files are gone.

The big questions was "can Linux read the drive?" A Web search of "linux usb jumpdrive pro" gave me hope that my kernel, 2.4.18 on SuSE 8.0, would recognize the drive in question. So, as root, I typed:


    # tail -f /var/log/messages

and plugged the drive into a USB socket. Here's what appeared; I removed "Aug 5 01:32:15 linux kernel:" from each line below):


    usb.c: registered new driver usb-storage
    scsi0 : SCSI emulation for USB Mass Storage devices
    usb-uhci.c: interrupt, status 3, frame# 1313
      Vendor: LEXAR     Model: JUMPDRIVE PRO     Rev: 0   
      Type:   Direct-Access                      ANSI SCSI revision: 02
    Attached scsi removable disk sda at scsi0, channel 0, id 0, lun 0
    SCSI device sda: 1001952 512-byte hdwr sectors (513 MB)
    sda: Write Protect is off
     sda: sda1
    WARNING: USB Mass Storage data integrity not assured
    USB Mass Storage device found at 4
    USB Mass Storage support registered.

Encouraged by that report, I tried this:


    # dd if=/dev/sda of=/tmp/r1 bs=512

which reported that 1,001,952 blocks had been transferred. I then unplugged the drive and did the rest of my work using the image stored in /dev/sda.

Condition of the Boot Sector

The master boot record, which is the boot sector for the entire drive and its first sector, has a partition table, as well as other interesting things:


    # od -Ax -tx1 /tmp/r1 | less
    ...
    *
    0001b0 00 00 00 00 00 00 00 00 48 04 07 c9 00 00 80 01
    0001c0 01 00 06 0f ff e0 3f 00 00 00 b1 45 0f 00 00 00
    0001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    *
    0001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa

The boot sector has a reasonable-looking partition table with one entry. It began at offset 0x1be, the two bytes 80 01. Your favorite search engine can give you other information about the partition table, but I note two things here. First, the entry has an LBA32 format--starting logical sector 0x3f, length 0xf45b1. Now, 0xf45b1 is 1000881 decimal. That plus 63 (0x3f) is 1000944. The difference between the 1001952 and this 1000944 is 1008, that is, 63*16. I guess this has something to do with cylinder boundaries. The second thing of note is the byte at 0x1c2, with value 06; this is the partition type. What does 06 mean?

Typing fdisk /dev/hda as root and giving the command l to list, shows that type 6 is:


 0  Empty           1c  Hidden Win95 FA 65  Novell Netware  bb  Boot Wizard hid
 1  FAT12           1e  Hidden Win95 FA 70  DiskSecure Mult c1  DRDOS/sec (FAT-
 2  XENIX root      24  NEC DOS         75  PC/IX           c4  DRDOS/sec (FAT-
 3  XENIX usr       39  Plan 9          80  Old Minix       c6  DRDOS/sec (FAT-
 4  FAT16 <32M      3c  PartitionMagic  81  Minix / old Lin c7  Syrinx         
 5  Extended        40  Venix 80286     82  Linux swap      da  Non-FS data    
 6  FAT16           41  PPC PReP Boot   83  Linux           db  CP/M / CTOS / .
 ...

So, it's FAT16.

Now, if I had been watching carefully, I would have known from the line sda: sda1 in /var/log/messages that the partition table was okay and contained only one entry.

Finding the FATs

When I actually started looking, however, I wasn't really sure if this was a FAT16 vs FAT12. The drive's capacity of 512MB suggested it could be either FAT16 or FAT32. I also somehow had the impression that the partition could have contained a FAT32 filesystem in the same partition type. As I continued to look through the filesystem, I noticed this:


# od -Ax -w8 -tx1 -tc /tmp/r1 | less
    045400 4c 45 58 41 52 20 4d 45 L   E   X   A   R       M   E
    045408 44 49 41 28 00 00 00 00 D   I   A   (  \0  \0  \0  \0
    045410 00 00 00 00 00 00 4b 5a \0  \0  \0  \0  \0  \0   K   Z
    045418 33 2b 00 00 00 00 00 00 3   +  \0  \0  \0  \0  \0  \0
    045420 41 52 00 53 00 54 00 55 A   R  \0   S  \0   T  \0
    045428 00 4c 00 0f 00 9a 6f 00 \0   L  \0 017  \0 232   o  \0
    045430 67 00 2e 00 78 00 6c 00 g  \0   .  \0   x  \0   l  \0
    045438 73 00 00 00 00 00 ff ff s  \0  \0  \0  \0  \0
    045440 52 4e 41 4c 4f 47 7e 31 R   S   T   L   O   G   ~   1
    045448 58 4c 53 20 00 b8 03 61 X   L   S      \0    003   a
    045450 50 30 e4 30 00 00 ca 74 P   0      0  \0  \0      t
    045458 4b 30 f2 6a 00 3e 00 00 K   0      j  \0   >  \0  \0
    ...

On a side note, I recently discovered the hard way that CMD | less doesn't do what you want it to if the output of CMD is too long. In this case it was okay to use, but it isn't always; this probably is system-dependent. If you have enough space on your hard drive, it may pay to do something like this:


    # od -Ax -w8 -tx1 -tc /tmp/r1 > /tmp/r2; less r2

or


    # hexdump -C /tmp/r1 > /tmp/r2; less r2

So this looks like the start of a directory. Immediately above that area, though, I saw this:


    042420 00 00 00 00 00 00 14 dd 15 dd 16 dd 17 dd 18 dd
    042430 19 dd 1a dd 1b dd 1c dd 1d dd 1e dd 1f dd 20 dd
    042440 21 dd 22 dd 23 dd 24 dd 25 dd 26 dd 27 dd 28 dd
    042450 29 dd 2a dd 2b dd 2c dd 2d dd 2e dd 2f dd 30 dd
    042460 31 dd 32 dd 33 dd 34 dd 35 dd 36 dd ff ff 00 00
    042470 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    *

That looked like an allocation chain with 16-bit entries. If these had taken the form 31 dd 00 00 32 dd 00 00 rather than 31 dd 32 dd, I might have thought I was looking at FAT32.

I had heard somewhere that typically two FATs can be found together, one right after the other. I told less(1) to find another line resembling the line at 0x42460, by typing ?31 dd 32 dd 33 dd. In response, less(1) showed me this:


    023a20 00 00 00 00 00 00 14 dd 15 dd 16 dd 17 dd 18 dd
    023a30 19 dd 1a dd 1b dd 1c dd 1d dd 1e dd 1f dd 20 dd
    023a40 21 dd 22 dd 23 dd 24 dd 25 dd 26 dd 27 dd 28 dd
    023a50 29 dd 2a dd 2b dd 2c dd 2d dd 2e dd 2f dd 30 dd
    023a60 31 dd 32 dd 33 dd 34 dd 35 dd 36 dd ff ff 00 00
    023a70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    *
    026a00 f8 ff ff ff 03 00 e6 02 21 03 a0 03 15 03 91 03
    026a10 ff ff 0a 00 0b 00 ff ff 0d 00 0e 00 0f 00 10 00

The data at 0x42460 and at 0x23a60 are the same; this told me that the offset between tables was:


    0x42460 - 0x23a60 = 0x1ea00

because 0x26a00 is the start of FAT#2. Therefore, the start of FAT#1 should be at


    0x26a00 - 0x1ea00 = 0x08000

But when I looked there, I saw this instead:


    007c00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    *
    012400 01 52 02 52 03 52 04 52 05 52 06 52 07 52 08 52
    012410 09 52 0a 52 0b 52 0c 52 0d 52 0e 52 0f 52 10 52
    012420 11 52 12 52 13 52 14 52 15 52 16 52 17 52 18 52

Somebody had written a whole mess of 0xff bytes. I guess this was part of the corruption.

At this point, 0x12400 looked okay, but was it? What's in the corresponding place in FAT#2?


    0x12400 + 0x1ea00 = 0x30e00
    030e00 01 52 02 52 03 52 04 52 05 52 06 52 07 52 08 52
    030e10 09 52 0a 52 0b 52 0c 52 0d 52 0e 52 0f 52 10 52
    030e20 11 52 12 52 13 52 14 52 15 52 16 52 17 52 18 52

Luckily, this looked okay too. In fact, FAT#2 might be completely okay even though the first 40KB or so of FAT#1 had been corrupted.

Repair Attempt #1

All of this has been interesting, but the point of this exercise was to repair the filesystem and read the data. So I now turned to my friend fsck for the repair work, in particular fsck.msdos, err and dosfsck(8). I took the filesystem image and did what needed to be done with a spare loop device:


    # losetup /dev/loop2 /tmp/r1
    # fsck.msdos /dev/loop2

But according to fsck.msdos(8), the "disk" claimed to have something near 165 FATs, whereas fsck.msdos only supports two. Apparently, some filesystem parameters were messed up severely.

Shortcut to Filesystem Repair

I started looking at the source code for mkfs.msdos, also known as mkdosfs(8), but then came up with a better idea. What if I could create a filesystem with the FAT parameters arranged so that the FATs and the directory in this new filesystem were in the same place where the FATs and directory were in the disk image I already had? The bytes that read LEXAR MEDIA probably were the volume name. Maybe, by giving the right parameters to mkfs.msdos(8), I could create a filesystem image wherein 0x08000 would point to the first FAT, 0x26a00 would point to the second FAT and 0x45400 would point to the volume label.

On the mkdosfs(8) manpage, I found:


SYNOPSIS
       mkdosfs  [  -A ] [ -b sector-of-backup ] [ -c ] [ -l file
       name ] [ -C ] [ -f number-of-FATs ] [ -F FAT-size ]  [  -i
       volume-id  ] [ -I ] [ -m message-file ] [ -n volume-name ]
       [ -r root-dir-entries ] [ -R number-of-reserved-sectors  ]
       [ -s sectors-per-cluster ] [ -S logical-sector-size ] [ -v
       ] device [ block-count ]

Therefore, I specified -f 2 for two FATs and -n mkfs__msdos--that is, a string I could find easily--for the volume name. This way I could tell where the vol-name landed.

How about the other parameters? I saw above that the FATs were 0x1ea00 bytes apart; if they landed the wrong distance from each other, I could tweak -F and maybe -s. I found on-line that for a filesystem of this size, the clusters would be 8192 bytes; in other words, there would be 16 512-byte sectors per cluster. The cluster is the file allocation unit described by the FAT. Hence, it would be -s 16.

As for where to create the filesystem, it wouldn't do to put it on the USB drive. Instead, I created a file the same size as the drive image but filled with zeroes:


    # dd if=/dev/zero of=/tmp/r2x bs=512 count=1001952

After creating the filesystem, I figured I'd mount it and create a file. The file would have enough data in it that we could see a reasonable allocation chain. To accomplish this, I wrote a script and prepared to call it with parameters until I happened to find everything where I wanted it. I called it b.sh:


    #!/bin/bash
    # parameters added to mkfs.msdos....
    ARGS="$*"
    if mount | grep /tmp/r2d; then umount /tmp/r2d; fi
    losetup -d /dev/loop2 
    losetup /dev/loop2 /tmp/r2x
    mkfs.msdos -n mkfs__msdos -s 16 $ARGS /dev/loop2
    mount -t vfat /dev/loop2 /tmp/r2d
    yes hello | dd bs=8192 count=3 of=/tmp/r2d/foo.txt
    umount /tmp/r2d

My plan was to try running this script with different parameters until I got it right. 0x8000 is 32KB. In 512-byte sectors, that's 64. Because the first FAT started at 0x8000, I decided to try -R 64, like this:


    # sh b.sh -R 64
    mkfs.msdos 2.8 (28 Feb 2001)
    Loop device does not match a floppy size, using default hd params
    2+1 records in
    2+1 records out
    #

The surprising thing was my first guess turned out to be right, at least as far as the FAT placement:


# hexdump -C /tmp/r2x | less
 ...
 00008000  f8 ff ff ff 03 00 04 00  f8 ff 00 00 00 00 00 00  |..........|
 00008010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 *
 00026a00  f8 ff ff ff 03 00 04 00  f8 ff 00 00 00 00 00 00  |..........|
 00026a10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 *
 00045400  6d 6b 66 73 5f 5f 6d 73  64 6f 73 08 00 00 71 89  |mkfs__msdos...q.|
 00045410  0f 31 0f 31 00 00 71 89  0f 31 00 00 00 00 00 00  |.1.1..q..1......|
 00045420  41 66 00 6f 00 6f 00 2e  00 74 00 0f 00 65 78 00  |Af.o.o...t...ex.|
 00045430  74 00 00 00 ff ff ff ff  ff ff 00 00 ff ff ff ff  |t.....|
 00045440  46 4f 4f 20 20 20 20 20  54 58 54 20 00 00 71 89  |FOO    TXT ..q.|
 00045450  0f 31 0f 31 00 00 71 89  0f 31 02 00 00 50 00 00  |.1.1..q..1...P..|
 00045460  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 *
 00049400  68 65 6c 6c 6f 0a 68 65  6c 6c 6f 0a 68 65 6c 6c  |hello.hello.hell|
 00049410  6f 0a 68 65 6c 6c 6f 0a  68 65 6c 6c 6f 0a 68 65  |o.hello.hello.he|
 ...

I didn't check the directory size, but it apparently it was okay as well--more on that below.

Grafting Filesystems

I now had a boot sector that would tell fsck.msdos to expect the FATs and the root directory at all the right places. So what if I created a filesystem image where the first sector was that one, but all the rest of the sectors contained data from the USB drive? Then, fsck.msdos would read the boot sector; I'd tell it to use FAT#2 to repair everything; and we'd see how it turned out.

Repair Attempt #2

To summarize exactly what fixed the USB device:

  • Step 1: create a filesystem image of the right size, with FATs and the directory in the right places:


    # dd if=/dev/zero of=/tmp/r2x bs=512 count=1001952 
    # losetup /dev/loop2 /tmp/r2x
    # mkfs.msdos -n mkfs__msdos -s 16  -R 64 /dev/loop2

  • Step 2: copy bytes from the corrupt image, except the boot sector, onto the filesystem image created in step 1:


    # dd if=r1 of=r2x bs=512 skip=1 seek=1

  • Step 3: execute filesystem repair on that image:


    # fsck.msdos -f -r /dev/loop2

Because I knew that FAT1 was bogus, I told it to use FAT2, and it reported success. It asked me whether to write the changes, and I said yes.

The filesystem images in /tmp/r2x and /dev/loop2 now were consistent. The acid test was to try to mount the filesystem:


    # mkdir /tmp/r2d
    # mount -t vfat /dev/loop2 /tmp/r2d
    # ls -lRA /tmp/r2d

After which all kinds of good stuff appeared.

Note: A good result to ls -lR showed that I was lucky in one other way: I didn't know if the boot sector had a good value for the size of the root directory, the -r parameter to mkfs.msdos. I simply used the default and it turned out fine.

Burning CDs

At this point, I decided I had better burn a CD. I burn and read CDs all the time on Linux, but I rarely burn CDs to be read by Windows. Again I did a Web search, and a page from IBM's DeveloperWorks site turned up. I had searched "linux burn CD windows" or something like that. So I tried this:


    # mkisofs -J -r -v /tmp/r2d | \
          cdrecord -v -pad -eject fs=4m speed=4 dev=0,0,0 -

I wasn't 100% sure that Windows would like this CD, but fortunately I have Windows95 under Win4Lin. Its sole purpose for me is to run Quicken and TurboTax, but I fired it up and pointed Windows Explorer at the just-burned CD-ROM. Explorer loved it. I used gimp(1) to capture a screenshot and e-mailed the image to my friend's brother--he was ecstatic.

APPENDIX: The Bash Script Explained

Shell jockeys need not read this.


  1 #!/bin/bash
  2 # parameters added to mkfs.msdos....
  3 ARGS="$*"
  4 if mount | grep /tmp/r2d; then umount /tmp/r2d; fi
  5 losetup -d /dev/loop2 
  6 losetup /dev/loop2 /tmp/r2x
  7 mkfs.msdos -n mkfs__msdos -s 16 $ARGS /dev/loop2
  8 mount -t vfat /dev/loop2 /tmp/r2d
  9 yes hello | dd bs=8192 count=3 of=/tmp/r2d/foo.txt
 10 umount /tmp/r2d

Line 1 identifies to exec(2) that this is supposed to be run by the shell. I've become accustomed to bash, the Bourne again shell.

Line 2 simply explains line 3, that the parameters you type after b.sh are parameters to add to the mkfs.msdos command line.

Lines 4-6 establish /dev/loop2 as the block device whose contents are in the filesystem image kept in /dev/r2x. Line 4 unmounts the artificial filesystem if it was mounted; this is done because we're about to make some changes to it. Lines 5-6 make sure that /dev/loop2 is connected to /tmp/r2x and only to /tmp/r2x.

Line 7 creates an artificial filesystem image with whatever additional parameters the user gave--remember $ARGS from line 3?.

Line 8 mounts the filesystem onto /tmp/r2d. Line 9 creates a file of about 24KB (three clusters), so I have a filename to look for at the beginning of the directory.

Line 10 then unmounts the artificial filesystem image, so the kernel does not think there are inconsistencies if I play with /tmp/r2x.

Bookmark/Search this post with:
digg digg | reddit reddit
» add new comment | email this page | printer friendly version | digg digg | reddit reddit | 45982 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

linux

Submitted by mp3 music (not verified) on Mon, 2007-09-24 17:17.

"This will which first released ninth our development GNOME 2.20.0, later and [gnome-announce]: is be this month..." for release candidate , . Bye., . Thanks.,

» reply | email this page

LINUCKS SAVES TEH DAY!!1

Submitted by Anonymous (not verified) on Tue, 2007-07-03 17:27.

LINUCKS SAVES TEH DAY!!1

» reply | email this page

stoped at the first step

Submitted by katu (not verified) on Mon, 2007-08-06 09:23.

Hi, I tried to follow your instructions to recovery data froma a camera, but when i try to make dd if=/dev/sda of=file from a live cd of kubuntu, my computer doesn´t work. any help? thanx a lot

» reply | email this page

This Makes sense, I've had a similar experience with Linux

Submitted by MAPWORLD (not verified) on Tue, 2007-03-27 12:48.

This makes sense. I once had a friend who had a Microsoft Word document on his flash drive which was corrupted on Windows. I stuck the flash drive in my laptop run Ubuntu v 6.06 (Freeware/shareware Linux) and opened the file up without any problems whatsoever. It just worked.

» reply | email this page

FAT was (probably) not corrupted

Submitted by Helmut from Germany (not verified) on Sat, 2007-01-13 06:02.

The many FF FF entries in the beginnning of the FAT were probably correct:

In FAT systems, unfragmenting utilities usually deploy directories at the beginning of the partition. If there are not too many entries in a directory, the 32-bit entries in the directories all fit in one block (e.g. in a system with 8k-blocks, there are 256 entries per block), so the directory "file" has the lenght of one block. It is marked in the fat with FFF (FAT12), FF FF (FAT16) or FF FF FF FF (FAT32).

So I would expect to see many FF bytes in the start of a FAT. Did you ever look whether the FAT contained FF bytes at the beginning or not?

The boot sector of a fat disk partition (not the MBR) contains drive parameters , if the FAT boot sector is damaged, you will have some problems. I suppose windows damaged that sector due to some bug, not the FAT. Writing there, e.g the information that there are about 165 fats ;-)

The disk parameters consist of head/cylinder/sector information (which can safely be ignored (and is ignored, I guess, even by Windows) since it is purely redundant, the relevant data being in the MBR, plus some routine information (e.g. bytes per sector, which is virtually alway 512), plus vital information as number reserved sectors at the start of the partition, which has to be known to calculate the start of the first FAT. Whether the information about the number of FATs is vital depends on the probability that someone created a FAT system with only one FAT ...

» reply | email this page

Collin - Thank you, thank you, thank you

Submitted by Craig (not verified) on Wed, 2006-12-20 13:14.

Collin - Thank you for posting this article! Yesterday, I accidently shut down my Windows XP computer without ejecting my Lexar Jumpdrive. Afterward, I got the "Drive not Formatted" error. Fortunately I found your article, and have a second computer running Redhat. Following essentially the same precedure you described, I was able to recover all the data from my disabled Jumpdrive. You saved the day. Thanks again!!

» reply | email this page

512MB USB corrupted in Windows XP

Submitted by Sash (not verified) on Tue, 2006-05-23 03:09.

Hey! i just happened to jump to this page, nice comments here. my problem is that this very incidence happened with me 2 weeks back. my 512 MB kingston usb disk got corrupted. i have tried numerous programs to read it. it is shown in windows xp as removabe drive but it cannot be formatted. the system is shown as raw. some hard disk tools do show the exact size of the disk but can anyone tell me how to get the disk back! i will appreciate the replies to my email at [email protected]

Thankx!

» reply | email this page

Recovery Tool!!!

Submitted by Surowi (not verified) on Thu, 2006-05-25 14:08.

PPL before you try anything you must check this.

http://www.runtime.org/gdb.htm

I managed to recover my files after 3 mins and with beautifull interface. This software works perfect. I tryed dozens dos recovery tools and noone worked

Try this mmaybe it helps you out

Regards

» reply | email this page

GetDataBack Sucks

Submitted by Anonymous (not verified) on Sat, 2007-02-10 20:54.

this is just another hostage holding, overpriced data reocovery tools that won't recover jack until you pay the outrageous $$$

» reply | email this page

gpart can help you too

Submitted by Anonymous (not verified) on Wed, 2005-09-28 19:11.

The information here was invaluable, because it gave me some encouragement in trying to recover the 250GB USB drive that Win2K suddenly didnt want to accept after I missed that a VMWare Virtual machine had mounted the drive...

I lost the partition table and have no disk large enough to create an image to.

With the above info I started dumping 50MB of the disk and found that no physical damage was evident, but still no way of finding the appropriate data for the approach above.
Knowing that it was a single 250GB FAT32 partition I downloaded GPart
http://www.stud.uni-hannover.de/user/76201/gpart/

It guessed what I believe correctly and after reboot the disk was mountable (in fact it automounted which maybe wasnt the best approach). I ran fsck.vfat with no reported errors!

Once again, I had great help of the info you supplied and hope that more people in desperate need will find your article.

» reply | email this page

fsck.msdos reporting to find 165 FATs

Submitted by Joerg (not verified) on Tue, 2005-07-19 07:20.

Thanks for the great article. When trying fsck.msdos on a damaged CF card image, I stumbled over the same problem:

the "disk" claimed to have something near 165 FATs

(actually 191 in my case). I then noticed that you mounted the MBR instead of the FAT partition. When specifying the partition offset with losetup (I believe 0x8000 in your case), e.g.

losetup -o 32768 /dev/loop2 /tmp/r1

then fsck.msdos should be able to find 2 FATs (the number of FATs should be at offset 0x10 from start of the partition) and your Rapair Attempt #1 might work.

If the partion table was intact, dd'ing /dev/sda1 instead of /dev/sda would give you the partition image only, thus eliminating the need to care about the partition offset.

Unfortunately, in my case FAT#2 amd the directory table is damaged. I'm going to write a restore program exploiting the FAT chains...

Thanks,
Joerg

» reply | email this page

links to FAT info?

Submitted by sean darcy (not verified) on Wed, 2005-06-22 02:38.

Great article. I'm now trying to recover an 11gb FAT32 partition from my wife's windows machine. The MBR seems OK ( fdisk sees the partition - as does w2k disk manager ). Somehow the partition itself is messed up ( virus? sunspots? ). windows doesn't think it has a file system. fsck.vfat thinks it has 255 FATs.

This article is exactly what I was looking for in trying to fix this. But it assumes a lot of info about FAT file systems:

"That looked like an allocation chain with 16-bit entries. If these had taken the form 31 dd 00 00 32 dd 00 00 rather than 31 dd 32 dd, I might have thought I was looking at FAT32."

Where do you find this info? This is way beyond my meager knowledge. Any references would be *really* useful.

» reply | email this page

re: links to FAT info?

Submitted by collin (not verified) on Sun, 2005-07-03 16:24.

Where do you find this info? This is way beyond my meager knowledge. Any references would be *really* useful.

Sorry, I don't remember. But I remember reading somewhere that what FATxx means is is that the cluster numbers take xx bits. Hence seeing two consecutive 16-bit numbers (31 dd 32 dd) gave me the clue that it was FAT16. Unless a FAT filesystem is severely fragmented, one would expect to see consecutive cluster numbers either in a file allocation chain (cluster numbers within a file) or the free list.

NOTE: I haven't verified the full technical accuracy of the following links, your mileage may vary, not a statement of my employer, don't try this at home, etc etc etc., but these are just links that appear useful, just based on a casual glance:

I just googled on "fat16 fat32 structure cluster number" (no quotes); the first couple of links look very useful. One of them contains a pointer to http://home.teleport.com/~brainy/fatgen102.pdf, which looks pretty useful.

Another place to look is http://www.storagereview.com/guide2000/ref/hdd/file/clustChaining.html, which I think you'll find helpful.

» reply | email this page

FAT info

Submitted by KultiVator (not verified) on Sun, 2005-06-26 14:31.

Here, very cool site!

http://home.freeuk.net/foxy2k/disk/disk1.htm

Of course, advisable to handle with care, preferrably messing with an image file first. :-)

» reply | email this page

What to do when FAT1/FAT2 aren't working?

Submitted by rob (not verified) on Sat, 2005-06-18 12:43.

Salve!
Thank you for this artikel, but what to do when both FATs are gone?

I have a problem wit a 128 MB SMartMedia Card (FAT16 formated with
axxpac, a memory extention for PalmIIIx http://www.axxpac.de),
probly due ejct during writing and befor unounting it with Linux.

What I found out (with help from Debian-User-German) ML befor finding your article:

cfdisk 2.12p
Disk Drive: /dev/hde
Size: 131072000 bytes, 131 MB
Heads: 16 Sectors per Track: 32 Cylinders: 500
Name Flags Part Type FS Type [Label] Size (MB)
Pri/Log Free Space 0,03*
hde1 Boot Primary FAT16 131,05*

From a fresh formateded Card wrote I 64 Bit to the Card I can't mounted
dd if=/hd1/smartmedia128/neu/ddsmart128-neu-050616 ibs=1 skip=24063 bs=1 \
count=64 seek=24063 of=/dev/hde

Now cfdisk tells:
hde1 Boot Primary FAT16 [AXXPAC/TH ] 131,05*

The card is now mountable, I see all directories and all files, *g*
but only files smaller then 0x4000Bit (16384) can be read without
read/write errors. dosfsck will truncate all files to

» reply | email this page

Continue.... What to do when both FAT not working...

Submitted by rob (not verified) on Sat, 2005-06-18 12:48.

dosfsck will truncate all files to

» reply | email this page

continue ... does the forum don't like "

Submitted by rob (not verified) on Sat, 2005-06-18 12:52.

read/write errors. dosfsck will truncate all files to smaller 16384 and
is no help. Is this 0x4000 limit based on cluster - that there is
information to jump to the next cluster missing?
With fsck I got also the errormessage that multiple files or
directories use the same cluster.

ls -lRA is working, but some files and dirs have the date of the
crash day - but they are older and not written on that day.

So "ls -lRA" is not the final "Acid Test" for the recovery, a "cp -Rp *"
would not work in my case.

-What can I do insted of msfsck?
-When I something with "ls -lRA" the FAT1/FAT2 are still there?
-How can I extract the starting adresses for each file I see
with ls -lRA?

I know that there are tools like autopsy, but I hope that
the FATs are still recoverable.

Greetings
rob

PS: Sorry for my fragmentated post ;)

» reply | email this page

The FATs

Submitted by KultiVator (not verified) on Sun, 2005-06-26 15:00.

From what I understood, you've overwritten both FATs with "clean" ones? Bad move, sorry..
FAT = File Allocation Table .. basically, holds the information of where different parts of files are on a disk.

The root directory (which is located right after the FATs), however, holds some other info: file names, attributes, sizes, dates, and starting clusters' numbers, that's why you can see the files and can access them, but only the first 16kB, since that is the size of a cluster on that particular disk..

» reply | email this page

Was the drive corrupted or filesystem?

Submitted by mangoo (not verified) on Sat, 2005-06-18 03:35.

This piece is confusing: "According to Lexar tech support, there is a bug with Windows 2000 (that MS never bothered to fix) and can corrupt the drive when it is removed without proper eject.".

Does this mean that the drive was damaged because of this bug (and you could say good bye my $70 USB), or was it the filesystem on this drive (so that reformatting that USB would make it usable again)?

» reply | email this page

re: Was the drive corrupted or filesystem?

Submitted by collin (not verified) on Sun, 2005-07-03 15:56.

Just the filesystem was corrupted. The electronics in the card were fine.

» reply | email this page

Prosoft engineering

Submitted by Anonymous (not verified) on Thu, 2005-06-16 13:50.

Prosoft engineering produces a range of data rescue software for the Mac which I have personally used with much success. They support everying from HDD to memory cards to USB flash drives.

address is: http://www.prosofteng.com

There are various products from Image Rescue to Data Rescue, they are not open source but there are trial versions available for download which will allow you to recover a file at a time. Of course all of this is for the mac only. They are famous for their official Novell client for Mac as well.

» reply | email this page

Shameless ad.

Submitted by WeAllHateSpam (not verified) on Wed, 2007-02-14 12:05.

Shameless ad.

» reply | email this page

Mounting under /tmp

Submitted by Duncan Thomas (not verified) on Wed, 2005-06-15 05:44.

Nice article, printed and saved for if I ever need to do something similar. A very quick point though - beware of mounting under /tmp on systems that aren't yours; some badly written tmp cleanup scripts can go in and clean up your newly mounted filesystem :-( I've lost filesystems to this a couple of time before I worked out what was going on...

» reply | email this page

JPEG images

Submitted by Gerrit Hannaert (not verified) on Wed, 2005-06-15 05:05.

I approached a similar problem I had differently - I knew I was looking for JPEG images and that there was a chance they were intact. I assumed the FAT was unreliable, and admittedly forgot about the second FAT copy.

I also assumed there was no fragmentation, and then proceeded to 'dd' anything that looked like an EXIF header +1.5MB to disk as individual files.

Something along these lines:
hexdump -C sda1_flashdisk.dd | grep -i "exif" >headers.hex
cat headers.hex | awk --non-decimal-data '{print (sprintf("%d","0x"$1)/512)}' > headers.512.dec
cat headers.512.dec | xargs --replace bash -c "dd skip={} if=sda1_flashdisk.dd bs=512 count=3000 of=./{}.jpg"

I was lucky on several counts: all files were of predicatable content, there was little if any fragmentation, and JPEG seems to tolerate excess junk added to the end of file :-)

» reply | email this page

I wish I had known about this

Submitted by Anonymous (not verified) on Wed, 2005-06-15 00:37.

I wish I had known about this about a month ago. My Father had a 128MB Sandisk USB flash disk which he had been using under Windows 2000 and it died in a rather spectacular way, the only difference here though is that when I fed it to my linux workstation, it did not find any partition records - it was royally goosed. I had an identical USB flash disk to my Father's and tried to overwrite the partition records from my chip to his, but with no success. Extremely interesting article though and very well researched. Congratulations on recovering your data - I challenge any Windows user to even come close to that without having to write a suite of programs.

» reply | email this page

Overall, pretty good

Submitted by Azhrei (not verified) on Tue, 2005-06-14 09:35.

I've done this type of recovery myself, using basically the same approach: look at hex dumps and compare the corrupted media with uncorrupted media, looking for areas that match, then copying individual sectors in an attempt to produce something readable. I wouldn't have thought of using the backup FAT table for fsck.msdos, though. :) Usually, fixing a partition table would do it. Or I would've copied FAT#2 on top of FAT#1 and just tried to mount it...

» reply | email this page

Overwriting FAT#1 with FAT#2 (or vice-versa)

Submitted by KultiVator (not verified) on Sun, 2005-06-26 15:20.

Overwriting FAT#1 with FAT#2 (or vice-versa) - this isn't necessarilly a good idea if there's a chance that both are more or less corrupted.
In such cases, trying to "merge" them (with extreme care, of course) would be better..
Well, just using the copy #1 at one time, and #2 at the other, while both times trying to access & restore files and see which time you get better results ("proper" file contents), would be the most cautious way.

» reply | email this page

thanks for your comment

Submitted by Collin (not verified) on Tue, 2005-06-14 21:03.

Thanks for posting your comment! Those are both good ideas, which I didn't think of. Well, I thought of trying to fix the partition table but I didn't know enough about the boot sector contents (and still don't).

The important thing is that GNU/Linux distros give us enough tools so that we can approach these problems in different ways... even problems caused by closed operating systems!

Thanks again for your note.

» reply | email this page

Question: How would gpart hav

Submitted by Anonymous (not verified) on Wed, 2005-06-15 03:54.

Question: How would gpart have performed on that job?

I recovered once a whole laptop for someone in the office with half a years work of source code on it...

» reply | email this page

"I recovered once a whole lap

Submitted by Mason Deaver (not verified) on Thu, 2005-06-16 07:58.

"I recovered once a whole laptop for someone in the office with half a years work of source code on it"

Can we infer from your comment that your colleague whose work you saved is a software developer who had not backed up his or her work in six months? Yeesh! I'm getting queasy just thinking about this; I need to go lie down for a while ....

» reply | email this page

Windows tools to test

Submitted by Anonymous (not verified) on Thu, 2005-06-16 04:00.

Hey, there are some windows based tools that exist but I don't think they work for flash drives, but you could always try.

Driverescue 1.9d (scans through bios in windows).

Testdisk - http://www.cgsecurity.org/testdisk.html

PCInspector - which looks exactly like Driverescue before driverescue went commerical http://www.pcinspector.de/file_recovery/UK/welcome.htm

» reply | email this page

Using windows Disk Manager

Submitted by QiQiangZhu (not verified) on Sat, 2006-06-17 13:40.

Using windows Disk Manager to creat a new partition for your usb drive, although may lost 8MB space, still is the best way to do the rescure for a USB flash drive.

» reply | email this page

Or using Partition Magic

Submitted by QiQiangZhu (not verified) on Sat, 2006-06-17 22:18.

Or using Partition Magic Boot Disk to repartition this USB drive, you will gain back all the space on it.

» reply | email this page

Recover HD after Ghosting

Submitted by need help (not verified) on Sat, 2006-09-09 16:40.

Could you use this same process to recover files after Ghosting a HD? If not; what would you suggest I use? If it's possible at all to recover files after Ghosting a HD.

» reply | email this page