==== Simulated data recovery with Testdisk and Photorec ====
=== Introduction ===
Testdisk is a data recovery tool. Its capabilities depend on the filesystem it's operating on. E.g., it can read backup ext3 partitions, or manually specify partition types. It's also good with deleted files or incomplete files.
In this column, I'm going to test testdisk and photorec with a simulated corrupt filesystem.
=== Procedure ===
- Create a filesystem,
- Populate it
- Corrupt the filesystem
- Attempt to recover the files
=== Create the filesystem ===
For testing I'm going to use a loopback filesystem on Linux. This means I can work within files and spare the pain of working with disks.
blacktower:~# dd if=/dev/zero of=./baddisk.img bs=1024 count=50000 50000+0 records in
50000+0 records out
51200000 bytes (51 MB) copied, 0.26165 s, 196 MB/s
blacktower:~# mkfs -t ext3 ./baddisk.img mke2fs 1.41.12 (17-May-2010)
./baddisk.img is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
12544 inodes, 50000 blocks
2500 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=51380224
7 block groups
8192 blocks per group, 8192 fragments per group
1792 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
blacktower:~# mkdir /mnt/baddisk; mount -o loop ./baddisk.img /mnt/baddisk
Now to create some test files.
i=0; while [ $i -lt 1000 ]; do /usr/games/fortune >> /mnt/baddisk/fortunes.txt; i=$((i+1)); done
blacktower:~# cp /var/pub/testfiles/* /mnt/baddisk/
blacktower:~# ls -l /mnt/baddisk/
total 1216
-rw-r--r-- 1 root root 114616 Apr 6 11:03 fortunes.txt
-rwxr--r-- 1 root root 17920 Apr 6 11:24 libreoffice writer.doc
-rwxr--r-- 1 root root 24391 Apr 6 11:24 libreoffice writer.odt
-rwxr--r-- 1 root root 42101 Apr 6 11:24 libreoffice writer.pdf
-rwxr--r-- 1 root root 18541 Apr 6 11:24 libreoffice writer.rtf
drwx------ 2 root root 12288 Apr 6 10:55 lost+found
-rwxr--r-- 1 root root 962190 Apr 6 11:24 testimage.bmp
-rwxr--r-- 1 root root 17444 Apr 6 11:24 testimage.jpg
-rwxr--r-- 1 root root 18523 Apr 6 11:24 testimage.png
This test is very synthetic because the filesystem has never had a delete. There's no fragmentation and all the files are sitting in an optimal location.
=== Making the filesystem more realistic ===
Let's fragment! (warning, dangerous script)
#!/bin/bash
# pick a file, copy it, delete the original
targetdir=/mnt/baddisk
mkdir "$targetdir"/tmp
i=0
while [ $i -lt 10000 ]
do
# echo $i
set -- "$targetdir"/*
shift $(($RANDOM%$#))
if [[ -f $1 ]]
then
cp "$1" $targetdir/tmp/
rm "$1"
mv "$targetdir/tmp/`basename \"$1\"`" "$targetdir"
fi
i=$(($i+1))
done
MD5 hashes permit us to check the file integrity later. They're stored separately from the filesystem.
blacktower:~# md5sum /mnt/baddisk/*
0b5080a051d53ba2432b666c90d7c0b4 /mnt/baddisk/fortunes.txt
8f301122918ab70d8cdccebdac46c8c5 /mnt/baddisk/libreoffice writer.doc
45f1fbff8dcd6a92eef33e5639837c81 /mnt/baddisk/libreoffice writer.odt
6dbeeb8ff8b82d35296dfc5265897f92 /mnt/baddisk/libreoffice writer.pdf
9d4cb7f894983e667a7c2173f9fa59c4 /mnt/baddisk/libreoffice writer.rtf
md5sum: /mnt/baddisk/lost+found: Is a directory
ed0eda5ba4f73f8d41eade656db27517 /mnt/baddisk/testimage.bmp
3c90cdc73c1b155b7cdd5e0a13584207 /mnt/baddisk/testimage.jpg
8b40c922b02ef43be7c986fa59536260 /mnt/baddisk/testimage.png
md5sum: /mnt/baddisk/tmp: Is a directory
=== Corrupting the filesystem ===
First we'll unmount the filesystem and Check to see that it's okay.
blacktower:~# umount /mnt/baddisk
blacktower:~# fsck.ext3 ./baddisk.img
e2fsck 1.41.12 (17-May-2010)
./baddisk.img: clean, 20/12544 files, 7703/50000 blocks
To corrupt the disk, I'm throwing random nulls throughout the image with the code below.
blacktower:~/testdisk# cat ./corrupt.bash
#!/bin/bash
i=0
while [ $i -lt 100 ]
do
target=$(( RANDOM % 5120 ))$(( RANDOM % 10000 ))
echo $target
#dd if=/dev/zero of=./baddisk.img bs=1 count=1 seek=$target 2> /dev/null
./corrupt ./baddisk.img $target
i=$((i+1))
done
blacktower:~/testdisk# cat ./corrupt.c
#include
/* program to write to specific location of a file */
int main(int argc, char* argv[])
{
FILE* fh;
if( argc != 3 ) {
printf( "\nUsage: %s filename\n\n Where \"filename\" is the file to corrupt and \"location\" is the offset in bytes\n", argv[0] );
return(1);
}
fh = fopen( argv[1], "r+" );
if( fh == NULL ) {
printf( "\nCould not open existing file for writing!\nUsage: %s filename\n\n Where \"filename\" is the file to corrupt and \"location\" is the offset in bytes\n", argv[0] );
return(1);
}
fseek( fh, atoi(argv[2])-2, SEEK_SET );
fputc( 0, fh );
fclose( fh );
return 0;
}
blacktower:~/testdisk# ./corrupt.bash
blacktower:~/testdisk# mount -o loop ./baddisk.img /mnt/baddisk/
blacktower:~/testdisk# md5sum /mnt/baddisk/* 2> /dev/null
0b5080a051d53ba2432b666c90d7c0b4 /mnt/baddisk/fortunes.txt
8f301122918ab70d8cdccebdac46c8c5 /mnt/baddisk/libreoffice writer.doc
45f1fbff8dcd6a92eef33e5639837c81 /mnt/baddisk/libreoffice writer.odt
6dbeeb8ff8b82d35296dfc5265897f92 /mnt/baddisk/libreoffice writer.pdf
9d4cb7f894983e667a7c2173f9fa59c4 /mnt/baddisk/libreoffice writer.rtf
24896c06121360c4b3daa1deca7e854f /mnt/baddisk/testimage.bmp
3c90cdc73c1b155b7cdd5e0a13584207 /mnt/baddisk/testimage.jpg
8b40c922b02ef43be7c986fa59536260 /mnt/baddisk/testimage.png
blacktower:~/testdisk#
Looks like we hit mostly empty space. Only the bmp is corrupted.
Change the 100 random bytes to 10000 and run again:
blacktower:~/testdisk# fsck.ext3 ./baddisk.img
e2fsck 1.41.12 (17-May-2010)
./baddisk.img: clean, 20/12544 files, 7703/50000 blocks
blacktower:~/testdisk# md5sum /mnt/baddisk/* 2> /dev/null
e04f2a97d51432d04a64f642e350b1f0 /mnt/baddisk/fortunes.txt
8f301122918ab70d8cdccebdac46c8c5 /mnt/baddisk/libreoffice writer.doc
fc2475f7a1a70f2f31ae50ca59b96cb7 /mnt/baddisk/libreoffice writer.odt
6037209ee8c78156a1e6acea00b09dd1 /mnt/baddisk/libreoffice writer.pdf
16a93d89a8dedfd23ab629006104cbca /mnt/baddisk/libreoffice writer.rtf
aaf2de72c3f2966bf414d598435dc432 /mnt/baddisk/testimage.bmp
3c8a111a8c9baaa09aea73e7d59e890e /mnt/baddisk/testimage.jpg
e3912ad720969f46308cdccad73a66f8 /mnt/baddisk/testimage.png
Only the .doc file is safe now.
Again.
blacktower:~/testdisk# fsck.ext3 ./baddisk.img
e2fsck 1.41.12 (17-May-2010)
./baddisk.img: clean, 20/12544 files, 7703/50000 blocks
blacktower:~/testdisk#
fortunes.txt is visibly corrupted, and every file fails an MD5, but the filesystem hasn't reported any problems yet.
{{:article-testdisk:corruption.png?nolink&300 |}}
Different method:
blacktower:~/testdisk# tail -n +10 ./baddisk-2.img > baddisk-2b.img
blacktower:~/testdisk# ls -l
total 299852
-rw-r--r-- 1 root root 50674559 Apr 10 18:10 baddisk-2b.img
-rw-r--r-- 1 root root 51200000 Apr 6 21:45 baddisk-2.img
blacktower:~/testdisk# mount -o loop ./baddisk-2b.img /mnt/baddisk/
mount: you must specify the filesystem type
blacktower:~/testdisk# mount -t ext3 -o loop ./baddisk-2b.img /mnt/baddisk/
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
blacktower:~/testdisk# fsck.ext3 ./baddisk-2b.img
e2fsck 1.41.12 (17-May-2010)
fsck.ext3: Superblock invalid, trying backup blocks...
fsck.ext3: Bad magic number in super-block while trying to open ./baddisk-2b.img
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193
blacktower:~/testdisk# fsck.ext3 -b 8193 ./baddisk-2b.img
e2fsck 1.41.12 (17-May-2010)
fsck.ext3: Attempt to read block from filesystem resulted in short read while trying to open ./baddisk-2b.img
Could this be a zero-length partition?
blacktower:~/testdisk# ls -l ./baddisk-2b.img
-rw-r--r-- 1 root root 50674559 Apr 10 18:10 ./baddisk-2b.img
blacktower:~/testdisk#
The partition table is missing and the offsets for the backups are all messed up.
=== Testdisk ===
non-partitioned
=== photorec ==
PhotoRec 6.11, Data Recovery Utility, April 2009
Christophe GRENIER
http://www.cgsecurity.org
Disk ./baddisk-2b.img - 50 MB / 48 MiB (RO)
Partition Start End Size in sectors
P Unknown 0 0 1 6 41 1 98974
14 files saved in /root/testdisk/recup_dir directory.
Recovery completed.
txt: 14 recovered
blacktower:~/testdisk/recup_dir.3# head -1 *.txt
==> f0000000.txt <==
Just to have it is enough.
==> f0000128.txt <==
men proud.
==> f0003072.txt <==
==> f0004352.txt <==
it classified?
==> f0004480.txt <==
rail2\hyphmax0}\aspalpha\ltrpar\langfe255\lang255\cf0\kerning1\hich\af3\dbch\af4\afs24\lang255\loch\f0\fs24{\rtlch \ltrch\loch
==> f0006400.txt <==
wiss\fprq2\fcharset0 Arial;}{\f3\fnil\fprq2\fcharset0 Andale Sans UI{\*\falt Arial Unicode MS};}{\f4\fnil\fprq2\fcharset0 Tahoma;}{\f5\fnil\fprq0\fcharset0 Tahoma;}}
==> f0007552.txt <==
us.
==> f0014848.txt <==
ker is a fellow who lends you his umbrella when the sun is shining
==> f0017024.txt <==
ought, and the wisdom never to use either.
==> f0017280.txt <==
owntown Newark is in your future.
==> f0017664.txt <==
outh gets trashed.
==> f0078080.txt <==
e
==> f0095488.txt <==
amenco dancer's
==> f0095616.txt <==
up to his neck in sand?
blacktower:~/testdisk/recup_dir.3#
=== Try harder ===
Paranoid : Yes (Brute force enabled)
14 files
=== Expert Mode ===
79 files saved in /root/testdisk/recup_dir directory.
Recovery completed.
txt: 79 recovered
Unless we find the superblock, it looks like the fragmentation of the files is going to make it nearly impossible to find any files.
We could create a signature to find the superblock... hmmm...
http://www.virtualblueness.net/Ext2fs-overview/Ext2fs-overview-0.1-12.html
=== Additional Information ===
http://www.cgsecurity.org/wiki/TestDisk
http://www.cgsecurity.org/wiki/PhotoRec