Friday, 23. January 2009, 12:04:22
For information on File Carving, please see the wiki article below.
http://www.forensicswiki.org/wiki/CarvingSo, you deleted a file, and can't seem to find it now. What do you do?
On Linux systems, you may have checked the "Trash", and searched for the file, etc, etc.
On Windows systems, you hopefully checked the "Recycle Bin", and may have searched for the file as well.
Nothing turns up. What are you to do?
Well, thankfully, that file isn't gone yet. At least, hopefully.
Before I get into the recovery of the file, I find it to be best if I explain what exactly happens when you delete that file.
For Linux users, you may have actually deleted it, rather than "Move to Trash".
Same for Windows users, only difference being that you send it to a "Recycle Bin" not a Trash folder.
Yes, Windows users, you can add a context menu to delete the file, rather than move it to the "Recycle Bin".
So, what exactly happens if you do one of those first two, moving it to the "Recycle Bin" or "Trash"?
Well, on your computer's hard disk is what is called a filesystem. The filesystem is responsible for taking your hard disk and dividing it up for file storage. The filesystem is also responsible for keeping track of where files are on the disk.
When you move the file to the "Recycle Bin" or "Trash" you're basically just copying the file from "Folder X" to "Folder Y". Folder X being the original location, Folder Y being the holding place for trash items.
All this does is tell the filesystem to now point to a new folder, not to actually remove the file.
Ah, okay, so the file is still there. Easy enough to recover (not diving into that).
Now, what happens if you were to say, empty the "Recycle Bin" or "Trash"?
Well, what you've essentially done is told the filesystem, "Okay, I'm done with this file, remove it from your database of files".
So, it's now removed from the database of files that all filesystems keep, but wait, I never said it's actually erased.
That's because it's not. By removing from the database you now allow the filesystem to use the space that the file had occupied. It doesn't mean it's actually being used yet, or that the file's information is gone.
Regardless of your Operating System, all computers use some filesystem to manage the files and directories on a hard disk. Due to that fact, the methods listed below should suffice for data recovery on a multitude of Operating Systems. I personally have only tested it on Linux (Fedora, Ubuntu, Gentoo) and Windows (XP, Vista, and 2000).
Now that, that's out of the way, how do we get our file back?
Below is a step-by-step guide on how to accomplish this task, broken down into sections, Section Seven is a step-by-step example. Hopefully this will help at least one person out.
In the event your hard disk is encrypted these steps will not be of assistance for recovering your file.
I am currently working on a way to hopefully accomplish this seemingly impossible task though as well.See my note at the bottom of the post for how to carve data from an encrypted drive.
[
Please note - Once you have completed sections one through three you can use a faster way to obtain the files. Some Linux distros do not come with a pre-compiled version of the tools you may use and building software on a LiveCD will take quite a bit of time. If you are interested in recovering the data as quickly as possible, please have a look at the link below:
http://www.ubuntugeek.com/recover-deleted-files-with-foremostscalpel-in-ubuntu.htmlAlso, worth mentioning, the instructions are for Ubuntu based systems, but the same tools should be available on other LiveCD distros as well, esp those for data recovery/forensics/security.]
Section One - Preserving the DataThe very first thing we need to do, is shutdown your PC.
Why? Well, if you accidentally erased that file, it could now be overwritten by anything you do. Yes, by reading this article you could overwrite the file with your temporary internet files. So shutdown your system and use a LiveCD distro to continue on. Windows users, you will need to use the LiveCD method even for external media, Linux users you don't have to if it's an external medium, but you will if it is the disk your system is running on.
Section Two - File ReconIn order to carve a file from an image, we first need to know a few key things about the file.
What kind of file was it?
Was this an image, a text file, a music or video file, or something else all together?
Of those, do we know what the extension was?
Assuming it was an image: Was it a jpg? png? gif? tiff? bmp?
[
Please note - If you are using the faster method mentioned above, you can skip everything below this, and follow the directions mentioned on the link provided.]
Hopefully these are things you can remember about the file. If not, the carving process may take you much longer than anticipated.
Once we know what kind of file it was, and the extension, have a look at the website below about file signatures:
http://www.garykessler.net/library/file_sigs.htmlOn this page we want to locate our file's format/extension if they have it listed.
If not, you may want to use google to see if you can find it on there. A good place to begin is by searching for
"File Signature" "EXTENSION"
Where EXTENSION is the extension of the file you're looking for. eg: PNG, GIF, etc... You'll want to do the search exactly as I have it listed, only changing the extension, if this search does not yield any results for your file's signature in a hex format, try some other searches.
Once you have your files hex signature, keep it handy, you will need it later. Also take note of the Trailers if they have one listed. This will make recreating the file that much easier.
Section Three - Selecting a MethodThe safe method (recommended but takes longer):Use the Linux command "dd" to take an image of your hard disk.
In order to do this you'll need to have a backup medium that is either the exact same size as your original disk, or one that is larger. Larger is recommended.
I could go into detail on how to image the disk using dd, but there's a lot of info on Google about that topic.
The unsafe method (faster)The unsafe method involves you directly accessing you original disk.
This is called the unsafe method because it can cause data corruption beyond the loss of the original file.
It is a good way to quickly recover a file that has to be recovered immediately, but if you have a couple of days to do the image, imaging is recommended.
Either way you go, you should either now have an image of the disk to work with, or you should have the disk itself.
WARNING!!! If you use the unsafe method, I take no responsibility for a corrupted or damaged disk! You have been warned of the dangers!Section Four - Picking an EditorThis isn't the most important step, it's not even a required step, it's one that makes the recovery job a little more custom to suit your needs and preferences.
Then why is this important? Well, I don't like to tell people what they
HAVE to use to do a job.
It's like my boss coming up to me and telling me I have to use a stapler when a paper clip gets the job done just as well.
My personal preference is Hexedit, a terminal based program which has an excellent way of working with files. See the link below for information on working with files in Hexedit.
http://prigaux.chez.com/hexedit.html(for my example I will be using Hexedit to do the data recovery.)
Another choice is GHex2, which is a Gnome Hex Editor.
There are several other editors out there, pick one you like.
Again, this is all personal preference
Section Five - Locating the DataHere is where the real fun begins. We now need to recover the file from the drive.
If you had a lot of files of the same format you're searching for, you'll probably have a lot of results when you're searching for your file's signature. This can heavily impact the amount of time it will take for us to recover your specific file or set of files.
In order to carve the file from our hard disk or image we open up our medium in our hex editor.
If you're using the hard disk, it will be something like /dev/hda1 or /dev/sda1.
If you're using the disk image, it will be the image name you created. Example: /media/disk-1/myimage.img
Now, to find the files that relate to our particular type of file we are searching for.
We'll want to use your hex editors search function to find the file signature you noted earlier.
Now that we have located that, take note of where this specific set of information is.
In Hexedit you'll find it at the bottom of the terminal as 0x6B34161/0x8EDC400 where the first number is the offset you are currently at, and the last number is the total offsets in the file. GHex2 also notes the offset at the bottom of the window.
Next we will want to search for the trailer signature that you had noted down earlier.
Again, we want to note down it's offset. (Noting it down is a good idea so that you don't use the same offset twice if we have multiple files of the same type)
Section Six - Extracting the DataNow to actually take that information that was once our file, and recover it to a new file.
We now want to copy all the information that is between the two offsets you just noted down, including the two offsets themselves.
Once we have that copied, we want to extract it into a new file, with the same extension that our original file had.
Section Seven - An Example Case StudyNow that you know how to recover the data, let's prove it works.
[
Please note - I'm running all the commands below as the root user.]
For this, I'm using my 512Mb USB Drive.
First, I create the partition using CFdisk:
cfdisk /dev/sde
I'm using a 150Mb Partition, because it is easiest to work with for me. Nice and small, but still provides enough space to actually place a file or a couple of files on the disk.
I want to zero out the partition to ensure that there is no previous data left on the disk.
dd if=/dev/zero of=/dev/sde1 conv=sync
Next I create a filesystem on the newly formed disk (I use XFS you can use ext2, ext3, or any filesystem you like.)
mkfs.xfs -f -d agcount 4 /dev/sde1
The "-f" option should be used to force the creation of the filesystem. The "-d agcount=4" is totally optional.
Now we have a fresh partition, hopefully free of any information.
Well, that's not actually the case, just doing a low level format will still leave at least trace amounts of information from files on your old system. That's not the case though here, because I zeroed out the blank platter before hand.
Following the creation of the filesystem, I want to copy over a file, so that there is something on the disk.
cp /home/user/myimage.jpg
And we want to sync the disks to be sure that the information was written to the filesystem.
sync
And now I want to "delete" the file"
rm -f /media/disk-3/myimage.jpg
the "-f" option here tells us to force it to remove the file, preventing it from asking me if I'd like to.
And we want to sync the disks again to be sure that the information was written to the filesystem.
sync
Next step, is to unmount the filesystem so that the information on the disk does not get overwritten blindly.
umount /dev/sde1
Following the unmount, we now need to image the disk. Since there is no information I care about on the disk, I
COULD do a direct disk carving, but to show an example of how to do this step-by-step I'm going to take an image.
dd if=/dev/sde1 of=/home/user/149.8Mb.img conv=fdatasync,fsync,sync
Now that we have our image, we want to carve our file from it to make sure the carving process works correctly.
hexedit /home/user/149.8Mb.img
To locate the file's signature I use Hexedit's search feature
CTRL+S
Here, CTRL+S is a keypress, not a command to be typed in the terminal.
We get promted for what to search for, since my file was a JPG file, I use that files type's signature:
FFD8FF
There are actually two for JPG file formats: FF D8 FF E0 and FF D8 FF E8, to make searching simpler and hopefully faster, I just shortened it. It may lead you to a non image-format string though. To be sure look for JFIF somewhere close by on the text side of your hex editor.
For my particular example, the string was found, and it is an image. The offset is 0x6B34000
Now that we have that part of the file (the beginning) I want to search down and look for the trailer of the file. For this I search for the JPG trailer of FFD9
CTRL+S
FFD9
In this example my offset is 0x6B44AE7.
With the two offsets known, we now need to copy the information.
CTRL+SPACE
Sets the marker to the active offset.
Once you have used the page up key to select the contents back up to the beginning offset, you'll want to actually copy the information.
ESC+W
And Paste it into a new file.
ESC+Y
You will be prompted for a place to put the information, I used /home/user/recovered.jpg
Upon opening the file you will see that it is the exact image that you had deleted before. Actually, if you have thumbnails, you'll notice it before you even open the file.
Amazing isn't it?
As a small side note, I'd like to mention, this is by no means the only ways to recover a file. This is simply the method I use as it's tried and proven.
This example is me carving data from a running system. I didn't actually use a LiveCD, and you don't have to if it's an external medium. LiveCDs are used for disks that your system is running on.
I'll be uploading screenshots of this in a bit and adding them to this post, as well as posting the test results from my distro testing.
(Yes, I've actually finished typing up the white-paper, now I just need to put it on here)
Note 1: Update - I have found a way to carve files from an encrypted disk. It requires you to actually mount the disk though. As an example, I have an LVM on a Luks encrypted section of my hard disk. In order for me to carve the file from that particular disk, I would use /dev/mapper/VolGroup00-LogVol00 as the input file for dd.
If you have any issues or questions about this, please post a comment and I'll assist you as best I can.As always,
Happy Hacking
PiklesOnFire