[1+1=2]

OneAndOneIs2

« Coding a random filesystem diagramYay! Kill the spam-bots! »

Thu, Jan 04, 2007

[Icon][Icon]Why deleting just isn't enough

• Post categories: Omni, In The News, Technology, Helpful

Every few months, a slow news day leads to somebody, somewhere, buying an old PC, hard drive, or flash memory card off ebay, and then writing a story about how they were able to restore all the files that the previous owner had tried to erase prior to selling.

If you want to sell hardware and you're not sure how some people can recover data from supposedly-erased hard drives, this article is for you.

I'm going to use this diagram to explain the whole thing: It represents data stored on a PC filesystem, such as a hard drive or Flash memory such as you get in digital cameras. It's hugely reduced in size (even a floppy disk would be more than 2000 times bigger than this!) to simplify the explanations, but it's good enough to illustrate the principles:

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
b  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
g  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
h  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
i  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
j  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
k  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
l  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
m  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
n  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
o  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
p  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
q  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
r  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
s  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
t  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
u  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
v  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
w  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
x  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
y  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
z  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

It's currently a totally blank disk. Each zero represents one byte of data.

Now, no working disk drive looks like this, even when it's empty. The first thing that a disk has is a partition table. Most Windows PCs only have one partition, very slightly smaller than the capacity of the hard drive. But you can have up to four partitions on a normal disk drive (or even more, depending on your operating system.)

So we partition our disk drive, and now the computer knows where it can store data:

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  p a r t i t i o n 1 = b a - z z 0 0 0 0 0 0 0 0 0 0
b  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
g  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
h  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
i  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
j  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
k  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
l  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
m  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
n  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
o  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
p  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
q  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
r  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
s  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
t  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
u  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
v  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
w  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
x  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
y  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
z  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

We've defined a single partition that occupies the disk from the start of the second row (ba) to the end of the last row (zz). We can't start storing data before 'ba', because hard drives devote a certain amount of space to partition tables, and in our case, it's the whole of the first row.

Next, we need to format our partition - in Windows, that means either NTFS or FAT. Other OSes use other filesystems. We're going to use an imaginary one to keep things simple. (To save space, I'm not going to show all the empty lines in the following diagrams)

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  p a r t i t i o n 1 = b a - z z 0 0 0 0 0 0 0 0 0 0
b  f o r m a t = c a - z z 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Again, we've devoted a complete row, this time to information about our formatted partition. However, the remaining 24 lines of space are now ready for writing. With this particular filesystem, the first row of the partition tells the computer where the files' contents are stored. We're going to add a file "credit.txt", a text file that holds our credit-card number.

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  p a r t i t i o n 1 = b a - z z 0 0 0 0 0 0 0 0 0 0
b  f o r m a t = c a - z z 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  c r e d i t . t x t = d a - d s 0 0 0 0 0 0 0 0 0 0
d  1 2 3 4 - 3 2 1 2 - 3 4 5 6 - 5 4 3 2 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

The file and its location are now added (in green), and the contents (in orange) clearly visible from a simple scan of the disk.

Now this is where the problems start. We want to sell this drive, so we need to delete our credit card details from it. We delete the file, and this, we think, will delete the credit card details.

Right. . ?

Wrong. This is our filesystem after we delete the file:

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  p a r t i t i o n 1 = b a - z z 0 0 0 0 0 0 0 0 0 0
b  f o r m a t = c a - z z 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
d  1 2 3 4 - 3 2 1 2 - 3 4 5 6 - 5 4 3 2 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

The file's entry has been removed from row 'c' - The computer is presented with what it thinks is a blank disk. But the contents of the file are left untouched: Only row 'c' has been altered. The file has been logically deleted, because to the computer, the disk appears empty. But it has not been physically deleted: It's still there.

Perhaps, instead, we should have simply deleted the whole partition? Let's see what this would have achieved:

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
b  f o r m a t = c a - z z 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  c r e d i t . t x t = d a - d s 0 0 0 0 0 0 0 0 0 0
d  1 2 3 4 - 3 2 1 2 - 3 4 5 6 - 5 4 3 2 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Oh dear, this is even worse! The partition is gone, but all the information about the formatted filesystem and its contents are still there. It's very easy, with the data we've got, to simply re-create the partition table and restore all files within it. This makes it even easier for our malicious buyer to grab our credit card details!

The problem, in a nutshell, is that deleting never actually deletes the information. At best, it removes references to the information while leaving the information itself untouched.

In order to delete a file safely, what we really need to do is get at the actual contents. At this point, my bias starts to show through, because I think Linux users are considerably better off than Windows users here: Linux usually comes with a tool that does this very thing. It's called shred. If you're a Windows user, either get hold of a Linux LiveCD such as Knoppix, or look up a Windows-specific secure deletion program on Google. I'm going to continue by talking about shred, but the principles are the same whatever you use.

Shred and its brethen simply over-write file contents with random data. As an example, let's see what would happen if we shred credit.txt

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  p a r t i t i o n 1 = b a - z z 0 0 0 0 0 0 0 0 0 0
b  f o r m a t = c a - z z 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  c r e d i t . t x t = d a - d s 0 0 0 0 0 0 0 0 0 0
d  k 2 v @ ( j 5 Z £ ^ ! k a 8 * N 8 A , 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

That's better! The file is still there, but the contents are of no use to anybody. Shred learned from the green row, 'c', that credit.txt's data was located from 'da' to 'ds' and then wrote random data to that area of the disk. If we now delete the file as usual, we can be sure that this disk drive has no clue as to our credit card number.

But what if we had a file with our credit card details in it that we deleted several months ago? What if its contents are still there, somewhere?

The only way to make absolutely sure that no recoverable data is left on the disk at all is to shred the whole thing. This does what we really wanted to do right at the start: Removes absolutely everything from the disk. Because Windows locks the files that it is currently using, and all OSes tend to write to the disk from time to time, you can't do this from within a normal OS. You need to use something that can function independantly: Knoppix is really handy at this point! Do, of course, bear in mind that what you're doing here is permanently and irreversibly wiping a disk drive completely, so make sure you remove or at least unplug any drives that you don't want wiped! Accidents do happen. . .

From within Knoppix, you would open up a terminal and use fdisk -l to tell you what disk drives it can detect. It should show you at least two: The CD you booted from, and the drive you want to wipe.

The naming system is a bit arcane if you're used to Windows and "C:" and "D:" for the hard drive and CD-ROM, but it's simple enough to follow. All hard drive names start with "/dev" which simply means "device" - all the PC's hardware has a name beginning with "/dev". Typically, a hard drive will be "hd" if it's IDE, or "sd" if it's SATA. It will also have a letter following it: The first hard drive will be "a", the second "b", and so on.

So if you have a simple IDE hard drive, it will be called "/dev/hda". If you have a SATA drive with two partitions, the disk will be "/dev/sda" and the partitions will be "/dev/sda1" and "/dev/sda2"

Simple enough, once you get the hang of it.

So, if your hard drive is a standard IDE, it will be /dev/hda you want to erase, and you would issue the command shred /dev/hda and then go and find something else to do for a while, because this takes quite some time: There's a lot of data to write. By default, shred will overwrite the whole drive 25 times! If you have a 100GB disk, that means writing 2500GB of data. To just do it once, you would type shred -n 1 /dev/hda, but bear in mind that this is less secure.

Eventually, shred will leave you with a filesystem that looks like this:

   a b c d e f g h i j k l m n o p q r s t u v w x y z 

a  n # Y v C n $ } I / . ` b 0 J r n v 9 8 N % I : 3 ? 
b  = Y ` K c E b x x f W S p y \ g L l $ C ? ) , 8 k o 
c  O ! w | \ 7 2 v A i O I p w 5 v O k 1 \ I ` s T u a 
d  N g h j t y - 2 n c k m r 1 ( W 1 r . i < M _ L ' + 
e  @ } G L ^ ^ f ( t S = ] i ( D q ! r E 5 = K _ y 0 7 
f  % _ Z a o g I 2 . K v u O h D q q , A ` 2 0 E " g ? 
g  K | k g 6 A " j % S ? Z v a p t Z l x z < r P 3 D v 
h  > # n ( A e D * < _ [ N e x 7 i r T c a z f R t _ 3 
i  9 M i # / K m E Z & k M ; m | C b * - > , _ * f i d 
j  | ( \ i m c o 3 k H & 5 G ; Q + ] m M w M 0 ) J E ? 
k  u ! T M r c ; 7 ` w < F , M \ 9 } a q # C j 0 Z u < 
l  O I p A : , D H } \ q 5 O 9 x z : C t { b > O ` G ; 
m  m V [ M p ` U p @ i C v n ' , s P | t I U Y T , / n 
n  h # h n i a J I R y b S y 0 A I W r U C 4 o F # b X 
o  - E ^ \ Q [ l U I + # u v { Y ( U _ @ = o ) h J _ m 
p  ^ L n t J # A ; V . ] m ! ] c a _ { , " l m X \ o e 
q  % 6 n c g H x G 2 ^ , T ` " " / 0 > U X 8 % . 3 / 5 
r  ] f H f r h M ! c j W = 3 | I k | 6 J | X K f 3 T , 
s  Y A > U / 0 Z $ y . C n T + & L } K o M m h { | s x 
t  _ o p L ] y g > _ N B & H 4 ; Y 3 B - j T m F . F o 
u  Q ? / F C ! Z j 3 : t E 9 s a o } _ H " \ : q ] W # 
v  z ; w j W 2 : B * o P Q ! % 6 " 9 L m z I t r 8 _ + 
w  = l V { h n 9 I t Y A r f r L d V H C $ s g ! { s J 
x  L ] I r E + q b Q \ y B & Q 3 I # $ W b , y x V Y t 
y  f $ ^ ' c O } @ 5 B _ 5 \ w 0 N Q j ( b - I w & ( ? 
z  ^ . y \ " 2 F x ` V s # H 5 ; t ! } ! y 5 y ? e w # 

If you'd rather it was returned to the pristine block of zeros we started with, add -z to the command: shred -z /dev/hda and the final pass of shred will write zeros instead of random data. We thus end up with this:

   a b c d e f g h i j k l m n o p q r s t u v w x y z

a  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
b  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
c  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
e  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
g  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
h  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
i  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
j  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
k  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
l  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
m  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
n  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
o  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
p  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
q  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
r  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
s  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
t  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
u  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
v  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
w  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
x  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
y  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
z  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

And your disk is now about as safe as it can be, short of placing it in solvent and leaving it there until it dissolves. In theory, the data can still be recovered after multiple random over-writes, but you'd need very expensive forensic equipment to manage it: Not something the average ebayer is likely to have.


52 comments

Alison
Comment from: Alison [Visitor] · http://www.creativehedgehog.com
You've left out the explanation about why computers are so "unsecure" in this way.

It's because they are trying to be efficient. Instead of spending ages of time and processor power restoring the hard drive to zeros, it simply says: "we don't need this anymore, so that space is free for me to use however I want." (ie, putting new data in that spot.)

Nice article though!
05/01/07 @ 05:23
phani bhushan
Comment from: phani bhushan [Visitor]
can u plz tell me how one can recover the dats if the file is deleted!!!!
02/02/07 @ 04:47
Steve
Comment from: Steve [Visitor]
In the popular FAT-type filesystems, you are likely to see even LESS of the data deleted, much to the chagrin of the end-user. You're likely to see "Undelete Programs" (on sites like NoNags, for instance) which tell you that you can still undelete the file named "~redit.txt" as long as you type in the first letter... As though line C read; "~redit.txt=da-ds"
02/04/07 @ 00:48
Martin
Comment from: Martin [Visitor] Email
Linux is very secure at this. You can hardly recovery files that are deleted. (I don't know exactly why, maby you can write something? ;) )
13/07/07 @ 08:11
Devin
Comment from: Devin [Visitor] Email · http://devhen.wordpress.com
Great article! However, it leaves me wondering how in the hell computer forensics people can recover information even after it has been randomly overwritten several times. Or better yet, been replaced by zeros. What are these forensics experts looking at? How can you recover information from a drive that is truly empty (all zeros) ?!

I really like your articles and I think you do a great job of explaining technical things in a common sense way so I'm hoping maybe you can shed some light on this hard drive forensics stuff for me. ?

Thanks! -Devin
02/11/07 @ 00:29
UxiTuxi
Comment from: UxiTuxi [Visitor] Email
the testdisk founded on
http://www.cgsecurity.org/index.html?testdisk.html
is i.e. a grate (partition- and data-) recovery tool...
09/11/07 @ 13:56
HKV
Comment from: HKV [Visitor] Email
"How can you recover information from a drive that is truly empty (all zeros) ?!"

-> Even if you write zeros, there *are* still magnetic traces left on the disk, which are not readable by the read-write head of the harddrive but can be read through special tools that work directly on the plates of the disks.
For simple understanding, even if you erase something on a paper that was written by a pencil and is now compeletely invisible to naked eye, there still are tools *other than* the eye itself that can read what was written there. ;)
21/12/07 @ 20:04
Rick
Comment from: Rick [Visitor] Email
Glad I found this site! Your explanation is very much to the point.

In a related situation, I have an old "dead" hard drive on which I have a lot of data I would like to retrieve. It just got old fast and stopped booting. I had it replaced (still under warranty), but I was allowed to keep it in order to attempt to retrieve the data. However, the expense of having it done for me is out of the range of the value of the data.

I now wonder if it's either the drive partition data or the FAT/NTFS data that is just jumbled? The symptoms began with it refusing to boot only sporadically.

I will be back to your site when I have more time to search for more tips.

Thanks again!
17/01/08 @ 18:50
Tao
Comment from: Tao [Visitor] Email
Why not just download to a flopy "DSCRUB" it is free and does the same thing overwriting then rewriting your drive to all ZEROs any where from once (as I use in my service workshop to kill really bad viruses). An up to 35 times overwrite for top secrete government info, like I use on old Centrelink & Defence force PC's?
17/01/08 @ 23:07
Tao
Comment from: Tao [Visitor] Email
Also a note to Rick visitor of17/01/2008 @ 18.50 hours

Get your self a few copy's of really good PC Mags, I use them all the time to pick up new tests and useful programes. I get APC here in Australia and the free DVD it comes with has given me several retreval and boot tools. (Note nothing will ever get back info from a "DSCRUBed" drive)as this programe was developed in co-operation with the USA and British goverments to provide "Safe" PC Distruction....Tao
17/01/08 @ 23:13
Sayyan
Comment from: Sayyan [Visitor] Email
Tao, it's still quite possible for a forensic laboratory to restore data that has been "DSCRUBed", they use several different tools, one of which has a read/write head that is incredibly more accurate than a standard hard drive one.

When you write to the hard drive, tiny innacruacies of the write head cause it to not overwrite data, but rather leave concentric rings of data. Using the extremely precise read head, they are able to read all the different operations that the head has made.

If you want to protect your data from a well funded forensic labarotary, your only really option is to melt the disk.
21/01/08 @ 10:22
Dislexsick
Comment from: Dislexsick [Visitor] Email
note @ Tao. That method is still insecure. The only way to truly get rid of the data is physically destroying the HDD [The polide supposedly put a drill straight through all the plates].

If people are buying the PC used, then a 40GB HDD should do, and that would only cost them $30 or so nowadays.
21/01/08 @ 11:47
Miamiguy01
Comment from: Miamiguy01 [Visitor] Email
I have nothing to add, substantively. I have nowhere near the level of expertise reflected in the meaty comments of the other guests.

Perhaps that is why I have the "outside" perspective to say "Brilliant!" You've made the maddeningly obscure comprehensible to me, and that's important. After all, the problem of "scattered data" is no mere academic inquiry, these days. Thank you, thank you, for taking the time.

God bless.
22/01/08 @ 07:38
Michael
Comment from: Michael [Visitor] Email
What is the best program, or you recommend, to use for securely deleting files on Windows XP or vista? Can you delete individual files or do you have to delete the whole hard drive?

If you place a strong magnet on a drive, will that destroy a drive enough that it cant be recovered?

Thanks
25/01/08 @ 14:49
oneandoneis2
Comment from: oneandoneis2 [Member] · http://geekblog.oneandoneis2.org/
http://portableapps.com/apps/utilities/eraser_portable is pretty good - install on a USB drive and take it wherever you go..
26/01/08 @ 18:42
chaosjim
Comment from: chaosjim [Visitor] Email
I have read that Evidence Eliminator is the best utility for deleting sensitive data from drives,does anyone know what it does that makes it so good?
28/01/08 @ 23:25
Jach
Comment from: Jach [Visitor] Email · http://www.nincheats.net
Wow, second time I've stumbled upon this. Nice one, I've known about shred for a while. But in reality your data is never completely gone from the Feds unless you pour acid all over your drive and use magnets and other crap to completely destroy it.

So don't think shred will save you when the FBI come knocking. =P
29/01/08 @ 01:57
Aystin C
Comment from: Aystin C [Visitor]
i just wanna know why everyone wants to completely shit can their disk? lol
29/01/08 @ 05:09
Big Poppa
Comment from: Big Poppa [Visitor] Email
I want to destroy my hard drive because my porn collection is such a complete work of art that I'd be worried if it got out more men would enjoy the complete sense of well-being and relaxation that I've achieved resulting in the loss of my competitive advantage.
30/01/08 @ 01:22
PGP is free for individuals and offers whole disk encryption, the ability to create hidden, secure drives for your porn & private pics, and the ability to send encrypted zip files and emails.

It is uncrackable and transparent when using.

Also, it has a desktop shredder for securely deleting files, drag and drop.

And finally, it can scramble the unused spaces on a current drive (shreds everything previously deleted) while XP is running. The program can create a drive that is totally scrambled at all times with whole disk encryption.

No one, even if I allow a Fed techie to have my computer to study, can retrieve the data I don't want them to. The physical recovery of data does not matter because the data is useless without the password key.

Without the secret drive mounted, there are no pictures, porn, bank info, etc available to find even while the tech guys are working on your computer.
A search for pictures, video files comes up empty, but in 15 sec, I can mount the drive and access them like they were there all the time.

TaDa...
31/01/08 @ 01:54
aninnymouse
Comment from: aninnymouse [Visitor] Email
People always point to encryption as a security solution, but as I understand it, with encryption and encryption programs you're really just trading one form of insecurity for another.

Sure, you can encrypt your entire partition or folder or whatever, but how would you access it? You'd need the same program to unencrypt the data. How would that program know who you are, that you are someone authorized to see the unencrypted data? Usually with a password. So, what do you do if you, as a hacker, stumble upon an encrypted folder/partition and you want to break in? Just bruteforce the password using the original encryption program. Granted, some encrypters take steps to eliminate this, but not all do. Am I wrong in my understanding here
31/01/08 @ 22:10
Slokunshialgo
Comment from: Slokunshialgo [Visitor] Email
That always depends on the type of password being used, and how the encryption program decides to work it. Now, this is an example I just made up on the spot, but it might work, and may be used by some.

Say you enter a password, and it gets the MD5 hash of it to do its encryptions. The word "password" has a hash of 5f4dcc3b5aa765d61d8327deb882cf99. Ok, so, if you didn't know at all what the program did, this would be a good enough key for your encryption. However, as you said, the people trying to get it likely would have the program.

This is where the idea of password strength comes in. One of the only problems is that 2 strings could possibly have the same hash, as there is a limited number. But as we can see, there are 32 characters, from a-z, and 0-9, so 32 sets of 36 characters. 36^32 = 6.334028666e+49 possibilities, making it highly unlikely that anybody would come up with a random set of characters with the exact same hash as you did.

Now, as for brute-forcing it, running 2000 checks a second (which is reasonable with the average computer nowadays), it would take ~1.003566283e+39 years to go through every possibility. Not to say that we cannot hack it, as it could be one of the first few we try, but it is unlikely just guessing will work. This is where the strength of your password comes in.

If you're using any dictionary word, then people could get it with a simple "dictionary attack", which is to run through every word in a dictionary and see if any of them match. This can take some time, but works for a lot of things, because people can, and do, use simple passwords like this to keep themselves safe. When you get a strong password, however, it becomes less likely to crack. Such as h7d$jgk!3%f. it means nothing, but one could memorize it, and use it. The chances of a brute-force attack coming across this is downright nill, but still there.

So, I guess when it comes down to it, the only real security for encryption is how the key is generated, and how secure the password is. Also, keep in mind, that once somebody has the physical hardware themselves, there is nothing stopping them but their own knowledge and access to resources.
01/02/08 @ 15:22
Michael
Comment from: Michael [Visitor] Email · http://www.digitalgemstones.com
This is a great article, and I know the comments address it somewhat, but I think it could greatly benefit from you covering why random overwriting is not actually random, and why with enough effort the 'random' bytes can be reversed back to their original values. Thanks again for explaining it in such straightforward English.
07/02/08 @ 06:12
Michael
Comment from: Michael [Visitor] Email · http://www.zbuffer.com
For Windows XP Professional users, the cipher command may be used to wipe the free space on a drive. Just use the /w:<drivespec> switch, as in:
cipher /w:c

This command would wipe the free space on drive C.
09/02/08 @ 15:39
Alphanaut
Comment from: Alphanaut [Visitor] Email
Other tools than shred.

# cat /dev/urandom > /dev/hda1
Break w/ Ctrl+c

# dd if=/dev/urandom if=/dev/hda1

if u want zeros, use /dev/zero instead.
17/03/08 @ 11:34
kqrdeb
Comment from: kqrdeb [Visitor] Email
Wow! I just have to say I've read but your post on defragging and this one, and they're so good! You explain things I've always asked myself many times, and you do it in a way that I'm able to understand even though I'm tired or something...

Thanks many times! :):):)
19/03/08 @ 12:39
Mike Roquemore
Comment from: Mike Roquemore [Visitor] Email · http://mikeroquemore.com
I have the best solution to protecting my hard drive.

Thermite.

I have a hard drive sized amount of thermite above my hard drives. One turn of a key, and it will ignite taking the hard drives and bottom of the case with it.
27/03/08 @ 05:09
u24
Comment from: u24 [Visitor] · http://www.puremango.co.uk
Mike Roquemore: I think you've been reading "Stealing the Network" too much ;)
27/03/08 @ 15:39
thermite
Comment from: thermite [Visitor] Email
Mike, what do you use to ensure the thermite combusts. Chemical? Magnesium strips?
27/03/08 @ 17:30
caleb cushing
Comment from: caleb cushing [Visitor] Email · http://xenoterracide.blogspot.com
very good. this one could use a couple of additions though. I see one comment says it's harder to recover deleted info from a linux system. The opposite may be true if you don't know what you are doing. most linux file systems are journaling, which means that even if you shred the file you will have to take care of it's journal.

you perhaps should also have covered the concept of inodes, and other special file system markers. deleting doesn't delete the data. it just marks that space for overwriting.
28/03/08 @ 21:29
Indijan
Comment from: Indijan [Visitor] Email
You can "delete" the journal with:
tune2fs -O ^has_journal /dev/hdXX

But can you shred the journal?


11/05/08 @ 00:12
Jason SW
Comment from: Jason SW [Visitor]
Just wondering, but if they can detect the "magnetic traces" left on the disk, then what about just subjecting the whole thing to a powerful electromagnet, or something? :P
20/06/08 @ 03:15
Sasha
Comment from: Sasha [Visitor] Email
Why whipe the whole drive when you can just whipe the free space? I'm not sure if any existing applications do this but it seems much more efficient, and this way you don't have to move anything back and forth.

Good article, I learned a couple things. :)
05/07/08 @ 04:21
snowboarder13
Comment from: snowboarder13 [Visitor]
I have a correction to Alphanaut's post. The 'dd' command has an input file (if) and an output file (of). Thus, the command listed should be changed from:

# dd if=/dev/urandom if=/dev/hda1

to

# dd if=/dev/urandom of=/dev/hda1

I was wondering, though, does it really make much of a difference to wipe with, say, /dev/urandom before /dev/zero ? Either way, the disk ends up with 0's ...
06/09/08 @ 03:40
Mason
Comment from: Mason [Visitor]
If you are reselling or giving away your drive, simply overwriting with zeros is more than enough to keep anyone short of the government or a millionaire hacker from getting any of the data. Using shred is really only useful for the paranoid who don't know how to make thermite (and don't want to burn their house down). For anyone without a cleanroom and electron microscope, if the drive r/w head can't read it, it might as well not exist.
20/10/08 @ 08:41
Joel
Comment from: Joel [Visitor]
I recommend Darik's Book and Nuke boot CD for securely wiping hard disks. It's quick and effective.

URL: http://www.dban.org/
30/10/08 @ 22:53
Gond
Comment from: Gond [Visitor]
A couple of things: 1) Wiping your hard drive via a bulk-eraser (e.g. - an electromagnet) should be the best option short of total destruction BUT, more than likely, your hard drive would be unusable afterwards because the powerful magnet would probably destroy the read-write heads of the drive. 2) Regarding super-ultra-mega secure encryption: One caveat that should be noted is that if your data was EVER unencrypted on that same drive, it would be vulnerable to the forensic tools mentioned. For instance, you read this article's comments and say, "Hey! I'll just go ahead and encrypt my porn collection right now, just to be safe!" 2 weeks from now, you get busted, and much to your chagrin, they find evidence on your hard drive. Why? Because your files WERE unencrypted at one point, leaving a magnetic "shadow" behind AFTER encryption. This "shadow" is the same one that the forensic tools can pick up after deletion, overwrite, whatever.
G-
01/12/08 @ 17:55
Penguinific
Comment from: Penguinific [Visitor]
First of all, thanks to the author.
To Gonds' post, that would depend on the state of the unencrypted files, ie; were they unencrypted in ram/software, or did they ever reside on the actual drive in an unencrypted state.
The super powerful magnet idea is actually much trickier than most would think. First of all it is not a matter of simply placing it on the drive, you must pass alternating north and south fields over it. Also most hard drive cases would protect the actual platters inside from all but an obscenely powerfull electromagnet, and I am talking about something powerfull enough to pick up a locomotive here, but even with the platters exposed you would be surprised how strong a field it would take to do even a modest job.
For encryption, the real pros (FBI) after running a few dictionary attacks (languages) and a few other tricks, (assuming they know what encryption software was used, without which a key is useless) will then revert to attacking the actual encryption. For a very basic example lets say I've encrypted the word "shoulder". In the english language there are very common letter groupings such as "sh" or "ou' and "er" They can run pattern finding algorithms to start piecing it all back together.
03/12/08 @ 04:12
Penguinific
Comment from: Penguinific [Visitor]
P.S. To the vast majority of users, shred at 25 overwrites is overkill, why complicate things?
03/12/08 @ 04:21
David
Comment from: David [Visitor]
I have just read the manual for "shred" on my Debian 4 system. Apparently it fails if used on an ext3 file system with journaling. Shame! This needs to be sorted.
03/12/08 @ 22:19
Travis
Comment from: Travis [Visitor]
The surest way to erase any disk is to drill a hole through the casing, through the platters, and then heat the disk to a temperature greater the the Curie temperature for the disk material. The Curie temp for iron is ~770C, nickel ~360C, and cobalt ~1130C. Above those respective temperatures, the magnetic domains are perfectly randomized via thermal entropy throughout the disk, making it unreadable by ANY means, including Scanning/Tunneling EM, rendering the disk completely inoperable and illegible.
17/02/09 @ 22:06
Anon
Comment from: Anon [Visitor]
Errr, i used a screwdriver to open up the drive and then heavily scratched the platters surface and also used a pair of tweazers to remove the thread on the read/write head. I think this did a pretty good job of making all the data pretty hard to get at and possibly i could have done slightly more with a small magnet and a pair of even quite a blunt old pair of paper scissors.

I didn't go that far because the objective i was aiming for was to recover the drive and make it usable again. Opening the case was kinda ok but it rapidly and unexpectedly went downhill fast from there :( Luckily i didn't actually need the data, i just wanted the extra drive space :)
16/04/09 @ 11:23
Daniel
Comment from: Daniel [Visitor]
To those who say their cryptography will protect them:

http://xkcd.com/538/
20/04/09 @ 00:15
freebirth one
Comment from: freebirth one [Visitor]
Well, considering this site:

http://www.heise.de/newsticker/Sicheres-Loeschen-Einmal-ueberschreiben-genuegt--/meldung/121855

overwriting the whole drive once is enough.

Sorry for this being a German articel; I didn't found an English pendant
06/07/09 @ 01:06
Catkin
Comment from: Catkin [Visitor]
YO! DANGER!!!

These commands would delete the whole file system, not just make deleted files un-recoverable!

# dd if=/dev/urandom of=/dev/hda1
# dd if=/dev/zero of=/dev/hda1

These commands make deleted files un-recoverable without affecting the other files.

# dd if=/dev/urandom of=[path of a dummy file on the file system]
# dd if=/dev/zero of=[path of a dummy file on the file system]

When the fie system fills up, delete [path of a dummy file on the file system]. Best to stop processes that get upset when the file system fills up before running the dd command(s).
29/07/09 @ 18:27
Daniel Werner
Comment from: Daniel Werner [Visitor] Email
»In theory, the data can still be recovered after multiple random over-writes, but you'd need very expensive forensic equipment to manage it«

This decade-old claim has apparently been refuted:
http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html

Quoting from the Heise article:
»a single bit whose precise location is known can in fact be correctly reconstructed with 56 per cent probability (in one of the quoted examples). To recover a byte, however, correct head positioning would have to be precisely repeated eight times, and the probability of that is only 0.97 per cent. Recovering anything beyond a single byte is even less likely.«

Sounds reasonable to me.
11/01/10 @ 20:39
poxy rebel
Comment from: poxy rebel [Visitor]
Well, as someone who has actually HAD their hard drive seized by the authorities I can comment with some knowledge rather than theorising.
Its to the advantage of FBI and assorted friends to convince everyone they have godlike powers of forensic analysis.
I was actually told upon siezure that the lab would recover every single key that had ever been pressed on the machine.
Yeah right what a pile of BS. I made 4 passes on sensitive items with the pgp wipe tool and 3 passes for free space.
So did they recover anything? In short, no. I actually recieved access to the forensic report on my hard drive. It contained such heartwarming phrases (heartwarming to me) like "data is no longer present on the machine" and "it is impossible to ascertain the nature of the data".
In other words they can't get it for all their posturing, the only reason they knew those files existed was from email records that I had kept on purpose.
One caveat though, fragments of data were recovered from unallocated space, which is not the same as free space on a partition.
So to ensure security use boot time partition tools to take care of unallocated space that might be storing who knows what.
03/04/10 @ 07:36
metal roof
Comment from: metal roof [Visitor] · http://www.bratexusa.com/
Aw, this was a really quality post. In theory I'd like to write like this also - taking time and real effort to make a good article... but what can I say... I procrastinate alot and never seem to get anything done... Regards
30/04/10 @ 13:25
cara meninggikan badan
Comment from: cara meninggikan badan [Visitor]
If you want to sell hardware and you're not sure how some people can recover data from supposedly-erased hard drives, this article is for you.
01/06/10 @ 03:26
tinggi badan
Comment from: tinggi badan [Visitor] · http://peninggi-badan.com
or flash memory card off ebay, and then writing a story about how they were able to restore all the files that the previous owner had tried to erase prior to selling.
02/06/10 @ 03:42
kurt don joe
Comment from: kurt don joe [Visitor]
Thanks for a super post, and some very nice comments, now I have one question for 'poxy rebel': what OS and file system did you use, before the FBI impounded your system?

ext3/4 NTFS or was it Fat32 ? I'm just a curious paranoid Joe Don
19/02/11 @ 11:35
demian0
Comment from: demian0 [Visitor]
so i assume about 10 shreds will make any forensic machine useless :)

nice post, im on my way subscribing :)
12/11/11 @ 21:41
 

[Links][icon] My links

[Icon][Icon]About Me

[Icon][Icon]About this blog

[Icon][Icon]My /. profile

[Icon][Icon]My Wishlist

[Icon]MyCommerce

[FSF Associate Member]


May 2017
Mon Tue Wed Thu Fri Sat Sun
 << <   > >>
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31        

Search

User tools

XML Feeds

eXTReMe Tracker

Valid XHTML 1.0 Transitional

Valid CSS!

[Valid RSS feed]

powered by b2evolution