File Manipulation

From Slackadelic Wiki

Mass Alter File Permissions

We've all done it; fubared an entire directory of permissions. I blame microsoft backups.

In this example, we have user Tom, who needs to fix the permissions of his home directory, and wants them set to //read write execute// for himself, and //read only// for everyone else.

find /home/tom -type f -print0 |xargs -0 chmod 644
find /home/tom -type f -print0 |xargs -0 chown tom:users

To alter directories, change -type f to -type d

For further chmod usage, see the Slackbook Please see college admission essay

Remove Spaces From File Names

Typing spaces in CLI, particuarly those formed in Microsoft file systems, is a PITA, even with tab complete.

Use sed to fix it.

for file in *; do mv "$file"
'echo "$file |sed 's/\ //g'
';
done

Mass unzip/unrar files

Requires zip and/or unrar installed on your system. If you don't have them, see your $OS packaging system

for file in *.zip; do unzip $file; done

Convert DOS EOF to UNIX EOF

tr -d '\r'

WARNING: Be careful with the translate command (tr), you could easily mistype and nuke the file you are working with

CD Images

Requires mkisofs and cdrecord or dvdrw+tools to be installed. In this example, we are making a CD Image of Tom's home directory, called homeimage. The gzip command is used as a go-between to prevent errors.

mkisofs -r /home/tom |gzip > homeimage.iso.gz

To burn it to disc:

gzip -dc homeimage.iso.gz |cdrecord dev=0,0,0

In the above example dev is the location of your cd burning device. Your computer may be different, but for most people, 0,0,0 will be the default location. See cdrecord's MANPAGE for more information

The DD Utility

Like nmap, dd is the swiss army of file manipulation tools:

dd can blow away harddrives, using a reasonably secure method:

dd if=/dev/zero of=/dev/XdX bs=1M

WARNING: This will nuke everything on that drive, even the file system. You have been warned

dd can easily copy an iso from a cd (assuming it is not protected):

    dd if=/dev/cdrom of=/home/tom/isoimage.iso

dd can recover a dying hard drive (as long as it spins, you can recover the infomration):

   dd bs=512 if=/dev/XdX of=/some device/dir conv=noerror,sync

Keep in mind, this makes a bit for bit copy of the drive, and fills in errors with a null or zero. In other words, your destination directory needs to be big enough to handle the entire hard drive.

WARNING: using dd in this manner can further destroy a failing hard disk

DD as a Hard Drive Image Utility

Using the above command, we can extend this into making Hard Drive images. Who needs Norton Ghost when you have dd?

We will do this in 3 steps; copy the image (in this case, an external HD /dev/sdc), copy the partition information, then illustrate a command that will restore the image:

     dd if=/dev/sda conv=sync,noerror bs=64K | gzip -c  > /mnt/sdc1/sda.img.gz
    
  
     fdisk -l /dev/sda > /mnt/sda1/sdc_fdisk.info
     

     gunzip -c /mnt/sdc1/sda.img.gz | dd of=/dev/sda conv=sync,noerror bs=64K 

You might also be wanting the MBR:

     dd if=/dev/sda of=/dev/sdc  bs=446 count=1
    

NOTE: DDRescue was meant for this sort of operation, and is much more flexible about it than regular dd

WARNING: DO NOT try a dd copy/restore with a set of raided harddrives, without removing the partitions on the slave drives beforehand

WARNING: BE CAREFUL! One ill thought out command or mistype could hose your system. I use dd as an emergency, there are better tools around for when you were smart enough to plan ahead

Searching For Text Inside Files

If you are wanting to prove someone said something in an IM conversation, or just can't find that report you typed up for the boss, there is no need for a fancy desktop search system:

   find -type f -maxdepth 1 |xargs grep -E "search-string"

NOTE: The -maxdepth switch tells find not to dig more than one directory deep. Removing it will cause find to transverse the entire directory tree

For programmers who just have too much to do than organize their files:

   find -name "*.[ch]" |xargs grep -E "search-string"