Updated hint

[linux_from_scratch_hints.git] / backing-up-lfs.txt

AUTHOR:         Mike McCarty    Mike dot McCarty at sbcglobal dot net

DATE:           2010-03-04

LICENSE:        GNU Free Documentation License as of 2010-03-04.

SYNOPSIS:       How to and what to back up in an LFS system.

PRIMARY URI:    N/A

DESCRIPTION:    This hint is intended for those who need help
                planning a backup system for LFS.

ATTACHMENTS:    N/A

PREREQUISITES:  Have a bootable LFS system and external removable medium
                for storage of the backup.

HINT:

Ok, you've built your shiny new LFS system, and it boots. The book
suggests that you back up your system, and then proceed to build
whatever portions of BLFS appeal to you. Unfortunately, the book doesn't
tell you what you need to backup, and how to do it.

The first thing to take note of is that there are more than one thing
called a "backup", and that they differ significantly from each other,
and in what they are good for.

One type of backup, sometimes called "bare metal" or "disaster
recovery", is one which one can simply put in a CD-ROM, boot, answer a
few prompts, and after some amount of time and CD-ROM swapping have the
system back in the state it was in. This hint does not address that type
of backup.

Another type of backup, sometimes called "full" stores all the
information which is needed in order to start from a freshly installed
system, and then restore the system back to the state it was previously
in. For an LFS system, in which "install" is synonymous with "build",
this type of backup does not make much sense.

Another type of backup, sometimes called "incremental", "differential",
or similar terms, allows one, in conjunction with a "full" backup, to
bring a freshly installed system back into a previous state, by first
restoring from a "full" backup, and then applying changes from the
incremental or differential backup. Again, since "fresh install" is
synonymous with "build", this type of backup is not quite applicable to
LFS, and is also not covered in this hint.

Ok, so what is covered?

This hint describes backups which allow you to recover from accidental
deletion of files, and also do disaster recovery from bare metal, but do
not themselves contain any means to boot. Both a "full" and
"incremental" or "differential" style of backup can be created, but a
separate boot medium must be supplied by the LFS user. The hint covers
what needs to be backed up, and how to create a true "snapshot" which
represents the system state at a single point in time. It does not cover
the style of backup medium nor how to secure the backups themselves,
though suggestions are made.

The first consideration in any backup, is that it actually represents a
single moment in time. Making the backup must be an atomic action, that
is, one which appears to have taken place in an instant. The only way
absolutely to guarantee this, is for all file systems to be in a
consistent state, and for them not to be modified for the duration of
the backup. More than one technique for accomplishing that have been
suggested, but the simplest to achieve with an LFS system is simply to
reboot to single user mode. How this is accomplished depends upon what
you chose as your boot method, so the details of that can't be covered
in full here. However, if you use GRUB, as suggested in the LFS book,
then you can create an entry for single user mode by adding an entry
to /boot/grub/menu.lst similar to this:

        # LFS 6.5 Single User Mode for Rescue and Backups
        title LFS 6.5 Single User Mode
        root (hd0,3)
        kernel /boot/lfskernel-2.6.30.2 root=/dev/hda4 single

It is not adequate to boot your system and then use

        # init 0

since that leaves you still logged in as yourself, and you won't be able
to unmount your file systems. You also need to make sure that root can
log in, and has a good password.

Reboot, and select the single user mode entry, and when prompted for the
root password, enter it. You should then be given a single user mode
maintenance prompt.

Now that you are rebooted to single user mode, you need to unmount all
file systems, except the root file system. So, for each file system you
have mounted, simply do an unmount. Obviously, you can't unmount your
root file system /, so it must be remounted read only. To find out what
you have mounted, simply use the mount command. I'll give the example
here of what happens when you have /dev/hda1 for /boot, /dev/hda2 for /,
and /dev/hda5 for /home mounted. You'll need to adapt your procedure to
your setup.

        # mount
/dev/hda2 on / type ext3 (rw)
/dev/hda1 on /boot type ext3 (rw)
/dev/hda5 on /home type ext3 (rw)

I've omitted the entries for /proc, and others, since these are not true
file systems. You want to make note of each file system which
corresponds to one of your hard disc partitions.

Now, for each file system you intend to back up, except for /, which you
have mounted, issue the umount command.

        # umount /boot
        # umount /home

Finally, remount the root file system read only

        # umount -n -o remount,ro /

You need the -n so umount doesn't attempt to modify /etc/mtab after the
file system is marked read only.

At this point, we have ensured that the file systems won't be modified
by any background stuff going on, but we don't yet know that the file
systems are in a consistent state. For that, we need to run fsck on each
of the file systems. So, going through the list you got from mount
earlier, run fsck on each file system. Still using the example above

        # fsck -c /dev/hda1
        # fsck -c /dev/hda2
        # fsck -c /dev/hda5

If you have other file systems which you sometimes mount, but which are
not mounted automatically at boot, then use fsck on them as well. Be
prepared for the fscks to take some time.

If all the checks pass, then you are ready to begin your backup. You
need to mount all your file systems read only, except for the one to
receive the actual backup. It's common to put the backups in
/var/backups. In that case, you need to mark the file system containing
it as read write, and all others as read only. It's common, however, for
/ not to be large enough to contain the actual backups, so /var or
/var/backups to be on a separate partition. In that case, you can leave
/ mounted read only. This example shows

        # mount -n -o remount,rw /
        # mount -o ro /dev/hda1 /boot
        # mount -o ro /dev/hda5 /home

If you are not marking / as rw, and putting the backup in
/home/var/backups, for example, you would use

        # mount -n -o ro /dev/hda1 /boot
        # mount -n /dev/hda5 /home

In this case, you need the -n to prevent attempting to modify /etc/mtab
when it is on a read only file system.

Now mount any other file systems which are not automatically mounted at
boot, and which you intend to back up, but be sure to mount them read
only.

At this point, you are ready to begin the backup procedure. Well, the
obvious question is, what needs to be in the backup? The answer,
unfortunately, is "It depends." Partly, it depends on how you set your
system up, and it also depends on what you want on each backup set you
make. However, for guidance to those who are relatively new to UNIX
system's administration, here is a list of what you'll typically see on
a freshly built LFS system, what is in each piece, and suggestions to
help you determine whether you want to back it up.

/bin            this is the main program directory, and should be
                included in all backups which intend to back up the
                system itself, but not necessarily on data only backups.

/boot           this is usually a collection of kernels, boot
                configuration files, and the like, and should be
                included in all backups which intend to back up the
                system itself, but not necessarily on data only backups.

/dev            this is the devices directory, usually managed by udev
                on LFS systems, and should not be included in backups.

/etc            this is necessary configuration files and scripts and
                should be included in all backups which intend to back
                up the system itself, but not necessarily on data only
                backups.

/home           this is user information and data, and is not needed to
                back up the system itself, but would be included in all
                data only backups.

/lib            this is various library files, and should be included in
                all all backups which intend to back up the system
                itself, but not necessarily on data only backups.

/lost+found     this is a file system maintenance directory, and should
                not be included in any backups.

/media          this is used to hold mount points for various removable
                medium drives, like USB disc drives, floppy discs,
                CD-ROM drives and the like. Unless you are creating a
                backup which is intended to contain a file system which
                is mounted on, say, /media/my-usb-disc, then it should
                not be included in any backups.

/mnt            this is used as a temporary mount point by root, and
                normally should not be included in any backups.

/opt            this contains add on packages, and should normally be
                included in backups which are intended to back up the
                system add ons from, for example, BLFS, unless you put
                your add ons into /usr/local, or the like.

/proc           this is not a real file system, but rather a view port
                into the kernel's view of the system, and should not be
                included in any backups.

/root           this is normally root's home directory, and should be
                included in all backups which intend to back up the
                system itself, but not necessarily on data only backups.
                This directory should be pretty spare.

/sbin           this is the main maintenance program directory, and
                should be included in all backups which intend to back
                up the system itself, but not necessarily on data only
                backups.

/sys            this is not a real file system, but rather a view port
                into the kernel's view of the system, and should not be
                included in any backups.

/tmp            this is a temporary directory, and should not be
                included in any backups.

/tools          this is the main LFS build tools directory, and is not
                needed to back up the system itself, but would be
                included in all data only backups.

/usr            this is user information and data, and is not needed to
                back up the system itself, but would be included in all
                data only backups. In fact, you might not have anything
                in it at all, if you put all your add ons into /opt. You
                may have only man pages in it.

/var            this is user information and data, and is not needed to
                back up the system itself, but would be included in all
                data only backups. In fact, you might not have anything
                in it at all, if you put all your add ons into /opt.
                Note that, if you put you backup into /var/backups, then
                you need to be careful not to include /var/backups in
                the backup set.

Ok, so now you've rebooted, and you've decided what to put into your
backup set. How do you create your backup set? Well, that depends on
what you like. I like to use tar, and use it to create backups. One tool
which I've found very useful is yakup, which has very nice abilities to
split the archive into CD-ROM or DVD-ROM sized pieces, and manage
inserting and swapping media etc. That can be done after the backup is
made, and with the system on line. Others may prefer to use cpio style
backups. It's up to you how you chose to build your backup set. I highly
recommend yakup, but that's just my personal preference.

At this point, you have your backup set made, and are ready to reboot to
normal user mode, after which you write your backup to removable media,
unless it already is on one, like a USB disc. If you have your root file
system mounted read only, then you need to remount it, so it can be
marked "clean" when you go down. Otherwise, skip the first command
below.

        # mount -o remount,rw /
        # reboot

What you do now depends heavily on what other software you have
installed. You need some way to get the backup set off your machine, and
save it away, preferably at a remote location. I like to use DVD-ROMs
for my backups, and so I have growisofs and cdrecord installed. How to
do that is beyond the scope of this hint.

One other thing you need to do is make a backup of the scripts or
programs you use to restore files. If you simply use a tarball which
fits on one DVD-ROM or CD-ROM, then this may not be necessary, as you'll
need a boot disc, and it should contain tar. If you use a script like
yakup, then one of your backup discs should have that script as one of
the files on it, not in the tarball.

Ok, disaster has struck! How do you get your information back on your
disc? If all that has happened is you accidentally deleted a file, then
simply pull that one file back from your backup set, using whatever
means is appropriate for the format. This may be as simple as putting a
DVD-ROM into the drive, issuing a mount command, and running tar. I
recommend testing every backup set you make by restoring at least one
file to another location, and ensuring that it contains the same data as
the file it supposedly backs up.

How about if, as root, you fat fingered

        # rm -rf /fred

and typed instead

        # rm -rf / fred

and your root directory is gone?

You need to get a bootable CD-ROM with some version of Linux on it. I
like to use KNOPPIX. Others like sysrescuecd. Almost anything will work.
Boot your rescue medium, insert your backup medium, like DVD, into the
drive, and mount it. This may require using two DVD drives, and is
something you should think about before disaster strikes. Another
possibility is to use something like Feather Linux, or Puppy Linux,
which can load entirely into RAM, and free up the drive. Depending upon
how serious the disaster is, you may need to repartition your disc, and
create file systems. How to do that is highly dependent upon how you set
your system up, and some of the details are in the LFS book. When you
have your disc set up where it is ready to accept the restored files,
then copy the restore scripts you intend to use, and use them to pull
all files back from the backup. Recreate any directories you have on
your system which are not on the backup proper, but are needed for
boot. This is usually just /dev, which you don't back up, but udev
needs in order to get set up. You then need to reinstall and set up
your boot block, like GRUB, per its needs. At this point, you should
have a bootable system.

I highly recommend doing at least one simulated disaster recovery, using
the rescue boot medium you hope never to have to use later, and verify
that you can recover at least one file from your backup. A real test
would require putting a blank hard drive in and recreating your system.
Doing that is very instructive, but also requires some outlay in money.
I highly recommend it. Don't be learning how to do disaster recovery
while trying to recover from a disaster.

ACKNOWLEDGMENTS:        N/A

CHANGELOG:

Include timestamped changes that have occurred in the hint. Add latest
entries at the end. Entries in this section should be as follows:

    * 2010-03-08
    *  * slightly obfuscate e-mail address
    *  * fix error in mount of /dev/hda1 to indicate read only
    *  * add information about recreating some necessary directories
    *    which may not be present on the backup, but necessary for boot