Backup and Restore the complete file system

[eljohnsmith]

Can someone tell me the commands to backup and restore the complete file system. I have an installation on a 1GB drive and I am ready to move to a bigger drive but I have spent so many hours configuring this that it is just terrifying to think about starting from scratch. I would like to use this command as well to backup to another hard drive in case of a catastrophe.

Thanks,

John Smith.

[slycat]

For a one-time deal, you could just dd the boot drive to the new drive but only on a non-running instance I believe. E.g.: plug the original and new drive to a linux machine (possibly windoze with cygwin), fdisk -l to find their mount points, then
$this->bbcode_second_pass_code('', 'dd if=/dev/<original> of=/dev/<new>')

However, for actual backups without needing another machine or using a non-running instance, I use this.
$this->bbcode_second_pass_code('', 'tar -cvpzf POGO`date +%m%d%y`.tgz --exclude=/proc --exclude=/mnt --exclude=/sys --exclude=/dev --exclude=/lost+found --exclude=/media /media/Storage/Backups /')
For fear of HDD catastrophe, you would of course backup to a new drive. Your modified rootfs should be compressed to <700Mb if you keep your pacman clean, so you can use your 1gb flashdrive to hold 1 backup after you migrate to the HDD. I use a 4gb to hold the latest 5-6 backups.

When you need to extract it to a new drive you can from the same instance, but if the boot drive is the problem, slap it into a linux (cygwin?) machine and I run a simple script:
$this->bbcode_second_pass_code('', '# After formatting the destination drive
# AS ROOT! NOT "sudo". Use "sudo su" if you have to.
cd /<path>/<to>/<backup dir>/
tar -xvzpf POGO######.tgz -C /<path>/<to>/<destination drive>/
cd /<path>/<to>/<destination drive>/
mkdir media
mkdir mnt
mkdir proc
mkdir dev
mkdir sys')

Coincidentally I had done this last night . Everything migrates perfectly.

Since you are switching to a new drive, though, there may come a problem with how quickly the drive comes up (causing the plug to not boot). Chances are low, but if you give this a shot and the plug isn't running, there is still a fix to this. I am not sure exactly as I have never run into this issue, but in the event of this happening, search the forums about "boot delay" or "blparam".

Hope that's your ticket!

[eljohnsmith]

That is exactly what I was looking for. Quick, simple, and painless. Thanks a lot. One thing I had to change was the parameter at the end. It seems that the last parameter is supposed to be the directory being backed up. Here is the final product in my case, hoping it can help someone else. I also added a bit of code to automatically delete the older backup sets. I am hoping that I can set this to run weekly and forget about it. I will try to keep 5 weeks worth of backups in the USB HD connected to the pogo plug. I apologize in advance for any unconventional stuff in the script. I am not an expert on shell scripting, I just know how to copypasta.

$this->bbcode_second_pass_code('', '
#! /bin/bash
# script to create a tar backup file
# of the pogo plug device

# directory to place backups
BACKUPDIR=/media/mybook/backup/POGO

# file name YYYY-MM-DDHH:MM:SS.tgz
BACKUPNAME=`date +%F%T`.tgz

# remove dashes and colons from file name
BACKUPNAME=${BACKUPNAME//[-|:]/}

# root directory for the backup
SOURCE=/

# number of sets to keep
NUMSETS=5

# perform the backup
tar -cvpzf $BACKUPDIR/$BACKUPNAME \
--exclude=/proc --exclude=/mnt --exclude=/sys \
--exclude=/dev --exclude=/lost+found \
--exclude=/media $SOURCE

# delete older files greater than $NUMSET
cd $BACKUPDIR
LINS=$((NUMSETS + 1))
if [[ $(ls $BACKUPDIR | wc -l) > $NUMSETS ]]
then
stat -c "%Y %n" * | sort -rn | tail -n +"$LINS" | \
cut -d ' ' -f 1 --complement | xargs -d '\n' rm
fi

')

The restore piece works very well, I tested it and it is nice. Thanks a lot for your answer slycat.

[slycat]

My mistake, you are right. I copied the terminal window from putty and I guess I forgot that last "/" glad you caught it though.

My script is neanderthal-style, hah! I haven't mastered the art of elegant (ba)sh scripting. I had it added to crontab to perform every other week (5am on Sundays). But I never had an auto-erase script. I hope you don't mind if I take that bit for myself.

Cheers!

[eljohnsmith]

I don't mind at all. I am glad someone else can use it. Thanks.

[minmao]

Sorry for noob question...I run the code starting with "tar" to backup my system drive, but nothing happend.
What did I do wrong? Thanks.

[pentolino]

Another noob question

What's the correct way of restoring? Is it:
- start with a fresh installation
- tar -xvzpf POGO######.tgz -C /<path>/<to>/<destination drive>/

or is there a way to produce a tar archive just like the one you download for initial installation?
Both way would be fine for me, I'm just thinking that the second one would be faster.

Thank you anyway.

[slycat]

Restoring from a backup tar would be the same procedure as restoring with the downloaded clean rootfs (except you don't have to do the mac_addr part, for pogos, as it is already there from your backup). I am not sure what device you are using but the best way would be to

Attached desired drive where rootfs will be on a linux machine (not plug)
Format said drive to ext3(PPPro)/ext2(v1/dockstar) (unsure of fs for other systems, check before you do this)
cd to where your backup tar is
tar -xvzpf <backup_file.tgz> -C </path/to/rootfs/>

To first restore to a clean rootfs, boot, and then force replace folders/files may cause some hiccups since you are basically editing all the files in a running instance. I wouldn't attempt it, but if you feel this possible method better suits you, try it but when you know you have a good backup. Don't blame me if you get a headache afterwards

[pentolino]

I'm sorry I wasn't completely clear about my point and goal: I don't have the need to do a complete online backup, I just plan to do a full backup every once in a while (every month or so), simply stopping my dockstar and plugging the disk to a different PC to take a snapshot of it; I don't wanna use dd because I want to exclude some specific directories (which contain a lot of data I am not interested in backing it up).
My goal is just to have a quick recover path in case my hard disk crashes (maybe I am unlucky, but I had 2 hard disk and 2 flash drive failing to various degrees in the last 2 years), with my complete standard setup (netatalk, falexget, samba, dnsmasq, deluge and a few others); I just want to "tar -xvzf" something on a shiny new disk and then be completely running in minutes, not a few hours like I had to spend a few days ago (it takes time to install and configure everything, also because of some small bugs in some package dependencies).

Of course I am already running regular daily backups of selected directories (mainly /etc and one user home where flexget and deluge config is)! It took me days overall to fine tune everything, I absolutely don't wanna loose it! That can be easily done
with rsync or tar <anything>.

Thank you for your reply anyway

[slycat]

To be honest, I don't know a quicker recovery path. If you use tar to backup your root partition (only the bits you want) and grab a shiny new disk, you'll just have to partition the new disk to what you want and extract the tar (from a linux machine since your plug would be inoperable without a root drive).

However, if you plug is still operational and you want to make a copy of YOUR rootfs sans a few directories, you could rsync and the --exclude flag. Basically from your plug you would have root (/) as source and /path/to/shinynewdrive/ as destination but only after you format it, of course. So it may take time, but the process would be: attach shiny, partition, mkfs, mount, rsync, shutdown, remove old disk, set shiny to boot (usually just plug it into the old drive's slot), boot.
I don't really use rsync, but this may help.

If speed is real important, use pigz to make backups (in case of failures, not to replace the above method). You can use --exclude= to skip folders/files
$this->bbcode_second_pass_code('', '# From plug itself
cd /path/to/backup/storage/
tar -cvpf - --exclude=/media/* --exclude=/proc/* --exclude=/lost+found --exclude=/sys/* --exclude=/dev/* --exclude=/mnt/* / | pigz -6 > backup.tgz
# Should only take less than 10minutes for 500MB (numbers from my PPPro)

# From linux machine
mkfs.ext3 /dev/shiny
mount /dev/<partition> /media/shiny
pigz -dc backup.tgz | tar xf - -C /media/shiny
# More labor, but no other way when the plug is inoperable')

Either way, manual intervention is needed, but it should (unless hiccups occur) be a matter of minutes, not hours.