Rsnapshot Server on Windows

[Migrated from tuzig.com]

2009 Aug 14 Update: Looks like rsnapshot is now packaged in cygwin! Thanks to pseudo-anonymous coward for the comment! Some of the information below may still be of interest, so it’s left unchanged.

Rsnapshot is a great tool for backing up data, keeping versions and using relatively minimal disk space. From the rsnapshot website:

“rsnapshot is a filesystem snapshot utility for making backups of local and remote systems.
Using rsync and hard links, it is possible to keep multiple, full backups instantly available. The disk space required is just a little more than the space of one full backup, plus incrementals.”

There are quite a few guides and howto’s on installing and running rsnapshot, including how to backup windows dekstops/servers. However, I couldn’t find any guide on installing rsnapshot server on windows. The rsnapshot website itself is pretty vague about it and it is not strictly mentioned as a supported operating system (The rsnapshot homepage currently states: “rsnapshot is written entirely in Perl. It should work on any reasonably modern UNIX compatible OS, including: Debian, Redhat, Fedora, SuSE, Gentoo, Slackware, FreeBSD, OpenBSD, NetBSD, Solaris, Mac OS X, and even IRIX.”

Why would we want to do that?
Well… good question.
Most people would probably be more interested in running rsnapshot on Unix. However, since rsnapshot works using pull mechanism, if you want to back up a Unix box to a Windows server using rsnapshot – you’d have to install rsnapshot on Windows.

Would it definitely run on Windows?
Even though I couldn’t find any concrete mention of it, there shouldn’t be a reason why not. It is written in perl, and using rsync and ssh – all supported under windows using Cygwin. I know us Windows people are generally confused when it comes to Cygwin stuff, but I promise you – it’s pretty easy. I can’t remember how long ago I first experimented with Cygwin, but it used to look like very scary stuff. Now it’s totally neat with a wizard interface to install anything you need (apart from rsnapshot), and you needn’t worry about too many command line switches… well… that’s arguable :)

Caveats
The only problem I’ve experienced was with filenames that are not compatible with Windows / NTFS. Files with colon (:), asterisk (*) etc in their name cannot be copied to the Windows box ! If you plan to do a full backup of a Unix box, it may be a show-stopper. However, for backing up most important files that should be good enough. I’m trying to look for some reasonable work-around, but couldn’t find anything solid at the moment

Update:
There’s a way to store unix files on windows with cygwin. This can be achieved with managed mounts. That means filenames will look a bit funny on windows, but that’s not a major issue because the main purpose for storing the files is for backup. For further information have a look in the cygwin FAQ

Step by step instructions
I’ll try to capture the most important steps:

Firstly, download and run cygwin. This is an online setup program that downloads only the components you require from the internet. You can run it any time after the installation to remove/update/add components.

Simply run setup, click Next, choose ‘Install from Internet’, choose the folder you want it under (I installed under D:\Dev\cygwin) with the default settings – i.e. Install for All Users and use the Unix / binary text file type. Select any local package directory (it’s only used to store temporary files, I pointed it to some tmp folder) and unless you’re connecting behind a proxy, use Direct Connection. Then choose a mirror, preferably somewhere you can identify as being nearby to where you are.

Choosing the right packages
This is the most important part of the cygwin install. Don’t worry though, if you miss something out you can always run setup again.

Install the following packages:
Admin->Cron and Cygrunsrv
Devel->make
Net->openssh (which automatically marks openssl for you) and rsync
Perl->perl

I also installed Editors->vim (or nano, or joe) – whichever unix editor you’re reasonably comfortable with. Nano is probably the easiest to understand for the Windows crowd. Of course you can edit any file in Notepad, but most unix files wouldn’t appear properly, so you may consider PSPad or some more robust text editor for windows.

click Next and it will download everything and install it for you. If it takes ages, you might have chosen the wrong mirror. Click Cancel (don’t worry, it won’t screw up anything) and go back and pick another mirror.
Click Finish

Downloading rsnapshot
Get the Generic UNIX (don’t worry, it WILL run on Windows – promise) rsnapshot-1.2.1.tar.gz file from the download page. Save it in your previously selected temp folder (I used D:\tmp). For some reason, when using Internet Explorer, it wanted to save the file as rsnapshot-1.2.1.tar.tar (I wonder why) – rename it to tar.gz if it does that

Launching Cygwin and installing rsnapshot
Click on the Cygwin icon on your desktop / Start menu. It will open a unix-like command shell (bash).
Change to the temporary folder. For those not familiar with Cygwin (myself included, until not too long ago), cygwin maps the windows C:, D:, …, Z: drives to a folder under /cygdrive. So to go to D:\tmp simply type:

$ cd /cygdrive/d/tmp

note the forward slashes and also that folders and files are case sensitive !

open the archive file using this command:

$ tar -zxf rsnapshot-1.2.1.tar.gz

This will put all files in a subfolder with a same name, without the tar.gz suffix so change to this folder

$ cd rsnapshot-1.2.1

Now I simply followed the installation instructions (follow carefully including all dots and slashes):

$ ./configure --with-perl=/usr/bin/perl --with-rsync=/usr/bin/rsync

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking whether make sets $(MAKE)... yes
checking for working aclocal... missing
checking for working autoconf... missing
checking for working automake... missing
checking for working autoheader... missing
checking for working makeinfo... found
checking whether make sets $(MAKE)... (cached) yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for cp... /usr/bin/cp
checking for rm... /usr/bin/rm
checking for ssh... /usr/bin/ssh
checking for logger... no
checking for du... /usr/bin/du
configure: creating ./config.status
config.status: creating Makefile
config.status: creating rsnapshot
config.status: creating rsnapshot.conf.default

Now type “make install” to install the program.

After rsnapshot is installed, don’t forget to copy
/usr/local/etc/rsnapshot.conf.default to /etc/rsnapshot.conf


$ ./configure --sysconfdir=/etc

$ make install
make[1]: Entering directory `/cygdrive/d/tmp/rsnapshot-1.2.1'
/bin/sh ./mkinstalldirs /usr/local/bin
/usr/bin/install -c rsnapshot /usr/local/bin/rsnapshot
/bin/sh ./mkinstalldirs /etc
/usr/bin/install -c -m 644 ./rsnapshot.conf.default /etc/rsnapshot.conf.default

make install-man1
make[2]: Entering directory `/cygdrive/d/tmp/rsnapshot-1.2.1'
/bin/sh ./mkinstalldirs /usr/local/man/man1
mkdir /usr/local/man
mkdir /usr/local/man/man1
/usr/bin/install -c -m 644 ./rsnapshot.1 /usr/local/man/man1/rsnapshot.1
make[2]: Leaving directory `/cygdrive/d/tmp/rsnapshot-1.2.1'
make[1]: Leaving directory `/cygdrive/d/tmp/rsnapshot-1.2.1'

setting up key-based SSH to remote Unix host
I won’t go into too much details. You can google for it and there are plenty of resources. The commands I used (on the Windows box) are:


$ cd ~
$ mkdir .ssh
$ cd .ssh
$ ssh-keygen -b 1024 -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/Administrator/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/Administrator/.ssh/id_rsa.
Your public key has been saved in /home/Administrator/.ssh/id_rsa.pub.
The key fingerprint is:
2f:c4:23:df:b6:99:aa:ee:b5:65:63:df:d1:d6:33:ce Administrator@server2003

$ scp id_rsa.pub user@remotehost.somewhere.com:/home/user/id_rsa.pub
Password:
id_rsa.pub 100% 234 0.2KB/s 00:00

$ ssh user@remotehost.somewhere.com
Password:
...
user@remotehost:~$ cd .ssh
user@remotehost:~/.ssh$ cat /home/user/id_rsa.pub >>authorized_keys
user@remotehost:~/.ssh$ cat /home/user/id_rsa.pub >>authorized_keys2
user@remotehost:~/.ssh$ exit
logout
Connection to remotehost.somewhere.com closed.

Now check that it actually works

$ ssh user@remotehost.somewhere.com

If it doesn’t ask for password then you’re all sorted. If it does, you’re on your own here…

Setting up the managed mount
This is done to allow full unix file names to be stored on the windows box. It needs to be done once from the command line, and as long as rsnapshot will be run from the same user account (Administrator), there’s no need to run it again


$ mkdir /.snapshots
$ mount -o managed d:/dev/cygwin/.snapshots /.snapshots

Setting the rsnapshot configuration file
I can’t go into too much details, but here are the important bits:


$ cd /etc
$ cp rsnapshot.conf.default rsnapshot.conf
$ nano rsnapshot.conf

And the important parts of rsnapshot.conf:

# All snapshots will be stored under this root directory.
snapshot_root /.snapshots/

# Leave cmd_cp commented out !
#cmd_cp /usr/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.
cmd_rm /usr/bin/rm

# rsync must be enabled for anything to work.
cmd_rsync /usr/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.
cmd_ssh /usr/bin/ssh

# Comment this out to disable syslog support.
#cmd_logger /path/to/logger

# Uncomment this to specify a path to "du" for disk usage checks.
cmd_du /usr/bin/du

backup user@remotehost.somewhere.com:/etc/ remotehost_backup_folder/

Note that rsnapshot is extremely sensitive about spaces and tabs. Tabs separate the fields, so pay attention and avoid using spaces in the wrong place.

That should be it. Try it out using:

$ rsnapshot configtest

and

$ rsnapshot hourly

Read the rsnapshot howto guides for further details on hourly, daily, the rsnapshot.conf file etc

What about hard links?
rsnapshot is based on hard links. I won’t go into too much details about it, but basically it allows saving loads of space by simply ‘pointing’ a number of files to one location on disk. Cygwin on Windows 2000 onwards fully supports hard links. I was quite confused though, because when you store multiple copies (hourly.0, hourly.1 etc) and you view them on Windows explorer – They all seem to take very similar space. However, the space on disk is preserved. If you want to check it out you can use a microsoft tool called HLScan

Final Notes
If you want to schedule the backups, you can use the cygwin cron (which I won’t go into details about). I preferred using Windows scheduler / Tasks / AT and created a batch and a script file to run them directly from windows. Simply create a one-line script file, preferably under /home/Administrator (or your user account), e.g.
hourly.sh

$ cd ~
$ echo rsnapshot hourly >hourly.sh
$ chmod 750 hourly.sh

Then copy the cygwin.bat file to hourly.bat and add the script name at the end of the command line:

bash --login -i hourly.sh

15 Responses to “Rsnapshot Server on Windows”

  1. a-caveman

    Finally! Someone put up instructions on how to do this!

    Thank you!

    rsnapshot works great for backing up to a linux box and using almost no resources, but all the software for pulling backups to a windows box hogs every last bit of bandwidth. I’ll be trying to implement this in the morning! Hopefully I’ll be replacing our bandwidth and resource hogging piece of junk backup programs with rsnapshot in the very near future…

    Thanks again!

  2. yoav

    Thanks. Please note things aren’t always perfect though. I did keep encountering problems with certain files. I think it was to do with folder nesting levels. At some points, and with particularly long filenames it didn’t seem to work as smoothly as expected. YMMV. Would like to hear other people’s experience / suggestions

  3. a-caveman

    I tried this this afternoon and it worked fine, except for rsnapshot complaining about file names, but that seems to be a windows thing.

    One thing, for those not necessarily familiar with .bat files, you need to edit:
    /Cygwin.bat
    using your favorite text editor and tack the mentioned line at the bottom:
    bash –login -i /path/to/shell/script

    Also, for your bash script that the .bat file references, I had to have a proper shell script for it to work:
    #!/bin/bash
    /usr/local/bin/rsnapshot hourly
    exit

    But otherwise, your instructions worked quite well.

  4. Pseudo-Anonymous Coward

    Rsnapshot is actually packaged in cygwin now (Hurray!).

    I would caution anyone against using anything older than the 1.7 release because of the path length restriction. The older version of cygwin doesn’t fail nicely when that limit is reached and it will kill your backups. All someone has to do to break your backups is make a folder tree that’s too deep.

  5. Michiel Bruijn

    Haleluja

    Thanks a lot. I really really really wanted to use rsnapshot to backup my home network.
    Many many times I tried to configure a dedicated linux box because I liked the idea of seperated OS’es.
    After a few days of strugling with mounting drives and most important increase samba speed (15 MB/sec on gigabit!) I gave up. I searched for a windows alternative.
    I don’t know how, but for some reason rsnapshot didn’t let me go and I ended up on your website.
    This is really fits my needs. I can control my backup pc with a familiar OS, and do the backup with cygwin + rsnapshot, and do the cron with administrative tasks. (and let the computer wake up for backup job)
    Thanks a million.

  6. Richard

    Great tutorial. I’m getting a problem with the following:

    $ mount -o managed w:/backup/.snapshots /.snapshots
    mount: invalid option – ‘managed’

    Any ideas? I’ve had a look at the mount command man pages and can’t see anything about “managed”.

    Thanks,

    Rich

  7. Yoav Aner

    Hi Rich,

    It’s been quite a few years since I posted this guide, so things might have changed quite a lot with cygwin. I since don’t use Windows at all, so can’t even try this out. I could still find this mentioned on the cygwin faq page though…

  8. Dudemanbubba

    Worked like a charm. Just got a WD Sentinel with the Windows 2008 Storage Server Essentials on it. Followed your tutorial to the T except for the manage mount at the end. I just entered the folder path to the snapshot directory I made on the Windows server /cygdrive/d/snapshots/ and it worked fine on my test case which was an Ubuntu 12.04 LTS server. It appears the managed mount was to help with file name compatibility. Does anyone know of a significant reason for concern? Is it for long file names only? I read that the managed mount was dropped from the cgwin…

  9. Yoav Aner

    This post is really old, and I haven’t used Windows much since, so unfortunately I can’t help much. I’m glad it’s still useful though, but I wouldn’t be surprised if a lot has changed since I wrote it.

  10. Anti Disinformation Campain

    Please add this sentence to this post:

    It is a really stupid idea to use rsnapshot on windows, because it will never backup your OPENED FILES. There are other tools that are able to utilize the Windows Volume Shadow Service – search for this to find some tools that actually make sense and forget this stupid rsnapshot on windows idea – it makes no sense at all.

  11. Yoav Aner

    Please add this sentence to your comment: I like to make rude comments without even reading the post properly, or trying to understand it.

    This was written in 2007, a lot has changed since then. Nevertheless, the purpose of the post was running the rsnapshot *server* on windows, as a backup/storage server primarily for other linux servers/clients, not to back up Windows itself.

Leave a Reply

css.php