Discussion:
[Bacula-users] What to exclude?
Michael Munger
9 years ago
Permalink
TLDR; = what can I safely exclude from a bacula backup of a Linux server and still be able to fully recover (bare metal) from a disaster?

Details:
I am looking for a best practice here, and I am working from two sources:

1. http://www.bacula.org/5.0.x-manuals/en/main/main/Disaster_Recovery_Using_Bac.html#SECTION003960000000000000000
2. https://wiki.debian.org/FilesystemHierarchyStandard

In order to recover a client machine that is a linux server (not the bacula-dir or bacula-sd) in accordance with Bacula docs, I have setup the FileSet for such a server as noted below.

FileSet {
Name = "Example Linux Server"
Include {
Options {
signature = SHA1
compression = GZIP
}

File = /
}

Exclude {
/opt/bacula/working
/tmp
/proc
/sys
/.journal
/.fsck
/media
}
}

My reasons for excluding these are as follows:

/opt/bacula/working
Not needed - just pid files and temp stuff.
/tmp
no point in backing up temp files
/proc
can't backup processes
/sys
This is the filesystem point for exporting kernel objects. Kernel does this on the fly. No point in backing it up.

.journal
not needed. journal log file.

.fsck
not needed. fsck log file

/media
no reason to backup external devices (unless you have a reason, which I don't.)


It should be noted, that I am using ClientRunBeforeJob to dump MySQL databases to /var/local/mysqlbackups, so I have indepdenent copies of them there, but since I am NOT excluding /var/lib/mysql, the raw databases should also be recoverable from bare metal.

Does anyone have a different (better) best practice?

Michael Munger, dCAP, MCPS, MCNPS, MBSS
High Powered Help, Inc.
Microsoft Certified Professional
Microsoft Certified Small Business Specialist
Digium Certified Asterisk Professional
***@highpoweredhelp.com
Dimitri Maziuk
9 years ago
Permalink
Post by Michael Munger
TLDR; = what can I safely exclude from a bacula backup of a Linux
server and still be able to fully recover (bare metal) from a disaster?

Not an answer to your question, but we are
- using raid-1s for system disks,
- using SSDs for system disks,
- dd'ing the system disk before the machine goes into service and
keeping the copy on the shelf,
- backing up /etc and crontabs and not caring about the rest of /.

I'll pick any of the above over a bare metal recovery from a backup
system (SSD being the best option ATM).

FWIW
--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
Michael Munger
9 years ago
Permalink
Your answer is appreciated. And, I was trying to keep my question limited to what I wanted to know, but you bring up a good point: bacula is only part of the strategy. These systems are cloned with Clonezilla, the image of which sits on the shelf as a base. We use RAID 5/6/10 depending on how critical the machine is to slow the impact of a disk failure.

So, the restoration process would actually go: restore from clonezilla image, then restore from bacula.

But, I would prefer that bacula was as complete as possible in case there was an issue with the original image.

None of these machines are SSDs because SSDs fail instantly and catastrophically. And while the speed is sexy, the way they fail is ... not. But, that's neither here nor there. Just a personal choice for me.

Michael Munger, dCAP, MCPS, MCNPS, MBSS
High Powered Help, Inc.
Microsoft Certified Professional
Microsoft Certified Small Business Specialist
Digium Certified Asterisk Professional
***@highpoweredhelp.com


-----Original Message-----
From: Dimitri Maziuk [mailto:***@bmrb.wisc.edu]
Sent: Wednesday, January 27, 2016 5:02 PM
To: bacula-***@lists.sourceforge.net
Subject: Re: [Bacula-users] What to exclude?
Post by Michael Munger
TLDR; = what can I safely exclude from a bacula backup of a Linux
server and still be able to fully recover (bare metal) from a disaster?

Not an answer to your question, but we are
- using raid-1s for system disks,
- using SSDs for system disks,
- dd'ing the system disk before the machine goes into service and keeping the copy on the shelf,
- backing up /etc and crontabs and not caring about the rest of /.

I'll pick any of the above over a bare metal recovery from a backup system (SSD being the best option ATM).

FWIW
--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
Heitor Faria
9 years ago
Permalink
Hello Michael: bacula director client stock fileset exlude is a good starting point if you want to restore a full linux box (/tmp, /proc, /.journal etc.).
Already did full restores this way.

Regards,
===========================================================================
Heitor Medrado de Faria - LPIC-III | ITIL-F |  Bacula Systems Certified Administrator II
Do you need Bacula training? http://bacula.us/video-classes/
+55 61 8268-4220
Site: http://bacula.us FB: heitor.faria
===========================================================================

Enviado por TypeApp
...
Michael Munger
9 years ago
Permalink
Having setup and just tested, I ran into the "will not descend into different file systems" issue as mentioned in this post: http://bacula.10910.n7.nabble.com/Will-not-descend-from-into-dev-td57408.html, and it gives a very good reason for explicitly defining what to include.

So, I have amended my previous file set to now explicitly INCLUDE:

/bin/
/boot/
/etc/
/home/
/lib/
/opt/
/root/
/sbin/
/srv/
/usr/
/var/

While still excluding the other items.

Does anyone see an issue with this? (Trying to figure out if this is good enough now... before I have to recover and realize something is not quite right...).

Michael Munger, dCAP, MCPS, MCNPS, MBSS
High Powered Help, Inc.
Microsoft Certified Professional
Microsoft Certified Small Business Specialist
Digium Certified Asterisk Professional
***@highpoweredhelp.com


-----Original Message-----
From: Michael Munger [mailto:***@highpoweredhelp.com]
Sent: Wednesday, January 27, 2016 4:43 PM
To: bacula-***@lists.sourceforge.net
Subject: [Bacula-users] What to exclude?

TLDR; = what can I safely exclude from a bacula backup of a Linux server and still be able to fully recover (bare metal) from a disaster?

Details:
I am looking for a best practice here, and I am working from two sources:

1. http://www.bacula.org/5.0.x-manuals/en/main/main/Disaster_Recovery_Using_Bac.html#SECTION003960000000000000000
2. https://wiki.debian.org/FilesystemHierarchyStandard

In order to recover a client machine that is a linux server (not the bacula-dir or bacula-sd) in accordance with Bacula docs, I have setup the FileSet for such a server as noted below.

FileSet {
Name = "Example Linux Server"
Include {
Options {
signature = SHA1
compression = GZIP
}

File = /
}

Exclude {
/opt/bacula/working
/tmp
/proc
/sys
/.journal
/.fsck
/media
}
}

My reasons for excluding these are as follows:

/opt/bacula/working
Not needed - just pid files and temp stuff.
/tmp
no point in backing up temp files
/proc
can't backup processes
/sys
This is the filesystem point for exporting kernel objects. Kernel does this on the fly. No point in backing it up.

.journal
not needed. journal log file.

.fsck
not needed. fsck log file

/media
no reason to backup external devices (unless you have a reason, which I don't.)


It should be noted, that I am using ClientRunBeforeJob to dump MySQL databases to /var/local/mysqlbackups, so I have indepdenent copies of them there, but since I am NOT excluding /var/lib/mysql, the raw databases should also be recoverable from bare metal.

Does anyone have a different (better) best practice?

Michael Munger, dCAP, MCPS, MCNPS, MBSS
High Powered Help, Inc.
Microsoft Certified Professional
Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional ***@highpoweredhelp.com



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Martin Simmons
9 years ago
Permalink
Beware that including both / and /bin/ will save two copies of /bin/ (assuming
it is on the same filesystem as /).

The article actually suggests explicitly listing each filesystem, not each
directory.

__Martin
...
compdoc
9 years ago
Permalink
Post by Michael Munger
what can I safely exclude from a bacula backup of a Linux server and still
be able to fully recover (bare metal) from a disaster?

I would think with bare metal recovery you couldn't afford to exclude
anything. As for myself, I tend to replace the failed part and install from
scratch, then restore the applications and settings. Installing from scratch
lets me know the condition of the hardware. It should install smoothly and
with no crashes/freezes/unexplained occurrences.

The purpose of each server can determine what directories gets backed up.
For my Ubuntu Server running Bacula, using caja/nautilus I actually looked
through the directories and found all I want are these:

File = /boot/grub
File = /etc
File = /home
File = /root
File = /sbin
File = /opt
File = /var/crash
File = /var/log

Btw, if you have bacula spooling to /var while you're trying to back it up,
expect unexplained occurrences.
compdoc
9 years ago
Permalink
Post by Michael Munger
what can I safely exclude from a bacula backup of a Linux server and
still be able to fully recover (bare metal) from a disaster?
Oh, and don't forget mysql.
Michael Munger
9 years ago
Permalink
Good reminder. I am using a ClientBeforerunJob script to dump all the
databases to .sql files before backing up.
...
Randy Katz
9 years ago
Permalink
Is this list complete? What about stuff in /usr/local for example?
And /var ?
...
Gary R. Schmidt
9 years ago
Permalink
Post by Randy Katz
Is this list complete? What about stuff in /usr/local for example?
And /var ?
By definition, /usr/local (or, more standards-like, opt/local) is yours,
no-one else knows what is in it, and whether it should be included.

Frex, I have squid installed in /opt/local on one machine, and for that
I could exlude /opt/local/squid/var, as there are only the squid cache,
log, and run directories there.

As for /var - you'll have to have a look at yours, as, again, some
important libraries and configuration files can end up there, as well as
cache files and logs.

There is no really complete list, as no two machines are identical
(except if they are in a cluster!! ;-) ).

Now, having said all of that, with LTO-6 tapes and an 8-slot autoloader,
I just whack in the sensible top level mount points, exclude /tmp,
/proc, /opt/bacula, and any other obvious things to avoid, and wear the
minuscule overhead of storing logs and cache files and other dross.

Cheers,
Gary B-)

Continue reading on narkive:
Loading...