[Bacula-users] Problem: deleted files entrys are still in the database and are restored

[Vladimirs Vecgailis]

Hi all!


Following situation:

1) A Full backup of a client is done.

2) After then i create a new file

/var/log/myfile1

3) Now i do an incremental backup.
The new file is backuped.

4) now i delete this file.

5) Doing an incremental backup again.


And now, when i do a restore with "restore" and choosing the

(5) - "Select the most recent backup for a client"

the "myfile1", that is actually deleted in the "most recent version" of the client,
is also restored!


That's mean, that the entrys about file deletions are not updated in the database!
Or am i misunterstanding something?

I mean, if i restore a most recent version of a client, then i definetly do not want to
restore all deleted & old files! If i want to find some file, which is not present in the last
backup, i will seek for it with

(7) - "Enter a list of files to restore"
or
(8) - "Enter a list of files to restore before a specified time"


Help!



Mit freundlichen Grüßen

Vladimirs Vecgailis

--

==================================
Vladimirs Vecgailis
Colocation IT
primacall Vertriebs-GmbH
Postfach 08 04 55
10004 Berlin

Leipziger Str. 124
10117 Berlin
GERMANY

Phone [phone]
eMail ***@primacall.de
Web http://www.primacall.de
==================================

Amtsgericht und Sitz: Berlin, HRB 98348 B
Steuernummer: 37-199-20919
Geschäftsführer: M. Vukusic

Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können
rechtlich bedeutsam sein. Sie sind ausschließlich für den Adressaten
bestimmt und jeglicher Zugriff durch andere Personen ist unzulässig. Falls
Sie nicht der beabsichtigte Empfänger sind, ist jegliche Veröffentlichung,
Verteilung oder sonstige in diesem Zusammenhang stehende Handlung untersagt
und unter Umständen rechtswidrig. Herkömmliche E-Mails sind zudem nicht
gegen Zugriff geschützt und können mit und ohne Einfluss von Dritten
verloren gehen, verändert oder verfälscht werden. Eine Haftung für die
Unversehrtheit von E-Mails oder hieraus entstandene Schäden kann daher nicht
übernommen werden.

The information in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee and access to the e-mail
by anyone else is prohibited. If you are not the intended recipient, any
disclosure, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Common e-mails are not
protected against access by third persons and therefore might be lost,
changed or falsified, with or without any interference by third persons. We
cannot accept any responsibility for the integrity of e-mails or for any
damages resulting out of these circumstances.

[Silver Salonen]

Post by Vladimirs Vecgailis
Hi all!
1) A Full backup of a client is done.
2) After then i create a new file
/var/log/myfile1
3) Now i do an incremental backup.
The new file is backuped.
4) now i delete this file.
5) Doing an incremental backup again.
And now, when i do a restore with "restore" and choosing the
(5) - "Select the most recent backup for a client"
the "myfile1", that is actually deleted in the "most recent version" of the client,
is also restored!
That's mean, that the entrys about file deletions are not updated in the database!
Or am i misunterstanding something?
I mean, if i restore a most recent version of a client, then i definetly do not want to
restore all deleted & old files! If i want to find some file, which is not

present in the last

Post by Vladimirs Vecgailis
backup, i will seek for it with
(7) - "Enter a list of files to restore"
or
(8) - "Enter a list of files to restore before a specified time"
Help!

Hello.

Well, this is a known limitation - see
http://www.bacula.org/?page=projects "Item 1" ;)

[Vladimirs Vecgailis]

Post by Silver Salonen
Hello.
Well, this is a known limitation - see
http://www.bacula.org/?page=projects "Item 1" ;)

:-[ ]

But this bug renders the incremental backups (almost) useless...

Imagine a Maildir-directory with many thousands of mails.
Some hundrets are knowingly deleted.
Then a crash comes and data is restored - WITH ALL THE OLD FILES/MAILS...


Exactly this is my situation - i backup a Zimbra-Server (a mail & groupware-server).
It saves the mails in a maildir-like directory-structure in separate files.
Now i make backups of approx. 900 000 files, and the amount is bigger every day.

Now i tried a restore it to a test-machine and got all the old data back, what was deleted weeks ago!

Further, my database grows very fast!


What shoul be done in that kind of situation???

:-(




Mit freundlichen Grüßen

Vladimirs Vecgailis

--

==================================
Vladimirs Vecgailis
Colocation IT
primacall Vertriebs-GmbH
Postfach 08 04 55
10004 Berlin

Leipziger Str. 124
10117 Berlin
GERMANY

Phone [phone]
eMail ***@primacall.de
Web http://www.primacall.de
==================================

Amtsgericht und Sitz: Berlin, HRB 98348 B
Steuernummer: 37-199-20919
Geschäftsführer: M. Vukusic

Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können
rechtlich bedeutsam sein. Sie sind ausschließlich für den Adressaten
bestimmt und jeglicher Zugriff durch andere Personen ist unzulässig. Falls
Sie nicht der beabsichtigte Empfänger sind, ist jegliche Veröffentlichung,
Verteilung oder sonstige in diesem Zusammenhang stehende Handlung untersagt
und unter Umständen rechtswidrig. Herkömmliche E-Mails sind zudem nicht
gegen Zugriff geschützt und können mit und ohne Einfluss von Dritten
verloren gehen, verändert oder verfälscht werden. Eine Haftung für die
Unversehrtheit von E-Mails oder hieraus entstandene Schäden kann daher nicht
übernommen werden.

The information in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee and access to the e-mail
by anyone else is prohibited. If you are not the intended recipient, any
disclosure, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Common e-mails are not
protected against access by third persons and therefore might be lost,
changed or falsified, with or without any interference by third persons. We
cannot accept any responsibility for the integrity of e-mails or for any
damages resulting out of these circumstances.

[Ryan Novosielski]

Post by Vladimirs Vecgailis

Post by Silver Salonen
Hello.
Well, this is a known limitation - see
http://www.bacula.org/?page=projects "Item 1" ;)

:-[ ]
But this bug renders the incremental backups (almost) useless...
Imagine a Maildir-directory with many thousands of mails.
Some hundrets are knowingly deleted.
Then a crash comes and data is restored - WITH ALL THE OLD FILES/MAILS...
Exactly this is my situation - i backup a Zimbra-Server (a mail & groupware-server).
It saves the mails in a maildir-like directory-structure in separate files.
Now i make backups of approx. 900 000 files, and the amount is bigger every day.
Now i tried a restore it to a test-machine and got all the old data back, what was deleted weeks ago!
Further, my database grows very fast!
What shoul be done in that kind of situation???
:-(

I can point out that you SHOULD have read the documentation, as it is
not like this was a secret. I'm glad that it sounds like you caught it
before trying this on a production system. It is NOT a bug, it is the
way things currently work, and it is a common limitation.

For a maildir store, you should be doing full backups every night.

- --
---- _ _ _ _ ___ _ _ _
|Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Systems Programmer II
|$&| |__| | | |__/ | \| _| |***@umdnj.edu - 973/972.0922 (2-0922)
\__/ Univ. of Med. and Dent.|IST/AST - NJMS Medical Science Bldg - C630

[Vladimirs Vecgailis]

В Thu, 10 Jan 2008 09:19:40 -0500

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I can point out that you SHOULD have read the documentation, as it is
not like this was a secret. I'm glad that it sounds like you caught it
before trying this on a production system. It is NOT a bug, it is the
way things currently work, and it is a common limitation.
For a maildir store, you should be doing full backups every night.

I've READ the documentation, but maybe i passed exactly this part,,,
though i remember to have read about something in this vein...


* pulling my hair out * ...

As it stands at http://www.bacula.org/?page=projects the date, when
this item/limitation was noted/registered was the "28 November 2005"
and is now the "Item 1" on the project-list.

Maybe you know how is the stand at the moment?


Greetings.

Vladimirs Vecgailis


==================================
Vladimirs Vecgailis
Colocation IT
primacall Vertriebs-GmbH
Postfach 08 04 55
10004 Berlin

Leipziger Str. 124
10117 Berlin
GERMANY

Phone [phone]
eMail ***@primacall.de
Web http://www.primacall.de
==================================

Amtsgericht und Sitz: Berlin, HRB 98348 B
Steuernummer: 37-199-20919
Geschäftsführer: M. Vukusic

Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können
rechtlich bedeutsam sein. Sie sind ausschließlich für den Adressaten
bestimmt und jeglicher Zugriff durch andere Personen ist unzulässig. Falls
Sie nicht der beabsichtigte Empfänger sind, ist jegliche Veröffentlichung,
Verteilung oder sonstige in diesem Zusammenhang stehende Handlung untersagt
und unter Umständen rechtswidrig. Herkömmliche E-Mails sind zudem nicht
gegen Zugriff geschützt und können mit und ohne Einfluss von Dritten
verloren gehen, verändert oder verfälscht werden. Eine Haftung für die
Unversehrtheit von E-Mails oder hieraus entstandene Schäden kann daher nicht
übernommen werden.

The information in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee and access to the e-mail
by anyone else is prohibited. If you are not the intended recipient, any
disclosure, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Common e-mails are not
protected against access by third persons and therefore might be lost,
changed or falsified, with or without any interference by third persons. We
cannot accept any responsibility for the integrity of e-mails or for any
damages resulting out of these circumstances.

[Dan Langille]

Post by Vladimirs Vecgailis
As it stands at http://www.bacula.org/?page=projects the date, when
this item/limitation was noted/registered was the "28 November 2005"
and is now the "Item 1" on the project-list.
Maybe you know how is the stand at the moment?

To my knowledge, nobody is working on it.

[Jason A. Kates]

I wouldn't say that this is a bug. The shrink wrapped backup softare
that you will pay 10's of K for work in the same way. If it's that
much of an issue just do fulls for that file system.
-Jason

Post by Vladimirs Vecgailis

Post by Silver Salonen
Hello.
Well, this is a known limitation - see
http://www.bacula.org/?page=projects "Item 1" ;)

:-[ ]
But this bug renders the incremental backups (almost) useless...
Imagine a Maildir-directory with many thousands of mails.
Some hundrets are knowingly deleted.
Then a crash comes and data is restored - WITH ALL THE OLD FILES/MAILS...
Exactly this is my situation - i backup a Zimbra-Server (a mail & groupware-server).
It saves the mails in a maildir-like directory-structure in separate files.
Now i make backups of approx. 900 000 files, and the amount is bigger every day.
Now i tried a restore it to a test-machine and got all the old data back, what was deleted weeks ago!
Further, my database grows very fast!
What shoul be done in that kind of situation???
:-(
Mit freundlichen Grüßen
Vladimirs Vecgailis
--
==================================
Vladimirs Vecgailis
Colocation IT
primacall Vertriebs-GmbH
Postfach 08 04 55
10004 Berlin
Leipziger Str. 124
10117 Berlin
GERMANY
Phone [phone]
Web http://www.primacall.de
==================================
Amtsgericht und Sitz: Berlin, HRB 98348 B
Steuernummer: 37-199-20919
Geschäftsführer: M. Vukusic
Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können
rechtlich bedeutsam sein. Sie sind ausschließlich für den Adressaten
bestimmt und jeglicher Zugriff durch andere Personen ist unzulässig. Falls
Sie nicht der beabsichtigte Empfänger sind, ist jegliche Veröffentlichung,
Verteilung oder sonstige in diesem Zusammenhang stehende Handlung untersagt
und unter Umständen rechtswidrig. Herkömmliche E-Mails sind zudem nicht
gegen Zugriff geschützt und können mit und ohne Einfluss von Dritten
verloren gehen, verändert oder verfälscht werden. Eine Haftung für die
Unversehrtheit von E-Mails oder hieraus entstandene Schäden kann daher nicht
übernommen werden.
The information in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee and access to the e-mail
by anyone else is prohibited. If you are not the intended recipient, any
disclosure, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Common e-mails are not
protected against access by third persons and therefore might be lost,
changed or falsified, with or without any interference by third persons. We
cannot accept any responsibility for the integrity of e-mails or for any
damages resulting out of these circumstances.
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Bacula-users mailing list
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Vladimirs Vecgailis]

В Thu, 10 Jan 2008 09:34:00 -0500

Post by Jason A. Kates
If it's that
much of an issue just do fulls for that file system.
-Jason

Well, the problem is, the Zimbra-Server is not the only one, which
have this problem.
There are also other server, that have the same "problem"...

To do Fulls for all of them whould be the worst solution... :-\
There are about 2 TB of data that is written to tapes on every full-backup round weekly...

Vladimirs

[Chris Hoogendyk]

Post by Jason A. Kates
I wouldn't say that this is a bug. The shrink wrapped backup softare
that you will pay 10's of K for work in the same way. If it's that
much of an issue just do fulls for that file system.
-Jason

Actually, some do and some don't. You have to check the documentation
for each one to be sure.

For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.


---------------

Chris Hoogendyk

-
O__ ---- Systems Administrator
c/ /'_ --- Biology & Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst

<***@bio.umass.edu>

---------------

Erdös 4

Post by Jason A. Kates

Post by Vladimirs Vecgailis

Post by Silver Salonen
Hello.
Well, this is a known limitation - see
http://www.bacula.org/?page=projects "Item 1" ;)

:-[ ]
But this bug renders the incremental backups (almost) useless...
Imagine a Maildir-directory with many thousands of mails.
Some hundrets are knowingly deleted.
Then a crash comes and data is restored - WITH ALL THE OLD FILES/MAILS...
Exactly this is my situation - i backup a Zimbra-Server (a mail & groupware-server).
It saves the mails in a maildir-like directory-structure in separate files.
Now i make backups of approx. 900 000 files, and the amount is bigger every day.
Now i tried a restore it to a test-machine and got all the old data back, what was deleted weeks ago!
Further, my database grows very fast!
What shoul be done in that kind of situation???
:-(
Mit freundlichen Grüßen
Vladimirs Vecgailis
--
==================================
Vladimirs Vecgailis
Colocation IT
primacall Vertriebs-GmbH
Postfach 08 04 55
10004 Berlin
Leipziger Str. 124
10117 Berlin
GERMANY
Phone [phone]
Web http://www.primacall.de
==================================
Amtsgericht und Sitz: Berlin, HRB 98348 B
Steuernummer: 37-199-20919
Geschäftsführer: M. Vukusic
Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können
rechtlich bedeutsam sein. Sie sind ausschließlich für den Adressaten
bestimmt und jeglicher Zugriff durch andere Personen ist unzulässig. Falls
Sie nicht der beabsichtigte Empfänger sind, ist jegliche Veröffentlichung,
Verteilung oder sonstige in diesem Zusammenhang stehende Handlung untersagt
und unter Umständen rechtswidrig. Herkömmliche E-Mails sind zudem nicht
gegen Zugriff geschützt und können mit und ohne Einfluss von Dritten
verloren gehen, verändert oder verfälscht werden. Eine Haftung für die
Unversehrtheit von E-Mails oder hieraus entstandene Schäden kann daher nicht
übernommen werden.
The information in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee and access to the e-mail
by anyone else is prohibited. If you are not the intended recipient, any
disclosure, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Common e-mails are not
protected against access by third persons and therefore might be lost,
changed or falsified, with or without any interference by third persons. We
cannot accept any responsibility for the integrity of e-mails or for any
damages resulting out of these circumstances.
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Bacula-users mailing list
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Alan Brown]

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.

When did Amanda gain that feature? I didn't see it last time I looked...

[Chris Hoogendyk]

Post by Alan Brown

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.

When did Amanda gain that feature? I didn't see it last time I looked...

It may not be specifically listed as a feature in the documentation.

Amanda uses the native utilities to actually do the backups. On Solaris
that would be ufsdump. I did a series of tests in which I deleted files
and moved files from one directory to another after a full backup and
before an incremental backup. When I did the restore of the full, the
deleted and moved items were there in their original positions, but when
I did the restore of the incremental on top of the full, they magically
disappeared and moved items ended up in their proper locations.

Gnutar also will handle deleted files:
http://www.gnu.org/software/automake/manual/tar/Incremental-Dumps.html ;
and, if you were to use Amanda on linux, you would do it with gnutar.

Post by Alan Brown

Post by Chris Hoogendyk
I wouldn't say that this is a bug. The shrink wrapped backup softare
that you will pay 10's of K for work in the same way. If it's that
much of an issue just do fulls for that file system.

Jason is correct, there are only a couple of (very expensive) backup
packages which do deletion tracking over differential/incremental backups.
I have a similar situation with maildir format areas. My current solution
is weekly full backups.

Actually EMC Retrospect is not "very expensive". However, I'm not sure
whether it meets other high end features and needs that people may have.
EMC pushes it for the SMB market and has their own high end expensive
product, Legato Networker. Years ago I used Retrospect for company wide
tape backups for a small company (50 Macs), and now I use it for small
workgroups to handle their own backups to a disk archive on my servers
(less than $500 U.S. for a one "server" and 20 client license). But I'm
not going to consider commercial applications for my primary
server/network backups. Don't need to. Open source handles it quite
well. I just don't want to push that on the non techy small workgroups,
and I don't have the manpower to take direct responsibility for their
backups.

In the free arena, besides Amanda, there is rsync, and a google search
of "incremental deleted files" turns up a number of others (as well as
commercial products).

Interestingly, that google search also turns up a patent that was issued
to IBM in 2005 for "a system and associated method for restoring a file
system from incremental backups in the presence of deletion, without
restoring deleted files." Amazing how big companies can get patents for
ideas that have been around for a while.

More interestingly, Network Appliance also has a patent that involves
specialized header types for deleted files, Star Softcomm (Singapore)
has a patent that includes references to incrementals handling deleted
files, Teradactyl LLC has a patent on a backup procedure that includes
metadata for deleted files, HP has a patent that includes identifying
deleted files between full and delta backups, and Howard Lee and
Benjamin Cutler have a patent for an archival system that tracks file
deletions. (I just added patentstorm to my search to sift all that to
the top.)

So, without going out and examining all our competitors in detail, I do
think that's sufficient evidence that it is inaccurate to say "only a
couple".



---------------

Chris Hoogendyk

-
O__ ---- Systems Administrator
c/ /'_ --- Biology & Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst

<***@bio.umass.edu>

---------------

Erdös 4

[m***@uphs.upenn.edu]

In the message dated: Fri, 11 Jan 2008 15:57:02 EST,
The pithy ruminations from Chris Hoogendyk on
<Re: [Bacula-users] Problem: deleted files entrys are still in the database and
are restored> were:
=>
=>
=> Alan Brown wrote:
=> > On Thu, 10 Jan 2008, Chris Hoogendyk wrote:
=> >
=> >> For example, in the commercial realm, EMC Retrospect will properly
=> >> restore incremental backups with deleted files removed. In the open
=> >> source realm, Amanda will also properly handle deleted files in
=> >> incremental backups.
=> >
=> > When did Amanda gain that feature? I didn't see it last time I looked...
=>
=> It may not be specifically listed as a feature in the documentation.
=>
=> Amanda uses the native utilities to actually do the backups. On Solaris

Similarly, BackupPC also handles deleted files correctly, subject to the
capability of the underlying mechanism (gnutar and rsync work correctly
while smb and some vendor versions of tar would not).

Mark


[SNIP!]

=>
=> ---------------
=>
=> Chris Hoogendyk
=>
=> -
=> O__ ---- Systems Administrator
=> c/ /'_ --- Biology & Geology Departments
=> (*) \(*) -- 140 Morrill Science Center
=> ~~~~~~~~~~ - University of Massachusetts, Amherst =
=>
=>
=> <***@bio.umass.edu>
=>
=> --------------- =
=>
=>

----
Mark Bergman ***@uphs.upenn.edu
System Administrator
Section of Biomedical Image Analysis 215-662-7310
Department of Radiology, University of Pennsylvania

http://pgpkeys.pca.dfn.de:11371/pks/lookup?search=mark.bergman%40.uphs.upenn.edu




The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.

[Martin Simmons]

Post by Alan Brown

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.

When did Amanda gain that feature? I didn't see it last time I looked...

It is a feature of tar.

__Martin

[Dan Langille]

Post by Martin Simmons

Post by Alan Brown

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.

When did Amanda gain that feature? I didn't see it last time I looked...

It is a feature of tar.

Do you recall what feature or option in tar? I'm asking for hints when
I search the man page. I didn't find 'delete' mentioned specifically.

[C M Reinehr]

Post by Dan Langille

Post by Martin Simmons

Post by Alan Brown

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.

When did Amanda gain that feature? I didn't see it last time I looked...

It is a feature of tar.

Do you recall what feature or option in tar? I'm asking for hints when
I search the man page. I didn't find 'delete' mentioned specifically.

Dan,

I found this documented on the tar website:

http://www.gnu.org/software/tar/manual/tar.html#SEC88

"Incremental backup is a special form of GNU tar archive that stores
additional metadata so that exact state of the file system can be restored
when extracting the archive."

The online manual goes into much more detail than the "man" manual page.

HTH

cmr

[Dan Langille]

Post by C M Reinehr

Post by Dan Langille

Post by Martin Simmons

Post by Alan Brown

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.

When did Amanda gain that feature? I didn't see it last time I looked...

It is a feature of tar.

Do you recall what feature or option in tar? I'm asking for hints when
I search the man page. I didn't find 'delete' mentioned specifically.

Dan,
http://www.gnu.org/software/tar/manual/tar.html#SEC88
"Incremental backup is a special form of GNU tar archive that stores
additional metadata so that exact state of the file system can be restored
when extracting the archive."
The online manual goes into much more detail than the "man" manual page.

Thank you for the URL.

As pointed out in the manpage, this features uses tar extensions which
may not be implemented in other implementations of tar.

The strategies used by GNU tar may be worth study by anyone implementing
similar features in Bacula.

[Martin Simmons]

Post by Dan Langille

Post by C M Reinehr

Post by Dan Langille

Post by Martin Simmons

Post by Chris Hoogendyk
For example, in the commercial realm, EMC Retrospect will properly
restore incremental backups with deleted files removed. In the open
source realm, Amanda will also properly handle deleted files in
incremental backups.
When did Amanda gain that feature? I didn't see it last time I looked...

It is a feature of tar.

Do you recall what feature or option in tar? I'm asking for hints when
I search the man page. I didn't find 'delete' mentioned specifically.

Dan,
http://www.gnu.org/software/tar/manual/tar.html#SEC88
"Incremental backup is a special form of GNU tar archive that stores
additional metadata so that exact state of the file system can be restored
when extracting the archive."
The online manual goes into much more detail than the "man" manual page.

Thank you for the URL.
As pointed out in the manpage, this features uses tar extensions which
may not be implemented in other implementations of tar.
The strategies used by GNU tar may be worth study by anyone implementing
similar features in Bacula.

I think that idea has been rejected already, because the strategies are too
specific to unix and do not work on Windows.

Also, I'm not sure if GNU tar's implementation of incrementals is working in
all cases yet.

__Martin

[Alan Brown]

Post by Jason A. Kates
I wouldn't say that this is a bug. The shrink wrapped backup softare
that you will pay 10's of K for work in the same way. If it's that
much of an issue just do fulls for that file system.

Jason is correct, there are only a couple of (very expensive) backup
packages which do deletion tracking over differential/incremental backups.

I have a similar situation with maildir format areas. My current solution
is weekly full backups.

[Vladimirs Vecgailis]

Am Fri, 11 Jan 2008 15:04:15 +0000 (GMT)

Post by Alan Brown

Post by Jason A. Kates
I wouldn't say that this is a bug. The shrink wrapped backup
softare that you will pay 10's of K for work in the same way. If
it's that much of an issue just do fulls for that file system.

Jason is correct, there are only a couple of (very expensive) backup
packages which do deletion tracking over differential/incremental backups.

Wow, have not thought, that the situation is so critical with commercial
software.

Post by Alan Brown
I have a similar situation with maildir format areas. My current
solution is weekly full backups.

Well, i do also weekly backups... But i also do daily inc's...




Mit freundlichen Grüßen


Vladimirs Vecgailis
Geschäftsführer

---
RedStar IT - Linux Consulting, Support, Hosting
Inhaber Vladimirs Vecgailis
Storkower Str. 225
10367 Berlin

Tel. [phone]
Fax. [phone]

E-Mail: ***@redstar-it.de
WEB: http://www.redstar-it.de

[Dan Langille]

Post by Vladimirs Vecgailis

Post by Silver Salonen
Hello.
Well, this is a known limitation - see
http://www.bacula.org/?page=projects "Item 1" ;)

:-[ ]
But this bug renders the incremental backups (almost) useless...

This is not a bug at all. This is a common characteristic of many
backup systems.

BTW: when posting to a public mailing list, your huge signature is
rather bulky and inappropriate. It is certainly not confidental. If

Post by Vladimirs Vecgailis
==================================
Vladimirs Vecgailis
Colocation IT
primacall Vertriebs-GmbH
Postfach 08 04 55
10004 Berlin
Leipziger Str. 124
10117 Berlin
GERMANY
Phone [phone]
Web http://www.primacall.de
==================================
Amtsgericht und Sitz: Berlin, HRB 98348 B
Steuernummer: 37-199-20919
Geschäftsführer: M. Vukusic
Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können
rechtlich bedeutsam sein. Sie sind ausschließlich für den Adressaten
bestimmt und jeglicher Zugriff durch andere Personen ist unzulässig. Falls
Sie nicht der beabsichtigte Empfänger sind, ist jegliche Veröffentlichung,
Verteilung oder sonstige in diesem Zusammenhang stehende Handlung untersagt
und unter Umständen rechtswidrig. Herkömmliche E-Mails sind zudem nicht
gegen Zugriff geschützt und können mit und ohne Einfluss von Dritten
verloren gehen, verändert oder verfälscht werden. Eine Haftung für die
Unversehrtheit von E-Mails oder hieraus entstandene Schäden kann daher nicht
übernommen werden.
The information in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee and access to the e-mail
by anyone else is prohibited. I you are not the intended recipient, any
disclosure, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Common e-mails are not
protected against access by third persons and therefore might be lost,
changed or falsified, with or without any interference by third persons. We
cannot accept any responsibility for the integrity of e-mails or for any
damages resulting out of these circumstances.

[Vladimirs Vecgailis]

Post by Dan Langille

Post by Vladimirs Vecgailis
But this bug renders the incremental backups (almost) useless...

This is not a bug at all. This is a common characteristic of many
backup systems.

well, ok,

this limitation renders the incremental backups (almost) useless.

Post by Dan Langille
BTW: when posting to a public mailing list, your huge signature is
rather bulky and inappropriate. It is certainly not confidental. If

np.

Vladimirs.

[Dan Langille]

Post by Vladimirs Vecgailis

Post by Dan Langille

Post by Vladimirs Vecgailis
But this bug renders the incremental backups (almost) useless...

This is not a bug at all. This is a common characteristic of many
backup systems.

well, ok,
this limitation renders the incremental backups (almost) useless.

I can see why you think that. Other will disagree.

With heavily changing file systems, a full backup is your best solution.

There is a project to achieve what you want. To date, nobody has done
the work.

Post by Vladimirs Vecgailis

Post by Dan Langille
BTW: when posting to a public mailing list, your huge signature is
rather bulky and inappropriate. It is certainly not confidental. If

np.

Thank you. :)

[Vladimirs Vecgailis]

В Thu, 10 Jan 2008 10:04:03 -0500

Post by Dan Langille

Post by Vladimirs Vecgailis
well, ok,
this limitation renders the incremental backups (almost) useless.

I can see why you think that. Other will disagree.

???
But it the situation, when a file is knowingly removed/email deleted/whatever is deleted
is in 99% of cases. Knowingly, i repeat. And if this file is still restored
this leads the whole restore-process "ad absurdum"...


V.

[R.I. Pienaar]

Hello,

Post by Vladimirs Vecgailis
But it the situation, when a file is knowingly removed/email deleted/whatever is deleted
is in 99% of cases. Knowingly, i repeat. And if this file is still restored
this leads the whole restore-process "ad absurdum"...

As is the case with most tools, you need to evaluate them and discover
the best way to implement in your environment.

You've done this and discovered the best solution would be full
backups each night.

I've worked with several commercial backup systems, most of them have
the same "feature" yes its annoying but its the nature of the beast.

[Jason A. Kates]

2TB of nightly backups isn't that much to backup each night. If it's
that much of an issue I would move to fulls.

I upgraded from Netbackup to Bacula and each will restore the base files
then the incremental files.
-Jason

Post by Vladimirs Vecgailis
В Thu, 10 Jan 2008 10:04:03 -0500

Post by Dan Langille

Post by Vladimirs Vecgailis
well, ok,
this limitation renders the incremental backups (almost) useless.

I can see why you think that. Other will disagree.

???
But it the situation, when a file is knowingly removed/email deleted/whatever is deleted
is in 99% of cases. Knowingly, i repeat. And if this file is still restored
this leads the whole restore-process "ad absurdum"...
V.
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Bacula-users mailing list
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Vladimirs Vecgailis]

В Thu, 10 Jan 2008 10:41:59 -0500

Post by Jason A. Kates
2TB of nightly backups isn't that much to backup each night.

I thing it's rather the question of your retention period.

Post by Jason A. Kates
I upgraded from Netbackup to Bacula and each will restore the base files
then the incremental files.
-Jason

You mean, Netbackup has the same limitation???


Vladimir

[Eric Bollengier]

Post by Vladimirs Vecgailis

Post by Dan Langille

Post by Vladimirs Vecgailis
But this bug renders the incremental backups (almost) useless...

This is not a bug at all. This is a common characteristic of many
backup systems.

well, ok,
this limitation renders the incremental backups (almost) useless.

Why not run a command like 'find . -type f > file_list' and store the result
in a file before the backup ? (It will work better with a snapshot)

You will be aware of all "current" files...

Bye

[Vladimirs Vecgailis]

Am Fri, 11 Jan 2008 18:50:50 +0100

Post by Eric Bollengier

Post by Vladimirs Vecgailis

Post by Dan Langille

Post by Vladimirs Vecgailis
But this bug renders the incremental backups (almost) useless...

This is not a bug at all. This is a common characteristic of many
backup systems.

well, ok,
this limitation renders the incremental backups (almost) useless.

Why not run a command like 'find . -type f > file_list' and store the
result in a file before the backup ? (It will work better with a
snapshot)
You will be aware of all "current" files...
Bye

Bingo! That's it. At least for unix-based systems it will work...

... make a list, and then do "recover" and give the filelist to
bacula...


Mit freundlichen Grüßen


Vladimirs Vecgailis
Geschäftsführer

---
RedStar IT - Linux Consulting, Support, Hosting
Inhaber Vladimirs Vecgailis
Storkower Str. 225
10367 Berlin

Tel. [phone]
Fax. [phone]

E-Mail: ***@redstar-it.de
WEB: http://www.redstar-it.de

[Eric Bollengier]

Hi,

Post by Dan Langille

Post by Vladimirs Vecgailis
As it stands at http://www.bacula.org/?page=projects the date, when
this item/limitation was noted/registered was the "28 November 2005"
and is now the "Item 1" on the project-list.
Maybe you know how is the stand at the moment?

To my knowledge, nobody is working on it.

I'm working on it with kern, we have a solution but it will takes
some time to implement it.

Bye

[Vladimirs Vecgailis]

Am Fri, 11 Jan 2008 19:19:07 +0100

Post by Eric Bollengier
Hi,

Post by Dan Langille

Post by Vladimirs Vecgailis
As it stands at http://www.bacula.org/?page=projects the date, when
this item/limitation was noted/registered was the "28 November
2005" and is now the "Item 1" on the project-list.
Maybe you know how is the stand at the moment?

To my knowledge, nobody is working on it.

I'm working on it with kern, we have a solution but it will takes
some time to implement it.
Bye

! :-)

Good news! :-)





Mit freundlichen Grüßen


Vladimirs Vecgailis
Geschäftsführer

---
RedStar IT - Linux Consulting, Support, Hosting
Inhaber Vladimirs Vecgailis
Storkower Str. 225
10367 Berlin

Tel. [phone]
Fax. [phone]

E-Mail: ***@redstar-it.de
WEB: http://www.redstar-it.de