<---topics
 Persistence Alone is Omnipotent

Drop Deleted Files From Previous Snapshots

How to Update a Copy of Fileset with Files Added, Files with Changes and Some Files Deleted

June 2, 2025 12:04 am

The "rsync" command can handle all options in copying file sets, including deleting files from previous copies when the files have been deleted from the original file set. In simplest terms, adding the --delete option deletes files from previous copies of filesets:

rsync --delete path_to_production_files path_to_backup_copy_of_production

The backup scripts drop files from on backup snapshots if the files have been deleted form production.

All the backups are made by making snapshot copies. Deleted files are dropped from the backcup copies using the rsync command.

Rsync does the following

  1. If a file is deleted from its source system, the file is deleted from most recent snapshot
    When a book is dropped from the most recent snapshjot to stay in sync iwth thge original source, older snapshots hang onto the file as part of the record of what our system had as of the specific dates of those snapshots.
  2. If a file is changed, a copy of the new file is created in the backup library
  3. If a file has not changed, the file in the backup library is left untouched, but can be referenced as if each shapshot has its own copy of the files. We could not have 10% of the snapshots of our system dating back years without that file sharing tirck

    Copies of the files as ther existed by date are retained in the backup libary. A record of rsync options applied to a previously saved book are stored in the database system log.

Use RSYNC to make library of snapshots usi\ng minimal disk space

The "rsync" command can handle all options in copying file sets, including deleting files from previous copies when the files have been deleted from the original file set. In simplest terms, adding the --delete option deletes files from previous copies of filesets:

rsync --delete path_to_production_files path_to_backup_copy_of_production

The backup scripts drop files from on backup snapshots if the files have been deleted form production.

I wrote an explanation and provided a sample script but then my post would not appear in this subreddit. That longer post is now available on my website at when the from the most recent snapshots

  1. All the backups are made by making snapshot copies.
  2. Deleted files are dropped from the backcup copies using the rsync command.
  3. Rsync does the following (whatever happens is recorded the log produced by the backup script)
    1. If a file is deleted from its source system. The file is deleted from most recent snapshot Older snapshots hang onto the file as part of the record of what our system had as of specific dates/
    2. If a file is changed, a copy of the new file is created in the backup library
    3. If a file has not changed, the file in the backup library is left untouched, but can be referenced by all the snapshots as if each shapshot has its own copy of the files. Thanks to this disk-saving mode, we have hundreds of different snapshots of all our systems. If snapshots could not use on copy of a file for all the snapshots when the files had not changed, every snapshot would have to use the same amount of diskspace as our entire propduction system. Instead, the average disk storage for our snapshots is less the 3% the size of the whole system. Every snap shot is whiole and complete but needs sp

Sample Script for making snapshots of productipn file sets, while deleting from snapshops books previously included in snapshots

  1. 1 snapshot for each of the previous 24 hours
  2. 1 snapshot for each day from day 2 through day 14 in history
  3. 1 snapshot for each week from day 15 - same day ten weeks prior
  4. 1 snapshot per two weeks month from 10 weeks prior through same dat previous year
  5. 1 snapshot per month from the previous year through end of year three prior.
  6. 1 snapshot per quarter starting 3.25 years prior,

Sample Script for making snapshots of productipn file sets, while deleting from snapshops books previously included in snapshots

This a script that does most of those details:

#First, the script promotes snapshot copies one cay
mv /auxo/backups/att_web_dream/www/backup.01.019 /auxo/shithole/.auxo.backups.att_web_dream.www.backup.01.019_"$(date)"
mv /auxo/backups/att_web_dream/www/backup.01.018 /auxo/backups/att_web_dream/www/backup.01.019
mv /auxo/backups/att_web_dream/www/backup.01.017 /auxo/backups/att_web_dream/www/backup.01.018
mv /auxo/backups/att_web_dream/www/backup.01.016 /auxo/backups/att_web_dream/www/backup.01.017
mv /auxo/backups/att_web_dream/www/backup.01.015 /auxo/backups/att_web_dream/www/backup.01.016
mv /auxo/backups/att_web_dream/www/backup.01.014 /auxo/backups/att_web_dream/www/backup.01.015
mv /auxo/backups/att_web_dream/www/backup.01.013 /auxo/backups/att_web_dream/www/backup.01.014
mv /auxo/backups/att_web_dream/www/backup.01.012 /auxo/backups/att_web_dream/www/backup.01.013
mv /auxo/backups/att_web_dream/www/backup.01.011 /auxo/backups/att_web_dream/www/backup.01.012
mv /auxo/backups/att_web_dream/www/backup.01.010 /auxo/backups/att_web_dream/www/backup.01.011
mv /auxo/backups/att_web_dream/www/backup.01.009 /auxo/backups/att_web_dream/www/backup.01.010
mv /auxo/backups/att_web_dream/www/backup.01.008 /auxo/backups/att_web_dream/www/backup.01.009
mv /auxo/backups/att_web_dream/www/backup.01.007 /auxo/backups/att_web_dream/www/backup.01.008
mv /auxo/backups/att_web_dream/www/backup.01.006 /auxo/backups/att_web_dream/www/backup.01.007
mv /auxo/backups/att_web_dream/www/backup.01.005 /auxo/backups/att_web_dream/www/backup.01.006
mv /auxo/backups/att_web_dream/www/backup.01.004 /auxo/backups/att_web_dream/www/backup.01.005
mv /auxo/backups/att_web_dream/www/backup.01.003 /auxo/backups/att_web_dream/www/backup.01.004
mv /auxo/backups/att_web_dream/www/backup.01.002 /auxo/backups/att_web_dream/www/backup.01.003
mv /auxo/backups/att_web_dream/www/backup.01.001 /auxo/backups/att_web_dream/www/backup.01.002
echo 'done bumping copies up a level.'
echo 'Starting the copying level 01 to level 02, using links instead of files where files are unchanged.'
cp -al /auxo/backups/att_web_dream/www/backup.01.000 /auxo/backups/att_web_dream/www/backup.01.001

The RSYNC statement for deleting files n lopnger in produciton shystem

rsync -avzpt
>delete \
no-owner \
ignore-times \
checksum \
no-group \
no-perms \
delete-excluded \
exclude-from=att_web_dream.www.1_exclude \
links \
log-file=../logs/att_web_dream.rsync.log \
root@255.255.255.1:/var/www/ \\
/auxo/backups/att_web_dream/www/backup.01.000