taters

I’ve had experiences with certain software manipulating iptables behind ufw and those changes do not show up when I viewed ufw’s status.

Docker is a well known example of this. Simply searching docker ufw in a search engine will bring up this issue immediately. I’ve also had experiences with cloudflared (the program used for cloudflare’s tunnel service) manipulating iptables behind ufw as well.

It’s possible the vpn is doing something similar.

I’m beginning to learn to check iptables as well when it comes to troubleshooting network issues.

No worries, glad I can help :)

If you are interested, I made a tool based around rsync. It’s what I use to handle my backups plus transferring files and folders between my network connected devices.

https://codeberg.org/taters/rTransfer

With my tool, I’ll make the following files in a directory:

.
├── .sync-computer-fs_01_root
├── .sync-computer-fs_02_boot
├── .sync-computer-fs_03_boot-efi
└── .sync-computer-fs_04_home

and then enter the Rsync Backup command information into the appropriate fields of each file. I can run my command with those files and it’ll do a dry run followed by a confirmation to continue with the actual transfer.

There’s a command option flag to reverse the transfer direction which can act as a “restore” in the case of an OS backup.

If you happen to give it a try, any feedback would be appreciated.

Whatever you choose to do, good luck and be sure to test everything more than you think you should. Both backups and restores. I know it’s all to easy to fuck up data transfer in any direction :)

I’ve been working with the rsync command for a while now and I have created a pretty solid backup system on my own with that command. I’ve had some terrible luck using back/restore programs like Timeshift on Linux Mint and so I went out of my way to learn how to make a “simpler” backup/restore process for myself.

I am using Alpine Linux so the following commands are based around that file system. It’s possible to modify the commands to fit your Operating System’s file system by using the lsblk command to see all the mountpoints for your OS. Each mountpoint requires a separate rsync command.

For myself, I have four mount points. / for the root directory. For the boot directory, I will need a command for /boot/ and /boot/efi/ directories. Since my home directory is in a separate partition, I also need a command for /home/.

The following are all the expanded commands I use to backup my entire OS which is backed up to the /backup/ directory:

# BACKUP  
# /root/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=/backup/*' '--exclude=/boot/*' '--exclude=home/*' '--exclude=proc/*' '--exclude=sys/*' '--exclude=dev/*' '--exclude=tmp/*' '--exclude=run/*' '--exclude=mnt/*' '--exclude=media/*' / /backup/`  

# /boot/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=lost+found' /boot/ /backup/boot/`  

# /boot/efi/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=lost+found' /boot/efi/ /backup/boot/efi/`  

# /home/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=lost+found' '--exclude=.cache/*' /home/ /backup/home/`  

The root partition is the most involved command so I’ll break it down slightly. You can check out the rsync man page for more detailed information about each command option.

• --dry-run- is used to perform a test run. If you have any verbose options, it can be used to see what files will be moved without performing any actions. This is important for testing but I also use it to view or review any changes before I commit anything. This option must be removed if you want to commit any changes. I’m leaving it here incase anyone decides to blindly run this command. I don’t want to be responsible for any lost data.
• --archive --acls --one-file-system --xattrs --hard-links --sparse are all used to preserve any file/folder attributes and permissions. --archive is commonly used, even for single files while the rest are more important for the operating system in the event of a restore. You can check the man page for more information as I’ve kind of forgotten their purposes over time.
• --verbose --human-readable --partial --progress is both for verbose outputs and for continuing in case a previous transfer was incomplete.
• --delete rsync will compare the source directory and the destination directory for any changes and will delete any files or directories in destination directory to match the source directory.
• --numeric-ids is used to preserve ownership in number form. I find it useful since I keep multiple device backups on an external SSD.
• --exclude=... This excludes files or directories from being synced. For the / directory it’s important to exclude /backup/. If you don’t, you will end up recursively saving to the /backup/* directory until your storage is full. /boot/* home/* are excluded because they will be handled by the following rsync commands. proc/* sys/* dev/* tmp/* run/* mnt/* media/* are directories that are used to run the operating system or devices and will not be required after a system reboot.
• / /backup/ is two parts. / is the source directory, what you want backed up. /backup/ is the destination directory. It’s where you want your data to end up.
• --exclude=lost+found is not important and can be excluded
• --exclude=.cache/* can also be excluded from the home directory if you want to save space.

You’ll most likely need to create the /backup/ directory in order to perform the backup commands.
Running the commands in order as listed, it will first perform a backup on the root directories while creating empty directories for the following commands. After that, each command will be able to complete it’s transfer.

Restoring is as simple as using the same command but reversing the Source and Destination directories. If backing up root was / /backup/, then to restore you now use /backup/ /. After performing a restore, it’s wise to reboot your system.

# RESTORE  
# /root/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=/backup/*' '--exclude=/boot/*' '--exclude=home/*' '--exclude=proc/*' '--exclude=sys/*' '--exclude=dev/*' '--exclude=tmp/*' '--exclude=run/*' '--exclude=mnt/*' '--exclude=media/*' /backup/ /`  

# /boot/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=lost+found' /backup/boot/ /boot/`  

# /boot/efi/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=lost+found' /backup/boot/efi/ /boot/efi/`  

# /home/  
`rsync --dry-run --archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress --delete --numeric-ids '--exclude=lost+found' '--exclude=.cache/*' /backup/home/ /home/`  

For docker containers, I usually shut them all down before a backup, perform the backup and start the containers again with the following commands:

# CONTAINERS DOWN  
docker stop --timeout 30 $(docker ps -a -q)  

# PERFORM BACKUP  

# CONTAINERS UP  
docker start $(docker ps -a -q)  

For a restore with docker containers, I usually shut down the containers, perform the restore and reboot my system. If you have any containers that require building, you may need to spend time rebuilding after a restore. I haven’t figured out how to address that issue yet. It may be helpful to run the command docker system prune -a && docker volume prune -f to remove all docker containers before a restore. Sometimes docker containers remain after performing a restore and must be deleted by hand. You can see which directories can be removed by performing a dry run and looking for the delete errors.

You don’t have to use any of this but for me I at least got some insight into how backing up and restoring a file system works with the help of rsync. Hopefully if can give you some insight as well for your needs.

I use these commands near daily since I like to keep my systems as neat and organized as possible. Often I’ll create a backup, do my experimenting, perform a restore, make any changes I want and perform another backup with the latest changes. I’ve had more consistent results going this route compared to using other backup programs.

I know there is a way to create smaller backups using links but I don’t think I do that. I prefer a full backup copy anyways since I run my systems in a more minimal way. Space isn’t an issue for me to worry about.