Using tar, rclone, bash, AWS CLI, systemd services, timers - to schedule a website back up to an S3 bucket
Using AWS CLI to create an S3 bucket and to attach IAM policies
Configuring rclone to access the AWS S3 bucket
Creating a bash script to back up the website folder and upload it to an S3 bucket
Using a systemd service and timer to schedule website backup
Using AWS CLI to create an S3 bucket and to attach IAM policies
In this tutorial, we will back up the web site to a file and we will copy the resulting backup file to an AWS S3 bucket with rclone, at 3AM every day, via a systemd timer and service.
We will use AWS CLI to list and create a bucket to upload the backup archive.
You can add temporary credentials to the file~/.aws/credentials; make sure they allow creation of a bucket and attaching IAM policies:
[default]
aws_access_key_id=xx
aws_secret_access_key=xx
aws_session_token=xxxx
After which, you can run commands - list buckets for example
$ aws s3 ls
2024-11-20 20:05:58 cf-templates-1gphkm0byc8yz-eu-west-2
2024-11-18 20:18:16 elasticbeanstalk-eu-west-2-590184137142
2024-11-13 05:10:36 george-aws1-bucket-1
2024-11-20 22:23:03 s3bucket-yaml-mys3bucket-teepjyjfkdjb
Let’s create a bucket (bucket names have to be unique across AWS, as they are accessed via URLs like web sites):
$ aws s3api create-bucket --bucket georgetech-backup --region eu-west-2 --create-bucket-configuration LocationConstraint=eu-west-2
{
"Location": "http://georgetech-backup.s3.amazonaws.com/"
}
Run the following command to create a new IAM user with programmatic access:
$ aws iam create-user --user-name rclone-s3-bucket
{
"User": {
"Path": "/",
"UserName": "rclone-s3-bucket",
"UserId": "xx",
"Arn": "arn:aws:iam::590184137142:user/rclone-s3-bucket",
"CreateDate": "2025-03-30T12:43:16+00:00"
}
}
Generate access keys (Access Key ID and Secret Access Key) for the user rclone-s3-bucket:
$ aws iam create-access-key --user-name rclone-s3-bucket
{
"AccessKey": {
"UserName": "rclone-s3-bucket",
"AccessKeyId": "xxx",
"Status": "Active",
"SecretAccessKey": "t/xx+JHV9ptOY",
"CreateDate": "2025-03-30T12:47:40+00:00"
}
}
Next, we will create a policy that defines the permissions needed by rclone to access the georgetech-backup bucket and to list all buckets in the account (one long line):
$ aws iam create-policy --policy-name RcloneS3BucketAccess --policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:CreateBucket"
],
"Resource": [
"arn:aws:s3:::georgetech-backup",
"arn:aws:s3:::georgetech-backup/*"
]
}
]
}'
The result of the command will be similar to the below:
{
"Policy": {
"PolicyName": "RcloneS3BucketAccess",
"PolicyId": "xx",
"Arn": "arn:aws:iam::account-id:policy/RcloneS3BucketAccess",
"Path": "/",
"DefaultVersionId": "v1",
"AttachmentCount": 0,
"PermissionsBoundaryUsageCount": 0,
"IsAttachable": true,
"CreateDate": "2025-03-30T12:50:49+00:00",
"UpdateDate": "2025-03-30T12:50:49+00:00"
}
}
We will attach the policy to the account, you need to get the <account-id> from the AWS console (IAM > Users > user rclone-s3-bucket):
$ aws iam attach-user-policy --user-name rclone-s3-bucket --policy-arn arn:aws:iam::<account-id>:policy/RcloneS3BucketAccess
Configuring rclone to access the AWS S3 bucket
We will install rclone:
sudo -v ; curl https://rclone.org/install.sh | sudo bash
Net we will add the AccessKeyId and SecretAccessKey to rclone via rclone config:
$ rclone config
Configuration complete.
Options:
- type: s3
- provider: AWS
- access_key_id: xx
- secret_access_key: t/xx+xx
- region: eu-west-2
- location_constraint: eu-west-2
- acl: private
The corresponding ~/.config/rclone/rclone.conf file will look like this:
[s3]
type = s3
provider = AWS
access_key_id = xx
secret_access_key = xxx
region = eu-west-2
location_constraint = eu-west-2
acl = private
We can list the files in the backup bucket:
$ rclone ls s3:georgetech-backup/
548 .bashrc
Creating a bash script to back up the website folder and upload it to an S3 bucket
Let’s create a bash script to automatically back up the Hugo site with tar and copy it to the AWS S3 bucket with rclone.
The backup_date variable will be the current date, and with backuppath it will be used in the script twice, to avoid mistyping.
We will also check for errors returned by tar or rclone and will add log entries with logger.
The rclone --config /path/to/rclone.conf should be added, as the commands will be run by systemd as the root user, and we are the admin user:
Don’t forget to make the file executable with chmod +x backup.sh:
#!/usr/bin/env bash
backup_date=$(date '+%d%m%Y_%H%M')
filename=hugo-site-$backup_date.tgz
backuppath=/home/admin/backups/
hugopath=/home/admin/quickstart/
rcloneconfigpath=/home/admin/.config/rclone/rclone.conf
tar czf $backuppath$filename $hugopath
if [ $? -eq 0 ]; then
logger -p user.info "Hugo backup - $filename - successfully created."
else
logger -p user.err "Hugo backup - $filename\ - creation failed."
fi
rclone --config $rcloneconfigpath copy $backuppath$filename s3:georgetech-backup/
if [ $? -eq 0 ]; then
logger -p user.info "Hugo backup - $filename - successfully copied to s3 bucket."
else
logger -p user.err "Hugo backup - $filename - copy to s3 bucket failed."
fi
We can check the result in the systemd log file via journalctl -r, which will list the latest log entries first:
Mar 30 14:35:48 ip-172-31-34-177 admin[5520]: Hugo backup - hugo-site-30032025_1435.tgz - successfully copied to s3 bucket.
Mar 30 14:35:48 ip-172-31-34-177 admin[5512]: Hugo backup - hugo-site-30032025_1435.tgz - successfully created.
We can also use the AWS CLI to list files in the bucket:
$ aws s3 ls s3://georgetech-backup/
2025-03-30 15:27:08 11971349 hugo-site-30032025_1427.tgz
Creating a systemd service and timer to schedule website backup
We will create a systemd service to run a few commands:
- archive the
/home/admin/quickstart/folder and its files to atararchive whose name include the date and time - use the
loggercommand to add a system log entry that the site has been archived and uploaded to the bucket; an error log entry will be added upon failure.
We can use the editorvimto create the file:
sudo vim /etc/systemd/system/hugo-site-backup.service
The contents of the systemd unit:
oneshotis used for a series of sequential tasks that terminate after running- we specify each task to run with
ExecStart - we use
bash -cto start the shell, and between quotes we use absolute paths (/home/admin/backupsinstead of~/backups, because the unit will be run as a different user,root:
[Unit]
Description=Backup Hugo Site
[Service]
Type=oneshot
ExecStart=/bin/bash -c "/home/admin/backups/backup.sh"
We will create a timer unit named /etc/systemd/system/hugo-site-backup.timer, to run the service above every day at 3:00 AM:
[Unit]
Description=Backup Hugo Site
[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
[Install]
WantedBy=timers.target
We will the enable timer, but it will only run when scheduled:
sudo systemctl enable hugo-site-backup.timer
We can get its status, it is waiting for the trigger (3AM) to start the hugo-site-backup.service:
$ systemctl status hugo-site-backup.timer
● hugo-site-backup.timer - Backup Hugo Site
Loaded: loaded (/etc/systemd/system/hugo-site-backup.timer; enabled; preset: enabled)
Active: active (waiting) since Sun 2025-03-30 14:45:39 UTC; 11min ago
Trigger: Mon 2025-03-31 03:00:00 UTC; 12h left
Triggers: ● hugo-site-backup.service
If we want to manually start the associated service to test its functionality, we can do so:
sudo systemctl start hugo-site-backup.service
The journalctl -r command shows that the hugo-site-backup.service ran successfully:
Mar 30 14:52:31 host systemd[1]: Finished hugo-site-backup.service - Backup Hugo Site.
Mar 30 14:52:31 host systemd[1]: hugo-site-backup.service: Deactivated successfully.
Mar 30 14:52:31 host root[5806]: Hugo backup - hugo-site-30032025_1452.tgz - successfully copied to s3 bucket.
Mar 30 14:52:30 host root[5796]: Hugo backup - hugo-site-30032025_1452.tgz - successfully created.
Mar 30 14:52:30 host bash[5793]: tar: Removing leading `/' from member names
Mar 30 14:52:30 host systemd[1]: Starting hugo-site-backup.service - Backup Hugo Site...
Mar 30 14:52:30 host sudo[5787]: pam_unix(sudo:session): session opened for user root(uid=0) by admin(uid=1000)
Mar 30 14:52:30 host sudo[5787]: admin : TTY=pts/0 ; PWD=/home/admin/backups ; USER=root ; COMMAND=/usr/bin/systemctl start hugo-site-backup.service
We can list the systemd timers, ours is in the list, waiting to be triggered at the scheduled time:
$ systemctl list-timers
NEXT LEFT LAST PASSED UNIT ACTIVATES
Mon 2025-03-31 02:19:03 BST 4h 39min left Sun 2025-03-30 07:05:51 BST 14h ago apt-daily.timer apt-daily.service
[..]
Mon 2025-03-31 03:00:00 BST 5h 20min left - - hugo-site-backup.timer hugo-site-backup.service
[..]