Backup of fly.io volumes post

content/posts/20250430-creating-backups-for-fly-io-volumes.md

@@ -0,0 +1,102 @@
+---
+title: "Creating backups for fly.io Volumes"
+date: 2025-04-30
+tags: ["self-host", "fly.io", "backup"]
+slug: creating-backups-for-fly-io-volumes
+---
+
+I have a few applications running on [fly.io](https://fly.io), and some of them need to
+keep data in the file system persistently (more precisely, an SQLite database file and
+user-submitted data) so that it is not lost after a redeploy or when the Fly Machine running my application is
+restarted.
+
+To achieve that, I use [Fly Volumes](https://fly.io/docs/volumes/overview/) which are local
+persistent storage for Fly Machines, mounted in my server just like a regular directory. This setup works fine, 
+but I began considering how to back up the data stored there.
+
+[Volume snapshots](https://fly.io/docs/volumes/snapshots/) are created automatically on a daily basis and 
+retained for 5 days by default. However, there doesn't seem to be an easy (or well-documented) way to implement
+a custom backup policy. I wanted the ability to copy the entire directory's content using tools like `rsync` or upload
+it to an S3 bucket on my own schedule.
+
+I explored solutions involving `cron` jobs running inside my Fly Machine, but they became overly complicated. 
+These approaches required modifying my `Dockerfile` to install additional applications, and I wasn't sure 
+how to manage the schedule effectively, especially since I configured my machines to auto-stop to save resources.
+
+Direct SSH connections requires me to use `flyctl` CLI and it wasn't clear to me how to handle authentication
+in this case. After some research, I found that I can use [access tokens](https://fly.io/docs/security/tokens/)
+to connect to the machines using SSH allowing me to send commands there in an automated way.
+
+## Generating your access token
+
+First step is to create an access token that allows me to send commands to my machine without requiring any
+manual form of authentication. This can be done using the following command:
+
+```bash
+fly tokens create ssh -n my-token-name
+```
+
+Check the [command documentation](https://fly.io/docs/flyctl/tokens-create-ssh/) for more options. The output of 
+this command will be as the following, where `<TOKEN_CONTENT_STRING>` will be a very long string that
+you need to store and don't share it publicly.
+
+```bash
+FlyV1 <TOKEN_CONTENT_STRING>
+```
+
+Add the token to an environment var in the machine you will run the backup script:
+
+```bash
+export FLY_SSH_TOKEN=<TOKEN_CONTENT_STRING>
+```
+
+## Data location
+
+The volume is mounted in `/data` directory, defined in our application `fly.toml` file:
+
+```
+[[mounts]]
+  source = 'app_data'
+  destination = '/data'
+```
+
+## Creating a backup script
+
+With the token, we can now create two scripts: one to run locally on the machine that will receive the backup data,
+and another to be executed remotely on your Fly Machine. The example scripts are very simple, but you can improve
+them by adding more capabilities, error handling, uploading the data to S3 buckets, and so on.
+
+```
+# local_backup.sh
+
+# This call is needed to "wake-up" your machine if stopped
+curl -s -o /dev/null https://your-app.fly.dev/
+
+# Execute the script that will generate a tarball of /data/ directory
+fly ssh console -C 'sh remote_backup.sh' -t $FLY_SSH_TOKEN
+
+# Copy the generated tarball to our local machine
+fly ssh sftp get "/root/data_content_$(date +%F).tar.gz" -t $FLY_SSH_TOKEN
+```
+
+```
+# remote_backup.sh
+
+tar czf "/root/data_content_$(date +%F).tar.gz" data/
+```
+
+You need to send `remote_backup.sh` to your Fly Machine. Adding the following line to your `Dockerfile` should be enough:
+
+```
+COPY remote_backup.sh /remote_backup.sh
+```
+
+Deploy your application again, and you can run `local_backup.sh`.
+
+## Run it periodically
+
+Now you can add `local_backup.sh` to your `crontab` schedule, or even adapt the procedure described here
+to be executed in other environments, like defining a GitHub Action or another way to schedule jobs.
+
+I know this is not the most complete way to implement a backup policy, but it is working for my current projects.
+In the future, as I improve my scripts, I will possibly update this post to make it more complete.

themes/hugo-tania

@@ -1 +1 @@
-Subproject commit c52acfb01bb0188da4f76b1378d9045041772a99
+Subproject commit 6c4173f56e24a9192398f94d6e71d3e3da6d52de