diff --git a/rclone.md b/rclone.md new file mode 100644 index 0000000..a3fd514 --- /dev/null +++ b/rclone.md @@ -0,0 +1,144 @@ +# rclone + +## What is it + +rclone is a command-line tool for syncing, copying, and moving files between +local storage and remote destinations. It supports over 60 backends — SFTP, +S3-compatible object storage, WebDAV, SMB, and cloud providers like Backblaze +B2, Hetzner Object Storage, and Filen among many others. + +Think of it as rsync, but with native support for cloud and network remotes +baked in, and significantly better transfer progress output. + +## What it is mainly used for + +- Copying files to/from remote servers over SFTP +- Syncing directories to object storage (backups, offsite copies) +- Moving large amounts of data between locations reliably and resumably +- Replacing scp in scripts (scp is deprecated; rclone handles resume, + verification, and progress properly) + +## Installation + +```bash +brew install rclone # macOS +sudo apt install rclone # Debian/Ubuntu +``` + +## Configuring a remote + +rclone uses named remotes — connection profiles you configure once and +reference by name forever after. Run the interactive wizard: + +```bash +rclone config +``` + +Select `n` for new remote, give it a name (e.g. `seedbox`), then select the +backend type. For an SSH/SFTP server: + +- Type: `sftp` +- Host: `ds113081.seedhost.eu` +- User: `god` +- Port: `22` (default, just press Enter) +- Password: leave blank if you use SSH key auth +- Key file: leave blank to use your default `~/.ssh/` keys + +Once saved, the remote is available by name in all rclone commands. + +Config lives at `~/.config/rclone/rclone.conf`. If running rclone as root +(e.g. from a cron job via sudo), copy the config: + +```bash +sudo mkdir -p /root/.config/rclone +sudo cp ~/.config/rclone/rclone.conf /root/.config/rclone/rclone.conf +``` + +## Core commands + +### copy +Copies source to destination. Skips files that already exist and are identical. +Does not delete anything at the destination. + +```bash +rclone copy /local/path seedbox:/remote/path -P +``` + +### sync +Makes destination identical to source. **Deletes files at destination that +don't exist at source.** Use with care. + +```bash +rclone sync /local/path seedbox:/remote/path -P +``` + +### move +Copies files to destination then deletes them from source. + +```bash +rclone move /local/path seedbox:/remote/path -P +``` + +### ls / lsd / lsl +List files, directories, or files with timestamps. + +```bash +rclone lsd seedbox:/remote/path # list directories +rclone ls seedbox:/remote/path # list files +rclone lsl seedbox:/remote/path # list with timestamps and sizes +``` + +### check +Verifies that source and destination are identical without transferring anything. + +```bash +rclone check /local/path seedbox:/remote/path +``` + +## Real examples + +Copy a local folder to the seedbox over SFTP: +```bash +rclone copy /Volumes/Storage/Airport seedbox:/home/god/downloads/otw/ -P +``` + +Sync a folder to Hetzner Object Storage: +```bash +rclone sync /opt/docker/backups hetzner:my-bucket/backups -P +``` + +Copy from seedbox to local, limiting to 4 parallel transfers: +```bash +rclone copy seedbox:/home/god/downloads/ /Volumes/Storage/ -P --transfers=4 +``` + +Dry run to see what would happen without actually doing it: +```bash +rclone copy /source/ seedbox:/dest/ --dry-run +``` + +## Useful flags + +| Flag | What it does | +|------|-------------| +| `-P` / `--progress` | Live progress — files transferred, speed, ETA, overall size | +| `--transfers=N` | Number of parallel file transfers (default 4) | +| `--dry-run` | Show what would be copied/deleted without doing it | +| `--checksum` | Compare by checksum instead of size+date (slower, more accurate) | +| `--ignore-existing` | Skip files that exist at destination regardless of content | +| `--max-age=24h` | Only transfer files newer than the given age | +| `--exclude=PATTERN` | Exclude files matching a pattern (e.g. `--exclude="*.tmp"`) | +| `--bwlimit=10M` | Limit bandwidth (e.g. `10M` = 10 MB/s) | +| `--log-file=/path` | Write output to a log file (useful for cron jobs) | +| `--retries=N` | Number of retries on failure (default 3) | + +## Notes + +- Unlike rsync, rclone does not use `user@host:/path` syntax directly. + Always configure a named remote and reference it by name. +- rclone's `-P` progress is far more useful than rsync's `--progress` — + shows overall transfer stats, ETA, and per-file progress simultaneously. +- `copy` is the safe default. Use `sync` only when you explicitly want + destination to mirror source (it deletes files). +- scp is officially deprecated in OpenSSH. rclone over SFTP is the modern + replacement for scripts that need to move files to remote servers.