# rclone

## What is it

rclone is a command-line tool for syncing, copying, and moving files between
local storage and remote destinations. It supports over 60 backends — SFTP,
S3-compatible object storage, WebDAV, SMB, and cloud providers like Backblaze
B2, Hetzner Object Storage, and Filen among many others.

Think of it as rsync, but with native support for cloud and network remotes
baked in, and significantly better transfer progress output.

## What it is mainly used for

- Copying files to/from remote servers over SFTP
- Syncing directories to object storage (backups, offsite copies)
- Moving large amounts of data between locations reliably and resumably
- Replacing scp in scripts (scp is deprecated; rclone handles resume,
  verification, and progress properly)

## Installation

```bash
brew install rclone          # macOS
sudo apt install rclone      # Debian/Ubuntu
```

## Configuring a remote

rclone uses named remotes — connection profiles you configure once and
reference by name forever after. Run the interactive wizard:

```bash
rclone config
```

Select `n` for new remote, give it a name (e.g. `seedbox`), then select the
backend type. For an SSH/SFTP server:

- Type: `sftp`
- Host: `ds113081.seedhost.eu`
- User: `god`
- Port: `22` (default, just press Enter)
- Password: leave blank if you use SSH key auth
- Key file: set this explicitly — see SSH key authentication below

Once saved, the remote is available by name in all rclone commands.

Config lives at `~/.config/rclone/rclone.conf`. If running rclone as root
(e.g. from a cron job via sudo), copy the config:

```bash
sudo mkdir -p /root/.config/rclone
sudo cp ~/.config/rclone/rclone.conf /root/.config/rclone/rclone.conf
```

## SSH key authentication

rclone does not use the macOS SSH agent. It reads key files directly using
its own SSH library, which only reliably supports the older PEM format —
not the modern OpenSSH private key format that recent versions of ssh-keygen
produce by default.

If you point rclone at a standard modern key and it falls back to password
auth with no clear error, this is why.

### Generate a dedicated rclone key

```bash
ssh-keygen -t rsa -b 4096 -m PEM -f ~/.ssh/rclone_rsa -N "" -C "rclone"
```

No passphrase (`-N ""`), RSA 4096, explicitly PEM format. Keep this key
separate from your main SSH key.

### Add the public key to the remote host

```bash
ssh-copy-id -i ~/.ssh/rclone_rsa.pub user@host
```

### Set the key in your rclone remote config

Either via `rclone config` (set the `key_file` field), or edit
`~/.config/rclone/rclone.conf` directly:

```ini
[hetzner-vps]
type = sftp
host = vps-hetzner-01.warthog-rockhopper.ts.net
user = egeidal
key_file = ~/.ssh/rclone_rsa
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum
```

## Core commands

### copy
Copies source to destination. Skips files that already exist and are identical.
Does not delete anything at the destination.

```bash
rclone copy /local/path seedbox:/remote/path -P
```

### sync
Makes destination identical to source. **Deletes files at destination that
don't exist at source.** Use with care.

```bash
rclone sync /local/path seedbox:/remote/path -P
```

### move
Copies files to destination then deletes them from source.

```bash
rclone move /local/path seedbox:/remote/path -P
```

### ls / lsd / lsl
List files, directories, or files with timestamps.

```bash
rclone lsd seedbox:/remote/path       # list directories
rclone ls seedbox:/remote/path        # list files
rclone lsl seedbox:/remote/path       # list with timestamps and sizes
```

### check
Verifies that source and destination are identical without transferring anything.

```bash
rclone check /local/path seedbox:/remote/path
```

## Real examples

Copy a local folder to the seedbox over SFTP:
```bash
rclone copy /Volumes/Storage/Airport seedbox:/home/god/downloads/otw/ -P
```

Sync a folder to Hetzner Object Storage:
```bash
rclone sync /opt/docker/backups hetzner:my-bucket/backups -P
```

Copy from seedbox to local, limiting to 4 parallel transfers:
```bash
rclone copy seedbox:/home/god/downloads/ /Volumes/Storage/ -P --transfers=4
```

Dry run to see what would happen without actually doing it:
```bash
rclone copy /source/ seedbox:/dest/ --dry-run
```

## Useful flags

| Flag | What it does |
|------|-------------|
| `-P` / `--progress` | Live progress — files transferred, speed, ETA, overall size |
| `--transfers=N` | Number of parallel file transfers (default 4) |
| `--dry-run` | Show what would be copied/deleted without doing it |
| `--checksum` | Compare by checksum instead of size+date (slower, more accurate) |
| `--ignore-existing` | Skip files that exist at destination regardless of content |
| `--max-age=24h` | Only transfer files newer than the given age |
| `--exclude=PATTERN` | Exclude files matching a pattern (e.g. `--exclude="*.tmp"`) |
| `--bwlimit=10M` | Limit bandwidth (e.g. `10M` = 10 MB/s) |
| `--log-file=/path` | Write output to a log file (useful for cron jobs) |
| `--retries=N` | Number of retries on failure (default 3) |

## Notes

- Unlike rsync, rclone does not use `user@host:/path` syntax directly.
  Always configure a named remote and reference it by name.
- rclone's `-P` progress is far more useful than rsync's `--progress` —
  shows overall transfer stats, ETA, and per-file progress simultaneously.
- `copy` is the safe default. Use `sync` only when you explicitly want
  destination to mirror source (it deletes files).
- scp is officially deprecated in OpenSSH. rclone over SFTP is the modern
  replacement for scripts that need to move files to remote servers.