Self-hosting an automation platform for free

"Mom, can I have Zapier?" "No, we have automation at home."

Do you ever need to run simple automations for personal projects? What do you use?

Zapier is terrific. It’s powerful and it provides a generous free tier. But when you need something more complex than simple if-then logic, like “send me an email when this event happens, but only if the event data contains an order ID”, you run into the free tier’s limits pretty quickly.

If you need one complicated automation (or more than five simple ones), for me, Zapier doesn’t make sense for projects that don’t make money.

But guess what! There are some great self-hosted options. In particular, n8n is an automation platform built on Node with a commercial tier, but it’s “source-available” and pretty robust.

And guess what else! You can actually host it for free using fly.io.

Fly is a hosting service that works by turning Dockerfiles into proper, honest-to-goodness virtual machines. It feels as easy to use as Heroku, with a fancy command line client and even SSH access to your VMs.

Their free tier includes three quite low-powered VMs running 24/7 (1 shared virtual CPU and 256MB RAM), and up to 3gb of attached storage. It’s not much, but it’s enough to run n8n! So we can have our own Zapier-like automation tool for small automations. (If I outgrow my private n8n instance I’ll happily pay for Zapier — it’s an awesome tool!)

If you’d like to try the same thing, here’s how I got n8n running on Fly.

1. Install flyctl and sign up

If you haven’t already, install flyctl, the Fly CLI. On a Mac with Homebrew installed, it’s a quick brew install flyctl to get going. (flyctl is available on other platforms, though. Check out their getting-started guide for installation and signup instructions.)

After it’s installed, you can set up an account and authorize the CLI client by running fly auth signup.

2. Initialize a new project

Create and navigate to a new directory where your fly.toml file (and Git repository, if you’re so inclined) will live.

mkdir mynewproject && cd mynewproject

n8n provides a Docker image, and Fly supports launching from an image as well as local code! So all we really need here is a fly.toml file, which flyctl will generate for us:

fly launch

This will walk you through the process of initializing a new Fly app. It will prompt you to name your app, and choose the region where your app will run.

3. Create a new volume

Because Fly VMs are ephemeral, n8n needs to store its data in a volume that persists between deploys and restarts.

Let’s create one:

fly volumes create n8n_data --size=1

This creates a 1GB volume (--size=1) named n8n_data. The default size is 3GB; I went with 1 just in case I want to launch more applications within the free tier as I’m experimenting with Fly. (For my very basic use case, 1GB is also plenty.)

4. Customize fly.toml

The fly launch command created a fly.toml for us and added an application to our Fly account. (You can view details by logging into the Fly interface, or by running fly status in your project directory.)

But it didn’t actually run anything, since we haven’t written any code for it to run (or pointed it to a container to run)! So we basically have an empty application and no running machines.

We’ll use fly.toml to point Fly at a container image; we will also use it to configure the application, by e.g. setting environment variables.

You can see a complete reference to fly.toml in their documentation.

n8n also provide lots of information for its configuration options in their documentation.

There are a few things we need to do:

  • Tell Fly what docker image to build
  • Set n8n environment variables
  • Tell Fly how our application is accessed (ports, protocols, concurrency, etc.)
  • Tell fly where our volumes are for persistent storage (as Fly machines are ephemeral)

Here’s each section of my fly.toml, annotated.

Basic config

app = "lively-grass-5918" # auto-assigned by Fly, unless you specified a name when you created the application.
primary_region = "iad"

[build]
  image = "n8nio/n8n:latest"Code language: TOML, also INI (ini)

The build section here tells Fly which Docker image to pull. It’s best to set it to a specific version (e.g. n8nio/n8n:0.232.0) as upgrades can break things, and if you’re on latest, upgrades happen whenever you deploy.

Next:

Environment variables

[env]
  EXECUTIONS_PROCESS = "main"
  N8N_DIAGNOSTICS_ENABLED = false
  N8N_LOG_LEVEL="debug"
  N8N_LOG_OUTPUT="console"
  N8N_EDITOR_BASE_URL = "https://lively-grass-5918.fly.dev"
  N8N_SMTP_HOST = "your-smtp-provider.com"
  N8N_SMTP_USER = "[email protected]"
  TZ = "America/New_York"
  WEBHOOK_URL = "https://lively-grass-5918.fly.dev"Code language: TOML, also INI (ini)

EXECUTIONS_PROCESSES="main" tells n8n to do everything on the main thread. Normally it runs things in parallel, but since we’re on the smallest VM, using multiple threads seems to crash n8n. Or, it did in my testing awhile back. So we do everything on the main thread, which means every automation is a blocking process.

So far this hasn’t been a problem for me as my automation doesn’t run often, and it’s not super time-sensitive. I’d upgrade to hosted n8n or Zapier if I needed more juice though.

The other environment variables:

  • Disable n8n’s telemetry, which is on by default
  • Set our log level and output for help debugging (you can view your app’s STDOUT in the Fly dashboard or via fly logs)
  • Set the “editor base URL” — that is, what URL will we use to access the n8n interface?
  • Set the SMTP credentials n8n should use to send us emails re: account signups, or any other system alerts
  • Set our timezone
  • Set the URL that other services will hit when calling our self-hosted webhooks

Services

We can define how Fly will route traffic to our app:

[[services]]
  protocol = "tcp"
  internal_port = 5678
  processes = ["app"]

  [[services.ports]]
    port = 80
    handlers = ["http"]
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]

  [[services.tcp_checks]]
    interval = "15s"
    timeout = "2s"
    grace_period = "1s"
    restart_limit = 0
Code language: TOML, also INI (ini)

It should route traffic from ports 80 and 443 to the internal TCP port 5678. We also tell it how often to run its app health checks, but this is optional.

If you’d like, you can also set a hard limit on concurrent connections under services.ports:

  [services.concurrency]
    type = "connections"
    hard_limit = 2500
    soft_limit = 2000Code language: JavaScript (javascript)

Volumes

Fly also needs to know which volumes to connect to the application:

[[mounts]]
  source = "n8n_data"
  destination = "/home/node/.n8n"
  processes = ["app"]Code language: TOML, also INI (ini)

This points Fly at the volume we created earlier, and binds it to the /home/node/.n8n directory in the application, where n8n stores its sqlite database. (You could use Fly Postgres if you wanted to, though.)

Final result

Here’s what the complete fly.toml looks like:

app = "lively-grass-5918" # auto-assigned by Fly, unless you specified a name when you created the application.
primary_region = "iad"

[build]
  image = "n8nio/n8n:latest"

[env]
  EXECUTIONS_PROCESS = "main"
  N8N_DIAGNOSTICS_ENABLED = false
  N8N_LOG_LEVEL="debug"
  N8N_LOG_OUTPUT="console"
  N8N_EDITOR_BASE_URL = "https://lively-grass-5918.fly.dev"
  N8N_SMTP_HOST = "your-mail-provider.test"
  N8N_SMTP_USER = "[email protected]"
  TZ = "America/New_York"
  WEBHOOK_URL = "https://lively-grass-5918.fly.dev"

[[mounts]]
  source = "test_data"
  destination = "/home/node/.n8n"
  processes = ["app"]

[[services]]
  protocol = "tcp"
  internal_port = 5678
  processes = ["app"]

  [[services.ports]]
    port = 80
    handlers = ["http"]
    force_https = true

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]
Code language: TOML, also INI (ini)

5. Deploy!

Save your fly.toml and in your project’s directory, run:

fly deploy

If everything goes according to plan, you should be able to navigate to your app’s URL and create your first n8n account!

There are some additional steps to get a proper URL pointed at your n8n instance. The documentation on the topic is great. It will depend on your DNS provider. But once you have your HTTPS cert provisioned for your domain and Fly is seeing the custom domain for the app, you should be able to update the [env] section of fly.toml to use your new domain, then fly deploy once more.

Troubleshooting

I noticed that n8n would crash on the default Fly machine with only 256MB of RAM. I’m not sure why this happened; Fly’s metrics show that the machine has a bit of spare capacity. But, one or two test runs of my automation would crash n8n and Fly would reboot it.

Scaling the machine’s RAM up solved the issue for me:

fly scale memory 512

That should scale your machine and reboot your application. I imagine that if my instance got enough traffic, I’d need to scale it up again, but so far Node and n8n are running smoothly with 512mb.

Happy trails

From there, you can start building your automations!

n8n isn’t as mature and robust and easy-to-use as something like Zapier. But it’s still great! The existing integrations work well enough, and there’s a lot of power in the fact that it lets you write code if you need something not already built-in.

For instance, I have to use it to pull data out of a webhook call. The JSON from the webhook is full of deeply-nested data structures, but I only need a tiny subset of the data. Some automation platforms I’ve used expect you to either poke your way through (usually quite limited) interfaces to get the data you want, and leave you no option if their GUI doesn’t have affordances for what you’re trying to do; others expect you to learn funky query languages to get at data deep in JSON.

n8n, on the other hand, lets me write some simple JavaScript as a step in the automation, which then passes its result to the next step. It’s great!

Incidentally, n8n also has really nice testing affordances. You can drop in sample data, manipulate it, and test-run your workflows easily without emailing real customers or whatever.

Fly has also been awesome for this. Fly makes it extraordinarily easy to host and scale applications without having to worry about ops. It feels very Heroku-ey, which is great. I set this app up almost a year ago and have barely touched it since; it’s been accepting webhooks without complaint this whole time.

Fly’s documentation is excellent, the CLI is fast and full-featured, and their forums are active (the Fly team even posts there!). If my invoices from them were more than $0.00, I’d happily pay them. I initially set this up on Fly out of curiosity, but I’ll definitely use their hosting service again if any apps I build graduate from my basic DigitalOcean VMs.

Have fun!