commit 2ca6ccd8796707dd8231ce578da9183153b8632f
author Skyler Grey <minion@clicks.codes> Sat Oct 14 22:56:43 2023 +0000
committer Skyler Grey <minion@clicks.codes> Sat Oct 21 01:17:41 2023 +0000
tree 12cfe8db12313e5c7e0fce64485af7a9c5066564
parent a7cc4e9da15538085ae617919cf8f3ff8c3acf09

Replace caddy with nginx

Before this commit, we used Caddy. Here's what was needed to make that setup go:
- As we needed the layer4 plugin, we had to compile caddy with layer4 using an
  unsupported technique in nix
- As the layer4 plugin did not support caddyfile, we had to use caddy.json
  - We wrote some nix code to setup reverse proxy routes easily however this was
    a challenge to reasonably maintain, and caddy.json is far
    less-well-documented than caddyfile leading to a significant amount of pain
    if we ever needed to break out
- Several modules in NixOS use nginx by default (privatebin, nextcloud etc.), we
  need to disable and then replicate their setup. This is often nontrivial

Nginx has some distinct advantages for Clicks specifically on NixOS:
- "Streams" are supported Out-Of-The box, meaning no layer4 plugin
- Nginx has a standard nixified interface for virtual hosts, meaning there's no
  breaking out into nginx config for http
  - Note how you may still have to break out for TCP/UDP streams
- Nginx configurations, including relatively complex ones (e.g. nextcloud) are
  often included with modules by default. Choosing nginx will avoid us rewriting
  these, as well as all the pain and debugging that comes with that

Additionally:
- Nginx has excellent integration with the builtin ACME module. I am satisfied
  with its ability to replace caddy on that front
- A major point in the favor of caddy is usability. Unfortunately, this really
  only seems to apply to caddyfile with caddy.json being significantly more
  cumbersome to write, even with nix helper functions
- It is trivial to write some simple but decent helper functions for nginx

Alternatives considered:
- Stick with caddy
  - Too much maintenance burden
  - We struggle to update it frequently, due to the plugin install bits
- Use traefik
  - From a short look, it's too complex for our needs
  - It doesn't have configs built into nix in the same way as nginx
- Use apache2
  - Apache has no analogue to nginx streams
  - It doesn't have configs built into nix in the same way as nginx

This commit will need a followup in future to improve the typing of the Service
type. We have a very basic implementation but it would be nice to be able to
check for types that are inside it rather than just the top level.

Change-Id: I25e7ba48cec6b9308e6aa9a14f57a8c192918c92