e6edea78f7ab30eca1c67739f76938bd2c1c4e80
howto/nixos.md
| ... | ... | @@ -0,0 +1,175 @@ |
| 1 | +# NixOS |
|
| 2 | + |
|
| 3 | +NixOS is a declarative Linux distribution based on the Nix package Manager. In this post I'll explain how I setup dn42 in this environment. I currently only peer with wireguard and use bird2. NixOS uses configuration files to manage the system state and has a builtin container module. |
|
| 4 | + |
|
| 5 | +## container disclaimer |
|
| 6 | + |
|
| 7 | +I had a spare IPv4 Address so I decided to use a container without a NAT and keep my host "clean" from dn42 Wireguard Interfaces and IP routes. However it's pain full to debug since nixos-rebuild restarts the container on every minor change. So every time you change a firewall rule or debug a DNS setting nixos-rebuild restarts the container before the change takes effect and since BGP is BGP, it can be really frustrating. |
|
| 8 | + |
|
| 9 | +You may also want to have a look at this [Issue](https://github.com/NixOS/nixpkgs/issues/43652) and [Pull Request](https://github.com/NixOS/nixpkgs/pull/80169) |
|
| 10 | + |
|
| 11 | +If you still want to give it a try, here you'll find some inspiration from my setup. You can also use some of these nix expression in a non-container environment. |
|
| 12 | + |
|
| 13 | +## Building the container |
|
| 14 | + |
|
| 15 | +Defining the container environment is the base part of the setup. Beginning with network setup, Private Network disables the passthrough of Host Interfaces into the container and adds a bridged Interface to the host default Interface (e.g. eth0). The localAddress is the container side address and the hostAddress is the one the Host gets. Inside the ```container.<name>.config```, you can basicly import the same nix expression as from the Host and don't need to add some special container parts. |
|
| 16 | + |
|
| 17 | +```nix |
|
| 18 | + containers.dn42 = { |
|
| 19 | + hostAddress = "192.168.254.1"; # Transfer Network |
|
| 20 | + hostAddress6 = "2001:db08::42"; # Transfer Network |
|
| 21 | + localAddress = "116.203.1.5"; |
|
| 22 | + localAddress6 = "2a01:4f8:c0c:4f7a::2/128"; |
|
| 23 | + privateNetwork = true; |
|
| 24 | + autoStart = true; |
|
| 25 | + |
|
| 26 | + config = { config, pkgs, ... }: { |
|
| 27 | + imports = [ |
|
| 28 | + ./peers # Folder with a config for every Peer |
|
| 29 | + ./dns.nix # Bind with the litschi.dn42 zone deligated |
|
| 30 | + ./bird.nix # Bird config for BGP Routing |
|
| 31 | + ./networking.nix # Static Network configuration (with firewall) |
|
| 32 | + ./nginx.nix # nginx config for litschi.dn42 |
|
| 33 | + ]; |
|
| 34 | + environment.systemPackages = with pkgs; [ |
|
| 35 | + # Network debug tools |
|
| 36 | + dnsutils |
|
| 37 | + mtr |
|
| 38 | + tcpdump |
|
| 39 | + wireguard-tools |
|
| 40 | + ]; |
|
| 41 | + } |
|
| 42 | + } |
|
| 43 | +``` |
|
| 44 | + |
|
| 45 | +In theory the container should now be starting and you can get shell access with ```sudo nixos-container root-login <name> ```. |
|
| 46 | + |
|
| 47 | +I mounted some host paths into the container for dns zone files and static homepage since the container is the only one providing .dn42 webservers. |
|
| 48 | + |
|
| 49 | +```nix |
|
| 50 | + containers.dn42 = { |
|
| 51 | + bindMounts = { |
|
| 52 | + "/var/www/dn42" = { |
|
| 53 | + hostPath = "/var/www/dn42"; |
|
| 54 | + isReadOnly = true; |
|
| 55 | + mountPoint = "/var/www/dn42"; |
|
| 56 | + }; |
|
| 57 | + "/var/dns/dn42" = { |
|
| 58 | + hostPath = "/var/dns/dn42"; |
|
| 59 | + isReadOnly = true; |
|
| 60 | + mountPoint = "/var/dns"; |
|
| 61 | + }; |
|
| 62 | + }; |
|
| 63 | + } |
|
| 64 | +``` |
|
| 65 | + |
|
| 66 | +### Network Setup |
|
| 67 | + |
|
| 68 | +As mentioned above, I got a spare public IPv4 Address, but by adding it as ```localAddress```, the container Part is configured static enough. But to forward traffic between Intferfaces ```/proc/sys/net/``` should configured |
|
| 69 | + |
|
| 70 | +```nix |
|
| 71 | + boot.kernel.sysctl = { |
|
| 72 | + "net.ipv4.ip_forward" = 1; |
|
| 73 | + "net.ipv6.conf.all.forwarding" = 1; |
|
| 74 | + }; |
|
| 75 | +``` |
|
| 76 | +This allows our firewall to configure forwarding between peers and other tunnels. What is allowed to be forwarded can be configured in the firewall. Ferm has only few NixOS Options, but is pretty basic. Its configured with the ```services.ferm.config``` options, that contains just a string. Within this string there's standard plain ferm config. Example config is attached below. |
|
| 77 | +If the dn42 address is not bound at any other Interface, you need to add it to the lo Interface to use it as source IP when routing via peers with dedicated transfer net. |
|
| 78 | +```nix |
|
| 79 | + networking.interfaces.lo = { |
|
| 80 | + ipv4.addresses = [ |
|
| 81 | + { |
|
| 82 | + address = "172.23.73.65"; |
|
| 83 | + prefixLength = 32; |
|
| 84 | + } |
|
| 85 | + ]; |
|
| 86 | + ipv6.addresses = [ |
|
| 87 | + { |
|
| 88 | + address = "fd67:24bd:a1ea::1"; |
|
| 89 | + prefixLength = 128; |
|
| 90 | + } |
|
| 91 | + ]; |
|
| 92 | + }; |
|
| 93 | +``` |
|
| 94 | + |
|
| 95 | +#### Ferm example |
|
| 96 | +```nix |
|
| 97 | +services.ferm = { |
|
| 98 | + enable = true; |
|
| 99 | + config = '' |
|
| 100 | + domain ip table filter chain INPUT proto icmp ACCEPT; |
|
| 101 | + domain ip6 table filter chain INPUT proto (ipv6-icmp icmp) ACCEPT; |
|
| 102 | + domain (ip ip6) table filter { |
|
| 103 | + chain INPUT { |
|
| 104 | + policy DROP; |
|
| 105 | + interface lo ACCEPT; |
|
| 106 | + interface intern-+ ACCEPT; |
|
| 107 | + # website |
|
| 108 | + proto tcp dport (http https) ACCEPT; |
|
| 109 | + # wireguard |
|
| 110 | + proto udp dport ( <Wireguard Ports> ) ACCEPT; |
|
| 111 | + # bgp |
|
| 112 | + proto tcp dport (179) ACCEPT; |
|
| 113 | + # dns |
|
| 114 | + proto (udp tcp) dport domain ACCEPT; |
|
| 115 | + mod state state (INVALID) DROP; |
|
| 116 | + mod state state (ESTABLISHED RELATED) ACCEPT; |
|
| 117 | + } |
|
| 118 | + chain OUTPUT { |
|
| 119 | + policy ACCEPT; |
|
| 120 | + } |
|
| 121 | + chain FORWARD { |
|
| 122 | + policy DROP; |
|
| 123 | + # allow intern routing and dn42 forwarding |
|
| 124 | + interface dn42-+ outerface dn42-+ ACCEPT; |
|
| 125 | + interface intern-+ outerface intern-+ ACCEPT; |
|
| 126 | + interface intern-+ outerface dn42-+ ACCEPT; |
|
| 127 | + # but dn42 -> intern only with execptions |
|
| 128 | + interface dn42-+ outerface intern-+ { |
|
| 129 | + proto (ipv6-icmp icmp) ACCEPT; # Allow SSH Access from dn42 to devices behind intern-+ Interfaces |
|
| 130 | + proto tcp dport (ssh) ACCEPT; |
|
| 131 | + mod state state (ESTABLISHED) ACCEPT; |
|
| 132 | + } |
|
| 133 | + } |
|
| 134 | + } |
|
| 135 | + ''; |
|
| 136 | + }; |
|
| 137 | +``` |
|
| 138 | + |
|
| 139 | +### Peering with wireguard |
|
| 140 | + |
|
| 141 | +Explained above, every peer gets a dedicated wireguard Interface and so a dedicated file. In the container config folder theres a peer subfolder and within a folder for dn42- (extern) Peers and intern- configs e.g. my Home Router or mobile devices. |
|
| 142 | + |
|
| 143 | +A sample wireguard config may look like this: |
|
| 144 | +```nix |
|
| 145 | +{config, pkgs, ...}: |
|
| 146 | +{ |
|
| 147 | + networking.wireguard.interfaces.dn42-peer = { |
|
| 148 | + privateKey = ""; |
|
| 149 | + allowedIPsAsRoutes = false; |
|
| 150 | + listenPort = 42420; |
|
| 151 | + |
|
| 152 | + peers = [ |
|
| 153 | + { |
|
| 154 | + publicKey = ""; |
|
| 155 | + allowedIPs = [ "0.0.0.0/0" "::/0" ]; |
|
| 156 | + endpoint = "42.42.42.42:42421"; |
|
| 157 | + } |
|
| 158 | + ]; |
|
| 159 | + postSetup = '' |
|
| 160 | + ${pkgs.iproute}/bin/ip addr add 169.254.0.1/32 peer 169.254.0.0/32 dev dn42-peer |
|
| 161 | + ${pkgs.iproute}/bin/ip -6 addr add fe80::1220/64 dev dn42-peer |
|
| 162 | + ''; |
|
| 163 | + }; |
|
| 164 | +} |
|
| 165 | +``` |
|
| 166 | + |
|
| 167 | +As seen, the IP configuration is applied via ip-commands in the postSetup. This kinda works but isn't a fancy solution. There's room for improvements e.g. configuring static addresses and routes with networkd. |
|
| 168 | + |
|
| 169 | +### BGP Routing with bird2 |
|
| 170 | + |
|
| 171 | +Like ferm, Bird2 is configured by ```services.bird2.config``` containing a string. In there the example bird2 config from [wiki.dn42](https://wiki.dn42/howto/Bird2) can be imported. Roa tables can be generated or downloaded from host providing them. |
|
| 172 | + |
|
| 173 | +### services |
|
| 174 | + |
|
| 175 | +I also run services like a nameserver for .litschi.dn42 zones and a nginx webserver within this container. Since Host path for ```/var/www/dn42``` and ```/var/dns/dn42``` are booth binded into the container, zone config and e.g. website and be edited directly from Host without need the rebuild the hole container. |