1 # [Make your own distro](#) 2 ## — 17 January, 2016 3 4 Happy new year everyone! For the new year I've decided to focus more on 5 projects, and less on "ricing" which bores me more and more... 6 So here is a fun project I'm working on: Making my own linux distro! 7 8 I consider building a linux distribution from scratch to be one of the coolest 9 things one could do with their computers. And I finally got into it! 10 11 I've been writing my "tales" in my 12 [journal](gopher://z3bra.org/0/distro/journal.txt). Be sure to read it for a 13 more "day-to-day" like approach. 14 This post is only meant to detail the tools used and the workflow I have when 15 I work on it. 16 17 First off all, I have never built an [LFS](https://linuxfromscratch.org) system, 18 and will probably never do (their documentation is great though). 19 20 ### Scope 21 22 So here I am, sitting in front of my computer, deciding what this "new distro" 23 should be. I've always liked the idea of [sta.li](http://sta.li), an similar 24 distributions ala [rlsd2](http://rlsd2.dimakrasner.com). Minimalist systems 25 trying to bring fresh new ideas on the table of linux distributions. So I 26 decided to list the things I wanted in my distro: 27 28 * statically compiled 29 * stripped down userland 30 * simplified file system 31 * port-tree featuring plan9's mk(1) 32 * featuring new/obscure softwares when possible 33 34 I chose each point because I though it would be and interresting challenge, 35 allowing me to learn more about linux distributions in the process. I'm not 36 trying to argue that X is better than Y, or anyhing amongst those lines. 37 38 ### Toolchains 39 40 There are, from my experience, three important toolchains when building a 41 distro: 42 43 * Compilation toolchain 44 * Package management toolchain 45 * Testing toolchain 46 47 Each one is as important as the two others, because they will define how 48 efficient you'll be for the whole project. 49 50 #### Compilation toolchain 51 I am using [pcc](http://pcc.ludd.ltu.se) as my complier of choice, and 52 [musl](http://musl-libc.org) as my libc. 53 54 To this day, I couldn't find a suitable replacement for GNU's binutils, but am 55 still looking forward to elfutils and elftoolchain. 56 57 To make it easier to work with this toolchain, I decided to build a 58 cross-compiler featuring both. I also had to create one using gcc, until I can 59 figure out a patch for softwares that can't compile with pcc. 60 The script used to generate the cross-compiler can be found 61 [here](http://git.z3bra.org/cross/file/cross-gcc.html). 62 63 In order to use it, add the path to the toolchain in your path. Calling 64 `x86_64-linux-musl-gcc` will automatically call libs/include from the cross 65 prefix. 66 67 <video controls> 68 <source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-xcompiler.webm" type="video/webm"> 69 </video> 70 <span class='caption'>Using the cross-compiler to build a software statically 71 against musl libc.</span> 72 73 74 #### Package management toolchain 75 There are a lot of them. But the most challenging one will be the one I'll write 76 myself. This is why I wrote [pm](http://git.z3bra.org/pm/log.html), my pack 77 manager. It's still a work in progress, but it can install, delete and inspect 78 packs in a given directory, which is perfect for my needs. 79 80 For the packaging of softwares, I decided to go with an mk(1) based port-tree 81 ala BSD. Inspired by the [morpheus port-tree](http://git.2f30.org/ports/log.html), 82 it follows the way the *BSD handle their own port-tree, while using mk(1) from 83 plan9, which is, in my opinion, more predictable. For more information on mk, 84 check the link [maintaining files on plan9 with 85 mk](http://plan9.bell-labs.com/sys/doc/mk.html). 86 87 The ports are available [here](http://git.z3bra.org/mkports/log.html). To build 88 a port, you need to configure the `config.mk` file to match your environment, and 89 run `mk <port>`. This will build all the deps required to build you port, and 90 install them under your cross-compilation root directory. 91 To clean the tree from this deps, run `mk clean`. The tarball will be packed in the 92 port directory of your tool, eg `pm/pm#1.0.tar.bz`. Theses file are suitable for 93 installation via [pm](http://git.z3bra.org/pm/log.html). 94 95 <video controls> 96 <source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-mkports.webm" type="video/webm"> 97 </video> 98 <span class='caption'>Show casing the creation of a port within the mk port-tree 99 and building/installing it under a specific directory</span> 100 101 #### Testing toolchain 102 To actually test the distro, you need a way to boot from it. I chose to use linux 103 containers (LXC) over traditionnal VMs (virtualbox, qemu) as I'm not playing with 104 different CPU architectures (yet!). 105 This way I can treat my distro as a simple chroot and modify it from outside in real 106 time. It also let me play with those containerization technologies I'm not 107 familiar with. 108 109 The first thing to do is to install the packs under your chroot. For this purpose, I 110 chose `/ns/pm/rootfs`. After installing a few packs, you can fire up a container 111 using this directory as its root to check wether the distro boots, and if the tools 112 you installed work as expected. 113 114 As of today, spawning gettys doesn't work. I'll need to figure out why, but otherwise, 115 it's been a pretty successful (and fun!) experience. 116 117 <video controls> 118 <source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-containers.webm" type="video/webm"> 119 </video> 120 <span class='caption'>Demonstrating the whole testing process, from creating the 121 containers and saying "hello world!" from within this container</span> 122 123 In the process, I learnt how to spawn containers "the hard way". Here is a quick 124 peak at it: 125 126 ip link add veth0 type veth peer name eth0 127 ip netns add foo 128 ip link set eth0 netns foo 129 ip netns exec foo ip addr add 192.168.0.100/24 dev eth0 broadcast + 130 ip netns exec foo ip link set eth0 up 131 ip netns exec foo ip route add default via 192.168.0.254 132 brctl addif br0 veth0 133 ip netns exec unshare -fpium --mount-proc env -i chroot /ns/pm/rootfs /sbin/init 134 135 This is far from a final-run-everywhere solution of course, because you'll miss /dev 136 mounts, ttys and such, and won't be able to log within this container, or inject 137 processes in to it. 138 139 But. 140 141 That's neat.