monochromatic

monochromatic blog: http://blog.z3bra.org
git clone git://z3bra.org/monochromatic
Log | Files | Refs

make-your-own-distro.txt (6139B)


      1 # [Make your own distro](#)
      2 ## — 17 January, 2016
      3 
      4 Happy new year everyone! For the new year I've decided to focus more on
      5 projects, and less on "ricing" which bores me more and more...  
      6 So here is a fun project I'm working on: Making my own linux distro!
      7 
      8 I consider building a linux distribution from scratch to be one of the coolest
      9 things one could do with their computers. And I finally got into it!
     10 
     11 I've been writing my "tales" in my
     12 [journal](gopher://z3bra.org/0/distro/journal.txt). Be sure to read it for a
     13 more "day-to-day" like approach.
     14 This post is only meant to detail the tools used and the workflow I have when
     15 I work on it.
     16 
     17 First off all, I have never built an [LFS](https://linuxfromscratch.org) system,
     18 and will probably never do (their documentation is great though).
     19 
     20 ### Scope
     21 
     22 So here I am, sitting in front of my computer, deciding what this "new distro"
     23 should be. I've always liked the idea of [sta.li](http://sta.li), an similar
     24 distributions ala [rlsd2](http://rlsd2.dimakrasner.com). Minimalist systems
     25 trying to bring fresh new ideas on the table of linux distributions. So I
     26 decided to list the things I wanted in my distro:
     27 
     28 * statically compiled
     29 * stripped down userland
     30 * simplified file system
     31 * port-tree featuring plan9's mk(1)
     32 * featuring new/obscure softwares when possible
     33 
     34 I chose each point because I though it would be and interresting challenge,
     35 allowing me to learn more about linux distributions in the process. I'm not
     36 trying to argue that X is better than Y, or anyhing amongst those lines.
     37 
     38 ### Toolchains
     39 
     40 There are, from my experience, three important toolchains when building a
     41 distro:
     42 
     43 * Compilation toolchain
     44 * Package management toolchain
     45 * Testing toolchain
     46 
     47 Each one is as important as the two others, because they will define how
     48 efficient you'll be for the whole project.  
     49 
     50 #### Compilation toolchain
     51 I am using [pcc](http://pcc.ludd.ltu.se) as my complier of choice, and
     52 [musl](http://musl-libc.org) as my libc.
     53 
     54 To this day, I couldn't find a suitable replacement for GNU's binutils, but am
     55 still looking forward to elfutils and elftoolchain.
     56 
     57 To make it easier to work with this toolchain, I decided to build a
     58 cross-compiler featuring both. I also had to create one using gcc, until I can
     59 figure out a patch for softwares that can't compile with pcc.
     60 The script used to generate the cross-compiler can be found
     61 [here](http://git.z3bra.org/cross/file/cross-gcc.html).
     62 
     63 In order to use it, add the path to the toolchain in your path. Calling
     64 `x86_64-linux-musl-gcc` will automatically call libs/include from the cross
     65 prefix.
     66 
     67 <video controls>
     68 	<source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-xcompiler.webm" type="video/webm">
     69 </video>
     70 <span class='caption'>Using the cross-compiler to build a software statically
     71 against musl libc.</span>
     72 
     73 
     74 #### Package management toolchain
     75 There are a lot of them. But the most challenging one will be the one I'll write
     76 myself. This is why I wrote [pm](http://git.z3bra.org/pm/log.html), my pack
     77 manager. It's still a work in progress, but it can install, delete and inspect
     78 packs in a given directory, which is perfect for my needs.
     79 
     80 For the packaging of softwares, I decided to go with an mk(1) based port-tree
     81 ala BSD. Inspired by the [morpheus port-tree](http://git.2f30.org/ports/log.html),
     82 it follows the way the *BSD handle their own port-tree, while using mk(1) from
     83 plan9, which is, in my opinion, more predictable. For more information on mk,
     84 check the link [maintaining files on plan9 with
     85 mk](http://plan9.bell-labs.com/sys/doc/mk.html).
     86 
     87 The ports are available [here](http://git.z3bra.org/mkports/log.html). To build
     88 a port, you need to configure the `config.mk` file to match your environment, and
     89 run `mk <port>`. This will build all the deps required to build you port, and
     90 install them under your cross-compilation root directory.
     91 To clean the tree from this deps, run `mk clean`. The tarball will be packed in the
     92 port directory of your tool, eg `pm/pm#1.0.tar.bz`. Theses file are suitable for
     93 installation via [pm](http://git.z3bra.org/pm/log.html).
     94 
     95 <video controls>
     96 	<source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-mkports.webm" type="video/webm">
     97 </video>
     98 <span class='caption'>Show casing the creation of a port within the mk port-tree
     99 and building/installing it under a specific directory</span>
    100 
    101 #### Testing toolchain
    102 To actually test the distro, you need a way to boot from it. I chose to use linux
    103 containers (LXC) over traditionnal VMs (virtualbox, qemu) as I'm not playing with
    104 different CPU architectures (yet!).
    105 This way I can treat my distro as a simple chroot and modify it from outside in real
    106 time. It also let me play with those containerization technologies I'm not
    107 familiar with.
    108 
    109 The first thing to do is to install the packs under your chroot. For this purpose, I
    110 chose `/ns/pm/rootfs`. After installing a few packs, you can fire up a container
    111 using this directory as its root to check wether the distro boots, and if the tools
    112 you installed work as expected.
    113 
    114 As of today, spawning gettys doesn't work. I'll need to figure out why, but otherwise,
    115 it's been a pretty successful (and fun!) experience.
    116 
    117 <video controls>
    118 	<source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-containers.webm" type="video/webm">
    119 </video>
    120 <span class='caption'>Demonstrating the whole testing process, from creating the
    121 containers and saying "hello world!" from within this container</span>
    122 
    123 In the process, I learnt how to spawn containers "the hard way". Here is a quick
    124 peak at it:
    125 
    126 	ip link add veth0 type veth peer name eth0
    127 	ip netns add foo
    128 	ip link set eth0 netns foo
    129 	ip netns exec foo ip addr add 192.168.0.100/24 dev eth0 broadcast +
    130 	ip netns exec foo ip link set eth0 up
    131 	ip netns exec foo ip route add default via 192.168.0.254
    132 	brctl addif br0 veth0
    133 	ip netns exec unshare -fpium --mount-proc env -i chroot /ns/pm/rootfs /sbin/init
    134 
    135 This is far from a final-run-everywhere solution of course, because you'll miss /dev
    136 mounts, ttys and such, and won't be able to log within this container, or inject
    137 processes in to it.
    138 
    139 But.
    140 
    141 That's neat.