Bootstrapping Sortix 1.1 on Sortix 1.0

(Draft - not published)

Sortix 1.0 is a self-hosting operating system that can build new versions of its base system, but it isn't able to build all its ports of third party software natively. Sortix 1.1 on the other hand is fully self-hosting and the releases including all the ports are built natively.

It's possible to bootstrap a full Sortix 1.1 system on Sortix 1.0, even though the differences are significant, thanks to foresight and careful compatibility. Here's how it can be done in a few phases, incrementally becoming Sortix 1.1.

Installing 1.0

The first step is to simply install Sortix 1.0.

Out of the box, it already contains a gcc-5.2.0 toolchain with C and C++ support with the system source code in /src, ready for tinkering. It's easy to recompile the base operating systems (without ports) and upgrade the current installation by running:

cd /src && make sysmerge

This is how Sortix 1.1 will be built but the difficulty lies in the ports.

Obtaining the new source code

Next let's obtain the Sortix 1.1 source code.

Sortix 1.1 isn't released yet, so for now, I'm using the latest development code.

Sortix 1.0 did not have networking and another operating system must be used to help obtain the new source code.

The tarballs of the upstream releases cannot be downloaded at build time and needs to be cached ahead of time.

cd /src && make mirror

Sortix 1.0 contains a defective patch(1) port since GNU patch assumed the O_RDONLY constant is 0 which is not true on Sortix, so we'll need to extract patch with patches applied in order to bootstrap a new patch(1).

cd /src && make extract-ports PACKAGES=patch

The source code can be be transferred into the installation using one of these methods:

• Booting a Sortix 1.1 live environment on the machine which comes with the source code.
• Mounting the installation's filesystem from another operating system.
• Transferring it over the serial line using serial-transfer(7).
• Entering all the new source code into the editor.

Sortix 1.1 with old ports

Once the new source code is in /src, let's upgrade Sortix 1.0 to a Sortix 1.1 base system with the Sortix 1.0 ports.

Sortix 1.1 is intentionally largely ABI compatible with Sortix 1.0. There's been a few incompatible ABI changes but they've intentionally been limited to networking, which won't affect the system build.

The build tools used during the system build are intentionally backwards compatible with the previous stable release and makes it possible to build the new release:

make install-build-tools

The build tools were built with the old libc, clean them before rebuilding the whole base system with the new libc:

make clean-build-tools

The new base system needs to be built without any ports since most of them don't build with the 1.0 base system.

make sysmerge-wait PACKAGES=

The sysmerge-wait makefile target rebuilds the system and schedules a system upgrade on the next boot. Reboot Sortix to become the new Sortix 1.1 base system with 1.0 ports. The boot will warn about the lack of a /boot/random.seed file which will fixed soon after when the bootloader configuration is upgraded.

Sortix 1.1 with mixed ports

Sortix 1.0 ships several buggy and outdated ports that needs to be bootstrapped in the right order in order to build the Sortix 1.1 ports.

patch

patch in Sortix 1.0 is broken and needs to be updated using the source code which was previously extracted and patched on another operating system. Otherwise ports will fail to be patched after extraction.

make sysmerge-now PACKAGES=patch

This command will build and install the patch port using the cache of upstream tarballs. It'll reuse the preextracted patch directory since it is the correct version.

The port can be installed through an in-place system upgrade, rather than upgrading on reboot as before, since the kernel didn't change and no there are no ABI changes.

gawk

gawk in 1.0 has a bug that writes integers as %.0f due to printf not supporting formatting floating point numbers:

make sysmerge-now PACKAGES=gawk

dash

dash in 1.0 has a bug where it can stack overflow because it uses alloca instead of malloc:

make sysmerge-now PACKAGES=dash

sed

sed in 1.0 has a bug that doesn't handle lines without a terminating newline:

make sysmerge-now PACKAGES=sed

m4

m4 in 1.0 is defective:

make sysmerge-now PACKAGES=m4!!

The exclamation mark in m4! builds m4 along with its mandatory and optional dependencies.

flex

flex in 1.0 fails to handle character ranges properly which is needed to build wget and maybe others.

make sysmerge-now PACKAGES=flex!!

make

make needs to be updated to avoid a crash in realpath(3) when building qemu:

make sysmerge-now PACKAGES=make!!

perl

perl is a new port added after 1.0 and is needed to build some ports:

make sysmerge-now PACKAGES=perl!!

texinfo

texinfo is a new port added after 1.0 and is needed to build some ports:

make sysmerge-now PACKAGES=texinfo!!

pkg-config

pkg-config in 1.0 has a bug where it fails in the C locale. pkg-config requires a functional pkg-config to build itself and its dependencies, so the old pkg-config needs to be wrapped with a workaround.

mkdir /tmp/pkg-config
echo '#!/bin/sh' > /tmp/pkg-config/pkg-config
echo 'unset LC_ALL' >> /tmp/pkg-config/pkg-config
echo 'exec /bin/pkg-config "$@"' >> /tmp/pkg-config/pkg-config
chmod +x /tmp/pkg-config/pkg-config
dash -c 'ln -s pkg-config /tmp/pkg-config/$(uname -m)-sortix-pkg-config'
PATH="/tmp/pkg-config:$PATH" make sysmerge-now PACKAGES='pkg-config!!'
rm -rf /tmp/pkg-config

libssl

libssl needs to be updated in order to bootstrap the new python.

make sysmerge-now PACKAGES=libssl!!

python

python needs to be updated in order to build qemu.

make sysmerge-now PACKAGES=python!!

binutils

Update to the latest binutils toolchain.

make sysmerge-now PACKAGES=binutils!!

gcc

Update to the latest gcc toolchain. The gcc in 1.0 doesn't have all the needed feature, and has a bug where it crashes during the vim regexp build.

make sysmerge-now PACKAGES=gcc!!

TODO: Don't we want to throw in a new binutils here for good measure?

Sortix 1.1 full build

It's now possible to do a full system rebuild with all the new ports. First clean everything, so we rebuild the base system and all ports with the new toolchain.

make mostlyclean clean-repository

Then rebuild the whole system:

make sysmerge-full

Reboot to complete the full upgrade. The new ports are installed onto the root filesystem, and the full sysmerge will uninstall any ports that don't exist anymore. The upgrade is performed in-place and the new operating system is running after the command finishes.

Sortix 1.1

Sortix 1.1 is fully self-hosting and can rebuild all its ports out of the box. It's ideal to do a second full system rebuild, since the new ports were built with the old ports which may cause bugs not present in the actual release.

make mostlyclean clean-repository
make sysmerge-full

Congratulations on your fully featured self-hosting Sortix 1.1 operating system!