Bootstrapping Sortix 1.1 on Sortix 1.0

(Draft - not published)

Sortix 1.0 is a self-hosting operating system that can build new versions of its base system, but it isn't able to build all its ports of third party software natively. Sortix 1.1 on the other hand is fully self-hosting and the releases including all the ports are built natively.

It's possible to bootstrap a full Sortix 1.1 system on Sortix 1.0, even though the differences are significant, thanks to foresight and careful compatibility. Here's how it can be done in a few phases, incrementally becoming Sortix 1.1.

Installing 1.0

The first step is to simply install Sortix 1.0.

Out of the box, it already contains a gcc-5.2.0 toolchain with C and C++ support with the system source code in /src, ready for tinkering. It's easy to recompile the base operating systems (without ports) and upgrade the current installation by running:

cd /src && make sysmerge

This is how Sortix 1.1 will be built but the difficulty lies in the ports.

Obtaining the new source code

Next let's obtain the Sortix 1.1 source code.

Sortix 1.1 isn't released yet, so for now, I'm using the staging branch as it contains the new ports system, which will soon be merged to master. The master branch is also self-hosting, but the new ports system introduces some new challenges that we're exploring here.

Sortix 1.0 did not have networking and another operating system must be used to help obtain the new source code.

The tarballs of the upstream releases cannot be downloaded at build time and needs to be cached ahead of time.

cd /src && make mirror

Sortix 1.0 contains a defective patch(1) port since GNU patch assumed the O_RDONLY constant is 0 which is not true on Sortix, so we'll need to extract patch with patches applied in order to bootstrap a new patch(1).

cd /src && make extract-ports PACKAGES=patch

The source code can be be transferred into the installation using one of these methods:

• Booting a Sortix 1.1 live environment on the machine which comes with the source code.
• Mounting the installation's filesystem from another operating system.
• Transferring it over the serial line using serial-transfer(7).
• Entering all the new source code into the editor.

Sortix 1.1 with old ports

Once the new source code is in /src, let's upgrade Sortix 1.0 to a Sortix 1.1 base system with the Sortix 1.0 ports.

Sortix 1.1 is intentionally largely ABI compatible with Sortix 1.0. There's been a few incompatible ABI changes but they've intentionally been limited to networking, which won't affect the system build.

The build tools used during the system build are intentionally backwards compatible with the previous stable release and makes it possible to build the new release:

make install-build-tools

The new base system needs to be built without any ports since most of them don't build with the 1.0 base system.

make sysmerge-wait PACKAGES=

The sysmerge-wait makefile target rebuilds the system and schedules a system upgrade on the next boot. Reboot Sortix to become the new Sortix 1.1 base system with 1.0 ports. The boot will warn about the lack of a /boot/random.seed file which will fixed soon after when the bootloader configuration is upgraded.

Sortix 1.1 with mixed ports

Sortix 1.0 ships several buggy and outdated ports that needs to be bootstrapped in the right order in order to build the Sortix 1.1 ports.

patch

patch in Sortix 1.0 is broken and needs to be updated using the source code which was previously extracted and patched on another operating system. Otherwise ports will fail to be patched after extraction.

make sysmerge PACKAGES=patch

This command will build and install the patch port using the cache of upstream tarballs. It'll reuse the preextracted patch directory since it is the correct version.

The port can be installed through an in-place system upgrade, rather than upgrading on reboot as before, since the kernel didn't change and no there are no ABI changes.

gawk

gawk in 1.0 has a bug that writes integers as %.0f due to printf not supporting formatting floating point numbers:

make sysmerge PACKAGES=gawk

dash

dash in 1.0 has a bug where it can stack overflow because it uses alloca instead of malloc:

make sysmerge PACKAGES=dash

sed

sed in 1.0 has a bug that doesn't handle lines without a terminating newline:

make sysmerge PACKAGES=sed

m4

m4 in 1.0 is defective:

make sysmerge PACKAGES=m4!!

The exclamation mark in m4! builds m4 along with its mandatory and optional dependencies.

flex

flex in 1.0 fails to handle character ranges properly which is needed to build wget and maybe others.

make sysmerge PACKAGES=flex!!

make

make needs to be updated to avoid a crash in realpath(3) when building qemu:

make sysmerge PACKAGES=make!!

perl

perl is a new port added after 1.0 and is needed to build some ports:

make sysmerge PACKAGES=perl!!

texinfo

texinfo is a new port added after 1.0 and is needed to build some ports:

make sysmerge PACKAGES=texinfo!!

pkg-config

pkg-config in 1.0 has a bug where it fails in the C locale. pkg-config requires a functional pkg-config to build itself and its dependencies, so the old pkg-config needs to be wrapped with a workaround.

mkdir /tmp/pkg-config
echo '#!/bin/sh' > /tmp/pkg-config/pkg-config
echo 'unset LC_ALL' >> /tmp/pkg-config/pkg-config
echo 'exec /bin/pkg-config "$@"' >> /tmp/pkg-config/pkg-config
chmod +x /tmp/pkg-config/pkg-config
dash -c 'ln -s pkg-config /tmp/pkg-config/$(uname -m)-sortix-pkg-config'
PATH="/tmp/pkg-config:$PATH" make sysmerge PACKAGES='pkg-config!!'
rm -rf /tmp/pkg-config

libssl

libssl needs to be updated in order to bootstrap the new python.

make sysmerge PACKAGES=libssl!!

python

python needs to be updated in order to build qemu.

make sysmerge PACKAGES=python!!

Sortix 1.1 ports

It's now possible to do a full system rebuild with all the new ports.

make sysmerge-full

The new ports are installed onto the root filesystem, and the full sysmerge will uninstall any ports that don't exist anymore. The upgrade is performed in-place and the new operating system is running after the command finishes.

Sortix 1.1

Sortix 1.1 is fully self-hosting and can rebuild all its ports out of the box. It's best to do a second full system rebuild, since the new ports were built with the old ports which may cause bugs not present in the actual release. Some of the ports in the mixed phase were also built without optional dependencies.

make mostlyclean clean-repository
make sysmerge-full

Congratulations on your fully featured self-hosting Sortix 1.1 operating system!