os-test has been funded by NGI Zero Commons

I'm pleased to announce that Sortix has been selected for a generous grant from the NGI Zero Commons Fund!

In particular, we will be expanding os-test into a much larger POSIX test suite as proposed:

os-test measures interoperability and differences between every POSIX operating system (Linux, BSD, macOS, and many more). This project expands os-test with full coverage for the POSIX standard library for the C programming language. This new test coverage will check that each C header properly provides all the mandated definitions, and that each function succeeds on basic inputs. Detailed new suites will be written for the areas where defects or deviation from the standard are likely, or where edge cases otherwise might not be correctly implemented or even standardized.

os-test continuously publishes test results for every POSIX OS as open data. os-test improves interoperability, since application vendors are able to know what behaviors they can actually use to write portable applications for all operating systems, operating system vendors can identify and fix their conformance issues, and the POSIX standard authors can measure adoption/rejection of the new POSIX.1 2024 standard. os-test is developed as a side project to fully implement POSIX in the new Sortix operating system.

If you have an idea on how to make the Internet a better place, you can apply for funding from the NLnet Foundation's NGI Funds! Thank you so much to them for funding free software projects in need. The project is made possible thanks to funding from European Commission's Next Generation Internet programme.

Benefits of os-test

The os-test lab is an important part of Sortix development, where we can easily write hundreds of tests and run them on every Unix-like operating system. This data helps Sortix implement POSIX and de-facto behavior in the best interoperable manner. We realized os-test can help everyone and is a public benefit for the community.

Importantly, os-test has found numerous bugs in Sortix and every other Unix-like operating system. It's my experience that if you read a sentence in POSIX (especially rare cases), write some tests, run the tests everywhere, then it's likely that someone has implemented it incorrectly. Alternatively, often I just find fascinating differences in behavior, which are perfectly allowed. Systems like OpenBSD, Midipix, Redox, Onyx, and more have already used os-test to fix implementation bugs.

We're about to take os-test up to 11 -- 1.21 gigawatts! Over the next months, os-test will grow exponentially in size, and at the current rate, it's going to find a lot of bugs and differences in all Unix-like operating systems. os-test is all about scale. If we can produce actionable data at a large enough scale, then the OS vendors will come to us, and we don't need to file a myriad of issues separately and have the same discussion repeatedly.

Interoperatibility is important to build the Next Generation Internet. If we are to reap the benefits of innovation in new operating systems such as Sortix and others, then we need portable applications. os-test helps upcoming operating systems compete by being interoperable and being able to innovate and differentiate themselves in other aspects. Let's deduplicate testing efforts and do it together.

I'm excited to get started! Stay tuned to hear more about the progress! Until then, have a look at os-test, and don't forget you easily can submit your own tests if you have any portability questions.