Sortix

os-testing POSIX headers everywhere

Jonas 'Sortie' Termansen on 2025-09-28

What's the best POSIX operating system? Let's measure it and find out! We're gonna find bugs in every operating system when we start poking data at scale.

My methodology:

  1. I downloaded the POSIX.1-2024 standard as html.
  2. I wrote a posix-parse.c program that parses the HMTL into machine readable .api header declaration files.
  3. I generated a test file for each declaration, testing it exists and has the right type.
  4. I ran the new include test suite everywhere on every single Unix system I could get my hands on.
  5. I published the data as os-test and found a ton of bugs in every single operating system.
  6. Hopefully operating systems use this data and improve so portable programs truly work everywhere.

Thank you to the NGI Zero Commons Fund for graciously funding os-test and thank you to cfarm for providing access to the proprietary POSIX systems AIX, macOS, and Solaris.

AIX 7.3 - 92%

AIX is a quite strict POSIX system. I did not detect any modern extensions commonly found elsewhere that have since been standardized in POSIX-1.2024, AIX appears to implement the POSIX-1.2008 standard quite well, with some incompatibilities. I was pleased to find a lot of common software in /opt/freeware/bin on the system I have access to. Apparently I have to use the openxlC 17.1.1 toolchain with an ibm-clang rather than a cc in PATH. The particular system I can access has 40 cores but is somehow very slow to compile on my user.

The AIX 7.3 test score for POSIX-1.2024 is 92%. the remaining parts are largely new declarations introduced in POSIX-1.2024, and various subtle incompatibilities where types and function signatures were subtly wrong. The score is a bit lower because it doesn't have libintl, which was recently standardized.

For instance, AIX does not have the newly standardized features that were common extensions: <endian.h>, asprintf(3), ppoll(2), qsort_r(3), and others. I imagine it will likely be easy for the AIX developers to add many of these features with a modest effort.

Meanwhile AIX has 16 cases where the type signature is subtly wrong, for example:

struct pollfd {
  unsigned short events;
  unsigned short revents;
  long fd;
};
int poll(void *, ulong_t, long);
int getpwnam_r(const char *, struct passwd *, char *, int, struct passwd **);

Which should have been:

struct pollfd {
  short events;
  short revents;
  int fd;
};
int poll(struct pollfd *, nfds_t, int);
int getpwnam_r(const char *, struct passwd *, char *, size_t, struct passwd **);

Fixing these issues upstream may require incompatible ABI changes or special compatibility logic and symbol redirection. The issues are likely not going to be noticed or matter when porting ordinary programs.

AIX is generally a great choice if you want to test on a strict POSIX-1.2008 system without many nonstandard extensions.

DragonFly BSD 6.4 - 90% (93%)

DragonFly BSD 6.4 scores 90% and would have scored 93% if the feature macros are adjusted for extensions that have been standardized in POSIX.1-2024. Overall a decent middle tier score.

A number of POSIX.1-2024 additions are still missing.

Dragonfly BSD has a lot of the new POSIX-1.2024 features that were previously de-facto common extensions. The score is a bit lower because it doesn't have libintl, which was recently standardized, but I should probably install it as an optional package. The C11 <threads.h> header is missing, which lowers the score.

There are 10 cases where the type signatures are subtly wrong, for example:

char *initstate(unsigned long, char *, long);
int setpriority(int, int, int);
struct tm { /* ... */
  char *tm_zone;
};
int getnameinfo(const struct sockaddr *restrict, socklen_t, char *restrict,
                size_t, char *restrict, size_t, int);

Which should have been:

char *initstate(unsigned int, char *, size_t);
int setpriority(int, id_t, int);
struct tm { /* ... */
  const char *tm_zone;
};
int getnameinfo(const struct sockaddr *restrict, socklen_t, char *restrict,
                socklen_t, char *restrict, socklen_t, int);

Otherwise Dragonfly BSD's headers are quite complete with no big missing areas.

FreeBSD 14.3 - 92% (94%)

FreeBSD is the top scoring BSD system scores 92% and would have scored 94% if the feature macros are adjusted for extensions that have been standardized in POSIX.1-2024.

A number of POSIX.1-2024 additions are still missing such as PTHREAD_NULL, pthread_mutex_clocklock, sem_clockwait, sig2str.

FreeBSD has a lot of the new POSIX-1.2024 features that were previously de-facto common extensions. The score is a bit lower because it doesn't have libintl, which was recently standardized, but I should probably install it as an optional package. <fcntl.h> also misses a bunch of the S_ file bit constants.

Unfortunately linking with libstdthreads is required to use the C11 threads.h API, which POSIX only allows to be in libc or libpthread, and that no special option is required to use the API.

There are 6 cases where the type signatures are subtly wrong, some of which shared with DragonFly BSD, but also some of its own:

typedef struct { /* ... */
  int dsize;
} datum;
int pthread_mutexattr_getrobust(pthread_mutexattr_t *restrict,
                                int *restrict);

Which should have been:

typedef struct { /* ... */
  size_t dsize;
} datum;
int pthread_mutexattr_getrobust(const pthread_mutexattr_t *restrict,
                                int *restrict);

These functions are declared but cannot be linked with:

Overall FreeBSD is a solid POSIX system that's keeping up to date with de-facto extensions.

Haiku - 90% (91%)

Haiku is doing very well for an upcoming operating system and scores 90% and would have scored 91% with a hint of feature macro adjustments.

Haiku suffers from network portability problems, as many of its network functions (net/if.h, netinet/in.h, arpa/inet.h, netdb.h, and sys/socket.h) has been put in libnetwork for some reason. POSIX doesn't allow this behavior, and only allows a limited list of functions to possibly be in libxnet. However, nobody actually does so, according to os-test data. I recommend Haiku either merge libnetwork into libc, or add it to the default set of standard libraries in the compiler link spec. Otherwise everyone porting software to Haiku will need to deal with this unnecessary complexity.

There are 6 curious cases where the type signatures are subtly wrong:

long random(void);
const char *strsignal(int);
size_t c16rtomb(char *restrict, char32_t, mbstate_t *restrict);
struct tm { /* ... */
  char *tm_zone;
  int tm_gmtoff;
};
struct sockaddr { /* ... */
  uint8_t *sa_data;
};

Which should have been:

int random(void);
char *strsignal(int);
size_t c16rtomb(char *restrict, char16_t, mbstate_t *restrict);
struct tm { /* ... */
  const char *tm_zone;
  long tm_gmtoff;
};-
struct sockaddr { /* ... */
  char *sa_data;
};

Curiously Haiku actually provides the POSIX.1.2024 Device Control extension that attempts to replace ioctl, as the only system to do so:

int posix_devctl(int, int, void *restrict, size_t, int *restrict);

Unlike ioctl, posix_devctl is absolutely unusable. The standard never defined how the function can actually be invoked, and there is no established tradition. I haven't found the haiku documentation yet for this function, and I couldn't find its source code with a quick search in git.

Haiku is a great system to test on if you want to be compatible with the long tail of systems. Some weirdness inherited from BeOS may affect portability and might make people care less about Haiku.

Hurd - 92% (95%)

GNU Hurd benefits greatly from sharing glibc and scores 92% and would have scored 95% with a hint of feature macro adjustments.

Unfortunately I am testing the Debian Hurd 2023 i386 release, as the 2025 release does not work and cannot be installed in qemu without crashing. If you're a Hurd developer, please feel free get in touch because I really would love for it to work in a standard qemu virtual machine.

The headers and available functions are largely the same as the Linux glibc implementation, so I'll describe them together next. However, I should note that Hurd scores somewhat lower on the os-test suites that test runtime behavior.

Linux (glibc) - 93% (96%)

glibc is a top standard library and scores 93% and would have scored 95% with a hint of feature macro adjustments. glibc scores a third place, almost tied with OmniOS.

glibc has 16 incompatible type signatures, including:

typedef __fsid_t long long unsigned int;
struct statvfs { /* ... */
  __fsid_t f_fsid;
};
struct stat { /* ... */
  __fsid_t st_dev;
};
int lio_listio(int, struct aiocb *const restrict *restrict, int,
               struct sigevent *restrict);
struct flock { /* ... */
  int l_whence;
  int l_type;
};

Which should have been:

struct statvfs { /* ... */
  unsigned long f_fsid;
};
struct stat { /* ... */
  dev_t st_dev;
};
int lio_listio(int, struct aiocb *restrict const [restrict], int,
               struct sigevent *restrict);
struct flock { /* ... */
  short l_whence;
  short l_type;
};

glibc is a common system that everyone should test for, even though it can be slightly incorrect at times, but don't forget all the other systems before making too many glibc-specific assumptions.

Linux (musl) - 96%

musl is the top scoring standard library, scoring 96%. There are just 19 cases where the feature macros needs to be adjusted for POSIX.1-2024.

musl does not have any incompatible type signatures. Impressive. musl has been tested with libc-test, whose methodology inspired this os-test include suite, but doesn't cover POSIX.1-2024.

These functions are declared but cannot be linked with:

musl is a great system to test as a stricter system with most modern interfaces, and a great second platform to test on in addition to glibc. There isn't a musl predefined macro to recognize musl: You're supposed to use e.g. configure to check for the features available at compile-time.

macOS - 87% (88%)

macOS is the lowest scoring production-ready POSIX operating system, narrowly behind OpenBSD. macOS scores 87% and would have scored 88% with a hint of feature macro adjustments.

macOS has been historically very slow to adopt POSIX.1-2008 features, but has since been adding some of the interfaces over time, and I expect POSIX.1-2024 adoption to be similarly slow.

In particular, it's slow adoption of endian.h has been a common source of portability problems, but might possibly improve now that endian.h has been standardized in POSIX.1-2024.

macOS doesn't have the C11 threads.h and uchar.h headers. Meanwhile a lot of _l suffixed locale function variants are missing.

There are 4 incompatible type signatures:

char *setstate(const char *);
int lio_listio(int, struct aiocb *const *, int, struct sigevent *);
int shm_open(const char *, int, ...);
struct tm { /* ... */
  char *tm_zone;
};

Which should have been:

char *setstate(char *);
int lio_listio(int, struct aiocb *restrict const [restrict], int,
               struct sigevent *restrict);
int shm_open(const char *, int, mode_t);
struct tm { /* ... */
  const char *tm_zone;
};

macOS is overall backwards, indicating the Apple priorities, and features seem to only be added years later when someone decides it's finally time.

Minix - 77% (81%)

Minix is the lowest scoring POSIX operating system and scores 77% and would have scored 81% with 116 feature macro adjustments.

Unfortunately Minix development appears to have been abandoned and is included here for historical interest, comparison, and completion. Overall Minix is the lowest scoring OS on the several os-test suites, suggesting an overall low implementation quality. It appears that much of the userland has been copied from NetBSD, but many system calls and functions cannot be linked with or does not exist at runtime. Notably Minix does not have pthreads or C11 threads.

Minix has 6 type signature incompatibilities:

struct utmpx { /* ... */
  __uint16_t ut_type;
};
int setkey(const char *);
int lio_listio(int, struct aiocb *const *restrict, int,
               struct sigevent *restrict);
int encrypt(char *, int);
size_t iconv(iconv_t, const char **restrict, size_t *restrict,
             char **restrict, size_t *restrict);
struct tm { /* ... */
  char *tm_zone;
};

Which should have been:

struct utmpx { /* ... */
  short ut_type;
};
void setkey(const char *);
int lio_listio(int, struct aiocb *restrict const [restrict], int,
               struct sigevent *restrict);
void encrypt(char [64], int);
size_t iconv(iconv_t, char **restrict, size_t *restrict,
             char **restrict, size_t *restrict);
struct tm { /* ... */
  const char *tm_zone;
};

These are 95 functions that are declared but cannot be linked with, including:

The os-test runtime suites also suggest that a number of interfaces will fail with ENOSYS at runtime.

Minix is not worth testing for, but on the other hand, if it works on Minix, it will likely work everywhere.

NetBSD - 90% (92%)

NetBSD 10.1 scores 90% and would have scored 92% if the feature macros are adjusted for extensions that have been standardized in POSIX.1-2024. Overall a decent middle tier score.

NetBSD has 6 type signature incompatibilities, similar to Minix, but with a difference in sigev_notify_attribute:

struct utmpx { /* ... */
  uint16_t ut_type;
};
int setkey(const char *);
int lio_listio(int, struct aiocb *const *restrict, int,
               struct sigevent *restrict);
int encrypt(char *, int);
struct tm { /* ... */
  char *tm_zone;
};
struct sigevent { /* ... */
  void *sigev_notify_attributes;
};

Which should have been:

struct utmpx { /* ... */
  short ut_type;
};
void setkey(const char *);
int lio_listio(int, struct aiocb *restrict const [restrict], int,
               struct sigevent *restrict);
void encrypt(char [64], int);
struct tm { /* ... */
  const char *tm_zone;
};
struct sigevent { /* ... */
  pthread_attr_t *sigev_notify_attributes;
};

OmniOS - 93% (95%)

OmniOS is a top scoring POSIX system with 93%, only behind musl, and would have scored 95% if the feature macros are adjusted for extensions that have been standardized in POSIX.1-2024.

Illumos has diverged considerably from Solaris, which makes both systems very interesting to measure and compare.

Illumos has only 3 type signature incompatibilities:

char *setstate(const char *);
struct entry { /* ... */
  char **data;
};
int getlogin_r(char *, int);

Which should have been:

char *setstate(char *);
struct entry { /* ... */
  void **data;
};
int getlogin_r(char *, size_t);

Illumos may be a good choice to test on, if you're interested in supporting Solaris, but you don't own Solaris. However, the systems have diverged considerably, OmniOS has become more modern, and testing on Illumos is not enough to know that software works on Solaris.

OpenBSD - 87%

OpenBSD 7.7 is the lowest scoring BSD system with 87%. There are only 22 cases where the feature macros need to be adjusted.

OpenBSD is the only production-ready POSIX system without the aio.h, monetary.h, and wordexp.h headers. Additionally C11 threads.h is missing. Overall these absences speak to how rarely used those headers are in practice.

Meanwhile we must thank OpenBSD for participating in the standardization process and successfully inventing and standardizing several interfaces, including: strlcpy, strlcat, reallocarray, getentropy, and more. Additionally they suggested several more interfaces such as arc4random that failed to be standardized at this time.

There are 27 incompatible type signatures in OpenBSD, for example:

struct shmid { /* ... */
    int shm_segsz;
};
int pthread_barrier_init(pthread_barrier_t *, pthread_barrierattr_t *,
                         unsigned int);
int vprintf(const char *restrict, __va_list_tag *);
void psignal(unsigned int, const char *);
struct tm { /* ... */
  char *tm_zone;
};
int getnameinfo(const struct sockaddr *, socklen_t, char *, size_t, char *,
                size_t, int)

Which should have been:

struct shmid { /* ... */
    size_t shm_segsz;
};
int pthread_barrier_init(pthread_barrier_t *restrict,
                         const pthread_barrierattr_t *restrict,
                         unsigned int);
int vprintf(const char *restrict, va_list);
void psignal(int, const char *);
struct tm { /* ... */
  const char *tm_zone;
};
int getnameinfo(const struct sockaddr *restrict, socklen_t, char *restrict,
                socklen_t, char *restrict, socklen_t, int);

Redox - 78%

Redox is the lowest scoring actively developed POSIX system, scoring 78%.

Redox has a new standard library relibc that is written in Rust with generated C headers. Overall its standards compliance is lacking, but improving. relibc appears to have a systematic pattern of stubbing out interfaces, so they can be linked with, but doesn't work at runtime. The real test scores may be lower when the next os-test phase starts invoking each function to see if they work at runtime. Meanwhile many standard library functions appear to emit warnings to the standard error, which causes os-test failures since the output is incorrect. Redox has just gotten to the point where it's able to run os-test, which is an impressive feat for an upcoming operating system developed from scratch.

A number of facilities are missing: aio.h, monetary.h, nl_types.h, semaphore.h, tar.h, threads.h, uchar.h, and wordexp.h. Meanwhile a lot of standards comformance constants are missing from limits.h and unistd.h.

There are 21 incompatible type signatures in Redox, including many wrong const types, for example:

struct lconv { /* ... */
    const char **grouping;
};
char *if_indextoname(unsigned, char *);
size_t read(int, const void *, size_t);
int pthread_attr_setguardsize(pthread_attr_t *, int);
struct sigaction { /* ... */
    unsigned long sa_flags;
};
int select(int, fd_set *, fd_set *, fd_set *, struct timeval *);
struct netent { /* ... */
    unsigned long n_net;
};
struct msghdr { /* ... */
    size_t msg_iovlen;
};

Which should have been:

struct lconv { /* ... */
    char **grouping;
};
const char *if_indextoname(unsigned, char *);
size_t read(int, void *, size_t);
int pthread_attr_setguardsize(pthread_attr_t *, size_t);
struct sigaction { /* ... */
    int sa_flags;
};
int select(int, fd_set *restrict, fd_set *restrict, fd_set *restrict,
           struct timeval *restrict);
struct netent { /* ... */
    uint32_t n_net;
};
struct msghdr { /* ... */
    int msg_iovlen;
};

These functions are declared but cannot be linked with:

Redox is not yet stable as a platform to test on, but it's fun to try out and see how far they've come implementing POSIX in Rust.

Solaris - 83% (95%)

Solaris 11.4 would have been a top system, however it only scores 83%, because the standard library headers does not use range checks for the feature macros, and the POSIX.1-2024 feature macros are not recognized. If the POSIX.1-2008 feature macros are used instead, then it would have scored 95%.

Overall Solaris is quite complete, and does have some of the new POSIX.1-2024 features that were previously de-facto extensions, but it does lack the other half of the new interfaces. Meanwhile Solaris POSIX environment can be quite backwards, such as the PATH containing very old tools without features that were standardized long ago. /usr/bin/getconf can be used to find /usr/xpg6/bin and /usr/xpg4/bin that provides programs with POSIX.1-2001 features. However, it fails to provide /usr/xpg7/bin with POSIX.1-2008 features, which requires invoking /usr/xpg7/bin/getconf directly.

Solaris has only a single type signature incompatibility:

struct entry { /* ... */
  char **data;
};

Which should have been:

struct entry { /* ... */
  void **data;
};

Solaris may be interesting as a platform to test on, if you're looking for a proprietary system that's somewhat strict and old style, but still has some modern extensions.

Sortix - 83% (84%)

Sortix 1.1.0-dev (2025-09-28) has most facilities and scores 83%, and would have scored 84% if 13 minor feature macro issues were fixed.

Sortix is a new POSIX operating system developed from scratch and os-test was developed to guide its development with interoperability data. Features are only present in the standard library if the features are implemented correctly, so the compile-time include score should match the future runtime os-test suite scores. I intentionally did not modify the Sortix standard library while developing this os-test include suite, to ensure the results were fair and unbiased.

Sortix is in an interesting state where it implements a lot of POSIX.1-2024 already as one of the first systems to do so, and often has the modern functions, but might lack older and deprecated interfaces. Some interfaces are missing on purpose.

A number of facilities are missing: aio.h, cpio.h, monetary.h, nl_types.h, tar.h, threads.h and wordexp.h. The features that are missing are the ones that weren't needed to port software in practice, as libc and kernel features are implemented whenever they are needed.

The libm is a partial import of an old NetBSD libm and has a considerable number of unintentional ommisions and defects. It needs to be replaced by another libm of higher quality and compatible licensing, possibly the one from musl.

Sortix makes some different ABI and API choices than other POSIX systems, such as 64-bit time_t, off_t, ino_t, gid_t, uid_t; and pointer-sized dev_t, pid_t, timer_t. There are a number of different choices allowed by POSIX, which Sortix attempts to use, which creates some portability issues in practice.

Sortix has 4 incompatible type signatures:

struct termios { /* ... */
  size_t ws_col;
  size_t ws_row;
};
int pthread_mutexattr_gettype(pthread_mutexattr_t *restrict, int *restrict);
int mprotect(const void *, size_t, int);

Which should have been:

struct termios { /* ... */
  unsigned short ws_col;
  unsigned short ws_row;
};
int pthread_mutexattr_gettype(const pthread_mutexattr_t *restrict, int *restrict);
int mprotect(void *, size_t, int);

As the developer, I can say the struct termios issues were originally on purpose and is a remnant of a time where the Sortix terminal interfaces were very weird, and the other two const issues are just typos. I may have some ABI compatibility issues fixing the termios struct and haven't decided on the final approach yet. Now that I've published the os-test data and made it easy to find these problems, I look forward to diving into my Sortix test results and increasing the conformance.

These functions are declared but cannot be linked with:

As the developer, I can say those functions are actually implemented, and I just forgot to add the source files to the libc makefile.

Sortix is a great choice to test on, if you're looking for a semi-strict system that has the modern interfaces, but doesn't have obsolete interfaces. Porting programs to Sortix usually involves removing assumptions that are not required by POSIX, which makes your programs more portable. If your program works on Sortix, it's likely compatible with the future direction of POSIX. However, some programs require features that are not implemented yet and cannot be ported at this time.

Conclusion

There are a ton of ways to look at this huge data set and a simple write-up cannot communicate all the many interesting variances in the os-test include suite. There are also a ton of optional parts of POSIX that could be worth talking about, including the big XSI option. You can find all of those results at the bottom of the os-test pages.

musl on Linux is the top scoring standard library for POSIX systems, with Illumos (OmniOS) and glibc close behind. The proprietary AIX and Solaris systems can feel a bit backwards and strict, but technically score very high on the previous POSIX.1-2008 standard, and may be very able to adopt POSIX.1-2024 in the near future. The BSD systems have a range of results, with a shared historical base, and divergent implementations of the recent standards. GNU Hurd is a lot like glibc on Linux, though some features are missing.

In the upcoming operating systems: Haiku has the most POSIX coverage in the standard library, although the runtime os-test suites have lower scores; Sortix has most of POSIX with notable missing parts, but the runtime scores are generally high as os-test was built to test Sortix; Redox has much of POSIX with big missing parts and generally low runtime test scores in areas under development; and Minix has the lowest scores and is no longer active development.

If you're looking for a strict OS to test if your POSIX application is truly portable without extensions, AIX and Solaris may be great choices if you can obtain access. If you're looking for a freely available standard library with most modern facilities and extensions, musl on Linux is a great choice. If you're looking to try out the new POSIX.1.2024 interfaces, Sortix is actually is good choice since it was one of the first systems to implement much of the new standard.

Copyright 2011-2025 Jonas 'Sortie' Termansen and contributors.
Sortix's source code is free software under the ISC license. 		#sortix on irc.sortix.org
@sortix_org