os-test > regex

Legend

Good. A cell is good if the declaration is present in the header and can be linked with libc. This outcome means the test succeeded with the appropriate -D_POSIX_C_SOURCE=202405 or -D_XOPEN_SOURCE=800 feature macros and the standard libraries libc, libpthread, libm, librt, and libxnet.

Missing optional. A cell is missing_optional if the test could not be compiled, but the declaration is optional in POSIX and need not be provided.

Outside libc. A cell is outside_libc if the test could be compiled, but had to be linked with another standard library than the ones prescribed by POSIX: libc, libpthread, libm, librt, and libxnet. This may not conform to POSIX depending on the linking options provided by getconf(1)/confstr(3).

Extension. A cell is extension if the test could not be compiled with the _POSIX_C_SOURCE and _XOPEN_SOURCE feature macros, but it could instead be compiled with other system-specific feature macros that provide the entire API. This outcome means the system header feature macro logic does not support the latest POSIX.1-2024 standard and only receives partial credit. This outcome typically indicates a declaration new to POSIX.1-2024.

Previous posix. A cell is previous_posix if the test could not be compiled with the -D_POSIX_C_SOURCE=202405 or -D_XOPEN_SOURCE=800 feature macros from the POSIX.1-2024 standard, but the test could instead be compiled with the older -D_POSIX_C_SOURCE=200809L or -D_XOPEN_SOURCE=700 feature macros from the older POSIX.1-2008 standard. This outcome means the system header feature macro logic does not support the latest POSIX.1-2024 standard and only receives partial credit. In particular, the header did not check if the value was higher than the supported value, but instead hard-coded the supported values, and failed to be forward compatible with new standard versions.

None. A cell is none if there is no test data result for that operating system.

Compile error. A cell is compile_error if the test could not be compiled and the error message was not recognized as a more precise error test outcome.

Incompatible. A cell is incompatible if the declaration existed in the header, but had a signature that is incompatible with the standardized declaration. This outcome typically indicates a type error in the declaration. Addressing the issue may require an incompatible ABI change or special compatibility logic.

Missing header. A cell is missing_header if the header did not exist.

Undeclared. A cell is undeclared if the header did not contain the declaration.

Undefined reference. A cell is undefined if the test could be compiled, but could not be linked with the standard library.

Unknown type. A cell is unknown_type if the test could not be compiled and failed because a required type was not declared. In some cases, functions fail because the header omitted a required type for a function and instead declared the function in an alternate fashion with an incompatible alias for the type.

§. The § link on the left of each row links to that row.

regex

This suite tests regular expressions.

Each feature in POSIX regular expressions is tested in both BRE (Basic Regular Expression) mode and ERE (Extended Regular Expression) mode. BRE is not actually a regular expression language, despite the name, because it lacks alternations, and importantly because it has back references, which are mathematically impossible to implement efficiently (unless P=NP IIRC). ERE is the real fully feature POSIX regular expression language, which can be implemented efficiently due to the lack of back references. However, POSIX optionally allows BRE implementations to have all ERE features via a special backslash syntax.

This suite is split into the strict base POSIX tests, which tests every sentence that is mandatory in POSIX; and the nonstandard tests, which tests all the optional behavior as well as common de facto extensions.

Each regular expression feature is exhaustively combined with each other regular expression feature to ensure that the features work together. More complicated combinations are tested that involve three or more features in some interesting cases. Each flag to regcomp and regexec are tested as well with their relevant features.

The POSIX requirements on how regular expressions choose their matches are very specific. They can be a little difficult to interpret at times and can be even more tricky to implement properly. See Chapter 9 in POSIX.1-2024 along with the description of regcomp+regexec for all the official rules. Many tests use a truth table approach where every possible reasonable input is given to each regular expression, so it's more likely that implementation bugs are discovered.

The nonstandard tests check every extension I know of, along with UTF-8 and unicode support, including case sensitivity and the behavior on invalid UTF-8 sequences. It's technically fine to disagree with the nonstandard tests, and they mostly exist to study the availability of extensions, and to make sure they are implemented properly in a common way. However, POSIX does mandate that if some nonstandard features like \| are implemented in BRE, then their semantics must match that of the ERE counterparts. If an implementation chooses to have those extensions, then some of the nonstandard tests must be complied with as well.

Overall this test suite is very comprehensive and should provide a near exhaustive coverage of the regular expression features. I read every relevant sentence in POSIX and made sure a test case covered each sentence. The only exceptions are are as follows:

The REG_MINIMAL flag and the ??, ?, +?, and {}? ERE features are not tested because the POSIX.1-2024 specification appears to have been a rush job. The original proposal was fundamentally flawed, and the attempt to fix it does not appear to have been deliberated in detail. I was left with many questions after reading the text. No system implements those features yet, so I don't have any reference data to make test expectations with. The text is full of potential contradictions that I am unclear how to resolve. For instance, does "X??" prefer to match the empty string or "X"? The cited example of ".?c" matching "abcabc" says that it prefers the "abc" match, but is that really how ungreedy matching works? It contradicts the longest overall match rule, and I believe instead usually ungreedy matching loses to over competing greedy matching expressions, but the overall longest match is preferred. Then there's "a*?" where * is usually required to match at least one thing, if at all possible, so on the input "aa" does this pattern prefer the empty string or "a"? There is also the curious case of (X){1,1}?, where it's unclear if the whole X subexpression itself becomes ungreedy? I don't really have enough information to implement the features, let alone test them.

The test suite may miss more advanced implementation bugs that only appear on specifically complicated regular expressions. I had to stop somewhere with the combinatorial explosion of test cases.