We regularly test the XGC compilers against the ACVC test suite, and against its successor, the ACATS tests. While both of these are intended for compilers that no built-in restrictions, they offer good coverage of the Verison 1.8 compilers.
We have identified 3487 tests from ACATS Version 2.5 that are applicable to restricted compilers. Table 9-1 lists the number of tests in each section, and how many of those tests the compiler passes.
Table 9-1. The Validation Test Classes
| Group | Description | Number of Tests | Number of Applicable Tests | Number of Passes |
|---|---|---|---|---|
| A | Class A tests check for acceptance (compilation) of language constructs that are expected to compile without error. | 75 | 61 | 61 |
| B [a] | Class B tests check that illegal constructs are recognized and treated as fatal errors. | 1510 | 1510 | 1510 |
| C | Class C tests check that executable constructs are implemented correctly and produce expected results. | 2307 | 1835 | 1562[b] |
| D | Class D tests check that implementations perform exact arithmetic on large literal numbers. | 4 | 4 | 4 |
| E | Class E tests check for constructs that may require inspection to verify. | 32 | 9 | 6 |
| L [c] | Class L tests check that all library unit dependencies within a program are satisfied before the program can be bound and executed, that circularity among units is detected, or that pragmas that apply to an entire partition are correctly processed. | 89 | 68 | 68 |
| Notes: a. B tests are expected to fail with compilation time errors. Ones that are not applicable due to restrictions may therefore fail for different reasons from the original intention of the test, but nevertheless fail to compile and are therefore treated as passes. b. In group C, 303 tests did not print PASSED but terminated with an unhandled exception. In all 303 cases the exception was correctly raised then not handled because of the restriction on exception propagation. c. L tests are expected to give errors at compile time, bind time or link time and not to run. | ||||