Home • About Us • A-Z Index • Search * Contact Us • Register • Login • Press • Shop

The Open Brand -- Problem Reporting and Interpretations System

Problem Report 2740 Details

Help Show help | Quick Search | Submit a Test Suite Support Request | Click here to view your privileges

This page provides all information on Problem Report 2740.

Report 2740 Actions

    Problem Report Number 2740
    Submitter's Classification Test Suite problem
    State Resolved
    Resolution Test Suite Deficiency (TSD)
    Problem Resolution ID TSD.X.1431
    Raised 2022-06-11 08:34
    Updated 2022-06-15 15:08
    Published 2022-06-15 15:08
    Product Standard Internationalised System Calls and Libraries Extended V3 (UNIX 03)
    Certification Program The Open Brand certification program
    Test Suite VSX5 version 5.3.18NW
    Test Identification /tset/C99.hdr/misc/T.tgmath_4 assertion 7



    /tset/C99.hdr/misc/T.math_1 assertion 7

    Specification Base Definitions Issue 6
    Location in Spec ISO/IEC 9899:1999 Section 7.12 paragraph 7
    Problem Summary The criterion used to determine PASS/FAIL for some tests is arbitrary and unsupported by the

    standard.
    Problem Text The tests for assertions math_1{7} and tgmath_4{7} are based on language in Section 7.12
    paragraph 7 of the ISO/IEC 9899:1999

    standard. That paragraph states:







    ? . 7 ?The macro FP_FAST_FMA is optionally defined. If defined, it indicates that the fma
    function generally executes about as fast

    as, or faster than, a multiply and an add of double operands.193) The macros

    FP_FAST_FMAF



    FP_FAST_FMAL



    are, respectively, float and long double analogs of FP_FAST_FMA. ?







    The test description for tgmath_4{7} states:



    ? Test Description:



    If the header file contains a macro definition for FP_FAST_FMA,



    FP_FAST_FMAF and/or FP_FAST_FMAL:



    The macro expands to the integer constant 1, and the fma(), fmaf()



    and/or fmal() function (respectively) generally executes about as



    fast as, or faster than, the equivalent multiply and add



    operations.?







    The test works by computing a large number of fma() calls, a large number of direct multiply-adds,
    comparing the respective elapsed

    times, and using this criterion as stated in a comment in the test:







    /*



    * The spec says "generally executes about as fast as, or



    * faster than, a multiply and an add of double operands", so



    * we need to allow a reasonable amount of leeway:



    * a) for the "generally" (in case the values we used are atypical)



    * b) to account for the vague "about as fast" wording.



    * In view of (a) we do a very lenient test for speeds being



    * "about as fast": it doesn't fail until one is at least twice



    * as fast as the other.



    */







    While this seems reasonable on the face of it, it is completely unsupported by the language in the
    standard. Moreover, it replaces a

    vague criterion with a sharp criterion. As you can see in our test results we (sometimes) fail with
    ratios barely above 2. If 1.99999

    qualifies as ?about as fast? then 2.006273 should also qualify as ?about as fast?.





    Test Output 400|895 7 1 09:32:07|IC Start



    520|895 7 00037158 7 1|Feature test macros: -D_XOPEN_SOURCE=600 -

    D_POSIX_C_SOURCE=200112



    520|895 7 00037158 7 2|Compiler or run-time messages or results:



    520|895 7 00037158 7 3|FP_FAST_FMAF is defined



    520|895 7 00037158 7 4|Multiply-add float operation using fmaf() used



    520|895 7 00037158 7 5|2.004709 times as much CPU time as the same direct



    520|895 7 00037158 7 6|multiply-add float operation. Expected < 2.0



    220|895 7 1 09:32:07|FAIL



    410|895 7 1 09:32:07|IC End







    400|885 7 1 09:30:47|IC Start



    520|885 7 00031436 4 1|Feature test macros: -D_XOPEN_SOURCE=600



    520|885 7 00031436 4 2|Compiler or run-time messages or results:



    520|885 7 00031436 4 3|FP_FAST_FMAF is defined



    520|885 7 00031436 4 4|Multiply-add float operation using fmaf() used



    520|885 7 00031436 4 5|2.006273 times as much CPU time as the same direct



    520|885 7 00031436 4 6|multiply-add float operation. Expected < 2.0



    520|885 7 00031436 7 1|Feature test macros: -D_XOPEN_SOURCE=600 -

    D_POSIX_C_SOURCE=200112



    520|885 7 00031436 7 2|Compiler or run-time messages or results:



    520|885 7 00031436 7 3|FP_FAST_FMA is defined



    520|885 7 00031436 7 4|Multiply-add double operation using fma() used



    520|885 7 00031436 7 5|2.023618 times as much CPU time as the same direct



    520|885 7 00031436 7 6|multiply-add double operation. Expected < 2.0



    220|885 7 1 09:30:47|FAIL



    410|885 7 1 09:30:47|IC End

    Review Information

    Review Type TSMA Review
    Start Date 2022-06-11 08:34
    Last Updated 2022-06-14 11:55
    Completed 2022-06-14 11:55
    Status Complete
    Review Recommendation Test Suite Deficiency (TSD)
    Review Response This is accepted as a fault in the test suite.

    In view of the vague statement in the standard, these tests should give a
    Warning result instead of Failed.

    Review Type SA Review
    Start Date 2022-06-14 19:55
    Last Updated 2022-06-15 15:07
    Completed 2022-06-15 15:07
    Status Complete
    Review Resolution Test Suite Deficiency (TSD)
    Review Conclusion A test suite deficiency is granted.

    Problem Reporting System Options:

     

    Back   

Contact the Certification Authority