Home • About Us • A-Z Index • Search * Contact Us • Register • Login • Press • Shop

The Open Brand -- Problem Reporting and Interpretations System

Problem Report 0602 Details

Help Show help | Quick Search | Submit a Test Suite Support Request | Click here to view your privileges

This page provides all information on Problem Report 0602.

Report 0602 Actions

    Problem Report Number 0602
    Submitter's Classification Test Suite problem
    State Resolved
    Resolution Rejected (REJ)
    Problem Resolution ID REJ.X.0206
    Raised 1970-01-01 08:00
    Updated 2003-03-13 08:00
    Published null
    Product Standard Commands and Utilities V2 (UNIX 95)
    Certification Program The Open Brand certification program
    Test Suite VSC version 4.1.4
    Test Identification POSIX.cmd/sh 90
    Problem Summary PG4C.00070 This test may fail due to a timing problem in an expect script.
    Problem Text
    This test may fail due to a timing problem in an expect script.

    The test in question verifies that the KILL character erases a
    line and repositions the cursor awaiting more input. However,
    there is a typo in the expect script for this test. The last
    five lines of the expect script are:

    send "mkdir YY$TESTID\r"
    expect "$ "
    mkdir "exit\r"
    expect eof
    exit

    Clearly, the 'mkdir "exit\r"' line should be 'send "exit\r"'. This
    is verified by (a) comparing it to the other test scripts, and (b)
    making the change and running the tests.

    On our system, with the existing code the test fails about one time
    out of every three or four. When the test is changed it passes on
    our system every time. We constructed the following scenario file
    to run the test 25 times in succession:

    shtest

    /tset/POSIX.cmd/sh/sh_02.ex{90,90,90,90,90,90,90,90,90,90,90,90,90,90,90,90,90,9
    0,90,90,90,90,90,90,90}

    Here is the "nistreport" for this run with the original code::

    VSC Journal
    1996/01/22
    Test Start: 11:08:21



    POSIX.cmd/sh 11:08:24
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 FAIL
    90 PASS
    90 PASS
    90 PASS
    90 FAIL
    90 PASS
    90 FAIL
    90 PASS
    90 PASS
    90 PASS
    90 FAIL
    90 PASS
    90 FAIL
    90 PASS
    90 FAIL
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 FAIL
    90 PASS
    25 Assertions. 18 PASS; 7 FAIL ; 0 UNRESOLVED; 0 UNSUPPORTED; 0 UNTESTED;
    0 NOT IMPLEMENTED; 0 UNAPPROVED ASSERTIONS; 0 NORESULT; 0 OTHER


    End Test at 11:10:47

    PASS: 18
    FAIL: 7
    UNRESOLVED: 0
    UNSUPPORTED: 0
    UNTESTED: 0
    NOT_IMPLEMENTED: 0
    UNAPPROVED_ASSERTION: 0
    NORESULT: 0
    OTHER:0

    TOTAL: 25

    A similar run with the modified code (changing "mkdir" to "send")
    yielded the following nistreport:

    VSC Journal
    1996/01/22
    Test Start: 11:15:54



    POSIX.cmd/sh 11:15:57
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    90 PASS
    25 Assertions. 25 PASS; 0 FAIL ; 0 UNRESOLVED; 0 UNSUPPORTED; 0 UNTESTED;
    0 NOT IMPLEMENTED; 0 UNAPPROVED ASSERTIONS; 0 NORESULT; 0 OTHER


    End Test at 11:17:58

    PASS: 25
    FAIL: 0
    UNRESOLVED: 0
    UNSUPPORTED: 0
    UNTESTED: 0
    NOT_IMPLEMENTED: 0
    UNAPPROVED_ASSERTION: 0
    NORESULT: 0
    OTHER:0

    TOTAL: 25

    We believe that this amply demonstrates that (a) this is indeed a typo in
    the test case, (b) our system conforms to the requirements of sh assertion 90.
    Test Output
    -----------------start of test output------------------

    400|1 90 1 11:08:51|IC Start
    200|1 1 11:08:51|TP Start
    520|1 1 12009 1 1|Assertion #90 (C): the kill char; clrs the linel; wh/ in
    insert mode
    520|1 1 12009 1 1|Command failed: '[ -d YY90 ] && [ ! -d XX90 ]'
    220|1 1 1 11:08:56|FAIL
    410|1 90 1 11:08:56|IC End

    ------------------end of test output-------------------

    Review Information

    Review Type TSMA Review
    Start Date null
    Completed null
    Status Complete
    Review Recommendation No Resolution Given
    Review Response
    We recommend this request be refused.

    The submitter is correct in pointing out the typo mentioned
    above. But he is incorrect in statig that this causes
    intermittent failures. This issue can never cause this test
    suite to fail.

    We believe that this failure is an intermittent expect timing
    problem. The proper way to deal with this issue is to submit a
    separate journal showing that the IUT can pass this test as the
    waiver for test FAILure if it occurs during the final branding
    run.

    Review Type SA Review
    Start Date null
    Completed null
    Status Complete
    Review Resolution Rejected (REJ)
    Review Conclusion
    This request is refused.

    Problem Reporting System Options:

     

    Back   

Contact the Certification Authority