1. Problem Reporting System Overview

1.1 Introduction

This Problem Reporting System provides functionality to submit, search and manage problem reports related to a specific Certification Program.  It uses a web front-end, which gives access to the repository of Problem Reports relating to the program.  The web pages presented to a user of the system are tailored to their privileges on the system, which are controlled using their web authorization by a username and password.  When a Problem Report is submitted a clear distinction is made between public and confidential information provided by the submitter. When searching, viewing and reviewing problem reports, only the Certification Authority (CA) reviewers will see the confidential information. Public information is accessible by any member of the system.

The problem reporting system is accessed using a series of URLs that can be book-marked in a browser.  However, the most natural starting points are the actions of searching the repository for existing problem reports, and raising new problem reports.

Access to the problem report system requires privileges on the hosting web server.  Privilege levels are granted based on group or access list memberships on the server.  For example, users in one of the reviewer lists may review problem reports using the web interface, but users who are not in a reviewer list may only search for public information on problem reports.

The roles in the problem reporting system are listed below. In the list, TSMA is the Test Suite Maintenance Authority (the group of people who maintain the test suites), SA is the Specification Authority (the group of people who develop the specifications), and CA is the Certification Authority (the group of people who manage the certification program):

Member	Any user who has access to the Problem report system, but has no review privileges.  A member may only search for and view published problem reports, and they will only see public information in those problem reports, i.e. the anonymous problem description and only the resolution information for the reviews that occurred during the problem report resolution process.
TSMA Reviewer	A user with privileges to make comments in a Test Suite Maintenance Authority (TSMA) review.  A TSMA Reviewer may search for and view unresolved and public problem reports.  They will only see public information in a problem report, and only resolution information for any Certification Authority (CA) or Specification Authority (SA) reviews in a problem report.  However, they will see full review discussion details for any TSMA reviews, and will be able to take part in the review discussion by submitting their own comments.  Controlled by group membership.
TSMA Responder	A user with privileges to make the final response that closes a Test Suite Maintenance Authority (TSMA) review.  A TSMA Responder has TSMA Reviewer privileges, and additionally may provide the final public response to a TSMA review. Controlled by group membership.
SA Reviewer	A user with privileges to make comments in a Specification Authority (SA) review.  An SA Reviewer may search for and view unresolved and public problem reports.  They will only see public information in a problem report, and only resolution information for any Certification Authority (CA) or Test Suite Maintenance Authority (TSMA) reviews in a problem report.  However, they will see full review discussion details for any SA reviews, and will be able to take part in the review discussion by submitting their own comments.  Controlled by group membership.
SA Responder	A user with privileges to make the final response that closes a Specification Authority (SA) review.  A SA Responder has SA Reviewer privileges, and additionally may provide the final public response to a SA review. Controlled by group membership.
CA Reviewer	A user with privileges to make comments in a Certification Authority (CA) review.  A CA Reviewer may search for and view unresolved and public problem reports.  They will see both public and confidential information in a problem report, and but only resolution information for any Test Suite Maintenance Authority (TSMA) or Specification Authority (SA) reviews in a problem report.  They will see full review discussion details for any CA reviews, and will be able to take part in the review discussion by submitting their own comments.  Controlled by group membership.
CA Responder	A user with privileges to make the final response that closes a Certification Authority (CA) review.   A CA Responder has CA Reviewer privileges, and additionally may provide the final public response to a CA review.  The CA Responders are also responsible for notifying the submitter of the problem report of its progress through the system, and updating the system based on the submitter's response. Controlled by group membership.

1.3 Life-Cycle of a Problem Report

The Problem Reports tracked in the system go through the life-cycle described below. This description covers those parts of the life-cycle that have software support, which does not include all the possible states in the Problem Reporting and Interpretations process.

Member Action � Raise a Problem Report

All members can submit problem reports. Submitters provide their name, contact, company and product information, which are all kept confidential. Submitters also provide information on the problem, including their own classification of the type of problem (see below), a problem description, and references to any test suites or specifications involved. This information is public and will be visible to all users of the Problem Reporting system.

When a member first submits an problem report, it enters a review state based on the submitter�s classification of the problem. If the classification is a Test Suite problem a TSMA review is started, If the classification is a Certification System problem a CA review is started, and otherwise a SA review is started. The appropriate reviewers will receive a notification email indicating that the Problem Report has been raised and the review started. Whatever the review, the CA and SA reviewers will always be notified of new reports raised.

Reviewer Action � Discuss a Problem Report in a Review

All users in the review group for a given review type may join in the discussions on the Problem Report using the web interface and the URL given them in the review notification email. Comments will be accumulated in the review, but will only be visible to users of the particular review group, not to reviews in other review groups (e.g. the CA cannot see the TSMA review comments).

Responder Action � End a Problem Report Review

After reviewers have come to a conclusion about a Problem Report, one of the responders provides the official response to the review and this closes the review. This comment is the only information concerning a review that is publicly visible. If it is a CA or TSMA review, a SA Review is then automatically started (and SA Reviewers notified). If it is a SA Review, then the CA is notified.

CA Responder Action � Notify submitter of SA review outcome

After the SA review of a Problem Report, the CA notifies the submitter using the web interface. The review may not have been conclusive, and in which case an Expert Group Review will automatically be started. If the review was conclusive the Problem Report will change state.

SA Action � Conduct an Expert Group Review

For Problem Reports whose resolution is still inconclusive after the SA review, the SA carries out an Expert Group review in exactly the same way as other reviews. An expert group review cannot return an inconclusive outcome.

CA Responder Action � Publish Problem Report

The CA updates the Problem Report to publish it.

CA Responder Action � Mark Problem Report as Appealed

The CA updates the Problem Report to mark it as appealed.

CA Responder Action � Notify submitter of appeal outcome

If a submitter has appealed against the resolution of their Problem Report, the appeal process will be used to resolve the issue. This is not tracked in the Problem Reporting system. However, once the appeal outcome is known the CA notifies the submitter and publishes the Problem Report with the final outcome.