IsoveraDL Peer Review System Software Requirements Specification

Document Summary

This document specifies the requirements for the BEN Collaborators Peer Review Tool. The peer review tool will be an open-source software system written for the PHP/MySQL platform. Although the system will be written in an extensible way, its primary focus will be the provision of the BEN Collaborative with a suite of tools which facilitate its mission of providing a federated network of peer-reviewed biological learning resources and their metadata. The peer review tool will be developed as a complement to the BEN Collaborators Collections Tool, aka “Dewey”, which was released in Sept. 2006. The peer review tool will be released as a plugin for Dewey.

Revision History

Date Version Description of Document Updates Author
11/22/2006 0.1 Write up system overview and first draft of use cases. ssachs
11/22/2006 0.2 Revise workflow management section for improved flexibility. ssachs
12/05/2006 0.3 Revisions and comments ccollins
12/05/2006 0.4 Update overview section and add more comments regarding feedback from authors and routing resources into the workflow system. ssachs
12/06/2006 1.0 First draft released ssachs
2/26/2007 1.1 Incorporated review feedback,. ssachs, syoumans
3/07/2007 1.2 Final review and revision; circulate to BEN ccollins, sdemi, smostafa
4/17/2007 1.3 Incorporated branching smostafa

Introduction

Purpose

The primary goal of the peer review tool is to facilitate online peer review of educational resources whose metadata is maintained by digital libraries within the BEN Collaborative.

Scope

The peer review tool will provide users with all of the tools they need to conduct peer reviews of educational resources, within the context of a catalog of metadata which describes those resources. The peer review tool does not provide any features for reviewing the quality of the metadata; the Dewey collections tool already includes such features. The peer review tool will leverage, but not significantly extend, the input form design functionality found in Dewey

System Overview

The Peer Review Tool will operate within the context of the Dewey collections tool released in Sept. 2006. The peer review tool will be released as a plugin to the Dewey collections tool, as provided by the CakePHP framework which underlies the technical architecture of Dewey.

The lifecycle of an educational resource in a metadata catalog, and the position of peer review in that lifecycle, is depicted in Figure 1. Two methods for accessioning records into a collection are depicted. One, labeled “cataloging”, enables trusted users to add resources known to be of high quality into the collection. Another, labeled “submission”, enables untrusted users to add resources of any quality into the collection. Once a resource is submitted, it is peer reviewed to ensure that it is of high quality. If a resource is accepted into the collection, its metadata is reviewed separately. Once the metadata is fully validated, both the resource and the metadata are published in the collection, and are made available for harvesting and browsing by external users.


Figure 1 The resource and metadata lifecycle

The Peer Review Tool will facilitate the following tasks in a smoothly integrated manner:

- Managing a peer review workflow that is flexible and easily changeable.

- Assigning review tasks to users or groups of users.

- Tracking interactions with authors and generating emails from templates.

- Designing forms which accept input on the quality of a learning resource.

- Monitoring workload and review performance.

The peer review tool does not include facilities for users to review the quality of the metadata describing a resource, although such facilities are included in Dewey. Reviewers will have the capability to change the metadata which describes a resource as needed, but this capability will not extend beyond the capabilities already provided by Dewey.

Administrators will be able to manage the peer review workflow using a model which is flexible and easy to reconfigure. The model can accommodate a wide variety of different review processes, while guaranteeing that all resources will be subject to a final review step where decisions of acceptance or rejection can be made definitively. The system provides for assignments, reviews made by multiple reviewers in parallel, and decisive reviews made by a single person. Administrators will be able to create new user groups in order to properly assign duties in the review system, and will be able to assign users to those user groups as needed.

Administrators will also be able to design and deploy forms which may be used at different steps of review. The form design process will leverage the power of the Dewey input process management functionality. The capture of form input will be conducted in a way that facilitates inclusion of reviewer input on form letters sent to authors.

In order to review the activity in the system, administrators will be able to monitor workload and review the performance of peer reviewers. Monitoring allows administrators to decide who should be assigned review tasks and to ensure that reviewers are completing reviews in a timely manner. Monitoring is complemented by a system for automatically reminding reviewers of their pending reviews.

The system main persistence layer will be a database built on the MySQL 4.1 platform. The system will be built in PHP version 4.4, using the CakePHP v1.0 framework.

References

NSDL Metadata Policy for Collections – http://policy.comm.nsdlib.org/cgi-bin/wiki.pl?Policy_Drafts_MS-1

BEN Metadata White Paper – http://www.biosciednet.org/docs/BEN_Metadata_White_Paper_V5.pdf

IEEE Learning Objects Metadata Specification – http://ltsc.ieee.org/wg12/files/LOM_1484_12_1_v1_Final_Draft.pdf

The Open Archives Initiative Protocol for Metadata Harvesting – http://www.openarchives.org/OAI/openarchivesprotocol.html

Use Cases

A use case describes how a user of the proposed system will interact with the system to perform a unit of work. It describes an interaction over time that has meaning for the end user (person, machine or other system), and leaves the system in a complete state.

· A use case typically has requirements and constraints that describe the essential features and rules under which it operates.

· A use case may have an diagram or description illustrating behavior over time - who does what and to whom, when.

· A use case typically has scenarios associated with it that describe the work flow over time that produces the end result. Alternate work flows (to capture exceptions, etc.) are also allowed.

The following use cases are included in the collection tool:

Workflow Management

The peer review workflow is the process by which all resources are reviewed. It is a chronological series of steps specified by the administrator. The first step is always the submission of the resource. Subsequent steps are either a) assignments, b) parallel reviews, or c) single-person reviews.

In an assignment step, one user orders one or more other users to carry out some other step.

Parallel reviews are a set of reviews conducted by one or more people. While the resource is undergoing parallel review, it may not progress until a specified minimum number of reviews are performed.

A single-person review is a review conducted by exactly one person, and may result in one of the following decisions:

1. Accept – The resource is accessioned into the collection, and may not be removed until an administrator de-accessions it. No further review may be conducted.

2. Feedback – Feedback is requested from the resource contact, and/or the resource contact is requested to revise the resource. When the resource contact answers the request from feedback, the single-person review step is repeated.

3. Redirect – The reviewer indicates that the resource should undergo an additional assignment or single-person review

4. Reject – The resource is rejected from the collection. No further review may be conducted.

The collection administrator and manager are free to create and modify the workflow in any way, subject to these restrictions:

- The first (submitted) step must be followed by an assignment or a single-person review.

- An assignment must be followed by another assignment, a parallel review or a single-person review.

- A parallel review must be followed by an assignment or a single-person review.

- A single-person review may be followed by an assignment, or it may be the final step in the process.

- The final step must be a single-person review.

All resources in the system are reviewed through the same workflow. When changes are made to the workflow, the old version of the workflow is saved. All resources whose peer review was already underway continue with the workflow version that was in effect when they were submitted. All resources submitted after the changes are made will be reviewed under the new workflow.

The substantive work of assessing resource quality is governed, in large part, by the questions asked on review forms which are completed by reviewers in the course of performing reviews. Review forms are managed separately from the workflow. See the section “Review Form Management” for more details.

Branching on Vocabulary

For each parallel or single-person review step, the admin must create a “default branch”, consisting of a review form and a user group which performs the reviews. The admin may also define a “branching vocabulary”.

If the admin defines a branching vocabulary, then he or she can then add as many “alternate branches” as he or she wants (subject to these constraints):

1. Each alternate branch consists of a review form which reviewers must complete; a user group which performs the reviews; and one or more terms which “select” the resources to be routed to that alternate branch; those terms are drawn from the branching vocabulary defined in step 1. For example, if I create a single-person review and choose Discipline as my branching vocabulary, then I can create an alternate branch for those resources which are marked as Ecology, and another alternate branch for those resources which are marked as Microbiology. For the Ecologcy branch, I would create a form with ecology-specific questions and maybe a group of users who are experts in ecology, and assign them to perform the reviews.

2. More than one term may be chosen for an alternate branch, e.g. I could have a single alternate branch selected by both the terms “Ecology” and “Zoology”

3. The same term cannot be used to select for more than one alternate branch within the same review step.

4. The same user group can be used to perform more than one alternate branch. For example, if all I have is one giant pool of Peer Reviewers, then I would always want to assign all of the branches of a parallel review step to them.

View the peer review workflow

Summary The user views the peer review workflow.
Rationale Allows administrators to review the steps in the workflow.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager.
Basic Course of Events 1. The user clicks “View peer review workflow”. 2. The system displays the path through the workflow, beginning with the “Submitted” step and progressing through the final decision step. The display will consist of a simple listing of the steps of the workflow in linear fashion, including specification of the detailed options for each step.
Alternative Paths None
Post conditions No change

Design the peer review workflow

Summary The user describes the peer review workflow.
Rationale Allows peer review to commence according to a well-defined and auditable process.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager.
Basic Course of Events 1. The user clicks “Specify peer review workflow”. 2. The system displays a screen which displays the first step of the current workflow, which is the Submitted step. At the top of the screen is a summary of the workflow, if other steps have already been specified. 3. After the summary, the system displays a list of checkboxes, one for each input process. By checking of an input process, the administrator specifies that resources from that input process should be subject to the peer review workflow. 4. To view or change the details for another step, the user may click on that step in the summary at the top of the screen. 5. For all steps in the process, the bottom of the screen will display some of the following buttons, subject to the restrictions described above: a. Add an assignment step before this step b. Add an assignment step after this step c. Add a single-person review step before this step d. Add a single-person review step after this step e. Add a parallel review step before this step f. Add a parallel review step after this step g. Save changes to this step h. Delete this step i. Save workflow j. Cancel 6. The user adds steps to the workflow or deletes them from the workflow as described in the next several use cases. 7. The user clicks “Save”. 8. The system displays the full details of the workflow, and prompts the user with buttons to “Finish and save”, “Go back” and “Cancel” 9. The user clicks “Finish and save”.
Alternative Paths 1. In Step 8, the workflow is found to have violated one or more restrictions. The system does not display the “Finish and save” button, and displays the reason why the workflow is invalid. The user must go back and fix the problem, or cancel the changes. 2. In Step 8 the user clicks “Go back”. The system returns to Step 2, and the user continues working. 3. In Step 8, the user clicks “Cancel”. The system does not save the workflow to the database, and the user’s work is discarded.
Post conditions A new version of the workflow is created. All resources submitted from this point forward are reviewed according to the new version. All resources whose review was already underway continue with the workflow version which was in effect when they were submitted.

Add an assignment step to the workflow

Summary The user adds an assignment step to the workflow.
Rationale The assignment step allows the system to manage the workload of peer review by distributing it among multiple reviewers or editors.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is in Step 5 of “Design the peer review workflow”.
Basic Course of Events 1. The user clicks “Add an assignment”. 2. The system displays the following inputs: a. Assigning group – a checkbox of user groups to indicate the user groups which are collectively responsible for making the assignment. The administrator will also be able to create a new group which will assume the assigning group role (see “Add new user groups”). b. Working groups – a checkbox of groups in the system from which users may be assigned to work on the resource. The administrator will also be able to create a new user group. c. Minimum number of assignments – a text input specifying the minimum number of assignments which must be made. d. Maximum number of assignments – a text input specifying the maximum number of assignments which may be made. e. Time to complete step – a text input specifying the number of days in which the assignment should be completed once the resource reaches that step in the workflow. f. “Add step” button g. “Cancel” button 3. The user clicks “Add step”. 4. The system verifies that the input is valid. The assignment step is added to the workflow.
Alternative Paths 1. In Step 3, the input is not valid. The user is returned to Step 2, and the errors are reported. The following conditions are errors: a. No assigning group is chosen b. No working group is chosen c. The maximum number of assignments is less than the minimum number of assignments. 2. In Step 3, the user clicks “Cancel”. The use case is ended, and no changes are made to the workflow.
Post conditions The assignment step is added to the workflow, and the workflow summary is updated. Note that no changes are saved to the database at this point. Further, the system will generate a warning, and not activate the workflow when saved, if the next step in the workflow is not a parallel review. If a time to complete is specified, then any users who do not complete assignments in that completion time will receive reminder emails, if those users belong to groups which are set to receive reminder emails.

Add a parallel review step

Summary The user adds a parallel review step to the workflow, making multiple parallel assessments of the quality of the resource possible.
Rationale Multiple parallel assessments allow different users to perform the same review on the same resource, thereby reducing possible bias in the review process.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is in Step 5 of “Design the peer review workflow”.
Basic Course of Events 1. The user clicks “Add a parallel review”. 2. The system displays the following inputs: a. Form – A drop-down list which shows all of the input forms in the system. The administrator selects which form the reviewers must complete as part of the review. b. Assessment options – The administrator specifies the options users have to indicate their assessment. The administrator enters the options in a text area, in a manner similar to the method of entry of terms in a controlled vocabulary in the Cataloging Tool. c. Minimum number of reviews required – A text input for specifying the minimum number of reviews which must be completed in this step before the resource can progress to the next step. d. Time to complete step – a text input specifying the number of days in which the review should be completed once the resource reaches that step in the workflow. e. Enable Branching on Vocabulary section – including a way of specifying a vocabulary to branch on, a way of mapping branches to user-groups for review, a way of specifying which form will be used for each branch, a way to add more branches, and a way to remove branches. f. “Add step” button g. “Cancel” button 3. The user clicks “Add step”. 4. The system verifies that the input is valid. The assignment step is added to the workflow.
Alternative Paths 1. In Step 3, the input is not valid. The user is returned to Step 2, and the errors are reported. The following conditions are errors: a. No assessment options are specified. b. The minimum number of reviews is more than the maximum number of assignments in the preceding step. 2. In Step 3, the user clicks “Cancel”. The use case is ended, and no changes are made to the workflow.
Post conditions The parallel review step is added to the workflow, and the workflow summary is updated. Note that no changes are saved to the database at this point. Further, the system will generate a warning, and not activate the workflow when saved, if the minimum number of reviews required is greater than the maximum number of assignments in the preceding step, or if the preceding step is not an assignment step. If a time to complete is specified, then any users who do not complete reviews in that completion time will receive reminder emails, if those users belong to groups which are set to receive reminder emails.

Add a single-person review step

Summary The user adds a single review step to the workflow, making decisive and final assessments of the quality of the resource possible.
Rationale Single reviews allow a single person to make a decisive assessment on the quality of the resource, thereby guiding its future course through the system.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is in Step 5 of “Design the peer review workflow”.
Basic Course of Events 1. The user clicks “Add a single-person review”. 2. The system displays the following inputs: a. Performing Users – A checkbox to indicate the user group(s) which are together responsible for performing the review, if this step is not preceded by an assignment step. The checkbox will also include options for the users who conducted each of the preceding assignment steps. The administrator will also be able to create a new user group which will collectively be assigned for performing the review. b. Form – A drop-down list which shows all of the input forms in the system. The administrator selects which form the reviewers must complete as part of the review. c. Available redirections – A checkbox to indicate which other steps in the workflow are available for a redirect decision. d. Email template – A text area where the template for the email which authors receive when feedback is requested, the resource is accepted, or the resource is rejected. The template can include the reviewer’s comments, the reviewer’s decision, the resource title, the resource contact’s name, and the resource contact’s email address. e. Time to complete step – a text input specifying the number of days in which the review should be completed once the resource reaches that step in the workflow. f. Enable Branching on Vocabulary section – including a way of specifying a vocabulary to branch on, a way of mapping branches to user-groups for review, a way of specifying which form will be used for each branch, a way to add more branches, and a way to remove branches. g. “Add step” button h. “Cancel” button 3. The user clicks “Add step”. 4. The system verifies that the input is valid. The assignment step is added to the workflow.
Alternative Paths 1. In Step 3, the input is not valid. The user is returned to Step 2, and the errors are reported. The input is not valid if no performing users are selected, and the step is not preceded by an assignment. 2. In Step 3 the user clicks “Cancel”. The use case is ended, and no changes are made to the workflow.
Post conditions The single-person review step is added to the workflow, and the workflow summary is updated. Note that no changes are saved to the database at this point. Further, the system will generate a warning, and not activate the workflow when saved, if no performing users are specified, and this step is not preceded by an assignment step. If a time to complete is specified, then any users who do not complete reviews in that completion time will receive reminder emails, if those users belong to groups which are set to receive reminder emails.

Add new user groups

Summary The administrator creates a new user group, for the purpose of making that group responsible for one or more steps in the review process.
Rationale This use case makes it possible for the administrator to create groups of reviewers and/or editors who will collectively be responsible for one or more steps in the workflow.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is in Step 2.a or 2.b of “Add an assignment step to the workflow” or Step 2.a of “Add a single-person review step”.
Basic Course of Events 1. The user clicks “Add a user group”. 2. The system prompts the user for the user group name, whether users in that group are to receive reminder emails, and how often they are to receive emails (daily, weekly, or monthly). 3. The user clicks “Add group”. 4. The system verifies that the input is valid, and adds the group.
Alternative Paths 1. In Step 3, no group name is specified, or the email frequency is not provided. The user is returned to Step 2, and the error is reported. 2. In Step 3 the user clicks “Cancel”. The use case is ended, and no changes are made to the workflow.
Post conditions The user group is added to the system, and the set of checkboxes for indicating user groups which may perform a review or assignment is updated. Note that users must be assigned to user groups separately, via the user management features in the Cataloging Tool. Users in this group will have reminder emails sent to them at the period of frequency specified.

Review Form Management

View review forms

Summary All of the review forms in the system are displayed.
Rationale Provides a list of all review forms, allowing administrators to make changes as deemed appropriate.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager.
Basic Course of Events 1. The user clicks “View review forms”. 2. The system displays the list of review forms, including name, description, and assigned review states.
Alternative Paths None
Post conditions No change

Add or edit a review form

Summary A new review form is added to the system, or an existing form is changed.
Rationale This use case makes it possible to create a new review form, or modify an existing form, for use in assessing the quality of a resource.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager.
Basic Course of Events 1. The user indicates that a new review form is to be added by clicking “Add review form” 2. The system displays a screen with four panes: the top pane has inputs for specifying the title, description and help text for the form; the middle pane has inputs for adding a new question to the form; the bottom left pane has inputs for editing or deleting each of the questions on the form; and the bottom right pane displays a preview of the form. At the bottom of the screen are buttons marked “Save” and “Cancel”. 3. The next several use cases describe the process of creating or editing the form. 4. When the user is finished creating or editing the form, the user clicks “Save”. 5. The system displays a preview of the review form, as it will appear to the reviewer. At the bottom of the screen are buttons marked “Finish and save”, “Go back” , and “Cancel”. 6. The user clicks “Finish and save,” and the form is saved to the database.
Alternative Paths 1. In Step 4, the user clicks “Cancel”. All of the user’s work is discarded, and the user is returned to the View all review forms screen. 2. In Step 6, the user clicks “Go back”. The user is returned to Step 2 and may continue working on the form. 3. In Step 6, the user clicks “Cancel”. All of the user’s work is discarded, and the user is returned to the View all review forms screen.
Post conditions The new review form is created, or the existing form is modified as specified by the user. This use case builds on the metadata structure, controlled vocabulary, and input process management capacities of Dewey. Each review form corresponds to a single metadata structure and a single input process, with each review form question corresponding to a single metadata structure field and a single input process field. If a review question uses a drop-down list or a set of checkbox or radio button options, the set of options is saved as a controlled vocabulary.

Specify form properties

Summary This use case is a part of “Add or edit a review form”, and takes place during Step 3 of that use case. In this use case, the user specifies properties which apply to the review form as a whole.
Rationale This use case makes it possible to specify the title, handle, description, and help text for the review form.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is creating a new review form or editing an existing one.
Basic Course of Events 1. The system displays the following inputs: a. Title – text input, required b. Handle – text input, required, must be alphanumeric and unique among all other review forms c. Description – text area d. Help text – text area 2. The user enters the title, handle, description, and help text. 3. The user clicks “Submit”. 4. The system verifies that the title is provided, and that the handle is provided, contains only alphanumeric characters, and is unique among all other review forms. 5. The user is returned to the review form editing pane. In the bottom right-hand corner, the preview of the review form is updated.
Alternative Paths 1. In Step 4, the system finds an error in the user’s input. The user is notified that there is a mistake in the form, and the error is indicated. The preview of the form is not updated. The user is returned to Step 2.
Post conditions The user’s input is added to the review form, but is not yet saved to the database. It will be saved to the database when the user saves the entire form to the database.

Add a new question to a review form

Summary This use case is a part of “Add or edit a review form”, and takes place during Step 3 of that use case. In this use case, the user creates a new review question to add to the end of the review form.
Rationale This use case allows the user to add a new review question onto the end of a review form.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is creating a new review form or editing an existing one.
Basic Course of Events 1. The system displays the following inputs: a. Prompt – text input, required b. Handle – text input, required, must be alphanumeric and unique among all other review questions for this form c. Brief help text – text area d. Extended help text – text area e. Data type – drop down list, includes integer, float, string, date, and controlled vocabulary f. Input type – drop down list whose options depend on the data type: single-line text, multi-line text, drop-down list, checkbox, or radio button g. Required – radio button, “Yes” or “No” h. Multiplicity – radio button, “Single” or “Multiple” i. Options – text area, required if the input type is drop-down list, checkbox or radio button; user may enter a hierarchically arranged controlled vocabulary with identifiers for each term. 2. The user enters all the inputs. 3. The user clicks “Submit”. 4. The system verifies that the prompt is provided, that the handle is provided, contains only alphanumeric characters, and is unique among all other questions for this form, that the input type is compatible with the data type, and that the options are provided and are a valid controlled vocabulary, if the input type is drop-down list, checkbox, or radio button. 5. The user is returned to the review form editing pane. In the bottom right-hand corner, the preview of the review form is updated.
Alternative Paths 1. In Step 4, the system finds an error in the user’s input. The user is notified that there is a mistake in the form, and the error is indicated. The preview of the form is not updated. The user is returned to Step 2.
Post conditions The user’s input is added to the review form, but is not yet saved to the database. It will be saved to the database when the user saves the entire form to the database. The new question is added at the end of the review form.

Edit an existing question in the a review form

Summary This use case is a part of “Add or edit a review form”, and takes place during Step 3 of that use case. In this use case, the user modifies a question which was previously added to the review form.
Rationale This use case makes it possible for the user to change properties of the question if the preview of the form reveals that the question did not appear as originally intended.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is creating a new review form or editing an existing one.
Basic Course of Events 1. The system displays the same inputs as those depicted in Step 1 of “Add a new question to a review form”, as well as the buttons “Submit”, “Disable”, “Move up”, and “Move down”. If the question is the first in the form, then “Move up” is not displayed. If the question is the last in the form, then “Move down” is not displayed. 2. The user enters all the inputs. 3. The user clicks “Submit”, “Move up”, or “Move down”. 4. The system verifies that the input is valid and updates the form to include the new inputs. If “Move up” was clicked, the question is switched with the preceding question in the form. If “Move down” was clicked, the question is switched with the succeeding question in the form. 5. The user is returned to the review form editing pane. In the bottom right-hand corner, the preview of the review form is updated.
Alternative Paths 1. In Step 3, the user clicks “Disable”. The question is made inactive on the form, so that reviewers will no longer view it when performing reviews. The preview of the form is updated. 2. In Step 4, the system finds an error in the user’s input. The user is notified that there is a mistake in the form, and the error is indicated. The preview of the form is not updated. The user is returned to Step 2.
Post conditions The user’s input is added to the review form, but is not yet saved to the database. It will be saved to the database when the user saves the entire form to the database.

Preview a review form

Summary A preview of the review form, as it will be displayed to the reviewer, is displayed.
Rationale This use case allows the administrator to verify that the review form appears as intended, before it is used by reviewers.
Users Collection Administrator, Collection Manager
Preconditions The user is logged in as the Collection Administrator or Collection Manager, and is looking at the list of review forms.
Basic Course of Events 1. The user clicks “Preview review form” next to a review form 2. The system displays the review form as it will be displayed to the reviewer during the review process.
Alternative Paths None
Post conditions No change

Assign and Perform Reviews

View pending reviews

Summary The user views all the reviews which are not yet completed and included in the reviewer’s workload.
Rationale The use case allows the user to view pending workload. The pending workload includes a) group or single-person resources assigned explicitly to the user via an assignment step, b) single-person reviews which are the user’s personal responsibility by virtue of the user having assigned the resource for group review in the past and c) single-person reviews which are the responsibility of one or more groups which the user belongs to.
Users Reviewer or editor. Note that precise user groups and tasks for those user groups are specified during workflow management.
Preconditions The user is logged in as a reviewer or editor.
Basic Course of Events 1. The user clicks “View pending reviews” 2. The system displays a list of reviews which are pending for the user. The reviews are grouped by their step in the workflow. 3. For each review, the following items are displayed: a. The resource title b. The other reviewers assigned (whether the name of a group of users assigned, or the names of individuals assigned.) c. The status of the review (incomplete, in progress, completed, unassigned) d. A link to complete the review e. A link to view the record’s review history f. A link to view the complete metadata record 4. At the top of the page, the system displays a link for “View past reviews”.
Alternative Paths If the user clicks the “View past reviews” link, the display will refresh with a list of all past reviews the user has completed, including completed reviews. The link at the top of the page will display “View pending reviews”. The user is returned to Step 2.
Post conditions No change

Perform a review

Summary The user begins a review, resumes a previously-saved review, or completes a review.
Rationale The use case makes it possible for a reviewer to indicate his or her quality assessment of the record. When the review is complete, the record advances to the next step in the workflow.
Users Reviewer or editor. Note that precise user groups and tasks for those user groups are specified during workflow management.
Preconditions The user is logged in as a reviewer or editor, and is looking at the list of pending reviews.
Basic Course of Events 1. The user clicks “Perform review” 2. The system displays a screen with four panes. a. The top pane displays basic details of the record, including its title, summary, URL and step in the workflow process. b. The second pane includes a link to view the complete metadata record, to edit the record, to view the review history for the record. Each link opens in a new window. c. The third pane includes the review form for this record and workflow step. d. The fourth pane includes inputs for a review decision and the reviewer’s overall comments on the resource. At the bottom of the screen are buttons for “Complete review”, “Save but do not complete”, and “Cancel”. 3. The user completes the review form and selects “Complete review” or “Save but do not complete”. 4. The system verifies that the review form was correctly completed, and that an appropriate review decision was issued. 5. The system displays a preview of the review form, the review decision, and the overall comments. At the bottom of the screen are buttons for “Go back”, “Cancel”, and “Confirm”. 6. The user clicks “Confirm”. 7. The review is saved to the database and the system returns the user to the View pending reviews screen. 8. If the review is a single-person review and the user has indicated that the resource is accepted, that the resource is rejected, or that feedback is required on the resource, the use case “Request feedback from or send decision to resource contact” starts.
Alternative Paths 1. In Step 3, the user clicks “Cancel”. The user’s work is discarded, and the user is returned to the View pending reviews screen. 2. In Step 4, the user’s input is not verified as valid. The system returns the user to Step 3, and indicates the user’s errors. 3. In Step 6, the user clicks “Go back”. The user is returned to Step 3 and allowed to continue working. No information is saved to the database. 4. In Step 6, the user clicks “Cancel”. The user’s work is discarded, and the user is returned to the View pending reviews screen.
Post conditions If the user clicked “Complete review”, then the review is saved to the database and marked as complete. The resource advances to the next step of the review workflow, and is no longer pending for the user. If the user clicks “Save but do not complete”, then the review is saved to the database and marked as in progress. The resource does not advance to the next step, and the user is required to return to it at a later time and complete it.

Request feedback from or send decision to resource contact

Summary The reviewer asks the resource contact for more information about the resource, or to request revisions in the resource.
Rationale Allows reviewers to work collaboratively with resource contacts to improve the quality of the resource.
Users Reviewer or editor. Note that precise user groups and tasks for those user groups are specified during workflow management.
Preconditions The user is logged in as a reviewer or editor, and has completed a single-person review, and has indicated that feedback from the resource contact is required.
Basic Course of Events 1. The system displays an email form listing the name and email address of the resource contact, as well as input for the email subject and body. The input for the email body is pre-populated with the email template specified in workflow management. 2. The user modifies the email subject and body as desired. 3. The user clicks “Send”. 4. The system sends the email.
Postconditions If the review decision was Feedback, then the resource is marked as “Awaiting feedback”, and the reviewer will find it when viewing pending reviews. Once the resource contact responds to the email, the reviewer will be able to complete another review on the resource.

View reviews awaiting assignment

Summary The editor sees a list of those reviews awaiting assignment to a reviewer, and assigns them to eligible reviewers.
Rationale Allows editors to move resources through the system in a timely fashion.
Users Reviewer or editor. Note that precise user groups and tasks for those user groups are specified during workflow management.
Preconditions The user is logged in as a reviewer or editor.
Basic Course of Events 1. The user clicks “View reviews awaiting assignment” 2. The system displays a list of reviews which are awaiting assignment. The reviews are grouped by their step in the workflow process. 3. For each resource, the following items are displayed: a. The record title b. A drop-down list displaying all the eligible reviewers, and their current pending workload in parentheses. The list includes “no assignment”. By default, the “no assignment” option is selected for all reviews in the list. c. If it is possible to assign more than one reviewer for the resource, an “Add another reviewer” button will be displayed; clicking it will pop up a another drop-down list, from which the user may select another reviewer. 4. At the bottom of the screen is a button for “Assign reviews.” The user clicks the Assign reviews button. 5. The assignments are saved to the database. The user sees the updated list of pending reviews, and is notified that the assignments succeeded. The assigned reviewers receive an email notifying them of the review.
Alternative Paths None
Post conditions The assigned resources transition to the next step in the workflow.

Review monitoring

View summary statistics

Summary The user sees a numerical overview of the activity in the system.
Rationale Allows administrators to assess the activity in the system.
Users Collection Administrator and Collection Manager
Preconditions The user is logged in as a Collection Administrator or Collection Manager.
Basic Course of Events 1. The user clicks “View summary statistics” 2. The system displays the following statistics a. The number of resources submitted in the system b. The number of resources accessioned into the collection c. The number of resources rejected from the collection d. The number of resources currently under peer review, grouped by workflow step. e. The number of reviewers active in the system, for each group of users eligible to perform reviews or assignments f. The total number of assignments, parallel reviews, and single-person reviews made in the system
Alternative Paths None
Post conditions No change

Monitor user workload

Summary The user sees the list of reviewers and editors in the system, and displays the past performance and current workload for each.
Rationale Allows users making assignments to maximize the efficiency of the system by balancing the workload among each reviewer and editor.
Users Any user making an assignment
Preconditions The user is logged in as a reviewer or editor, and is in the process of making assignments.
Basic Course of Events 1. The user clicks “Monitor user workload” 2. The system displays a list of reviewers and editors in the system, grouped by user group. 3. For each user, the system displays: a. The number of reviews assigned to the user which are not yet completed (the current workload) b. The number of reviews assigned to the user which are in progress c. The average time to complete a review d. The total number of reviews assigned e. The total number of reviews completed
Alternative Paths None
Post conditions No change

View resource peer review progress

Summary The user sees the list of resources in the system, grouped by workflow step
Rationale Allows administrators to review the progress of resources through the peer review system.
Users Collection Administrator or Collection Manager
Preconditions The user is logged in as a reviewer or editor.
Basic Course of Events 1. The user clicks “View resource peer review progress” 2. The system displays a list of resources in the system at the first step of the workflow. If all the resources are being displayed, the resources are grouped by workflow step. 3. For each user, the system displays: a. The resource title b. A link to review the resource peer review history 4. The system displays a link to view all the resources in each of the other steps in the review process, as well as a link to view all the resources in the system at once. 5. The system returns the user to Step 2, displaying the resources at the appropriate step.
Alternative Paths None
Post conditions No change

View resource review history

Summary The user sees a history of the actions taken and comments made on a resource.
Rationale Allows reviewers to see the aggregate quality assessments made by others throughout the peer review process.
Users Reviewer or editor. Note that precise user groups and tasks for those user groups are specified during workflow management.
Preconditions The user is logged in as a reviewer or editor, and is a) performing a review on the resource, b) assigning reviews for the resource or c) viewing the progress of resources through the system.
Basic Course of Events 1. The user clicks “View review history” 2. The system displays a list of the actions taken on the resource, the resource’s progress through the workflow steps, the name of the user who took each action and the time the action was taken. 3. For reach review performed on the resource, the system displays a link to the review form. The user clicks on the link to a review form. 4. The system displays the review form as completed by the reviewer, including the review decision and the comments made on the resource overall. The system displays a link to return to the top of the page. 5. The user clicks the link to return to the top of the page. 6. The system returns the user to Step 2.
Alternative Paths None
Post conditions No change

Reminder Email System

Send reminder email

Summary The system sends the user a reminder, listing all of the tasks which the user is expected to complete, and those which are overdue.
Rationale Ensures that review tasks will be completed in a timely manner.
Users None (all actions performed by the system automatically)
Preconditions None
Basic Course of Events 1. Each night, a reminder email job runs automatically. 2. An email is sent to all users who have been designated to receive reminders. 3. The email contains a summary of the tasks which the user must complete, listing: a. How many assignments the user must make; b. Of those assignments, how many are overdue c. How many single-person reviews the user must complete; d. Of those single-person reviews, how many are overdue e. How many parallel reviews the user must complete; f. Of those parallel reviews, how many are overdue. 4. The email specifies a link to the the web page where the user should log in to complete the assignments. 5. The format of the email will be similar to the following:

Dear Jane Doe,

You have several tasks due in the peer review system:
3 assignments, 1 of which is overdue

5 single-person reviews, 2 of which are overdue
10 parallel reviews, 4 of which are overdue

Please go to http://www.biosciednet.org/dewey/users/login to log in and complete these tasks.

Thank you,

Peer Review System Administrator (admin@biosciednet.org)
Alternative Paths None
Post conditions No change