ResumeLinkedIn design.raviteja@gmail.com

"Transforming Peer Review Process in
SUNY Education system"

Client

My Role

UI UX Designer &
Usability Lead

Timeline

8 Months

Product

EdTech - Web Application

Tools

Figma, Miro, Discord,
Google Sheets, Jira

About Project

The primary objective of the SUNY and IBM partnership was to create an innovative Calibrated Peer Review (CPR) system that leverages advanced technologies to enhance the peer review process within SUNY's educational environment.

Problem Statement

The traditional peer review process is both time-consuming and places a heavy burden on instructors to evaluate student assignments, often resulting in challenges to deliver timely feedback to the students.

The Solution
Research, Strategize, and design an end-to-end PWA (Progress web application) experience to automate the manual peer process, with a focus to launch MVP product in 10 SUNY universities to validate initial concept testing on real time users.

The design process we adapted!

Empathize - 2 Weeks
  • Project introduction
  • Defining Project goal
  • Project Timeline
  • Communications
  • User research
Ideate - 3 Weeks
  • Heuristic evaluation
  • Personas
  • Empathy Mapping
  • Hierarchal task analyses V 2.0
  • Identifying MVF
  • Information architecture
Design - 4 Weeks
  • Initial Design Ideas
  • Low fidelity wireframes
  • Design System
  • Review sessions
  • High fidelity wireframes
  • Interactive Prototypes
  • Dev handoff
Testing - 3 Weeks
  • Expert Evaluation
  • User testing
  • Feedback review
  • Data analysis
Iterations - 2 Weeks
  • Design Iterations
  • Stakeholder presentations
  • Suggestions future scope
  • Handoff documentation.
The Process
Familiarizing Current Product
Activities:
  • Heuristic Evaluation
  • Hierarchical task analysis
  • Analyzing existing usability report
Key Takeaways
  • Lack of design discipline
  • No Visibility of system status
  • Lack of consistency in the navigation structure
  • Lack of control and freedom for instructor
  • Features and functionality are broken.
  • Lack of accessibility consideration
Detailed Analysis
Initial version of the product
Dashboard
Assignments Screen
Rosters Screen
User Research
Conducting some ground work interviews

To validate our assumptions we conducted primary research and get into the roots of the problem.

The research revealed a different story!

01

Instructor needs weren’t fulfilled

"I've employed peer review tools; however, those platforms didn't provide the capability to customize as per my preference."

02

Availability of the data

"The course materials and assignment listings I've curated hold significant importance for me. There are instances when I intend to leverage them for future semesters."

03

Ability to manage student data

"If needed I should be able to manage or edit student information."

Competitive Analysis
But, how the existing products workings?

The insights generated from the  interviews guided us in evaluating the existing products to identify the strengths, weakness and opportunities

Canvas
Brightspace

Outlining the primary users

Name: Catharine

Behavior: Active, Distracted,
Friendly

Secondary User - Student
Goals and Needs:
  • Students want to receive email notifications about updates to the Calibrated Peer Review application to stay informed of any changes that may affect their work or submissions.
  • I would like the ability to communicate with my teammates within the application, especially in cases where I don't know my teammates well.
Pain Points:
  • I'm worried that my friends might become upset if they find out that my feedback contributed to them losing marks.
  • I often find myself unfamiliar with my classmates in a given course, making it quite frustrating to search for their email addresses when I need to communicate.

Identified MVF for MVP launch

Instructor Minimum Viable Feature
  • Personalized navigation feature that allows users to easily view their past semester/assignment coursework.
    ‍
  • Instructor should be able to download all course materials based on their preferences.' This includes the
    ability to download by course, assignments, peer reviews, student, teams, and years.
    ‍
  • Instructor should have the ability to create assignments without peer review.
    ‍
  • Instructor should have the ability to view students' grade overviews by individual, teams, and assignments.
    ‍
  • Instructor should be able to manage student and team information.
    ‍
  • Instructor should be able to edit or assign final grades for peer review assignments.

"Worked collaboratively with the team to create user flows for all the user task!"

Mapped user stories with hierarchical task analysis with a clearer perspective.

After finalizing the minimum viable features for version 2.0, we created an updated hierarchical task analysis that incorporates new features and functionalities, which allowed us to identify areas for potential improvement in the CPR user experience.

Detailed Analysis

Low-Fidelity Prototype

We started sketching out our initial ideas for dashboard/home screen. All the sketches are done to get the initial feedback from the stakeholders.

Accessibility Guidelines - WCAG Level AA

Our team is proud to announce that the CPR web application has been designed in compliance with the WCAG Level AA guidelines. Although it would have been optimal to prioritize all scenarios of web accessibility guidelines, we recognize that the MVP launch was subject to certain constraints that prevented the full implementation of these guidelines.

Design System

Presenting CPR Web Application

Solutions - 1
Personalized navigation feature

The implementation of a personalized navigation feature greatly facilitate instructors in efficiently accessing and managing pertinent information from both current and previous semesters or years.

Solutions - 2
Bulk Download

The bulk-download feature enables instructors to efficiently download all course materials, including course information, assignments, peer reviews, student and team details, and past years' data, based on their preferences.

Solutions - 3
Instructor Manage Student Teams

In version 2.0, we have enhanced the instructor's autonomy in the application by enabling them to efficiently manage student team information, providing them with greater flexibility and control over the student team dynamics.

Solutions - 4
Detailed Grades Overview

Instructors now have the capability to sort and view student grades based on specific criteria, such as assignments, individual students, and teams.

Solutions - 5
Assignment With out Peer Review

Instructors now have the ability to customize peer review assignments, enabling or disabling them based on the specific requirements of the course/assignment.

Solutions - 6
Alert System

The integration of the alert system feature into the CPR application has enhanced user experience significantly. This enhancement involves the provision of email notifications as well as in-app notifications to both user categories, namely instructors and students.

High-Fidelity Prototype

Peer Review Submission

Peer Review Assignments Details

Instructor - Create a Course

Instructor - Create a Assignment

Instructor - Send Peer review

Instructor - Team Management

Instructor - Roster

Instructor - Peer review Matrix

Testing our solution with users

We have conducted successful usability sessions involving more than 25 participants, comprising both instructors and students. The experiment centered on creating courses within the application, managing teams, distributing peer review assignments to students, and submitting peer reviews.

Design Iteration - 1

Ensuring constant visibility of navigation content: A considerable number of instructor users currently face difficulties in locating the method to access course materials from previous semesters.

Before User Testing

After User Testing

Design Iteration - 2

Taking into user insights from qualitative data, we iterated bulk download as a global functionality. This feature will be consistently accessible within the main navigation.

Before User Testing

After User Testing

20+

Instructors have expressed satisfaction with the new navigation structure of the application, reporting improved ease of use.

82%

Users are able to create course & assignment in less than 1 minute.

88%

Instructor users are able to manage students and teams in less than 90 seconds.

92%

of instructor users were able to download student assignments based on their preferred specifications.

Suggested improvements in version 3.0

  • Accessibility: Ensure that the application is supportive of all assistive technologies and overcomes any accessibility barriers or issues that may be encountered by users with disabilities.
  • Enhanced Collaboration: Incorporate features to facilitate better collaboration among team members, file-sharing capabilities.
  • Increased Customization: Provide greater flexibility for users to customize the application to their preferences.
  • Improved Performance: Optimize the application's performance to enhance speed and reliability.