Open edX open response assessments: when a team sets them up as a separate project
A course team or program owner who needs graded written or file-based work, such as essays, lab reports or project files, has to decide whether to set that work up in-house or bring in a partner. For most teams the decision turns on whether the setup can be scoped, owned and accepted as a separate project before the first course opens. Open response assessments (ORA) are a course component in Open edX that learners complete inside a course and that can be graded by peers, by the learner through self-assessment, or by staff. Because the component is not a single form, its setup touches several parts of the platform at once.
The work crosses four areas. In Studio, course authors create the assessment, its prompts, grading criteria and deadlines as course content. In the LMS, learners submit responses and reviewers grade them, so the grading flow and each role must be tested with real learner and staff accounts. File-based submissions depend on storage the platform can reach, and the permitted file types and size limits need an explicit decision made before authors start building. The grading process itself also needs decisions: who grades, how many reviewers each response receives, and who handles a review that never gets completed.
Confirm the release your platform runs before planning anything. Open edX Verawood is the current named release, published on 30 July 2026, and Open edX supports only the latest named release at any time. Validate every ORA setting, workflow and file rule against the documentation for the release your platform actually runs. If your platform runs an earlier release, the first task is an upgrade decision, and that is a separate piece of work that should be scoped and dated before the assessment project begins.
DeployOpen plans and implements the ORA setup inside your infrastructure, whether that is your cloud accounts, data center or air-gapped network, and it documents the configuration so your team can operate it afterward. DeployOpen does not issue assessment certifications or audit reports. If an accrediting body or auditor needs evidence of how grading was run, that party issues the assessment, and the setup work supplies the configuration records and test results it will examine. Ask any partner, including DeployOpen, to put the scope, deliverables and acceptance tests in writing before work starts.
What Open edX open response assessments do, according to the platform docs
Open response assessments combine three grading modes, a set of learner submission types, a staged path from submission to grade, and reminder notifications. Each part shapes how much grading work the course team carries and how quickly learners get feedback, so confirm each capability against the ORA documentation for the named release before you plan around it.
Peer grading
Learners assess other learners' submissions against the rubric the course team wrote. Peer grading spreads the grading work across the cohort, so the number of active learners and the number of peer reviews each submission receives shape both turnaround and grade reliability. Check in the ORA documentation for your release how peers are assigned, how learners are handled if they have not completed their own submission, and how staff can step in when reviews stall.
Self assessment
Learners rate their own response against the same rubric criteria that peers and staff use. Self assessment builds reflection, and course teams often combine it with another grading mode in the same assessment. Confirm whether self assessment contributes to the final grade in your release, and how its result combines with peer or staff scores when both appear in one problem.
Staff grading
Instructors or designated staff grade submissions directly against the rubric. This mode gives course teams the most control over grading quality, but the workload sits with staff rather than being distributed across learners. Confirm in the documentation which roles can grade, how grader access is granted in the LMS, and how grading progress is tracked across a large cohort.
Submission types and file rules
The docs describe learner submissions as responses that can include text and uploaded files. File uploads need the most planning: ORA2 file storage applies rules on which file types learners may upload, and some types are prohibited. Confirm the accepted and prohibited file types, the upload limits and the storage location for your release, and test each rule with a sample upload before the course opens.
Staged flow from submission to grading
An open response problem moves learners through an ordered set of steps, from submitting a response through peer, self or staff assessment to a final grade. The step order controls when learners can see feedback and grades, and it determines what a learner must complete before the next step opens. Confirm the step names, the transition rules, and what happens to learners who miss a step or a deadline in your release.
Reminder notifications in Verawood
The Verawood release added reminder notifications for open response assessments, which the topic context attributes to that release. Reminders help keep peer and self assessment steps moving, which matters when grading depends on other learners finishing their work. Confirm in the Verawood release notes which events trigger a reminder, who receives it, and whether it is enabled by default and configurable for your course.
Which grading mode and file setup fit each course
This table lists the configuration choices a course team makes for each open response assessment, with the options, the courses each one suits and the cost of choosing it wrong. Read each row before the course opens, because choices such as file types and visibility are hard to change once learners have submitted work. Exact setting names and values belong to the Open edX ORA documentation for your release, so confirm them there before you build.
| Decision | Options | When it fits | Cost of getting it wrong |
|---|---|---|---|
| Grading mode | Peer, self, staff, or a combined flow with several steps | Peer fits large cohorts where learners can assess each other. Staff fits high-stakes work that needs expert judgment. | Peer grading against an unclear rubric produces inconsistent marks and learner complaints. |
| Text or file uploads | Text only, or text plus file uploads | Text fits short written answers. Uploads fit work such as diagrams, code or designs. | Uploads add storage, review time and file handling that a text-only question never needed. |
| Allowed file types | An allow list of specific formats, or a prohibited list of types to block | An allow list fits known deliverables. A prohibited list fits open-ended work where the risky types are known. | A prohibited list can miss formats that learners use or that pose a risk. Test each type before launch. |
| Submission visibility | Learners see only their own work, peers see submissions in peer steps, or staff only | Peer steps need peers to see work. Graded assessments often keep submissions visible to staff only. | Exposing submissions beyond the intended reviewers can breach assessment integrity and learner privacy expectations. |
| Reminder notifications | Off, or scheduled reminders for each open step (added in Verawood) | Reminders fit courses with deadlines across several steps, where learners tend to stall between them. | Reminders set after a deadline has passed, or with no named owner, leave learners unsure what is due. |
| Step timing and deadlines | Fixed dates for each step, or open windows | Fixed dates suit cohorts on a schedule. Open windows suit self-paced courses. | Overlapping or out-of-order step dates block learners from later steps and generate support requests. |
| Grader workload | Size of the staff grading pool and how submissions are allocated | Staff grading fits when reviewers are planned for the expected number of submissions. | Underestimated grading load delays results, and learners wait longer for feedback than the course promised. |
How learner file uploads and prohibited file types are set up
File-based open response submissions need three decisions before launch: where uploaded files are stored, which file types learners are prohibited from submitting, and who is responsible for scanning and retention. This page does not list the storage configuration or the prohibited-type settings for the ORA2 assessment component. Confirm the exact keys, defaults and accepted formats with the Open edX documentation for your release, then check them against your own environment, because the values that work in a single-server test may not hold in a multi-node production deployment.
Start with the storage backend. Learner files are written to whatever file storage the deployment is configured to use, so the team needs to know whether that is a local volume, shared storage visible to every node serving the LMS, or object storage in the client's own cloud or data center. If the deployment runs on more than one node, confirm that a file uploaded through one node can be read by staff graders on another. Ask who provisions capacity, who watches growth, and how the storage tier is sized for the expected number of learners and the file sizes in each assessment.
Backup and retention come next. Uploaded files are part of the learner record, so the file store must be backed up on the same schedule as the database that holds submission records. If the two drift apart, a restored assessment can point to files that no longer exist. Agree a retention period with the privacy or legal owner, and decide what happens to files when a course run ends or a learner asks for deletion. Confirm whether removing a course or a learner's data also removes associated uploads, and test that deletion rather than assuming it.
Uploads also carry personal data the course team did not plan for. A learner may attach an identity document, a medical note or another person's records, even when the assessment prompt does not ask for them. A prohibited-type list controls which formats are accepted, but it does not inspect content. Decide whether the platform, a gateway in front of it or a separate malware scanning service will check files, who maintains that service, and who reviews flagged submissions. Regulatory obligations are covered on the compliance page, but the operational owner for these decisions needs to be named in the project plan.
Test file restrictions before any learner uses the assessment. Build a short test matrix in a staging environment that uses the same storage type as production. It should confirm that each prohibited type is rejected with a clear message, that each permitted type uploads and displays for a grader, that the size limit behaves as documented, that a backup restore returns the files, and that deletion removes them. Ask for the testing effort, the staging environment and the sign-off steps to be written into the fixed-scope proposal, so the team knows which of these checks the partner runs and which remain with its own staff.
Open edX open response assessments rollout in sequence
This sequence moves an open response assessment program from planning to live grading in ordered steps. Each step closes with a named output, so the team confirms the release, rubrics, file policy and pilot results before grading reaches learners.
Confirm the named release and Tutor version
The platform team confirms the Open edX named release the site will run. Verawood is the current named release, and Open edX supports only the latest named release. The team checks that the Tutor version deploys that release, then records both versions in the environment record. Done when the operations lead has approved the release and Tutor versions in writing and they appear in the change log for the target environment.
Define the assessment design for each course
Course leads and instructional designers decide, for each course, which assessments use peer, self or staff grading. They set the number of responses each learner writes, the number of peer reviews each submission receives, and the criteria and point scale in each rubric. They also agree who resolves disputed grades. Done when each course has a signed-off rubric and assessment map approved in writing by the course owner.
Set up storage and file rules in staging
Platform engineers configure file storage for ORA2 submissions in a staging environment. They then define the permitted file types, the maximum file size and the prohibited file types that the institution's policy requires. Testers upload allowed and blocked files, confirm that learners and graders can open what they should, and check how long submitted files are kept. Done when a tested file policy and a staging test log show each rule behaving as written.
Configure grading modes and reminders
The administrator sets the grading mode for each assessment, whether peer, self or staff, along with the rules for peer assignment and staff overrides. They then configure the reminder notifications added in Verawood, agreeing the audience and timing with course owners. Done when a configuration review record lists each assessment's grading mode and reminder settings, and a staging learner account has received the expected notifications.
Run a pilot with a small grader group
Course staff run one or two assessments in a pilot course with a small group of graders and test learners, following the approved rubric. The team checks peer assignment, grade calculation, staff override handling and whether graders apply the criteria consistently. Done when the pilot grading report records completed grades, reported issues, rubric changes and the fixes approved before launch.
Load graders and rubrics into live courses
The course team loads the approved rubrics into each live assessment and assigns grader roles, including staff graders and any named peer pool. The access model agreed with the institution governs these assignments. Platform staff check that permissions match the grader roster and that no grader can reach courses outside their assignment. Done when a roster and rubric check, signed by the course owner, matches the live configuration.
Launch and monitor grading workload
The team opens the assessments to learners, then tracks submission volume, pending peer reviews, ungraded staff submissions, file rejections and reminder delivery. Course owners review these figures against the grading staffing plan at intervals they agree in advance, and the team adjusts grader assignments or reminders where needed. Done when the monitoring report shows all assessments closed with grades released and each open issue has a named owner.
What drives the effort and cost of an assessment setup
This table lists the factors that change how much design, configuration, testing and handover an Open edX open response assessment setup needs. Read each row as a question to answer early, since most drivers are set by the course team before any build starts. The main takeaway is that grading volume, file handling and release validation usually matter more than the number of assessment settings.
| Driver | Why it changes effort or cost | How to keep it down |
|---|---|---|
| Number of courses and assessments | Each ORA needs its own prompt, rubric, dates and grading settings, so configuration and test passes grow with every course that runs assessments. | Standardize a small set of assessment templates and reuse them across courses. Confirm the total count before quoting. |
| Grading mode mix (peer, self, staff) | Peer grading adds reviewer assignment rules, calibration and appeal handling. Staff grading adds grader accounts and queue management. Mixed modes need more end-to-end testing. | Use the mode that fits each task. Limit peer grading to tasks where reviewers can judge quality without subject expertise. |
| File-based versus text-only submissions | File uploads add storage configuration, allowed and prohibited file type rules, virus or content checks where required, and download and retention testing. | Use text-only responses where they meet the learning goal. If files are needed, agree an allowed file type list early and keep it short. |
| Rubric complexity | Rubrics with many criteria, point scales and conditional feedback take longer to design, test for scoring accuracy and explain to graders and learners. | Keep criteria few and observable. Reuse rubric structures across assessments so graders learn one format. |
| Number of graders and their training | More graders increase grader account setup, calibration exercises, support requests and the need to check inter-grader consistency. Training is usually a client task, which DeployOpen can support with documentation. | Train a small grader group first and reuse the calibration material. Confirm who owns grader onboarding before the build. |
| Storage and backup requirements | Uploaded files and submission data are stored outside the database in the storage backend you choose. Retention periods, backup schedules and restore tests must be designed and verified. | Set retention and file size rules before launch. Ask for a restore test as part of handover rather than assuming backups work. |
| Integration with the LMS gradebook and reporting | Scores must reach the gradebook and reports correctly. Each grade export, report view or external reporting pipeline adds data mapping and reconciliation work. | Limit reporting needs to the fields the course team uses. Confirm which reports are native and which need custom work. |
| Number of Open edX releases to validate against | Open edX supports only the latest named release. Each upgrade may change assessment behavior, so each release needs a regression pass on the ORA workflows you run. | Schedule upgrades on a planned cadence and keep a fixed set of ORA test cases. Confirm the target release before design starts, using the Open edX release notes. |
Grading workload at scale and reminder notifications
Grading load in an open response assessment grows with the number of submissions, and in peer grading it grows faster than enrollment. Each submission is reviewed by several peers, so the number of review tasks is roughly submissions multiplied by the reviews assigned per submission. Peer grading spreads that work across learners, but it does not remove it, and a cohort that doubles in size doubles the review tasks that must be completed before a deadline. Confirm the peer assignment settings for each assessment, including how many peers review each submission and how many reviews each learner must complete, before estimating effort for a course.
Staff grading does not spread across learners, so it needs a planned queue before the course opens. Decide who owns each grading queue, how submissions are assigned to graders, how many graders cover each time zone, and who covers the final days before a deadline when submissions peak. Staff grading also carries the most risk after a peer or self-assessment step closes, because every submission that moves to staff review lands on the same few people. Build reviewer capacity from the course calendar, then test it against a pilot cohort rather than assuming a graders-per-learner ratio will hold.
Reminder notifications help learners finish peer review steps and help graders clear queues on time. Verawood added reminder notifications for open response assessments, but the team must confirm the exact behavior in the Verawood release notes and in the installed version. Check which events trigger a reminder, who receives it (learners, graders or both), when reminders are sent, whether each assessment can set its own timing, and how the message is delivered. Notification volume also scales with cohort size, so confirm that the mail service in your infrastructure can send the expected number of messages within a reminder window.
A load test in staging should use submission counts that match the expected cohort, not a small sample. Build test data from the real course structure, including the number of assessment steps, the number of peer reviews per submission and the number of graders assigned to each queue. Simulate the deadline spike, when many submissions and reviews arrive in the same hours and reminders fire at the same time. Measure page response times, grading queue lag, background task backlog and notification delivery, and record the thresholds the course team accepts before go-live. Ask the client to confirm the expected counts in writing.
Grader calibration and dispute handling need a written plan before the first grade is released. Calibrate graders by having each one score the same set of sample responses against the rubric, then compare results and agree on the acceptable spread with the instructor. Repeat calibration during the course if grading drifts, and record each change to the rubric or guidance. Define the escalation path for disputed grades: who receives the learner request, who performs the second review, how long the decision may take, and who has final authority. Keep a log of every dispute and its outcome so the course team can explain decisions afterward.
How open response assessments connect to the rest of Open edX
An open response assessment touches authoring, grading, roles, sign-in, notifications and file storage at once, so a change in one area can surface in another. The cards below list each connection, what to decide about it, and what to confirm before you finalize the design. Reporting through Aspects, if your organization uses it, consumes the same learner activity and should be planned alongside these.
Studio authoring flow
Course teams add open response assessments to units in Studio, where they define the prompt, rubric, grading steps and due dates. The authoring decisions affect what learners and graders see later, so course designers and the platform team should agree on a template before content is built. Check how the component is added and edited in the Studio version you run, and confirm whether Verawood's Studio sidebar changes the steps your authors follow. Test an assessment end to end in a sandbox course before copying the pattern to live courses.
LMS gradebook and learner progress
Scores and completion from assessment steps feed the LMS gradebook and the learner progress views. Decide with the registrar or academic office which grade is authoritative when peer, self and staff steps combine, and how late or resubmitted work is recorded. Confirm with your own team how exported grades reach any external system, because the export path is a separate piece of work. Verify the gradebook output with a test learner and a test grader before the first real cohort starts.
Course roles and instructor access
Who can see learner responses, submit grades and override outcomes depends on course roles such as staff, instructor and course team membership. Map these roles to the people who actually grade, and keep the list short so that access reviews stay manageable. Review how instructor dashboard changes in Verawood affect what instructors can see for assessments. Document who can change roles, and confirm the process with your own team before course launch.
Identity provider sign-in
Learners and graders sign in through the identity provider you configure. The Open edX docs list OAuth2 providers, SAML 2.0, Shibboleth, LTI, and older provisional options such as CAS and SSL client certificates, so the choice affects how graders are provisioned and how accounts are matched. Confirm with your identity team which protocol your directory supports, how account attributes map to course roles, and what happens when a grader leaves. Check the current third-party authentication page in the Open edX docs before you commit to a provider.
Platform notifications for graders and learners
Open edX sends notifications about assessment activity, and Verawood added reminder notifications for open response assessments. Reminders can change how many peer responses are completed on time, so decide with course teams which reminders are needed and when they should fire. Confirm which notification channels your deployment uses, and whether any messages need to pass through your email relay or a mail policy. Test the messages with real addresses in a staging environment, since delivery problems often appear only there.
File storage in your infrastructure
Learners can attach files to responses, and the platform must store those files somewhere you control. Define the allowed and prohibited file types with your security team before launch, since the prohibited list is the main control over unsafe uploads. Confirm which storage backend holds the files, how long they are kept, who can retrieve them, and how they are included in backups. Plan the storage capacity against the expected volume of uploads for each course, and review the setting after each upgrade.
Questions buyers ask before hiring help with Open edX open response assessments
What drives the cost of setting up open response assessments in Open edX?
The cost depends on how many assessment types you run (peer, self and staff), how many courses use them, and how much of the storage, identity and deployment work already exists. One-time costs usually include discovery, configuration of the assessment settings, file storage setup, grader workflow design, testing and handover documentation. Recurring costs include the storage you pay your cloud or data center provider, hosting, upkeep after platform upgrades and any grading operations you buy from a partner. Your internal time for writing rubrics, selecting graders and approving test results is also a cost, even though it may not appear on a vendor invoice. Ask for a quote that separates one-time implementation from recurring operations, names who performs each task, and lists the assumptions about storage and course volume.
How long does an open response assessment setup take to complete?
The timeline depends on the scope agreed in writing, and this page does not publish a duration. The main drivers are the number of assessment types, the file storage backend and its access approvals, the state of your existing Tutor deployment, the number of courses in the first release, and how many rounds of rubric and grader review your team requires. Timelines also move with your internal approval steps and the release schedule of the Open edX named release you run. Before you commit, ask for a written schedule with milestones, the dependencies on your team and storage owners, and the test cycle that must finish before any live course opens to learners.
What does our team need to provide before open response assessment work starts?
Your team needs to provide several inputs before implementation begins. These include the rubrics for each assessment, grader lists for any staff-graded items, and a decision on which file types learners may upload and the maximum file sizes. Your storage team must grant access to the bucket or file store, with credentials handled under your own secrets process. You also need a named approver who can accept test results and sign off on production. Test learners, a list of pilot courses and the reminder notification preferences should be ready early. Ask the implementation partner to confirm this list in writing so gaps appear during scoping, not during rollout.
Can our own course team set up open response assessments without outside help?
A course team can create a single open response assessment in Studio without outside help, provided the platform already runs and the storage behind file uploads works. Scaling the setup is a different task. Teams need to configure the shared storage, check permissions for uploads, verify file type and size rules, set up the cache and deployment settings for multi-node operation, and test reminder notifications. Platform engineers also need to plan how upgrades will affect assessment data. If your platform team already runs Tutor and understands Django-based services, the work is manageable in-house. If not, a written scope with a clear handover lets you decide which parts to keep.
Can DeployOpen run grading operations for our open response assessments?
DeployOpen can run grading operations only when both parties agree to a retainer in writing. Most engagements are fixed-scope, so grading operations are not part of the standard implementation unless the written scope says so. Where a retainer is agreed, the scope should state which tasks are included, such as monitoring grading queues, supporting staff grader assignments, reviewing storage use, handling escalations and applying platform fixes. It should also state response expectations, reporting, and what your team still approves. Ask for the retainer terms in the same document as the implementation scope so responsibilities are clear from the start.
What should a written scope and quote for open response assessments include?
A written scope should name each assessment type and course, the storage and identity systems involved, and the environments covered, such as development, test and production. It should list deliverables, including configuration records, test evidence, runbooks and training materials, along with the acceptance criteria that define completion. The quote should separate one-time work from any recurring retainer, state which costs are paid by you directly (cloud storage, licenses where applicable) and which are paid to the partner, and list every assumption about course volume and file storage. It should also name your approver, the change process for scope additions and the process for handover after go-live.
What usually breaks first with file uploads in open response assessments?
File uploads usually fail at the boundaries between systems rather than inside the assessment logic. Common problems include storage permissions that differ between environments, file type rules that do not match the policy your security team approved, and size limits enforced by a reverse proxy or load balancer before the platform sees the request. Missing or inconsistent cache settings in multi-node deployments can also cause odd behavior during grading. Reminder notifications may stop after a configuration change if the mail path is not tested. Verify each path with test learners, check that files remain readable after an upgrade, and confirm that retention and deletion rules match your data policy. Confirm the exact behavior of your release with the Open edX release notes.
How DeployOpen scopes and delivers open response assessment setups
DeployOpen scopes each Open edX open response assessment setup as a written, fixed-scope piece of work inside your own infrastructure. We plan the grading model, file upload rules and reviewer workload before configuration begins, then build, test, document and hand over the setup. Ongoing support is available as a retainer.
How we scope
- Written scope that names each assessment type: peer, self and staff grading
- File upload rules, including prohibited file types, agreed with your assessment owners
- Grading workload plan per course, with reviewer capacity confirmed before launch
How we build
- Configure assessment prompts, rubrics and due dates in a test course before production use
- Set file storage and prohibited file type rules, then test uploads from each learner role
- Configure Verawood reminder notifications and confirm delivery against your mail setup
How we hand over
- Runbook covering assessment setup, file storage settings and grading reviewer procedures
- Test records showing peer, self and staff flows completed from submission to grade
- Upgrade notes for future Open edX releases, since only the latest named release is supported
How we support
- Optional retainer for monitoring, configuration changes and release-specific upgrade work
- Support for grading workload questions as course enrollment and submission volume grow
- Confirm your current release and file storage backend with your team before scoping starts
