Overview
Most assessment problems fall into three groups. Validity: the task does not actually require the outcome it claims to assess (a multiple-choice test for 'design a marketing plan'). Reliability: two markers give the same script very different grades because criteria are vague. Usefulness: results are recorded as a single grade and cannot tell the program which outcomes students struggle with.
We address all three. Assessment tasks are redesigned from the outcomes backward, with authentic formats where the outcome demands them. Rubrics are written with distinct criteria and observable performance descriptors at each level, piloted on real (anonymized) student work, and refined after a calibration exercise with your markers.
Because well-built rubrics produce criterion-level scores, the same data can feed assurance of learning or program assessment reports — for example AACSB assurance of learning or ABET student outcome assessment — without a separate testing exercise.
Who it’s for and the problems it solves
This is for you if you are…
- Program leaders preparing assurance-of-learning or outcome assessment evidence
- Departments with marking inconsistency between sections, campuses or teaching assistants
- Faculty rethinking assessment in light of generative AI tools
- Professional programs designing practical, clinical or portfolio assessment
- Training centers introducing competency-based assessment
Common problems we help with
- Rubric criteria overlap, so one weakness is penalized three times
- Descriptors say 'excellent', 'good', 'poor' without saying what those look like
- Grade distributions differ widely between markers or campuses
- Assessment data cannot be broken down by outcome for accreditation reports
- Take-home essays are now easy to complete with AI tools and staff are unsure what to change
- Students contest grades because criteria were not shared or explained
What’s included — and what isn’t
Included
- Review of current assessment tasks and their alignment with course and program outcomes
- Program-level assessment map: which outcomes are assessed where, how often and by what method
- Redesign of selected tasks, including authentic assessments, staged submissions and oral components
- Analytic or holistic rubrics, single-point rubrics for formative feedback, and marking guides for exams
- Performance descriptors written in observable terms, with anchor examples where available
- Marker calibration and moderation workshop using anonymized scripts
- Assessment-for-AI-era review: which tasks are vulnerable and redesign options aligned to your policy
- Templates to extract criterion-level data for assurance of learning or program review
- Student-facing guidance: assignment briefs and 'how you will be assessed' explanations
Not included
- Marking student work or setting final grades
- Writing assessments whose content the responsible faculty have not reviewed and approved
- Determining misconduct cases or AI-detection judgments on individual students
What you receive
- Assessment audit
- A table of current tasks with alignment, validity concerns and workload notes, plus prioritized recommendations.
- Program assessment map
- Outcome-by-course matrix showing summative assessment points and any outcome that is never assessed.
- Rubrics and marking guides
- Editable rubrics, ready to import into your LMS, with guidance for markers.
- Calibration report
- Results of the moderation exercise, where markers diverged and the rubric edits that followed.
- Evidence templates
- Spreadsheets that convert rubric scores into outcome-level results with targets and action-plan fields.
How it works
Discovery
Agree the courses and outcomes in scope, collect current briefs, rubrics and anonymized samples of student work.
Design
Draft redesigned tasks and rubrics with the course team in one or two working sessions.
Pilot and calibrate
Markers independently score the same anonymized scripts; we analyze agreement and facilitate a moderation discussion.
Refine
Rubrics and briefs are revised based on the calibration, then approved by the course or program lead.
Handover
Final rubrics, LMS-ready files, evidence templates and a short guide for new markers.
Assessment design principles and tools
- Backward design
- Wiggins and McTighe's sequence — decide the evidence of learning before planning instruction — applied to each task.
- Analytic vs holistic rubrics
- Analytic rubrics give criterion-level feedback and data; holistic rubrics are faster for complex performances. We recommend per task.
- Single-point rubrics
- One column of criteria at the expected standard, with space for comments above and below, useful for formative feedback.
- Inter-rater agreement
- Simple percentage agreement and, where numbers allow, weighted kappa or intraclass correlation to show whether calibration improved consistency.
- Authentic assessment
- Tasks that mirror professional practice (briefs for a client, case analyses using local data, lab design) and make outsourced or AI-generated answers less useful.
- Assurance of learning cycle
- Define goals, measure with direct and indirect evidence, analyze, act, and re-measure — the closing-the-loop logic accreditors look for.
Anatomy of a rubric markers can agree on
Reliable rubrics have criteria that are independent of one another, levels described by observable features rather than adjectives, and a manageable number of levels (three to five). Weighting reflects the outcomes, not the effort involved.
- Criterion: 'Use of evidence' — not 'Quality of work'
- Descriptor at meets-expectations level: 'Supports each main claim with relevant, cited sources and explains how the source supports the claim'
- Avoid double-counting: grammar judged once, not inside every criterion
- Anchor examples: a short excerpt showing each level where possible
Rethinking assessment alongside generative AI
No single detector or ban solves the problem, and AI-detection scores are uncertain indicators, not proof. We help teams classify tasks by vulnerability and choose proportionate responses: in-class or oral components, process evidence such as drafts and reflections, tasks requiring local data or personal experience, or explicit permitted use of AI with acknowledgment. The approach must match your institution's policy.
What the work can look like
Illustrative example — fictional data
Illustrative excerpt: calibration result
Six fictional markers on a large first-year business writing course scored the same eight anonymized reports before and after a rubric revision. Numbers are invented.
Before revision
Markers agreed on the same level for the 'Argument' criterion in 44% of cases; two markers systematically placed work one level higher.
Changes made
The 'Argument' and 'Structure' criteria were separated; the descriptors replaced 'clear and logical' with observable features (thesis stated in introduction, each section advances a claim). Two anchor excerpts were added.
After revision
Agreement on 'Argument' rose to 75% in a second round. Remaining disagreement concerned use of sources, which was scheduled for the next moderation meeting.
Illustrative example with invented numbers; actual results depend on your tasks and markers.
What you’ll need to provide
- Course and program learning outcomes
- Current assignment briefs, exams, rubrics and marking schemes
- Anonymized samples of student work at different grade levels, with appropriate consent or policy basis
- Marker availability for a calibration session
- Institutional assessment regulations, grading scale and AI-use policy
You don’t need everything ready to ask for a quote — tell us what you have and we’ll say what’s missing.
What affects price and turnaround
Price depends on
- Number of courses, tasks and rubrics in scope
- Whether a calibration workshop is included and how many markers take part
- Program-level mapping and evidence templates versus course-level work only
- Availability of anonymized student work for piloting
Turnaround depends on
- Assessment calendar: rubrics need approval before tasks are released to students
- Marker availability for calibration sessions
Every project is quoted individually. How we price our work.
Questions people ask before starting
Can rubrics be imported into our LMS?
Yes. We provide rubrics in a structure that can be entered or imported into Moodle, Canvas or Blackboard rubric tools, and we can build them directly if we have course access.
How do we handle student work in calibration without breaching privacy?
We use anonymized scripts selected by your staff under your data protection rules and consent practices. Work is shared through channels you approve and deleted after the engagement.
Will this satisfy our accreditor's assurance-of-learning requirements?
It produces the kind of direct, criterion-level evidence that assurance-of-learning processes rely on. Whether your overall process meets a standard is judged by the accrediting body, and you remain responsible for running the cycle.
Should we stop using essays because of AI?
Not necessarily. Essays still assess valuable outcomes. The question is how to make the process visible and the task specific enough that producing it without learning is harder or less useful. We help you decide task by task.
Do you design exams too?
Yes, including exam blueprints and marking guides. For large banks of objective items, see our question bank service, which adds item writing and item analysis.