10. Worked Project — From Question to Defensible Conclusion
10. Worked Project — From Question to Defensible Conclusion
This studio applies the whole course to one project.
1. Topic, puzzle, and question
Topic: peer feedback in research writing.
Puzzle: review-level evidence is broadly positive, but “peer feedback” combines different task designs, and reported perceptions do not show which comments produce better revisions.
Question:
Among first-year undergraduates in a six-week methods module, what is the effect of rubric-guided peer feedback, compared with usual peer comments, on blind ratings of research-abstract quality, and which feedback features are taken up in revision?
The first part is causal and quantitative; the second is process-oriented.
2. Scope and feasibility
| Decision | Choice | Reason |
|---|---|---|
| participants | 48 consenting students in one module | accessible within six weeks |
| product | 250-word abstract, draft and final | comparable unit of writing |
| intervention | four rubric prompts | clearly distinguishable condition |
| comparison | usual open comments | realistic alternative |
| primary outcome | change in blind 20-point rubric score | directly represents selected quality criteria |
| process evidence | comment-to-revision links | tests uptake mechanism |
| boundary | one course and institution | prevents over-generalisation |
Ethical issues include consent, power relations, anonymisation of student writing, and ensuring that course grades do not depend on research participation.
3. Search trail
Core query:
("peer feedback" OR "peer review")
AND ("academic writing" OR "research writing" OR abstract*)
AND (undergraduate* OR "higher education")
Selection rule: retain studies or reviews that examine peer feedback on higher-education writing and report writing, revision, feedback, or implementation evidence.
Challenge searches:
"peer feedback" AND writing AND (null OR limitation OR challenge)
"peer feedback" AND writing AND (rubric OR structured OR uptake)
4. Mini evidence matrix
| Source | Evidence | Contribution | Boundary |
|---|---|---|---|
| Huisman et al. (2019) | meta-analysis of 24 quantitative higher-education writing studies | average performance evidence; task design as moderator | few studies in some comparisons; heterogeneity |
| Wei & Liu (2024) | systematic review of 60 empirical studies from 2014–2024 | maps benefits, challenges, samples, and methods | many undergraduate and qualitative studies |
| Lu et al. (2023) | path analysis with 114 undergraduates | links feedback features, acceptance, revision, and performance | observational pathway in one context |
| Zyska et al. (2026) | public proposal–feedback–revision dataset with human ratings | demonstrates criterion-level linkage of drafts, reviews, and revisions | computer-science course; preprint/dataset paper |
The studies perform different jobs. None alone answers the proposed question.
5. Synthesis and gap
Synthesis
Peer feedback is generally associated with improvement in higher-education writing, but effect estimates combine varied comparisons and task designs. Process evidence suggests that comment characteristics and acceptance matter, while newer draft–feedback–revision datasets make criterion-level uptake observable. The literature therefore supports testing a structured pathway—prompt → comment → uptake → revision—rather than treating exposure to feedback as the mechanism.
Gap
It remains uncertain whether rubric-guided peer comments improve short research abstracts for novice university writers relative to usual comments, and whether any score difference is accompanied by more criterion-relevant uptake.
This gap is narrower than “peer feedback has not been studied.” It identifies a design, population, product, comparison, and mechanism.
6. Design alignment
| Element | Decision | Threat controlled | Threat remaining |
|---|---|---|---|
| assignment | randomise within seminar | baseline group differences | non-compliance |
| rating | two raters blind to condition/stage | expectation bias | rubric captures limited dimensions |
| primary analysis | compare mean draft-to-final change | common baseline/time trend | small-sample uncertainty |
| process analysis | code comment specificity and matched revision | tests proposed pathway | coding subjectivity |
| missing data | report by condition; analyse assigned groups | selective completion | assumptions about missing outcomes |
Primary outcome and coding rules should be fixed before viewing group results.
7. Simulated results
| Outcome | Structured feedback | Usual comments | Estimated contrast |
|---|---|---|---|
| mean draft score / 20 | 12.1 | 12.4 | −0.3 |
| mean final score / 20 | 15.7 | 14.2 | +1.5 |
| mean change | +3.6 | +1.8 | +1.8 |
| actionable comments | 72% | 41% | +31 percentage points |
| comments linked to a revision | 63% | 38% | +25 percentage points |
Simulated 95% confidence interval for the difference in score change: 0.4 to 3.2 points.
What may be said
In this simulated randomised class study, rubric-guided feedback increased the average draft-to-final score change by an estimated 1.8 points relative to usual comments. More comments were actionable and linked to revisions, which is consistent with the proposed uptake pathway.
What may not be said
Structured peer feedback always improves university writing by 1.8 points.
The study estimates an average effect for one task and setting. The process pattern supports, but does not uniquely prove, the mechanism.
8. A C–E–W–L discussion paragraph
Rubric-guided feedback produced greater improvement than usual comments in this class (claim). The structured group showed a larger mean score change, and its comments were more often actionable and matched to revisions (evidence). Because the prompts targeted the same criteria used to assess the abstract, they may have helped reviewers identify changes that writers could apply (warrant). However, the small single-module sample, imperfect measure of uptake, and short follow-up limit precision and transfer to other writing tasks (limit).
9. Abstract built from research jobs
Background: Peer feedback can support academic writing, but outcomes vary with task design and uptake. Question: We tested whether rubric-guided feedback improved first-year students' research abstracts relative to usual comments. Method: In this simulated study, 48 students were randomised within seminars; blinded raters scored draft and final abstracts, and comments were linked to revisions. Results: The structured group showed a 1.8-point greater mean improvement (simulated 95% CI 0.4–3.2), with more actionable and revision-linked comments. Conclusion: Structured prompts improved outcomes in this class and the process evidence was consistent with greater uptake, but the small, single-task setting limits generalisation.
Each sentence has one job. No sentence claims more than the simulated design supports.
10. Revision after peer review
| Reviewer comment | Decision | Revision |
|---|---|---|
| “Quality sounds too broad.” | accept | define the four rubric criteria |
| “The mechanism is proven.” | accept | change to “consistent with” and name rival pathways |
| “Add student satisfaction.” | adapt | report as secondary evidence, not a quality outcome |
| “Generalise to all undergraduates.” | decline | retain the one-module boundary |
The best revision changes inference, not merely wording.
11. Five-minute defence
- Question: local causal effect plus uptake pathway.
- Strongest evidence: randomised contrast in blind score change.
- Main limit: one small module and one short text genre.
- Rival explanation: differential engagement or interaction between students.
- Next study: replicate across disciplines and test whether criterion knowledge moderates the effect.
Transfer the pattern to your project
Replace the topic, but preserve the chain:
puzzle → bounded question → transparent search → compared evidence → precise gap
→ aligned design → calibrated result → bounded conclusion → documented revision
Return to the course overview or copy the templates from the research workbook.