Crowdsource Grading on Digital Assignments

This semester, I’m trying new assessment methods with my graduate students: contract grading and crowdsource grading (for major projects). For me, crowdsource grading is not just about students making decisions about the quality of their peers’ work; it’s also about working together as a class to decide what in fact constitutes quality work. (Or in our case, what constitutes an “A,” “B” and “C” grade on major projects.)

I’ve been collaboratively creating assignment criteria with my digital writers for a couple of years now. My method is informed by the work of **Chanon Adsanatham, who leads students through a number of scaffolded activities that ultimately result in student-generated grading criteria, and ***Jody Shipka, who asks students to turn in a “Statement of Goals and Choices” (SOGC), a statement that documents their rhetorical decisions, along with their digital projects. My method is best described as Adsanatham+Shipka+genre analysis. I ask students to collaboratively conduct a simplified version of a genre analysis (what is successful/effective about this project? What is ineffective/unsuccessful about this project?) as a way to generate assessment criteria. Based on our collaborative work together, I design a rubric, share it with the students, and make any adjustments they feel is necessary. My students also turn in a reflection essay (or what Shipka would call SOGC) along with all digital projects.

For this digital project, I opted not to do the Tanya method in class. Rather, I asked students to make a list of assessment criteria and from these lists, we’ll generate the “final” version together on Thursday in class.

As I’ve been reviewing the articles I assigned this week, I keep coming back to my learning goals and outcomes for this class. One stated outcome is “Students will possess knowledge about how theory and concepts related to digital writing (as studied and taken up in the discipline of Composition/Rhetoric) informs the production of digital writing.” (I should have written analysis and circulation as well!) Over the past four weeks, we’ve read, discussed and pulled apart various theories and concepts. If we thought of how all of these theories and concepts inform the production of digital writing all at once, our heads would explode. If we, as digital writers, had to constantly return to, and consider, everything we’ve learned about say, the function of images, while we’re composing, our heads would explode.

In both of my DW courses, I keep reminding students that writers make choices. Digital writers make choices. We think about how we might best achieve our communicative purpose with regards to our audience. We think about how we can achieve particular effects, how we can convey meaning, how we can utilize modalities to make our work accessible to a wide range of people, for example.

This brings me back to assessment and the question I’d like us to ask ourselves: what theories, concepts or themes might inform the choices we make as digital writers while composing this project. When we discuss assessment criteria for this project, I will offer up one way we can think about assessing this project: focus our attention on one or two concepts, themes or theories, allow them to guide us in composing the project, and use them to determine what constitutes quality work (or in other words assessment criteria). (I think it’s important for us to really think about the fact we are creating moving images not still images. Much of the scholarship we’ve read focuses on still images, and there’s a big difference between the two.)

Below are some possibilities (many of which overlap):

1. Audience involvement, identification, and role in the meaning making process
2. Modality affordances (individually and collectively) and their meaning-making capabilities3. Intertextuality (the ways images work in relation to each other: reference, difference, similarities)
4. The psychology of images (or in our case moving images) and how material functions as a persuasive mechanism
5. The use and function of persuasive appeals (ethos, pathos, logos)
6. The relationship between text and image
7. Design elements (Kress and Leewuen’s description of layout)
8. The use of rhetorical strategies often used in alphabetic text like metonymy, synecdoche, amplification
9. The use of pictorial and non-pictorial icons, and their role in the meaning-making process

This assessment method calls for a reflection essay, one that will allow students to really engage with the chosen concept, theme or theory in relation to their piece of digital writing. It’s also an opportunity for them to talk about how they executed or were unable to execute the vision they had for their project (due to constraints like using Fair Use, CC, and/or public domain material). So I imagine the assessment criteria will focus on both the video and the reflection, or students may even just want the reflection to be graded.

I’m very interested to hear what other folks in the class came up with. I’m very much looking forward to Thursday (god willing it doesn’t snow for the umpteenth!)

**Chanon Adsanatham. “Integrating Assessment and Instruction: Using Student-Generated Grading Criteria to Evaluate Multimodal Digital Projects.”
***Jody Shipka. “Negotiating Rhetorical, Material, Methodological and Technological Difference: Evaluating Multimodal Designs.” College Composition and Communication 61.1 (2009): 343-366.