Crowdsource Grading on Digital Assignments

This semester, I’m trying new assessment methods with my graduate students: contract grading and crowdsource grading (for major projects). For me, crowdsource grading is not just about students making decisions about the quality of their peers’ work; it’s also about working together as a class to decide what in fact constitutes quality work. (Or in our case, what constitutes an “A,” “B” and “C” grade on major projects.)

I’ve been collaboratively creating assignment criteria with my digital writers for a couple of years now. My method is informed by the work of **Chanon Adsanatham, who leads students through a number of scaffolded activities that ultimately result in student-generated grading criteria, and ***Jody Shipka, who asks students to turn in a “Statement of Goals and Choices” (SOGC), a statement that documents their rhetorical decisions, along with their digital projects. My method is best described as Adsanatham+Shipka+genre analysis. I ask students to collaboratively conduct a simplified version of a genre analysis (what is successful/effective about this project? What is ineffective/unsuccessful about this project?) as a way to generate assessment criteria. Based on our collaborative work together, I design a rubric, share it with the students, and make any adjustments they feel is necessary. My students also turn in a reflection essay (or what Shipka would call SOGC) along with all digital projects.

For this digital project, I opted not to do the Tanya method in class. Rather, I asked students to make a list of assessment criteria and from these lists, we’ll generate the “final” version together on Thursday in class.

As I’ve been reviewing the articles I assigned this week, I keep coming back to my learning goals and outcomes for this class. One stated outcome is “Students will possess knowledge about how theory and concepts related to digital writing (as studied and taken up in the discipline of Composition/Rhetoric) informs the production of digital writing.” (I should have written analysis and circulation as well!) Over the past four weeks, we’ve read, discussed and pulled apart various theories and concepts. If we thought of how all of these theories and concepts inform the production of digital writing all at once, our heads would explode. If we, as digital writers, had to constantly return to, and consider, everything we’ve learned about say, the function of images, while we’re composing, our heads would explode.

In both of my DW courses, I keep reminding students that writers make choices. Digital writers make choices. We think about how we might best achieve our communicative purpose with regards to our audience. We think about how we can achieve particular effects, how we can convey meaning, how we can utilize modalities to make our work accessible to a wide range of people, for example.

This brings me back to assessment and the question I’d like us to ask ourselves: what theories, concepts or themes might inform the choices we make as digital writers while composing this project. When we discuss assessment criteria for this project, I will offer up one way we can think about assessing this project: focus our attention on one or two concepts, themes or theories, allow them to guide us in composing the project, and use them to determine what constitutes quality work (or in other words assessment criteria). (I think it’s important for us to really think about the fact we are creating moving images not still images. Much of the scholarship we’ve read focuses on still images, and there’s a big difference between the two.)

Below are some possibilities (many of which overlap):

1. Audience involvement, identification, and role in the meaning making process
2. Modality affordances (individually and collectively) and their meaning-making capabilities3. Intertextuality (the ways images work in relation to each other: reference, difference, similarities)
4. The psychology of images (or in our case moving images) and how material functions as a persuasive mechanism
5. The use and function of persuasive appeals (ethos, pathos, logos)
6. The relationship between text and image
7. Design elements (Kress and Leewuen’s description of layout)
8. The use of rhetorical strategies often used in alphabetic text like metonymy, synecdoche, amplification
9. The use of pictorial and non-pictorial icons, and their role in the meaning-making process

This assessment method calls for a reflection essay, one that will allow students to really engage with the chosen concept, theme or theory in relation to their piece of digital writing. It’s also an opportunity for them to talk about how they executed or were unable to execute the vision they had for their project (due to constraints like using Fair Use, CC, and/or public domain material). So I imagine the assessment criteria will focus on both the video and the reflection, or students may even just want the reflection to be graded.

I’m very interested to hear what other folks in the class came up with. I’m very much looking forward to Thursday (god willing it doesn’t snow for the umpteenth!)

**Chanon Adsanatham. “Integrating Assessment and Instruction: Using Student-Generated Grading Criteria to Evaluate Multimodal Digital Projects.”
***Jody Shipka. “Negotiating Rhetorical, Material, Methodological and Technological Difference: Evaluating Multimodal Designs.” College Composition and Communication 61.1 (2009): 343-366.

Digital Writers + Copyright: It’s Complicated

If the relationship between digital writers and copyright law was announced on Facebook, it would be: “it’s complicated.” All of my students this week are likely overwhelmed with the plethora of information they read about copyright, fair use, public domain and creative commons. So many things to remember as digital writers. So many “scary” (at least in the eyes of the law) consequences if they are ignored. So many things we need to stand up for, talk out against, or protect, depending on how you understand and take up arguments around intellectual property rights.

As an educator, I’ve grappled with guiding my students in how to negotiate the complicated terrain of copyright. I incorporate digital writing into all of my writing classes (not just those identified as Digital Writing), so I feel a tremendous sense of responsibility to have these complicated conversations. While designing my graduate Digital Writing class last semester, I was ecstatic to come across old+old+old=new: a copyright manifesto for the digital age, a smart digital text written by smart folks who are dedicated to initiating conversation about current intellectual property laws in the digital world. I found Grace’s reflection on her digital work to be particularly compelling. She describes a project she created in class about Hurricane Katrina. Grace notes that her video is technically protected by Fair Use (as it was indeed created for educational purposes), and then poses these important questions.

At what point am I no longer protected under Fair Use, given the fact that many of the images, songs, and video are copyright protected? If the journal my piece is published in is academic, does that give my piece continued protection when published as an original work? Further, if I do not directly profit monetarily from my piece being published, and the journal I am published in is online free-of-charge, it seems as though no direct profit is to be made by any of the parities. Although I hold true to the power and potential or the fair and open use of multiple media, I still have doubts about how my work is interpreted under the law when published.

This brought me back to my days at Wheaton College, where I taught my first Digital Writing course. Two of my students created a brilliant remix about S.978–a bill proposed in 2012 that called for stricter copyright laws on the Internet. Both students were active participants in YouTube culture, each with their own YouTube channel. They were genuinely concerned about the bill, as, if it was passed, their “channels could be shut down” and they could “even be severely fined.” After they created the remix, they uploaded it to YouTube, clearing indicating that the video was created in a course (and hence fell under “Fair Use”). They included an extensive bibliography of the material used in the video. Several days after it was published, this happened:

The students’ video was no longer made available to the public; their smart, savvy and extremely persuasive argument was no longer available to the people who would be most affected by this bill nor to the people that may have power in getting this bill passed or rejected.

Several months later, this happened:

Yes, you read that correctly. Their remix, complete with all copyrighted material, was published in TheJUMP: The Journal for Undergraduate Multimedia Projects. So to answer Grace’s question, yes, if published in an academic journal, all is well. But guess what, if it’s published on YouTube and is clearly marked as an educational film, no one gets to see your amazing work.

But guess who’s reading the academic journal my students were published in? People like me, people like us, who want to give students every opportunity to create, to invent, to make meaning, and make a difference.  They are people like my students, young folks interested in digital work who can only see their peers’ smart digital work trapped in a wing of the ivory tower (okay…so a tower doesn’t have wings, but you get the point). They are people like the folks who wrote old+old+old=new. Guess who’s likely not reading it? The people who have power to reject bills that may hamper my students’ creativity.

As an educator, I feel a tremendous responsibility to help my students navigate the complicated terrain of copyright. As an educator, I also feel a tremendous responsibility to try to push these conversations into spaces outside of the academy. So the question I pose to myself and to other educators is: how are we going to actively do this? How are we going to talk about the relationship between copyright laws and student learning? Luckily, I get to talk with my graduate students about this Thursday and again during the remix unit.

**After writing this post, I came across student and instructor “unofficial” guidelines for composing multimedia projects. I’m wondering to what extent instructors use this guideline and/or enforce it in the classroom.

Kress Meets Pony and Bear Are Friends

This week in Digital Writing, we’re working with the concept of multimodality, reading some interesting articles that offer us a way to understand, think about, complicate, and use different vessels of communication to make meaning. While reading Kress’ “Gains and losses: New forms of texts, knowledge, and learning,” I was particularly struck by two passages that relate to my current work on digital reading practices:

Passage 1: “Because words rely on convention and on conventional acceptance, words are always general, and therefore, vague. Words being nearly empty of meaning need filling with the hearer and/or reader’s meaning…we treat that as the act of interpretation. With depiction and with images the situation is different: that which I wish to depict I can depict, at the moment of any rate. I can draw whatever I like whenever I like to draw it. Unlike words, depictions are full of meaning; they are always specific. So on the one hand there is a finite stock of words–vague, general, nearly empty of meaning; on the other hand there is an infinitely large potential of depictions–precise, specific, and full of meaning…this leads to the paradox of speech and writing as having a finite number of open, relatively vague elements in fixed order, and image or depiction having a possibility of infinitely many full, specific elements in an open order….speech and writing tell the world; depictions shows the world.”

Passage 2: “Reading as taking meaning and making meaning from many sources of information, from many different sign-systems, will become the new common sense….another crucial change in the meanings of reading…(is) what I have called “reading as design” (Kress, 2004). In relation to message entities (texts?) such as that, the reader finds her or his way around the matter presented on that page, and orders it according to principles, as I suggested, that arise from the reader’s life-world…the reader designs a coherent complex sign that corresponds to the needs that she or he has.”

The discussion of writing and images, of reading and what it means to “read,” made me think of a book a friend gave my baby, The Pony and Bear Are Friends by Sigrid Heuck. Ever since I got this book, I’ve been intrigued by it. I don’t remember seeing anything like it before, though maybe I did as a child and don’t remember.

The book is fascinating. The traditional writing is attached to its traditional counterpart medium, the book. Like a book, we have one pathway in, we’re directed to read from left to right, from line to line. Yet unlike many books that use accompanying images, these images are part of the narrative, woven among and between the words; words and images working harmoniously to create a narrative.

I decided to do a little experiment. I was interested to hear how both a child and adult read this “children’s” book. How did they engage with a multimodal text that takes this form? How did they “read” the words? How did they “read” the images? When I think about Kress’ article, I wondered: what is the difference between how one engages with the words and the images, and how they engage with them both together? Is the reader taking meaning or making meaning? Is there a difference between how children (who likely first learned to read via pictures) and adults (many of who equate text with words, and writing with the alphabet)?” I asked my 8-year old neighbor and my 35 year-old husband to read the text aloud and respond to a couple of questions.

Here is the audio from the 8 year-old:

Unfortunately I deleted the audio recording of my husband’s session by mistake (damn voice memo on iphone), but I do remember his responses to my questions:

Me: What was your experience like reading this book?
Him: It was entertaining.

Me: Does this look like texts you normally read or does it look different?
Him: It looks different because it has pictures interspersed with text.

Me: Was it difficult for you to engage with the images or did it come pretty natural?
Him: Some of the pictures were difficult to ascertain.

There’s a lot of things to pull apart here, but one thing I find particularly interesting is my neighbor’s comment about school texts (alphabetic) and this text (alphabetic+pictures). She says this book is easier to read than her books at school, but toward the end of the conversation, said her school books are easier to read because she’s now used to them. My husband talked about the difficulty of engaging with some of the images, making meaning with some of them in relation to the words, despite the fact he engages with images all the time albeit on the screen though not in the same form/genre. I also found my neighbor’s comment about how she read as a little girl interesting, saying her and her mother took turns “reading” the words and “reading.”

Kress claims we fill words with meaning while images are already full of meaning. He talks about writing “having a finite number of open, relatively vague elements in fixed order,” and image or depiction having an “open order.” But what happens when an image is situated in genre that calls for readers to engage with it like they do words, read it from left to right, in a linear, coherent way?  Is the notion of “reading as taking meaning and making meaning” really a new way to think about reading, or is the way we as children understand reading (and then redefine it once we enter an alphabetic saturated (or so we think re: texts, anyway) world)? What is gained and what is lost in using words and images in this way?  Thinking about this book in relation to notions of multimodality, the function of images and words, and Kress’ article makes my brain ache. I’m needing some more distance from this experiment to gain some clarity and make some stronger connections.  I look forward to hearing what other folks think.

Welcome to the Parlor

At the end of teaching my first graduate course, I remember asking the students an inevitability ego-shattering question: “how do you think I could improve this course?” (aka “in what ways did my teaching suck this semester?”) My very gracious students gave some interesting (and only a wee bit ego-bruising) feedback. One student’s response stands out: “I think you should assign one or two books over the semester rather than a bunch of articles a week. The articles are hard to connect. I never really knew what you wanted us to do with them.” My response, without a second of contemplation, was something along the lines of: “I’m open to changing a lot of things about my pedagogy, but I’m not open to changing that.” After I finished the sentence, I was actually surprised by my impassioned reaction. After all, I knew exactly how she felt. A lot of my previous professors did the same thing, which is likely exactly why I do it as well. One of my favorite profs from my doctoral program was notorious for assigning seeming reading randomness. Her readings weren’t just a couple of articles, they were a couple of full articles, some parts of an article, and sometimes even a couple of paragraphs from an article. On several occasions, I remember her emailing us the night before class and telling us to read yet another article or another excerpt or another definition. At the time, I found it (not her, I loved her) extremely annoying. It wasn’t until later that I figured out what she was doing: she was mapping out a conversation in the field, providing us with voices and ways we could understand a topic of inquiry.

When I sat down to write this post, I had absolutely no intention of talking about readings in graduate courses or curricular decisions. But I felt compelled to do so after asking my students to read a long chapter from a book and 10 or so blog posts about digital rhetoric. I envisioned my students sitting in front of their computers or maybe in the library saying to themselves: “So much reading. So many different perspectives. So little context. Is she insane?” My decisions probably didn’t do me any favors in making a good first impression. But yes, there was a bunch of reading this week, likely the heaviest load we’ll have all semester. But I think it’s good. I think it’s great. I think it’s worth it.  Why? Because this, my friends, is what rhetorician Kenneth Burke calls the parlor. So welcome to the parlor, welcome to a space where conversations are held, perspectives are given, and knowledge is built, enhanced, and complicated.  Listening to these voices gives you a glimpse into what people are talking about when they talk about digital writing.

With that said, I’m thinking forward to class Thursday night. I’m hoping to engage in some good discussion about these readings. But man, we could talk about so much. These scholars offer up so many different lines of inquiry we could engage with, but which ones should we take up?

I set out to use this post as a way to provide some kind of discussion map for us, some things we may consider talking about. When I had this idea, a synthesis activity I do with my undergraduates immediately came to mind. So I decided to do the assignment, identify some discussion themes, and do some connection-making among and between the voices I hear in this parlor.  I initially planned to include some of the activity in this post, but it grew out of control and is definitely not suitable for a blog post. So you can journey here to check it out.

Oh PS, Daniel Anderson is amazing. If you haven’t checked out his work, you must.

PSS, the picture of my son above has absolutely nothing to do with this post, but I had to include it for a couple of reason: today was his first time in the snow; he looks kind of like Ralphie’s brother from A Christmas Story, and he’s too damn cute not to share with the world.