Exercises and feedback

Learning means doing

For skills courses, active learning is a must. That means exercises, and lots of them.

Students need feedback. Grading has two goals. The more familiar goal is summative assessment, which measures the student. If Molly gets a B+ in a writing course, that tells you about Molly. Knowing that she got a B+ doesn’t help Molly learn how to write.

For students to learn from an exercise, they need formative feedback. That is, a list of things they did right and wrong.

Students should have a chance to correct their mistakes on exercises, and resubmit. Why? So they read the feedback, think about their mistakes, and figure out how to correct them. This maximizes the learning students get for their effort on an exercise.

Notice that the feedback has to be individual. Molly gets feedback about her work, Kevin gets feedback about his, Sandra gets feedback about hers, and so on.

So we need:

  • Tasks
  • Individual formative feedback
  • A chance to improve

Computers can’t replace people

Computers can’t grade essays, reports, programs, etc. It takes a human to give formative feedback. If there are 50 students in a course, with 20 exercises per student per semester, that’s 1,000 submissions to be graded.

What usually happens here is that professors give up. It isn’t practical for them to grade 1,000 submissions. They switch to test banks and multiple choice questions. Student learning suffers.

If we take skill learning seriously, we can’t do this. The facts are clear: students need formative feedback from humans. We need to find a way.

Cyco makes it economically feasible to give formative feedback at scale. Cyco does two things:

  • Streamlines the grading process, using “clickable rubrics.”
  • Allows grading work to be distributed to many people.

Let’s see how this works.

Writing exercises

Authors create many exercises, and embed them in their Cycos. They create rubrics for each exercise. A rubric is a standard for assessing exercise solutions. Graders use the rubrics to assess student work.

An example. Here’s an author creating an exercise:

Authoring an exercise

Notice the token `[custom:rival-school]`. Authors create whatever tokens they want. The tokens are replaced by different text in different contexts. For example, one of the University of Michigan’s rivals is Michigan State University. Here’s what a student at the University of Michigan might see:

Student view

Neat, huh?

The author isn’t done yet. S/he must also create a grading rubric for the exercise. It consists of one or more rubric items. Here’s the author creating a rubric item:

Creating a rubric item

The feedback phrases on the right are comments given to students, depending on whether they met the standard, got part way there, or didn’t meet it at all. Authors don’t have to complete comment lists ahead of time. Graders can add new phrases as they go.

Exercises can have as many rubric items as authors want. This one has two:

Rubric item list

Authors can also give model solutions, and grading notes. They’re optional.

Authors don’t type exercises directly into course pages. They create exercises separately, and then embed the exercises in course content. Here’s content from the course the author is working on:

!(bordered)/sites/default/files/bucket/authoring-content.png(Embedding an exercise)

Separating exercises and content lets authors work on them separately. Authors can change exercises, add rubric items, etc., without needing to mess with course content.

Also, a team of authors can divide up the work. One might do content, and another exercises. They can work separately, without stepping on each others’ toes.

Student work

This is what students see:

Course page

Stu is a student in the class. He clicks the `Work on it` link to…, er, work on it. A separate popup windows appears, and the user enters his/her solution:

First attempt

Notice that students can type text, or attach files.

Grading

A grader assesses each submission. Here’s a grader about to start grading the submission:

Grading

The rubric items defined by the author are on the right. Notice the feedback phrases in blue. The text fields let graders add new phrases. Over time, the phrase set will become more complete. This is an example of continuous improvement, important if courses are to last a long time.

The grader clicks two phrases, and the `Create feedback` button. Those three clicks generate a feedback message:

Graded

The text in the lower right was written by Cyco, not the grader. The message has the student’s name (Stu), and the grader’s (Jen). Graders define greetings, signatures, etc., in their own voices, using tokens. Here are some greetings and signatures in different voices:

  • Surfer dude greeting: Yo! [feedback:student_first_name]!
  • Formal greeting: Dear [feedback:student_first_name] [feedback:student_last_name],
  • Hippie signature: Peace out. Pat.
  • Rocker signature: Rock on, y’all! Jiiiiiimmmmyyyyy.

Graders define as many of these as they like. If a grader has four signatures, Cyco will choose one randomly.

Notice that it took only three mouse clicks to generate the personalized formative feedback message. The grader can edit the message at will, adding suggestions, links, whatever s/he wants. However, the basic three-click assessment could be done in less than a minute.

You don’t need a Ph.D. to do the grading. There are potentially millions of people around the world who could do this work.

Trying again

Next time Stu (the student) logs in, he sees:

Feedback available

The student clicks `Review`, and sees:

Review

Stu has work to do. He creates a new version of his submission:

New version

He corrects the first line of the limerick:

New version

The grader looks at it, and gives feedback. This time, the grader decided that Stu had completed the exercise.

Stu sees:

What the student sees

Here’s the message:

Message

Stu can get a report on all of his submissions. It shows:

Report

There’s a completion badge for the exercise. A badge count could be the basis for a leader board.

Summary

To learn skills, students need to do exercises and get feedback. They need a specific list of things they did right and wrong. Every student needs feedback about his/her own work, individually.

They also need a chance to correct their mistakes, and resubmit. This makes them look over their work, see where it’s lacking, and fix it.

This is a lot of grading. However, if we want to take skill learning seriously, there is no choice but to do the work. Cyco has a grading interface designed for formative feedback at scale. Graders can be anywhere in the world. They need not be professors.