[ARCHIVED] Help me understand the differences among criteria, outcomes, and rubrics

[espertus]

My background is in engineering, not education, and I am having trouble figuring out the differences among criteria, outcomes, and rubrics. I've searched the documentation, but everything I've found either describes the mechanics of setting them up or uses too vague of terms for me to understand.

Here's my situation: I want to use mastery learning and have created 6 rubrics for my computer architecture course. Each consists of multiple criteria. For example, the criteria for the Digital Logic rubric are:

1. Converting a problem description into a truth table
2. Finding the minimal sum-of-products formula with a Karnaugh map
3. Converting a formula to a logic diagram
4. Creating a wiring diagram
5. Building and testing a full-adder in hardware

Should each of these criteria be a learning outcome? If so, there would be about 30 for the course, which seems high. What's the difference between a group of outcomes and a rubric?

Since few, if any, assignments test all of the criteria on a rubric, it generally doesn't make sense to attach one to an assignment. Instead, I would like to attach specific criteria (learning outcomes?) to assignments, but that doesn't seem to be an option.

In case my criteria and rubrics are too esoteric, here is an analogous list of criteria for a rubric on building a brick wall in a hypothetical course on building porcine houses:

1. Finding materials suitable for making bricks.
2. Forming bricks.
3. Baking bricks.
4. Making mortar.
5. Laying bricks and mortar.

Since no single assignment tests all of these criteria, should they be groups of learning outcomes instead? Would rubrics be overlapping subsets of learning outcomes (generally within the same group) unique to a specific assignment?

Thank you.

Ellen

[leslie_irvine]

Hi Ellen,

I'm sure others will be able to expand on this, but my short answer is that learning outcomes describe what the students will be able to do or understand after learning the related material. So if your objective is to have students understand the process of making bricks, the outcome (based on your list) would be that they can use what they learned in a real-life situation (e.g., "Students will correctly identify the procedures for making bricks). 

I haven't used the Mastery Learning yet, but maybe someone else can help you with specifics. 

Leslie

[espertus]

Thanks for explaining the terms to me.

[espertus · Accepted solution]

I think I've figured it out, at least relative to Canvas, not necessarily matching the external definitions of terms.

Instead of creating rubrics with criteria directly, I am creating learning outcomes for everything I want to test. To continue the example from my original post, I would create these outcomes:

1. Finding materials suitable for making bricks.
2. Forming bricks.
3. Baking bricks.
4. Making mortar.
5. Laying bricks and mortar.

I would put them into an outcome group Building Brick Houses.

For the brick-making assignment, I create a rubric Making Bricks that contains the first 3 outcomes. I have a mortar-making assignment that contains the fourth outcome and a brick-laying assignment that contains the last outcome.

When students go to the Learning Mastery tab in their gradebook, they can see their progress to each of the outcomes.

[J_Trabue]

At the basic level, an outcome is something students should be able to do at the end of the class (given that "to do" can mean "know" in the sense of "demonstrate knowledge of"). That is, outcomes can be skill or knowledge based. Some fields find it useful to distinguish -- so one outcome might be "correctly define key terms related to the structure of an argument" whereas another is "write an analysis of an argument that makes use of the key terms." 

Outcomes can have different levels of granularity, or detail, depending on your needs as a teacher. Using your example, your outcome might be as simple as "build a brick house." Can students build a house by the end of the class? Great! They succeeded in meeting the outcome. You didn't care if they bought the bricks, made the bricks, found the bricks at a construction site and stole them, or whatever. You didn't care if the bricks they used were of particular quality in and of themselves, or if they were all the same color, or whatever -- just -- is there a house now?  -- if so, they succeeded in meeting the outcome.

Or you can write outcomes at any level of detail you want, so that you can then measure their success with various stages or sub-routines, so to speak. You identified five areas to measure or assess in your example. It really just depends on what data you want to capture for purposes of feedback and assessment. 

However, I must complicate this explanation by saying that sometimes outcome is used functionally -- operationally -- in a different, more restricted sense, to refer only to those outcomes designed and implemented at levels higher than the individual instructor or assignment. For instance, it's common in some environments to use "outcome" only to refer to an outcome that has been assigned to a class or program or assignment by a higher authority -- a program, a dean, an institution, a national accrediting body. 

A rubric is essentially a scale, usually expressed as a grid,  for measuring the degree of success a student achieves in meeting an outcome or set of related outcomes. The rubric can be a simple as three cells : the name of the outcome and a binary assessment: fail / succeed. Or it can be as huge and complicated as you want, say measuring 50 outcomes with 100 degrees of partial success and partial failure. 

The main use of a rubric is to clearly express assessment expectations and standards. 

Criteria is a technical term within the world of rubric design and discussion, and it overlaps to a great degree with "outcome." A criterion is the thing measured. Any outcome measured in a rubric is also a criterion.  If I have an unmeasured outcome, or I'm simply discussing outcomes out of the context of rubrics, I don't refer to it as a criterion.  Conceivably, I could also have a criterion that isn't an outcome, depending on how I operationalize the term (see above).

For instance, I might be tasked with measuring certain outcomes by my program (e.g. "build a brick house" and the five sub-outcomes you listed above), but I, as a teacher, might add "the house should be beautiful" or "the house should be energy efficient" or something like that because I think it's important even though it's not in the list of "outcomes" I was handed by my department.  

In that case, when discussing my rubric, I might refer to the criteria that I was handed by the program as the outcomes and the outcomers I made up for myself and my students as criteria, in a distinction that would make sense in the context but be confusing in the abstract. 

For instance, imagine a metal sphere. If I use it as a ball bearing, I would call it a ball bearing, not a metal sphere, even though it is a metal sphere. If I encountered it out of context, I would probably just refer to it as a metal sphere, even though used in a specific way it would take on that extra label. 

I hope that makes sense.

In my experience, assessing student work is a constant tug-of-war between two equally valid but mutually exclusive goals: granularity, or level of detail, and ease of use, both for the faculty member and the student. If you're thinking hard about assessment, you're thinking hard about the best place to land on that scale, the best way to balance those two goals. 

--JT

[MSearcy9]

Wow, My virtual classes is having a hard time trying to go live. So, thanks for the examples and explanations.