Hacking CrowdMark for 2-Stage Tests: Delightful Efficiency!

I LOVE 2-stage tests.

(Students write a test individually, then join a group to write a second portion of the test.  I use them for multiple choice, but there are lots of ways to do this.)  

Lots of evidence for 2-stage tests here, and an article about them here, plus my own experience of the sound of the room and the general happiness of my students.  (And the fact that in my classes they have consistently voted in favour of 2-stage tests, after having tried them, even when the individual portion of the test was too long).

BUT!  Super Annoying:

The administrative hassle of 2-stage tests: Matching the students with their groups, matching their groups with their results.

They have one scantron sheet per group (which is important for getting buy-in to the group discussion), but you can only bubble one name on the form.  So, matching the group to the results requires manually matching a group identification sheet to the bubble sheets.  In a big (350+ student) class, I estimate that this used to take at least a half-day, just for the matching.  Ugh.

Side question: How do other people handle this matching?  Am I the only person that feels like it takes way too long?

In any case:

Since SFU is currently licensed* for CrowdMark, and I am happily using it for my test and worksheet grading, here’s my one-page semi-automated** solution with that software:

For my 450 student class, we have about 120 groups; this matching takes me about 20 minutes in prep, and half an hour to match the pages in crowdmark.  Hurrah efficiency!

1.  Make a 4-person “assignment” and CrowdMark it to be unique and barcoded.


2.  Using one-sided scantrons, print the group identifier page on the back of the scantron 

Note: need to first split the CrowdMarked pdf (super easy, using Automator, included with mac osx) into even and odd pages, so that I’m only printing the cover pages.

3.  During the test, students bubble the Crowdmark test ID onto the bubble sheet.


For us, I use “Special Code” on our scantrons.  

Also, incidentally, I actually get the students to do this for their individual multiple choice scantron as well, to help match their info if (when) some of them mess up their student numbers.

Side note: On our photocopier, I always do a quick scan-to-pdf of my bubble sheets before running them through the scantron (or giving to document services).  It’s way easier to find a numbered pdf page using the scantron output spreadsheet, than it is to hunt through a stack of papers.

4. A TA or myself uses CrowdMark to match the groups.

This is easy to do because the class list is imported; it takes about 25-45 minutes for the 450 students in 120 groups. 

5.  Match the group scantron spreadsheet with the crowdmark spreadsheet.


This uses the magic that is VLOOKUP(),which is probably my favourite excel formula of all time.  Seriously!  How did I survive before it?  This is the most boring thing to gush about, I know… but still.  I like VLOOKUP even better than Pivot tables.  I like it so much that I actually spell it in all-caps, non-lazily.  I would not be surprised if there were a love-for-VLOOKUP emoji out there.

5.  Go do something more valuable with your time!




Slightly annoying things that I would like CrowdMark to fix:
  1. You track the grading time.  Since I often have a TA doing the matching, can you also track and show me the time it takes to do the matching assessments?
  2. I can’t make a pdf with fewer than the number of students — but on group tests, I only need about 1/4 of the tests.  Easy to hack around this, but it’d be nice to not have to.
  3. I’m only using this to match students, but there’s no way for me to make a “1-page” assignment.  So I have to split the even-numbered pages out before printing, and all the assessments are shown as ‘incomplete.’
  4. Level-up nitpick: I would like the test ID to be unique between both individual and group.  e.g. Individual test IDs go from 1-400, group test IDs go from 401-500 (or whatever number I want to start them at).  Easy to hack around this, but it’d be nice to not have to.


*I’m not in the pocket of CrowdMark at all; I just find that their software solves some problems for me.  If you’re not using CrowdMark, there are other reasonably good solutions to this same problem.  The amazing team of Karen Smith and Craig Kornak from UBC Microbiology have another solution for group matching.  They give each group a piece of paper showing a unique 3-letter group code (the letters include A through E), and the students they bubble the code into questions 1-3 on their scantron.  Pros and cons: You don’t have to get the software for this, but you do have to print the codes out, hand them out, and include an extra group identification sheet as well.  That’s a bunch of extra pages per group, and you still have to do some the manual matching of student IDs with each other.  What do other people do for this?

** It will be fully automated on the happy day, somewhere in the future I hope, when CrowdMark incorporates bubble/OMR tech into their software!


Image at the top is from http://beacharts.ca/scantronart/, which is a hilarious and awesome collection.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s