Technology roundup: Scantron alternatives

Following up on this tweet!

For the past couple years, I’ve been happily using REMARK Office software to make customized scannable OMR bubble sheets (“Optical Mark Recognition”).  They’re awesome as a Scantron replacement, because you can print and scan using your department photocopier – you don’t need to buy a specialized machine or specialized forms ($$$).  Additionally, and even more fun, you can make your own worksheets and lots of other forms.  We’ve found that they’re great for reducing the administrative load on worksheet grading (for participation or otherwise) and exam grade inputting.

Some examples – 1 exam grading page & 2 worksheets:

(I know Jared Stang and Lisa McDonnell have many more creative uses of their own bubble-form worksheets!  And I’m pretty sure that I got the 2nd DNA question from Robin Young!)

I was able to use the software because it was purchased by the CWSEI (thanks Warren!) Having moved on from my postdoc there, I’m on the hunt for an alternative.  Even though I was satisfied with REMARK, I wasn’t a big fan of the price so I decided to look at the software competition.  I ended up finding several “Scantron Alternatives” options:

Software Website Cost Runs on…
Akindi https://akindi.com/ Free? Online server
Auto-Multiple-Choice http://home.gna.org/auto-qcm/ Free! Mac/Linux
FormReturn https://www.formreturn.com/ $699 USD for one license Mac/PC/Linux
FormScanner http://www.formscanner.org/ Free! Mac/PC
InspiroScan http://www.inspiroscan.com/ $1500 USD for a year, plus $399 per year thereafter, plus they design forms for $299 per form. PC
REMARK Office http://remarksoftware.com/ $1195 USD PC

For me to decide which I wanted, I made a priority list and ran them through it.

 

My priorities/criteria:
  1. Customizable: Software that allows me to make customizable, photocopier-scannable bubble sheets, including incorporation of diagrams/figures.  (Need)
  2. Software installs/works fine: I have to be able to get up and running, or at least a trial version (Need… obviously)
  3. Licenses: Ability to install on multiple computers – for TAs to run, and for myself to use at home or at work (Want)
  4. Mac compatibility: I’ve got Mac computers, so ideally I’d like to install on my own computer to use in my office.  But, if there is excellent windows-only software, I could possibly get it installed elsewhere in the department, or run windows on my mac.  (Want)
  5. Integrated real-time error-checking: where you can see the spreadsheet and the image of filled-in bubbles at the same time, without having to go looking. (Luxury)
  6. Cost is free!  (Luxury)
  7. LMS integration: So that we can return work to students without them going through piles of papers.  (Currently a Dreaming-in-Technicolour Luxury; and it requires extensive LMS support so I didn’t even bother putting it in the table!)
Here’s how the different software packages rated:
My priorities:

Need:

Customizable form creation

Need:

Software install worked

Want:

Multiple licenses

Want:

Mac compatible

Luxury: Real-time error-checking

Luxury:

Is it free?

Akindi

No*

Yes Yes Yes Not sure No

Auto-Multiple-Choice

Yes No Yes Yes No? Yes
FormReturn

Yes

Yes No Yes No

No

FormScanner

Yes

Yes Yes Yes No

Yes

InspiroScan

No** No*** No No Not sure

No

REMARK Office

Yes Yes No No Yes

No

* Their website says to contact them about customizability.

** Unless it’s a “survey” (not quiz) which you design, or if you pay $299 per form, for them to design it…

***  Free demo avalable, but PC-only

That’s my short rundown.  For more verbosity…

Having tried to run all of these, I made some further notes about my experiences installing, making worksheets, and grading them.

(PS – has anyone had better luck than me with table formatting in wordpress.com?)

Software Notes from installation and test run Learning Curve — time to finish install and test Pros Cons
Akindi Doesn’t let you customize forms, so it didn’t meet my needs. But it looks excellent & very easy to use for regular multiple choice. A few minutes, but this doesn’t include any form design.
  • Very nice interface, for scantron-only type forms — they have nice templates.
  • No software to download; everything run through online server.
  • Not clear how we could make custom forms.
  • Online software, therefore data stored elsewhere – on US servers? (how are schools like Ryerson using this?  Waivers?)
Auto-Multiple-Choice Installation didn’t work so I gave up in frustration Impenetrable (Spent >1 futile hour just trying to install – it didn’t even get working by this point; plus even if I had, the lack of GUI and possible use of LaTeX has its own steep learning curve.)
  • Impossible to install without better-than-novice terminal/linux skills… so even if I did figure it out, TA training and installation on their computers could be a barrier.
FormReturn Not totally intuitive, but tutorial videos were helpful. Hit a snag with the test set of forms filled out by “students” – some forms were rejected due to how the bubbles were filled out. 1.5 hours
  • Forms designed within the software, which is good if and only if you have access to it at your own computer.
  • Prompt tech support replies.
  • Software recognized marks from pen, pencil, marker.
  • Form design includes lots of steps required for printing unique info on each page, so if you’re just photocopying worksheets, your form creation is a little cumbersome.
  • Some errors (e.g. scrawling across two bubbles) cause form not to be read at all, and checking this is annoyingly not integrated.
FormScanner Installation was pretty easy. I had a small hiccup with Java, which was easily solved by consulting their online forum. About 45 minutes
  • Free
  • Very straightforward to set up forms.
  • Clearly designed for just this purpose, so there aren’t a lot of extraneous/annoying options to have to select.
  • Good and current discussion forum for questions/issues.
  • Output parsing and error-checking using their excel macros works well for the default form provided, but haven’t tried yet for non-default forms.
  • No real-time error-checking in the software (yet), but their output setup in MS Excel means I could customize reasonably well for what I want.
  • Minimal error-checking is supported, but not integrated between images and spreadsheet: it outputs a csv, and an excel file with macros is provided for parsing the data.
  • Software had trouble reading light pen marks, but solid pen, pencil, marker were okay.
InspiroScan Couldn’t do it (runs on windows only) Don’t know (they have a demo version but it’s PC-only)
  • From watching their video: Error checking interface looks good.
  • And, according to them, one advantage is: “InspiroScan uses USB!” (?!?)
  • Cost
  • Windows-only.
  • Can only self-design “survey” forms for free, not test forms.
REMARK Office Ran smoothly, with just a few hiccups while I was learning the software. (Disclaimer – I’ve used this software for the past year, so I can’t fully recall the install/testrun) About an hour
  • Error checking interface is extremely strong; it runs well, and and connects the spreadsheet with the scanned images in real time, in one interface.
  • Can build forms right in MS word, with a nice bubble font that they provide.
  • Good tech support and they will (promptly) review your own forms for you.
  • Windows-only.
  • Further notes from having used it for a year:
    • Sometimes the bubbles font is a little finicky, especially if doing mac/pc conversions.
    • Sometimes, inexplicably, it has a hard time recognizing a form, especially if they have been scanned in large batches.
    • On our computer, this software freeze/crashed more often than I would have liked (though this certainly could’ve been our old mac laptop that was running windows).

REMARK’s software is really strong, and its error-checking is by far the best.  However, for me, the lack of Mac compatibility, and the inability to share my license with my TAs or my home computer, makes it too much of a hassle.  None of the others could really compete with the (free!) software FormScanner.

Based on this, I think I’m going to try out FormScanner in the summer term – looking forward to it!  And, I’m really happy that it comes out of a physics education group from Nicholls State University (along with a Java developer) – they’ve even published a paper about their work.

 

 

I’ll keep you posted for how it ends up working.  Has anyone else tried any of these packages, and would like to chime in?

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s