I like to make many resources available on my module web pages (pdf files of the annotated slides produced using a tablet PC during lectures, audio recordings of lectures, etc.) Many students make use of these resources, and appreciate them. However, other students appear to do less work than if these resources were not available. Perhaps the existence of these materials makes these students feel too secure? So, while the feedback I receive from the students is overwhelmingly positive, the exam performances are often disappointing.
(See my case study for more details of some of the things I have been trying to do with podcasting and a tablet PC)
Some undergraduates appear to be looking for the most “efficient” way to obtain the highest possible class of degree. They frequently ask me whether I can give them more information than I already do about which proofs might come up in the exam. In fact, I do tell them exactly which proofs are “examinable as bookwork”, but their problem is that there are quite a lot of them!
Now, as I have noted in my earlier posts, if we follow the path of least resistance, we may not be acting in the best interests of the students. However, perhaps we can take advantage of the students’ desire to know what is coming up, so that they can “drill” on it? There may be some aspects of mathematical logic, language and use of quantifiers, for example, where drilling may result in at least some progress. (This is high up my priority list at the moment, because I recently marked a total of over 200 2nd year and 3rd year honours analysis exam scripts!)
With this in mind, I have initiated a project here in Nottingham to produce a variety of online, multiple choice questions involving quantifiers and mathematical language. The idea is, eventually, to have a reasonable variety of question types, each of which has several random elements including the specific numbers which turn up, the names of the variables, and the choice of equivalent mathematical English phrases used. This would produce an essentially unlimited set of questions available online to the students (and, indeed, to anyone in the world) for self-testing on this material.
Now you may well say “this will just be another online resource which most of the students will ignore”, and that might be right, except for the second part of my plan! I suggest that students could be told that there will be a written (but machine-marked), multiple-choice, Class Test, on this material, where the questions will be precisely of the type available on this online resource. Under these circumstances, my theory is that most students will drill themselves using the online resource in order to maximize their mark in the Class Test. From discussions with the Nottingham Learning Team, I can confirm that my theory fits with the observed facts when something similar (but lower level) has been tried in Engineering.
So, is it hard to produce such an online resource involving quantifiers? That may depend on how beautiful you want it to look! My first plan was to write some scripts to automatically generate lots of web pages using, perhaps, MathML. However, I have been granted a teaching-free semester (this semester), to allow me to get some research done. So, showing considerable restraint, I have managed to resist the temptation to dive in and spend a person-week writing these scripts.
What I did do (first) was to use ClassMarker to make a very small set of questions, so that I could ask colleagues for feedback. (Thanks to Matt Heath for pointing out ClassMarker‘s existence to me.) If you want to see my very limited selection, you can log in there with my “Test Student”, username test2947 and password fptbmul
What I did not want to do was to start generating lots of questions by hand: I am convinced that intelligent, random elements are important here. By good fortune, however, we have just appointed a new member of staff, Peter Rowlett, whose PhD included producing online resources with exactly the kind of facilities I was looking for. I had a chat with Peter, and he put together an initial prototype over the weekend. You can see what he has done so far at http://www.maths.nott.ac.uk/personal/pmzpjr/quantifiers/
This is, of course, just the start. So far, just some of the numbers and quantifiers are random, with some simple templates and rules generating appropriate questions from these. Suitably random variable names, and a far greater variety of questions, should (hopefully) eventually follow.
I would envisage (suitably randomized) quite a variety in the language used: for example, as appropriate, we might see “there is/are”, “there is at least one”, “there exist/exists”, etc., as well as the symbolic versions. Some of the questions would ask students to translate between symbols and language. I have also had a request to include “To every 
If you look at the type of question already available from the two resources above, you will begin to see the type of question I currently have in mind. With the exception of the “translation” questions, the student needs to decide whether a statement is true or false (hopefully easy) AND pick an appropriate, concise, reason from a set of alternatives on offer. For the easier, concrete, single-quantifier questions, the student will probably need reassurance that this is not a trick question, and that the truth or otherwise of the given statement is meant to be obvious.
I would very much like to have some feedback on this. In particular, I have the following questions.
- Do you think that this could be a useful resource?
- Do you know of a similar resource that is already available for this material?
- Which variations in mathematical language would you like to see included?
- How should the symbolic versions of the questions be punctuated? (We are currently using commas and not colons.)
- Which other types of question could be dealt with in this way? For example, can we help students to start their proofs correctly using an appropriate “Let” statement?
- Will students simply use pattern recognition to solve these questions, or can sufficient suitable randomization of symbols and language make this “less efficient” for the students than actually understanding what is going on?
- If students DO use pattern recognition to answer these questions, might they still have gained something? Will it, at least, improve the chances that they will use the correct quantifiers when they are asked to write down standard definitions from the notes?
Joel Feinstein 20/3/09
Explaining mathematics 
This seems like a very nice idea! An alternative system, which lots of people seem to like, is STACK: http://stack.bham.ac.uk/. This has a computer algebra backend. However, it seems that for MCQ questions as you are proposing, you don’t really need a computer algebra package; any decent programming language would allow the degree of “randomness” that you propose, so maybe STACK isn’t that useful here. AFAIK Stack only integrates with Moodle (which means I cannot use it here in Leeds). Also, the website linked above really fails to work with the version of Firefox which I’m using…
My thought would be that the more variation in the use of “English mathematical phrases”, the better. I try to do this in my lectures, to give the impression that “For all epsilon … there exists …” isn’t some magic incantation, but just one of many precise ways of saying something. My hope is that students will try to look for the underlying meaning, not just try to memorise one form. I’ve no idea if this works, but it seems you are after the same goal, and are proactively going after it.
My guess is that, with enough variation, students will start to “think” rather than rely on “pattern recognition”. The example questions on Peter’s site seem excellent for this: you’re _not_ asking for the meaning of a mathematical phrase, but rather asking for an example or counter-example, which I would have thought would stimulate thinking in the students.
It’s interesting that you intend to use a paper-based summative assessment. Why not use the computer? Because of fears of plagiarism? Or because of admin problems (giving students enough time in front of a computer?)
Good luck!
LikeLike
Thanks for the comments!
We used some computer-based assessed class tests this year for our first year. Unfortunately, with large numbers of students taking the test simultaneously, under controlled conditions, this caused significant administrative and IT headaches!
If you allow the students to take the assessment in their own time, then plagiarism, or “help from friends”, becomes an issue.
Putting that together leads me to want to use paper-based, computer-marked, multiple-choice questions for these class tests.
Joel
LikeLike
I wouldn’t be particularly concerned about “pattern recognition” responses. But perhaps my major classroom goal is unusually simple: I want students to be able to express their ideas in standard “mathspeak,” and correctly translate/interpret standard “mathspeak” into whatever form they use internally. This type of exercise would help students determine how well they were matching the intended interpretations.
For beginners, order is a major issue. So I start with language that clearly indicates order and is parallel to the language used in a proof. So “Given any epsilon, one can choose N so that n>N implies |s_n -s|0 be given. … [setup N] … Then n>N implies… [estimate] …”
For a transition to mathematical logic, I use, “Given any epsilon, there exists…” Because I like keeping the proof language and definition language parallel, I stay with that style for a long time. The upside-down A gets introduced essentially as an abbreviation. Maybe that’s heresy to some.
Only much later, once I’m fairly convinced that students have internalized the order and appropriate conceptual interpretation, would I introduce a definition with “For each/any/all/every epsilon, …” And those four words each have their own issues, as the mathematical use (interchangeable) has notable conflicts with the connotations of the words in ordinary speech.
Anyway, this topic is extremely challenging for students. Pretty much any help would be useful.
LikeLike