Understanding is not cued knowledge: performance is never the sum of drills; problems are not exercises; mastery is not achieved by the unthinking application of algorithms. – Wiggins, 1993
Now when I say We, I really mean Curtin University. I’m going to assume other universities teach in a similar way. To start off, let us have a look at the break down of some second and third year Computer Science (did I say Computer Science? Surely I mean Bachelor of Science (Science) majoring in Computing!) units at Curtin.
In this table, Tests and Exams are things where you have to sit for a certain amount of time with nothing but pen and paper and complete questions. Programming and Written Assessments include things like projects, programming tasks and reports, usually these things are mixed together as an ‘Assignment’.
| Unit | No. Tests | % Tests | No. Prog/Writ | % Prog/Writ |
|---|---|---|---|---|
| DAA300 | 3 | 90% | 1 | 10% |
| AMI300 | 3 | 85% | 1 | 15% |
| OS200 | 2 | 85% | 1 | 15% |
| SE200 | 3 | 80% | 1 | 20% |
| CG200 | 2 | 65% | 1 | 35% |
| FCC220 | 1 | 50% | 3 | 50% |
Now if I was asked to rate these units on a combination of how effective I learned the unit outcomes, overall enjoyment and how much I remember the concepts to this day I would probably produce this list.
Now I know correlation doesn’t mean causation, so I’ll state it clearly.
I hate tests.
There are so many reasons why tests suck in computing that I just have to make a list.
1. It Is a Bad Test of Understanding
When designing computer programs, we usually judge the requirements on inputs and outputs. So if I input X into your program or function, it should output Y. We write test cases around this principal in order to see if the program does what it is supposed to. Unfortunately, people are not programs. When given a programming assignment, if I get given the test case, I can easily produce a program that outputs the solution in one line after calculating the solution manually and telling the program to print the solution. Unfortunately I would get zero marks after my tutor inspected the code.
So if simple input/outputs isn’t enough to test a student’s code, why should it be enough to test their knowledge of the concept. A graduate programmer will seldom be far from a computer able to produce any of the information that has been rote learned at university, so why do we focus so much on this technique?
2. It Does Not Help Students Learn
I can now hear the voices of all my lecturers at once:
“Tests aren’t for learning Jason! They’re for assessing!”
They sure are, Unnamed Lecturer. That’s why every unit ever has somewhere around half of the assessment marks as a formal, summative exam at the end of the semester, a time when everyone stays up late memorising every useless fact in their book in the hopes that they will recall an obscure fact which gives them a higher mark than their peers. Maybe, just maybe this will get them a higher grade so that Company Inc. will think they will be a better employee. This is so disconnected from reality I cannot fathom the reasons why we would want more of this type of assessment.
When studying for a test, what do students do? Normally, we would go through lecture slides, tutorials and past tests/examinations to see what topics are generally tested. We then repeatedly do practice questions (in the case of algorithms or calculations) or try to commit things to recall memory by writing notes and repetition. This is a good strategy to pass tests but this is not a good way to learn.
To contrast, a well structured assignment task will encourage students to go and research concepts and implement them (in the case of programming tasks) or write about them (in the case of reports/essays). While you can easily learn via rote in order to pass tests, implementing an algorithm requires a deeper understanding. The marker can then confirm this understanding by reading your implementation and comments around it. There is also a bonus, programming and to a smaller degree essay writing, has the rote learning of concepts built in. With both programming and writing, you inherently read and reread concepts over and over as you debug and edit your work.
3. There Is No Portfolio
When you complete a programming task, your result automatically goes into a portfolio of code you can publish as an example of your skills when joining the workforce. This is a bonus for people who are vocation oriented (read: at university just to get a job). While high marks always look attractive on a resume, employers will also be looking for experience in certain languages and fields of development. Saying “I did good in a test” is much less impressive than “I implemented an encryption protocol in C in two weeks”.
4. There Are No Goals
Goals are important. There are entire industries based around teaching people to set goals and achieve them. The current buzz around education and marketing (both which are weirdly linked) is gamification: the use of things like scores, points and achievements to help motivate people. Gamification, when used correctly, can increase user engagement and retention. A programming assessment, probably unknowingly, borrows many concepts from gamafication. Usually they will consist of a few different features worth a certain amount of marks (or points, if you see where I’m headed here). When a student completes a feature, they can test it using test input and once they get the correct output, they have “achieved” that feature.
The difference in these programming assessments which makes them almost automatically gamified compared to assignments in other degrees is the ability for instant gratification. A programmer knows when they have reached a milestone, because they can test each part compartmentally. Many Computer Science students tend to enjoy playing video games, so much so that a large chunk of the events our local CS club run are LAN’s. I don’t think this is a coincidence. Our teachers should acknowledge this connection and start designing assessment tasks with it in mind.
You Just Need to Study Harder, You’re an Adult! Formulate Your Own Learning Goals!
This is the response I expect to get from educators reading this. This is what I would expect they would think to themselves when they read my unit feedback (almost every one has been “Less tests, more assignments”). I reject this assumption completely.
University is not an accreditation factory, we don’t attend for the tests, we attend to learn. In this day and age of Khan Academy, Code Academy, Coursera and with all the fancy university offering free MOOCs (Massively Open Online Courses), the difference between learning online and attending a university has narrowed. Students expect that the university will not only make learning content available and test their summative knowledge in a 60 minute session a few times per semester. Students expect that their learning institution will assist them in setting achievable goals which are formative across the unit.
A good way to do that is more practical, programming assessments with theoretical reports/analysis. Giving students feedback through both automatic methods such as providing sample test input/output as well as feedback from instructors which goes beyond “this did not fit the marking guide, this was the answer” will improve student engagement, retention and results.
Lastly, the one issue with encouraging more practical assignments is the issue of cheating and ghost writing. This is the one thing exams are good for, and why they will always have their place at the end of a unit as the summative assessment for the entire semester.
Not going to lie; I disagree with a lot of the things you say. But I think you’re right on with this one. <3
I feel like your argument is fundamentally flawed. If you progressively learn the concepts over the course of the unit, then tests will be no problem. Of course, you would revise before a test – but the idea of slaving over the lecture slides/past exams/tutorials to learn things last minute means you have tackled learning incorrectly. Tests don’t support this at all. It’s the students that try to coast and cram that blame tests.
In addition, building a portfolio out of assignments gives you little credit. Why would a professional want to see a portfolio comprised of work that is hardly of your own making? Assignments are designed with clear goals in mind, limited scope and require little creativity.As students, you fill in the blanks of what the assignment specs to produce the end result.
Unfortunately, student feedback on units is also heavily skewed. Only students that really care (whether positive or negative) about the units tend to respond. You miss to much of the total student base to produce a meaning result.
This is true and applies to all academic study not just computing, but how are you progressively learning these concepts? You could read the text book, summarize it in notes, do a few practice quiz questions and revisit it every few weeks. This is a decent way to memorize anything. But have you actually learned it? Would you be able to effectively apply that information outside of the sandbox that is a test specifically designed to check your memory on the unit content? I would say no. Computing isn’t best learned like that – people study computing because they like to interact with algorithms, not read a book.
You could decide to go and implement every algorithm you come across during the unit, or at least the interesting ones. Then the economy of time comes into play – while you’ll get a deeper understanding, it takes a lot longer and it isn’t as easily compartmentalized the same way the first strategy is. It just doesn’t make sense time management wise for a student to spend a big chunk of their study time coding if it isn’t required for the unit.
Then you really have to ask yourself, are you at university to pass tests? I like to think no.
I have to completely disagree on this one. Many prospective employers will ask for your GitHub account to have a look at the way you code. Believe it or not, there are still people despite all these tests who graduate without being able to code. I’d also argue that even though assignment are generally limited in scope, they require more creativity than your average test.
It’s very easy to say that it’s the student’s fault, but with the extremely high attrition rate of computing students compared to other degrees, I don’t really think it’s fair to rest the blame solely on the student’s shoulders.