The answer is that I don't grade students on actually solving a problem, I grade them on how they go trying to solve it.
Some background: once upon a time, during my first year teaching, a student...let's call her 'M'...asked for help. I pulled up a chair and sat down, and M said, "Ok, I'm going to explain what's going on, but I don't want you to say anything. Then I'm going to ask you a question and I want you to just answer that question. Ok?" I agreed, and halfway through her explanation she got excited and said, "Oh, go away! Go away! I got it!"
I always loved that story, because I wanted all of my students to have that attitude, but it took many years for me to notice the pattern that is the real lesson of that story: students would ask me for help, and while explaining the problem they would spontaneously realize what the problem was. It seemed that these novice programmers had everything they needed in their heads, but they didn't have experience organizing it and moving methodically through it. In finding words to explain it they would see what was going on.
And I think there was another effect: we are rewarded year after year through formal schooling, for being able to quickly whip off correct answers to questions. We are taught to believe that it is a sign of intellectual weakness if we have to spend time pondering something. So we blindly rush forward.
But when we have to wrap words around thinking it forces us to slow down and think about what we are doing.
So a few years ago I tried an experiment: instead of asking students to submit their final work, I asked them to keep 'journals' of their progress (in Jupyter notebooks, which I will talk about in another post) in solving a puzzle, and explained that the grade would be based entirely on how thoroughly and clearly they documented their attempts. Instead of just changing their code, they first copied and paste it, then change the copy, and explain what their thinking was.
The first time I tried this they had only been coding for two weeks. I assigned everybody the first Project Euler challenge, "Multiples of 3 and 5." The results amazed me. I haven't done a controlled study, so this is my subjective impression, but the success rate was much higher than I've come to expect from novice coders.
One girl's journal really stood out. She clearly had been telling herself (after only two weeks) that she lost and confused and behind (largely, I expect, because the art of memorization she had been trained in for the previous 10 years was not applicable here), and her journal entries conveyed an overall message of "I have no idea what I'm doing but the teacher said to write about my struggles and I can at least do that."
But over and over again these entries started with her writing, almost despondently, about how it wasn't working, but ended with a glimmer of an idea. Slowly, step-by-step, she figured things out, and fixed bugs, and a couple dozen entries later she solved the challenge.
For the rest of the term, each week each student was asked to submit a journal of their attempts to solve a Project Euler problem. They chose any problem that interested them, with the understanding that their grade would be based entirely on the quality of the journal, and not on whether or not they solved the problem. The grading rubric evolved over the term, but the basic expectations were:
- The bigger the change in the code, the more important that they explain the thinking behind it.
- Presentation, including grammar and spelling, mattered. My logic was that the more thought they were required to put into the writing, the harder they would have to think about it.
- It was ok to ask others for help, and even use the Internet, but they must thoroughly document any help received, and that while they wouldn't be penalized for doing so they also wouldn't get credit for it.
- They got extra points for creativity, illustrations, etc. This was just to try to get them to engage more with the material.
One implication of the rules was that if they chose a problem that they solved too easily and/or quickly, they wouldn't get a very good grade, because there wouldn't be enough to the story of how they solved it.
Not everybody got it right away. Occasionally a student would solve a problem in one or two tries and submit that, but a C or even D quickly cured that. And there were a couple of incidents, over several terms of teaching this particular class, in which students quite obviously Googled the solution. But in their notebooks this appeared as huge conceptual leaps with no explanation (or understanding), and their grades suffered accordingly. That took care of that.
My rationale was that they could cheat on these notebooks, if they wanted to spend the time composing "fake" journal entries, but:
- If they could fake convincing entries they would actually be learning exactly what I wanted them to be learning.
- They would quickly discover that it's more efficient to just write real entries.
