RT01- Closed Book Exams Could Even Kill George the Jungle!

"If we want real change in the students' learning mindset, just rethink and transform our assessment model(s) to become more authentic and relevant... It is that simple! Changing the rest (e.g. curriculum) is easy, except changing the mindset of a seasoned lecturer stuck in the 20th century factory paradigm. However, the impossible is possible, right?" - Zaid Ali Alsagoff

RT-SERIES

This article is the first of a series exploring my thoughts and experiences on what I think is wrong with Higher Education in Malaysia (and beyond), and how we can actually transform it to be more relevant without too much effort. 'RT' refers to Rethink and Transform.

As for how many episodes this RT-Series will include, I have no idea at this moment (fuzzily exploring and sharing). Let's just see how it goes as we explore different aspects of the current education systems/methods/approaches/etc. commonly being applied in Higher education. Also, I have for sure not reached that intellectual level that I am stuck with my opinion and ideas (not moulded yet!), so please challenge me on every aspect if you can (find time!). By the time you challenge my reflections, I might have already changed my mind. In other words, these posts are just snapshots of what I was thinking at the time of writing, and my ego will hopefully not allow me to be stuck in one direction, if my intellect has been convinced otherwise.

Importantly, I am not going to refer to particular Universities, but instead focus on what I have experienced, observed, read, discussed, learned, and reflected over the last 15 years. Let's face it, you will find great lecturers and courses in all Universities, but you will also find crap in every University (including Harvard!). The best Universities have the least crap (if that is a measure of excellence)! What I mean by crap, well that will be explored in the RT-series in the coming weeks and months, as I reflect out loud what I am honestly thinking.

As for now...Lets cut the babble crap, and explore assessment and specifically closed book exams, which I believe is that ultimate weapons of mass destruction to inspiring, innovative and lifelong learning.



WHY CLOSED BOOK EXAMS?
But before I start slamming closed book exams, let's reflect why we still keep on using them, even though many of us know that they have some potentially serious side effects to authentic learning and constructive learning habits.

Here are a few pointers for why using closed book exams makes sense:

• Administratively convenient and efficient
  
  You can stuff hundreds (even thousands) into examination halls at the same time (if you have the facilities) and get it over within a short period (e.g. 2-hour exam). If you are using multiple choice questions (closed-ended questions), better yet you don't even have to correct them, as the system will basically do everything for you (including item analysis), whether online or using scanners. What other assessment method can be done as efficiently as exams? Usually, at least 60% (mid-term = 20%, Final exam = 40%) of the total course marks are assigned for closed book exams, which is still widely practiced for Diploma, Bachelor and Master programme courses. In terms of efficient assessment administration, exams are the Holy Grail for assessing students (but not necessarily the most effective method!).
  
  
• Final Result (Summative)
  We want to know at the end of the course, whether students have fulfilled the learning outcomes set (according to Bloom's taxonomy) and then reward them with a grade based on what they answer. This makes sense, if final exams really measure what they are supposed to.
  
  
• Coverage
  You can cover the essential components of the course curriculum with (mid-term or final) exams, and especially if you are using multiple choice exams, you can basically cover every chapter in the book. Now, that is awesome! Isn't it?
  
  
• Reliable and Objective
  Exams results are often predictable and you can be quite objective, especially if you are using closed-ended questions. A-scoring students will mostly get 'A', B-scoring students will mostly get 'B', and F-scoring students will mostly get 'F'. Now that is what I call consistency or perhaps a self-fulfilling prophecy... Not sure! But exams are certainly a good indicator whether the students are hard-working and academically inclined, just like IQ tests measure your intelligence. Wait a second! Could both be reliable, objective but not really valid and relevant. What do you think?
  
  
• Knowing
  If you are doing an open-book exam, we don't really know that you know. But if you are only using a pen(cil), eraser (sometimes calculator) and paper, you are on your own, and what you answer is what you know. In short, you can replicate what the lecturer or learning outcome wants without any additional tools. Isn't that great, you can vomit out what you have learned exactly the same way it was learned (or memorized), and then if the lecturer is kind you might score an 'A' by just memorizing the course PowerPoint slides (Been there, done that!). Whether you understand the concept or idea is secondary, as long as you can write out the answer. What matters is what you wrote in the exams, not what you actually understand, right? If both are achieved then great, but if not, you are still going to get an A, because the exam script is evidence enough to prove that you know your stuff. What more do you need to prove?
  
  
• Yes for Quantitative Subjects
  
  For subjects that are infused with a lot of mathematical and scientific formulas, it makes sense to use closed book exams. Using pen(cil) and paper (and scientific calculator) is an authentic way (or close to) for assessing such type of learning areas. Don't you think so?

Alright, I could point out a few more reasons why closed book exams are still being used, but let's now instead focus on why exams or more specifically why closed book exams could kill...Alright, let's leave George the Jungle out of this discussion...

WHY NOT?

If you were asked to retake all the (mid-term and final) exams you took during your University days, would you be able to do as well as you did?

Looking back at my Bachelor (Psychology) and Masters (IT Management) programmes (mostly mid-term = 20%, Final = 40%), I can safely say that I would probably fail 80-90% of those exams, if I had taken them today. Forget about scoring 'A' and 'B' (and some 'C'), I would literally fail most of the exams! So, what did those closed book exams actually measure?

"Education is what remains after one has forgotten everything he learned in school." - Albert Einstein

Until today, I am still clueless to that question. Though, we should not necessarily blame the exam format itself, because it is the lecturers who set the questions for the exam, right? what do you think?

The best answer I can think of is SNAPSHOT. Most closed book exams that I have experienced measure what I have memorized or understood before the exam (temporary memory), but does not necessarily measure what I know (long-term memory) or can do now. So, what is the point of allocating say 60% of the course evaluation on something so shallow and trivial (snapshot learning). Is it because education is a business, and it makes sense from that point-of-view, or is it perhaps because we simply don't know better? Please, enlighten me!

Universities around the world do tons of research related on teaching and learning, but how much of that output has been used to transform education. Educational science as a tool for transforming education is probably as influential as Bart Simpson's mood swings. So, what to do?

Let's put educational science aside, and let's use our common sense and experiences to reflect why closed book exams should be history for most courses or subjects. Perhaps stimulating the logical and affectionate mind (left and right brain!) can transform change, or perhaps our old ways are so ingrained in our ego, it is simply impossible to change. Impossible is possible!

Here are three (3) more reasons for why closed book exams should be benched (substituted):

• Cramming
  If 60% of your grade is allocated for exams (40% for coursework), I can guarantee you that many students will schedule their learning to cramming before exams, and produce great snapshot learning experiences. Although, I am a great fan of PowerPoint for teaching, I am not sure it is the ultimate tool for learning (no matter how detailed your slides are!). I have seen (and even practiced for boring subjects) students depend on the course PowerPoint slides for everything, and worse yet, managed to score 'A' by simply memorizing the slides (without understanding). Some students can actually graduate from a Bachelor programme without reading a single course related book. Isn't that cool? Yes, until the graduated student goes for his first job interview, and realize he just wasted 4-5 years of his life learning Nothing (Stephen Hawking, we were created from Nothing and the Laws of Science...Yeah, why not use your right brain for crying out loud!). Although, we should not blame the exam format, because it is the lecturers who set the questions. However, closed book exams seem to often encourage consciously and unconsciously cramming and gear students towards scoring rather than learning. Who cares what you learn, as long as you get 'A', right?
  
• Closed Book Exam = Authentic Learning?
  If you are asked to deal with a problem, find an answer, make a decision, or perhaps solve a mystery, don't you use whatever tool legally (hopefully!) possible to master it (picture above)? When you are working, does your boss give you a work related challenge, and then asks you to use no tools except a pen(cil) and paper? Of course not (at least most of the times)!
  
  But, why do we conduct closed book exams, if it is totally against common sense to finding an answer. Worse yet, closed book exams subconsciously teach us to not look beyond our crammed brain of knowledge to find an answer. I always find it amusing when newly hired staff gets stress and close-to-nervous breakdowns, when their boss give them a challenge beyond what they have learned, without a book or PowerPoint slide to save their day.
  
  In the 21st century, we have Zorro-bytes to store information, and the Wolfram|Alpha, Google Squared or Answer.com (etc.) to memorize and summarize, so that we can instead focus on applying our knowledge and skills to solve problems, make better decisions and be more creative and innovative about how we do things.
  
  I know it is tougher to set questions for open-book exams (or any tool exam!), but at least we will be encouraging healthy learning habits from day one, so that students' habitize to think beyond and use whatever tool appropriate to answer the question. That is how it works in the real world, so let's simulate that! Why not?
  
• Measuring the Right Stuff?
  It depends, but mostly from my experience with (Bachelors and Masters) closed book exams is terribly frustrating experiences. Obviously! But, what annoys me more is that closed book exams using especially just pen(cil) and paper hardly measure what they are supposed to. For example, I am doing a course exploring Business Leadership, and then 60% of my grade is based on what I write and tick in two (2) exams. It is obviously administratively convenient to measure that way, but shouldn't most of my grade be based on how I do in learning activities that are closer to the real thing (authentic), rather than ticks and a few paragraphs of synthesized memory? Yes, I am a great business leader if I can score A in that subject.

SUBSTITUTE?
So, what should closed book exams be substituted with to ensure more authentic and relevant learning?

Today, we have amazing possibilities to facilitate learning and assessment environments that sizzle, and I will explore some of these in RT02... Coming soon (after a couple of pending articles). But, what is important for now, is to realize that closed book exams should be benched for good (for most courses), and that we need to explore other assessment methods to transform the way we learn and assess. If we continue to stick with our old ways of assessing, we will soon be irrelevant, including the necessity to have a degree to get a decent job. Have you noticed that organizations and companies are increasingly skeptical to graduates' knowledge and skills (specialization), communication/language/leadership/teamwork skills, etc. I seriously don't blame them.

Well, that is what I think! But, what about you, what do you think? Let's together explore the future of learning and assessment. I can change, you can change, we can change! Impossible is possible, so let's think that we can do it. If others can't, we can :)