Metacognition for Scholars: How to Engage in Deep Work

By Charity S. Peak, Ph.D. (Independent Consultant)

True confession: I’m addicted to shallow work. I wouldn’t say I’m a procrastinator as much as I am someone who prefers checking small things off my list or clearing my inbox over engaging in more complex tasks. I know I should be writing and researching. It’s just as much of my job as teaching or administrative duties, but I get to the end of my day and wonder why I didn’t have time for the most critical component of my promotion package – scholarship.

It turns out I’m not the only one suffering from this condition (far from it), and luckily there is a treatment plan available. It begins with metacognition about how one is spending time during the day, self-monitoring conditions that are most distracting or fruitful for productivity, and self-regulating behaviors in order to ritualize more constructive habits. Several authors offer suggestions for how to be more prolific (Goodson, 2013; Silvia, 2007), especially those providing writing prompts and 15-minute exercises, but few get to the core of the metacognitive process like Cal Newport’s (2016) recent Deep Work: Rules for Focused Success in a Distracted World. Newport, a professor of computer science at Georgetown and author of 5 books and a blog on college success, shares his strategies for becoming a prolific writer while balancing other faculty duties.

Newport claims that deep work is the ability to focus without distraction on a cognitively demanding task. It is arguably the most difficult and crucial capability of the 21^st century. Creative thinking is becoming progressively rare in our distracted world, so those who can rise above shallow work are guaranteed to demonstrate value to their employers, especially colleges and universities. In order to be creative and produce new ideas, scholars must engage in deep work regularly and for significant periods of time. Instead, Newport argues that most people spend their days multitasking through a mire of shallow work like email, which is noncognitively demanding and offers little benefit to academia, let alone an individual’s promotion. In fact, he cites that “a 2012 McKinsey study found that the average knowledge worker now spends more than 60 percent of the workweek engaged in electronic communication and Internet searching, with close to 30 percent of a worker’s time dedicated to reading and answering e-mail alone” (Newport, 2016, p. 5). Sound like someone you know?

The good news is that if you carve out space for deep work, your professional career will soar. The first step is to become metacognitive about how you are spending your time during the day. One simple method is to self-monitor how you use your work days by keeping a grid near your computer or desk. At the end of every hour throughout your day, record how much time you actually spent doing your job duties of teaching (including prep and grading), writing and research, and service. Like a food diary or exercise journal, your shallow work addiction will become apparent quickly, but you will also gain metacognition about when and under which conditions you might attempt to fit in time for deep work.

Once you have a grasp of the issue at hand, you can begin to self-regulate your behavior by blocking off time in your schedule in which you can engage in a deeper level of creative thinking. Each person will gravitate toward a different modality conducive to an individual’s working styles or arrangements. The author offers a few choices for you to consider, which have been proven to be successful for other scholars and business leaders:

Monastic: Eliminate or radically minimize shallow obligations, such as meetings and emails, in an effort to focus solely on doing one thing exceptionally well. Put an out-of-office response on your email, work somewhere other than your workplace, or take a year-long sabbatical in order to completely separate from frivolous daily tasks that keep you away from research and writing. Most teaching faculty and academic leaders are unable to be purely monastic due to other duties.
Bimodal: Divide your time, dedicating some clearly defined stretches to deep pursuits and leaving the rest open to everything else. During the deep time, act monastically – seek intense and uninterrupted concentration – but schedule other time in your day for shallow work to be completed. One successful scholar shared the possibility of teaching a very full load one semester but not teaching at all during the next as an example of engaging deeply in both critical duties.
Rhythmic: Also called the “chain method” or “snack writing,” create a regular habit of engaging in deep work, such as every morning before going into work or at the end of each day. Blocking off one’s calendar and writing every day has been proven to be one of the most productive habits for scholars attempting to balance their research with other duties (Gardiner & Kearns, 2011).
Journalistic: Fit deep work into your schedule wherever you can – 15 minutes here, an hour there. Over time you will become trained to shift into writing mode on a moment’s notice. This approach is usually most effective for experienced scholars who can switch easily between shallow and deep work. Inexperienced writers may find that the multitasking yields unproductive results, so they should proceed cautiously with this method.

The key is to do something! You must ritualize whichever method you choose in order to optimize your productivity. This may take some trial and error, but with your new-found metacognition about how you work best and some alternative strategies to try, you will be more likely to self-regulate your behaviors in order to be successful in your scholarly pursuits. If you try new approaches and are still not engaging in enough deep work, consider joining a writing group or finding a colleague to hold you accountable on a regular basis. Again, like diet and exercise, others can sometimes provide the motivation and deadlines that we are unable to provide for ourselves. Over time, your addiction to shallow work will subside and your productivity will soar… or so they tell me.

Resources:

Gardiner, M., & Kearns, H. (2011). Turbocharge your writing today. Nature 475: 129-130. doi: 10.1038/nj7354-129a

Goodson, P. (2013). Becoming an academic writer: 50 exercises for paced, productive, and powerful writing. Los Angeles: Sage.

Newport, C. (2016). Deep work: Rules for focused success in a distracted world. New York: Grand Central Publishing.

Silvia, P. J. (2007). How to write a lot: A practical guide to productive academic writing. Washington, D.C.: American Psychological Association.

Fighting Imposter Syndrome Through Metacognition

By Charity S. Peak, Ph.D.

Have you ever felt like an imposter at work? Taught a class that was not your expertise? Felt intimidated before giving a presentation? Nearly every faculty member experiences this imposter phenomenon at some point. After all, as faculty we work around incredibly smart and talented people who shine from being experts in their field. Additionally, people drawn to academia naturally feel compelled to be knowledgeable and often find themselves to be inadequate when they are not (Huston, 2009).

Imposter syndrome is “an overwhelming sense of being a fraud, a phony, of not being good enough for [a] job, despite much evidence to the contrary” (Kaplan, 2009). Apart from accomplishing significant professional milestones, people cannot seem to internally acknowledge their success or feel deserving. This sense of being an imposter is prevalent among women but is increasingly being revealed by men as well. Although the condition is often referred to as a syndrome, it is important to understand that it is NOT actually a diagnosable mental illness found in the DSM-V. Instead, it is an affliction, similar to test or performance anxiety, experienced by a variety of high-achieving individuals that can be treated successfully using metacognition and self-regulation.

Reactions to imposter syndrome vary widely and by individual. Typically, imposter phenomenon starts with a self-sabotaging internal dialogue, such as:

Who do I think I am? I’m not smart enough to teach this class or present on this topic.
What if my students ask me a question that I can’t answer?
What if someone finds out I don’t know what I’m talking about?
I’m not cut out for this. I really can’t do this.

A physical reaction similar to other stressful situations (fight, flight, or freeze) often follows:

Increased blood pressure
Blushing
Sweating
Shaking
Tonic immobility (i.e., mental block or “deer in headlights”)

Faculty in these situations tend to respond in one of two ways:

Undercompensating by becoming submissive, overly agreeable or even apologetic
Overcompensating with defensive, bossy and aggressive behaviors

Recognize symptoms when they arise and recenter yourself through breathing:

Assume a comfortable posture
Close your eyes if possible
Focus on the sensations of your body
Breathe in through your nose and out through your mouth
When your mind wanders, gently bring it back to your breath
Breathe in, breathe out
Repeat for at least 10 breaths and up to 5 minutes

Reconstruct a new, positive internal dialogue. Talk to yourself as you would a good friend by being supportive and confidence-building.
Posture yourself as confident. It turns out that “fake it till you make it” works with regard to physical posture. People who use Power Poses for 2 minutes demonstrate higher levels of confidence-building hormones (testosterone) as opposed to stress-inducing hormones (cortisol) (Carney, Cuddy & Yap, 2010; Cuddy, 2012).
Acknowledge the limits of your knowledge. Instead of hiding your lack of expertise, build a repertoire of ways to deflect difficult questions, such as:

What do you think?
I don’t know. Does anyone want to look it up and tell us the answer?
Great question. Can we talk about that more after class (or meeting)?
Let’s not dive too deeply into that issue because it might distract us from today’s agenda.
Good thought. Does anyone want to collaborate to address that concern?
Here is what I know, and here is what I don’t know (Huston, 2009).

Avoid “teaching as telling.” Rather than lecturing, which requires great preparation and pressure to be the expert in the room, move toward new pedagogical models of facilitation which turn the teaching burden over to the students, such as jigsaw and gallery walk.
Know that you are not alone. It is plausible that nearly everyone in the room has felt this way at one point or another in their careers, even though they may not readily share these thoughts with others. Normalizing the feelings to yourself will start to defuse your anxiety.
Share the issue with others you trust. A mentor or even a small community of colleagues can collaboratively strategize about how to address the issue.
Recognize external factors that might contribute. Often people blame themselves for toxic situations which were created by outside circumstances. If the situation persists, consider declining future involvement to avoid setting yourself up for difficulties.

“Awareness is half the battle” really does apply to imposter syndrome. Through metacognition, you can conquer the self-defeating thoughts and behaviors that might prevent you from succeeding in your personal and professional life. Intentional self-monitoring of negative internal dialogue followed by practicing self-regulation through the simple strategies outlined above is the antidote to imposter syndrome. So next time you feel yourself break into a sweat (figuratively or literally), assume a Power Pose and leverage metacognition to triumph over your doubts!

Metacognition promotes success by helping us overcome self-defeating thoughts. Share on X
Resources:

Carney, D. R., Cuddy, A. J., & Yap, A. J. (2010). Power posing brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychological Science, 21(10), 1363-1368. doi: 10.1177/0956797610383437

Cuddy, A. (2012, October 1). Your body language shapes who you are [Video file]. Retrieved from https://www.ted.com/talks/amy_cuddy_your_body_language_shapes_who_you_are?language=en

Huston, T. (2009). Teaching What You Don’t Know. Cambridge, MA: Harvard University Press.

Kaplan, K. (2009). Unmasking the impostor. Nature, 459(21): 468-469. doi: 10.1038/nj7245-468a

Metacognitive Judgments of Knowing

Roman Taraban, Ph.D., Dmitrii Paniukov, John Schumacher, Michelle Kiser, at Texas Tech University

“The more you know, the more you know you don’t know.” Aristotle

Students often make judgments of learning (JOLs) when studying. Essentially, they make a judgment about future performance (e.g., a test) based on a self-assessment of their knowledge of studied items. Therefore, JOLs are considered metacognitive judgments. They are judgments about what the person knows, often related to some future purpose. Students’ accuracy in making these metacognitive judgments is academically important. If students make accurate JOLs, they will apply just the right amount of time to mastering academic materials. If students do not devote enough time to study, they will underperform on course assessments. If students spend more time than necessary, they are being inefficient.

As instructors, it would be helpful to know how accurate students are in making these decisions. There are several ways to measure the accuracy of JOLs. Here we will focus on one of these measures, termed calibration. Calibration is the difference between a student’s JOL related to some future assessment and his actual performance on that assessment. In the study we describe here, college students made JOLs (“On a scale of 0 to 100, what percent of the material do you think you can recall?”) after they read a brief expository text. Actual recall was measured in idea units (IUs) (Roediger & Karpicke, 2006). Idea units are the chunks of meaningful information in the text. Calibration is here defined as JOL – Recalled IUs, or simply, predicted recall minus actual recall. If the calibration calculation leads to a positive number, you are overconfident to some degree; if the calculation result is negative, then you are underconfident to some degree. If the result is zero, then you are perfectly calibrated in your judgment.

The suggestion from Aristotle (see quote above) is that gains in how much we know lead us to underestimate how much we know, that is, we will be underconfident. Conversely, when we know little, we may overestimate how much we know, that is, we will be overconfident. Studies using JOLs have found that children are overconfident (predicted recall minus actual recall is positive) (Lipko, Dunlosky, & Merriman, 2009; Was, 2015). Children think they know more than they know, even after several learning trials with the material. Studies with adults have found an underconfidence with practice (UWP) effect (Koriat et al., 2002), that is, the more individuals learn, the more they underestimate their knowledge. The UWP effect is consistent with Aristotle’s suggestion. The question we ask here is ‘which is it’: If you lack knowledge, do your metacognitive judgments reflect overconfidence or underconfidence, and vice versa? Practically, as instructors, if students are poorly calibrated, what can we do to improve their calibration, that is, to recalibrate this metacognitive judgment.

We addressed this question with two groups of undergraduate students, as follows. Forty-three developmental-reading participants were recruited from developmental integrated reading and writing courses offered by the university, including Basic Literacy (n = 3), Developmental Literacy II (n = 29), and Developmental Literacy for Second Language Learners (n = 11). Fifty-two non-developmental participants were recruited from the Psychology Department subject pool. The non-developmental and developmental readers were comparable in mean age (18.3 and 19.8 years, respectively) and the number of completed college credits (11.8 and 16.7, respectively), and each sample represented roughly fifteen academic majors. All participants participated for course credit. The students were asked to read one of two expository passages and to recall as much as they could immediately. The two texts used for the study were each about 250 words in length and had an average Flesch-Kincaid readability score of 8.2 grade level. The passages contained 30 idea units each.

To answer our question, we first calculated calibration (predicted recall – actual recall) for each participant. Then we divided the total sample of 95 participants into quartiles, based on the number of idea units each participant recalled. The mean proportion of correct recalled idea units, out of 30 possible, and standard deviation in each quartile for the total sample were as follows:

Q1: .13 (.07); Q2: .33 (.05); Q3: .51 (.06); Q4: .73 (.09). Using quartile as the independent variable and calibration as the dependent variable, we found that participants were overconfident (predicted recall > actual recall) in all four quartiles. However, there was also a significant decline in overconfidence from Quartile 1 to Quartile 4 as follows: Q1: .51; Q2: .39; Q3: .29; Q4: .08. Very clearly, the participants in the highest quartile were nearly perfectly calibrated, that is, they were over-predicting their actual performance by only about 8%, compared to the lowest quartile, who were over-predicting by about 51%. This monotonic trend of reducing overconfidence and improving calibration was also true when we analyzed the two samples separately:

NON-DEVELOPMENTAL: Q1: .46; Q2: .39; Q3: .16; Q4: .10;

DEVELOPMENTAL: Q1: .57; Q2: .43; Q3: .39; Q4: .13.

The findings here suggest that Aristotle may have been wrong when he stated that “The more you know, the more you know you don’t know.” Our findings would suggest that the more you know, the more you know you know. That is, calibration gets better the more you know. What is striking here is the vulnerability of weaker learners to overconfidence. It is the learners who have not encoded a lot of information from reading that have an inflated notion of how much they can recall. This is not unlike the children in the Lipko et al. (2009) research mentioned earlier. It is also clear in our analyses that typical college students as well as developmental college students are susceptible to overestimating how much they know.

It is not clear from this study what variables underlie low recall performance. Low background knowledge, limited vocabulary, and difficulty with syntax, could all contribute to poor encoding of the information in the text and low subsequent recall. Nonetheless, our data do indicate that care should be taken in assisting students who fall into the lower performance quartiles to make better calibrated metacognitive judgments. One way to do this might be by asking students to explicitly make judgments about future performance and then encouraging them to reflect on the accuracy of those judgments after they complete the target task (e.g., a class test). Koriat et al. (1980) asked participants to give reasons for and against choosing responses to questions before the participants predicted the probability that they had chosen the correct answer. Prompting students to consider the amount and strength of the evidence for their responses reduced overconfidence. Metacognitive exercises like these may lead to better calibration.

References

Koriat, A., Lichtenstein, S., Fischoff, B. (1980). Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory, 6(2), 107-118.

Koriat, A., Sheffer, L., & Ma’ayan, H. (2002). Comparing objective and subjective learning curves: Judgments of learning exhibit increased underconfidence with practice. Journal of Experimental Psychology: General, 131, 147–162.

Lipko, A. R., Dunlosky, J., & Merriman, W. E. (2009). Persistent overconfidence despite practice: The role of task experience in preschoolers’ recall predictions. Journal of Experimental Child Psychology, 102(2), 152-166.

Roediger, H., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255.

Was, C. (2015). Some developmental trends in metacognition. Retrieved from

https://www.improvewithmetacognition.com/some-developmental-trends-in-metacognition/.

How Do You Increase Your Student’s Metacognition?

Aaron S. Richmond

Metropolitan State University of Denver

How many times has a student come to you and said “I just don’t understand why I did so bad on the test?” or “I knew the correct answer but I thought the question was tricky.” or “I’ve read the chapter 5 times and I still don’t understand what you are talking about in class.”? What did you say or do for these students? Did it prompt you to wonder what you can do to improve your students’ metacognition? I know many of us at Improve with Metacognition (IwM), started pursuing research on metacognition because of these very experiences. As such, I have compiled a summary of some of the awesome resources IwM bloggers have posted (see below). These instructional strategies can be generally categorized into either self-contained lessons. That is a lesson that can teach some aspect of metacognition in one or two class sessions. Or metacognitive instructional strategies that require an entire semester to teach.

Self-Contained Instructional Strategies

In Stephen Chew’s Blog, Metacognition and Scaffolding Student Learning, he suggests that one way to improve metacognitive awareness is through well-designed review sessions (Chew, 2015). Chew suggests that students would metacogntively benefit by actively participate and incentivize participation in study review sessions. Second, Chew suggests that students should self-test before review so that it is truly a review. Third, have students predict their exam scores based on the review performance and have them reflect on their predictions after the exam.

Ed Nuhfer (2015) describes a way to increase metacognition through role-play. Ed suggests that we can use Edward De Bono’s Six Thinking hats method to train our students to increase their metacognitive literacy. In essence, using this method we can train our students to think in a factual way (white hat), be positive and advocate for specific positions (yellow hat), to be cautious (black hat), recognize all facets of our emotions (red hat), be provocative (green hat), and be reflective and introspective (blue hat). We can do this through several exercises where students get a turn to have different hats.

In David Westmoreland’s (2014) blog, he discusses a classroom exercise to improve metacognition. David created a “metacognitive lab that attempts to answer the question How do you know?” In the lab, he presents students in small groups a handful of “truth” statements (e.g., Eggs are fragile.). Then students must take the statement and justify (on the board) how it is true. Then the class eliminates the justifications if they know them not to be true. Then the students with one another about the process and why the statements were eliminated.

Course Long Instructional Strategies

Chris Was (2014) investigated whether “variable weight-variable difficulty tests” would improve students’ calibration (i.e., knowing when you know something and knowing when you don’t). Chris has his students take several quizzes. In each quiz, students can weight each question for varied amount of points (e.g., question 1 is easy so I will give it 5 points whereas question 4 is hard so I will only give it 2 points). Then students answer whether they believe they got the question correct or not. After each quiz is graded, a teaching assistant goes over the quiz and discusses with the students why they weighted the question the way they did and why the thought they would or would not get the question correct. Was found that this activity caused his students to become better at knowing when they knew or did not know something.

Similarly, Shumacher and Taraban (2015) discussed the use of the testing effect as a method to improve metacognition. They suggest there are mixed results of the testing method as an effective instructional method. That is, when students were repeatedly tested and were exposed to questions on multiple exams, only low achieving students metacognitively benefited.

John Draeger (2015) uses just-in-time teaching in attempt to improve metacognition. John asks students metacognitive prompting questions (e.g., What is the most challenging part of the reading?) prior to class and they submit their answers before coming to class. Although, he has not measured the efficacy of this method, students have responded positively to the process.

Parting Questions to Further this Important Conversation

There are many other instructional methods used to increase student metacognition described throughout IwM that are both self-contained and semester long. Please check them out!

But even considering all of what has been presented in this blog and available on IwM, I couldn’t help but leave you with some unanswered questions that I myself have:

What other instructional strategies have you used to increase student metacognition?
If you were to choose between a self-contained or semester long method, which one would you choose and why? Meaning, what factors would help you determine which method to use? Insructional goals? How closely related to course content? Time commitment? Level of student metacogntive knowledge? Level of course?
Once you have chosen a self-contained or semester long method, how should implementation methods differ? That is, what are the best practices used when implementing a self-contained vs. semester long technique?
Finally, often in the metacognition research in higher education, instructional strategies for improving metacognition are pulled from studies and experiments conducted in k-12 education. Are there any studies, which you can think of, that would be suitable for testing in higher education? If so, how and why?

References

Beziat, T. (2015). Goal monitoring in the classroom. Retrived from https://www.improvewithmetacognition.com/goal-monitoring-in-the-classroom/

Chew, S. (2015). Metacognition and scaffolding student learning. Retrieved from https://www.improvewithmetacognition.com/metacognition-and-scaffolding-student-learning/

Draeger, J. (2015). Using Justin-in-Time assignments to promote metacognition. Retrieved from https://www.improvewithmetacognition.com/using-just-in-time-assignments-to-promote-metacognition/

Nilson, L. B. (2015). Metacognition and specifications grading: The odd couple? Retrieved from https://www.improvewithmetacognition.com/metacognition-and-specifications-grading-the-odd-couple/

Nuhfer, E. (2015). Developing metacognitive literacy through role play: Edward De Bono’s six thinking hats. Retrieved from https://www.improvewithmetacognition.com/developing-metacognitive-literacy-through-role-play-edward-de-bonos-six-thinking-hats/

Shumbacher, J., & Traban, R. (2015). To test or not to test: That is the metacognitive question. Retrieved from https://www.improvewithmetacognition.com/to-test-or-not-to-test-that-is-the-metacognitive-question/

Was, C. (2014). Testing improves knowledge monitoring. Retrieved from https://www.improvewithmetacognition.com/testing-improves-knowledge-monitoring/

Westmoreland, D. (2014). Science and social controversy—A classroom exercise in metacognition. Retrieved from https://www.improvewithmetacognition.com/science-and-social-controversy-a-classroom-exercise-in-metacognition/

Using Just-in-Time assignments to promote metacognition

by John Draeger (SUNY Buffalo State)

In a previous post entitled “Just-in-time for metacognition,” I argued that Just-in-Time teaching techniques could be used to promote both higher-order-thinking and metacognition. Just-in-time teaching techniques require that students submit short assignments prior to class for review by the instructor before class begins (Novak 1999; Simkins & Maier, 2009; Schraff et al., 2011). In my philosophy courses, students send their answers to me electronically the night before class and I spend the morning of class using their answers to shape my pre-class planning. I’ve had success with higher-order-thinking questions, but I tended to ask students questions about their learning process only when the class had clearly gone off track. Since I’ve become convinced that developing good metacognitive habits requires practice, I’ve made metacognitive questions a regular component of my Just-in-Time assignments. In this post, I thought I would let you know how things are going.

Research shows that students learn more effectively when they are aware of their own learning process (I encourage you to surf around this site for examples). Borrowing from Tanner (2012) and Scharff (2014), I have asked students to think about why and how they engage in various learning strategies (e.g., reading, writing, reflecting). More specifically, I have asked: what was the most challenging part of the reading? Was the current reading more challenging than the last? What was the most useful piece of the reading? What was the most challenging piece of the reading? What was your reading strategy this week? How might you approach the reading differently next time? What was the most challenging part of the last writing assignment? How might you approach your next writing assignment differently? What are your learning goals for the week?

Responses from students at all levels have been remarkably similar. In particular, student responses fall into three broad categories: general student commentary (e.g., about the course, reading, particular assignment), content (e.g., students reframe the metacognition question and answer with use of course content), reflective practice (e.g., students actually reflect on their learning process).

First Type of Response: General Commentary

When asked to describe the most challenging part of the reading, students took the opportunity to observe that the reading too long, too boring, or it was interesting but confusing.
When asked to describe the most useful part of the reading, students often said that the question was difficult to answer because the reading was too long, too boring, or it was interesting but confusing.
When asked about their reading strategy, students observed that they did their best but the reading was too long, too boring, or interesting but confusing.
When asked about their learning goals for the week, students said that the question was strange, off the wall, and they had never been asked such a thing before.

Second Type of Response: Content

When asked to describe the most challenging part of the reading, students identified particular examples that were hard to follow and claims that seemed dubious.
When asked to describe the most useful part of the reading, students often restated the central question of the week (e.g., is prostitution morally permissible? should hate speech be restricted?) or summarized big issues (e.g., liberty argument for the permissibility of prostitution or hate speech).
When asked about their reading strategy, students often said that they wanted to understand a particular argument for that day (e.g., abortion, euthanasia, prostitution).
When asked their learning goal for the week, students said that they wanted to explore a big question (e.g., the nature of liberty or equality) and put philosophers into conversation (this is a major goal in all my courses).

Third Type of Response: Reflective practice

When asked to describe the most challenging part of the reading, students said that they didn’t give themselves enough time, they stretched it over multiple days, or they didn’t do it at all.
When asked about the most useful part of the reading, some students said that the reading forced them to challenge their own assumptions (e.g., “I always figured prostitution was disgusting, but maybe not”).
When asked about their reading strategies, some said that they had to read the material several times. Some said they skimmed the reading and hoped they could piece it together in class. Others found writing short summaries to be essential.
When asked about their learning goals for the week, some students reported wanting to become more open-minded and more tolerant of people with differing points of view.

Responses to the metacognitive prompts have been remarkably similar from students in my freshman to senior level courses. In contrast, I can say that there’s a marked difference by class year in responses to higher-order thinking prompts, possibly because I regularly use student responses to higher-order thinking prompts to structure class discussion. While I gave students some feedback on their metacognitive prompt responses, in the future I could be more intentional about using their responses to structure discussions of the student learning process.

I also need to refine my metacognition-related pre-class questions. For example, asking students to discuss the most challenging part of a reading assignment encourages students to reflect on roadblocks to understanding. The question is open-ended in a way that allows students to locate the difficulty in a particular bit of content, a lack of motivation, or a deficiency in reading strategy. However, if I want them to focus on their learning strategies, then I need to focus the question in ways that prompt that sort of reflection. For example, I could reword the prompt as follows: Identify one challenging passage in the reading this week. Explain why you believe it was difficult to understand. Discuss what learning strategy you used, how you know whether the strategy worked, and what you might do differently next time. Revising the questions so that they have a more explicitly metacognitive focus is especially important given that students are often unfamiliar with metacognitive reflection. If I can be more intentional about how I promote metacognition in my courses, then perhaps there can be gains in the metacognitive awareness demonstrated by my students. I’ll keep you posted.

References

Novak, G., Patterson, E., Gavrin, A., & Christian, W. (1999). Just-in-time teaching: Blending active learning with web technology. Upper Saddle River, NJ: Prentice Hall.

Scharff, L. “Incorporating Metacognitive Leadership Development in Class.” (2014). Retrieved from https://www.improvewithmetacognition.com/incorporating-metacognitive-leadership-development-in-class/.

Scharff, L., Rolf, J. Novotny, S. and Lee, R. (2011). “Factors impacting completion of pre-class assignments (JiTT) in Physics, Math, and Behavioral Sciences.” In C.

Rust (ed.), Improving Student Learning: Improving Student Learning Global Theories and Local Practices: Institutional, Disciplinary and Cultural Variations. Oxford Brookes University, UK.

Simkins, S. & Maier, M. (2009). Just-in-time teaching: Across the disciplines, across the academy. Stylus Publishing, LLC..

Tanner, K. D. (2012). Promoting student metacognition. CBE-Life Sciences Education, 11(2), 113-120.

Just-in-Time for Metacognition

By John Draeger, SUNY Buffalo State

This post brings metacognition to an already valuable teaching tool. Just-in-time techniques require that students submit short assignments prior to class. Instructors review those answers before class and use them to shape class time. In my philosophy classes, for example, I assign two short questions via a course management system (e.g., Blackboard). At least one of the questions is directly related to the reading. Students are required to submit their answers electronically by 11:00 p.m. the night before class. When I wake up in the morning, I read through their responses and use them to make decisions about how class time will be used. If students seemed to grasp the reading, then I spend less time reviewing the basic arguments and more time exploring deeper content and connections. If student responses displayed a misunderstanding of the reading, then we spend class time carefully examining passages in the text and digging out the relevant arguments.

Just-in-Time techniques have been used in a variety of disciplines and they have been shown to increase the likelihood that students will complete their reading assignments, read more carefully, and take ownership over their learning (Novak 1999; Simkins & Maier, 2009; Schraff et al. 2011). However, just-in-time assignments are typically used to prompt students to complete their assigned reading pages and gauge their basic comprehension. While both are valuable, I argue that the technique can also be used to promote other important skills.

For example, pre-class questions can be used to develop higher-order thinking skills. Students can be asked to examine an author’s point of view, underlying assumptions, or the implications of her view. Such questions prompt students to move beyond their knowledge of what is contained in the text towards active engagement with that text. Students can be asked to apply concepts in the reading (e.g., stereotype bias) to something in the news. And students can be asked to analyze the connections between related course ideas. In a previous post, “Using metacognition to uncover the substructure of moral issues,” I argued that students begin to “think like a philosopher” when they can move beyond the surface content (e.g., hate speech and national security) and towards the underlying philosophical substructure (e.g., rights, well-being, dangers of governmental intrusion). Like other skills, developing higher-order thinking skills requires practice. Because just-in-time assignments are a regular part of a student’s week, incorporating high-order thinking questions into just-in-time assignments can give students regular opportunities to practice and hone those skills.

Likewise, pre-class assignments can give students a regular outlet to practice and develop metacognition. Students can be asked to reflect on how they prepared for class and whether it was effective (Tanner 2012). Pre-class questions might include: how long did you spend with the reading? Did you finish? Did you annotate the text? Did you write a summary of the central argument? Did you formulate questions based on the reading for class discussion? Was this reading more difficult than the previous? If so, why? Did you find yourself having an emotional reaction to the reading? If so, did this help or hinder your ability to understand the central argument? Are your reading techniques adequately preparing you for class? Or, are you finding yourself lost in class discussion despite having spent time doing the reading? If pre-class questions related to higher-order thinking ask students to do more than simply “turn the pages,” then pre-class questions related to metacognition ask students to do more than simply engage with the material, but also engage with their own learning processes.

When just-in-time questions are a regular part of the ebb and flow of a course, students must regularly demonstrate how much they know and instructors can regularly use that information to guide course instruction. These techniques work because there is a consistent accountability measure built-in. I suggest that just-in-time assignments can also be used to give students regular practice developing both higher-order thinking and metacognition skills. I have been incorporating higher-ordering thinking into just-in-time assignments for years, but I confess that I have only given metacognition prompts when things have “gone wrong” (e.g., poor performance on exams, consistent misunderstanding of the reading). Responses to these questions have led to helpful conversation about the efficacy of various learning methods. Writing this blog post has prompted me to see the potential benefits of asking such questions more often. I pledge to do just that and to let you know how my students respond.

References

Novak, G., Patterson, E., Gavrin, A., & Christian, W. (1999). Just-in-time teaching: Blending active learning with web technology. Upper Saddle River, NJ: Prentice Hall.

Scharff, L., Rolf, J. Novotny, S. and Lee, R. (2011). “Factors impacting completion of pre-class assignments (JiTT) in Physics, Math, and Behavioral Sciences.” In C. Rust (ed.), Improving Student Learning: Improving Student Learning Global Theories and Local Practices: Institutional, Disciplinary and Cultural Variations. Oxford Brookes University, UK.

Simkins, S. & Maier, M. (2009). Just-in-time teaching: Across the disciplines, across the academy. Stylus Publishing, LLC..

Tanner, K. D. (2012). Promoting student metacognition. CBE-Life Sciences Education, 11(2), 113-120.

The Six Hour D… And How to Avoid It

This great essay by Russ Dewey (1997) evolved from a handout he used to give his students. He shares some common examples of poor study strategies and explains why they are unlikely to lead to deep learning (even if they are used for 6 hours…). He then shares a simple metacognitive self-testing strategy that could be tailored for courses across the disciplines.

http://www.psywww.com/discuss/chap00/6hourd.htm

Negotiating Chaos: Metacognition in the First-Year Writing Classroom

by Amy Ratto Parks, Composition Coordinator/Interim Director of Composition, University of Montana

“Life moves pretty fast. If you don’t stop and look around once in a while, you could miss it.” John Hughes, Ferris Bueller’s Day Off

Although the movie Ferris Bueller’s Day Off (Hughes, 1986) debuted long before our current first-year college students were born, the combined sentiment of the film remains relevant to them. If we combined Ferris’ sense of exuberant freedom with Cameron’s grave awareness of personal responsibility, and added Sloane’s blasé ennui we might see an accurate portrait of a typical first-year student’s internal landscape. Many of our students are thrilled to have broken out of the confines of high school but are worried about not being able to succeed in college, so they arrive in our classrooms slumped over their phones or behind computer screens, trying to seem coolly disengaged.

The life of the traditional first-year student is rife with negotiations against chaos. Even if we remove the non-academic adjustments of living away from home, their lives are full of confusion. All students, even the most successful, will likely find their learning identities challenged: what if all of their previous academic problem-solving strategies are inadequate for the new set of college-level tasks?

In the first-year writing classroom, we see vivid examples of this adjustment period play out every year. Metacognitive activities like critical reflective writing help students orient themselves because they require students to pause, assess the task at hand, and assess their strategies for meeting the demands of the task. Writing studies researchers know that reflection benefits writers (Yancey, 1998) and portfolio assessment, common in first-year program across the country, emphasizes reflection as a major component of the course (Reynolds & Rice, 2006). In addition, outcomes written by influential educational bodies such as National Council of Teacher’s of English (ncte.org), The Common Core State Standards Initiative (corestandards.org), and Council of Writing Program Administrators (wpacouncil.org) emphasize the importance of metacognitive skills and demonstrate a shared belief in its importance.

But students aren’t necessarily on board. It is the rare student who has engaged in critical reflection in the academic setting. Instead, many aren’t sure how to handle it. Is it busy work from the teacher? Are they supposed to reveal their deep, inner feelings or is it a cursory overview? Is it going to be graded? What if they give a “wrong” reflection? And, according to one group of students I had, “isn’t this, like, for junior high kids?” In this last question we again see the developing learner identity. The students were essentially wondering, “does this reflective work make us little kids or grown ups?”

If we want new college students to engage in the kind of reflective work that will help them develop transferable metacognitive skills, we need to be thoughtful about how we integrate it into the coursework. Intentionality is important because there are a number of ways teachers might accidentally perpetuate these student mindsets. In order to get the most from reflective activities in class, keep the following ideas in mind:

Talk openly with students about metacognition. If we want students to become aware of their learning, then the first thing to do is draw their attention to it. We should explain to students why they might care about metacognitive skills, as well as the benefits of investing themselves in the work. If we explain that reflection is one kind of metacognitive activity that helps us retrieve, sort, and choose problem-solving strategies, then reflection ceases to be “junior high” work and instead becomes a scholarly, collegiate behavior.
Design very specific reflective prompts. When in doubt, err on the side of more structure. Questions like “what did you think about the writing assignment” seem like they would open the door to many responses; actually they allow students to answer without critically examining their writing or research decisions. Instead, design prompts that require students to critically consider their work. For example, “Describe one writing choice you made in this essay. What was the impact of your decision?”
Integrate reflection throughout the semester. Ask students to reflect mid-way through the processes of drafting, research, and writing. If we wait until they finish an essay they learn that reflection is simply a concluding activity. If they reflect mid-process they become aware of their ability to assess and revise their strategies more than once. Also, reflection is a metacognitive habit of mind (Tarricone, 2011; Yancey, 1998) and habits only come to us through repeated activity.

These three strategies are a very basic beginning to integrating metacognitive activities into a curriculum. Not only do they help students evaluate the effectiveness of their attempts at problem solving, but they can also direct the students’ attention toward the strategies they’ve already brought to the class, thereby creating a sense of control over their learning. In the first-year writing classroom, where students are distracted and worried about life circumstance and learner identity, the sense of control gained from metacognitive work is especially important.

References

Chinich, M. (Producer), & Hughes, J.H. (Director). (1986). Ferris Beuller’s day off.[Motion picture]. USA: Paramount Pictures.

Reynolds, N., & Rice, R. (2006). Portfolio teaching: A guide to instructors. Boston, MA: Bedford St, Martin’s.

Tarricone, P. (2011). The taxonomy of metacognition. New York: Psychology Press.

Yancey, K.B. (1998). Reflection in the writing classroom. Logan: Utah State University Press.

(2013). First-year writing: What good does it do? Retrieved from http://www.ncte.org/library/nctefiles/resources/journals/cc/0232-nov2013/cc0232policy.pdf

(2014). Frameworks for success in postsecondary writing. Retrieved from http://wpacouncil.org/framework

(2014). English language arts standards. Retrieved from http://www.corestandards.org/ELA-Literacy/introduction/key-design-consideration/

Testing Improves Knowledge Monitoring

by Chris Was, Kent State University

Randy Isaacson and I have spent a great deal of time and effort creating a curriculum for an educational psychology class to encourage metacognition in preservice teachers. Randy spent a number of years developing this curriculum before I joined him in an attempt to improve the curriculum and use the curriculum to test hypotheses regarding improvement of metacognition with training for undergraduate preservice teachers. A detail description of the curriculum can be found in the National Teaching and Learning Forum (Isaacson & Wass, 2010), but I wanted to take this opportunity to give a simple overview of how we structured our courses and some of the results produced by using this curriculum to train undergraduates to be metacognitive in their studies.

With our combined 40+ years of teaching, we our quite clear that most undergraduates do not come equipped with the self-regulation skills that one would hope students would acquire before entering the university. Even more disappointing, is students lack the metacognition required to successfully regulate their own learning behaviors. Creating an environment that not only encourages, but also requires students to be metacognitive is not a simple task. However, it can be accomplished.

Variable Weight-Variable Difficulty Tests

The most important component of the course structure is creating an environment with extensive and immediate feedback. The feedback should be designed to help the student identify specific deficiencies in his or her learning strategies and metacognition. We developed an extensive array of learning resources which guide the student to focusing on knowing what they know, and when they know it. The first resource we developed is a test format that helps the students reflect and monitor their knowledge regarding the content and items on the test. In our courses we have students judge their accuracy and confidence in their responses for each item and having them predict their scores for each exam

Throughout the duration of the semester in which they were enrolled in the course students are administered a weekly exam (the courses meet Monday, Wednesday and Friday with the exams occurring on Friday). Each examination is based on a variable weight, variable difficulty format. Each examination contained a total of 35 questions composed of 15 Level I questions that were at the knowledge level, 15 Level II questions at the evaluation level, and 5 Level III questions at the application/synthesis level. Scoring of the exam was based on a system that increased points for correct responses in relation to the increasing difficulty of the questions and confidence in responses: Students choose 10 Level I questions and put those answers on the left side of the answer sheet. These 10 Level I questions are worth 2 points each. Ten Level II questions were worth 5 points each are placed on the left side of the answer sheet, and three Level III questions were worth 6 points each are placed on the left. Students were also required to choose the questions they were least confident about and place them on the right side of the answer sheet. These questions were only worth one point (5 of the 15 Level I and II questions, and 2 of the 5 Level III questions). The scoring equaled a possible 100 points for each exam. Correlations between total score and absolute score (number correct out of 35) typically range from r = .87 to r = .94. Although we provide students with many other resources to encourage metacognition, we feel that the left-right test format is the most powerful influence on student knowledge monitoring through the semester.

The Results

Along with our collaborators, we have conducted a number of studies using the variable weight-variable difficulty (VW-VD) tests as a treatment. Our research questions focus on whether the test format increases knowledge monitoring accuracy, individual differences in knowledge monitoring and metacognition, and psychometric issues in measuring knowledge monitoring. Below is a brief description of some of our results followed.

Hartwig, Was, Isaacson, & Dunlosky (2011) found that a simple knowledge monitoring assessment predicted both test scores and number of items correct on the VW-VD tests.

Isaacson & Was (2010) found that after a semester of VW-VD tests, knowledge monitoring accuracy on an unrelated measure of knowledge monitoring increased.

Metacognition and Reflective Thinking

By Steven C. Fleisher, California State University Channel Islands

Imagine that we are reading an assignment. As we read, do we think: “How long will this take?” “Will this be on the test?” If so, try this instead. Presume that we are reading the article as preparation for meeting later with an important person such as our supervisor to discuss the article. How would this situation change the questions we ask ourselves? Such thinking can make us aware of what constitutes satisfactory mastery of knowing and how to achieve it.

Think back for a moment to learning a psychomotor skill, such as learning to ride a bicycle. It is normal to master that skill with normal innate balance and strength. We might think: “That’s all there is to it.” However, watching cyclists in a serious bicycle race or triathlon, reveals that reliance only on innate ability cannot produce that kind of performance. That level of expertise requires learning to pedal with cadence, to deliver equal power from both legs, use the gearing appropriately, exploit position within a group of racers and pace oneself relative to challenges. Untrained innate ability can rarely get us far in comparison to the results of informed training.

The same is true in learning. Metacognitive skills (learnable skills) enhance academic performance. People with metacognitive skill will usually outperform others who lack such skill, even others with greater innate intelligence (natural ability). Metacognitive training requires developing three strengths: 1) metacognitive knowledge, 2) metacognitive monitoring, and 3) metacognitive control.

Metacognitive knowledge refers to our understanding about how learning operates and how to improve our learning. We should have enough of this knowledge to articulate how we learn best. For example, we can know when it is best for us to write a reflection about a reading in order to enhance our learning. We should be alert to our misconceptions about how our learning works. When we learn that cramming is not always the best way to study (Believe it!), we must give that up and operate with a better proven practice.

Metacognitive monitoring refers to developed ability to monitor our progress and achievement accurately. For example, self-assessment is a kind of metacognitive monitoring. We should know when we truly understand what we are reading and assess if we are making progress toward solving a problem. When we become accurate and proficient in self-assessment, we are much better informed. We can see when we have mastered certain material well enough, and when we have not.

Metacognitive control. This competency involves having the discipline and control needed to make the best decisions in our own interests. This aspect of metacognition includes acting on changing our efforts or learning strategies, or taking action to recruit help when indicated.

Putting it together. When we engage in metacognitive reflection, we can ask ourselves, for example, “What did we just learn?” “What was problematic, and why?” “What was easy, and why?” “How can we apply what we just learned?” Further, when we gain metacognitive skill, we begin to internalize habits of learning that better establish and stabilize beneficial neural connections.

Reflective Exercises for Students:

Metacognitive knowledge. Consider three learning challenges: acquiring knowledge, acquiring a skill, or making an evidence-based decision. How might the approaches needed to succeed in each of these three separate challenges differ?
Metacognitive monitoring. After you complete your next assignment or project, rate your resultant state of mastery on the following scale of three points: 0 = I have no confidence that I made any meaningful progress toward mastery; 1 = I clearly perceived some gain of mastery, but I need to get farther; 2 = I am currently highly confident that I understand and can meet this challenge.
Next, see if your self-rating causes you to take action such as to re-study the material or to seek help from a peer or an instructor in order to achieve more competence and higher confidence. A critical test will be whether your awareness from monitoring was able to trigger your taking action. Another will come in time. It will be whether your self-assessment proved accurate.
Metacognitive control. To develop better understanding of this, recall an example from life when you made a poor decision that proved to produce a result that you did not desire or that was not in your interests. How did living this experience equip you to better deal with a similar or related life challenge?

References

Chew, S. L. (2010). Improving classroom performance by challenging student misconceptions about learning. Association for Psychological Science: Observer, Vol. 23, No. 4. http://psychologicalscience.org/observer/getArticle.cfm?id=2666

Dunlosky, J. and Metcalf, J. (2009). Metacognition. Thousand Oaks, CA: Sage.

Leamnson, R. N. (1999). Thinking about teaching and learning: Developing habits of learning with first year college and university students. Sterling, VA: Stylus.

Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory Into Practice, 41(4), 219-225.

Wirth, K. (2010). The role of metacognition in teaching Geoscience. Science Education Resource Center, Macalester College. http://serc.carleton.edu/NAGTWorkshops/metacognition/activities/27560.html

Creating a Metacognitive Movement for Faculty

by Charity Peak, U.S. Air Force Academy*

Faculty often complain that students don’t complete reading assignments. When students do read, faculty yearn for deeper analysis but can’t seem to get it. With SAT reading scores reaching a four-decade low (Layton & Brown, 2012) and nearly forty percent of postsecondary learners taking remedial coursework (Bettinger & Long, 2009), it’s not surprising that college students are increasingly unable to meet the reading expectations of professors. Faculty sense the waning reading abilities of their students, but they struggle to identify how to address the problem. After all, they weren’t trained to be reading teachers.

In February 2012, a group of faculty gathered for a Scholarship of Teaching and Learning (SoTL) Circle at the U.S. Air Force Academy to discuss how to get students to read more critically. The topic spurred such great interest that an interdisciplinary faculty learning community on Reading Critically was formed to investigate the issue and share strategies to use in the classroom. What evolved was a collective movement by faculty to become metacognitively aware of why and how they were assigning and apprenticing students to read more critically within their disciplines.

Our first meeting tackled the big question, “What do we want to know about college reading?” Despite our interdisciplinary nature, we easily identified several common areas of concern: Compliance (completing reading assignments), Comprehension (understanding what they read), and Critical Analysis. These Three C’s of College Reading guided our discussions over the next two academic years and eventually led to the creation of a website to assist other faculty members struggling with the same issues.

As academics, our first inclination was to dive into the literature to determine what other institutions had discovered about this issue. Surely we weren’t the only faculty grappling with these concerns. Not surprisingly, the research literature confirmed that the vast majority of college students do not read assignments ahead of time and do not consider the textbook to be a critical component of learning (Berry et al., 2010). In fact, a number of studies find that college students only read textbooks about six hours per week (Spinosa et al., 2008), with just 20-30% reading compliance for any given day and assignment (Hobson, 2004). Faculty hoping to set the stage prior to class and engage learners in meaningful discussions during class must first address reading compliance among students.

Unfortunately, reading is not indicative of comprehension. The combination of students’ weak reading abilities (particularly marginalized students) and difficult textbook structure produce unskilled learners, which faculty are unprepared to handle. Hobson (2004) explains that most college teachers – content specialists – do not realize their students are struggling to comprehend assigned texts. Furthermore, if faculty insist on emphasizing reading as part of their course structure, then “helping students improve their reading skills should be the responsibility of every college-level teacher” (p. 4). Without specific strategies to address the reading needs of students, typically far outside the spectrum of the usual subject area specialist, faculty are rendered helpless in creating deep thinking environments in the classroom.

Because low reading compliance predicts nonparticipation (Burchfield & Sappington, 2000), college faculty must address the issue in an effort to drive deeper learning. Over the course of two years, our Reading Critically faculty learning community identified and shared several research-based strategies to assist faculty in improving reading compliance, comprehension, and critical analysis. With no budget and nothing more than a dedication to the cause, we invited speakers to our meetings from our own institution to share how they were apprenticing readers within their courses. We discovered the value of pre-class reading guides, concept mapping, equation dictionaries, and even reading aloud in class. The interdisciplinary connectedness and learning through a common academic concern became a welcome respite from the typical silos that exist in higher education.

By the end of our first year together, our faculty learning community had gathered a wealth of research-based practices that could be implemented in courses across all disciplines. While each of the group’s participants had learned a great deal, we weren’t sure how to spread the word and continue the movement. Then, we discovered Carnegie Mellon’s Solve a Teaching Problem website. Alas, a model for us to follow! We set out to design a website for faculty to Solve a Reading Problem. Collaboratively, we created a step-by-step way for faculty to address reading issues they were encountering in their courses:

Step 1: Identify a reading problem

Step 2: Investigate a reason for the problem

Step 3: Initiate a strategy to address the problem

Our learning community pooled resources together by suggesting various problems and solutions along with research-based literature to support our ideas. Faculty then submitted lesson ideas and classroom strategies they found successful in their own courses to support better reading compliance, comprehension, and critical analysis. While the website is still very much a work in progress, it represents two years of metacognition around why faculty assign readings and how to maximize those opportunities in the classroom.

Ultimately, our faculty learned that we have a responsibility to be metacognitive about our own teaching practices in order to improve learning. This group’s commitment to the cause created an interdisciplinary metacognitive movement among our faculty that is still developing. What metacognitive movement can you lead at your institution?

References:

Berry, T., Cook, L., Hill, N,. & Stevens, K. (2010). An exploratory analysis of textbook usage and study habits: Misperceptions and barriers to success. College Teaching, 59(1), 31-39.

Bettinger, E., & Long, B. (2009). Addressing the needs of underprepared college students: Does college remediation work? Journal of Human Resources, 44(3), 736-771.

Burchfield, C. M., & Sappinton, J. (2000). Compliance with required reading assignments. Teaching of Psychology, 27(1), 58-60.

Hobson, E. H. (2004). Getting students to read: Fourteen tips. IDEA Paper No. 40. Manhattan, KS: The IDEA Center.

Layton, L., & Brown, E. (September 24, 2012). SAT reading scores hit a four-decade low. Washington Post. Washington, D.C.

Spinosa, H., Sharkness, J., Pryor, J. H., & Liu, A. (2008). Findings from the 2007 administration of the College Senior Survey (CSS): National aggregates. Los Angeles: Higher Education Research Institute, UCLA.

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.