Tag: calibration

Metacognition and Scaffolding Student Learning

Effective scaffolding requires metacognitive awareness. #metacognition #learning Click To Tweetby Dr. Stephen Chew, Samford University, slchew@samford.edu

Scaffolding learning involves providing instructional support for students so that they can develop a greater understanding of a topic than they could on their own. The concept of scaffolding originated with the work of Vygotsky and was later developed by Bruner. Scaffolding is not simply giving students the answers, but helping students understand the chain of reasoning or evidence that leads to an answer. I argue that metacognition plays a crucial role in effective scaffolding. Without metacognitive awareness, attempts at scaffolding may only create overconfidence in students without any learning. Let’s examine a common scaffolding activity, review sessions for exams.

Early in my career I used to give review sessions until I realized that they weren’t being helpful to the students who needed them most. I gave students old exams to try to answer for their review. Since I change textbooks regularly, there were questions on the old exams on topics that weren’t covered in the current class. I thought the discrepancy would be obvious when students got to those questions, but only the very best students noticed. Most students answered the questions, basically by guessing, completely unaware that we had never covered the topic. In addition, many students would simply read the question and then check the answer to see if they had guessed correctly without trying to reason through the question or using it as an indicator of their degree of understanding. I realized that students hadn’t studied the material before the review session. They were using the session as a substitute for actually studying. Just going through the review session increased their (false) confidence that they had studied without increasing their learning. It was my first encounter with poor metacognition. The issue with a lot of the struggling students wasn’t the content, but their metacognition and study skills, which my review sessions weren’t addressing. So I stopped doing them.

In recent years, though, I’ve thought about bringing them back with changes to address poor metacognition. First, we know that students who most need review sessions are least likely to think they need them, so I would somehow require participation. This is one reason why I believe that brief formative assessments in class, where everyone has to participate, are better than separate, voluntary review sessions. If I were to reinstate separate review session, I might make participation worth a small portion of the exam grade. Second, I would somehow require that students had done their best to study for the exam BEFORE coming to the review session so it is truly a review. Third, the review session would have to induce students to use good study strategies, such as self-testing with feedback and reflection, or interleaving. I might require students to generate and post three good questions they want to know about the material as their entry ticket to the review session. This would require students to review material before the review session and question generation is an effective learning strategy. Finally, I would require students to utilize the feedback from the review to recognize the level of their understanding and what they need to do to improve. I might have them predict their exam grade based on their review performance. All of these changes should increase student metacognition. I’m sure I’d have to experiment with the format to try to figure it out, and my solution may not work for other classes or faculty. It’s never a simple matter of whether or not an activity such as review sessions are a good or bad idea, it’s how they are implemented.

Without metacognitive awareness, scaffolding can backfire. Consider how poor metacognition can undermine other scaffolding activities such as releasing PowerPoint slides of lectures, guided note taking, allowing a formula “cheat sheet” in STEM classes, and allowing students to discard a certain number of exam items they think they got wrong. If students lack metacognition, each of these activities can actually be counterproductive for student learning.


Executive Function: Can Metacognitive Awareness Training Improve Performance?

by Antonio Gutierrez, Georgia Southern University

In a recent meta-analysis of 67 research studies that utilize an intervention targeted at enhancing metacognitive awareness, Jacob and Parkinson (in press) argue that metacognitive interventions aimed at improving executive function processes are not as effective at improving student achievement as once believed by scholars and practitioners alike. In essence, the evidence in support of robust effects of these types of interventions in improving achievement is inconclusive. While descriptive research studies continue to report high associations between metacognitive awareness and performance or achievement measures, Jacob and Parkinson argue that the experimental evidence supporting a strong role of metacognitive training in improving student performance is scant. I have recently pondered a similar dilemma with research on the effect of metacognitive monitoring training on students’ performance, confidence judgments but especially calibration. The literature on these topics converges on the finding that metacognitive monitoring training improves performance and confidence in performance judgments but not necessarily calibration (see e.g., Bol et al., 2005; Gutierrez & Schraw, 2015; Hacker et al., 2008).

While Jacob and Parkinson’s meta-analysis is illuminating, I wonder whether, like the calibration literature, the conclusion that executive function interventions are not as effective at improving achievement may be due to very different conceptualizations of the constructs under investigation. In the case of calibration, the mixed findings may be due to the fact that the metacognitive monitoring interventions were not likely targeting the same thing. For instance, some interventions may have been targeting a reduction in calibration errors (overconfidence and underconfidence), others may have been targeting improvement in calibration accuracy, whereas yet others may have been targeting both, whether intentionally or unintentionally. Because these interventions were targeting different aspects of calibration, it could be that the inconclusive findings were due to a confounding of these various dimensions of calibration … comparing apples to oranges, if you will. Could the lack of robust effects of executive function interventions on achievement be due to a similar phenomenon? What if these studies were not targeting the same executive function processes, in which case they would not be as directly comparable as at first glance? Jacob and Parkinson’s (in press) study may lead some to believe that there is little to be gained in investing time and effort in executive function interventions. However, before we abandon these interventions, perhaps we should consider developing executive function interventions that are more specific and finer grained such as by targeting very specific aspects of the executive function rather than a more general approach.

References
Bol, L., Hacker, D. J., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy, and performance. The Journal of Experimental Education, 73, 269-290.

Gutierrez, A. P., & Schraw, G. (2015). Effects of strategy training and incentives on students’ performance, confidence, and calibration. The Journal of Experimental Education: Learning, Instruction, and Cognition. Advance online publication. doi: 10.1080/00220973.2014.907230

Hacker, D. J., Bol, L., & Bahbahani, K. (2008). Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection, and explanatory style. Metacognition Learning, 3, 101-121.

Jacob, R., & Parkinson, J. (in press). The potential for school-based interventions that target executive function to improve academic achievement: A review. Review of Educational Research. Advance online publication. doi: 10.3102/0034654314561338


Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments

This sometimes humorous article by Justin Kruger and David Dunning describes a series of four experiments that “that incompetent individuals have more difficulty recognizing their true level of ability than do more competent individuals and that a lack of metacognitive skills may underlie this deficiency.”  It also includes a nice review of the literature and several examples to support their study.

Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments, Journal of Personality and Social Psychology 1999, Vol. 77, No. 6. 121-1134


Evidence for metacognition as executive functioning

by Kristen Chorba, PhD and Christopher Was, PhD, Kent State University

Several authors have noted that metacognition and executive functioning are descriptive of a similar phenomenon (see Fernandez-Duque, et al., 2000; Flavell, 1987; Livingston, 2003; Shimamura, 2000; Souchay & Insingrini, 2004). Many similarities can be seen between these two constructs: both regulate and evaluate cognitions, both are employed in problem solving, both are required for voluntary actions (as opposed to automatic responses), and more. Fernandez-Duque, et al. (2000) suggest that, despite their similarities, these two areas have not been explored together because of a divide between metacognitive researchers and cognitive neuroscientists; the metacognitive researchers have looked exclusively at metacognition, focusing on issues related to its development in children and its implications for education. They have preferred to conduct experiments in naturalistic settings, as a way to maximize the possibility that any information gained could have practical applications. Cognitive neuroscientists, on the other hand, have explored executive functioning using neuroimaging techniques, with the goal of linking them to brain structures. In the metacognitive literature, it has been noted metacognition occurs in the frontal cortex; this hypothesis has been evaluated in patients with memory disorders, and studies have noted that patients with frontal lobe damage, including some patients with amnesia, had difficulties performing metacognitive functions, including FOK judgments (Fernandez-Duque, et al., 2000; Janowsky, Shimamura, & Squire, 1989; Shimamura & Squire, 1986; as cited in Shimamura, 2000). Additionally, source monitoring and information retrieval has also been linked with the frontal cortex; source monitoring is an important metacognitive judgment (Shimamura, 2000). As previously stated, executive functions seem to be located generally in the frontal lobes, as well as specifically in other areas of the brain, contributing to the growing body of literature indicating that executive functions are both correlated and function independently. To explore the link between executive functioning and metacognition, Souchay and Isingrini (2004) carried out an experiment in which subjects were first asked to make evaluations on their own metacognition; they were then given a series of neurological tests to assess their executive functioning. They not only found a “significant partial correlation between metamemory control and executive functioning” (p. 89) but, after performing a hierarchical regression analysis, found that “age-related decline in metamemory control may be largely the result of executive limitations associated with aging” (p. 89).

As it relates to executive functioning, Fernandez-Duque, et al. (2008) noted that “the executive system modulates lower level schemas according to the subject’s intentions . . . [and that] without executive control, information processing loses flexibility and becomes increasingly bound to the external stimulus” (p. 289). These authors use the terms executive function and metacognition as essentially interchangeable, and note that these functions enable humans to “guide actions” where preestablished schema are not present and allow the individual to make decisions, select appropriate strategies, and successfully complete a task. Additionally, the primary task of both metacognition and executive functions are top-down strategies, which inform the lower level (i.e.: in metacognition, the object level; in executive functioning, as the construct which controls the “selection, activation, and manipulation of information in working memory” [Shimamura, 2000, p. 315]). Reviewing the similarities between metacognition and executive function, it seems that they are highly correlated constructs and perhaps share certain functions.

Executive functions and metacognition, while exhibiting similar functions and characteristics have, largely, been investigated along separate lines of research. Metacognitive research has focused on application and informing the teaching and learning processes. Executive functions, on the other hand, have primarily been researched as they relate to structures and locations within the brain. Recent literature and research indicates that executive functions and metacognition may be largely the same process.

References

Baddeley, A. (2005). Human Memory: Theory and Practice, Revised Edition. United Kingdom; Bath Press.

Blavier, A., Rouy, E., Nyssen, A., & DeKeyster, V. (2005). Prospective issues for error   detection. Ergonomics, 7(10), 758-781.

Dinsmore, D., Alexander, P., & Loughlin, S. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational psychology review, 20(4), 391-409.

Dunlosky, J., Metcalfe, J. (2008). Metacognition. Los Angeles: Sage.

Fernandez-Duque, D., Baird, J., Posner, M. (2000). Executive attention and metacognitive regulation. Consciousness and Cognition, 9, 288-307.

Flavell, J. (1987). Speculations about the nature and development of metacognition. In F. Weinert and R. H. Kluwe, (Eds.) Metacognition, Motivation, and Understanding. Hillsdale, NJ: Lawrence Erlbaum.

Friedman, N. P., Haberstick, B. C., Willcutt, E. G., Miyake, A., Young, S. E., Corley, R.   P., & Hweitt, J. K. (2007). Greater attention problems during childhood predict        poorer executive functioning in late adolescence. Psychological Science, 18(10), 893-900.

Friedman, N. P., Miyake, A., Young, S. E., DeFries, J. C., Corley, R. P., Hewitt, J. K. (2008).  Individual differences in executive functions are almost entirely genetic in origin.  Journal of Experimental Psychology, General, 137(2), 201-225.

Friedman, N. P., Miyake, Corley, R. P., Young, S. E., DeFries, J. C., & Hewitt, J. K. (2006). Not all executive functions are related to intelligence. Psychological Science, 17(2), 172-179.

Georghiades, P. (2004). From the general to the situated: Three decades of metacognition.  research report. International Journal of Science Education, 26(3), 365-383.

Higham, P. A. & Gerrard, C. (2005). Not all errors are created equal: Metacognition and   changing answers on multiple-choice tests. Canadian Journal of Experimental   Psychology, 59(1), 28-34.

Keith, N. & Frese, M. (2005) Self-regulation in error management training: Emotion control and    metacognition as mediators of performance effects. Journal of Applied Psychology,  90(4), 677-691.

Keith, N. & Frese, M. (2008). Effects of error management training: A meta-analysis. Journal of Applied Psychology, 93(1), 59-69.

Lajoie, S. (2008). Metacognition, self regulation, and self-regulated learning: A rose by any other name? Educational Psychology Review, 20(4), 469-475.

Livingston, J. A. (2003). Metacognition: An overview. Online ERIC Submission.

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., & Howenter, A. (2000). The unity and diversity of executive functions and their contributions to complex         “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49-100.

Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new

findings. In G. H. Bower (Ed.), The Psychology of Learning and Knowing. Cambridge, MIT Press, p. 1-26.

PP, N. (2008). Cognitions about cognitions: The theory of metacognition. Online ERIC Submission.

Shimamura, A. (2000). Toward a cognitive neuroscience of metacognition. Consciousness and Cognition, 9, 313-323.

Souchay, C., & Isingrini, M. (2004). Age related differences in metacognitive control: Role of executive functioning. Science Direct. 56(1), 89-99.

Thiede, K. W., & Dunlosky, J. (1994). Delaying students’ metacognitive monitoring improves their accuracy in predicting their recognition performance. Journal of educational psychology, 86(2), 290-302.

Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J., Dunlosky, & A. Graessser (Eds.), Metacognition in educational theory       and practice, (p. 277-304). Hillsdale, NJ: Lawrence Erlbaum.


Comprehension Monitoring: The Role of Conditional Knowledge Part 2

by Antonio Gutierrez, Georgia Southern University

In my previous post, I discussed the role conditional knowledge (i.e., the why, when, and where to apply strategies given task demands) plays in learners’ ability to calibrate their performance against their actual performance. This is in light of debates about the relations between the various dimensions of metacognition. Calibration is a component of one’s ability to monitor comprehension, which is a regulatory function. Conditional knowledge, on the other hand, is part of the knowledge component of metacognition. As a researcher I often wonder whether instead of making assumptions that these various metacognitive functions are related whether perhaps we should empirically test these assumptions. In metacognitive research it is often assumed that the knowledge and regulation aspects of metacognition are related. From a theoretical perspective, this makes sense. However, for us to assume that this is the case with all samples and populations may be a stretch, especially given the diversity and individual differences among learners. In this vein, I am currently seeking ethics approval to conduct research with middle school students because this is an understudied population with respect to metacognition. In this proposed research I plan to not only investigate calibration among middle school students and the influence metacognitive strategy training has on learners’ calibration, but I plan to empirically assess the association between the eight dimensions of metacognition (Knowledge: declarative, procedural, and conditional; Regulation: planning, information management, debugging strategies; comprehension monitoring; and evaluation of learning). I will also attempt to test the predictive power of various components of metacognition on learners’ calibration. I am especially interested in empirically measuring the association between conditional knowledge and calibration as well as the predictive power of conditional knowledge on calibration. I expect that metacognitive strategy training will improve learners’ performance, confidence judgments, and also their calibration. I also suspect that those with greater conditional knowledge will have better calibration, and hence, I expect conditional knowledge to strongly predict calibration.

This particular study is one among a series of scientific investigations on the validity of theoretical claims made when researchers discuss metacognition. In my attempt to provide educators a toolkit of domain-general metacognitive strategies they can readily apply in their classrooms, this series of studies will help me provide the empirical evidence necessary to demonstrate the utility and relevance of metacognitive strategies to not only scholars but practitioners as well. These strategies have been adapted from an unpublished pilot study I conducted prior to my dissertation. This research will help me to continue to refine these strategies to better suit adolescents. Moreover, it will shed some light on the link between conditional metacognitive knowledge and calibration, which is a topic that began with earlier posts. Stay tuned for preliminary results of my first investigation.


Four cornerstones of calibration research: Why understanding students’ judgments can improve their achievement

“The target articles make significant advances in our understanding of students’ judgments of their cognitive processes and products. In general, the advances are relative to a subset of common themes, which we call the four cornerstones of research on metacognitive judgments. We discuss how the target articles build on these cornerstones (judgment bases, judgment accuracy, judgment reliability, and control) and how they are relevant to improving student achievement.” (p. 58) For more information about this article, follow the link below.

Dunlosky, J., & Thiede, K. W. (2013). Four cornerstones of calibration research: Why understanding students’ judgments can improve their achievement. Learning and Instruction, 24, 58-61.