Author: Lauren Scharff

The Challenge of Deep Learning in the Age of LearnSmart Course Systems

by Lauren Scharff, Ph.D. (U. S. Air Force Academy)

One of my close friends and colleague can reliably be counted on to point out that students are rational decision makers. There is only so much time in their days and they have full schedules. If there are ways for students to spend less time per course and still “be successful,” they will find the ways to do so. Unfortunately, their efficient choices may short-change their long-term, deep learning.

This tension between efficiency and deep learning was again brought to my attention when I learned about the “LearnSmart” (LS) text application that automatically comes with the e-text chosen by my department for the core course I’m teaching this semester. As a plus, the publisher has incorporated learning science (metacognitive prompts and spacing of review material) into the design of LearnSmart. Less positive, some aspects of the LearnSmart design seem to lead many students to choose efficiency over deep learning.

In a nutshell, the current LS design prompts learning shortcuts in several ways. Pre-highlighted text discourages reading from non-highlighted material, and the fact that the LS quiz questions primarily come from highlighted material reinforces those selective reading tendencies. A less conspicuous learning trap results from the design of the LS quiz credit algorithm that incorporates the metacognitive prompts. The metacognition prompts not only take a bit of extra time to answer, but students only get credit for completing questions for which they indicate good understanding of the question material. If they indicate questionable understanding, even if they ultimately answer correctly, that question does not count toward the required number of pre-class reading check questions. [If you’d like more details about the LS quiz process design, please see the text at the bottom of this post.]

Last semester, the fact that many of our students were choosing efficiency over deep learning became apparent when the first exam was graded. Despite very high completion of the LS pre-class reading quizzes and lively class discussions, exam grades on average were more than a letter grade lower than previous semesters.

The bottom line is, just like teaching tools, learning tools are only effective if they are used in ways that align with objectives. As instructors, our objectives typically are student learning (hopefully deep learning in most cases). Students’ objectives might seem to be correlated with learning (e.g. grades) or not (e.g. what is the fastest way to complete this assignment?). If we instructors design our courses or choose activities that allow students to efficiently (quickly) complete them while also obtaining good grades, then we are inadvertently supporting short-cuts to real learning.

So, how do we tackle our efficiency-shortcut challenge as we go into this new semester? There is a tool that the publisher offers to help us track student responses by levels of self-reported understanding and correctness. We can see if any students are showing the majority of their responses in the “I know it” category. If many of those are also incorrect, it’s likely that they are prioritizing short-term efficiency over long-term learning and we can talk to them one-on-one about their choices. That’s helpful, but it’s reactionary.

The real question is, How do we get students to consciously prioritize their long-term learning over short-term efficiency? For that, I suggest additional explicit discussion and another layer of metacognition. I plan to regularly check in with the students, have class discussions aimed at bringing their choices about their learning behaviors into their conscious awareness, and positively reinforcing their positive self-regulation of deep-learning behaviors.

I’ll let you know how it goes.

——————————————–

Here is some additional background on the e-text and the complimentary LearnSmart (LS) text .

There are two ways to access the text. One way is an electronic version of the printed text, including nice annotation capabilities for students who want to underline, highlight or take notes. It’s essentially an electronic version of a printed text. The second way to access the text is through the LS chapters. As mentioned above, when the students open these chapters, they will find that some of the text has already been highlighted for them!

As they read through the LS chapters, students are periodically prompted with some LS quiz questions (primarily from highlighted material). These questions are where some of the learning science comes in. Students are given a question about the material. But, rather than being given the multiple choice response options right away, they are first given a metacognitive prompt. They are asked how confident they are that they know the answer to the question without seeing the response options. They can choose “I know it,” “Think so,” “Unsure,” or “No idea.” Once they answer about their “awareness” of their understanding, then they are given the response options and they try to correctly answer the question.

This next point is key: it turns out that in order to get credit for question completion in LS, students must do BOTH of the following: 1) choose “I know it” when indicating understanding, and 2) answer the question correctly. If students indicate any other level of understanding, or if they answer incorrectly, LS will give them more questions on that topic, and the effort for that question won’t count towards completion of the required number of questions for the pre-class activity.

And there’s the rub. Efficient students quickly learn that they can complete the pre-class reading quiz activity much more quickly if they chose “I know it” to all the metacognitive understanding probes prior to each question. If they guess at the subsequent question answer and get it correct, it counts toward their completion of the activity and they move on. If they answer incorrectly, LS would give them another question from that topic, but they weren’t any worse off with respect to time and effort than if they had indicated that they weren’t sure of the answer.

If students actually take the time to take advantage of rather than shortcut the LS quiz features (there are additional ones I haven’t mentioned here), their deep learning should be enhanced. However, unless they come to value deep learning over efficiency and short-term grades (e.g. quiz completion), then there is no benefit to the technology. In fact it might further undermine their learning through a false sense of understanding.


Lean Forward, but Do It Metacognitively!

by Lauren Scharff, Ph.D. (U. S. Air Force Academy)

As the Director for the Scholarship of Teaching and Learning (SoTL) at my institution, a large part of my job description involves helping faculty intentionally explore new approaches and how they impact student learning. In other words – I work with forward-leaning faculty who are ready to try new things. So, I think a lot about how, when, and why faculty members adopt new pedagogies, tools, and activities, and about when, for whom, and in what contexts these new approaches enhance learning. This work dovetails nicely with the development and goals of metacognitive instruction.

As a reminder if you’re relatively new to our site, one of the premises we’ve previously shared here (e.g. Scharff, March 2015) and elsewhere (Scharff and Draeger, NTLF, 2015) is that Metacognitive Instruction involves the intentional and ongoing interaction between awareness and self-regulation, specifically with respect to the pedagogical choices instructors make as they design their lessons and then as they carry them out.

I was happy to see these connections reinforced last month at our 7th Annual SoTL Forum. Dr. Bridget Arend was invited to give a morning workshop and the keynote address. Along with James R. Davis, she is co-author of Facilitating Seven Ways of Learning: A Resource for More Purposeful, Effective and Enjoyable College Teaching. In her workshop Bridget dug into how to facilitate critical thinking, promote problem-solving, and support the building of skills (3 of the 7 ways of learning), while in her keynote she focused more strongly on the concept of matching student learning goals with the most effective teaching methods. She went beyond the usual discussion of tips and techniques to explore the underlying purpose, rationale, and best use of these [pedagogical] methods.

Dr. Bridget Arend giving the keynote address at the 7th Annual SoTL Forum at the U. S. Air Force Academy
Dr. Bridget Arend giving the keynote address at the 7th Annual SoTL Forum at the U. S. Air Force Academy

7_Ways_of_Learning
Books such as these can help support metacognitive instruction.

While Bridget did not explicitly use the term “metacognitive instruction,” it struck me that her message of purposeful choice of methods directly supported key aspects of metacognitive instruction, especially those related to awareness of our pedagogical decisions. We (instructors) should not incorporate pedagogies (or new tools or activities) just because they are the ones typically used by our colleagues, or because they are what was “done to us as students and it worked for us,” or because they are the “new, latest-greatest thing” we’ve heard about. Rather, we should carefully review our learning goals and consider how each possible approach might support those goals for our students and our context.

We should also be mindful of other factors that might influence our adoption of new approaches. For example, administrators or institutions often reward faculty who are leading the adoption of new technologies. Sometimes the message seems “the more new technologies incorporated the better” or “out with the old and in with the new” so a program or institution can market itself as being the most cutting edge in education. However, while many of us appreciate being rewarded or showcased for new efforts, we also need to pause to consider whether or not we’re really supporting student learning as well as we could with these practices.

Questions we should ask ourselves before implementation include, How will our new pedagogical approach or a new app really align with the learning goals we have for our students? Will all of our choices complement each other, or might they work at cross-purposes with each other? Realistically, there are a limited number of learning outcomes we can successfully accomplish within a lesson or even across a semester.

As we implement these new approaches and tools, we should ask additional questions. How are they actually impacting aspects of student engagement, attitudes towards learning, and ultimately, the learning itself? How might they be adjusted (either “in the moment” or in future lessons) as we use them in order to better support our learning goals for our students in our context? No group of students is the same, and the context also shifts over time. What worked well in the past might need adjusting or more radically changing in the future.

In sum, we know that no single approach is going to work for all learning goals or all students across all situations. But if we build our awareness of possibilities using resources such as Facilitating Seven Ways of Learning (and many other published papers and texts) to help guide our pedagogical choices; if we carefully attend to how our approaches affect students and student learning; and we if modify our approach based on those observations (and maybe using systematic data if we’re conducting a SoTL research project), then we WILL be more likely to enhance student learning (and our own development as metacognitive instructors).

Thus, lean forward as instructors, but do it metacognitively!

————————-

Davis, James R. & Arend, B. (2013). Facilitating Seven Ways of Learning: A Resource for More Purposeful, Effective and Enjoyable College Teaching. Stylus Publishing, Sterling, VA.

Scharff, L. & Draeger, J. (September, 2015). Thinking about metacognitive instruction. The National Teaching and Learning Forum, 24(5), p. 4-6. http://onlinelibrary.wiley.com/doi/10.1002/ntlf.2015.24.issue-5/issuetoc


Using Metacognition to Make International Connections

by Lauren Scharff, PhD, U. S. Air Force Academy and John Draeger, PhD, SUNY Buffalo State

If you’re one of our longer-term followers, you’ll notice that this post is a bit different from others on our site. We just wrapped up a fantastic week in Melbourne, Australia working with six colleagues from around the globe, and we want to share some of our metacognition endeavors and reflections with you. This experience was part of the second International Collaborative Writing Groups  (ICWG) that is an affiliate effort for the International Society for the Scholarship of Teaching and Learning (ISSoTL).

Eight groups were part of the ICWG. The groups formed in May and met virtually over the summer to focus their topics and develop an outline prior to the face-to-face meeting this past week. Our group’s topic was The Student Learning Process, and we focused our efforts on how metacognition would support the transfer of learning from one situation or context to another. We believe the transfer of learning is one of the ultimate goals of education because it supports lifelong learning and employability.

The group’s work on how metacognition supports the transfer of learning will be revealed when it’s published, but meanwhile, we will share some ways that metacognition was part of our experience of facilitating the group. We’ll start with some pictures to set the tone. The first shows our group working: from left to right, Lauren, Susan Smith (Leeds Beckett University, UK), Lucie S Dvorakova (Honors Student, University of Queensland, Australia), Marion Tower (University of Queensland), Dominic Verpoorten (IFRES-University of Liège, Belgium), Marie Devlin (Newcastle University, UK), and Jason M. Lodge (University of Melbourne, Australia), [John Draeger taking the pic]. The second gives you a sense of the overall setting, showing multiple groups all kept to task by savvy ICWG coordinators, Mick Healy (University of Gloucestershire, retired) and Kelly Matthews (University of Queensland). Fortunately, Mick and Kelly also built in some social time for community building. The third picture shows our group at the Victoria State Library, left to right: Dominique, Sam, Marion, Sue, Marion, John, Lauren and Jason.

ICWG_SLP_Working

ICWG_mult_groups

ICWG_SLP_Social

How Metacognition Found Its Way into Our Facilitating Experiences

If you read the home page of this site, you’ll notice that we loosely define metacognition as the intertwined awareness and self-regulation of a process/skill, specifically with the goal of developing that process or skill. Although the site is focused on metacognition as it relates to teaching and learning, it can refer to any skill or process. Facilitating a group can be much like teaching, but it involves some additional processes that might more traditionally be linked to leadership and communication.

We noticed ourselves using metacognition in the following aspects of our work:

Use of Language: Given the international character of the group, self-monitoring and self-regulation allowed us to navigate differences in language and underlying assumptions. For example, through our discussions we learned that academic faculty might be referred to as ‘staff,’ ‘tutor,’ ‘instructor’ or ‘professor.’ Individual courses might be referred to as ‘classes,’ ‘modules’ or ‘units’ of study.

Assumptions about education: Our discussion revealed differences in the structures of the university systems in different countries. When discussing how students might use their learning in one course to inform their learning in another, the two North Americans on the team (John and Lauren) tended to think about transfer learning between a diverse set of courses across a broad liberal arts core curriculum in addition to transfer across more closely related courses within a major. Because undergraduate education in Australia and the United Kingdom tend not to be structured around a broad core curriculum, members of the team from these countries tended to focus on transfer learning within a particular field of study.

As we drafted our text and created a survey that was to be used in four different countries, we each engaged in self-monitoring of the terms as the conversation was in progress and would regulate behavior accordingly. For example, someone would start by saying “I think that staff might…” but then quickly add “or perhaps you might say ‘professors.’” Similarly, we would use our newly developed awareness of the different educational structures to guide our discussion about how transfer of learning might be supported across all of our learning environments.

Management of Project Scope: Both transfer of learning and metacognition are vast areas of study. Given the wide variety of experiences and individual interests in our group, we explored a wide array of possible directions for our paper, some of which we decided we would table for follow-on papers (e.g. how student level of intellectual development might impact transfer of learning and the creation of a “toolkit” for instructors that would help them support transfer of learning). Moving the conversation in fruitful directions required that all of us remain mindful of the task at hand (i.e. working towards a 6000-word article). Self-monitoring allowed us to detect when an interesting discussion had gone beyond the scope of our current article and self-regulation more quickly brought us back to the task at hand.

In summary, the international character of the writing group added a depth and richness to the conversation, but it also increased the likelihood of misunderstanding and the challenge of group management. Self-monitoring and self-regulation allowed us to overcome those challenges.

Many thanks to our group members for a fantastic face-to-face experience, and we look forward to our continued exchanges as we finalize the paper and carry on with the follow-on papers.


The relationship between goals, metacognition, and academic success

In this article Savia Countinho investigates the relationship between mastery goals, performance goals, metacognition (using the Metacognitive Awareness Inventory), and academic success.

Countinho, S. (2007). The relationship between goals, metacognition, and academic success. Educate. 7(1), p. 39-47


Metacognitive Development in Professional Educators

Stewart, Cooper and Moulding investigate adult metacognition development, specifically comparing pre-service teachers and practicing teachers. They used the Metacognitive Awareness Inventory and found that metacognition improves significantly with age and with years of teaching experience, but not with gender or level of teaching (Pre-K though post-secondary ed levels).

Stewart, P. W., Cooper. S. S., & Moulding, L. R. (2007). Metacognitive development in professional educators. The Researcher, 21(1), 32-40.


Breaking the Content Mold: The Challenge of Shaping Student Metacognitive Development

by Dr. Lauren Scharff, U. S. Air Force Academy

We all know that it’s difficult to break long-term patterns of behavior, even when we’re genuinely motivated and well intentioned. It becomes significantly more difficult when we are trying to shift behavioral patterns of groups. This is true across a spectrum of situations and behaviors, but in this post I will focus on teachers and students shifting from a focus on content and basic skills to a focus on higher-level thinking and metacognitive skills.

These musing on “breaking the content mold” have become much more salient as I look forward to a new semester and I exchange ideas with colleagues about how we will approach our upcoming classes. I refer to the “content mold” as a way of illustrating how we, both students and teachers, have been shaped, or molded, by many years of prior experiences and expectations. Due to this shaping, the natural default for both groups is to teach or learn in ways that we have been exposed to in the past, especially if those approaches have seemed successful in the past. For many of us, this default is a focus on content and on disciplinary skills closely linked with the content. With conscious effort we can break out of that molded pattern of behavior to encourage interdisciplinary thinking and higher-level thinking skills that transfer beyond our course. However, when things get tough (e.g. when there are time constraints, high cognitive load situations, or pressures to achieve success as portrayed by exam scores), we tend to revert back to the more familiar patterns of behaviors, which for many of us means a focus on content and basic skills, rather than the use of higher-level thinking or metacognitive strategies.

Similarly, in an earlier post on this site, Ed Nuhfer points out that, “When students learn in most courses, they engage in a three-component effort toward achieving an education: (1) gaining content knowledge, (2) developing skills (which are usually specific to a discipline), and (3) gaining deeper understanding of the kinds of thinking or reasoning required for mastery of the challenges at hand. The American higher educational system generally does best at helping students achieve the first two. Many students have yet to even realize how these components differ, and few ever receive any instruction on mastering Component 3.”

One of the biggest challenges to breaking this molded pattern is that it will be far more likely to be successful if both the teacher and the student are genuinely engaged in the effort. No matter how much effort is put forth by an instructor, if value is not perceived by the student, then little change will occur. Similarly, even if a student has learned the value of higher-level thinking and metacognitive approaches, if a teacher doesn’t seem to value those efforts, then a student will astutely focus on what does seem to be valued by the teacher. A further challenge is that, over the course of a semester, the effort and motivation from both groups might wax and wane in a non-synchronous manner. As I explore these challenges, I will use myself and my less-than-successful efforts last semester as an example.

I taught an upper-level majors course in vision science, and because I have taught this course many times, I knew going in that the material is often unexpectedly challenging to students and most of them find the chapter readings to be difficult. (They contain a lot of brain biology and neural communication topics, and my students are not biology majors). Thus, I decided to build in a low-threat (with a small number of points), intentional, metacognitive reflection assignment for each lesson that had a reading. Students would indicate their level of reading completion (six levels encompassing a thorough reading with annotations, skimming, not at all) and their level of understanding of the material before class. If they had problems with any of the materials, they were supposed to indicate what steps they would take to develop understanding. They would record these and turn them in at mid-semester and at the end of the semester. I had hoped that this regular reflection would prompt their awareness of their reading behaviors and their level of learning from the reading, initiate proactive behaviors if they had poor understanding, and build habits by being completed regularly. I also took time at the start of the semester to explicitly explain why I was incorporating this regular reflection assignment.

Unfortunately, except for a couple of students, I would rate this assignment as a failure. I don’t believe it did any harm, but I also don’t believe that students used it as intended. Rather, I think most of them quickly and superficially answered the questions just so they could turn in their logs at the two required times. This type of reflection is not something that they have been asked to explicitly do in the majority (all?) of their prior courses, and they already had other strategies that seemed to work for their success in other classes For example, more than half way through the semester one student informed me that it was simply easier and faster to come to the teacher’s office and get reading guide answers (or homework problem solutions in other courses), rather than deeply read and try to figure it out on his own. Thus, if he didn’t understand as he skimmed, he didn’t worry about it. This approach wasn’t working well in my course, but up to that point he’d been very successful, so he persisted in using it (although I stopped answering his questions in my office until he could demonstrate that he’d at least tried to figure them out).

In hindsight, I believe that my actions (or lack of them) also fed into the failure. I assumed that students would bring their questions to class if they had them due to their increased awareness of them and the prompt about what they would do to increase their understanding. Thus, if there were no questions (typically the case), I used the class time to connect the readings with related application examples and demonstrations rather than reiterated what was in the readings. The students seemed engaged in class and showed no indication of specific problems with the readings. Their personal application reflection writing assignments (separate from the reading logs) were fantastic. However, their poor exam performance suggested that they weren’t deeply understanding the content, and I instinctively shifted back to my prior content-focused approaches. I also did not take time in class to directly ask them about their understanding of the readings, what parts they found most challenging, and why.

Thus, although I know I wanted to support the development of student metacognitive skills, and my students also seemed accepting of that goal when I introduced it to them at the beginning of the semester, both groups of us quickly reverted to old content-focused habits that had been “successful” in the past. I am not the first to note the challenges of developing metacognitive skills. For example, Case and Gunstone (2002) state the following, “Many … authors have emphasized that metacognitive development is not easy to foster (e.g., Gunstone & Mitchell, 1998; White, 1998). Projects to enhance metacognition need to be long-term, and require a considerable energy input from both teachers and students.”

So, what will I do in the future? My plans are to more regularly and explicitly engage in discussion of the reading reflection prompts (and other metacognitive prompts) during class. By giving class time to such discussion and bringing the metacognitive processes into the open (rather than keeping them private due to completion outside of class), I hope to indicate the value of the processes and more directly support student exploration of new ways of thinking about learning. Importantly, I hope that this more public sharing will also keep me from falling back to a simple content focus when student performance isn’t what I’d like it to be. Ultimately, metacognitive development should enhance student learning, although it is likely to take longer to play out into changed learning behaviors. I need to avoid the “quick fix” of focusing on content. Thus, I plan to shape a new mold for myself and openly display it my students. We’ll all be more likely to succeed if we are “all in” together.

——–

Nuhfer, E. (15 July 2014). Metacognition for Guiding Students to Awareness of Higher-level Thinking (Part 1). Improve with Metacognition. https://www.improvewithmetacognition.com/metacognition-for-guiding-students-to-awareness-of-higher-level-thinking-part-1/

Case, J. & Gunstone, R. (2002). Metacognitive Development as a Shift in Approach to Learning: An in-depth study. Studies in Higher Education 27(4), p. 459-470. DOI: 10.1080/0307507022000011561

 

 


Habits of Mind

by Arthur L. Costa, Ed. D. (Professor Emeritus, California State University, Sacramento). This paper summarizes 16 attributes of what human beings do when they behave intelligently, referred to as Habits of Mind.  Metacognition is the 5th mentioned (see a nice summary of all 16 on the final page). Dr. Costa points out that these “Habits of Mind transcend all subject matters commonly taught in school. They are characteristic of peak performers whether they are in homes, schools, athletic fields,organizations, the military, governments, churches or corporations.”


Parallels: Instructors’ Metacognition Practices and their Mindsets

by Lauren Scharff, U. S. Air Force Academy

A week ago my institution held our annual Outstanding Educator’s Award Ceremony, at which the Chancellor of the University of Colorado – Colorado Springs, Dr. Pamela S. Shockley-Zalabak, gave the keynote address. Her presentation was engaging and right on target for the event. It helped honor the winners of the awards, and at the same time it was applicable to all the other faculty members who were in attendance. But what is prompting this blog piece is the parallel I found between her points about instructor mindset and our efforts to explore and develop metacognitive instruction (check out our Phase I research project summary).

As many of you are likely aware, and as cited by Dr. Shockley-Zalabak, Dr. Carol Dweck has led the research on the concept of mindset with respect to how it might impact educational (and many other) behaviors. She found evidence for two types of mindset: fixed and growth. Individuals who show a fixed mindset believe that characteristics such as scholastic ability, leadership potential, and speaking skills are innate rather than developable. In contrast, individuals with a growth mindset believe that such characteristics can be developed, and they seek opportunities to do so. These mindsets impact the likelihood that individuals will seek out challenging experiences (growth mindset) as opposed to seeking out experiences that will reinforce their current level of skill and avoid failure (fixed mindset). (Here is a nice review of Carol Dweck’s book, Mindset: The New Psychology of Success.)

Typically, research on mindset and efforts to shift mindsets from fixed to growth are focused on students. For example, see the nice review and developmental activity shared in Charity Peak’s blog post from February of this year. This emphasis on student mindset is similar to the emphasis on student rather than instructor metacognitive practices. While a student focus is extremely important (ultimately students are the target population of our educational efforts), we shouldn’t neglect to acknowledge, study, and develop instructors with respect to their mindset (and their metacognitive practices).

Dr. Shockley-Zalabak’s keynote presentation pointed out that attending to instructor mindset, and not just student mindset, is key to creating great educational climates. Once you think about instructor mindset, the implications become obvious. Instructor mindset can have large influences on student learning because mindset can impact instructor expectations about their students’ abilities and about their own teaching ability.

Instructors with fixed mindsets tend to believe that students either are or are not capable within their discipline. It’s not hard to imagine that students in such a teacher’s course might not thrive unless they showed early promise and were tagged as “talented” by their instructor. For a teacher with a fixed mindset, it might be difficult to understand why effort should be put into those students who don’t seem capable. Such instructors prefer to only work with the top students, not realizing how much potential they might be overlooking and inadvertently not developing.

Instructor mindset can also apply to instructors’ views about teaching ability. If they believe that great instructors are born, not made, then they will likely resist opportunities for professional development. New approaches and pedagogies are threatening because they present the possibility of a decreased sense of efficacy as they move out of their comfortable routine. Growth mindset teachers, on the other hand, will continuously seek out new approaches, and if they don’t work well, view those experiences as learning opportunities.

Take a moment to evaluate yourself and your mindset tendencies. Where do you think you fall in the fixed-growth spectrum? (As with many evaluation tasks, it’s probably easier to roughly categorize some of your colleagues as having more fixed or growth mindsets about their students and their teaching than it is to accurately examine yourself.) Although it’s not always easy to attain, self-awareness is foundational for effective and intentional self-development.

Self-awareness is also one of the key components of metacognition, which leads me to wonder…   Will metacognition lead someone to recognize that she needs to develop a growth mindset? Will a growth mindset lead her to become a more metacognitive instructor? These questions lead me to believe that future phases of our metacognitive instruction research project should include explicit efforts to develop awareness of one’s mindset in addition to awareness and self-regulation of the teaching strategies that one chooses.*

————–

* If you are interested in participating in a future phase of the metacognitive instruction study, please contact one of the two lead investigators: Dr. Lauren Scharff (laurenscharff@gmail.com) or Dr. John Draeger (draegejd@buffalostate.edu).

 


What Do We Mean by “Metacognitive Instruction”?

by Lauren Scharff (U.S. Air Force Academy*) 

Many of you are probably aware of the collaborative, multi-institutional metacognitive instruction research project that we initiated through the Improve with Metacognition site.  This project has been invigorating for me on many levels. First, through the process of developing the proposal, I was mentally energized. Several of us had long, thoughtful conversations about what we meant when we used the term “metacognitive instruction” and how these ideas about instruction “mapped” to the concept of “metacognitive learning.”  These discussions were extensions of some early blog post explorations, What do we mean when we say “Improve with metacognition”? (Part 1 and Part 2). Second, my involvement in the project led me to (once again) examine my own instruction. Part of this self-examination happened as a natural consequence of the discussions, but also it’s happening in an ongoing manner as I participate in the study as an intervention participant. Good stuff!

For this post, I’d like to share a bit more about our wrangling with what we meant by metacognitive instruction as we developed the project, and I invite you to respond and share your thoughts too.

Through our discussions, we ultimately settled on the following description of metacognitive instruction:

Metacognitive instructors are aware of what they are doing and why. Before each lesson, they have explicitly considered student learning goals and multiple strategies for achieving those goals.  During the lesson, they actively monitor the effectiveness of those strategies and student progress towards learning goals.  Through this pre-lesson strategizing and during lesson monitoring awareness, a key component of metacognition, is developed; however, awareness is not sufficient for metacognition.  Metacognitive instructors also engage in self-regulation. They have the ability to make “in-the-moment”, intentional changes to their instruction during the lesson based on a situational awareness of student engagement and achievement of the learning objectives — this creates a responsive and customized learning experience for the student.

One of the questions we pondered (and we’d love to hear your thoughts on this point), is how these different constructs were related and / or were distinct. We came to the conclusion that there is a difference between reflective teaching, self-regulated teaching, and metacognitive instruction/teaching.

More specifically, a person can reflect and become aware of their actions and their consequences, but at the same time not self-regulate to modify behaviors and change consequences, especially in the moment. A person can also self-regulate / try a new approach / be intentional in one’s choice of actions, but not be tuned in / aware of how it’s going at the moment with respect to the success of the effort. (For example, an instructor might commit to a new pedagogical approach because she learned about it from a colleague. She can implement that new approach despite some personal discomfort due to changing pedagogical strategies, but without conscious and intentional awareness of how well it fits her lesson objectives or how well it’s working in the moment to facilitate her students’ learning.) Metacognition combines the awareness and self-regulation pieces and increases the likelihood of successfully accomplishing the process (teaching, learning, or other process).

Thus, compared to other writings we’ve seen, we are more explicitly proposing that metacognition is the intentional and ongoing interaction between awareness and self-regulation. Others have generally made this claim about metacognitive learning without using the terms as explicitly. For example, “Simply possessing knowledge about one’s cognitive strengths or weaknesses and the nature of the task without actively utilizing this information to oversee learning is not metacognitive.” (Livingston, 1997). But, in other articles on metacognition and on self-regulated learning, it seems like perhaps the metacognitive part is the “thinking or awareness” part and the self-regulation is separate.

What do you think?

——————
Livingston, J. A. (1997). Metacognition: An Overview. Unpublished manuscript, State University of New York at Buffalo. http://gse.buffalo.edu/fas/shuell/cep564/metacog.htm

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments

This sometimes humorous article by Justin Kruger and David Dunning describes a series of four experiments that “that incompetent individuals have more difficulty recognizing their true level of ability than do more competent individuals and that a lack of metacognitive skills may underlie this deficiency.”  It also includes a nice review of the literature and several examples to support their study.

Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments, Journal of Personality and Social Psychology 1999, Vol. 77, No. 6. 121-1134


Teaching Perspectives Inventory (TPI): The 5 Perspectives

There are a lot of free surveys/inventories “out there” for all sorts of things, most often related to some aspect of personality. If you use them in a reflective manner, they can help you better understand yourself – your . The TPI (also free) offers a chance for you to reflect on your teaching perspectives (one aspect of metacognitive instruction). The TPI suggests 5 perspectives: Transmission, Apprenticeship, Developmental, Nurturing, and Social Reform.

http://www.teachingperspectives.com/tpi/


The Teaching Learning Group at CSUN

Two years ago, eight faculty at California State University, Northridge, began studying how people learn as a grassroots effort to increase student success by focusing on what instructors do in the classroom. Our website shares our efforts, Five Gears for Activating Learning, as well as supporting resources and projects developed to date (e.g., documents, videos, and a yearlong Faculty Learning Community in progress). Although all five gears interact when people learn and develop expertise, our fifth gear, the Developing Mastery gear, focuses on assisting students in developing their metacognitive skills.

http://www.csun.edu/cielo/teaching-learning-group.html


The Six Hour D… And How to Avoid It

This great essay by Russ Dewey (1997) evolved from a handout he used to give his students. He shares some common examples of poor study strategies and explains why they are unlikely to lead to deep learning (even if they are used for 6 hours…). He then shares a simple metacognitive self-testing strategy that could be tailored for courses across the disciplines.

http://www.psywww.com/discuss/chap00/6hourd.htm


Despite Good Intentions, More is Not Always Better

by Lauren Scharff, U.S. Air Force Academy*

A recent post to the PSYCHTEACH listserv got me thinking about my own evolution as a teacher trying my best to help the almost inevitable small cluster of students who struggled in my courses, often despite claiming to “have studied for hours.” The post asked “Have any of you developed a handout on study tips/skills that you give to your students after the first exam?” A wide variety of responses were submitted, all of which reflected genuinely good intentions by the teachers.

However, based on my ongoing exploration of metacognition and human learning, I believe that, despite the good intentions, some of the recommendations will not consistently lead to the desired results. Importantly, these recommendations actually seem quite intuitive and reasonable on the surface, which leads to their appeal and continued use. Most of those that fall into this less ideal category do so because they imply that “More is Better.”

For example, one respondent shared, “I did correlations of their test scores with their attendance so far, the number of online quizzes they have taken so far, and the combined number of these two things. [All correlations were positive ranging from 0.35 to 0.57.] So I get to show them how their behaviors really are related to their scores…”

This approach suggests several things that all seem intuitively positive: online quizzes are a good way to study and attending class will help them learn. I love the empowerment of students by pointing out how their choice of behaviors can impact their learning! However, the message that more quizzes and simple attendance will lead to better grades does not capture the true complexity of learning.

Another respondent shared a pre-post quiz reflection assignment in which some of the questions asked about how much of the required reading was completed and how many hours were put into studying. Other questions asked about the use of chapter outcomes when reading and studying, the student’s expected grade on the quiz, and an open-ended question requesting a summary of study approaches.

This pre-post quiz approach seems positive for many reasons. Students are forced to think about and acknowledge levels and types of effort that they put into studying for the quizzes. There is a clear suggestion that using the learning outcomes to direct their studying would be a positive strategy. They are asked to predict their grades, which might help them link their studying efforts with predicted grades. These types of activities are actually good first steps at helping students become more metacognitive (aware and thoughtful) about their studying. Yea!

However, a theme running through the questions seems to be, again, “more is better.” More hours. More reading. The hidden danger is that students may not know how to effectively use the learning outcomes, how to read, how to effectively engage during class, how to best take advantage of practice quizzes to promote self-monitoring of learning, or what to do during those many hours of studying.

Thus, the recommended study strategies may work well for some students, but not all, due to differences in how students implement the strategies. Therefore, even a moderately high correlation between taking practice quizzes and exam performance might mask the fact that there are subgroups for which the results are less positive.

For example, Kontur and Terry (2013) found the following in a core Physics course, “On average, completing many homework problems correlated to better exam scores only for students with high physics aptitude. Low aptitude physics students had a negative correlation between exam performance and completing homework; the more homework problems they did, the worse their performance was on exams.”

I’m sure you’re all familiar with students who seem to go through “all the right motions” but who still struggle, become frustrated, and sometimes give up or develop self-doubt about their abilities. Telling students to do more of what they’re already doing if it’s not effective will actually be more harmful.

This is where many teachers feel uncomfortable because they are clearly working outside their disciplines. Teaching students how to read or how to effectively take notes in class, or how to self-monitor their own learning and adjust study strategies to different types of learning expectations is not their area of expertise. Most teachers somehow figured out how to do these things well on their own, or they wouldn’t be teachers now. However, they may never have thought about the underlying processes of what they do when they read or study that allowed them to be successful. They also feel pressures to cover the disciplinary content and focus on the actual course material rather than learning skills. Unfortunately, covering material does little good if the students forget most of the content anyway. Teaching them skills (e.g., metacognitive study habits) offers the prospect of retaining more of the disciplinary content that is covered.

The good news is that there are more and more resources available for both teachers and students (check out the resources on this website). A couple great resources specifically mentioned by the listserv respondents are the How to Get the Most out of Studying videos by Stephen Chew at Samford University and the short reading (great to share with both faculty and students) called The Six Hour D… and How to Avoid it by Dewey (1997). Both of these highlighted resources focus on metacognitive learning strategies.

This reflection on the different recommendations is not meant to belittle the well-intentioned teachers. However, by openly discussing these common suggestions, and linking to what we know of metacognition, I believe we can increase their positive impact. Share your thoughts, favorite study suggestions and metacognitive activities by using the comments link below, or submitting them under the Teaching Strategies tab on this website.

References

Dewey, R. (1997, February 12) The “6 hour D” and how to avoid it. [Online]. Available: http://www.psywww.com/discuss/chap00/6hourd.htm.

Kontur, F. & Terry, N. The benefits of completing homework for students with different aptitudes in an introductory physics course. Cornell Physics Library Physics Education. arXiv:1305.2213

 

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


Incorporating Metacognitive Leadership Development in Class

by Lauren Scharff, U.S. Air Force Academy*

During the spring 2014 semester I decided to try an explicitly metacognitive approach to leadership development in my Foundations for Leadership Development course in the Behavioral Sciences department at the United States Air Force Academy.

I had taught the course twice before and had many discussions with other course instructors. Overall, my sense was that many of our students didn’t intentionally and systematically connect what they were doing and learning in the course with their own personal leadership development. This is despite a paper that focused on a personal leadership development plan, and a video project that focused on implementing positive change within their squadrons.

This course is an upper-level, required core courses in our curriculum, and my section was one of more than 30 with approximately a dozen different instructors teaching sections. At our institution, much of the course structure within core courses is standardized across instructors, but I had 20% of the points with which to do what I desired, as long as 10% somehow assessed accountability for lesson preparation.

I was aware of a foundation of research in skill development (e.g. Svinicki, 2004), so I knew that in order to most effectively develop skills, people need multiple opportunities for practice coupled with feedback.  Feedback leads to awareness of strengths, shortcomings, and possible alternate strategies. This understanding of skill development became intertwined with my increasing focus on metacognitive approaches. I came to the conclusion that perhaps part of the less-than-ideal student connection to the course material and objectives occurred because our course activities that were designed to support that connection didn’t provide (require) enough opportunities for practice and continued awareness, especially beyond the classroom and course requirements.

As I prepared for the semester I drew on resources from The Learning Record, which outlines Five Dimensions of Learning that connected well with goals I had for my students’ leadership development: confidence and independence, knowledge and understanding, skills and strategies, use of prior and emerging experience, and critical reflection. The site also shares well-developed activities and assignments that supported my goal of using a metacognitive approach to promote my students’ leadership development.

Ultimately, I designed my course to be centered around journal entries, which I also completed.  During each lesson we shared our understandings, questions, and reflections based on the readings, as well as examples of personal observations of leadership and our reflections on how what we were learning might be effectively applied to real situations. More specifically, the journal entries included 1) answers to guided reflection questions about each reading for each lesson, and 2) at least two personal leadership observations and analyses each week. I created a simple grading system so that I wouldn’t be overloaded with assessing journal entries every lesson. (Journal assignment)

A quick poll of my students (N=13) indicated that none of them regularly kept personal journals, and only two had ever had any sort of journal assignment for a class. Thus, this requirement for regular journal writing required a change of habit for them that also represented increased time and energy for class preparation. Although there was some adjustment, when I asked for feedback after two weeks of class, the students were unanimous in their agreement that they were more deeply reading than if I had incorporated reading quizzes for accountability and that they preferred to continue the personal and reading reflections even though they involved frequent writing. Discussion during class was deeper and more engaging than in previous semesters.

Twice during the semester, students wrote evidence-based personal development evaluations, based on a shared example from The Learning Record. Students chose examples from their journals to support their evaluations of their own leadership development. These evaluation exercises forced them to thoughtfully review their observations across the weeks of the semester and develop ongoing awareness of their leadership strengths and weaknesses as well as an understanding of alternate strategies and when/how they might be useful for their leadership efforts. (Personal Development Evaluation assignment)

I also added a question each time that had them evaluate the journal approach and course design.  We made some tweaks at mid-semester. By the end of the semester, all but one student reported that the journal entries deepened their learning and personal awareness of their leadership development. While I will likely make some further tweaks in future semesters, I believe that this approach was a success, and that it could be scaled up for larger classes (see the simplified grading scheme in the Journal assignment). Below are sample comments from the final evaluation assignment (released with student permission):

“The leadership journal has had a tremendous effect on my personal development as a leader.  The journal has made me aware of my strengths and weaknesses…. The journal allowed me an avenue to give time and actually think about how I am doing as a leader and peer within my environment.”

 

“The personal observations were definitely helpful for documenting our successes and failures, which we can look back upon and improve. This relates not only to our personal leadership development, but to how we learn about ourselves.”

 

“These journals have taken all of us on a journey through the semester.  They undoubtedly began as something that we disliked and looked down upon each week.  However, I have really grown to love and understand this application of leadership growth.  They not only provide a chance for us to look back on our leadership gains and failures, they offer an opportunity for us to challenge ourselves in order to write about the things we want to see in ourselves.  The journals have become much more than a simple task of writing on a week-to-week basis.  They have grown into an outpouring of our character and lives as we turn the page from underclassmen to upper-class leaders and eventually to lieutenants in a few short months.  I believe that these journals are also a metaphor for many leadership challenges in that they will be frequent, difficult, and time consuming, but in the end they will let us all grow.  ….my reflections are not simply babble, …they actually represent significant growth and understanding of myself.”

References:

Svinicki, Marilla. 2004. Learning and Motivation in the Postsecondary Classroom. Bolton, MA: Anker Publishing Co, Inc.

The Learning Record: http://www.learningrecord.org/

————————————-

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.

 


Calls for Research Collaboration

If you have a planned, new, or ongoing project for which you’d like collaborators from other institutions, submit a short description and explain the type of collaboration you are requesting. For example, you might need statistical assistance, you might want someone to try your approach using a different student population, etc.

Use the comment feature to submit your post.


Request for Assistance in Project Design

If you are interested in designing a research project that investigates the impact of your incorporation of metacognitive strategies in your classroom, use the comment feature to submit a short description of your idea and questions. The Improve with Metacognition creators and consultants will try to assist you, and other site visitors might also comment on your post. Note that it is your responsibility on your end to apply for and receive IRB approval for the ethical use of human participants.


What is your favorite metacognition assignment?

Briefly explain the assignment. Help others understand how you’ve used it.  For example, in what level course have you used this?  What is the best thing about this assignment? What is a limitation of this assignment?

Share by using the comments feature for this post. If you have a handout to share with more detail, please email it with a short note linking it to your post, and the Improve with Metacognition creators will save it as an online resource and link it to your post. If you are comfortable doing so, you can include  your name and email contact so that others can contact you for further information.


What is your favorite in-class metacognition activity?

Briefly explain the activity. Help others understand how you’ve used it.  For example, have you used this in small or large classes? What is the best thing about this activity? What is a limitation?

Share by using the comments feature for this post. If you have a handout to share with more detail, please email it with a short note linking it to your post, and the Improve with Metacognition creators will save it as an online resource and link it to your post. If you are comfortable doing so, you can include  your name and email contact so that others can contact you for further information.