Changing how we teach genetics through learning goals, assessments, and interactive learning Michelle Smith University of Colorado Boulder Organization of the MCDB Science Education Initiative Jia Shi Michelle Smith STFs Faculty TAs LAs SEI Coordinators Jenny Bill Knight Wood What are learning goals? Statements that focus on the outcomes we expect of students when they complete the course Based on Bloom’s Taxonomy Students should be able to: Create: combine ideas to create something new. Evaluate: justify or defend a conceptual point of view. Analyze: compare and distinguish between related concepts. Apply: use learned information in a new situation. Understand: explain ideas or concepts. Remember: recall and restate learned information. Modified version of Bloom’s taxonomy: http://www.odu.edu/educ/llschult/blooms_taxonomy.htm Example of a genetics learning goal Syllabus topic: Pedigree Analysis Example of a course learning goal (10 total) After completing this course, students should be able to: Analyze phenotypic data and deduce possible modes of inheritance (e.g. dominant, recessive, autosomal, X-linked, cytoplasmic) from family histories. Sample of topic learning goals Draw a pedigree based on information in a story problem. Calculate the probability that an individual in a pedigree has a particular genotype. Define the terms “incomplete penetrance,” “variable expressivity,” and “sexlimited phenotype,” and explain how these phenomena can complicate pedigree analysis. Process of writing genetics learning goals Make goals departmental, not individual Genetic Instructors Sylvia Fromherz Ken Krauter Draft of learning goals Mark Winey Syllabus topics and classroom observations Michelle Smith Use information from both instructors to write new learning goal drafts Jia Shi Bill Wood Proof read, questioned importance of goals, suggested changes Genetics pre/post assessment Michelle Smith Bill Wood Jenny Knight MCDB faculty MCDB faculty The pre/post assessment is different from other genetics tests Assessment is 25 multiple-choice questions that address the 10 course learning goals. Jargon is used minimally in this assessment. Assessment is given pre and post to measure learning gains. The incorrect answers are designed to be attractive to students who do not fully understand genetics concepts. Observations during homework study sessions Student Interviews Questions validated by interviews with students and faculty members. Students at a variety of achievement levels helped with the development of the assessment A students: verify that students get the right answer for the right reasons B and C students: retain some misunderstandings that are useful as distracters D students: look for non-content clues to the right answer A single DNA nucleotide change of an A to a T occurs and is copied during replication; is this change in DNA sequence necessarily a mutation? a) Yes, it is a change in the DNA sequence. b) Yes, but only if the nucleotide change occurs in a sex cell (sperm or egg). c) Yes, but only if the nucleotide change occurs in the coding part of a gene. d) Yes, but only if the nucleotide change occurs in the coding part of a gene and alters the amino acid sequence of a protein. e) No, because A and T are similar enough, they can substitute for each other. Answer: a Student who earned a D in genetics: “I don’t like to see the word only in answers. Answers with only are never true. There are 4 yes answers and 1 no, so I will go with answer a).” Genetics assessment was given at three quite different institutions this fall 348 genetics students from CU-Boulder (majors and non-majors), Bridgewater College in Virginia, and Georgetown University in Washington, D.C. Prerequisites for students taking genetics are different Grade level of students: Bridgewater Bridgewater CU non-majors C U MC DB 1041 CU majors CU MCDB2150 Georgetown Georgetow n Univers ity Freshman Sophomore Freshman Freshman Sophomore Sophomore Sophomore Junior Junior Junior Senior Senior Senior Freshman Post-Grad Fres hman Sophomore Junior Junior Senior Post-Grad Post-Grad Pos t-Grad Senior Post-grad 1600 1400 1200 1000 800 600 400 200 0 University of Bridgewater Georgetown ColoradoCollege University Boulder Acceptance Rate (percent) Average SAT Score 2007 Class Institution entrance statistics: 100 90 80 70 60 50 40 30 20 10 0 University of ColoradoBoulder Bridgewater College Georgetown University Overall performance on the genetics pre-assessment 100 90 Average Score (%) 80 70 60 n=81 n=129 * CU CU non-majors MCDB1041 Non-majors CU CU majors MCDB2150 Majors 50 40 n=38 n=100 ** 30 Georgetown students have had a intro course that includes a genetics section 20 10 0 Bridgewater Bridgewater Georgetown Georgetown Pairwise comparisons between means were performed with a Tukey post-hoc test (significance level set at p<0.05). Example of wide-spread student conceptual problems Learning goal: Compare different types of mutations and describe how each can affect genes, mRNA, and proteins. 100 Bridgewater CU 1041 non-majors CU 2150 majors Georgetown Average Score (%) 90 80 70 60 50 * 40 30 * 20 10 0 Definition of a mutation Nonsense mutations and transcription Question Description Effects of frameshift mutations Pairwise comparisons between means were performed with a Tukey post-hoc test (significance level set at p<0.05). Most common conceptual problems on these topics Many students think that: 1). A DNA nucleotide change is defined as a mutation only if the nucleotide change occurs in the coding part of a gene and/or alters the amino acid sequence of a protein. 2). A stop codon stops transcription. 3). The insertion of a nucleotide into the coding portion of a gene cannot result in a shorter protein. Conclusions from the genetics assessment development process We have developed a genetics assessment where the wrong answers are attractive to students who do not fully understand genetics concepts. We have revealed several common student misunderstandings at all three institutions. Next we will… Bridgewater CU 1041 CU 2150 Georgetown 100 Address problems in the assessment. 90 Average Score (%) 80 70 60 50 40 30 20 10 0 Definitive evidence a gene of interest was identified Question Description Continue to work with genetics instructors at multiple institutions to verify that our assessment tool is a widely useful and reliable instrument. Genetics assessment will be used to gauge student learning and monitor curriculum change Compare scores on the pre and post assessment to measure learning gains Genetics student scores on an earlier 40 version of the assessment 35 Pre-assessment Percentage of Students Post-assessment 30 25 20 15 10 5 0 0-9% 1019% 2029% 3039% 4049% 5059% 6069% 7079% Score on Assessment Design tools to improve student conceptual learning 8089% 9099% 100% Biology Colorado Learning Attitudes about Science Survey (CLASS) Michelle Kate Smith Semsar Physiology STF Differences between novice and expert learners concerning their beliefs about science Novice Isolated pieces of information Expert content and structure Coherent framework of concepts Handed down by authority No connection to the real world source Describes nature Established by experiments Pattern matching to memorized recipes problem solving Use concept-based strategies. Widely applicable. (adapted from David Hammer,2000). Biology novices and experts Over 2,000 students took the survey this fall • General biology (Ecology and Evolutionary Biology) • Introduction to molecular and cellular biology (MCDB) • Genetics majors and non-majors (MCDB) • Anatomy (Physiology) 80 Ph.D. experts have taken the same survey Subdisciplines of Experts Molecular Physiology Ecology Other Molecular Physiology Ecology Other Biology CLASS statements designed to distinguish novice and expert beliefs Likert scale • Statements are based on the physics CLASS (Adams et al., 2004) • Student interviews on statements were conducted for clarity of interpretation (n=15) • Experts have 80% or greater agreement on 34 of 44 statements • Student responses are compared with experts Students tend to shift from expert to novice beliefs in science courses!! Statements are classified into categories (e.g.: personal interest, real world connections, problem solving) Work in physics, chemistry, and geology has shown shifts towards novice thinking in introductory science courses (Adams et al., 2006, Perkins et al., 2007, Unpublished data from: Langdon, Stempien and Bair) Preliminary evidence shows shifts towards novice thinking in General Biology (Ecology and Evolutionary Biology) Largest shifts towards novice thinking: It is important for the government to approve new scientific ideas before they can be widely accepted. Mathematical skills are important for understanding biology. I do not spend more than a few minutes stuck on a biology question before giving up or seeking help from someone else. Largest shift towards expert thinking: I think about the biology I experience in everyday life. Future questions to be addressed by the Biology CLASS •Is expert-thinking the same across biology subdisciplines? • Does thinking differ between academic and medical experts (university researchers & MDs)? In collaboration with Pawel Kindler at UBC • Is student-thinking the same across subdisciplines or among populations with different career goals? •Does student-thinking differ between introductory and upper division levels? •Do we select for expert-like thinkers or develop expert-like thinkers? Are interactive lectures or group tutorials better for learning genetics? Michelle Smith Ken Krauter Jenny Knight MCDB faculty MCDB faculty Experimental Design Monday and Wednesday: attend lectures in a traditional lecture hall and use clickers (~3 questions per class) On Fridays 140 students are split two equal-sized groups Interactive lecture Tutorial activities Content is the same in both sections Half way through the semester the groups switch treatments ~8.5 clicker questions and ~1.5 general questions posed to the class Facilitated by LAs, TAs and instructors Student performance is equivalent in both groups Monitor learning that day: At the end of each session there is a clicker quiz 70 Average Score (%) 60 * 50 40 30 20 10 0 Section 1: Lecture Section 1: Lecture Section 2: Group Tutorials Section 2: Group tutorials Section 1: Group tutorials Section 1: Group Tutorials Section 2: Lecture Section 2: Lecture No significant differences: Homework grades and Exam scores p<0.05 Students find the lectures more useful How useful are the Friday lectures/ group activities in helping you learn the course material? 50 Section 1: Lecture 45 Section 2:Lecture Group Tutorials Section 1: Section 2: Group Work 40 35 Percent 30 25 20 15 10 5 0 Never useful Not useful the majority of the time Useful Useful the majority of the time Always useful Significant difference between groups p<0.05, 2=26.18 Students confidence about learning the material similar in both groups 60 Section 1: Lecture 50 Section 2: Group Tutorials 40 Percentage On Fridays when I walk out of class, I am confident that I understand the material 30 20 10 0 Strongly disagree Disagree Neutral Agree Strongly agree Strongly disagree Disagree Neutral Agree Strongly agree 60 50 40 Percentage When I sat down to take the exam, I was confident that I understood the Friday material 30 20 10 0 Future directions for the interactive learning experiment • Determine if there are differences in retention between the two groups Final Exam Compare scores on questions that address topics covered in the first or Next semester students will be asked to answer genetics second half of the semester questions on line • Measure innovation in problem solving Future Directions for the Genetics Course Every faculty member teaching genetics will receive: • Learning goals • Validated content and attitude assessments tools • Information on common student misunderstandings • Activities, clicker questions, homework assignments aimed at maximizing learning and retention Many thanks to…. SEI at CU-Boulder Bio-CLASS Kathy Perkins Carl Wieman Science Teaching Fellows Mindy Gratny Angela Jardine Kate Semsar Georgetown University MCDB SEI Coordinators Jenny Knight Bill Wood MCDB faculty participating in the SEI Corrie Detweiler Christy Fillman Nancy Guild Michael Klymkowsky Ken Krauter Jennifer Martin Joy Power Jia Shi Ravinder Singh Quentin Vicens Mark Winey Ronda Rolfes Bridgewater College Robyn Puffenbarger Undergraduate Learning Assistants Jason Barr Amy Doubet Becca Green Jolene Hammond Tyler Long Lauren Snella Jill Terry THE GENETICS STUDENTS!!!! Example of wide-spread student conceptual problems Average Score (%) Learning goal: Analyze phenotypic data and deduce modes of inheritance from family histories. 100 90 80 70 60 50 40 30 20 10 0 Bridgewater CU 1041 non-majors CU 2150 majors Georgetown * Inherited Pedigree Mitochondrial diseases in analysis of an X- DNA inheritance women and the linked dominant patterns X chromosome inheritance pattern Question Description Pairwise comparisons between means were performed with a Tukey post-hoc test (significance level set at p<0.05). Most common conceptual problems on this learning goal Many students think that: 1). An inherited disease that primarily affects women and not men is likely to be caused by a mutation on the X chromosome. 2). X-linked dominant inheritance patterns cannot be distinguished from autosomal recessive and X-linked recessive inheritance patterns. 3). Mitochondrial DNA is inherited in the same way as nuclear DNA. 4). Women pass on mitochondrial DNA only to women.