Preparing Preservice Teachers Using a Civic Engagement Model: The Effect of Field Experience on Preservice Teacher

Abstract

Participating in a civic engagement partnership, Towson University preservice teachers deliver educational programming at the National Aquarium to students from local schools, focusing on Chesapeake Bay water quality and human impact.  Teaching Environmental Awareness in Baltimore (TEAB) is designed to engage students (both preservice teachers and K–12) in environmental issue investigations relevant to the local community and promote deep, critical thinking.  From a civic and socio-scientific viewpoint, our project has the following aims: (1) to focus on urban youth who may have limited personal experience with nature and/or have a limited understanding of local natural resources, (2)to assist preservice teachers in becoming confident, competent environmental educators through practical, hands-on professional development, (3) to enact a place-based environmental curriculum that meets both the instructional guidelines of local school districts and State content standards.

Introduction

A national movement, sparked by Richard Louv’s (2005) treatise Last Child in the Woods, has catalyzed collaborations among government agencies, schools, and nonprofit and community organizations, with the goal of reconnecting children with the environment. The positive impacts of spending time in nature on a child’s physical, cognitive, and social development have been well established in the literature (James, Banay, Hart, & Laden, 2015; Thompson Coon et. al., 2011; Rook, 2013). These impacts are especially crucial due the lack of public understanding in the United States of the importance and benefits of nature and the ecosystem services it provides (Duvall & Zint, 2007; Turnpenney, Russel, & Jordan, 2014).  

The State of Maryland contains rich and varied natural resources that provide both tangible and aesthetic value to its residents. These natural resources provide critical ecosystem services that maintain clean air and water and provide productive land to support its residents. Despite its aesthetic and economic value, Maryland’s natural resources face a multitude of long-term environmental threats. For instance, the Chesapeake Bay has been the focus of ongoing restoration efforts for more than two decades; yet, in recent years , the University of Maryland Center for Environmental Science assigned the Bay a D+ in overall health, based on six ecological indicators (University of Maryland Center for Environmental Science, 2018). Nutrient pollution from agriculture continues to be a problem in freshwater streams and rivers. Land development, especially along the shores of the Bay, continues at a rapid pace, and this land development threatens the water’s-edge ecosystems along the shores. Baltimore joins other post-industrial legacy cities in an uphill battle to modernize aging infrastructure and rehabilitate local waters stressed by generations of manufacturing outflow and inadequate regulation. Even as the industry of the Inner Harbor has been replaced by a revitalized waterfront and service economy, water quality continues to suffer as storm run-off and sewage overflows raise bacteria, nutrients, and debris levels well above of healthy levels.  Air quality, especially in central Maryland, ranks among the worst in the nation (Goldberg et. al., 2014). Critically evaluating local environmental problems and developing solutions is difficult and requires fundamental understanding of the interconnectedness of ecological systems and human impacts on them. The conservation, restoration, and long-term sustainability of Maryland’s natural resources are dependent on future generations of citizens who can serve as environmentally literate stewards of the state’s natural resources and can make informed decisions that will affect their families and their communities. 

Environmental education rooted in local, place-based issues is one way to ensure that our youth have the knowledge and skills necessary to address these complex socio-scientific issues as adults (Klosterman & Sadler, 2010). Furthermore, environmental literacy is a component of overall scientific literacy (Blumenstein & Saylan, 2011) and requires the same skills as other STEM fields (Jordan, Singer, Vaughan, & Berkowitz, 2009).  With the goal to create a more environmentally literate citizenry, the following initiatives have been implemented in Maryland K–12 schools over the past six years:

  • Environmental literacy standards for K–12 students were adopted. 
  • The state began requiring that all students enrolled in public schools are to engage in a “meaningful watershed educational experience” at least once at the elementary, middle school, and high school levels (Chesapeake Bay Watershed Agreement, 2020).
  • Beginning with the freshman class of 2013, all high school seniors must satisfy an environmental literacy graduation requirement (Maryland State Department of Education, 2019).  To date, Maryland is the only state to mandate this requirement, although several other states have adopted and implemented environmental literacy standards.  

These changes in K–12 education in Maryland Public Schools have created the need for school systems and institutions of higher education to reevaluate how they deliver instruction for both K–12 students and the preservice teachers who will eventually be teaching them.  School districts need support from outside partners to provide appropriate and meaningful watershed educational experiences for all students. Additionally, there is a pressing need to provide appropriate training to preservice and inservice teachers; they must have the content knowledge and pedagogical expertise to ensure their ability to plan instruction that will align with the new environmental literacy standards and meet the requirements for the Meaningful Watershed Educational Experience (MWEE). This will enable our students to eventually meet the environmental literacy graduation requirement.  

We aimed to address these needs by forming a partnership between an institution of higher education (Towson University) and an informal educational institution (National Aquarium).  In this partnership, Towson University preservice teachers deliver educational programming focusing on Chesapeake Bay water quality and human impact to students from local schools. Teaching Environmental Awareness in Baltimore (TEAB) is designed to engage students (both preservice teachers and K–12) in environmental issue investigations relevant to the local community and to promote deep, critical thinking. From a civic and socio-scientific viewpoint, our project has the following aims:

  1. To focus on urban youth who may have limited personal experience with nature and/or have a limited understanding of local natural resources, 
  2. To assist preservice teachers in becoming confident, competent environmental educators through practical, hands-on professional development, 
  3. To enact a place-based environmental curriculum that meets both the instructional guidelines of local school districts and State content standards.

We are also aiming to address the following more overarching civic issues through our project activities:

  • The infrequency of contact between children and nature and their lack of appreciation and awareness of the local environment,
  • A disproportionate lack of exposure to nature for at-risk urban youth,
  • The need for well-trained teachers who can provide experiential education opportunities that foster children’s affinity for nature and a stewardship ethic that is supported by knowledge.
  • Although our project involves several entities, and our goals stated above address more than one audience, the data presented here focus mainly on the effect of the project on preservice teachers.  In particular, we wanted to answer the following questions:
  • Can integrating non-formal educational field experiences that focus on local environmental issues into teacher preparation programs promote  enhanced  preservice teacher content and pedagogical knowledge, as perceived by preservice teachers?

Can integrating non-formal educational field experiences that focus on local environmental issues into teacher preparation programs promote more positive attitudes towards teaching environmental education, and perhaps toward the environment itself?

  The specific objectives of this study are as follows:

  • Preservice teachers will report deepened understanding of how environmental factors affect aquatic life in the Chesapeake Bay. 
  • Preservice teachers will feel confident teaching environmental education topics in non-formal settings. 
  • Preservice teachers will demonstrate increased personal interest in environmental issues affecting their local community. 
  • Preservice teachers will report strengthened pedagogical content knowledge in delivering science lessons.

Program Partners

The pilot semester of our project was financially supported by a SENCER-ISE grant awarded to Towson University and the National Aquarium. 

Since its opening in 1981, the National Aquarium has been a gem in the very heart of Baltimore’s Inner Harbor, and generations of Maryland families have walked through its doors and shared in the wonders of the undersea world. Its mission, to inspire the conservation of the world’s aquatic treasures, has motivated thousands of Marylanders to appreciate and protect the delicate habitats in their own backyards. The Aquarium educates more than 150,000 Maryland schoolchildren a year, both at the Aquarium and in the classroom. The Aquarium’s conservation and education programs, coupled with the many affordable-access programs offered to Maryland residents, ensure that nearly 400,000 Marylanders are able to visit the Aquarium each year. Urban conservation is a major theme in the Aquarium’s new Conservation Plan. Under this plan, the Aquarium is working to provide urban residents with the tools and skills to make changes in their communities. Because we are a coastal city, Baltimore’s urban communities are becoming increasingly impacted by environmental challenges.  To combat these challenges, an educated citizenry is necessary.

Towson University is recognized as Maryland’s preeminent teacher education institution and as a national model for professional educator preparation. The Fisher College of Science and Mathematics (FCSM) at Towson University has a distinguished history in the preparation of STEM classroom teachers and STEM education specialists. The Fisher College prepares STEM preservice teachers to become facilitators of active and inclusive learning for diverse populations of students. FCSM faculty, who comprise a diverse community of teacher-scholars, have a wide range of strengths and specialties. Academic programs require teacher candidates to demonstrate professional knowledge, skills, and dispositions that place students at the center of active learning and emphasize higher order thinking. Through innovative educational partnerships, TU’s certification programs provide teacher candidates with progressively responsible field and/or clinical experiences in a variety of settings. These rich experiences are designed to enable teacher candidates to merge theory with classroom practice and to develop and refine their knowledge of and skills in STEM teaching and learning.  

At the Aquarium, preservice teachers are able to directly apply their learning from postsecondary coursework in a practical setting. As a result, they gain valuable career experience while making a significant contribution to the local community and its children. By serving as educational interns, the preservice teachers serve the needs of the local community by fostering environmental awareness among urban youth. 

Methods

Research Design: Participants 

Subjects in this study were elementary education preservice teachers at Towson University who were enrolled in one section of SCIE 376: Teaching Science in the Elementary School. Maximum enrollment in these sections is 18. Typically, students are 19–23 years old, and most are female. There were 16 students enrolled in the Fall 2017 pilot semester and 13 students enrolled in the Fall 2018 semester. The study utilized convenience sampling; thus, any preservice teacher enrolled in the course could participate but was not required to. Students were recruited regardless of age, sex, or ethnicity. The research design and participant recruitment methods were approved by the university institutional review board.  

Research Design: Location

All activities were conducted at the National Aquarium in Baltimore, Maryland. The location of the National Aquarium was well suited for our purposes for two reasons.  First, the Aquarium is located on a major tributary of the Chesapeake Bay, making it a perfect venue for investigating the socio-scientific issues surrounding water quality and watersheds.  Second, the Aquarium is located in the same community where our target school-age population lives, allowing us to emphasize place-based educational strategies.   

Research Design: Task/Preservice Teacher Content 

The field study component that is required of a MWEE is often difficult for Baltimore City Schools to implement due to a lack of safe study sites within the local area. The National Aquarium is a logical partner for them, as it is located in the same neighborhood as the schools and students we are aiming to reach, and there are many accessible study sites on the aquarium property where students can safely access the water and examine human impact.  The “What Lives in the Harbor” program is designed to meet the Chesapeake Bay Agreement requirements for an MWEE and is aligned with the Baltimore City Public Schools sixth-grade curriculum. MWEEs are learner-centered experiences that focus on investigations into local environmental issues that lead to informed action and civic engagement. Educators play an important role in presenting unbiased information and assisting students with their research and exploration.  In our case, the field experiences take place at the National Aquarium, entirely outdoors.  Students begin their visit to the Aquarium’s waterfront campus with a brief discussion about their local Baltimore Harbor watershed and its place within the larger Chesapeake Bay. Students then rotate through three stations where they take water quality readings. At the request of City Schools the Aquarium uses Vernier equipment, which is the same equipment used in high schools. Each station is led by two preservice teachers and lasts approximately 25 minutes. At each station, students collect quantitative data that will help them determine which organisms on their organisms cards would be able to survive in the harbor, based on the data they have collected. All data are recorded on paper data sheets, and also on portable electronic devices, which save the data for reference later; the data are also sent to the classroom teacher for later use in synthesis and conclusion activities that take place in the classroom.  A brief description of each station appears below.

Plankton & Turbidity: Turbidity is defined and the consequences of low or high turbidity are discussed.  Human impact on turbidity is emphasized as well as the impacts of high turbidity, such as decrease in the amount of light available for photosynthesis and increased water temperature. Turbidity is measured with a Secchi disc. Students assess phytoplankton living in the harbor using handheld microscopes and observe water color to determine the species of phytoplankton present. The observation and discussion of plankton in the water emphasizes the key role that plankton play as a primary food source for the harbor’s food web. 

Dissolved Oxygen & Salinity: Dissolved oxygen and salinity are measured with Vernier probes.  Dissolved oxygen and salinity readings are taken both at the surface and closer to the harbor bottom. Human impact on these parameters is discussed, as well as what the measurements mean for the organisms living in the watershed.  Emphasis is placed on the impact that low dissolved oxygen levels have on the ability of aquatic organisms to survive in certain water systems and the impact of salinity changes as a stressor for marine ecosystems.  

Temperature and pH: Temperature is measured with a digital thermometer and pH is measured using pH strips.  Common household items (bleach, milk, orange juice) are used to relate the pH scale to the students more effectively.  Emphasis is placed on the influence of temperature and pH on the chemical and biological reactions in marine ecosystems.

After completing all of the stations, students analyze the data they have collected to determine which organisms would be able to live in the Baltimore harbor, and are asked to support their conclusions with evidence from the data.  To test their hypotheses, students survey and catalog what they find in bio-hut cages suspended off the Aquarium piers using the iNaturalist app on an iPad. The bio-hut is a double cage system where one side is filled with oyster shells that attract rapid colonization by microorganisms. The oysters are seeded with spat (juvenile oysters) that grow and serve as biological filters by filter feeding and removing algae from harbor water.  Mussels and barnacles that attach themselves to and grow on the oyster shells act as living filters in these urban waters. The outer cage is empty and provides only shelter, offering a predator-free zone for juvenile native fish. The double cage system of the bio-huts restores some of the ecological function once provided by the wetlands historically found in the area.
The group discusses whether their predictions were correct and why or why not. They also discuss what water quality parameters seem to be the most important to biodiversity.  Finally, preservice teachers have the students take inventory and count the living spat (oyster larva) on the oyster shells inside the bio-hut cages. These data are provided to the Aquarium’s Field Conservation Department and contribute to one of the Aquarium’s broad conservation goals. At the end of each school year, these spat will be added to the Aquarium’s recently created oyster reef, which provides a unique habitat to the urban wildlife of the Baltimore Harbor. This onsite action project will help inspire students to plan their own action projects, as they learn about how the Aquarium’s oyster reef, floating wetlands, and bio-huts are creating natural ecosystems that support the diverse life in the harbor.  Following their field experience, students complete an action project at their schools. During the pilot, students identified one water quality parameter that is negatively affecting organisms in the harbor and then worked in groups to brainstorm issues in their neighborhood that could impact water quality and aquatic species in the harbor. Students selected one issue and suggested an action they or others in their neighborhood could take to positively change these conditions. From this exercise, pilot schools conducted several different action projects, such as discussing and designing a small garden on the school’s property in the following school year; creating posters to promote improving water quality and reducing waste; writing letters to the principal and elected officials about the importance of the bay; and pledging to reduce, reuse, and recycle 10% more over summer break.

Data Collection

Survey Instrument: STEBI (Science Teacher Efficacy Belief Instrument)

The identification of various methods that can help to develop self-efficacy is becoming an increasingly important aspect of science education research and the professional development of teachers (Ginns, 1996).  The STEBI was used to measure science teaching self-efficacy and outcome expectancy in preservice teachers. Since our subjects are preservice teachers, we used the STEBI-B, which is designed for this audience (Riggs & Enochs, 1990).  The STEBI-B was chosen as an instrument in this study because it has been commonly used in science education research studies and because studies have found the survey instrument to have high validity and reliability (Bleicher, 2004; Bleicher & Lindgren, 2005; Settlage, 2000; Schoon & Boone, 1998).  The STEBI-B consists of 23 Likert scale response items and is broken up into two subscales, personal science teaching efficacy (PSTE) and science teaching outcome expectancy (STOE). The subscale personal science teaching efficacy measures the participant’s belief in the ability to teach the subject of science effectively (Deeham, Danaia, & McKinnon, 2017). Deeham et al. also describe the outcome expectancy subscale as a measure of the participants’ broad views of science teaching related to why students perform as they do. The items for the two subscales are randomly placed throughout the survey. A paired t-test was used to determine any significant difference in the pre and post survey answers.

Survey Instrument: Environmental Education
Attitudes Assessment

To assess preservice teacher attitudes and beliefs toward teaching science, specifically environmental education, an analogy was administered pre/post. Participants were asked to complete the analogy, “Teaching environmental education is like _____.” They were then asked to accompany their answer with a drawing that illustrated their thoughts. The analogies that the preservice students create and explain helps to capture their attitudes towards teaching, thereby giving us insight into their teaching self-efficacy (Hanson, 2018).  Data collected were coded based on the categories described in Table 1 .

After coding the data from the science teaching analogy, the analogy results were linked to the STEBI scores, to give insight into the preservice teachers’ teaching self-efficacy and their attitudes towards environmental education.

Survey Instrument: SALG (Student Assessment of Learning Goals)

The SENCER SALG was administered pre/post and was used as an evaluation tool to gather learning-focused feedback from students. The SALG has students assess and report on their own learning and on the degree to which certain aspects of the course have contributed to that learning. The SALG instrument may be one of many assessment practices that can assist in gathering feedback for both teaching and learning assessment (Scholl & Olsen, 2014).  

Weekly Reflections

Weekly reflections serve as an outlet for students to self-report their current attitudes towards environmental education and their assessment of their teaching. Included with each open reflection assignment is a required question for students to answer: What is your current attitude towards teaching environmental education? Have there been any changes since last week? Any positive/negative experiences?

Table 1: Coding Categories for EnvironmentalEducation Attitudes Assessment

Students completed six weekly reflections throughout the semester, and these weekly reflections were analyzed through open coding techniques using NVivo software. Interrater reliability was established through the use of two different coders to develop codes and observe trends in the data. Three weeks out of the seven were selected using a random numbers calculator, then those weeks were coded separately by both individuals.  From these three weeks, larger codes were developed: Negative Attitude, Positive Attitude, Self-Efficacy, and Classroom Management. The weekly reflections gave insight into the attitudes and self-efficacy of the students through self-reporting information. 

Results

Survey Instrument: STEBI  (Science Teacher
Efficacy Belief Instrument)

Attitude outcomes were measured through pre/post data taken from the STEBI, which was administered to all Towson University students enrolled in the course. Paired t-test results show that the experiences at the Aquarium led to an increase in both science teaching self-efficacy (p=.003) and teaching outcome expectancy (p=.031).  See Figure 1 for individual pre and post STEBI scores.  

Individual questions were analyzed to determine areas of largest growth in self-efficacy. The question showing the largest gains was “I know the steps necessary to teach science effectively”; the average pre assessment score was 36 while the post assessment average score grew 14 points to an average of 50 points. Another survey question that showed large gains was “I wonder if I will have the necessary skills to teach science”; the average pre-assessment score was 34 and the post assessment average was 47. This increase of 13 points suggests that the preservice teachers were not wondering whether they would have the necessary skills to teach science as much as they did before the field experience. These individual STEBI question results are meaningful because they suggest that the preservice teachers were feeling more capable of teaching science effectively after this non-formal educational field experience.

Figure 1: STEBI and Post Assessment Scores
Table 2: Results of Preservice Teacher Coding per Coding Classification

Survey Instrument: EEAA (Environmental Education Attitudes Assessment)

The results of the pre EEAA show that most of the preservice teachers’ attitudes towards teaching environmental education were coded as negative or a struggle (61.5%). After the field experience, we saw a shift in the responses, as only 8% were coded as negative or struggle. Instead of a predominately struggle or negative attitude in the preservice teachers in the pre-EEAA (61.5%), we saw predominately journey and positive attitudes in the post EEAA (69%). The largest area of growth was in the positive category; only one preservice teacher was coded as positive in the pre EEAA, but in the post-EEAA there were five preservice teachers whose responses were coded “positive.”  Samples of each coding description appear in Table 2 above. See Figures 2 and 3 for results by coding category. 

Figure 2: Individual Preservice Teacher EEAA Codings in Pre/Post Test
Figure 3: EEAA Pre to Post Coding

Survey Instrument: STEBI + EEAA

Linking the results

Table 3 illustrates the linkages between each participant’s pre/post STEBI score and pre/post EEAA.  Of the 13 preservice teachers who were administered the STEBI and EEAA, seven subjects (54%) demonstrated growth in both self-efficacy and in attitudes towards environmental education from pre to post. Five students (38%) demonstrated growth in one area but not the other and only one student (8%) demonstrated a decrease in both areas. Overall, there were nine out of 13 students who demonstrated growth in self-efficacy and nine out of 13 students whose attitudes towards environmental education became more positive over the course of the study. 

Weekly reflections

Qualitative data collected through analysis of weekly reflections support the findings presented from the SALG that personal interest in the civic issues being studied did increase among participants. These data show that overall students became more interested in socio-scientific issues and watershed issues in particular as a result of participating in this course.  A few students’ comments that were written in reflections at the conclusion of the course appear below. 

The journey has opened my eyes on topics that are related to and inside of the subject environmental science, and that I am certainly more comfortable handling and teaching the subject than I was prior to this experience.

I learned how to be respectful towards the environment.  It is important to teach this quality to kids at a young age.  

Students also felt that they gained skills that would help them be more effective teachers in the classroom.  It was evident to us through their written lesson planning and through teaching observations that their delivery methods improved over the course of the semester, but students also reported feeling more confident in teaching science content to children.

Seeing how much students were enjoying and engaged in the program, I can only be reassured that environmental education is a powerful and important element to elementary education.

The biggest change I have found is in my confidence level. My self-efficacy for teaching science has increased 100 percent. I feel like I know the content a lot better so I can teach my students without feeling unsure of the topics. 

As a teacher of science, I am growing more confident in this content and I hope to apply this knowledge to my future work.

Table 3: Individual EEAA and STEBI Pre/Post Linked Results
Figure 4: Attitude Coding for Weekly Reflections
Figure 5: Self-Efficacy vs. Classroom Management Codes by Week

The NVivo coded data reveal many fluctuations in preservice teacher attitudes throughout the study. In the final week, there were fewer than three negative attitude codes and more than 28 occurrences of positive attitude codes. In general, positive codes tended to increase as the study progressed, and negative codes decreased after a spike in Week 3.  Even though changing weekly factors at the field site, which will be noted in the Discussion section, seemed to affect preservice teacher attitude, overall there were more occurrences of positive attitudes in the last half of the field experience than in the first half (see Figure 4).  

Some student responses from midway through the course that displayed these positive attitudes appear below.

I believe that my attitude is more positive now because I feel like I am learning a lot about the science content, as well as flexibility, time management, and patience, which are essential teaching skills.

My attitude towards environmental education is at a semester-high as of right now. I have always seen the value in developing a sense of environmental awareness and responsibility in the students. It is definitely fun to work with students who come into our stations with open minds and positive attitudes. It is interesting to hear about what they know, and how they connect/relate that to the information at each station.

Along with attitudes, we analyzed weekly reflections for changes in self-efficacy and classroom management concerns/areas of improvement. Classroom management concerns and areas of improvement codes decreased from 38 in Week 1 to 23 occurrences in Week 6 (see Figure 5). Self-efficacy codes were more variable. The reflections for Weeks 4, 5, and 6 contained more self-efficacy codes than Weeks 1, 2, and 3.  Possible reasons for these variations are discussed below.  

Survey Instrument: SALG

Personal interest data through SALG

The SALG data show that students’ personal interest in civic issues increased over the course of the study. Additionally, students became more interested in watershed issues and tended to regard environmental education as more important in the post test.  

A few student comments taken from the post SALG survey appear below.

At the beginning of the semester, I had no idea what factors could affect water quality. Now, because of this internship, I know much more about turbidity, salinity, watersheds, conservation, etc. that I can take with me in my future.

I have gained many skills to help me teach science. I am much more interactive and believe science should be taught through experience after taking this class.

The content within environmental education is definitely something I will carry with me into my other classes, especially other science courses because it is super relevant. It is something I also hope to promote within my personal life among family and friends.

Figure 6: Self-Efficacy in Environmental Education: SALG Data

The quantitative data support these qualitative comments.  For example, pre assessment data show that only 25% of students scored themselves a four or five on the Likert scale for understanding the concept of a watershed, but in the post test, this increased to 81% of students.  When asked whether they understood the impact of human activities on water quality, 56% of students rated themselves as 4 or 5 on the pre test, while this increased to 81% on the post test.

The post SALG data reveal that student self-efficacy in teaching the subject of science and environmental education also increased. Students mentioned different aspects of growth; for example, they reported that their feelings of confidence and self-efficacy had increased and that they had overcome their fear of teaching science (see Figure 6).  A few student comments taken from reflections at the conclusion of the course appear below.

My confidence gained by this class will be taken with me throughout the rest of my teaching career.

The biggest change I have found is in my confidence level. My self-efficacy for teaching science has increased 100 percent. I feel like I know the content a lot better so I can teach my students without feeling unsure of the topics.

I was afraid of teaching science prior to this experience, but I have since gained confidence.

The SALG data indicate that the largest growth areas in socio-scientific issues were in development of knowledge of the watershed and how human activities can affect water quality. These areas grew by over 20%, showing that these students have developed a deeper understanding and connection with the environment and how they as individual community members impact that environment. Confidence about understanding of environmental education, self-efficacy in being able to teach environmental education, and the ability to develop lesson plans in this area were individual questions that reflected growth from pre to post. See Table 4 for a summary of responses pre/post.   

Table 4: Sample SALG Questions Demonstrating the Largest Gains

Discussion

Impacts on Preservice Teachers

The STEBI data demonstrate an increase in self-efficacy in the preservice teachers at the end of the non-formal education experience. The item of largest growth on the survey was “knowing the steps necessary to teach science effectively,” showing us that the preservice teachers have greater confidence in their ability to teach science effectively after the non-formal education experience. Raising self-efficacy levels in preservice teachers is essential; research has found that individuals who have a low sense of efficacy for accomplishing a certain task may avoid it (Schunk, 1991). Having high self-efficacy will help to ensure that the preservice teachers do not avoid teaching of environmental education, but instead feel confident enough in their abilities to be effective, capable, and enthusiastic environmental educators in their future classrooms. 

The EEAA data enable us to observe a shift in attitudes in our subjects as the study progressed. These enhanced attitudes towards environmental education have an impact on their effectiveness as teachers (Ozdemir, Aydin, & Akar-Vural, 2009). If teachers do not have positive attitudes toward the topic of environmental education, then little instruction in this area will be given in the classroom (Ham, 2010). Thus, the impact of this educational experience on the promotion of positive attitudes towards environmental education in preservice teachers is meaningful for the implementation of effective EE. 

Our data suggest that the more confident and competent these students felt in teaching environmental education, the more positive their attitudes became.  Again, promoting both these factors is important, because when teachers perceive their ability to perform the process of teaching science to be low, their resulting dislike of teaching the subject of science translates into the avoidance of teaching science (Koballa & Crawley, 1985).   

The weekly preservice teacher reflections revealed many fluctuations from positive to negative and vice versa in preservice teacher attitudes throughout the six weeks of the study. One factor that influenced these fluctuations was the school group who visited the Aquarium each week. The university students taught a different set of students from a different school each week; therefore each group of City school students was unique in level of preparedness for the trip and in background content knowledge pertaining to the trip. If the school group attending the program was well prepared and ready to participate, the preservice teachers tended to have more positive attitudes.  If the school group was less prepared—for example, if the students did not seem to have much prior knowledge on the purpose of the program and the science behind it—then the preservice teachers tended to have more negative attitudes.  Weather was another factor that affected the preservice teachers’ assessment of how well a given day went. (The educational experience is based outdoors, and the weather naturally varied from week to week.) We are able to relate these factors to certain spikes and dips in attitudes and self-efficacy throughout the six weeks. In Week 3 we saw the most notable affect from these factors: there was a dip in positive attitude and a spike in negative attitudes, which we attribute to the weather. That week it was cold and rainy, and preservice teachers and Baltimore City students therefore complained about the weather throughout the outdoor experience. This factor affected the timing of the activities, since the schedule was adjusted because of the weather; it also affected the data collection, because student data collection papers were getting wet, and had a negative effect on student behavior, as complaints ran high. The day was definitely a challenge for the preservice teachers. We consider these conditions responsible for a dip in positive attitude by five code occurrences and a spike in negative attitude by 11 codes compared to the week before. 

An opposite trend was observed in Week 5. During the programming for this week, we had several politicians and local dignitaries from Baltimore and the surrounding area observing the “What Lives in the Harbor” program.  There was a news media presence there as well, and some of our students were interviewed. Many of the politicians spoke of the “good things” the Aquarium and the preservice teachers were doing. They also mentioned the positive impact the program was having on the community. Coding for Week 5 revealed the lowest level of codes for negative attitudes towards environmental education, with zero instances of negative attitudes.  It appears that the university students were feeling as if they were making an impact and doing something important for their local community. It also created an increase in positive attitude codes.  The students seemed to be affected by this experience and the positive feedback they received from persons not directly associated with the project. Examples from Week 5 student reflections follow.

My attitude toward environmental education has remained positive throughout this week. It was nice to have our efforts at the aquarium validated through the speakers during the press day. I was also interested to learn that this project is important not only on a state level but on a national level.

This also benefits me as a teacher of environmental education as I was congratulated on teaching the science well from an outside party’s perspective. To me, this has the same effect as a parent saying I did well because while they may not understand and therefore won’t focus on the teaching aspect of it, they feel that I conveyed the information well and that means a lot to me.

Impacts on Baltimore City Students

The “What Lives in the Harbor?” program not only has an impact on the preservice teachers, but it is hoped that the program will also positively impact the Baltimore City school students. While the Baltimore City Schools students were not the focus of this study, the school system has stated that the goal of the program is to reach 3,600 students annually by the year 2021 and increase their (1) knowledge of watershed concepts, (2) positive attitudes towards watersheds, (3) inquiry and stewardship skills, and (4) aspirations to protect watersheds. Measurement of progress towards these goals will be conducted by independent program evaluators.  The “What Lives in the Harbor?” program plans to scale up to 67 schools by 2021, systematically adding 16–25 schools per year. As shown in Table 5, the Aquarium will use a tiered approach to serve more schools, teachers, and university interns each year over three years.

Conclusion

We believe it is essential to provide appropriate training to preservice teachers so that they have the content knowledge, self-efficacy, and attitudes to plan and facilitate instruction that will align with the new environmental literacy standards and create more environmentally literate students. We consider our project successful in view of the following accomplishments: 

  • Preservice teachers met the goals we had for the project and had mostly positive things to say about their experience.
  • University students and faculty worked effectively with Aquarium staff to deliver quality watershed education programs to Baltimore City Public Schools students.
  • A positive shift in attitude regarding environmental education was observed in the preservice teachers.
  • Preservice teachers reported a deeper understanding of the environmental issues affecting aquatic life and water quality in the Chesapeake Bay. 
  • Preservice teachers felt more confident teaching environmental education topics in non-formal settings. 
  • Preservice teachers reported strengthened pedagogical content knowledge in delivering science lessons.

From a socio-scientific viewpoint, we believe that Teaching Environmental Awareness in Baltimore (TEAB) did engage students (both preservice teachers and K–12) in environmental issue investigations relevant to the local community and promoted deep, critical thinking. Our initial aims, listed below, were well addressed throughout the project.

Table 5: The Aquarium’s Tiered Approach for Systemic Implementation by 2021
  1. To focus on urban youth who may have limited personal experiences with nature and/or have a limited understanding of local natural resources, 
  2. To assist preservice teachers in becoming confident, competent environmental educators through practical, hands-on professional development, 
  3. To enact a place-based environmental curriculum that meets both the instructional guidelines of local school districts and State content standards.

We were also able to address the following more overarching civic issues through our project activities:

  • Increasing the frequency of contact between children and nature and fostering appreciation and awareness of the local environment,
  • A disproportionate lack of exposure to nature for at-risk urban youth,
  • The need for well-trained teachers who can provide experiential education opportunities that foster children’s affinity for nature and a stewardship ethic that is supported by knowledge.
  • Through the STEBI, EEAA, weekly reflections and the SALG we were able to answer our main research questions:
  1.  Integrating non-formal educational field experiences that focus on local environmental issues into teacher preparation can promote better preservice teacher content and pedagogical knowledge in the majority of preservice teachers. 

This conclusion was supported by self-reported data from preservice teachers through the SALG assessment data as well as through the weekly reflections coding data and STEBI. The preservice teachers reported having a stronger content background and more pedagogical knowledge than they did at the beginning of the field experience. 

2.  Integrating non-formal educational field experiences that focus on local environmental issues into teacher preparation programs can promote more positive attitudes towards teaching environmental education.

This conclusion is supported by the EEAA results and the weekly reflections coding data. 

Due to the increased attention and focus on EE in K–12 schools and the need for effective EE teachers, implementing methods that enhance teaching self-efficacy and attitudes in the field of environmental education at the preservice stage of teaching could be of value to educators, preservice teachers, and the communities that they will eventually serve. We envision future iterations of this partnership that will include evaluating the preservice teachers who deliver EE programming using the same types of evaluation tools we might use in a formal education setting.  For example, lesson planning and delivery could be evaluated using instruments such as the Reformed Teaching Observation Protocol (RTOP) (Sawada et al., 2000) or the Danielson framework (Danielson, 1996).  We are also considering integrating Teacher Performance Assessment (edTPA) rubrics (Ledwell & Oyler, 2016) into the course in order to provide a more robust data set of preservice teacher progress.  Much as an estuary is a transition zone between freshwater habitats and the ocean, teacher preparation is a transition zone for development between preservice and inservice teaching. Having varied experiences flow into this preservice “estuary” can help to increase self-efficacy, create positive attitudes toward teaching, and enhance content knowledge. All of these factors can aid educators in preparing students to become effective future environmental educators.

Authors

Chelsea McClure

Chelsea McClure is a professor of biology and science education at Towson University.  Her research interests lie in the areas of preservice teacher education and environmental education.   

 

 

Sarah Haines

Sarah Haines is a professor of biology and science education at Towson University.  Her research interests lie in the areas of environmental and nonformal education and their effects on student achievement. She integrates the SENCER ideals into most of her courses at TU.

References

Bleicher, R. E. (2004). Revisiting the STEBI-B: Measuring self-efficacy in preservice elementary teachers. School Science and Mathematics, 104, 383–391.

Bleicher, R.E., & Lindgren, J. (2005). Success in science learning and preservice science teaching efficacy. Journal of Science Teacher Education, 16, 205-225.

Blumstein, D. T., & Saylan, C. (2011). The failure of environmental education (and how we can fix it). Berkeley, CA: University of California Press. 

Chesapeake Bay Watershed Agreement 2014. (2020). Retrieved from  www.chespeakebay.net

Danielson, C. (1996). Enhancing professional practice: A framework for teaching. Alexandria, VA: Association for Supervision and Curriculum Development.

Deehan, J., Danaia, L., & McKinnon, D. H. (2017). A longitudinal investigation of the science teaching efficacy beliefs and science experiences of a cohort of preservice elementary teachers. International Journal of Science Education, 39(8,), 2548–2573. doi: 10.1080/09500693.2017.1393706 

Duvall, J., & Zint, M. (2007). A review of research on the effectiveness of environmental education in promoting intergenerational learning. The Journal of Environmental Education, 38, 14–24. doi: 10.3200/JOEE.38.4.14-24

Ginns, I.S., & Watters, J.J. (1996) Science teching self-efficacy of novice elementary school teachers. Paper presented at the annual meeting of the National Association for Research in Science Teaching, St. Louis, MO.

Goldberg, D. L., Loughner, C. P., Tzortziou, M., Stehr, J. W., Pickering, K. E., Marufu, L. T., & Dickerson, R. R. (2014). Higher surface ozone concentrations over the Chesapeake Bay than over the adjacent land: Observations and models from the DISCOVER-AQ and CBODAQ campaigns. Atmospheric Environment, 84, 9–19, doi: https://doi.org/10.1016/j.atmosenv.2013.11.008

Ham, V. (2010). Participant-directed evaluation. Journal of Digital Learning in Teacher Education, 27(1), 22–29.

Hanson, D. (2018). Using analogies to capture personal beliefs of pre-service elementary teachers. Lecture presented at the 2018 International Conference of the Association for Science Teacher Education (ASTE), Baltimore, Maryland.

James, P., Banay, R. F., Hart, J. E., & Laden, F. (2015). A review of the health benefits of greenness. Current Epidemiology Reports, 2(2), 131–142.

Jordan, R., Singer, F., Vaughan, J., & Berkowitz, A. (2009). What should every citizen know about ecology? Frontiers in Ecology and the Environment, 7, 495–500. doi: 10.1890/070113

Klosterman, M. L., & Sadler, T. D. (2010). Multilevel assessment of scientific content knowledge gains associated with socioscientific issues-based instruction. International Journal of Science Education, 32(8), 1017–1043, doi: 10.1080/09500690902894512 

Koballa, T., & Crawley, F.E. (1985). The influence of attitude on science teaching and learning. School Science and Mathematics, 85(3), 222-32.

Ledwell, K., & Oyler, C.  (2016).  Unstandardized responses to a “standardized” test: The edTPA as gatekeeper and curriculum change agent.  Journal of Teacher Education, 67(2), 120–134.  

Lindgren, J., & Bleicher, R.E. (2005), Learning the learning cycle: The differential effect on elementary preservice teachers. School Science and Mathematics, v.105), 61-72. doi:10.1111/j.1949-8594.2005.tb18038.x

Louv, R. (2005). Last child in the woods: Saving our children from nature-deficit disorder. Chapel Hill, NC: Algonquin Books of Chapel Hill.

Lindgren, J., & Bleicher, R. E. (2005). Learning the learning cycle: The differential effect on elementary preservice teachers. School Science and Mathematics, 105(2), 61–72. 

Maryland State Department of Education. (2020). Maryland environmental literacy standards. Retrieved from http://marylandpublicschools.org/programs/Pages/Environmental-Education/LiteracyStandards.aspx

Ozdemir, A., Aydin, N., and Akar-Vural, R. (2009). A scale development study on self-efficacy beliefs through environmental education. Dokuz Eylül Üniversitesi Buca Eğitim Fakültesi Dergisi, 26, 1–8.

Riggs, I. M., & Enochs, L. G. (1990). Toward the development of an elementary teacher’s science teaching efficacy beliefs instrument. Science Education, 74(6), 625–637.

Rook, G. A. (2013). Regulation of the immune system by biodiversity from the natural environment: An ecosystem service essential to health.

Proceedings of the National Academy of Sciences of the USA, 110(46),18360–18367.

Sawada, D., Piburn, M., Turley, J., Falconer, K., Benford, R., Bloom, I., & Judson, E. (2000). Reformed teaching observation protocol (RTOP) training guide. (ACEPT Technical Report No. IN00-2). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.

Scholl, K., & Olsen, H. M. (2014). Measuring student learning outcomes using the SALG instrument. SCHOLE: A Journal of Leisure Studies and Recreation Education, 29(1), 37–50. doi: 10.1080/1937156X.2014.11949710 

Schoon, K., & Boone, W. J. (1998). Self-efficacy and alternative conceptions of science preservice elementary teachers. Science Education, 82(5): 553–568. doi: 10.1002/(SICI)1098-237X(199809)82:5<553::AID-SCE2>3.0.CO;2-8

Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26, 207–231.

Settlage, J. (2000). Understanding the learning cycle: Influences on abilities to embrace the approach by preservice elementary school teachers. Science Education, 84, 43–50. 

Thompson Coon, J., Boddy, K., Stein, K., Whear, R., Barton, J., & Depledge, M. H.

(2011). Does participating in physical activity in outdoor natural environments have a greater effect on physical and mental wellbeing than physical activity indoors? A systematic review. Environmental Science & Technology, 45(5),1761–1772.

Turnpenny, J., Russel, D., & Jordan, A. (2014). The challenge of embedding an ecosystem services approach: Patterns of knowledge utilization in public policy appraisal.

Environment and Planning C: Government Policy, 32 (2), 247–262.

University of Maryland Center for Environmental Science. (2018). 2018 Chesapeake Bay & Watershed report card. Retrieved from https://ecoreportcard.org/report-cards/chesapeake-bay/publications/2018-chesapeake-bay-watershed-report-card/

University of Maryland Center for Environmental Science. (2019). Chesapeake Bay & Watershed report card 2019.   Retrieved from https://ecoreportcard.org/report-cards/chesapeake-bay/publications/2019-chesapeake-bay-watershed-report-card/

Download the article:

Download (PDF, 1.6MB)

Service-Learning Curriculum Increases Climate Change Awareness

Abstract

National efforts to reform undergraduate education have highlighted the need to relate abstract concepts in biology to real-world examples, especially for non-majors who may undervalue scientific processes. We therefore decided to introduce a module titled “Climate Change, Sustainable Practices and Plastic Pollution,” utilizing such high-impact practices as service-learning. This module involved connecting the course objectives with three hours of community service. Our mixed-methods approach across two different course iterations (n=117) indicated that at the end of the course, non-majors were significantly more likely to agree with all the statements on an open-ended pre- and post-survey about civic engagement and sustainable practices, as adapted from Dauer and Forbes (2016). Focus group and free response data confirmed that students valued service-learning and connected the experience to both learning objectives and their everyday lives. We therefore recommend service-learning as an active engagement tool to teach concepts related to global climate change and environmental pollution.

Introduction

A large body of literature suggests that science educators need to adopt active-learning and inquiry-based curricula to enhance student learning and retention (Brame, 2016; American Association for the Advancement of Science [AAAS], 2009; Freeman et al., 2014). While a majority of these reforms are targeted towards science, technology, engineering, and mathematics (STEM) majors, very few studies have explored the impact of innovative curricula and high-impact practices for students majoring beyond the sciences, often referred to as“non-majors.” In fact, non-majors are less likely to have confidence in their ability to perform or understand science, despite the need for an informed scientific citizenry of tomorrow’s voters, workers, consumers, and policy-makers (Dauer & Forbes, 2016; Cotner, Thompson, & Wright, 2017). Non-major classes, which cater to diverse majors and student populations, often seek to connect biology to students’ day-today lives and can do so through student-centered pedagogical approaches (Knight & Smith, 2010). Therefore, it is incumbent upon institutions of higher education to design these student-centric curricula for non-majors that help them recognize the relevance of science to their lives. Service-learning is one such pedagogical innovation  that allows the student to implement knowledge from the classroom (Keupper-Tetzel, 2017) to serve the community and thus represents an example of a model active-learning experience (Lynch, 2016). Broadly defined, service-learning is a set of immersive activities related to concepts in the course material that allows students to relate abstract concepts to concrete examples and gives students transferable and applicable skills related to the material (Dauer & Forbes, 2016; Matthews, Dorfman, & Wu, 2015). Notably, service-learning has improved retention rates not only in biology but also across disciplines (Nigro & Farnsworth, 2009), making service-learning of particular interest in reforming education for non-majors and other diverse student populations. Importantly, in order for the experiences to be of quality, the instructor, with course objectives in mind, must interface with community partners with specific needs. Research has shown that when community partners were highly involved in the process (e.g., by reinforcing learning objectives) students demonstrated greater content learning gains (Little, 2012).

In the scientific community and in the classroom, much attention is being paid to  environmental science, especially in relation to plastic pollution and anthropogenic climate change (Hawkins & Stark, 2016; Schuldt, Konrath, & Schwarz, 2011; Lineman, Do, Kim, & Joo, 2015). Service-learning has a place in this discussion for its ability to show students the relevance of environmental science in their lives and to increase their critical thinking skills (Dauer & Forbes, 2016; Celio, Durlak, & Dymnicki, 2011; Herlihy et al., 2016; Wu, Lu, Zhou, Chen, & Xu, 2016; Harvey, 2018; Yokota et al., 2017; Haward, 2018; Galgani, Pham, & Reisser, 2017). In fact, students may not consider plastic pollution a concern unless they have participated in clean-up efforts, for example through service-learning (Yokota et al. 2017; Haward 2018; Galgani et al., 2017). There also exists a population that does not accept that climate change is occurring even when presented with supporting data. While many efforts seem content to simply inculcate a dogmatic belief in climate change, a superior pedagogical approach is to teach students how to interpret data and draw their own conclusions (Lineman et al., 2015; Schuldt et al., 2011; Dauer & Forbes, 2016). With good reason, many service-learning opportunities for non-majors couple objectives related to the scientific process and data analysis to environmental stewardship (Packer, 2009). Sustainability and environmental science are showing up more and more on course syllabi, and service-learning is a promising strategy to add hands-on stewardship activities to environmental course material.

For these reasons, service-learning was introduced to a non-major biology course at the University of Alabama at Birmingham (UAB), which is an urban, public, research-intensive institution in central Alabama. This study, in line with Vision and Change: A Call to Action (AAAS, 2009), was intentionally done with non-major students whose participation in this class may be the last STEM course of their college curriculum. We tested the hypothesis that a service-learning course module, which included a three-hour service-learning component and data-driven lectures, would affect non-major student attitudes about climate change and topics related to environmental stewardship, including sustainability and plastic pollution.

Methods

Course and Recruitment

This study was approved by the UAB Institutional Review Board IRB-300000955. Ninety-four students were enrolled in BY 101: Topics in Contemporary Biology in the fall semester of 2017 and 89 students in the fall semester of 2018. BY 101 is an elective course for non-majors at UAB that gives a general overview of biology. The course learning objectives, designated on the syllabus (see Supplemental Materials), are as follows:

  1. Understand the basic process of science
  2. Identify the valid sources of scientific literature
  3. Environmental consciousness and civic responsibility
  4. Analyze and apply scientific information to make everyday decisions
  5. Gain a basic understanding of the cell and its functions as it relates to health and wellness
  6. Understand the process of evolution and evidence behind it

The lecture component of the service-learning module included three guest lectures from climate scientists at UAB, including: Dr. James McClintock, Antarctic climate scientist and author of Lost Antarctica: Adventures in a Disappearing Land (St. Martin’s Griffin, 2014); Dr. Jeffrey Morris, who studies the impact of ocean acidification on marine microbial interactions; and Dr. Dustin Kemp, who is a coral reef ecologist. A special lecture titled “Plastic Pollution and Climate Change” was delivered by author Samiksha Raut. All students were required to complete pre-and post-surveys as well as three hours of service-learning. Since this was a high-enrollment class, we decided to limit service-learning to only three hours. Because of the large number of students in this class, service-learning assignments were generally scheduled during the class meeting times to avoid schedule conflicts. By the end of both semesters, only three students (1.6%) had dropped out, while six (3.3%) did not complete any of the required service-learning components. Out of the 174 remaining students, 118 (67.8%) students consented to their data being used in this study and 117 (67.2%) completed both pre- and post-surveys. Their demographic composition is shown in Supplemental Table 1.

Two students partner with UAB Sustainability to do campus litter pickup. Copyright Sarah Adkins.
Students partner with UAB Recycling to sort recyclable materials. Copyright Jon Paolone.

 

 

 

 

 

 

 

 

 

 

 

Speed-Matching Event and Service-Learning 

Early on in the semester, community partners approved and recommended by UAB’s Office of Service-Learning and Undergraduate Research were scheduled to visit the class in a unique “Speed-Matching Event.” All the community partners introduced their organization and their general mission to meet the needs of the community while embracing sustainable practices to combat climate change. This was done with an intent to enable the students to understand the community partner’s goals and how they related to the learning objectives discussed in the classroom. Students committed to a minimum of three hours with their service-learning partner, which, along with the required surveys, constituted 15% of the student’s final grade. To be cognizant of the students’ schedules as well as any transportation issues, all opportunities provided were on campus (UAB Sustainability, Figure 1), within a 10-minute walk of campus (Railroad Park), or had transportation provided (UAB Recycling, Figure 1). Some of the opportunities provided with UAB Sustainability and Railroad Park were scheduled during class time, so that students did not have to take extra time out of their week. This also helped to make these activities inclusive for students with obligations outside of class time. Students who had the physical inability to be outside for extended periods of time had the opportunity to build pamphlets for the Red Mountain State Park. These approaches enabled us to make these assignments inclusive for all our students. After the speed-matching event, a Google form was sent to the students to sign up for a day and time for their service-learning. Students received reminders about their assignments and also about their community partner’s expectations, such as timeliness and dress code. Each of the student groups was overseen by upperclass undergraduate students who had volunteered their time to function as “site leaders.” Their task was to make sure that the students were diligent in completing the assignment. The list of partners and total number of student participants is shown in Table 1. During the service-learning, community partners had students sign in and out to account for attendance.

Table 1: Service-Learning Partners and Number of Participants from 117 students in Both 2017 and 2018 Cohorts

Assessments

Prior to the implementation of the service-learning module, students were given a six-item paper survey adapted from Dauer and Forbes (2016) where students could agree, disagree, or state uncertainty with beliefs about the six statements and follow up by explaining their reasoning for their responses. This was done to ascertain the familiarity of the students with climate change, sustainable practices, and plastic pollution. (For survey forms, see Supplemental Materials.) Three additional items asked the students to reflect on their content knowledge and expectations for service-learning but were not included in this analysis, as they were beyond its scope. We are not aware of any validated existing surveys that cover the breadth of our research question. We therefore decided to adapt our survey from Dauer and Forbes (2016). Most important, their items were in the form of open-ended questionnaires focused on science literacy and decision-making. Completion of both assessments combined was worth 5% of the student’s overall grade. Responses from the consenting students were transcribed into a Google spreadsheet. Names were de-identified with assigned numbers to be later matched with post-survey assessments. After the completion of this module, post-surveys with the same items as the pre-surveys were administered to the students to determine any changes in student attitudes.

In addition to the students, we also surveyed two teaching professors and four community partner personnel with the same post-survey and collected their responses. These six expert respondents agreed to all statements and confirmed that the questions reflected the appropriate learning objectives, with the exception of one expert who did not fully agree to the statement about making additional changes to daily habits (data not shown). Paper-based pre- and post-surveys were transcribed for analysis and any unambiguous student misspellings or typing errors were corrected via spell check. Qualitative data from pre- and post-surveys were analyzed via two independent coders (S. A. and J. M.), who identified themes through emergent selective coding (Strauss & Corbin, 1998; Onwuegbuzie, Dickinson, Leech, & Zoran, 2009) and then shared their findings. A consensus was reached (100% agreement) by both coders on a theme that applied to each statement. Representative quotes were selected unanimously.

Focus Group Interviews

Following the final course examination, students had the opportunity to participate in a focus group interview to share their commentary and leave their feedback about the service-learning experience. Students were  compensated  with  light   hors   d’oeuvres as well as $10.00 scholarships for their time. Ten students from the 2017 class agreed to participate in a focus group discussion (which is 11.7% of that class). Questions that guided the discussion were as follows:

  1. What does it mean for a person to live sustainably?
  2. How do you think the service-learning experience will help you put your BY 101 course content into actual practice?
  3. Before today, had you heard about global climate change?
  4. Describe in your own words what you think global climate change is all about?
  5. Do you think global climate change is real?
  6. Do you think global climate change impacts human health?
  7. Do you feel plastic pollution in the environment impacts you?
  8. Do you think you need to change your daily habits in any way to minimize the impact on the environment?
  9. Do you think you need to inform people around you about global climate change, suggest/recommend to them about any lifestyle changes they need to make to attempt to minimize the impact on the environment? Recordings were later transcribed and analyzed for qualitative analysis. Two coders ( J. B. and D. M.) worked independently to identify themes within the transcribed focus group interviews using constant comparison analysis. Emergent themes were identified through open coding followed by iterative cycles of axial and selective coding (Strauss & Corbin 1998; Onwuegbuzie et al., 2009). Afterwards, the two coders discussed the findings and reached a consensus (100% agreement). Quotes which best represented overarching themes were then selected.
Statistical Analysis

Survey data were fitted to binomial mixed effects models using the glmer package in R. Students could respond “agree,”“disagree,” or“don’t know” to each question. Student responses coded“don’t know” were grouped with“disagree” during quantitative analysis, with the rationale that both “disagree” and“don’t know” represent non-expert attitudes. Thus, each question represented a binary choice, and our models asked whether the probability of a student’s expressing expert attitudes was affected by the course (i.e., differences in a given student’s response on pre- and post- surveys) or by a variety of demographic characteristics (course year, year in college, gender, underrepresented minority status, parent’s education level, highest level biology taken in high school, number of college biology courses taken previously, and whether or not the student was a nursing major or enrolled in an honors program). Student ID was included as a random effect in the model, allowing us to correct for possible different starting levels of agreement among the different students. Our strategy for model analysis was to initially fit a model using all   of the possible predictors as non-interacting fixed effects, and then to fit refined models that removed any predictors that were not significantly affecting student response. In these refined models, we then added interaction terms between all remaining predictors and pre/post to determine whether demographics predicted student receptiveness to course content; if these interaction terms were not significant, they were removed from the final analysis. Predicted levels of agreement for each question were computed from the final, refined model using the lsmeans package in R, and these were used to conduct pairwise comparisons between the questions. Predicted values from the model are expressed (e.g., in Figure 2) as log odds ratios, interpretable as the natural logarithm of p/1-p, where p is the probability of agreement and 1-p the probability of disagreement or “don’t know.”

Results

Quantitative

One hundred seventeen students completed the pre- and post-surveys across the 2017 and 2018 cohorts. Students were significantly more likely to agree with each of the six statements regarding global climate change and plastic pollution after completing the service-learning module (see Figure 2; effect of pre- vs. post- on the log odds ratio, +1.04 ± 0.17 , p = 1.2 x 10-9). Also, female students were significantly more likely than male students to agree with the questions (log odds ratio +0.82 ± 0.35, p = 0.02), and honors students were more likely than others to agree (log odds ratio +1.59 ± 0.61, p = 0.01). There was no statistically significant impact of parental education level, minority status, or other demographic categories on the likelihood of agreeing with the statements, nor was there evidence that any of the demographic categories predicted how much a student’s attitude would change over the course of the semester.

The likelihood of agreement varied dramatically among the questions as well. Students were significantly more likely to express familiarity with the concept of global climate change (Figure 2, Question 1) than to agree to any of the other statements. Students were also significantly more likely to accept the reality of global climate change (Figure 2, Question 2) than to express a feeling of responsibility for educating others about climate change or a concern about the impact of plastic pollution on themselves (Figure 2, Questions 5 and 6). There was no statistical evidence that agreement with any of the questions increased more than the others between the pre- and post-surveys; instead, they all increased by a similar amount.

Figure 2: Change in Student Attitudes About Climate Change

Students were asked six questions about their attitudes toward climate change; the natural logarithm of the odds of agreeing with each question vs. disagreeing are represented here by the bars, with error bars representing standard errors of the log odds estimate. For example, a value of 2 indicates that the odds of agreeing vs. disagreeing are e2, representing ~88% probability of agreement. A value of 0 would indicate equal odds of agreeing vs. disagreeing. Odds were calculated by averaging across all significant demographic predictors. Students were significantly more likely to agree with each question after taking the service-learning class (gray bars) than before taking it (white bars) (logistic mixed effects model, p < 0.0001). Lowercase letters represent significance groupings for pairwise least squares means comparisons among questions; note that there was no significant interaction between pre/post and question, i.e., the difference between pre-survey and post-survey is the same for all questions.

Qualitative

For a part of our qualitative analysis, we analyzed the same pre- and post-survey data set with particular attention not just to overall class trends, but also to the accompanying student justifications that were collected from the free-response portion of the questions. Twenty-one student responses changed from one or more of their pre- survey disagreements to agreement statements. A majority of the students who changed their minds to agreement expressed a realization of their responsibility as stakeholders in global climate change and  plastic  pollution (12 of the 21 student responses in this category); the rest of the students reported an increase in awareness about these issues (nine of 21 student responses in this category) (Table 3). On the other hand, 11 students remained either opposed or uncertain regarding one or more statements (Supplemental Table 4), with the most common being the need to inform others about climate change (Figure 2, Question 6). These 28 student responses reported apathy (three of 28), that it wasn’t their place to change minds (four of 17), that they were already doing what they could (five of 28), or that the issues presented were not actually problems or were not real (13 of 28) (Table 3). Stances that remained unchanged included a student going from, “The world is changing on its own. We have a miniscule impact on it. Show me hard evidence that we have truly caused climate change,” to “Because I don’t know what sources to trust.” Two other students reported on their post-surveys: “There is no real evidence….” and “…The science says it is real, but I question the integrity of the studies….” We note that the students who reported a lack of strong enough evidence were all from 2018, and interestingly, this cohort also included references to two political figures (Donald Trump and Al Gore), whereas the 2017 cohort did not (Supplemental Tables 2–4).

Fourteen students (12% of the overall 117) disagreed with post-survey statements who did not disagree with the pre-survey statements (Supplemental Table 3). Interestingly, these students’ views were similar to those of students who maintained disagreement, with the addition of some students who reported a change in awareness after the course (Table 3). Similar to the students who disagreed in both pre- and post-surveys, several 2018 students expressed concerns possibly related to emotionally charged political rhetoric. For instance, one student commented“Global climate change has become a loaded term in today’s society associated with a kind of man-made apocalypse” and another was uncomfortable “spreading that our world is getting worse and worse” despite being willing “to spread about recycling and no littering.” The 2017 cohort expressed no comparable sentiments. Across the spectrum of agreement and disagreement, however, students recapitulated themes addressed in the course as well as notions related to data or evidence that were presented in the data-driven lectures (Table 3, Supplemental Table 2).

Note: responses reflect total number of question responses rather than number of students. Question numbers (e.g. Q1) refer to the order of statements in Figure 2.

Table 2: Shifts in Pre- to Post-Survey Dispositions from Students in Both the 2017 and 2018 Cohorts

We then analyzed the focus group interview (n=1 interview with 10 students). Three themes emerged from our analysis of this focus group data, including student comments on course structure, a connection between the service-learning experience and the lecture component, and a connection of the material to the student’s everyday life. The themes and subthemes that emerged from the analysis are reflected in Table 3, along with representative quotations.

Interview responses from questions in Table 3 were coded into three themes (in dark blue boxes), each having its own subtheme (in light blue boxes) supported by student quotes from 10 different students from the 2017 cohort.

Table 3: Student Focus Group Data

Discussion

Service-learning is recommended to engage non-major students (Packer, 2009) and there exists a need for students to better understand scientific data (Lineman et al., 2015; Schuldt et al., 2011; Dauer & Forbes, 2016). In this study, we targeted two non-major biology courses with data-driven class discussions led by climate change scientists, followed by service-learning projects involving environmental pollution and sustainable practices.

Students were significantly more likely to agree with six statements about climate change in our survey after taking our revised course (Figure 2). Importantly, there was no effect on the results due to previous biology experience or to racial or socioeconomic demographics, suggesting that this curriculum can be used across student groups. The large majority of students were familiar with climate change and accepted its reality, but were much less likely to agree with statements suggesting that individuals had a responsibility to change their own behavior or encourage others to do so. Our curriculum did not explicitly encourage students to promote these practices for others, but as some of our students noted, this could be embedded into other curricula that target other behavior or disciplines (Table 4), such as public speaking or business courses.

The open-ended format of our survey allowed students to justify their responses, giving us insight into the thought processes leading to changes in agreement between pre and post surveys. For those students who changed their minds from disagreement to agreement, the data-driven lectures and service-learning were directly referenced in several student justifications and seemed to have had an effect on their perspectives about climate change as they (Table 3, Supplemental Table 2). For students who did not agree with the statements by the end of the semester (Supplemental Tables 3 and 4), the majority indicated that global climate change is not  a current problem, either because it is not real, not of significant magnitude to matter, or not under human control. Of these, several students cited a lack of scientific evidence, which suggests that students need more opportunities to judge the source of data in order to draw their own conclusions (Lineman et al., 2015; Schuldt et al., 2011; Dauer & Forbes, 2016), whereas other students referenced political reasons. Interestingly, we find these political sentiments only expressed by students in the 2018 cohort, possibly reflecting heated U.S. political discourse around climate change and the increasing polarization of U.S. politics following the controversial 2016 presidential election. It is possible that efforts to directly address the validity of differing political perspectives in the context of course material may improve the ability of these students to productively engage with the material, as have successful efforts to teach evolution to religious students (Barnes, Brownell, & Perez, 2017).

Three broad themes emerged from the focus group data: (a) students enjoyed the course structure, (b) students connected the service-learning experience to the classroom content, and (c) students connected their experiences to their day-to-day lives (Table 4). We know that service projects should be relevant and applicable to the learning objectives in the classroom so that students do not feel they are doing charity as busywork (Lynch, 2016; Chong, 2014), and when those connections are made, student mental networks of information are strengthened (Daniel & Mishra 2017; Lumpkin, Achen, & Dodd, 2015). When executed effectively service-learning has the capacity to foster student engagement at multiple levels: cognitive, behavioral, emotional, and social (Simonet, 2008; Celio et al., 2011). These components contribute to the learning process as well as to the student’s own personal development and sense of involvement (Nigro & Farnsworth, 2009). Our responses confirmed the student’s connections between the course learning objectives and their service-learning experience (Table 4).

In summary, we have shown promising effects for non-major students’ understanding of environmental stewardship in a three-hour service-learning module coupled with data-driven lectures. Notably, the demonstrated student gains in both specific learning objectives and civic engagement are on par with longer service-learning modules (Begley, 2013; Larios-Sanz, Simmons, Bagnall, & Rosell, 2011; Cain, 2013); students commented positively on the time commitment, making a graded three-hour requirement a feasible option for instructors considering service-learning. Students also applauded how a few of the service-learning opportunities were during the actual class hours as opposed to being scheduled outside class time. Moreover, although many community supervisors aligned students with learning objectives of the course, the engagement levels with students varied depending on the service-learning partner. When executed at UAB, this service-learning experience required the use of upper-level student supervisors to ensure students were participating for the entire time duration. We encourage interested professors to recruit teaching assistants and other student help for similar roles.

One limitation of this study is that students did not also answer formative, self-reflection questions about their overall experience, which is an important feature of the service-learning experience (Chong, 2014; Phelps, 2012; Soska, Sullivan-Cosetti, & Pasupuleti, 2010). Furthermore, this study did not tease out the degree to which the guest lectures, the professor lectures, or service-learning played a role in student gains, but rather approached these gains holistically, and we cannot be sure to what degree service-learning, as opposed to the broader curriculum, influenced the observed changes in student attitudes. We therefore recommend that future studies should attempt to analyze these components separately and should explicitly investigate how a student’s political beliefs could possibly influence their experiences in community-centered courses. Despite these limitations, we find that our service-learning curriculum was effective for our students. We therefore encourage other educators not only to consider service-learning as an educational pedagogy, but also to use such activities in the context of stimulating a dialogue on polarizing topics like global climate change (Hawkins & Stark, 2016; Yoho & Vanmali, 2016), as a means of engaging non-major biology students.

Acknowledgements

We thank Gabrielle Richards for data entry and Dr. Jeffrey Olimpo as a reference for qualitative data analysis. This work would not be possible without the UAB Office of Undergraduate Research and Service-Learning, and especially Ms. Amy Badham. A special thank you to all of our community partners and site leaders including Dr. Julie Price with the office of Sustainability at UAB. We would also like to acknowledge our guest speakers Drs. Dustin Kemp and James McClintock. A huge thank you also to all our site leaders from the introductory biology classes.

Funding

This material is based upon work supported by the National Science Foundation Research Coordination Networks in Undergraduate Biology Education [Grant No. 1826988] to J. J. M. and S. R. and the National Science Foundation Graduate Research Fellowship Program [Grant No. 1450078] to S. J. A. Student scholarships as an incentive to participate in the focus group interviews were supported by a mini-grant from the Office of Undergraduate Research and Service-Learning at UAB.

Authors

Daniel Mendoza

Daniel A. Mendoza is a Gates Millennium Scholar as well as a McNair Scholar. He graduated with a BS degree from the University of Alabama at Birmingham in 2019 and is currently a master’s student at The George Washington University, studying special education for culturally and linguistically diverse learners.

 

Sarah Adkins

Sarah Jeanne Adkins is a Ph.D. Candidate and National Science Foundation Graduate Research Fellow studying biology education and microbiology at the University of Alabama at Birmingham, with interests in reforming science using art and interdisciplinarity and in course-based undergraduate research experiences. Sarah serves as the Chair for STEM Education for the Alabama Academy of Science and co-founded the Research On STEM Education (ROSE) Network.

 

Jay Bhatt

Jay M. Bhatt is a research assistant professor at Creighton University in the Department of Pharmacology and Neuroscience. His biology education research focuses on engaging and mentoring undergraduate students in scientific life sciences research. His biomedical research focuses on understanding the molecular mechanisms that lead to human diseases.

 

J. Jeffrey Morris

J. Jeffrey Morris is an assistant professor at the University of Alabama at Birmingham and director of the ROSE Network, a NSF Research Coordination Network focused on the dissemination of evidence-based teaching practices to community colleges in order to improve the retention rates of transfer students into STEM programs at R1 universities. He also studies the ecology and evolution of marine bacteria, with a specific focus on the social lives of phytoplankton/bacterial consortia.

 

Samiksha Raut

Samiksha A. Raut is an associate professor at the University of Alabama at Birmingham and an associate director of the ROSE Network, a NSF-RCN funded initiative for the dissemination of evidence-based teaching practices to community colleges. Her current research interests are focused on analyzing the impact of professional development on undergraduate teaching assistants and assessing the efficacy of different interventions in majors and nonmajors biology courses. She has extensive experience in designing service-learning courses and is the recipient   of the Provost Award for Faculty Excellence in ServiceLearning at UAB. Address all correspondence to : S. Raut (sraut@uab.edu)

References

“UAB Student Data.” (2017). Office of Institutional Effectiveness and Analysis. http://www.uab.edu/institutionaleffectiveness/

images/factbook/sections/26_FactsFigures2018_StudentData. pdf

“Vision and change in biology undergraduate education, a call for action.” (2009). In: Carol Brewer DS (ed.). American Association for the Advancement of Science. Washington, DC.

Barnes, M.E., Brownell, S.E., and Perez, K.E. (2017).“A call to use cultural competence when teaching evolution to religious college students: introducing religious cultural competence in evolution education (ReCCEE).” CBE Life Sciences Education, 16(4), es4.

Begley, G. S. (2013).“Making connections: service-learning in introductory cell and molecular biology.” Journal of Microbiology & Biology Education: JMBE, 14(2), 213.

Brame, C. J. (2016).“Active learning.” Vanderbilt University Center for Teaching.

Cain, D. M. (2013).“Impact of a service-learning project on student success in Allied Health Microbiology course.” Journal of Microbiology & Biology Education: JMBE, 14(1), 129.

Celio, C. I., Durlak, J., & Dymnicki, A. (2011).“A meta-analysis of the impact of service-learning on students.” Journal of Experiential Education, 34(2): 164-181.

Chong, C. S. (2014).“Service-learning research: Definitional challenges and complexities.” Asia-Pacific Journal of Cooperative Education, 15(4): 347-358.

Cotner, S., Thompson, S., & Wright, R. (2017).“Do Biology Majors Really Differ from Non–STEM Majors?.” CBE—Life Sciences Education, 16(3), ar48.

Daniel, K. L., & Mishra, C. (2017). “Student Outcomes From Participating in an International STEM Service-Learning Course.” SAGE Open, 7(1), 2158244017697155.

Dauer, J. M., & Forbes, C. T. (2016). “Making decisions about complex socioscientific issues: a multidisciplinary science course.” Science Education & Civic Engagement: An International Journal, 8(2), 5-12.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014).“Active learning increases student performance in science, engineering, and mathematics.” Proc Natl Acad Sci USA, 111(23): 8410-8415.

Galgani, F., Pham, C. K., & Reisser, J. (2017). “Plastic Pollution.

Frontiers in Marine Science,” 4, 307.

Harvey, C. (2018).“Climate change is becoming a top threat to biodiversity.” Scientific American.

Haward, M. (2018).“Plastic pollution of the world’s seas and oceans as a contemporary challenge in ocean governance.” Nature Communications, 9(1), 667.

Hawkins, A. J., & Stark, L. A. (2016).“Bringing Climate Change into the Life Science Classroom: Essentials, Impacts on Life, and Addressing Misconceptions.” CBE—Life Sciences Education, 15(2), fe3.

Herlihy, N., Bar-Hen, A., Verner, G., Fischer, H., Sauerborn, R., Depoux, A., Flahault, A., & Schutte, S. (2016).“Climate change and human health: What are the research trends? A scoping review protocol.” BMJ Open, 6(12), e012022.

Keupper-Tetzel, C. (2017).“Service learning: An engaging teaching concept.” The Learning Scientists.

Knight, J. K., & Smith, M. K. (2010).“Different but equal? How nonmajors and majors approach and learn genetics.” CBE—Life Sciences Education, 9(1), 34-44.

Larios-Sanz, M., Simmons, A. D., Bagnall, R. A., & Rosell, R. C. (2011).“Implementation of a service-learning module in medical microbiology and cell biology classes at an undergraduate liberal arts university.” Journal of Microbiology & Biology Education: JMBE, 12(1), 29.

Lineman, M., Do, Y., Kim, J. Y., & Joo, G.-J. (2015).“Talking about Climate Change and Global Warming.” Plos One, 10(9): e0138996.

Little, A. M. (2012).“Service learning in non-majors biology: Learning outcomes and lessons from the field.”

Lumpkin, A., Achen, R. M., & Dodd, R. K. (2015).“Students perceptions of active learning.” College Student Journal.

Lynch, J. (2016).“What does research say about active learning?” Pearson.

Matthews, P. H., Dorfman, J. H., & Wu, X. (2015).“The impacts of undergraduate service-learning on post-graduation employment opportunities.” International Journal of Research on Service-Learning and Community Engagement, 3(1).

McLaughlin, J., Patel, M., Johnson, D. K., & de la Rosa, C. L. (2018). “The Impact of a Short-Term Study Abroad Program that Offers a Course-Based Undergraduate Research Experience and Conservation Activities.” Frontiers: The Interdisciplinary Journal of Study Abroad, 30(3).

Nam, Y., & Ito, E. (2011). “A climate change course for undergraduate students.” Journal of Geoscience Education, 59(4), 229-241.

Nigro, G., & Farnsworth, N. (2009).“The effects of service-learning on retention.” Northern New England Campus Compact.

Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L., & Zoran, A. G. (2009). “A qualitative framework for collecting and analyzing data in focus group research.” International journal of qualitative methods, 8(3), 1-21.

Packer, A. (2009). “Service Learning in a Non-majors Biology Course Promotes Changes in Students’ Attitudes and Values About the Environment.” International Journal for the Scholarship of Teaching and Learning, 3(1), n1.

Phelps, A. L. (2012). “Stepping from service-learning to SERVICE- LEARNING pedagogy.” Journal of Statistics Education, 20(3).

Schuldt, J. P., Konrath, S. H., & Schwarz, N. (2011).“”Global warming” or“climate change”?: Whether the planet is warming depends on question wording.” Public Opinion Quarterly, 75(1): 115-124.

Simonet, D. (2008). “Service-learning and academic success: The links to retention research.” Minnesota Campus Compact, 1, 1-13.

Soska, T. M., Sullivan-Cosetti, M., & Pasupuleti, S. (2010).“Service Learning: Community Engagement and Partnership for Integrating Teaching, Research, and Service.” Journal of Community Practice, 18(2-3): 139-147.

Strauss, A., & Corbin, J. (1998).“Basics of qualitative research: Techniques and procedures for developing grounded theory.” Thousand Oaks, CA: Sage publications.

Wu, X., Lu, Y., Zhou, S., Chen, L., & Xu, B. (2016).“Impact of climate change on human infectious diseases: Empirical evidence and human adaptation.” Environ Int, 86: 14-23.

Yoho, R. A., & Vanmali, B. H. (2016).“Controversy in biology classrooms—citizen science approaches to evolution and applications to climate change discussions.” Journal of microbiology & biology education, 17(1), 110.

Yokota, K., Waterfield, H., Hastings, C., Davidson, E., Kwietniewski, E., & Wells, B. (2017).“Finding the missing piece of the aquatic plastic pollution puzzle: Interaction between primary producers and microplastics.” Limnology and Oceanography Letters, 2(4): 91-104.

Supplemental Information

Pre-/Post-Reflection BY 101-1C (Fall 2017/Fall 2018)

(Responses to italicized questions were not included in the analysis.)

Please provide as much information as you can about your opinions and why you think that way.  There are no right      or wrong answers. We are just interested in knowing your views.

Your Name:  

Your selected Community Partner:  

  1. What does it mean for a person to live sustainably? Explain.
  2. How do you think the service-learning experience will help (or has helped) you put BY101 course content into actual practice? Explain.
  3. Before today, had you heard about global climate change? Agree / Disagree / Don’t know Explain the reasoning for your above-mentioned response.
  4. Describe in your own words what you think global climate change is all about.
  5. Do you think global climate change is real? Agree / Disagree / Don’t know Please explain your reasoning for your response.
  6. Do you think global climate change impacts human health? Agree / Disagree / Don’t know Please explain your reasoning for your response.
  7. Do you feel plastic pollution in the environment impacts you? Agree / Disagree / Don’t know Please explain your reasoning for your response.
  8. Do you think you need to change your daily habits in any way to minimize the impact on the environment? Agree / Disagree / Don’t know

Please explain your reasoning for your response.

  1. Do you think you need to inform people around you about global climate change, suggest/recommend to them about any lifestyle changes they need to make to attempt to minimize the impact on the environment? Agree / Disagree / Don’t know

Please explain your reasoning for your response.

Supplemental Table 1

Demographic Information on 117 Consenting Students

Supplemental Table 2

Explanations from Students Who Disagreed at the Beginning of the Semester, but Agreed with Statements at the End of the Semester

Supplemental Table 3

Explanations from Students Who Changed from Agreement (A) or Don’t Know (DK) at the Beginning of the Semester to Disagreement at the End of the Semester

Supplemental Table 4

Explanations from Student Who Disagreed with Survey Statements in both the Pre- and Post-Surveys

INTRODUCTORY BIOLOGY

(Non-Science Majors) – BY 101 2E

Topics in Contemporary Biology

Fall 2018

Instructor: Dr. Sami Raut Office: Campbell Hall – 104

Office Phone Number: (205) 934-9680 Email: sraut@uab.edu

Office Hours: By appointment on most days of the week

Lecture: Tuesday & Thursday (Section 2E) 2 pm – 3:15 pm (HB 105)

Textbook (Recommended): Biology: Science for Life with Physiology, 5th Edition, Belk & Borden Maier (Note: This book has can been customized and is now available as an e-book for $23.92) https://collections.pearsoned.com/#purchaseebook/1323549234

Another free reference book from Openstax: https://cnx.org/contents/s8Hh0oOc@12.1:Pj8cW7X1@5/Introduction

Course Description:

To begin with, this course will introduce you to the fundamental principles in Biology and the process of science in general. Besides, this course also aims at developing the critical thinking skills required to make well-informed, fact-based logical decisions and opinions related to personal, social and ecological issues. There is a special learning module on environmental issues and it is tied with service – learning. Service-learning is a form of teaching and learning strategy that integrates meaningful community service with instruction and reflection to enrich the learning experience, teach civic responsibility, and strengthen communities.

Course Learning Objectives:

Understand the basic process of science Identify the valid sources of scientific literature

*Environmental consciousness and civic responsibility

Analyze and apply scientific information to make everyday decisions

Gain a basic understanding of cell and its functions as it relates to health and wellness Understand the process of evolution and evidence behind it

*Includes a service-learning component

Class Policies:

Attendance:-

Lecture attendance is highly encouraged so that you can gain a better understanding of the material and do not fall behind. Note: The class will exactly start at the assigned time and therefore, please see that you come to class on time. Additionally, quizzes/ assignments, case studies, etc will be given at intervals. It is therefore, to your advantage to come to class and gain valuable participation points. There will not be any make-up quizzes or assignments, etc. If you do miss a class, then it is your responsibility to obtain lecture materials, handouts, assignments and class announcements from your fellow classmates. This also applies to additional material included in the lecture other than the textbook. There is a lot

of additional material in this course that will get incorporated from variety of different sources. We will have many guest lectures at intervals.

Class Ambience–

Please note that the class ambience is “highly social”! We incorporate many active learning techniques, which means you will be asked to collaborate with your immediate neighbor and exchange a few words or maybe complete an assignment. So, please see that you are seated next to someone. Many studies in the recent times have shown that students tend to learn better, when there is incorporation of active learning techniques in the classroom. This class attempts to create a positive and an inclusive learning environment for all so that no one feels inhibited to express themselves. Therefore, please be courteous to your classmates; do not indulge in unnecessary side/random conversations and all kinds of digital distractions.

Lecture Exams –

Attendance for all the exams is mandatory and is highly encouraged. All evaluated exams and quizzes/assignments have to be returned back to the instructor and are the sole property of the instructor. If you fail to do so, it will result in a“ZERO” for that particular exam or quiz/assignment. Bonus Quizzes/assignments will be announced or unannounced.

Make-up Exams-

Attendance for the scheduled exams is mandatory. Make-up exams are ONLY given in cases of medical in capacitance or extreme hardship. You must notify me before the exam if you will not be able to take the exam. Documentation clearly stating the date of the scheduled exam will be required. Failure to notify me within 24 hours of the scheduled exam will be an automatic 0. Please note: Make-up exams are essay exams. The make-up will be at the convenience of the instructor. Allow 3 hours for the make-up exam. Official university business that is in conflict with the exam will be considered excused if the student notifies me at the earliest date and provides a letter from the event’s sponsor.

Exam Format-

In general, the exam format will be multiple-choice and true/false.

DSS Accessibility Statement

UAB is committed to providing an accessible learning experience for all students. If you are a student with a disability that qualifies under Americans with Disabilities Act (ADA) and Section 504 of the Rehabilitation Act, and you require accommodations, please contact Disability Support Services for information on accommodations, registration and procedures. Requests for reasonable accommodations involve an interactive process and consist of a collaborative effort among the student, DSS, faculty and staff. If you are registered with Disability Support

Services, please contact DSS to discuss accommodations that may be necessary in this course. Students registered with Disability Support Services must provide a DSS accommodation request letter to their instructor via email before receiving any academic adjustments. If you have a disability but have not contacted Disability Support Services, please call 934-420 or visit http://www.uab.edu/dss or Hill Student Center Suite 409.

Title IX Statement

The University of Alabama at Birmingham is committed to providing an environment that is free from sexual misconduct, which includes gender-based assault, harassment, exploitation, dating and domestic violence, stalking, as well as discrimination based on sex, sexual orientation, gender identity, and gender expression. If you have experienced any of the aforementioned conduct, we encourage you to report the incident. UAB provides several avenues for reporting. For more information about Title IX, policy, reporting, protections, resources and supports, please visit http://www.uab.edu/titleix for UAB’s Title IX Policy, UAB’s Equal Opportunity, Anti-Harassment Policy and Duty to Report and Non-Retaliation Policy.

Withdrawing-

You may withdraw from a course and receive a grade of “W” up to and including October 19th. Please follow the University procedures to withdraw.

Cheating-

Please read and make sure you understand the UAB Academic Honor Code. Academic dishonesty will be reported to the appropriate university officials. Punishment is explained in the student handbook. Cheating is taken very seriously and will result in greater administrative action.

Grading-

Exams: 70%

Class Participation: 10 % Service Learning: 20 %

Service Learning: Out of the 20% allotted to service learning, 15% will be assigned to the complementation of three service-hours with the community partners and the remainder of 5% will be devoted to the pre (2.5%) and post-reflection (2.5%). There will be a sign up required to participate in service hours with the specified community partners. You cannot show up at the community partner’s site without a sign-up.

Three exams each worth 50 points (Please bring #2 pencils and an eraser for each exam. Answers marked on the scantron will only be taken into account and scantrons will not be re-run. So, please mark and erase your answers if there were a need on the scantron very clearly.)

Exams begin promptly at the scheduled time. You must be on time for exams. Note: If you are more than 10 minutes late then you won’t be allowed to take the exam.

Grades will be assigned as follows:

A: 90-100%

B: 80-89.99 %

C: 70-79.99 %

D: 60-69.99 %

F: under 59.99%

NOTE: There is NO EXTRA-CREDIT!!

A Teaching Assistant (TA) is available for this class. TA will conduct a review session prior to every exam.

Canvas-

All class power points will be uploaded on Canvas after the lecture. Note: The class power points simply supplement the lecture and hence, coming to class and taking notes will be helpful.

Electronic Gadgets-

Usage of cellular devices inside the classroom including texting is strictly prohibited! Texting in the class will result in a 10-point deduction from your overall grade each time you text. Laptops and ipads are ONLY allowed for taking notes. However, if you are doing anything else on these devices other than taking notes, this will result in banning you from future use of the laptop/ipad. Taking screen-shots of the blackboard with electronic devices is strictly prohibited as well.

Review Session Location & Hours: TBA

II. Tutoring Service at UAB-

To get a tutor please email: uasc@uab.edu or call 205-975-4884.

This service is free of charge to all enrolled UAB students and is offered by the University Academic Success Center.

Download the article:

Download (PDF, 3.52MB)

Disease and the Environment: A Health Disparities CURE Incorporating Civic Engagement Education

Introduction

Course-based undergraduate research experiences (CUREs) offer a novel avenue for engaging students in the scientific process (Bangera and Brownell, 2014). In contrast to traditional laboratories, CUREs are designed to foster autonomy through student-driven hypothesis generation, experimentation, data analysis, and dissemination of findings (Auchincloss et al., 2014; Spell, Guinan, Miller, and Beck, 2014). Current evidence suggests that participation in CUREs in the biological sciences leads to significant increases in students’ development of scientific process skills, ability to “think like a scientist,” and affective dispositions in the domain (Brownell, Kloser, Fukami, and Shavelson, 2012; Brownell et al., 2015; Jordan et al., 2014; Olimpo, Fisher, and DeChenne-Peters, 2016). Despite the importance of these documented benefits, few studies (e.g., Ballen, Thompson, Blum, Newstrom, and Cotner, 2018) have examined the mechanisms for establishing connections between students’ research and the larger community—what, in the CURE literature, is referred to as broader relevance—as well as the impact of those connections on cognitive and non-cognitive student outcomes. Review of published CUREs, including those cited in the CUREnet database (https://serc.carleton.edu/curenet/index.html), further suggest that this is especially true when considering civic engagement as a form of experiential learning and capacity building with the local community.

In this article, we describe the development and evaluation of the BIOL 1108: Health Disparities in the Border Region II CURE, which represents our efforts to address the aforementioned concerns through purposeful integration of civic engagement education into the CURE curriculum. A health disparities course theme was identified given the widespread health inequalities along the U.S.-Mexico border that have posed a challenge to the U.S. healthcare system (Bastida, Brown, and Pagán, 2008; Rosales, Carvajal, and de Zapien, 2016). In this context, civic engagement “encompasses actions wherein individuals participate in activities of personal and public concern that are both individually life-enriching and socially beneficial to the community” (AAC&U Civic Engagement VALUE Rubric, 2018). While the incorporation of civic engagement instruction into science, technology, engineering, and mathematics (STEM) pedagogy is not unique to our work, the research presented here is novel in several ways. First, the limited number of studies focusing on civic engagement within course-based research experiences have largely been conducted in inquiry- or discovery-oriented contexts (rather than in environments adopting a CURE model) (e.g., Ahmed et al., 2017; NASEM, 2015); conversely, the CURE may be structured such that it has public health implications, but students are not directly engaged with the public (e.g., Smyth, 2017). Secondly, our efforts and findings are responsive to recent work in the field (Ballen et al., 2018); we contend that this work provides a significant first step in examining broader relevance but that, due to methodological constraints, it misconstrues the level of importance of broader relevance in CUREs as being “insignificant,” particularly for non-major (i.e., non-biology) populations. Finally, we present robust assessment of student outcomes following engagement in the BIOL 1108 CURE in a manner that serves to highlight the strength of civic engagement as an alternative mechanism for achieving broader relevance beyond commonly employed approaches within CUREs, such as student co-authored publications or presentations (e.g., Kloser, Brownell, Chiariello, and Fukami, 2011; Laungani et al., 2018).

Specifically, a quasi-experimental, mixed methods design was used to examine the following research questions:

  1. What impact does engagement in the BIOL 1108 CURE have on students’ development of public health outreach skills?
  2. To what extent does participation in the BIOL 1108 CURE influence students’ sense of project ownership, science identity and networking skills development, and researcher self-efficacy?
  3. What perceptions do students hold of the BIOL 1108 CURE experience, particularly as it relates to their understanding of the relationship between science and society?

We hypothesized that student involvement in the BIOL 1108 CURE would lead to a significant increase in their public health outreach skills development and perceptions regarding the connections between science and the public, given the explicit focus on civic engagement within the context of the CURE. This assertion is supported by prior evidence in the field, which suggests that students highly value opportunities to engage with their community and report feeling equipped to do so following formal civic engagement instruction (Ahmed et al., 2017; Donovan and Schmitt, 2014). Furthermore, in concordance with empirical studies on the efficacy and benefits of CUREs in the biological sciences (e.g., Brownell et al., 2012; Fisher, Olimpo, McCabe, and Pevey, 2018; Mader et al., 2017; Olimpo et al., 2016), we anticipated that participation in the BIOL 1108 CURE would result in enhancement of students’ science identity and researcher development.

Course Description: Health Disparities in the Border Region II (BIOL 1108)

Health Disparities in the Border Region II (BIOL 1108) is the second course in a year-long, research-driven sequence within the Department of Biological Sciences at the University of Texas at El Paso (UTEP). Eighteen two-semester CURE series exist within the department and university as part of the Freshman Year Research-Intensive Sequence (FYRIS; https://fyris.utep.edu), an NIH-funded program modeled after the University of Texas at Austin’s Freshman Research Initiative (https://cns.utexas.edu/fri). Each course sequence possesses a distinct topical focus aligned with the lead faculty’s area of scholarship and enrolls a maximum of twenty-four students per section per term, with the intent of retaining the same cohort of students throughout the duration of the experience. Building upon the structure of Health Disparities in the Border Region I (BIOL 1107), which emphasized development of technical skills and experimental design (see Appendix 1 for the course syllabus), BIOL 1108 was developed to meet six core course objectives, as described in Table 1. During the 15-week term, class sessions occurred twice weekly for an average of 120 minutes each session. Students predominantly spent class time continuing to iteratively and collaboratively engage in the research projects that they had initiated in BIOL 1107, receiving feedback from their peers and the course instructors (J.T.O. and J.A.) about their progress, and outlining and implementing their civic engagement initiative as deemed feasible. This latter component of the BIOL 1108 course is unique in comparison to all other CUREs at the institution and was purposefully designed to connect students and their research with the communities in which that research occurred and which that research, at least in part, was intended to benefit (see Table 2 for alignment of student research interests and their corresponding civic engagement component).

In order to increase the fidelity of implementation of student outreach initiatives, research teams first constructed a community engagement plan during week #11 of the course (see Appendix 2 for the BIOL 1108 course syllabus). Specifically, this plan required that each group: (a) identify the individuals within the community with whom they intended to interact during the initiative; (b) describe what role those individuals would have in the outreach process; (c) articulate how contact would be made with external partners; and (d) generate an outline detailing how the outreach event would be organized, executed, and monitored. At the conclusion of the first session, students were invited to participate in a gallery walk, which allowed them to observe other team’s engagement plans and to provide feedback on those plans. Similarly, this allowed the course instructors to formatively assess student progress and address any questions or concerns that emerged. Research teams then used the constructive criticism provided by their peers to revise their community engagement plans during the second weekly session. 

Revised plans required subsequent approval from the course instructors, and, once finalized, teams could proceed to the implementation phase. In this context, it is important to note that the majority of research teams (n = 3) elected to initiate contact with community partners with minimal guidance and facilitation from the course instructors. For instance, members of the air quality monitoring team directly e-mailed the local organizer for the UTEP Earth Day Celebration to express their interest in the event and to request a table for their outreach activity, which included an “adverse effects of air pollution” matching activity for children and opportunities for adults to view and discuss existing air quality data for the region. Likewise, members of the HAI team identified and contacted a clinical professor in the UTEP School of Nursing, who provided them with access to collect data from and speak informally with nursing students who were currently participating in clinical rotations. Notably, all student groups were successful in executing one or more components of their outreach plan (see Table 2 for an overview).

We contend that this success is attributable to several factors. First, BIOL 1108 is a continuation of BIOL 1107. Accordingly, students have already established relationships with one another and are already invested in their research projects, with moderate to high levels of perceived project ownership reported (see Methods and Results sections below). Second, the BIOL 1108 CURE convened, on average, for four hours each week, which provided substantial time for peer-peer and peer-instructor discussion to occur with respect to each student team’s research and outreach agendas. Course deliverables, including weekly updates and the final civic engagement presentation, likewise held students accountable for their efforts and promoted metareflective practices among both the students and the instructors. Lastly, the course’s central focus on place-based health issues within the Paseo del Norte region likely encouraged students to formulate outreach plans that primarily necessitated interaction with individuals at UTEP or in the community, with whom they were already at least somewhat familiar.

Methods

Participant Recruitment

Participants (N = 16) represented a convenience sample consisting of all students enrolled in the BIOL 1108: Health Disparities in the Border Region II CURE at the University of Texas at El Paso in the Spring 2018 semester. As discussed previously, this course is a successor to BIOL 1107: Health Disparities in the Border Region I (Appendix 1) and is intentionally designed to provide students with opportunities to connect their independent research initiatives to the local community (see Course Description: Health Disparities II [BIOL 1108] and Appendix 2). The majority of the students (n = 13) completed BIOL 1107 prior to entering BIOL 1108; however, none of the participants had prior civic engagement or service-learning experience. Participants were predominantly female (62.5%) and majoring in STEM (93.8%), although the course was open to any individual whose degree requirements included BIOL 1108. Approval was received from the University of Texas at El Paso’s Institutional Review Board prior to conducting research involving human subjects.

Public Health Outreach Flowchart (PHOF)

Given the explicit focus of BIOL 1108 on research and civic engagement, we sought to examine the degree to which students were successful at constructing public health outreach plans prior to and following their participation in the course. To accomplish this objective, a modified version of the Scientific Process Flowchart Assessment (SPFA; Wilson and Rigakos, 2016), the PHOF, was developed and validated (via expert-panel review). Specifically, the PHOF presented students with a hypothetical scenario in which two introductory biology students were tasked with creating an outreach program to address the high incidence of asthma in their community due to widespread public exposure to pesticides. Participants were prompted to create a flowchart diagramming their plan and could use any text, arrows, and objects to accomplish the task (Appendix 3). Responses were blinded and scored using a modified version of the SPFA rubric (Wilson and Rigakos, 2016), which was likewise subjected to expert-panel review for the purposes of content and construct validation (Appendix 3). Each response was evaluated by two individuals with expertise in the social sciences and bioeducation research. High interrater reliability was achieved (K= 0.93; p < 0.001), with all disputes being resolved through discussion among the coders. Aggregate data were then entered into SPSS (v.23, IBM) and paired t-tests used to assess for pre-/post-semester shifts in performance.

Persistence in the Sciences (PITS) Survey 

As a complement to the PHOF, the PITS (Hanauer, Graham, and Hatfull, 2016) was utilized to assess the impact of the BIOL 1108 CURE on students’ sense of project ownership (content- and emotion-related), researcher self-efficacy, science identity development, scientific community values, and networking skills (post-only). An adapted version of the PITS was created for pre-semester utilization, in which the question stem was modified, where appropriate, to inquire about students’ initial beliefs and expectations (e.g., “I believe that the research I conduct this semester will help to solve a problem in the world”). Psychometric analyses indicated a high degree of construct validity (as established via expert-panel review) and reliability for both the pre-test (Cronbach’s  α= 0.943) and post-test (Cronbach’s  α = 0.857) versions of the instrument (Cronbach’s  α≥ 0.754 for each individual subscale). Given that all students in the course intended to continue to engage in research in subsequent semesters (as indicated in an end-of-semester one-minute response paper assignment), we did not inquire about their interest in persisting in conducting scientific research on the post-semester PITS diagnostic. Data were entered into SPSS (v.23, IBM), and, with the exception of the Networking scale, a series of paired t-tests were used to examine pre-/post-semester shifts in response. Descriptive statistics were tabulated for all Networking items.

Student Perceptions of the Course (SPC)

To better understand how the BIOL 1108 CURE impacted students’ beliefs about the relationship between science and civic engagement, we asked participants to respond to three open-ended prompts at the end of the term (Appendix 4; adapted from Lancor and Schiebel, 2018). Responses were analyzed using a descriptive interpretive approach (Tesch, 2013), with emergent themes identified via iterative cycles of open and axial coding. Each response was scored by two individuals with expertise in the social sciences and bioeducation research. High interrater reliability was achieved (K= 0.97; P < 0.001), with all disputes being resolved through discussion among the coders.

Results

Participation in the CURE Results in a Significant Increase in Students’ Development of Public Health Outreach Abilities. 

A series of paired t-tests were performed to examine pre-/post-semester shifts in participants’ PHOF responses with respect to the six rubric dimensions (Appendix 3). Results indicated a statistically significant increase in the total number of items reported (t(15) = 3.463; p = 0.003) and total flowchart rating (t(15) = 3.218; p = 0.006), as well as in the number of connections made between concepts (t(15) = 2.259; p = 0.039) and interconnectivity (t(15) = 2.360; p = 0.032), following engagement in the BIOL 1108 CURE (Figure 1). Significant increases in all other categories were likewise observed with the exception of the Measures of Success dimension (Figure 2).

Engagement in the CURE Enhances Students’ Sense of Project Ownership and Researcher Self-Efficacy 

Paired t-test analyses of student responses to the PITS revealed a significant, pre-/post-semester shift for both the Project Ownership (Content) scale (t(15) = 2.841; p = 0.012) and Researcher Self-Efficacy scale (t(15) = 3.381; p = 0.004) (Table 3). Remaining comparisons were not statistically significant. Descriptive analysis of networking data indicated that students engaged in research-related conversation most frequently with friends and least frequently with faculty external to the course (Figure 3).

Research-Civic Engagement Connections Are Evident in Students’ Post-Semester Written Questionnaire Responses 

In addition to examining the above cognitive and non-cognitive outcomes, we sought to understand the more globalized perceptions students possessed regarding connections between their research and the broader community. Qualitative analysis of SPC responses revealed, in a collective sense, that students valued the need for increasing community awareness of public health issues in the region and that this could be accomplished both through practical means (e.g., increased communication) and through professional means (e.g., students pursuing careers with a civic engagement focus). Furthermore, several students (n = 10; 62.5% of the participants) noted that the research projects that they initiated in the course could serve as a platform for engaging in future scholarship that served to “bring science to the public.” One student stated, for instance, that she “wanted to become a primary care physician one day” and hoped she could “continue doing research in the field of public health so [she could] better advocate for [her] patients’ lifelong health.” Another, in documenting what he believed he learned in the course that could enable him to effectively connect the broader community with issues in science, wrote that “among all of the typical things [he] discovered in the course (e.g., how to write a research proposal; laboratory methods), [he] learned not to hesitate to communicate ideas about the direction of research and how to make progress.” In doing so, he could then also “better communicate any possibility of something bad or beneficial [about his research] to the public in an effective manner.”  Comprehensive analysis of student responses, including identified themes, is presented in Tables 4A – C above. In interpreting these outcomes, it is important to note that across all open-ended prompts, more than 81% of responses were identified as belonging to two or more coding categories. 

Discussion

Since their inception, CUREs have sought to extend the benefits of research to an increasing number of undergraduates at all academic levels (Bangera and Brownell, 2014). Indeed, efforts within the discipline indicate that CUREs have the potential to promote the development of cognitive and non-cognitive student outcomes ranging from increased science literacy to science identity formation and persistence in STEM (e.g., Brownell et al., 2012; Brownell et al., 2015; Jordan et al., 2014; Olimpo et al., ,2016). While this is the case, few studies (e.g., Ahmed et al., 2017; Ballen et al., 2018) have expounded upon the extent to which those outcomes are fostered by purposeful integration of civic engagement education into the CURE curriculum.

In this article, we describe the structure of the Health Disparities in the Border Region II CURE, highlighting connections between student-driven research that examines health challenges within the students’ local community as well as the civic engagement/public outreach initiatives that course participants developed to connect their research to the broader society. Furthermore, we present both quantitative and qualitative evidence suggesting that participation in the CURE positively impacts students’ development of public health outreach skills, researcher autonomy and self-efficacy, and affective dispositions toward the role of science in society. These findings are consistent with several prior studies, which note that targeted instruction that establishes tacit links between student research projects and the public good increases students’ attitudes about the role of science in society, their understanding of the nature of science, and their appreciation and value for “doing” scientific work (e.g., Ahmed et al., 2017; Smyth, 2017).

In considering the outcomes reported here, we also wish to acknowledge the limitations associated with our work. Specifically, the structure of the FYRIS program and the resources allocated for the Health Disparities sequence (e.g., physical materials, financial incentives) were only intended to support a single implementation with a relatively finite population of students. There currently exists no opportunity to repeat the course sequence, although we are in the process of exploring alternate strategies to sustain and scale the CURE. In addition, although we believe it would be ideal to conduct a comparative examination of CURE and non-CURE courses with embedded civic engagement opportunities, no parallel non-CURE course presently exists within the department that incorporates direct outreach to the local community. While these caveats should be considered when evaluating reported outcomes both here and more broadly within the CURE literature (Brownell, Kloser, Fukami, and Shavelson, 2013), they also promote meaningful contemplation of future research directions in this area.

For instance, what factors are required to ensure that CUREs incorporating civic engagement education into the curriculum are both sustainable and scalable? Are these factors the same as those that are necessary to support sustainability and scalability of CUREs that do not integrate civic engagement experiences? In what ways do CUREs that promote civic engagement through science-society connections (ProCESS CUREs) allow us to examine as yet unexplored benefits of student participation in course-based research, and how do we effectively measure those outcomes? 

With specific regard to our own work, and in response to those limitations cited above, we likewise seek to engage in future studies that: (a) examine the replicability of the findings reported here (e.g., through analysis of outcomes in course iterations with larger student sample sizes); (b) implement multiple sections of the course in the same semester and vary whether or not students participate in civic engagement experiences, which will afford us an opportunity to more closely understand the direct impact of such experiences; and (c) collaborate with other UTEP CURE faculty to promote incorporation of civic engagement into their curricula and to conduct CURE-CURE comparative studies using similar methods as those described in this article. Pursuing these and other relevant areas of inquiry is a critical step toward understanding how CUREs can continue to foster growth in the classroom and beyond.

About the Authors

Jeffrey Olimpo

Jeffrey T. Olimpo, Ph.D., Assistant Professor in Biological Sciences at the University of Texas at El Paso (UTEP), is a discipline-based education researcher with more than five years of experience in the development, implementation, and evaluation of CUREs. His current research focuses on the cognitive and non-cognitive outcomes associated with novices’ participation in authentic research opportunities as well as the impact of professional development experiences on the career growth of graduate, postdoctoral, and faculty instructors. He is currently PI of the NSF-funded Tigriopus CURE and Ethics/RCR in CUREs initiatives and is a Tips and Tools Section Editor for the Journal of Microbiology & Biology Education. E-mail: jtolimpo@utep.edu; Phone: (915) 747-6923.

Jennifer Apodaca

Jennifer Apodaca, Ph.D., is Lecturer and Lab Coordinator in the Department of Biological Sciences at the University of Texas at El Paso, where she teaches classes covering topics in introductory biology, microbiology, molecular biology, comparative genomics, animal physiology, animal behavior, and evolutionary biology. Her primary research interest in bioeducation involves curriculum development and evaluation of course-based undergraduate research experiences and civic engagement in science activities that employ genome-scale experimental and computational approaches to topics in public health, microbiology, and genetics.

Aimee Herandez

Aimee A. Hernandez is an undergraduate Forensic Biology student at the University of Texas at El Paso, whose research experiences cover areas from virology to biology education. After completing her doctoral degree, she aspires to work as a forensic DNA analyst for the FBI. In addition to her interest in forensics, she plans to eventually teach at the high school or undergraduate level, ideally to inspire young scientists who are often underrepresented or underestimated to make a name for themselves in the scientific community.

Yok-Fong Paat

Yok-Fong Paat, Ph.D., is Associate Professor in the Department of Social Work at the University of Texas at El Paso. Her interests focus on family well-being, community participatory based research, and social integration.

Acknowledgments

We wish to thank the undergraduate researchers in the Health Disparities course sequence for their diligence and willingness to participate in this study. This research was supported in part through the HHMI PERSIST initiative, award no. 52008125. The opinions and views expressed in this article are those of the authors and do not necessarily reflect the opinions and views of the Howard Hughes Medical Institute and/or its constituents.

References

Ahmed, S., A. Sclafani, E. Aquino, S. Kala, L. Barias, and J. Eeg. 2017. “Building Student Capacity to Lead Sustainability Transitions in the Food System through Farm-based Authentic Research Modules in Sustainability Sciences (FARMS).” Elementa-Science of the Anthropocene 5: ar46. https://scholarworks.montana.edu/xmlui/bitstream/handle/1/14380/Ahmed_PB_2017.pdf?sequence=1 (accessed August 29, 2018).

American Association of Colleges and Universities (AAC&U). 2018. Civic Engagement VALUE Rubric. https://www.aacu.org/civic-engagement-value-rubric (accessed August 29, 2018).

Auchincloss, L.C., S.L. Laursen, J.L Branchaw, K. Eagan, M. Graham, D.I. Hanauer, G. Lawrie, C.M. McLinn, N. Pelaez, S. Rowland, M. Towns, N.M. Trautmann, P. Varma-Nelson, T. Weston, and E. Dolan. 2014. “Assessment of Course-based Undergraduate Research Experiences: A Meeting Report.” CBE-Life Sciences Education 13: 29-40. 

Ballen, C.J., S.K. Thompson, J.E. Blum, N.P. Newstrom, and S. Cotner. 2018. “Discovery and Broad Relevance May Be Insignificant Components of Course-based Undergraduate Research Experiences (CUREs) for Non-Biology Majors.” Journal of Microbiology and Biology Education 19(2): 19.2.63. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5976029/ (accessed August 29, 2018).

Bangera, G., and S.E. Brownell. 2014. “Course-based Undergraduate Research Experiences Can Make Scientific Research More Inclusive.” CBE-Life Sciences Education 13(4): 602-606.

Bastida, E., H.S. Brown, and J.A. Pagán. 2008. “Persistent Disparities in the Use of Health Care Along the US-Mexico Border: An Ecological Perspective.” American Journal of Public Health 98(11): 1978-1995.

Brownell, S.E., M.J. Kloser, T. Fukami, and R. Shavelson. (2012). “Undergraduate Biology Lab Courses: Comparing the Impact of Traditionally Based ‘Cookbook’ and Authentic Research-based Courses on Student Lab Experiences.” Journal of College Science Teaching 41: 36-45.

Brownell, S.E., M.J. Kloser, T. Fukami, and R. Shavelson. (2013). “Context Matters: Volunteer Bias, Small Sample Size, and the Value of Comparison Groups in the Assessment of Research-based Undergraduate Introductory Biology Lab Courses.” Journal of Microbiology and Biology Education 14: 176-182.

Brownell, S.E., D.S. Hekmat-Scafe, V. Singla, P.C. Seawell, J.F.C. Imam, S.L. Eddy, T. Stearns, and M. Cyert. 2015. “A High-Enrollment Course-based Undergraduate Research Experience Improves Student Conceptions of Scientific Thinking and Ability to Interpret Data.” CBE-Life Sciences Education 14: ar21. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4477737/ (accessed August 29, 2018).

Donovan, K., and E. Schmitt. 2014. “Service Learning in Science Education: A Valuable and Useful Endeavor for Biology Majors.” Beta Beta Beta Biological Society 85(3): 167-177.

Fisher, G.R., J.T. Olimpo, T.M. McCabe, and R.S. Pevey. 2018. “The Tigriopus CURE—A Course-based Undergraduate Research Experience with Concomitant Supplemental Instruction.” Journal of Microbiology and Biology Education 19(1): 19.1.55. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5969452/ (accessed August 29, 2018).

Hanauer, D.I., M.J. Graham, and G.F. Hatfull. 2016. “A Measure of College Student Persistence in the Sciences (PITS).” CBE-Life Sciences Education 15, ar54, doi:10.1187/cbe.15-09-0185.

Jordan, T.C., S.H. Burnett, S. Carson, S.M. Caruso, K. Clase, R.J. DeJong, J.J. Dennehy, D.R. Denver, D. Dunbar, S.C.R. Elgin, A.M. Findley, C.R. Gissendanner, U.P. Golebiewska, N. Guild, G.A. Hartzog, W.H. Grillo, G.P. Hollowell, L.E. Hughes, A. Johnson, R.A. King, L.O. Lewise, W. Li, F. Rosenzweig, M.R. Rubin, M.S. Saha, J. Sandoz, C.D. Shaffer, B. Taylor, L. Temple, E. Vazquez, V.C. Ware, L.P. Barker, K.W. Bradley, D. Jacobs-Sera, W.H. Pope, D.A. Russell, S.G. Cresawn, D. Lopatto, C.P. Bailey, and G.F. Hatfull. 2014. “A Broadly Implementable Research Course in Phage Discovery and Genomics for First-Year Undergraduate Students.” MBio 5(1), e01051-13, doi:10.1128/mBio.01051-13.

Kloser, M.J., S.E. Brownell, N.R. Chiariello, and T. Fukami. 2011. “Integrating Teaching and Research in Undergraduate Biology Laboratory Education.” PLoS Biology 9, e1001174, doi:10.1371/journal.pbio.1001174.

Lancor, R., and A. Schiebel. 2018. “Science and Community Engagement: Connecting Science Students with the Community.” Journal of College Science Teaching 47(4): 36-41.

Laungani, R., C. Tanner, T.D. Brooks, B. Clement, M. Clouse, E. Doyle, S. Dworak, B. Elder, K. Marley, and B. Schofield. 2018. “Finding Some Good in an Invasive Species: Introduction and Assessment of a Novel CURE to Improve Experimental Design in Undergraduate Biology Classrooms.” Journal of Microbiology and Biology Education 19(2): 19.2.68. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6022745/ (accessed August 29, 2018).

Mader, C.M., C.W. Beck, W.H. Grillo, G.P. Hollowell, B.S. Hennington, N.L. Staub, V.A. Delesalle, D. Lello, R.B. Merritt, G.D. Griffin, C. Bradford, J. Mao, L.S. Blumer, and S.L. White. 2017. “Multi-Institutional, Multidisciplinary Study of the Impact of Course-based Research Experiences.” Journal of Microbiology and Biology Education 18(2): 18.2.44. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5577972/ (accessed August 29, 2018).

National Academies of Science, Engineering, and Medicine (NASEM). 2015. Integrating Discovery-based Research into the Undergraduate Curriculum. Washington, DC: The National Academies Press.

Olimpo, J.T., G.R. Fisher, and S.E. DeChenne-Peters. 2016. “Development and Evaluation of the Tigriopus Course-based Undergraduate Research Experience: Impacts on Students’ Content Knowledge, Attitudes, and Motivation in a Majors Introductory Biology Course.” CBE-Life Sciences Education 15: ar72. https://www.lifescied.org/doi/pdf/10.1187/cbe.15-11-0228 (accessed August 29, 2018).

Rosales, C.B., S. Carvajal, and J.E.G. de Zapien. 2016. “Editorial: Emergent Public Health Issues in the US-Mexico Border Region.” Frontiers in Public Health 4(93), http://doi.org/10.3389/fpubh.2016.00093.

Smyth, D.S. 2017. “An Authentic Course-based Research Experience in Antibiotic Resistance and Microbial Genomics.” Science Education and Civic Engagement 9(2): 59-64.

Spell, R.M, J.A. Guinan, K.R. Miller, and C.W. Beck. 2014. “Redefining Authentic Research Experiences in Introductory Biology Laboratories and Barriers to Their Implementation.” CBE-Life Sciences Education 13: 102-110.

Tesch R. 2013. Qualitative Research: Analysis Types and Software. New York: Routledge.

Wilson, K.J. and B. Rigakos. 2016. “Scientific Process Flowchart Assessment (SPFA): A Method for Evaluating Changes in Understanding and Visualization of the Scientific Process in a Multidisciplinary Student Population.” CBE-Life Sciences Education 15: ar63. https://www.lifescied.org/doi/pdf/10.1187/cbe.15-10-0212 (accessed August 28, 2018). 

 

Download the Appendices

Download (PDF, 260KB)

Download (PDF, 261KB)

Download the Article

Download (PDF, 1.12MB)

Learning Without Borders: 
Qualitative Exploration of Service-Learning

Introduction

For the last four years, pharmacy, physician assistant, pre-medicine, and nursing students enrolled or associated with Butler University’s College of Pharmacy and Health Sciences (COPHS) and College of Liberal Arts and Science (LAS) have partnered with Barnabas Task to travel to the Dominican Republic (DR) for an annual medical mission trip. Barnabas Task, a nonprofit organization founded in Fort Wayne, Indiana, conducts multiple service trips every year with dental and medical professionals, as well as other volunteers, to the Dominican Republic, Cuba, or Guatemala. Barnabas Task’s mission is “community transformation through leadership development” (Barnabas Task, 2013), and they utilize community health evangelism (CHE) to accomplish this goal. During these mission experiences, students have the opportunity to assist medical providers through patient triage, medical scribing, and medication dispensing.  Students also work directly with community leaders to educate them on public health topics including nutrition, exercise, smoking cessation, dental hygiene, and mosquito-borne illnesses. These community leaders can then educate others and spread the knowledge through grass roots. This philosophy of developing a relationship with host communities mirrors the work of Olenick and Edwards (2016). Their article in Nursing for Women’s Health concludes that short-term health missions are more effective when they focus on a “long-term commitment rather than a quick fix.”

Students and volunteers work to form long-term commitments not only by educating community leaders in the DR, but also by working with local students who act as translators within the clinic. Most of the students who made the trip lacked fluency in Spanish, and all volunteers are therefore provided with a translator. Every clinic day, students from Oasis Christian School, which is a part of Santiago’s private school system, help translate for the students and medical volunteers. Students from the local Catholic medical school, Pontificia Universidad Católica Madre y Maestra (PUCMM), also join the clinic daily to translate, triage patients, and fill prescriptions. Some students keep returning to the clinic even after they graduate medical school and volunteer as healthcare providers to help their community. This includes a provider who has made a commitment to visit the clinic quarterly to follow up with patients whose medications for chronic diseases such as diabetes and hypertension may require adjustments. Interactions with the DR students and providers adds another layer of collaboration, where students can learn from one another while caring for underserved populations.

To strengthen these long-term commitments, Barnabas Task turned to Butler University Fairbanks Center for Communications and Technology in 2015 with the goal of developing an electronic means of carrying medical information during the mission trips and accessing these records during future medical trips, thus starting the relationship between Barnabas Task and the Engineering Projects in Community Service (EPICS) course at Butler University. Computer science and software engineering students enrolled in this course meet biweekly to complete a “supervised team software project for a local charity or non-profit organization” (Linos, 2012). This relationship initiated the development of an Electronic Medical Records (EMR) application prototype, which runs as an iOS app. Students in the EPICS course collaborated with Barnabas Task to meet their needs to provide continuity of care and formed a relationship with healthcare students from COPHS to format the iPad application. Currently in the fifth semester of collaboration between EPICS, Barnabas Task, and COPHS, the application continues to be updated and built upon and is now a stable prototype of a bilingual EMR that can preserve patient records, transcribe prescriptions to the clinic’s pharmacy, and maintain medication inventory.

Data on the benefits of EMRs are plentiful. A systematic review published in September 2017 established how EMRs significantly improve documentation of clinical information and enhance quality outcomes in the long-term acute care setting (Kruse et al., 2017). Similar effects can be seen in the inpatient hospital setting. Khalifa and colleagues found that after EMRs had been implemented in their health system, there was “an increase in information access, increased healthcare professionals productivity, improved efficiency and accuracy of coding and billing, improved quality of healthcare, improved clinical management (diagnosis and treatment), reduced expenses associated with paper medical records, reduced medical errors, improved patient safety, improved patient outcomes and improved patient satisfaction” (Khalifa, 2017). A comprehensive review by Keasberry, Scott, Sullivan, Staib, and Ashby (2017) ascertained that EMRs enhance patient safety by including alerts about drug interactions and adverse drug reactions. The utilization of an EMR also improves patient outcomes by increasing to guideline recommendations. EMRs stateside improve hospital processes and patient care, which explains the DR clinic’s need to obtain an EMR to improve clinic processes abroad.

We conducted a thorough search and determined that there are no similar efforts currently described in the literature. However, there are publications that discuss collaborations and active learning as well as the benefits of these types of interactions. A group at the University of Wisconsin created interprofessional groups that served both a local community and a global community in Malawi. They concluded that students had increased their level of understanding in values and ethics, roles and responsibilities, and teamwork as a result of the experience (Dressel et al., 2017). Johnson and Howell (2017) also discuss the benefits of service-learning and interprofessionalism. Healthcare students from different programs including pharmacy, medicine, physical therapy, and nursing traveled to Ecuador for a service-learning opportunity. The authors explain how the students had to work through communication barriers both with their patients and with other healthcare professionals, all of whom spoke a different language. Increasing cross-cultural and interprofessional learning will be crucial in the future due to the diversifying healthcare system. A nursing cultural simulation developed by Carlson et al. (2017) connected nursing students in Hong Kong and Sweden and ultimately ascertained that the intercultural experience developed collaborative skills, including communication, between the two groups of students as they worked to complete a case study. In our literature review we found plenty of interprofessional articles; however, the literature lacks information on students from different colleges collaborating on a project to better the community they plan to serve. Professionals in the healthcare field are being exposed to a wide array of people with different educational backgrounds, and it is important to confront these language and knowledge barriers.

This study was developed in order to (a) assess how information technology affects clinic processes, (b) identify student learning and cultural awareness when collaborating with students from different colleges and globally, and (c) understand how global missions are viewed by the communities being served.

Methods

When commencing this project we hypothesized that students would gain knowledge about how to work with other professionals, increase their skills within their various areas of expertise, and develop cross-cultural awareness while helping to improve a community’s health with the creation of an EMR. The institutional review board approved the anonymous survey that was sent to all sixty-five volunteers who worked in the underserved clinic in the DR and the EPICS students who helped develop the EMR but were unable to go to the DR. Using Qualtrics (Qualtrics, Provo, UT, May 2017), an online survey platform, the survey was created to consist of multiple choice and free response questions regarding demographics, role in the project, and experience in the clinic. Utilizing skip logic, participants answered questions written specifically for their role in the clinic (for example, healthcare student; computer science student; translator; etc). The original survey questions are listed in appendix 1. Results from the open-ended questions on the survey were analyzed based upon common themes and similar wording found throughout the participants’ answers. The institutional review board also approved an anonymous quality survey all patients at the clinic eighteen years of age and older had the opportunity to take. Those who participated answered four questions about their time spent in the different stations of the clinic, whether they would recommend the clinic to their friends or family members, and whether they believed the clinic brought hope to the community. If an entire family came to the clinic, one person from the family could complete the survey for their household. In total, 95 patients completed the survey using SurveyMonkey.

Results

Of the 65 clinic volunteers who were sent the survey, 51 elected to complete it for a response rate of 78.5%. The specific roles for each of the responses are illustrated in Figure 1. Starting with student learning, knowledge was gained through this experience through the various collaborations. The EPICS team, healthcare professionals, and Dominican volunteers all had participants who reported their top learning experience was in communication. Three out of five of the EPICS team members stated their top two non-technical learning experiences were in communication and teamwork. Students are also retaining the knowledge from this experience, as five out of five responses by the EPICS team stated they have used the knowledge gained in this course outside school or in another class. One EPICS member conveyed the importance of this class being able to “bridge the gap between those who are very technical, with little healthcare experience, and healthcare clinicians who possess little technical expertise.” Examining the development of technical skills, all of the EPICS students grew in both Xcode (Apple’s software development environment) and Swift (Apple’s programming language) (Apple, 2018). One EPICS student gained experience in setting up an onsite clinic with WiFi to make sure the EMR application could work within the clinic and the iPads could communicate with one another. Not only did EPICS members learn technical skills to be used in their future careers, but students also reported an improvement in their Spanish and an increase in knowledge about the Dominican healthcare system and culture.  Similarly, half of the healthcare students reported an increase in knowledge about the Dominican culture, lifestyle, and healthcare system as one of their top three learning experiences. Not only did American students learn from the Dominican students, but four of the six Dominican students who took the survey noted that one of the benefits of the clinic was being able to practice their English, while three of six students stated their main benefit from the clinic was refining their medical skills with the collaboration of American and Dominican providers.

The survey also included questions about the students’ experiences in intercultural and interprofessional relationships. Five out of six EPICS students reported a positive interaction when working with students with a healthcare background. One student, when asked to comment on his or her overall experience with the COPHS and EPICS students, remarked that it “was extremely fulfilling to witness how the efforts of a variety of students can put their knowledge and skills together to make something special happen.” Eleven out of thirteen healthcare students reported a positive experience when collaborating with the EPICS team and one stated specifically that the EPICS team is “important for our clinic running smoothly.”

While healthcare students, the EPICS team, and Dominican students gained great knowledge while working together, so did the healthcare professionals who helped run the clinic. Half of the providers stated there was a benefit to working in a different scope of practice in a different culture and stated that their biggest challenge was language barriers between their patients and sometimes their translators. However, the EMR application may have reduced this language barrier by means of prototype through an English-Spanish toggle. All of the providers who took the survey would be interested in using the application in the future. Three out four healthcare providers stated that the application improved the efficiency of the clinic, and one of the providers stated that the EMR improved patient safety by forgoing legibility issues of doctor’s handwriting and by allowing the provider to see previous visit history and ascertain a past medical history.

Improving clinic operations was important, but so was seeing the hard work come to life.  From one of the EPICS students who attended the trip: “There aren’t any words to put in for the experience of the trip. It was incredible and even better on our end to see the work we put in over the semester at work in real time helping people in need. It really gives us a different perspective. It has made me want to go back again next year.”

Both healthcare and EPICS student teams appreciated the each other’s knowledge base and were able to learn from one another. Seven out of seventeen students from EPICS and future healthcare providers suggested there be more meetings between the two student teams to allow more communication and form better relationships and to improve collaboration on the application prior to the trip. One student conveyed his or her suggestion for improved interactions by stating: “I wish the healthcare students could have had a larger impact when it came to some of the formatting in the app.” Another stated, “we could have been helpful when it came to inputting drug names and formatting it the way that most resembles a prescription.”

One example of the collaboration between the two groups was a simulation clinic on Butler University’s campus before heading to the DR. One EPICS student stated: “Witnessing and collaborating with the students who would actually be using the application was vital.

 

We were able to together identify the most effective and efficient designs for the app, as well as locate bugs throughout the app that we may not have otherwise noticed.” Four of thirteen healthcare students who attended the simulation said the simulation helped students learn how to use it before traveling to DR and six out of thirteen healthcare students noted there was value in the simulation because it worked out issues beforehand and allowed the EPICS team to add more features to application. More collaboration is necessary because while 10 out of 16 users of the EMR said it was a positive experience, five out of the 16 said there was need for improvements. While the EMR needs improvement, all of the 13 healthcare students who took the survey stated that their overall experience was positive.

Finally, knowledge was gained through this experience but so were friendships.

“The trip felt like a once-in-a-lifetime experience. It was incredible to witness both teams’ work and preparation pay off. Our group of students formed a tight-knit group with relationships that will likely last a lifetime. We were also able to form friendships with people there and share our cultures with one another. I greatly enjoyed the activities outside of the clinic—they provided inspiration on how we can continue to make a difference.”

While the application and learning is important for the students, for healthcare professionals the patient is the top priority, and for engineers the customer is the top priority. To ensure our patients were satisfied and to see how an EMR effects clinic processes we interviewed 95 patients to assess where there is room for improvement with our application and clinic in the future. Figures 2–5 represent how patients responded when asked about the amount of time it took to enter the clinic, register at the clinic, see the physician or healthcare provider, and receive their medications. Responses concerning the amount of time it took to enter the clinic were the most evenly distributed of the four figures, ranging from “very fast” to “normal” amount of time. The amount of time to be registered as well as to see a provider were very similarly distributed, with only a small percentage of patients reporting “too long” of a wait. The amount of time to receive medications followed a similar distribution to Figures 3 and 4; however, it was the largest report of “too long” a wait. Patients were also asked if they had attended the clinic previously, which 46 out of the 95 patients who completed the survey had.  Of the 60 patients who responded to the question about whether this clinic brings their community hope, all answered “yes” and all 95 patients who answered the survey said they would recommend this clinic to their friends and family.

Discussion

The professional world becomes more intertwined each day with professionals obtaining multiple degrees, technology advancing at a rapid pace, and the increased need for multiple professionals to be working together to achieve a common goal. Students with healthcare or computer science backgrounds will work together once they enter their careers, because healthcare is constantly in conjunction with, and reliant on, technology. Learning about other disciplines through collaboration towards a mutual goal helps prepare students of both colleges and disciplines to better communicate with people who have different educational backgrounds.

Beyond communication, other lessons learned through this experience included collaboration and teamwork. This project began through collaboration, as Barnabas Task has been collaborating since 2008 with people from varying cultures to facilitate CHE. Butler University began helping staff and supplying clinics in 2014, and the EPICS team was introduced in 2015 to create the EMR application (Barnabas Task, 2013). Similar to the mission trip described by Dressel et al. (2017), students reported an increase in their teamwork skills. The application continually evolves as innovative ideas develop from communication and teamwork between the EPICS and healthcare students. To improve both this learning experience and the application, the EPICS and healthcare teams need more collaborative meetings and communication, which have been set up via live simulated clinic days in the United States. The team views the application working in real time and can modify the application before arriving at the clinic. The need for more simulations was reiterated in the survey results: almost half of students wanted an increase in the number of meetings between the two groups prior to the trip. More meetings will allow for the healthcare students to help update the prescribing and diagnostics parts of the application and to provide recommendations for further clinical functions in the prototype application, including drug interaction reporting and other patient safety features.

It is important that the students gained knowledge from this collaboration, but ultimately the goal is to help the patients in the DR. An EMR application is warranted for helping track past medical records; over half of the patients who took the survey reported being seen in the clinic previously. With patients returning each year, there is clearly a need for the clinic, and the clinic is being utilized as routine care for many people. The application allows past medical records to be viewed, to see progression of disease states and to ensure that the patient is receiving the best care possible. The application improves patient safety by allowing allergies to be documented and viewed through their prior visit history. The support for EMRs improving patient safety has been shown in the work of Khalifa (2017), as there were fewer occurrences of medical error. Providers can also access medication histories to track clinical progression. Not only does the application help prevent medication errors, it also improves the processes of the clinic. Patients are quickly registered and triaged and then sent to see a provider, without the hassle of paper charts. Only two of the 95 patient respondents commented that any step of the clinic took too long. Future development and evolution of the application could help further streamline clinic processes and improve patient satisfaction.

Not only is the application evolving, but so is the EMR EPICS project. There has been a growing number of EPICS students interested in the collaboration with healthcare students. The EMR project continues to attract new and returning Computer Science and Software Engineering (CSSE) students, who find this project intriguing and realize the potential it has for experiential learning. The EMR project has spanned over six consecutive semesters and has currently attracted and engaged 35 CSSE students. The trip teaches students to collaborate with students of different educational backgrounds and helps students discern their future career paths. One of the EPICS students changed his major after exploring his passion for computer programming while working on the EMR project. All participants in the application collaboration group reported some form of educational growth.

Beyond their own education, this experience also exposes students to the education styles of the Dominican Republic. Medical school in the DR takes six years to complete as opposed to the eight years required to achieve a medical degree in the United States. Cultures differ not only in education but also in communication styles and language. Learning to respect the cultures and healthcare systems of other countries will help students become more adaptable and knowledgeable as they embark on their future careers. It is also beneficial to familiarize oneself with other cultures, because many medical professionals are obtaining their degrees abroad, while still wishing to practice in the United States. This trend was voiced by many of the medical students who acted as the group’s translators during the clinic in the DR. As of 2006, approximately 25% of physicians practicing in the United States obtained their medical degree abroad, a number that has been increasing since the 1960s (Boulet, Cooper, Seeling, Norcini, & McKinley, 2009). Not only are physicians with different educational backgrounds practicing medicine in the United States, there has also been an increase in the number of foreign-born United States citizens. With almost 13% of the United States’ population being born in another country, providers will be encountering patients with a variety of backgrounds (Singer, 2013). It is important for healthcare providers to adapt and be knowledgeable of cultures different from their own.  Cultural awareness is the main experience gained from clinics where US and DR students volunteering together.

In the future, it would be beneficial to continue to track patient surveys to ensure that the application keeps improving patient satisfaction and clinic efficiency. However, it is reassuring to see that a majority of patients did believe that their wait times were acceptable and that the clinic is currently working at an efficient pace. Looking forward, it would also be appropriate to start examining clinical outcomes of patients, as the EMR is able to track them on a yearly basis to see whether medical interventions are making a long-standing impact on patients’ disease states. As Kruse et al. assert (2017), EMR systems can improve quality outcomes for patients in the acute setting. Data collected from the DR clinic could be examined to determine whether these same improvements can be repeated. Overall, the collaboration between healthcare students and computer science students has led to the production of a functioning, affordable EMR application prototype to improve patient safety and satisfaction. It has also expanded technical and communication skills for students across Butler’s campus and among the DR students that Butler University connects with while in the DR. The goals of this project in the future would be to keep improving the application and eventually provide access to the application to other non-profit organizations to help them serve their patient population.

Acknowledgement

These data were presented at the National Center for Science & Civic Engagement Conference for Science and Engineering for Social Good in Atlanta, Georgia in February 2018. At the conference many people, including Edward Coyle, co-founder of both the Vertically-Integrated Projects (VIP) program and the Engineering Projects in Community Service (EPICS) gave us advice for proceeding with our project.

About the Authors

Courtney Cox

Courtney Cox is a current pharmacy student at Butler University and has traveled to the Dominican Republic three times with the team. After graduation in May 2018, she hopes to pursue a career that allows her to continue to work with an underserved population both in the United States and abroad.

 

 

Sarah Lenahan

Sarah Lenahan, Class of 2019 PharmD candidate at Butler University, has traveled to the Dominican Republic twice working with the Electronic Medical Record application and will be going again during May 2018. She hopes to pursue a career in pharmacy that allows her to integrate her passions of faith, learning, and pharmacy to help underserved patient populations.

 

Patricia S. Devine

Patricia Devine is an Associate Professor and Campus-Based Experiential Education Director at Butler University College of Pharmacy and Health Sciences. Her passion and research interests are in improving health globally.

 

 

Panagiotis K. Linos

Panagiotis Linos has been a professor of Computer Science and Software Engineering at Butler University since 2001. The birth of the EPICS program at Butler is the result of his passion for community service and experiential learning. Before joining Butler, he was the Chairperson of the Computer Science department at Tennessee Technological University.

References

Apple. (2018). Apple Worldwide Developers Conference. Retrieved from  https://developer.apple.com/wwdc.

Barnabas Task: Story-Teller of Many. (2013) Retrieved from http://www.barnabastask.org/.

Boulet, J. R., Cooper, R. A., Seeling, S. S., Norcini, J. J., McKinley, D. W. (2009). U.S. citizens who obtain their medical degrees abroad: an overview, 1992–2006. Health Aff (Millwood), 28(1), 226–233. doi:10.1377/hlthaff.28.1.226

Carlson, E., Stenberg, M., Chan, B., Ho, S., Lai, T., Wong, A., & Chan, E. A. (2017). Nursing as universal and recognisable: Nursing students’ perceptions of learning outcomes from intercultural peer learning webinars: A qualitative study. Nurse Educ Today, 57, 54–59. doi: 10.1016/j.nedt.2017.07.006

Dressel, A., Mikandawire-Valhmu, L., Deitrich, A., Chriwa, E., Mgawadere, F., Kambalametore, S., & Kako, P. (2017). Local to global: Working together to meet the needs of vulnerable communities. J Interprof Care,, 20, 1–3. doi:10.1080/13561820.2017.1329717

Johnson, A. M., & Howell, D. M. (2017). International service learning and interprofessional education in Ecuador: Findings from a phenomenology study with students from four professions. J Interprof Care 31(2), 245–254. doi: 10.1080/13561820.2016.1262337

Keasberry, J., Scott, I.A., Sullivan, C., Staib, A., & Ashby, R. (2017). Going digital: a narrative overview of the clinical and organisational impacts of eHealth technologies in hospital practice. Aust Health Rev 41(6), 646–664. doi: 10.1071/AH16233.

Khalifa M. (2017). Perceived benefits of implementing and using hospital information systems and electronic medical records. Stud Health Technol Inform, 238, 165–168.

Kruse, C. S., Mileski, M., Vijaykuma, A. G., Viswanathan, S. V., Suskandla, U., & Chidambaram, Y. (2017). Impact of electronic health records on long-term care facilities: Systematic review. JMIR Med Inform, 5(3), e35. doi: 10.2196/medinform.7958

Linos, P.K. (2012). Ten Years of EPICS at Butler University: Experiences from Crafting a Service-Learning Program for Computer Science and Software Engineering. In B. A. Nejmeh (Ed.), Service-Learning in Computer and Information Sciences: Practical Applications in Engineering Education (pp. 39–75). Hoboken, NJ: Wiley.

Olenick, P., & Edwards, J. E. (2016 ). Factors to consider when planning short-term global health work. Nurs Womens Health, 20(2), 203–209. doi: 10.1016/j.nwh.2016.01.003

Singer, A. (2013 ). Contemporary immigrant gateways in historical perspective. Daedalus, 142(3), 76–91. doi:10.1162

Appendix 1:

Demographics

Is this your first experience with Barnabas Task?

a. No

b. Yes

How many times have you worked with Barnabas Task?

a. 1-2 times

b. 3-5 times

c. 6 or more times

What was your role with the EMR app?

a. Healthcare Student

b. Healthcare Provider

c. EPICS Team

d. Translator (PUCMM or OASIS Student)

e. Clinic Organizer

Describe your major.

a. Pharmacy

b. Physician Assistant

c. Nursing

EPICS Team

Why did you select this project? What was your motivation behind selecting this project?

Name the top three non-technical learning experiences that you took away from the EMR project.

Name the top three technical learning experiences that you took away from the EMR project.

Comment on your overall assessment and grading of your performance throughout this project.

Did you participate in the trip to the DR?

a. No

b. Yes

Comment on your overall trip experience.

What did you learn from the PUCMM/OASIS students while working in the clinic?

Comment on the amount of time spent on devotions and reflection.

Did your faith change or grow? Comment on this.

Were you interested in going on the trip to the DR?

What prevented you from going on the trip?

Comment on your experiences of interacting with the healthcare students.

What suggestions do you have to improve the way the two teams interacted?

Did you participate in the EMR simulation in March?

a. No

b. Yes

What value did you find in this simulation?

How have you used the knowledge and skills from this course outside of the classroom?

Healthcare Students

Why did you decide to participate in this trip?

Name the top three learning experiences that you took away from this experience.

Comment on the amount of time spent on devotions and reflections.

Did your faith change or grow? Comment on this.

Comment on your experience with the EPICS team (those that went on the trip and those that did not).

What suggestions do you have to improve the way the two teams interacted?

Comment on your overall experience in the DR.

What did you learn from the PUCMM and OASIS students while working in the clinic?

Comment on your experiences using the EMR app to automate the patient care process in the DR.

What did you like about the EMR app? What would you improve or change?

Did you like the text boxes used for diagnosis?

a. No

b. Yes

Did you participate in the EMR simulation?

a. No

b. Yes

What value did you find in this simulation?

Healthcare Providers

What is your role and capacity of involvement in the clinic? Comment on your previous involvement with Barnabas Task medical clinics.

Comment on any benefits and challenges you had from your participation in this clinic.

Did you utilize the EMR app?

a. No

b. Yes

Describe your overall experience and impression of the EMR app. How did you find it useful? How could it be improved?

How do you think the app affected patient care?

Would you be interested in using it in the future?

a. No

b. Yes

Clinic Organizer

40.
What is your role and capacity of involvement in the clinic?

41.
Comment on any benefits and challenges you had from your participation with this clinic.

42. Did you utilize the EMR app?

a. No

b. Yes

43.
Describe your overall experience and impression of the EMR app. How did you find it useful? How could it be improved?

44. Would you be interested in using it in the future?

a. No

b. Yes

Translators (PUCMM or OASIS students)

45.
What was your role in the clinic? Comment on any previous experiences with Barnabas Task.

46. Comment on any benefits and challenges you had from your participation in the clinic.

47. What did you learn from the American students?

48. Did you use the EMR app?

a. No

b. Yes

49. Describe your overall experience and impression of the EMR app. How did you find it useful? How could it be improved?

50. Would you be interested in using it in the future?

a. No

b. Yes

Download the article:

Download (PDF, 4.24MB)

Community-Engaged Projects in Operations Research

 

Abstract

Community-engaged learning is not very common in technical fields, but including relevant projects in courses can make it feasible and successful. We present an implementation of an operations research course at a liberal arts college. Working with one of four nonprofit community partners to optimize aspects of their organization, students gained insight into relevant, real-world applications of the field of operations research. By considering many aspects of their solution when presenting it to community partners, students were exposed to some issues facing local nonprofit organizations. We discuss the specific implementation of this course, including both positive learning outcomes and challenging factors.

Introduction

Operations research, a “discipline that deals with the application of advanced analytical methods to help make better decisions” (INFORMS 2017), is used by many organizations. Southwestern University, a small liberal arts college, offers an operations research course cross-listed as business, computer science, and mathematics, which broadens opportunities for students to take computer science courses (Anthony 2012). While civic engagement is popular in colleges, its incorporation into the classroom is less prevalent in STEM disciplines (Butin 2006). Though some computer science courses incorporate community-engaged learning, it frequently occurs in a senior capstone experience (Bloomfield et al. 2014). An interdisciplinary course taken before the senior year can provide more realistic experiences in working with people from different backgrounds. Project-based courses are not uncommon in operations research; colleges are sometimes even paid by outside corporations for such projects (Martonosi 2012).

The operations research course’s popularity and increasing support on campus for community-engaged learning worked synergistically to have projects proposed by local community partners (nonprofit organizations) in 2014. The Southwestern University Office of Civic Engagement (OCE) helped facilitate these projects by aiding in the solicitation of partners, providing continuing education to the faculty member, and providing a student Community-Engaged Learning Teaching Assistant (CELTA), whose duties included serving as a liaison between student groups and community partners. The CELTA was a computer science major who had previously taken courses with the instructor and had worked for the OCE for multiple semesters. Together, the instructor and CELTA investigated the value that students found in the project experience, in terms of both more traditional goals of community-engaged learning and the content typical of an operations research course. In the four projects, students partnered with a hippotherapy organization, a local chamber of commerce, and two units on campus.

Methods, Projects, and Partners

Students engaged in a semester-long team project partnering with local nonprofit organizations to solve a problem in need of optimization. Four student teams, working both in class and on their own time, submitted a proposal, a poster with preliminary results, and a final report including an executive summary and full technical details. They also made a final presentation to classmates, the professor, and their community partners. The course is typically a student’s first introduction to operations research. Thus, students are learning the basics of the field while simultaneously applying the ideas presented in the course to their project with the community partner. Both quantitative and qualitative data were collected from students about their experiences, with approval from the university’s Institutional Review Board. Students were asked identical questions about their attitudes toward community service in general, taken from Bringle’s (2004) The Measure of Service Learning: Research Scales to Assess Student Experiences, before project groups were assigned and at the end of the semester, while final project reports were being prepared. All answers were given on a 1–7 Likert scale of likelihood (extremely unlikely to extremely likely) or agreement (strongly disagree to strongly agree). The qualitative data was collected from multiple sources, including meetings with the instructor and CELTA, peer and self evaluations, final exam questions, and course evaluations.

Two of the community partners came from area nonprofit organizations: Ride On Center for Kids (R.O.C.K.), a hippotherapy organization, and the Greater Leander (Texas) Chamber of Commerce. The other two partners were internal to the university: the Center for Academic Success and Records (CASAR) and the directors of the new incarnation of Paideia, an interdisciplinary curriculum program unique to Southwestern.

R.O.C.K. “provides equine-assisted therapies and activities to children, adults, and veterans with physical, cognitive, and emotional disabilities” (R.O.C.K.). R.O.C.K. aims to serve as many clients as possible while using limited resources (including staff, arena time, and horses) appropriately. Clients’ needs determine whether the therapy sessions are individual or small groups. Students formulated appropriate linear programs for modeling the constraints and objectives, and analyzed the solutions under various assumptions (such as the number of hours a horse can be used each day or week). They recommended that R.O.C.K. alter operating hours to better utilize resources while still serving the same number of clients and prioritize the acquisition of additional horses.

The Leander Chamber of Commerce (LCC) has four membership plans, with different prices and benefits. As a nonprofit, they want to be sustainable while providing value to their members. Students first used linear programming techniques to determine optimal pricing for each of the plans while keeping the same benefits, under the limiting assumption that members would stay on the same plan. They then used knapsack problem techniques to determine the ideal combinations of benefits in the plan that provide the most perceived value to the members for a given cost. As costs and perceived values change and new benefits are considered, LCC can use provided software tools to update offerings.

Currently at Southwestern, academic advisor/advisee assignments are made manually, a time-consuming and suboptimal process. Students worked with the Center for Academic Success and Records to convert their process into a flowchart, assigning measures for compatibility based on stated academic interest and predictors of transitional challenges. The assignment can now be considered as a transportation problem, maximizing the compatibility indicators of the entire incoming class while limiting the number of advisees assigned to any one advisor. The team used a Java program to parse data about students, fed that information to a tool called glpsol within the Gnu Linear Programming Kit (GLPK), to solve the transportation problem, and again used Java to present the output cleanly.

Beginning in Fall 2014, as part of a reconfigured Paideia program, all students are part of an interdisciplinary cluster, making connections across disciplines through a subset of required courses. There are numerous tradeoffs to be considered, for faculty, students, and the university as a whole, when considering the ideal number of clusters, courses, and faculty per cluster. Students developed an Excel tool to model these relationships that will be used by present and future Paideia directors in their decision making. Their recommendation of three new clusters per year provided an ideal balance of number of courses available to students and faculty in the cluster, while allowing for changes in class size in future years.

The creation of groups in a course project often poses an interesting dilemma. Each group had at least one person from each of the three predominant majors represented in the course: computer science, math, and business or economics. For the projects where it was anticipated that higher-level programming languages would be used (as opposed to Excel), multiple computer science majors were assigned. Students were required to complete a questionnaire with questions including their preferences among the projects, their willingness or ability to work with an off-campus partner, and published personality questions in a STEM text (Burger 2008). The instructor and CELTA then assigned groups, based on those responses and their prior experiences in the classroom.

Research on Student Experiences

In the following table, we report some of the statements that most students agreed or strongly agreed with. We also note that most disagreed with the claim “without community service, today’s disadvantaged citizens have no hope.”

Responses to the final survey were largely similar to the preliminary survey with regard to the number of students who felt an outcome was likely or agreed with a statement, but when quantified as described above, many of the averages for each question fell. (Given the small sample size, 21 students, we look more at general trends than actual numbers.) The other statements in Table 1 changed by at most 0.1 points.

The differences in the average responses are small. Students answering less enthusiastically (e.g., “somewhat likely” instead of “likely” or “agree” instead of “strongly agree”) may have felt no differently in the final survey and simply had a hard time discretizing their response. Alternatively, a slight decrease in enthusiasm in final responses may be indicative of end-of-semester fatigue. As students typically did not interact directly with clients of the nonprofit partners, they might not have been able to see the outcomes and benefits of their projects. They might have also recognized that many clients served by their partners are not socio-economically disadvantaged and perhaps not people whom they would see as “in need.”

Since team dynamics can play an important role in the success (or lack thereof) in any group project, students periodically evaluated the contributions of their group members. They rated each group member on a scale of 0 to 4, including themselves, indicating whether they were a team player, the amount of effort put forth, whether they were dependable, their intellectual contribution, and their overall contribution. Student were told that specifics would not be shared with the group members, but the instructor would be speaking with anyone who did not seem to be contributing adequately, in an effort to allow them to improve their performance. Additionally, evaluations would be considered in calculating each student’s participation grade, but except in extreme cases, would not affect the project grades. The provided instructions and reminder that it is highly unlikely that everyone is excellent at everything seemed to lead students to give considered answers. In addition, they wrote a single sentence for each group member (including themselves) about their overall impression of said member’s performance. These comments typically suggested most group members were pulling their weight. Sometimes their disciplinary backgrounds meant they were a stronger contributor in one area than another. For example, a student who had more accounting experience might be especially skilled at reading financial statements and explaining their contents to others who have more programming experience. This exercise, along with in-class discussions, seemed to help mitigate some of the tensions that occasionally arose with the differences between majors/backgrounds.

The final exam included questions eliciting the benefits and drawbacks of having a group project with a community partner. A few students felt the group project prevented them from learning additional course material because of the time devoted to working on the project. However, most enjoyed delving into a large and real problem.  One student noted that “it exposed us to another learning method,” another said through the projects students “saw applications of theory which reinforced the ideas learned in lectures,” and a third indicated that “‘What can I do with this class/theory?’ actually gets answered.” (In accordance with the IRB consent forms, student quotes are not being attributed to specific individuals.) While many people often think of the benefits of operations research first in terms of money (whether increasing profit or cutting costs), the projects helped students focus on other things that can be optimized, as illustrated in this response: “The group projects gave much more of a feel of the complexities of optimizing real world situations, particularly when profit is not the most important quantity to an organization.” Other students talked about the benefits of the project being in the “real world,” and of working in teams similar to their anticipated future work environments. A student summed up much of the motivation for doing the group project with community partners in the observation that “reading case studies or doing fictitious projects does not provide the same sense of urgency and rewards as doing a project for someone who can actually benefit from it.” The student comments echo many of the benefits purported in literature about community-engaged teaching, including deeper understanding of course material and the ability to transfer knowledge (Furco 2010).

Most drawbacks students reported were logistical in nature, either with their group members or community partners. Frequent concerns were difficulty scheduling meetings (with or without the community partner) and having access to information. One indicated that “people bringing different backgrounds was a benefit in tackling our project, but it was hard to balance the work and make sure everyone pulled equal weight,” which led to concerns about receiving a group grade for the project (cumulatively, twenty-five percent of the final course grade). Another stated that community partners “did not fully understand the benefits and applications an OR student can provide” and had nebulous expectations, whether expecting too much or too little. Only a few students indicated a concern that the project resulted in “less time learning concepts with the professor,” and most viewed the experiential learning as likely to be retained longer. Most students indicated a desire to keep this component of the course.

Just as the small sample size limits statistical analysis, the frequency of the course offering (typically once every two or three years) and the varying nature of the projects and partners limit meaningful longitudinal studies. One wonders whether such projects increase student engagement and satisfaction, possibly with positive impacts upon retention and graduation. Anecdotally, all non-visiting students in the course have in fact graduated from Southwestern, but given that the students were typically juniors or seniors, that is unsurprising. Likewise, with the variety of majors enrolled and the differences in the projects, other assessments of impacts on overall academic performance are limited. However, in the future it may be possible to determine whether there is a correlation between students’ performance on exams and the specific skills and techniques used in their projects.

Discussion: CELTA, Community Partner, and Instructor Reflections

Each team met with the CELTA three times. The first meetings were primarily introductory in nature. Each group had held its first meetings with community partners and was involved in initial planning stages. The two groups working with on-campus partners both had a strong start, with detailed plans in place to find their solutions. Likely because of the connection to campus and the professor’s connection to these projects, the expectations were communicated more clearly than those tied to the projects that were based off campus. In contrast, the off-campus partners had more of a vision to be interpreted than a concrete plan to be executed. Though students are often more comfortable with precise directions, the real-world experience of uncertainty and ambiguity is quite valuable.

In the second round of CELTA meetings, group members were still excited but now had some concern about partially completed projects and looming deadlines. The groups had all made substantial progress and were working on posters to be presented at a campus symposium. Three of the four groups were now experiencing more of the challenges of a real-world project, where the scope or goals can change over time. The Academic Advising group felt that some of the partner’s requests were growing beyond the original requirements, but had difficulty scheduling face-to-face meetings to discuss the limitations. The Paideia group had the fewest communication obstacles, likely because the primary contact is a professor in the math department. As such, many group members already had a working relationship with her, and would often drop by her office for immediate feedback.

At this point, groups had already considered the obvious stakeholders, but were now asked to reflect further on the non-obvious stakeholders affected by their project, which can be equally important when modeling problems. The Academic Advising group had identified students and professors as the obvious stakeholders, with counseling services and parents as non-obvious stakeholders; both are concerned with students’ overall well-being and stress levels, which can be impacted by advising. The Paideia group noted students as the obvious stakeholders, and considered professors as non-obvious stakeholders, due to teaching load and leave considerations. The projects with off-campus partners, not surprisingly, had different stakeholders, with interesting implications. The member working with R.O.C.K. identified the horses as a non-obvious stakeholder. While meeting the needs of obvious stakeholders (the clients, and if they are minors, their parents), it is important to ensure that the horses do not get overworked. Accordingly, group members had to familiarize themselves with seemingly restrictive regulations that R.O.C.K. adheres to concerning the number of hours a horse should work per day and needed to incorporate those into their problem formulation and solution. For the LCC, member organizations are obvious stakeholders, and group members identified residents of Leander as non-obvious stakeholders, since each new resident of Leander receives a directory of businesses that are chamber members, and said membership confers certain credibility. In all groups, students realized that projects can have far broader impacts than initially considered.

The final round of CELTA meetings occurred toward the end of the project, while groups were finalizing their linear programs and solutions and writing their final paper. The completed project portfolio was provided to the instructor and the community partner, and each group gave a final presentation to the entire class, inviting their community partners to attend. While not all partners were able to attend, the possibility that the partner would be present ensured that students had to thoroughly motivate the assumptions made for the project and explain why they were reasonable. All groups already had experience presenting as a team from the campus symposium. Additionally, the poster presentations had increased student enthusiasm when they realized how interested their peers and faculty were in their projects. This was especially true for the groups working with on-campus community partners; students and faculty were able to ask specific questions because they were already familiar with Paideia and the Academic Advising process, which alerted members of these groups to issues with their solution that they might not have previously considered. Many group members talked about broader implications of their projects. A Paideia group representative considered optimizing Paideia to be part of the legacy he leaves behind upon graduation. The R.O.C.K. representative appreciated that the project had relevant business applications, and was excited to be able to apply the knowledge learned in the real world. Overall, group members expressed the opinion that it was a positive, albeit challenging, experience.

During the semester, morale was often correlated with the level of engagement of the community partner; groups that maintained good communication with their partner felt more positive about their projects. Communication challenges occurred with both on- and off-campus partners. While the instructor reassured students that projects could earn good grades despite incomplete partner information (with students making reasonable assumptions based on the information they did have), students naturally wanted to deliver products that met their and their community partner’s expectations. Groups that believed their partner would implement the proposed solutions were more satisfied with the experience; yet implementation was not always feasible for the partner. Not surprisingly, when a community partner is more invested in a project, a group often does better work. Accordingly, in future offerings the instructor will have more up-front discussions with both the students and the partners about how to facilitate such communication and commitment.

All community partners gave positive feedback about the work completed by the students. The LCC president has benefitted from the tools (e.g. Excel spreadsheets that are easily updatable without any operations research background), the analysis from students, and recommendations from the group about plan offerings and costs. Likewise, R.O.C.K. appreciated the information and made plans to present it to their board. However, like many nonprofit organizations staffed primarily by part-time employees and volunteers, R.O.C.K. experiences frequent staff turnover; the main project contact left the organization shortly after the project was completed, so follow-up has been limited. Likewise, a new director for the Paideia program was selected from the faculty shortly before the class project was completed; she has since used the spreadsheet and tools created and has given positive feedback.

The tools for assigning advisors to advisees require ongoing updates and maintenance by people with sufficient Java knowledge to reflect annual changes such as the number of advisees an advisor currently has. In addition, since the students who need to be assigned are new each year, there is some data processing involved in converting the information students provide on a web form into the format needed for the Java programs and GLPK. Full implementation has not yet happened for various reasons unrelated to the course, but there is support from CASAR staff for eventual usage, and the instructor is willing to do the updates.

One final exam comment was positive overall about the project, but the student wished that the group had “had more time to do more.” This issue of the semester-long lifetime of the project is an issue the instructor continues to struggle with. While the deliverable at the end of the semester is expected to be useful to the community partner, often some continued involvement with the partner after implementation would be ideal. Some students may be able to continue the partnership as an independent study, allowing the community partners to have the model refined as they realize limitations, whether due to assumptions the students had to make or to factors that were not readily known in the original problem.   

We believe that these projects are in fact rightfully viewed as partnerships, with students acting in a consulting role for the organizations. While there are inherent dangers in community-engaged learning programs that try to “fix” what is “wrong” with a community (Cooks 2004),  the partners themselves responded to offerings of these optimization services, and they chose the problem or issue. And of course they also remain in control of how the resulting information is used. Though the instructor and students did have a role in deciding which projects were selected—which does confer a degree of power (Mitchell 2008)—choices were largely based on suitability of the problem for the course (i.e. an optimization problem, not a website redesign). The concern about developing tools without providing people and resources to maintain them long-term, paralleling the concerns of do-gooders who impose their will on others, is worth acknowledging (Illich 1968). We are up-front with the community partners about the time span and limitations, aim to provide useful tools that are easily modifiable, and typically use software (frequently Excel) that their organization already uses.

Partners greatly valued the community-engaged learning relationships with the university, but, consistent with the literature, logistics (student schedules) and communication issues are not easy to overcome (Vernon and Ward 1999). While partners were invested to some degree in the projects, the projects were not their highest priority (nor were they expected to be). The instructor can be more proactive in future years about outlining the expected time commitments and flexibility needed to both the partners when selecting projects and the students when they register for the course. Having tangible results from the 2014 offering may make it easier to solicit future projects, and partners may be more invested when they have a fuller understanding of expected benefits. 

Conclusion

This Operations Research course was a productive and positive experience for students and community partners alike. Students benefitted from the hands-on project that required them to apply their knowledge outside of the typical classroom, and gained experience working and solving problems in a large group. The Community-Engaged Learning Teaching Assistant and instructor witnessed student learning in and out of the classroom, and they were able to educate students about community-engaged learning in general while further motivating course content. Finally, the community partners each received a solution to a problem from skilled students, which further strengthened the partnership between Southwestern University and the Georgetown community.

The instructor is committed to continue offering this course with nonprofit partners. Since ideally each project ends with a “solved” problem, partners will often differ from year to year, unlike many community-engaged learning courses which are able to work with the same partners for extended periods of time. Yet organizations may have new problems in mind that are in need of optimization, and can be partners in future offerings. Including presentations from community partners early in the semester could be beneficial, since passion about a project often leads to stronger teamwork, dedication, and enthusiasm about the experience. Though there will always be logistical challenges in courses of this nature, offering a community-engaged learning component in an operations research course is a worthwhile endeavor that results in beneficial learning outcomes and hands-on experience for students, and in tangible products for the partners.

Acknowledgments

Thanks to Dr. Sarah Brackmann, Director of Community-Engaged Learning at Southwestern University, and to the community partners and their primary contacts: Bridget Brandt (LCC), Jerry Fye (R.O.C.K.), Dr. Alison Marr (Paideia), and Kim Morter (Center for Academic Success and Records).

About the Authors

Barbara M. Anthony, (anthonyb@southwestern.edu), the instructor for the operations research course, is an Associate Professor of Computer Science at Southwestern University in Georgetown, Texas. She received her PhD in Algorithms, Combinatorics, and Optimization from Carnegie Mellon University in 2008. She is active in the computer science education community, with a particular interest in introducing students from underrepresented groups to the discipline, and finds ways to bring her theoretical computer science interests into multiple courses.

Kathryn M. Reagan, (kathryn.m.reagan@gmail.com), the CELTA for the operations research course, is a class of 2016 graduate of Southwestern University in computer science. She is currently a consultant software developer for ThoughtWorks. Her passions lie in social and economic justice and computer science education, and she loves finding ways to work within the intersection of those passions.

References

Anthony, B. 2012. “Operations Research: Broadening Computer Science in a Liberal Arts College.” Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (SIGCSE ’12): 463–468.

Bloomfield, A., M. Sherriff, and K. Williams. 2014. “A Service Learning Practicum Capstone.” Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE ’14): 265–270.

Bringle, R., and M. Phillips. 2004. The Measure of Service Learning: Research Scales to Assess Student Experiences. Washington, DC: American Psychological Association.

Burger, E. 2008. Extending the Frontiers of Mathematics: Inquiries into Proof and Augmentation. Hoboken, N.J.: Wiley.

Butin, D. 2006. “The Limits of Service-Learning in Higher Education.” The Review of Higher Education 29 (4): 473–498.

Cooks, L., E. Scharrer, and M.C. Paredes. 2004. “Toward a Social Approach to Learning in Community Service Learning.” Michigan Journal of Community Service Learning 10 (2): 44–56.

Furco, A. 2010. “The Engaged Campus: Toward a Comprehensive Approach to Public Engagement.” British Journal of Educational Studies 58 (4): 375–390.

Illich, I. 1968. “To Hell with Good Intentions.” Address, Conference on Inter-American Student Projects (CIASP), Cuernavaca, Mexico, April 20, 1968.

Institute for Operations Research and the Management Sciences [INFORMS]. 2017. “What is Operations Research?” https://informs.org/About-INFORMS/What-is-Operations-Research (accessed May 27, 2017).   

Martonosi, S. 2012. “Project-Based ORMS Education.” In Wiley Encyclopedia of Operations Research and Management Science, J. Cochran, ed. Hoboken, N.J.: John Wiley & Sons.

Mitchell, T.D. 2008. “Traditional vs. Critical Service-Learning: Engaging the Literature to Differentiate Two Models.” Michigan Journal of Community Service Learning 14 (2): 50–65.

R.O.C.K. (Ride On Center for Kids). www.rockride.org (accessed May 27, 2017).

Vernon, A., and K. Ward. 1999. “Campus and Community Partnerships: Assessing Impacts and Strengthening Connections.” Michigan Journal of Community Service Learning 6: 30–37.

Download (PDF, 269KB)

 

Students as Partners in Curricular Design: Creation of Student-Generated Calculus Projects

 

Steve Cohen, Roosevelt University
Melanie Pivarski, Roosevelt University
Barbara Gonzalez-Arevalo, Hofstra University

Abstract

In recent years advanced undergraduate students have developed projects for our redesigned Calculus II classes. Our student designers create new mathematics projects and present their work at conferences and in local talks. They are often mathematically early in their college careers, and so we can involve students of all levels in research projects.

Our course redesign affected three groups of students: those taking the class, those designing projects for the course, and embedded tutors. This qualitative study examines how the second and third groups of students benefited from their experiences and how we can modify our program to improve it. Evidence was gathered from interviews, surveys, and observation of student research work and its implementation in the classroom. Tutors reported more confidence in their knowledge of calculus and insights into teaching it, and project designers experienced benefits similar to that of a traditional undergraduate research experience.

Introduction

The extensive use of undergraduate research in mathematics is fairly recent, dating back to the 1980s with the widespread introduction of the NSF-funded Research Experience for Undergraduates programs (Lopatto 2010). Most of these experiences are designed for advanced undergraduates who are in their junior or senior years, and they are often used to help prepare these students for graduate study. By using undergraduate students to develop projects for use in a Calculus II classroom, we can give freshmen and sophomores the opportunity to work on research. The purpose of their research is clear; our students are motivated by helping their peers learn. Developing the calculus projects as well as using them to teach calculus helps to contextualize the mathematics curriculum, which is seen as “a promising direction for accelerating the progress of academically underprepared college students” (Perin 2011).

The use of undergraduates as embedded peer tutors is common; see e.g. Evans et al. (2001) and Goff and Lahme (2003). Tutors attend most classes, and depending on the instructor, sometimes work with students during the class sessions. Tutors can connect more deeply with the material, increasing their calculus skills as well as their ability to communicate and collaborate effectively.

In order to avoid ambiguity, we use “embedded tutors” or simply “tutors” to refer to the embedded peer tutors and “project designers” or “student researchers” to refer to students who, after completing Calculus II themselves, worked on researching a project for use in a future Calculus II course. We refer to students who were currently taking the course simply as calculus students.

In the section Connecting Students to the Course, we briefly describe our Calculus II course and the overall role of the tutors and project designers. As this varied by semester, we elaborate with more details and context in the Experiences and Results section. In the section Curricular Design as Student Research, we discuss definitions of student research that occur in the literature and how these connect to our curricular design. In the Methodology section we describe the methodology used in our study. In the Experiences and Results section, we delve into the results of the study, providing and elaborating on themes found in the student responses. In the Conclusions section, we summarize our results with a list of best practices.

Connecting Students to the Course

At Roosevelt University, semester-long projects with a civic engagement component became a regular part of all sections of Calculus II in Spring 2010 (González-Arévalo and Pivarski 2013). Calculus projects help students explore STEM applications, acquire library research skills, and develop communication skills. Beginning in Fall 2010 each class was assigned an embedded undergraduate tutor who attended class at least once a week and helped students in and out of class. Starting in Summer 2011, undergraduate research students had the opportunity to work on designing materials for class projects. Their work involved picking a topic of civic importance, finding appropriate data sources, considering issues related to calculus, and linking these together. There are many possible outcomes for these projects: use in a Calculus II class, honors theses, research talks, and starter ideas for more advanced mathematical research. We consider all of these to be successful outcomes. We also had some unsuccessful outcomes where students failed to progress.

This course redesign originally developed as a result of our involvement with the Science Education for New Civic Engagements and Responsibilities (SENCER) project. Over the years, our continued involvement with SENCER helped us incorporate students as partners in our curricular design. At the end of 2013 we published a project report (González-Arévalo and Pivarski 2013) detailing the redesign of the course and what we then thought would be the benefits. The current paper provides a qualitative assessment of the newest components of this redesign, namely calculus project development by advanced undergraduate research students and the incorporation of embedded tutors. We provide a description of how we use the embedded tutors in class, as well as how students work on the design of calculus projects. Some of this is explained in our aforementioned project report but we have included it here also for the convenience of the reader.

Embedded Tutors

Each semester at Roosevelt University there are one or two Calculus II sections, each with between nine and thirty students. Because there are only one or two calculus tutors per semester, we do not have a formal tutor training process. Each section instructor informally trains their own tutor. Typically, an experienced instructor acts as a secondary faculty resource. The designers do not work directly with the tutors, except in the cases where an individual student acts in both roles. In that instance, the tutor has a deep knowledge of the goals of the calculus project; we elaborate on this in the section Theme A: Insight into better learning processes. We intend for tutors to

  • attend all classes,
  • hold regular office hours,
  • test out the computer labs ahead of time, and
  • work with groups both inside and outside of class.

In practice, we often are unable to find qualified students whose schedule allows them to attend all class meetings, and so we loosen the requirement to attendance at least once per week. Tutors are not needed as graders, as the homework is online. Instructors grade weekly quizzes by hand to gauge where the class is mathematically. Instructors also grade the project parts. Tutors are student workers paid hourly; their salary is part of the institutional budget, often including Federal Work Study.

The use of the tutor varies by instructor. Some embedded tutors help students when they are working on problems during class, but others merely observe the class. When they are made available, some tutors try the class’s computer assignments ahead of time. The tutors always help out during class periods involving computer use.

Project Designers

At Roosevelt University many students transfer in or take calculus their sophomore year, which means they are not ready for a traditional undergraduate research experience until their senior year. Therefore, students need to have research opportunities requiring less background knowledge. Project creation allows student researchers to choose an area for the calculus application.

In the initial course redesign process, research students compiled a literature review on calculus projects. This review and previous semesters’ calculus projects provide a foundation for our project designers. Although they are mathematically constrained to construct a modeling project for a calculus class, designers independently explore an application of their own choosing. We ask that it involve actual data and ideally a social justice component. As they develop their plans, we meet weekly with the research students to discuss their ideas, progress, and challenges. During the week, they work independently, although we are always available either in person or by e-mail. At Roosevelt, students are funded through an NSF STEP grant (Science, Technology, Engineering, and Mathematics Talent Expansion Program) shared with the sciences, and through our university’s honors program.  At a school without funding, project design can act as an independent study project. 

Some students had their own ideas for projects, and others modified existing projects. For example, one student found a project that involved studying population growth through a series of biology experiments. She wanted the project to be compelling to the many science majors taking the class. The original project involved studying population growth in simple life forms and in humans. Since growing cell cultures involved more lab time than was realistic for a calculus class, she arranged to use some existing yeast data from one of our biologists’ research labs. She investigated curve fitting with MAPLE, split the problem into discrete assignments, and structured the investigation to fit the topic schedule of the calculus course. We helped her with this process over the summer and made adjustments during the semester that we used her project.

Design typically happened over the summer, but it sometimes occurred during the semester.  At any given time there are at most two students working on design.  Although they had access to them, the designers did not formally review past projects, and they did not have formal discussions with tutors.  They instead drew informally from their own experiences and anecdotes from their friends.  The designers whose projects were used in courses saw the results of the students’ work through a STEM poster session.

Curricular Design as Student Research

The work that our students do creating calculus projects is a distinctive research experience that has much in common with a traditional undergraduate research experience. In the report “Mathematics Research by Undergraduates: Costs and Benefits to Faculty and the Institution” (MAA CUPM 2006), the Committee on the Undergraduate Program in Mathematics of the Mathematical Association of America lists four characteristics of undergraduate mathematics research:

  • The student is engaged in original work in pure or applied mathematics.
  • The student understands and works on a problem of current research interest.
  • The activity simulates publishable mathematical work even if the outcome is not publishable.
  • The topic addressed is significantly beyond the standard undergraduate curriculum.

Although these guidelines were originally designed to describe a traditional mathematics research project, they apply in many ways to the work that our research students do. Our research students create projects for use in a Calculus II classroom, and so theirs is more of an applied curricular design research project than a traditional mathematics research project. Because of this, the first item is only partly true; the work is often adapted for a Calculus II classroom from another source. The second item holds, and it was a significant motivator for our research students when they chose the topic of their project. The third holds in the sense that their work, when completed, is made public through use in our classrooms. This is similar to an applied project being used by a company. For our students, two of six projects reached this point. Others either lacked time or good data sets or transitioned from a Calculus II project into applied math research for an honors thesis. The final point applies in the sense that it takes them outside the traditional curriculum. While the mathematics might be found in an undergraduate math modeling course, the act of designing mathematics activities that relate to a social justice theme provides a deeper challenge. At the same time, this allows our student project designers the chance to work on research very early in their undergraduate studies.

Dietz (2013, 839) defines three levels of student research activities:

Guided discovery: In these classroom activities, students make step-by-step progress toward a standard (but unknown to them) mathematical formula, or other result, via open-ended, but guided questions.

Independent investigation: In these multi-day activities, the instructor asks open-ended questions that require independent exploration by the students. Results may not be surprising to professionals, but they cannot be easily found in the literature.

Scholarly inquiry: In these intense activities, students engage in scholarly work that is typical of a given field of inquiry.

Our research students engage in curriculum design, researching applied areas and educational theories in order to develop a guided discovery project for the Calculus II class. The process of creating a new calculus project is an independent investigation; for one of the students it moved beyond this into the area of scholarly inquiry where she analyzed the efficacy of her project. For another, her work extended beyond that of a typical Calculus II project and became scholarly inquiry in the area of actuarial science.

There are multiple layers of learning, where advanced students progress beyond Calculus II while helping students currently taking Calculus II. When surveying the literature, we have found a few instances where advanced students created mathematics materials for introductory students. In Duah and Croft (2012), four mathematics students worked with lecturers to create materials for a module in vector spaces and complex variables. The authors noted the call for student-led curricular design in the UK (Kay et al. 2007; Porter 2008), which other fields have responded to. The authors also noted that there was a paucity of literature on student-created mathematics curricula. At least two papers were written in response to Duah and Croft (2012). In Hernandez-Martinez (2013), two students at an English university worked to create mathematical modeling teaching and assessment tasks for a second-year mathematics for engineers course. In Swinburne University of Technology in Australia (Loch and Lamborn 2015), a team of engineering and multimedia students created videos for engineering students to demonstrate how mathematics is used in engineering. In Pinter-Lucke (1993), the program of Academic Excellence Workshops (AEW) at Cal Poly Pomona involved STEM upperclassmen as leaders of cooperative learning-based workshops for underclassmen in courses ranging from college algebra through calculus. Student facilitators selected materials and led weekly problem sessions. The facilitators met weekly with faculty who were teaching the course, and they went to an intensive two-day training session. Although the paper does not mention whether the problems are student-created or student-selected, the process of choosing appropriate course materials is an advanced one, and so this is a notable example of students contributing to the enhancement of mathematics curricula.

Some institutions involved with the SENCER project are also working with students to create curricular materials, notably in biology (Goldey et al. 2012), where students are used to create and update labs. At Guilford College students are creating a new course as a part of their independent study,  and at New England College a proposal is being piloted.  At the United States Military Academy students are doing in-depth assessment research of the university’s curricular design across the STEM disciplines (United States Military Academy 2014).

In many of these cases, a small number of students were selected to participate in this work, but without a particular common experience to draw upon. In our project we bring students into the experience systematically and intentionally, which leads to the following multi-level learning experience: students have the initial experience of working on a Calculus II project as students in the class, then are given the opportunity to work as a peer tutor or project designer (or both). Their subsequent work then impacts the next set of potential tutors and designers. The depth and detail of the work done by our project designers appears to go beyond that of the AEW leaders, and so the combination of multi-level learning with the depth of experience appears to be unique to our endeavor.

Methodology

In this qualitative study, which received IRB approval, we interviewed each student with several open-ended questions (Appendix A) to get them to reflect on how they were affected by the experience.

We created a survey after we interviewed a few of the students, and it included questions that were based on the interviews. The survey itself was anonymous, and it was used to corroborate the interviews. This qualitative study involves a relatively small number of potential subjects: six project designers, one of whom was also a tutor, and eight additional students who were embedded tutors. Eight students, four of whom were project designers, agreed to be interviewed; four of these also completed a follow up survey. Two individuals, including one project designer, completed the survey, but not an interview. Four did not respond to our contact request. Due to the small sample size it was not possible to conduct a quantitative study of these results, and we have therefore avoided all numerical data throughout the paper (since it would not be statistically valid). Instead, we present the results of the qualitative study of the interviews. The survey was only used to triangulate the results of the interviews.

To categorize the responses, the three authors independently reviewed the interview transcripts and labeled responses according to a variety of categories (Appendix B). The labels were compared and discussed until consensus was reached. The results are organized into three main themes as follows:

Theme A: Insight into better learning processes.

Theme B: Insight into applying mathematics/calculus.

Theme C: Feedback on improving the experience of embedded tutors and researchers.

Experiences and Results

In the first part of this section we will describe some of our observations made as course instructors and research advisors. In the second part of the section we will concentrate on the actual results of our interviews.

Experiences

Overall, our experiences have been positive. While some of our embedded tutors merely benefited from a review of calculus, others developed into expert teachers. All students surveyed confirmed that they gained in some way in varying amounts.

At the beginning, we hoped that the use of tutors would contribute to a sense of community among the students in the class and in our major. We also hoped that the class’s mathematical skill level would increase along with the tutor’s mathematical skills. We hoped for smoother computer labs, smoother group dynamics during the project, and a source of peer advice. Two of the tutors explicitly commented on the increased sense of community; we observed this as well, both in the classroom and among the tutors. Due to the small number of class sections observed it was difficult to discern whether embedded tutors consistently improved the mathematical skill level of the class and to assess their group dynamics. But tutors had a noticeable effect on the computer labs; these benefited greatly from the extra support. The amount of peer advice given varied by tutor; some of them commented on this in the interviews. Students in sections where the embedded tutors helped during the class period appeared to be more likely to work with the tutors outside of class.

There has not been a good mechanism for class feedback on the tutors; an online survey had a low response rate, but informally they praised tutors who were actively involved.

Our experiences with student researchers have also been mixed. They have definitely learned the difficulty of finding data, since much of what is found online is processed data that give only means, medians, and standard deviations rather than raw data. They found that government sites are usually a good data source. As a result of their work, we used two student-created projects in our course; these are on modeling population and modeling air pollution. Those student researchers gave talks on their projects, both internally and externally. Two students developed more involved research projects on actuarial and head injury models that were not used in class because they were too advanced for a Calculus II class but which resulted in internal and external talks. Two projects (population, actuarial modeling) developed into honors theses, with the first thesis also studying the impact of the population project on the class using it. Two projects were not finished. One of the student researchers, working on temperatures, was stalled in the data collection stage, and did not relate the topic to calculus. The other, working on planetary motion, had planned activities but lost the plans in a move. After this, we started making students type up their results part-way through their research project to prevent the loss of work.

In our experience, project designers have the best results when they fill out weekly timesheets rather than being paid in a lump sum for their summer research. Timesheets appear to help with their pacing and accountability. In a situation where a designer is working in an independent study, the structure of the independent study course can be used to aid in pacing.

Results

The student interviews indicate that the students benefited from their experience as tutors and designers as well as from working on the Calculus II projects. They also provide valuable feedback on the curricular design. Note that we have removed words such as “Uh, um, like” as well as repeated phrases from the transcription quotes without explicitly labeling each occurrence.

Theme A: Insight into Better Learning Processes

This theme encompasses the students’ sense of themselves as learners and tutors, how math instruction is enhanced by students working on open-ended problems, and the components of effective project design. All of the tutors and designers report gains in their understanding of calculus and in becoming better students themselves. All appreciate the value of a required Calculus II project.

Tutors and designers put considerable thought into what students need to be successful. All of the tutors helped with the technology. One noted that they wanted students to see that the computer is doing something you can do by hand but just much faster. Tutors noted the value of learning to work in teams and that talking about a project is a good way to communicate to outside people what you learned in the class. Tutors noted the value of sitting through the class a second time. They were able to work on their problem areas and to look for connections among the topics and applications. Having experienced the challenge of working on a project that is more open-ended than a typical homework problem, they are in a position to coach students through the process. One tutor spoke at length about the psychology of a student facing a difficult subject. Knowing that their tutor struggled with calculus when they first took the class can reduce the student’s own stress and self-doubt.

Project designers tried to include elements that connected naturally to particular calculus concepts. For example, population growth naturally associates with differential equations. But more importantly they tried to make the project connect to students’ own majors such as biology. The project designers discussed how they had to think about what calculus topics students needed to know and how the project could help them with difficult concepts. One project designer explained that conceptually, integration is difficult for students, and so he wanted the project to connect integration to a real life problem. They are interested in making the topics current such as using calculus to study greenhouse gasses. By putting more emphasis on a meaningful situation, students would naturally move away from a more mechanical view of calculus.

Several tutors viewed the project as motivating interest in math. Previously their math classes involved memorization and refinement of processes. As embedded tutors they appreciated a mathematically relevant context. One said, “I think that it was really interesting getting to do lots of different things, but I also think that it is something that students talk about especially within the same degree program. So if we did something that was more biological, population based… one semester when I had a classmate who did something that was more ecological, like the oil spill one, we could have those conversations about how we’re applying the same skills in a very sort of different context.”

It is evident that tutors and creators think a lot about the students. They care about whether the project is feasible and relevant to student interests. The majority of the students in Calculus II are science majors, so project designers looked for projects that related to biology and chemistry, as we do not offer a physics major at our institution. Typically, projects are related to an important social issue (e.g. climate change and overpopulation). Several tutors expressed empathy for the students and were motivated to help students practice, find related problems in the homework, and discover new ways to explain things.

Tutors took advantage of their unique relationship with the students. Tutors know what the students are hearing from the instructor; they can fill in gaps from the instructor to the students and can also give some of the students’ perspective back to the instructor. This advocacy for the students helps the instructor better understand the needs of the students. The tutor’s view is different from the instructor’s; their recent mastery of the material helps them to understand the students’ thought processes. Students often felt more comfortable talking to a peer.

One tutor had designed the project that was being used that semester.  This experience was especially fruitful, as they had thought very deeply about what they wanted to include in the project, how students learn, and where they were lacking in skills. They reported that this greatly increased their effectiveness as a tutor for the course; this self-reporting is consistent with our observation.

Theme B: Insight into Applying Mathematics/Calculus

Our main motivation for incorporating projects in Calculus II is to give all students the ability to talk about calculus and its uses. The project challenges students to think about the mathematical concepts in a contextualized situation that requires imagination and technological assistance. Our tutors and designers reflected about their time as calculus students, both here and elsewhere, in their interviews. Calculus II students must communicate among themselves about mathematical modeling in order to successfully complete the project. Many cited this communication as crucial.

One described group work in their previous calculus class at a different school: “It was never actually going out into the world and presenting your findings and being knowledgeable of what you were talking about, so I liked that as a component.” One said their experience as a Calculus II student here helped them talk to professionals at a job fair.

The project designers’ reflections deepened when discussing the thinking that went into designing a project. Project designers looked for ideas that were feasible for Calculus II students to complete in a semester. Designers wanted their projects to be socially relevant and therefore searched for an interesting area and then had to deconstruct it; one chose to study head injuries and came across the head injury index. That led to a new kind of analysis for her, working backwards from a formula to work out its derivation. The designers intended for students to experience how a model may be limited, but they still wanted students to make valid inferences about what formulas would be reasonable to try. One designer noted his own growth as a student through understanding why concepts are true rather than simply accepting them as an established principle.

The project designers applied knowledge acquired since having had Calculus II. One, an actuarial science major, designed a project using mortality tables. Reflecting on the project done and the project design led to the problem of data. The projects needed some publicly available data to analyze. They could see that the data used when doing the project as a Calculus II student had problems. Most of the designers expressed awareness of the difficulty of doing a project with real data, in particular, finding a good source and dealing with flaws in the data themselves.

There is consensus among the designers that the project brings value to the class. It gives insight into how calculus can be applied in the real world, and the learning that is needed to navigate the project provides an incentive for students to learn more about calculus itself.

Theme C: Feedback on Improving the Experience of Embedded Tutors and Researchers

Tutors and researchers gave feedback on how to run the different activities. The tutors felt strongly that more preparation and better coordination between instructors and tutors was needed. They gave suggestions about the structure of the class and insights on the value they should bring to it. Tellingly, the project designers did not express concerns about what was expected of them. Their biggest concern regarded the difficulties of finding good projects, particularly those with usable data sets. Because the designers met regularly with their research mentor, they remained informed of the goals and expectations of the project.

Most tutors saw the value and importance of integrating technology into the class, but most did not feel that their skill level improved while tutoring. Many pointed out the need for more training for students, tutors, and instructors. The tutors believe that students in the class need more formal instruction on using the software, noting that much class time is spent troubleshooting the difficulties students are having or getting them started. The tutors felt that more training for them would improve their effectiveness, as they were unable to answer some questions students had. Finally, there are indications that the instructors also need additional training, both on the software being used and on the way to utilize the tutors effectively. In some cases the instructor relied on the tutor to troubleshoot any problems arising with the software. Most tutors felt instructors only explicitly engaged them when technology was being used in that day’s class. In fact, many of the tutors were not active during class unless there was an activity involving computers.

It is not surprising then that communication was the most cited concern among tutors. Several of them said they wished they knew more about the instructor’s goals. The true value of the embedded tutor is to act as a partner of the instructor, and for this he/she needs to be aware of what the instructor is trying to accomplish. Some tutors tended to hold back and not be proactive about helping, in part because they had no direction and in part because of their own inexperience and lack of training.

Many noted the value of having the time structured so that tutors are available to students both in and outside of class. Opportunities to be active in the class were important to the tutors, though some needed more prompting from the instructor. This suggests that some changes in the structure would help facilitate the tutor’s activities. Possibilities include more training involving all members of the team, regular meetings between tutor and instructor where plans for the class are discussed, and a set of prompts for the instructor to help guide the tutor.

Conclusions

Our experiences with student researchers mirrored those of others, even though our student research had a curricular focus instead of a mathematical one. In Seymour et al. (2004), a survey of seventy-six student science researchers at four different liberal arts institutions was compared with literature from fifty-four different papers on hypothesized benefits of being a student researcher.

They found that students reported gains in many areas, including confidence in their ability to do research, finding connections between and within science, their organizational and computer skills, their enthusiasm, enhanced resumes, and their attitudes towards learning and working as a researcher. In our study, we also found these gains, giving evidence that this type of student research project has many of the benefits of a traditional research project.

The main advantage of research with a curricular focus is the possibility for students to work when they are just beyond the calculus level. In our study, designers and tutors gained a deeper knowledge of how to apply mathematics and use technology. Both reflected on what makes a good teacher, indicating this type of experience could greatly benefit undergraduates who are interested in teaching. They also provided thoughtful comments on how to improve the program, most notably the need for consistent communication between tutors and instructors.

4.1 Best Practices for Incorporating Students in Curricular Design

Given the extensive amount of research on embedded tutors, we will concentrate primarily on best practices for student researchers.

  • Meet student researchers and tutors at least weekly.
  • Be available for tech support, orienting all students to new software.
  • Pay students using timesheets rather than lump sums.
  • Encourage researchers to become embedded tutors for the course (both before and after creating a project).
  • Have a set of background literature, including previously used projects, available for new student researchers.
  • Don’t be too prescriptive. Let them brainstorm ideas and act as a sounding board for them.
  • Have at least two students working at the same time; they can give feedback to each other, and bounce ideas off each other.
  • Communicate your expectations to help them steadily progress.
  • Use file sharing (Dropbox, iCloud, etc.) to prevent the loss of student work.
  • Proofread and give feedback on projects and talks. Be supportive and encouraging.
  • Make students aware of speaking opportunities (with enough time to write an abstract, to plan a trip, etc.).
  • Provide internal venues where they can present their work.
  • If the topic gets too deep for calculus allow it to become a more traditional research project.

Recommendations for Further Study

We would love to see a quantitative study on our style of design process. For this, a large university or community college would have to undertake these activities in Calculus II or a similar course. We are also interested in more studies on the impact of doing research early on in college. In our specific work, it would be interesting to increase interactions between the embedded tutors and the project designers.  It would also be interesting to have new project designers formally review old projects.  This would structure their introduction to the design process and help them to think critically about issues involved in the design.  Similarly, when possible, one could have the tutors formally review the current semester’s project in the week prior to the semester as a form of tutor training.

Acknowledgements

We would like to thank Amy Dexter, Bethany Hipple, Sherri Berkowitz, and Amanda Fisher for giving us pointers on the qualitative research process. We would like to thank the SENCER project for the initial impetus to redesign our Calculus II course. Thank you to the reviewers for helpful feedback. Thank you to the NSF STEP grant and the honors program for supporting the student researchers financially, and to the Provost’s office for providing support for some travel and a student worker. Thank you to Janet Campos for her work transcribing the interviews.

About the Authors

Steve Cohen is an associate professor of mathematics at Roosevelt University. He teaches courses to both majors and non-majors throughout the curriculum with particular interest in the History of Mathematics and Abstract Algebra. He is a member of the steering committee of the Chicago Symposium on Excellence in Teaching Undergraduate Mathematics and Science. He earned an M.S. and a Ph.D. in Mathematics from the University of Illinois Chicago and served as a visiting assistant professor at Loyola University of Chicago. Steve likes to play undisclosed games of uncertain outcomes. He also bakes an excellent cheesecake whose outcome is much more certain.

Bárbara González-Arévalo is an associate professor of mathematics at Hofstra University and a SENCER Leadership Fellow. Previously she was an associate professor of mathematics, statistics, and actuarial science at Roosevelt University. Her current research interests include Statistics, Applied Probability and the Scholarship of Teaching and Learning Mathematics. She earned an M.S. and a Ph.D. in statistics from Cornell University, and worked as an assistant professor at the University of Louisiana at Lafayette. She enjoys baking and has two beautiful boys. It is important to note that she does not bake the boys.

Melanie Pivarski is an associate professor of mathematics at Roosevelt University and a SENCER Leadership Fellow. She is currently serving as the department chair for mathematics and actuarial science. She earned a Ph.D. in mathematics from Cornell University and worked as a visiting professor at Texas A&M University.  Her current research interests involve heat kernels and their applications in metric measure spaces. Recently, she has been inspired to include students in her research work. This led her to work in the scholarship of teaching and learning mathematics. She likes to eat her co-authors’ creations, as she is too busy chasing her toddler to bake on her own.

References

Dietz, J. 2013. “Creating a Culture of Inquiry in Mathematics Programs.” PRIMUS 23 (9): 837–859.

Duah, F., and T. Croft. 2012. “Students as Partners in Mathematics Course Design.” In CETL-MSOR Conference Proceedings 2011, D. Waller, ed. 49–55. York, UK: The Maths, Stats & OR Network.

Evans, W., J. Flower, and D. Holton. 2001. “Peer Tutoring in First-Year Undergraduate Mathematics.” International Journal of Mathematical Education in Science and Technology 32 (2): 161–173.

Goff, G., and B. Lahme. 2003. “Benefits of a Comprehensive Undergraduate Teaching Assistant Program.” PRIMUS 13 (1): 75–84.

Goldey, E., C. Abercrombie, T. Ivy, D. Kusher, J. Moeller, D. Rayner, C. Smith, and N. Spivey. 2012. “Biological Inquiry: A New Course and Assessment Plan in Response to the Call to Transform Undergraduate Biology.” CBE Life Sci Educ 2012 11 (4): 353–363. doi:10.1187/cbe.11-02-0017.

González-Arévalo, B., and M. Pivarski. 2013. “The Real-World Connection: Incorporating Semester-Long Projects into Calculus II.” Science Education and Civic Engagement: An International Journal (Winter 2013). http://seceij.net/seceij/winter13/real_world_conn.html (accessed December 19, 2016).

Handelsman, J., C. Pfund, S.M. Lauffer, and C.M. Pribbenow. 2005. “Entering Mentoring.” The Wisconsin Program for Scientific Teaching. http://www.hhmi.org/sites/default/files/Educational Materials/Lab Management/entering_mentoring.pdf (accessed December 19, 2016).

Hernandez-Martinez, P. 2013. “Teaching Mathematics to Engineers: Modelling, Collaborative Learning, Engagement and Accountability in a Third Space.” Mathematics Education and Contemporary Theory 2 Conference (MECT2), June 2013. http://www.esri.mmu.ac.uk/mect2/papers_13/hernandez.pdf (accessed December 19, 2016).

Kay, J., P.M. Marshall, and T. Norton. 2007. “Enhancing the Student Experience.” London: 1994 Group of Universities. http://old.1994group.ac.uk/documents/public/SEPolicyReport.pdf (accessed December 16, 2016).

Loch, B., and J. Lamborn. 2015. “How to Make Mathematics Relevant to First-Year Engineering Students: Perceptions of Students on Student-Produced Resources.” International Journal of Mathematical Education in Science and Technology 47: 29–44.

Lopatto, D. 2010. “Science in Solution: The Impact of Undergraduate Research on Student Learning.” Tuscon, AZ: The Research Corporation.

MAA CUMP (Mathematical Association of America Committee on the Undergraduate Program in Mathematics). 2006. “Mathematics Research by Undergraduates: Costs and Benefits to Faculty and the Institution.” http://www.maa.org/sites/default/files/pdf/CUPM/CUPM-UG-research.pdf (accessed December 16, 2016).

Perin, D. 2011. “Facilitating Student Learning through Contextualization.” Community College Research Center Working Paper No. 29, Teachers College, Columbia University. http://ccrc.tc.columbia.edu/media/k2/attachments/facilitating-learning-contextualization-working-paper.pdf (accessed December 16, 2016).

Pinter-Lucke, C. 1993. “Academic Excellence Workshops.” PRIMUS 3 (4): 389–400.

Porter, A. 2008. “The Importance of the Learner Voice.” The Brookes eJournal of Learning and Teaching 2 (3). http://bejlt.brookes.ac.uk/paper/the_importance_of_the_learner_voice-2/ (accessed December 16, 2016).

Seymour, E., A. Hunter, S. Laursen, and T. DeAntoni. 2004. “Establishing the Benefits of Research Experiences for Undergraduates in the Sciences: First Findings from a Three-Year Study.” Science Education 88 (4): 493–534.

United States Military Academy. Core Interdisciplinary Team. “Interdisciplinary Learning, Assessment, Accreditation, and SENCER Courses: How Do They All Fit Together?” Presentation at the SENCER Summer Institute, Asheville, NC, July 31–August 4, 2014.

Download (PDF, 644KB)

 

Flipping an Introductory Science Course Using Emerging Technologies

 

David Green,
University of Miami
Jennifer Sparrow,
Penn State University

Abstract

Today’s faculty members have tools available that enhance the learning experience of modern digital learners. Emerging technologies and innovative teaching practices update the STEM education learning process and facilitate student retention. In today’s hybridized educational world, the classroom stretches far beyond the traditional four walls, and students should be producers of content, rather than merely passive acceptors of information. This article explains how several emerging technologies were implemented and tested in a General Education marine science course for non-majors, describes the role of technologies in “flipping” the classroom, and summarizes student feedback on the learning experience. Using the global marine system and specific case study locations, the course covered major oceanography disciplines, critical environmental issues, and socio-economic conditions of urbanized coastal regions. Environmental sustainability was the integrative theme, highlighting the importance of economic growth while emphasizing that environmental responsibility and social well being must be foregrounded in the context of an exponentially growing human population.

Flipping the classroom using emerging technologies supplemented a rigorous schedule of project-based learning, laboratory activities, field excursions, and civic engagement commitments. Pre- and post- SALG surveys (Student Assessment of Their Learning Gains) were used to gauge student perspectives on the course redesign. They demonstrated improvements in knowledge, skills development, and integration of learning. The combination of activity-based, student-centered learning and emerging technologies make today’s STEM education classroom an exciting, interactive, and engaging experience by giving these sometimes reluctant students the tools they need to succeed in tomorrow’s professional world.

Introduction

A scientifically educated citizenry capable of innovation and leadership is a necessity for a functioning democracy. Many of today’s learners, however, are ambivalent about science and science education, and they lack understanding of how science relates to their daily lives (Burns 2011; Burns 2012; Green 2012). While today’s learners have been surrounded by technologies in the classroom throughout their entire academic journey, many lack the skills necessary to apply their learning and to produce content and are still passive acceptors of information. Educators now have a responsibility and the opportunity to introduce “high-impact educational practices” into curricular redesigns (Kuh 2008). A host of innovative teaching strategies in STEM education have emerged (Springer et al. 1999; Vatovec and Balser 2009; Brown et al. 2010; Prunuske et al. 2012; Green 2012) that can engage reluctant students, increase critical thinking abilities, foster collaborative relationships in the classroom, and enhance communication skills (oral, written, and digital). Matching appropriate emerging technologies with effective teaching practices (Brill and Park 2008) and gathering feedback on these STEM course redesigns is imperative as we continue to enhance our curricula.

With the advance of academic technologies, many educators have embraced the “hybrid” course design (Garrison and Kanuka 2004; McGee and Reis 2012). Hybrid courses (or blended course designs) are those in which a significant amount of quality online content is used to engage students (McGee and Reis 2012) while providing new teaching opportunities for educators (McGee and Diaz 2007; Brown et al. 2010; Green 2012). Modern learners have been called “digital natives,” while today’s educators have been named “digital immigrants,” but that terminology has generated some debate (Prensky 2001a and 2001b; Toledo 2007; Bennett et al. 2008). Although educators and learners may speak different languages in relation to technology and have different comfort levels regarding its use, it is easy to see the potential of hybrid course design for today’s multi-tasking, quick information- seeking, and media-socialized students. Using emerging technologies facilitates activity-based learning and provides students with ownership of the learning environment (Brill and Park 2008; Strayer 2012; Prunuske et al. 2012). Connecting sound pedagogical strategies with suitable technology usage creates a learning environment that matches the needs of modern learners, while providing them with the skills they need to succeed in their professional careers.

Inverting the teaching sequence, or “flipping” the classroom, has gained significant attention in recent years (Lage et al. 2000; Milman 2012; Strayer 2012; Khan 2012; Prober and Khan 2013). Essentially, traditional lecture-type material is provided to students in video or online format before face- to-face sessions. Then, during the face-to-face meetings, students are engaged in social-learning scenarios that promote interactions, engagement, and skills development by applying their knowledge. The role of the instructor changes and, in many ways, resembles an “academic coach” during the learning process rather than an “information presenter.”

Figure 1. A conceptual model of the “flipped classroom” scenario used in the course redesign is depicted. Before attending face-to-face sessions, students are expected to read introductory content, which includes both traditional readings and interactive web-based activities. During face-to-face class sessions, students engage in learner-centered approaches, including activity-based labs and experiential learning opportunities. By implementing combinations of project-based learning, case study analyses, and civic engagement strategies, students apply their learning, demonstrate higher-order thinking skills, and produce content that ultimately benefits the needs of the regional community.

Figure 1 outlines the course design conceptual model used in this curriculum redesign, which employed web-based reusable learning objects that students used before class sessions, so that experiential and activity-based learning activities could be conducted during face-to-face sessions. Reflective exercises and activities, like project-based and service-learning activities, are high-impact learning opportunities that promote academic responsibility and civic engagement. Using emerging technologies to “flip the course” provided the curricular flexibility to implement these innovative teaching strategies. “Marine Systems” is an introductory general education science course for non-science majors that has traditionally been taught as a lecture-based course with embedded laboratory exercises. This paper describes a curriculum redesign that used a“flipped” course model, learner-centered approaches, and embedded service-learning opportunities, and it provides student perspectives on the learning process. The use of emerging technologies in the curriculum facilitated the course delivery, so that students developed an understanding of ecology and its relevance to their daily lives, increasing their civic engagement and awareness (fig. 2).

Figure 2. By using emerging technologies to facilitate the learning process, students gain an ecological perspective related to the marine science concepts they are introduced to. This helps them retain information and connect it to their daily lives, and, following successful completion of the course and civic engagement activities, they leave as engaged citizens.

The primary goals of this course redesign were

  1. To enhance the educational experiences of non-major science students by engaging in learner-centered approaches and web-based techniques;
  2. To demonstrate the potential pedagogical benefits of coupling emerging technologies with innovative teaching practices in a STEM education setting;
  3. To assess student perspectives of their learning gains related to their adoption of emerging technology in a “flipped classroom” scenario.

Methods

The course redesign began by linking course objectives and learning outcomes to a “Guiding Question” which reads:

“Given the current degree of human impacts on the marine world, how can tomorrow’s generations of all inhabitants continue to benefit from the natural goods and services a healthy marine system provides, if we better understand our role as citizens today?”

From this follows the “Primary Course Objective” for this course:

“Students will be able to positively influence both southwest Florida and global communities in mak- ing evidence-based decisions regarding human use and impacts of coastal and marine areas / resources.”

Lastly, the specific learning outcomes and skills development objectives are

  1. To enhance baseline scientific knowledge relating to marine systems and global sustainability by developing critical thinking skills;
  2. To gain an understanding of the ecology of regional ecosystems, the natural goods and services provided by these ecosystems, and how human interactions disrupt natural functions;

To introduce the concept of environmental sustainability and provide opportunities for students to apply this concept to practical real-life situations in an urbanized society.

Learner-centered Approaches

A variety of learner-centered approaches (experiential learning and project-based learning) were used to enhance student practice, learning, and contributions to the learning environment (fig. 3). Combinations of classroom and field-based learning exercises were used to describe the scientific method, to help explain key oceanographic concepts, and to provide encounters with local estuarine ecosystems. Students were given ownership of academic exercises, while the instructor facilitated, guided, and reinforced crucial learning content. Table 1 explains the calendar of individual learning modules with associated major academic themes and objectives. Multiple sources of information including the textbook, scientific journal articles, lab exercises, and personal observations were used. The textbook provided background information, while journal articles examined current issues and explored topics such as ocean acidification, human impacts, overexploitation of marine resources, and global climate change. Learner-centered laboratory exercises applied textbook concepts and provided a collaborative, activity-based learning environment. A reflective journal provided opportunities for student observations and personal reflections on the learning process. Field excursions engaged student interest by exploring coastal ecosystems and assisted with the understanding of ecosystem structure and function, coastal development, and marine research. The capstone project reinforced all class activities by relating environmental sustainability to the socio-economic and environmental issues previously explored. Civic engagement opportunities helped students leave the course as engaged citizens who are willing to apply their knowledge to meaningful projects that benefit our local informal science education partners.

Figure 3. Mapping teaching strategies used within the course design to student practice, learning, and contributions to the learning environment.

Virtual “Oceanographic Research Cruise” Capstone Project

Teams of students“virtually participate” in an oceanographic research expedition that visits a particular location of geological importance on the planet. The task reads: “You have been assigned positions aboard an oceanographic vessel exploring the far reaches of the planet! Your crew will arrive at a marine destination to use as your case study. At this location, your crew will explore and research the factors shaping the region as related to the information you learn in this class. At the end of your ‘research cruise,’ crews will present at our ‘Oceanographic Exploration and Research Collection Symposium!’ Collectively, we will explore the globe in its entirety, learning about the marine systems worldwide! You will incorporate concepts related to physical and chemical oceanography, marine geology, and marine ecology into your learning adventure!” The final project is submitted via a student-created webpage that summarizes the team’s virtual research expedition. The primary intention is to apply course content and learning in a social setting to a specific location that is unique to each team of students.

Ecosystems Visit Field Study and Formal Lab Report

In class, small groups of students chose a theme to investigate for a field research project. At this point, students brainstormed the parameters of the theme and arrived at a research question, formulating a testable hypothesis and designing an experiment to test their hypothesis. The instructor facilitated discussions and helped students choose gear that was needed for the field studies. Each student group created their own study and all groups worked their way through the scientific method during this project. At a field location, students collected their data and replicated their studies in multiple locations. Students created a formal lab report (complete with Excel graphs, figures, and tables) that summarized their research. Major academic concepts covered in this project included

  1. Natural Goods and Services
  2. Ecosystem Structure and Function
  3. Water Quality
  4. Limiting Factors
  5. Beach Profiles
  6. Flora and Fauna Analyses
  7. Estuarine Ecosystems Ecology
  8. Intertidal Zone, Beaches, and Dunes Evaluation
  9. Coastal Urbanization and Habitat Loss
  10. Environmental Sustainability
  11. Land Ethic and Wilderness Values
  12. Marine Conservation

Students were given ownership of this exercise from start to finish, and they explored the natural world the way a scientist would by applying their previous learning to real-world research opportunities.

Human Impacts Project

Breakout groups were formed, and each group was assigned a topic related to a human impact on the marine environment. Phase I (“Background Explorations—A Literature Scavenger Hunt”) included a literature review, where each group located peer-reviewed journal articles related to their topic. From this research, the breakout group synthesized a definition of the impact, explained why it is a problem in the context of an exponentially-growing human population, and described how future decisions should be made differently to improve the situation related to the negative human impact. During Phase II of the project (“From Jigsaw to Podcast”), new groups were formed so that each new group contained students who researched a different human impact during the first phase (similar to a “jigsaw” method of teaching). Students now assumed the role of “expert” for their original topic and they had to teach the new group about that human impact. Once the students had explained their synthesis from Phase I, the new group created an educational podcast script that was three minutes in length and appropriate for an audience of middle-school-aged children. To create the script, students had to summarize all of the human impact topics represented in their new group by answering the following questions:

  1. What is the size of the current human population and what is meant by exponential population growth?
  2. What are examples of modern-day human influences on the marine world?
  3. How and why are these human impacts a problem for the marine world under the context of an exponentially growing human population?
  4. Explain what humans can do differently in regard to future decisions made about ocean impacts.

This project helps students critically examine scientific research, use higher-order thinking skills, and produce educational content for a younger generation.

High-impact Learning Opportunities: Service-learning Projects and Civic Engagement

Partnering with regional informal science education centers, students assisted with tasks that met community needs by participating in field-based service-learning projects. These projects allowed students the opportunity to visualize previous human impacts on coastal ecosystems and mitigate the damage. Using “prompt” questions, students reflected on their experience in a written deliverable that connected their service-learning experiences to their learning in the course and personal development.   In previous iterations, students also delivered oral presentations with the regional partners in attendance. Serving the needs of the community and learning how to take a leadership role in civic engagement are the primary goals of this high-impact project.

Matching Emerging Technologies to Course Outcomes

A main focus of this course redesign was to match the use of appropriate technologies with non-traditional pedagogical strategies (table 2). Careful thought was given to the choice of technology in the course delivery and to desired outcomes. A description of the chosen technologies follows.

Reusable Learning Objects (RLOs): Traditional lecture sessions were replaced with web-based digital Reusable Learning Objects (RLO’s) that were created by the instructor. These highly-interactive presentations with audio, animated figures, text, pictures, and illustrations supplemented the curriculum and enhanced the experience of students by providing an interactive learning environment with real-time assessment and feedback.

GIS Mapping Software: A variety of Geographic Information Systems (GIS)-based learning opportunities were embedded within the course design. Students interpreted patterns they observed and improved their spatial analysis skills. They created their own maps of coastal ecosystems and water quality summaries by using handheld Global Positioning System (GPS) receivers and cloud-based GIS mapping software.

Podcasting: A podcast is an audio or video file that is broadcast over the internet. Following in-depth research on human impacts on the marine world, students created three-minute educational podcasts that are sharable with a younger audience.

Web 2.0 Tools (Weebly, Prezi, Blogs, etc.): Students used free Web 2.0 tools to create their own presentations and webpages. Using these tools, students went from passive acceptors of knowledge to active producers of learning content, which helped them utilize higher-order thinking skills.

Online Database Literature Searches: Students are expected to evaluate evidence and find reputable sources of scientific information. Peer-reviewed literature database searches were required throughout the course and exposed students to discipline-appropriate writing styles and the importance of the peer-review process.

TwitterTM Discussions: TwitterTM is a social networking system designed for quick comments and interactions. Students engaged in out-of-classroom discussions that followed face-to-face sessions and introduced upcoming class topics.

eTexts, Smartphones, and Tablet Computers: A variety of hardware choices by students facilitated the learning process. Our classroom was not conceptualized as a four-walled room with desks, but instead reached far beyond the traditional setup and allowed for real-time explorations of internet content and just-in-time teaching moments related to current events. While all course components are currently available for use on a tablet or computer via the learning management system, not all students own such a device, and any hardware choice by the student was acceptable.

SALG Survey and Data Analysis (Methods)

A Pre- and Post- Student Assessment of Learning Gains (SALG) survey was conducted to gain anonymous student perspectives on the course redesign. Students from single course, in each of two different semesters, was included in this analysis. Surveys included questions related to Knowledge, Skills, and Integration of Learning. Mean scores with Standard Errors were calculated for each question and compared across semesters. Table 3 displays the questions used in the SALG surveys. Because students withdraw from classes during the semester, the pre- and post- surveys have slightly different sample sizes. Results from the SALG surveys allowed for omnibus comparisons and cross-semester evaluations. Students were given an opportunity for free-write responses, as well, though those comments are not included in this manuscript.

Results

During the Fall 2011 semester, 77% of students self-reported GPA’s > 3.01 and 92% stated they were non-science majors (nFall 2011 Pre: 69; nFall 2011 Post: 59). During the Spring 2012 semester, 52% of students self-reported GPA’s > 3.01 and 95% stated they were non-science majors (nSpring2012 Pre: 60; n  t: 58).

Students responded to questions designed to measure their own perception of their understanding of core academic content (table 3—“Understanding” section). Across semesters, similar trends emerged. Students entered the course at or near the “Somewhat” comfortable level with their understanding of core academic concepts in all measured categories; students in both classes left the course feeling “A Lot” to “A Great Deal” more comfortable with their own understanding of core academic concepts (fig. 4). Students responded to questions designed to measure their own assessment of “Skills Development” (table 3—“Skills” section). Across semesters the data indicated that students entered the course at or near the “Somewhat” comfortable level with their perceptions of skills development; students in both classes left the course feeling “A Lot” to “A Great Deal” more comfortable with their own perceptions of skills development (fig 5). One specific skill (“Work Effectively with Others”) displayed no change in the pre- and post- surveys in either the Fall 2011 or Spring 2012 semesters (fig. 5).

Figure 4. Pre- and Post-SALG survey results from two semesters comparing “Understanding of Core Academic Concepts.” Question numbers on the x-axis can be cross-referenced with the actual questions in Table 3. Students responded with a 1-6 score, as illustrated on the y-axis (1=N/A; 2=Not at All; 3=Just a Little; 4=Somewhat; 5=A Lot; 6=A Great Deal). Mean and SE are reported.
Figure 5. Pre- and Post-SALG survey results from two semesters comparing “Skills Development.” Question numbers on the x-axis can be cross-referenced with the actual questions in Table 3. Students responded with a 1-6 score, as illustrated on the y-axis (1=N/A; 2=Not at All; 3=Just a Little; 4=Somewhat; 5=A Lot; 6=A Great Deal). Mean and SE are reported.

Embedded within this course were opportunities for civic engagement, GIS exercises to enhance geospatial analysis skills, and collaborative learning experiences for students. The omnibus dataset (table 3) reveals that students showed a strong increase in their understanding of how civic engagement activities help connect course content to real-world scenarios (MeanPre = 4.160 vs. MeanPost = 5.250).

GIS and geoliteracy skills were enhanced as students demonstrated a strengthened skillset related to their abilities to interpret GIS images to identify patterns (MeanPre = 2.879 vs. MeanPost = 4.448). Student attitudes remained neutral toward activity-based learning (MeanPre = 4.821 vs. MeanPost = 4.800). However, student perspective related to project- based learning displayed an increase (MeanPre = 4.353 vs. MeanPost = 4.650).

Figure 6. Pre- and Post-SALG survey results from two semesters comparing “Integration of Learning.” Question numbers on the x-axis can be cross-referenced with the actual questions in Table 3. Students responded with a 1-6 score, as illustrated on the y-axis (1=N/A; 2=Not at All; 3=Just a Little; 4=Somewhat; 5=A Lot; 6=A Great Deal). Mean and SE are reported.

Helping students integrate their new knowledge is an important goal in a general education course and is a key factor in matching teaching strategies to student practice, learning, and contributions to the learning environment (fig. 3). Students were asked if they were in the habit of connecting key ideas they learn in their classes with other knowledge, of applying what they learn in classes to other situations, of using systematic reasoning in their approach to problems, and of using a critical approach to analyzing data and arguments in their daily lives (table 3—“Integration of Learning” section). Learner perspectives showed an increase in each of these four categories related to the student integration of learning (fig. 6 and table 3 – “Integration of Learning” section).

Discussion

Spatially and technologically, tomorrow’s classroom will be very different from today’s, and the academic tools used in it may not yet even exist (McGee and Diaz 2007; Green 2012; Bolduc-Simpson and Simpson 2012). Yet we currently have many opportunities to engage modern learners with a variety of innovative strategies (Kuh 2008) and learner-friendly technological devices. We must continue to evaluate and assess the incorporation of emerging technologies into curricula redesigns, to ensure their academic soundness and their effectiveness in increasing student engagement. Entry-level STEM courses, like the one described in this article, provide us with the opportunity to transform the science education experience for reluctant learners (Green 2012).

Brundiers et al. (2010) stated the importance of embedding “real-world learning opportunities” into general education courses with an environmental sustainability focus. Overall, students responded favorably to project-based learning in this course redesign. When performing their own assessments, students clearly indicated an increased confidence in their learning gains. Increased skills development (critical thinking, communication, collaborative learning, and social interactions), which contributes to career and professional readiness, was demonstrated, as was an increase in integrating course content by connecting information gained in this course to other knowledge. Likewise, students perceived an increase in their ability to connect their knowledge gains from this class to other situations. In using the scientific method as a guide, students verified that they now are beginning to use systematic reasoning in their approaches to problem solving. Consistent with previous studies, students associated with this course redesign began to understand how civic engagement activities help connect course content to real-world scenarios that made course material relevant to them (Jacoby 2009; Green 2012).

While this course redesign was successful in many ways, it is important to recognize that not every student responds favorably to an inverted classroom design supported by technology. Most students are accustomed to note-taking during a traditional lecture, and any alteration to this structure makes some students uncomfortable. While these changes may not excite a student (as indicated in SALG Attitudes question about activity-based learning), other data presented in this paper show that learning did indeed take place. It is equally important to recognize that not all students learn in the same way, and some may not respond positively to non-traditional teaching strategies. This, however, is true of any teaching method, and it remains the responsibility of the instructor to adjust, assist, and guide each individual learner in the classroom, as needed. The instructor must also remember that learning happens at different paces, and that some students respond slowly to independent learning strategies that differ from their traditional classroom experiences, especially if they lack self-motivation. There are access issues with technology that must be understood by the instructor (i.e. costs, lack of ownership, etc.). Some students lack digital skills, and we must not assume that all have the same knowledge and experience when it comes to using digital tools, software, and hardware. Indeed, Toledo (2007) states that not all students are interested in a technologically-immersed learning environment, regardless of age or exposure. While the challenges listed here are not prohibitive, they must be understood for a successful course redesign aimed at increasing student engagement in the learning process.

In this study, emerging technologies proved to be an effective complement to the curriculum. Student responses generally showed an increase in learning and an increased confidence in subject matter as a result of the flipped classroom model that used emerging technologies as a teaching supplement. Classrooms tended to be lively, with animated students who were actively producing content. This is a much different scene from a traditional classroom with slideshows, dimmed lights, and quiet students taking notes. Thanks to the increased opportunities for one-on-one interactions during the face-to-face class time, struggling students were identified early in the learning process and assisted with their skills development and knowledge gains. This is consistent with Prunuske et al. (2012), who stated that they were able to spend more classroom time assisting students with higher- order learning development.

Using an inverted classroom delivery model required that the role of the instructor be modified into that of an academic facilitator, one who actively guides, rather than one who spouts information from the front of the room. Because self-motivated students were essential to the success of the course, there were challenges. “Borderline chaos” was tolerated in this active-learning scenario, yet the student energy was harnessed and used in a positive manner. Typically, breakout groups of students worked independently while the instructor circulated through the classroom. As a result, there was less reliance on slideshows and formal lectures. Instead, discussions, interactive exercises, and activity-based learning opportunities were emphasized, to promote student engagement and concept retention. Students must still be provided with proper guidance that includes “cognitive presence, teacher presence, and social presence” (Garrison and Cleveland-Innes 2005). Extra time and care should be given by the instructor to explain the new teaching methods, why they are important to the students, and what the learning outcomes are. Innovative teaching methods aside, best practices in teaching must be continued, which means that, regardless of pedagogical strategies, traditional study skills still need to be emphasized for proper learner development. (Brill and Park 2008; McGee and Reis 2012).

Many students have some underlying interest in the course on the first day, yet these same students may have had earlier experiences in science classes that alienated them. Some arrive with preconceived notions about what science is and isn’t. This interrupts their learning until the instructor can find ways to break through these barriers and reach the learner. Connecting textbook material with real world scenarios, case studies, and interactive exercises promotes stronger interest in the learning process and provides students with ownership of the class. Service-learning projects make students feel a sense of pride and accomplishment by directly serving the needs of regional organizations. Reaching reluctant learners and exciting them about science is an embraceable challenge that can be accomplished through the right mix of teaching methods and curricula design (Strayer 2012).

Learner-centered approaches to teaching were employed that relied upon innovative web-based techniques. By matching appropriate emerging technologies with learning outcomes in a STEM education classroom for non-science majors, reluctant students were reached and excited; these students were able to connect course content to other classes and to their daily lives, making their experience relevant and worthwhile. Gaining insight from students about the academic experience by understanding their perspectives is important as faculty experiment with new teaching strategies. To promote best practices in teaching, assessing learning gains and demonstrating student successes is an important follow-up for faculty members who experiment with non-traditional teaching methods and approaches. The incorporation of emerging technology into the course redesign allowed students to engage in a variety of learner-centered approaches designed to increase their knowledge, skills, and integration of learning. While students were neutral in their feelings toward activity-based learning, they displayed an increase in their enthusiasm toward project-based learning, which indicates that a successful social and collaborative learning environment was established with this course redesign. Student spatial skills were enhanced through the use of GIS mapping exercises and academic content was connected to their daily lives via a service-learning project at a coastal salt marsh, indicating student uses of higher-order thinking skills (Bloom 1956; Fink 2003). Our current students are our future decision-makers and leaders. It is vital to give them the tools they need to be well-rounded professionals who are educated and technologically advanced, and who approach their lives with ecological perspectives. As faculty members, it is our responsibility to ensure the teaching strategies we employ are as advanced and innovative as possible. Taking the time to understand the student perspective on innovative course redesigns can enable us to enhance the learning environment for all and might just help us save some of those reluctant science students.

Acknowledgements

A SENCER Post-Institute Implementation Award and an FGCU General Education Council Course Redesign Faculty Award helped fund this project. The authors wish to thank Douglas Spencer, Jessica Rhea, Mike Savarese, Donna Henry, Elspeth McCulloch, Aswani Volety, and the “ Tablet Computer Teaching Cell” at FGCU. Terry Cain, Lee County Parks and Recreation, and the Conservation 2020 Program assisted with civic engagement projects and field excursion logistics. Finally, many thanks to the “Students-as-Partners” who make this work possible and worthwhile! This study was completed at Florida Gulf Coast University before the lead author moved to the University of Miami.

About the Authors

David Green is an Instructional Designer for the Academic Technologies department at the University of Miami, where he is responsible for consulting with, guiding, and supporting faculty in the design and delivery of technology-enhanced courses and co-curricular activities. He is responsible for helping to design, develop, and implement the “Cane Academy,” which is a new initiative at the UM Miller School of Medicine to “flip the classroom” using short instructional videos coupled with companion assessment exercises. As a SENCER Leadership Fellow, he authored a SENCER   Model   Course   and has retrofit multiple university-level classes using the SENCER approach to pedagogy, assessed student response and engagement to the course redesigns, and helped recruit new faculty members to the program.

Jennifer Sparrow is the Senior Director for Teaching and Learning Technology (TLT) at Penn State University. TLT works to help PSU faculty take advantage of information technology to enrich the educational experiences of their students and to champion the creative and innovative uses of technology for teaching, learning, and research. She was previously Senior Director of Networked Knowledge Ventures and Emerging Technologies at Virginia Tech. For more than 15 years, she has championed the use of technology to engage students in the learning process. She has a passion for working with faculty to explore new technologies and their potential implementations in teaching and learning. She loves working with faculty who are willing to push the boundaries of the leading edge of technology in teaching, learning, and research. Her current projects involve the convergence of technologies and learning spaces to create interactive and engaged learning opportunities. Jennifer’s conversations around technology focus on increasing digital fluency for students, faculty, and life-long learners.

References

Bennett, S., K. Maton, and L. Kervin. 2008. “The ‘Digital Natives’ Debate: A Critical Review of the Evidence.” British Journal of Educational Technology 39 (5): 775–786.

Bloom B.S. 1956. Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co., Inc.

Bolduc-Simpson, S., and M. Simpson. 2012. “Social Places in Virtual Spaces: Creating a Social Learning Community in Online Courses.” Distance Learning for Educators, Trainers, and Leaders 9 (3): 33–42.

Brill, J.M., and Y. Park. 2008. “Facilitating Engaged Learning in the Interac-tion Age: Taking a Pedagogically-disciplined Approach to Innovation with Emergent Technologies.” International Journal of Teaching and Learning in Higher Education 20 (1): 70–78.

Brown, M., M. Auslander, K. Gredone, D.P.J. Green, B. Hull, and W. Jacobs. 2010. “A Dialogue for Engagement.” EDUCAUSE Review 45 (5): 38–56.

Brundiers, K., A. Wiek, and C.L. Redman. 2010. “Real-world Learning Opportunities in Sustainability: From Classroom into the Real World.” International Journal of Sustainability in Higher Education 11 (4): 308–324.

Burns, W.D. 2011. “But You Needed Me: Reflections on the Premises, Purposes, Lessons Learned, and Ethos of SENCER—Part 1.” Science Education and Civic Engagement: An International Journal 3 (2): 5–12.

———. 2012. “But You Needed Me: Reflections on the Premises, Pur- poses, Lessons Learned, and Ethos of SENCER—Part 2.” Science Education and Civic Engagement: An International Journal 4 (1): 6–13.

Fink, L.D. 2003. Creating Significant Learning Experiences. San Francisco: Jossey Bass.

Garrison, D.R., and H. Kanuka. 2004. “Blended Learning: Uncovering Its Transformative Potential in Higher Education.” Internet and Higher Education 7 (2): 95–105.

Garrison, D.R., and M. Cleveland-Innes. 2005. “Facilitating Cognitive Presence in Online Learning: Interaction Is Not Enough.” The American Journal of Distance Education 19 (3): 133–148.

Green, D.P.J. 2012. “Using Emerging Technologies To Facilitate Science Learning and Civic Engagement.” Science Education and Civic Engagement: An International Journal 4 (2): 18–33.

Jacoby, B. 2009. Civic Engagement in Higher Education: Concepts and Practices. San Francisco: Jossey-Bass.

Khan S. 2012. The One World School House: Education Reimagined. New York: Twelve.

Kuh, G.D. 2008. High Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, D.C.: Association of American Colleges and Universities.

Lage, M.J., G.J. Platt, and M. Treglia. 2000. “Inverting the Classroom: A Gateway to Creating an Inclusive Learning Environment.” The Journal of Economic Education 31 (1): 30–43.

McGee, P., and V. Diaz. 2007. “Wikis and Podcasts and Blogs! Oh My! What Is a Faculty Member Supposed To Do?” EDUCAUSE Review 42 (5): 28–40.

McGee, P., and A. Reis. 2012. “Blended Course Design: A Synthesis of Best Practices.” Journal of Asynchronous Learning Networks 16 (4): 7–22.

Milman, N.B. 2012. The Flipped Classroom Strategy: What Is It and How Can It Be Used?” Distance Learning for Educators, Trainers, and Leaders 9 (3): 85–87.

Prensky, M. 2001a. “Digital Natives, Digital Immigrants Part 1.” On the Hori¬zon 9 (5): 1–6.

———. 2001b. “Digital Natives, Digital Immigrants Part 2: Do They Really Think Differently?” On the Horizon 9 (6): 1–6.

Prober, C.G., and S. Khan. 2013. “Medical Education Reimagined: A Call to Action.” Academic Medicine 88 (10): 1407–1410.

Prunuske, A.J., J. Batzli, E. Howell, and S. Miller. 2012. “Using Online Lectures To Make Time for Active Learning.” Genetics Education 192: 67–72.

Springer, L., M.E. Stanne, and S.S. Donovan. 1999. “Effects of Small- group Learning on Undergraduates in Science, Mathematics, Engineering, and Technology: A Meta-analysis.” Review of Educational Research 69 (1): 21–51.

Strayer, J.F. 2012. “How Learning in an Inverted Classroom Influences Cooperation, Innovation, and Task Orientation.” Learning Environments Research: An International Journal 15: 171–193.

Toledo, C.A. 2007. “Digital Culture: Immigrants and Tourists Responding to the Natives’ Drumbeat.” International Journal of Teaching and Learning in Higher Education 19 (1): 84–92.

Vatovec, C., and T. Balser. 2009. “Podcasts As Tools in Introductory Environmental Studies.” Journal of Microbiology and Biology Education 10: 19–24.

Download (PDF, 1.68MB)

Persistent and Encouraging Achievement Gains on Common Core-Aligned Items for Middle School English Language Learners: ASAMI-Hands-On Astronomy for After-school Science and Math Integration

 

Jenifer Perazzo,
Pleasanton School District
Carl Pennypacker,
UC Berkeley and Lawrence Berkeley National Lab
David Stronck,
California State University, East Bay
Kristin Bass,
Rockman et al
Rainbow Lobo,
Winton Middle School
Gabriel Ben-Shalom,
Winton Middle School

Abstract

ASAMIAfterschool Science and Math Integration—integrates skills of mathematics with interesting concepts and hands-on activities in astronomy-based science in the middle school. Common Core Mathematics Standards and Next Generation Science Standards (NGSS) are used as ASAMI effectively teaches algebra standards/concepts with Hands-On Universe (HOU) curricula to engage 12–14-year-old English Language Learners (ELLs). In our 2014–15 school year pilot and field tests of ASAMI, students classified as ELL met twice a week for a total of four hours a week, at a middle school in California, USA. The evaluation of ASAMI shows that these learners improved their test scores on Common Core Mathematics Standards items [Gain = (post-test−pre-test)/pre-test] by 46 percent in our first six-week trial and by about 93 percent in our second semester in the school year. Four other pilots resulted in similar gains. The main algebraic focus and assessment items focused on ratios, proportion, and linear equations, which are used throughout the curriculum of the HOU. Our assessments show that ASAMI is a very effective tool to help focus instruction, and they demonstrate success in learning through the integration of math and science.   While the desire for integrated math and science curricula has been expressed for decades, few quantitative studies of achievement gains have surfaced (Czeniak et al. 1999).

Background and Introduction

Hands-On Universe

Afterschool Science and Math Integration (ASAMI) is based on Hands-On Universe (HOU) astronomy activities that are often computer/technology based.   HOU was based for many years at the Lawrence Hall of Science (LHS) at the University of California, Berkeley, and developed significantly within the Hall. Alan Friedman’s leadership at LHS in astronomy education help build the discipline of “Hands-On” astronomy. HOU has many linkages directly traceable to Alan, and the appendix describes the heritage of HOU through Alan.

Over its almost twenty-five years of activities, HOU has brought the wonder and the data of the Universe into classrooms all around the world. Approximately one thousand American teachers have been in HOU teacher workshops. Through the Galileo Teacher Training Program (GTTP), approximately 20,000 teachers in 100 nations around the world have ben in HOU workshops.   Formal external evaluations submitted to the U.S. National Science Foundation have usually demonstrated that HOU changed students’ attitudes positively towards STEM careers and helped students appreciate math, science, and technology. In HOU students measure objects on and off the computer and make models of celestial systems. We currently plan to start a new round of United States HOU Teacher workshops and are actively seeking funding. ASAMI is the most recent version of HOU. It uses HOU’s images, software, activities, and methods, adopted for ELL middle school students.

Program Goals

One goal of ASAMI is that students master enough math so that they can explore careers in STEM fields. Our pre-tests of the ELL students demonstrated that these students were lacking important skills and would have grave difficulties pursuing STEM careers. All citizens of the world are now facing major technological and scientific challenges. Every student needs to become an active, well-informed and educated citizen. The ELL students in our study required some remedial interventions in their education to succeed in the disciplines of math and science.   We wanted these students to to engage in and appreciate math and science, using hands-on, HOU-inspired activities, both on and off the computer.

NGSS Middle School Topics

The Next Generation Science Standards (NGSS) recommend that science education in grades K–12 be built around three major dimensions: scientific and engineering practices; crosscutting concepts that unify the study of science and engineering through their common application across fields; and core ideas in the major disciplines of natural science.   The Framework for K-12 Science Education (Quinn et al. 2012) also identifies seven crosscutting concepts that bridge disciplinary boundaries, uniting core ideas throughout the fields of science and engineering.   Among the seven crosscutting concepts presented in Chapter 4 of the Framework is the following: “Scale, proportion, and quantity. In considering phenomena, it is critical to recognize what is relevant at different measures of size, time, and energy and to recognize how changes in scale, proportion, or quantity affect a system’s structure or performance.”

The first three standards of Middle School – Earth Science Standards of NGSS (NGSS, 2013) support well our objectives in ASAMI:

(1)     MS-ESS1-1.     Develop and use a model of the Earth-sun-moon system to describe the cyclic patterns of lunar phases, eclipses of the sun and moon, and seasons.  [Clarification Statement: Examples of models can be physical, graphical, or conceptual.]

(2)     MS-ESS1-2.      Develop and use a model to describe the role of gravity in the motions within galaxies and the solar system.  [Clarification Statement:  Emphasis for the model is on gravity as the force that holds together the solar system and Milky Way galaxy and controls orbital motions within them. Examples of models can be physical (such as the analogy of distance along a football field or computer visualizations of elliptical orbits) or conceptual (such as mathematical proportions relative to the size of familiar objects such as their school or state).] [Assessment Boundary: Assessment does not include Kepler’s Laws of orbital motion or the apparent retrograde motion of the planets as viewed from Earth.]

(3)     MS-ESS1-3.      Analyze and interpret data to determine scale properties of objects in the solar system.  [Clarification Statement: Emphasis is on the analysis of data from Earth-based instruments, space-based telescopes, and spacecraft to determine similarities and differences among solar system objects. Examples of scale properties include the sizes of an object’s layers (such as crust and atmosphere), surface features (such as volcanoes), and orbital radius. Examples of data include statistical information, drawings and photographs, and models.] [Assessment Boundary: Assessment does not include recalling facts about properties of the planets and other solar system bodies.]

Such topics in the NGSS were included in ASAMI and were found in all of the activities. (See Appendix 2.)

Common Core Seventh- and Eighth-Grade Math

The NGSS clearly require the inclusion of the mathematical concepts of scale and proportion. Meanwhile the State of California has also adopted the Common Core Mathematics Standards,which include, for grade seven: “Analyze proportional relationships and use them to solve real-world and mathematical problems,” and for grade eight: “Understand the connection between proportional 
relationships, lines, and linear equations.” Many middle school students have had difficulty in understanding these concepts. The Trends in International Mathematics and Science Study (TIMSS) reports: “Students also found the proportionality items difficult. For example, one of the least difficult problems in this area asked about adding 5 girls and 5 boys to a class that was three-fifths girls. On average, fewer than two-thirds of the students across countries correctly answered that there would still be more girls than boys in the class” (Beaton 1996). Such students are subsequently unable to achieve mastery of algebra, the gatekeeper to more advanced mathematical and scientific courses. Research referenced in this article shows that an integrated curriculum provides opportunities for more relevant, less fragmented, and more stimulating experiences for learners.

Target Audience

ASAMI had its first pilot study done at a diverse middle school in El Cerrito, CA, during 2012–2013.  Then the leaders of ASAMI identified three middle schools in Hayward, CA, as appropriate schools for collecting research data about its effectiveness.   The principals of these schools wanted ASAMI to serve their many students who are English Language Learners.  Table 1 below indicates that ELLs are a significant segment of learners in California overall and in Hayward in particular. Our pre-tests indicate this population is very challenged to master the standards of Common Core Mathematics.

To meet the needs of the English Learners, the ASAMI program included several tutors who are bilingual in English and Spanish.  Although the lessons were taught in English, the tutors were always available to help the English Language Learners to understand the assignments and to feel accepted.  Here are data from Ed-Data of California from the year 2013–2014:

Table 1. Demographics of Schools in Target Area

School or Educational System Hispanic or Latino Students English Language Learners Free or Reduced Price Meals
California 53.3% 22.7% 59.4%
Hayward Unified School District 61.1% 30.9% 70.5%
Winton Middle School 76.9% 22.2% 78.9%
Bret Harte Middle School 51.1% 10.1% 67.8%
Cesar Chavez Middle School 67.3% 27.8% 81.1%

 

Figure 1. ASAMI student at work

The ASAMI program provides all of the hands-on materials and often sends the students home with items they constructed.  Leaders at the schools help greatly by recruiting the students, monitoring their attendance, and phoning the parents of absentees.  From interviews (to be published), it was very clear that parents want their children to succeed in STEM and are eager to cooperate with this after-school program.  Our interviews indicate that many English Language Learners struggle to learn a new language and simultaneously keep up with the pace of study in the classroom.

M. Calderon (2007) has stated: “The Hispanic dropout rate is the highest in history.” We have observed that ELL students often become discouraged, fail to compete, and are ready to drop out of participation in school activities.  The ASAMI program is achieving a caring, enjoyable environment where the students are making progress.

Fry observed: “An analysis of recent data from standardized testing around the country shows that the fast growing number of students designated as English language learners (ELL) are among those farthest behind” (2007, i)  The ASAMI project has been used successfully to serve this needy population.   The faculty of ASAMI have endeavored to use the best practices (Rolstad et al. 2005; Short and Echevarria 2004) to serve these students. Many of the previous studies tend to focus on language acquisition. The ASAMI program adds the acquisition of science and math literature. Integrating inquiry-based science and language learning brings success to ELLs, according to Stoddart, who wrote: “The authors of this article take the alternate view that the integration of inquiry science and language acquisition enhances learning in both domains” (2002, 664).

ASAMI Activities

Table of Some ASAMI Activities

An exemplary list of ASAMI activities is shown in Appendix 2.  Each activity usually required one to two hours in an after-school session.

Modeling Pedagogy and Support of the NGSS Practice Matrix
More ASAMI students at work

ASAMI endeavors to implement at the middle-school level the Modeling Pedagogy, which is widely used in many high-school physics classes.   The lead ASAMI teacher, Jennifer Perazzo, uses these instructional strategies. Moreover, creating and evaluating models is a major goal of NGSS. The table in Appendix 3 shows examples of the use of models in the NGSS.

The website of the American Modeling Teachers Association explains: “Modeling Instruction . . . applies structured inquiry techniques to the teaching of basic skills and practices in mathematical modeling [and] proportional reasoning” (http://modelinginstruction.org). Modeling Instruction has proven to be one of the most reliable pedagogies to improve student learning In the Modeling Instruction pedagogical approach, students work in groups of three. They voice their preconceptions, collect experimental data, build a model in their small groups, and document their ideas on whiteboards. Then the students assemble with their classmates for a “board meeting” to present their work and develop a class consensus model.

An example of how we implemented the model in ASAMI is shown in the diagram below.

Figure 3. Typical Modeling Pedagogy in Action (2-hour session)

ASAMI Assessments of Common Core Math

The first goal of the evaluation was to assess the effects of students’ participation in ASAMI on their understanding of proportional reasoning. To measure these outcomes, evaluators developed pre- and post-program content tests and surveys. Math assessments only were developed and implemented. The content tests contained five proportional reasoning items taken from four sources: (1) the California STAR test database, (2) the National Assessment of Educational Progress (NAEP) item database; (3) the New England Common Assessment Program; and (4) the Silicon Valley Mathematics Initiative’s Mathematics Assessment Collaborative project.

An exemplary assessment item is shown below.

Diagram 1. Typical ASAMI Common Core Math Assessment Item

The lead teacher and main content developer had not studied the assessments and was unaware of the detailed questions. Her focus was to develop and teach activities that were hands-on activities emphasizing Common Core math principles and tools.

Results of Assessments

Test Scores

We deployed our five assessment items in pre- and post-test sessions at the beginning and end of ASAMI. At Portola Middle School, only interviews were undertaken. All of the Common Core Math assessments were administered in the school years 2013–2014 and 2014–2015. While these assessments are viewed as a preliminary study, it is clear there was a gain in students’ capabilities. Before starting ASAMI, students’ skills were very low. Every group of ASAMI students had test scores that improved significantly beyond the control group’s gains. In summary, students had about double the learning gain, compared to a preliminary control class. Hence, we view the ~2X more learning as a lower limit, compared to traditional learning.

The number of students assessed was typically about twenty per class, and the standard deviations were usually around one point, When we combine the data, the results become much more significant, with the summed results approaching significance at greater (1/sqrt(4)), at a 4 sigma significance.  These results are very encouraging.

It is interesting to note that the eighth-grade ASAMI students, who had undergone normal math education for most of a year, had pre-test scores similar to those of entering ASAMI seventh graders.  These incoming eighth graders had learned little in the year and a half of math education since their entrance into middle school.

Table 2. ASAMI Pre- and Post-test Results

Student Interviews and Informal Observations

Interviews and observations were done at Portola Middle School, with parental consent and student assent forms per the UC Berkeley Committee for the Protection of Human Subjects Protocol # 2012-03-4125. These data suggest that students found the ASAMI activities to be highly engaging and quite different from typical classroom practices. Students worked diligently in groups on complex math and science problems, persisting on new and challenging tasks with the help of their ASAMI leaders. During one session, for instance, evaluators observed students using Salsa J software to calculate astronomical distances. A group of four students sat or stood in front of a computer, with one student running the program and the others providing guidance. The students were so engaged in the activity that they wanted only a brief snack break before returning to their work.

The root of ASAMI’s appeal may be in its “useful application” approach to mathematics. Rather than teaching proportional reasoning as an abstract skill, ASAMI embeds it into science problems that pique students’ interest. In fact, one student described the program as “an astronomy program which sneaks in math,” noting that she often didn’t “realize how much [math] you’re doing” until later. It was only in the hours after ASAMI that she felt the full mental impact of what she had done: “My brain’s tired. I’ve done too much math.

Another student also praised the ASAMI’s activities, calling them “Math in a fun way. You don’t know you’re doing math but you are,” she said. “I liked how they put the math. They didn’t just give you like a paper with math problems and say do this. It was in a way where it was math but it wasn’t just math, it was something else like astronomy.”  This same student commented that ASAMI was a very different from her regular math classes: “Most of the time now in school the teacher’s on the whiteboard, we do problems, we do our homework and our work, but it’s nothing like this, with measuring, with astronomy, with ratios, you know, it’s not like how they put it.” Before ASAMI, she didn’t think that mathematics had much to do with science. “I didn’t really think I needed science to do math. I just thought science was science and math was math and they were two different things.” Now that she has been through the program, she wishes that all students could have the same experience. “By them [math and science] being joined together it makes it more interesting and more fun because you’re not just doing math and you’re not just doing science, but you’re doing both of them at once.”

General Observations and Success Factors

We believe there are several reasons why ASAMI has worked well.

  • Individual Tutoring

We employed two or three Spanish-speaking high school and community college students in the ASAMI sessions. Hence, ASAMI participants received a lot of individual tutoring, and with the help of their own peer groups, were somehow convinced to undertake rigorous work and struggle with Common Core topics.

  • Fun and Exciting Activities

Math was always fun and often had instant consequences/feedback if you got things wrong. For example, in the playdoh recipe scaling activity, at least half of the students got the ratios wrong (many subtracted instead of using ratios!) and they made playdoh with much too liquid a consistency.   There was always fun and excitement in the hands-on activities, and we could keep them both involved and working rigorously, competing against other after-school activities. Students, when asked if this work was more fun and interesting than their normal math classes would give staff a condescending look and say “Duh…”

  • Parent and Community Support

We had great support from the parents. The leader of ASAMI community relations, Mr. Jesus Heredia, continually cultivated a strong relationship with the parents. The parents wanted ASAMI for their children, and if children did not attend the ASAMI sessions, the parents were informed, and usually the students came back. For these reasons, there was very low attrition in the student population (<12%). ASAMI was observed by staff to be desired by the parents as it promoted Common Core learning with an emphasis on technology, college, and jobs.

  • Strong Support from our Hosting Schools

Winton Middle School and Bret Harte Middle School provided superb hosting of our system. We had support from the administrators and from the after-school programs (Youth Enrichment Program), and custodial staff.

  • Strong Support from the School English Learner Advisory Committee (ELAC)

We undertook very careful communications and briefing with the ELAC, especially at Winton Middle School; they were convinced of ASAMI’s value, and they felt that ASAMI was their program.

  • Strong Support from the Hayward Unified School District (HUSD) Office and School Board

ASAMI benefitted from great support from the HUSD central office. The whole development of our program, the funding systems, the invoicing and multiple layers of approval (including School Board approval) were all undertaken with vigor and enthusiasm by District staff.

  • Undying Dedication to Rigor and Common Core Math in Every Instance!

We did not have to dig deeply to find how proportions and ratios are used in our science problems, so we could both emphasize Common Core and complete these activities. For example, students learned in HOU that proportion and scale are used widely in the Universe and that, in fact, the Universe makes no sense without proportion and scale.

  • Comments from an ASAMI Teacher

One new instructor, Mr. Ben-Shalom, writes of ASAMI: “At first I was skeptical that struggling students would want to participate in yet more academics during their after-school time, and yet this program has amazed me. ASAMI will not work for everyone, but those students who it has reached have shown a kind of dedication and enthusiasm about math and science that I thought not possible. And this is due to ASAMI’s solid repertoire of lessons and activities that are engaging and will help these students succeed.”

Future Work

We are confident of our test score gains and students’ indications of excitement about STEM topics. Future work (proposals are in the planning stages) will include a deeper study of these results and a more thorough explication of the success factors. As one local collaborator noted: “The ASAMI initiative has snowballed through the science department and inspired more student-centered and hands-on activities, generally.” We will endeavor to spread ASAMI throughout the Hayward Unified School District and then beyond into other California schools, many of which are blessed with students and families eager to master the Common Core STEM topics and need some extra help from ASAMI as their language acquisition and skills develop.

About the Authors

Kristin M. Bass, Ph.D., is a Senior Researcher at Rockman et al, a San Francisco-based external evaluation company. Kristin’s areas of expertise include assessment development and validation, program fidelity, research design, and quantitative analysis. At Rockman, she primarily directs projects related to formal and informal STEM education. Kristin has a B.A. in psychology from Yale University and a Ph.D. in education and psychology from the University of Michigan.

Gabriel Ben-Shalom is a recently graduated teacher, who finished his student teaching with Ms. Lobo at Winton Middle School and became available to teach ASAMI for eighth-grade students.  Gabriel benefitted in his own education from hands-on and conceptually deep activities, and he was eager to be involved in ASAMI, particularly as he witnessed U.S. Science and Math education move into an era of Common Core and NGSS.  He was very delighted when he found the students tackling hard problems and making progress in their own learning.  In fact, as we note in the paper, the eighth-grade students in Gabriel’s class had very large gains on the math Common Core assessment items, which is a tribute to his teaching skills.

Jesus Herdia is an English Language Learner (ELL) Specialist at Winton Middle School in Hayward, CA. He was formerly a teacher, but moved into ELL work when he saw the tremendous potential of these students, coupled with their strong need for activities that engaged and supported their core learning.  Hence, ASAMI spoke naturally to his sense of what the students needed.  Jesus was diligent in working with the families of the students, and through his efforts, we saw very low attrition in the ASAMI classes.  Jesus helped convince the English Language Advisory Committee that ASAMI was in their children’s best interest.   Jesus also played an essential role in the total running and management of ASAMI and was in the ASAMI classroom almost continuously.

Rainbow Lobo is a teacher in the Science Department at Winton Middle School in the Hayward Unified School District.  She teaches science and technology and has been an advocate of hands-on, student-centered learning for most of her career.  Students in her technology elective class demonstrated large gains in their grades after a year of Lobo’s class. She provided ASAMI’s home (her classroom), ideas on classroom management, and continuous input and ideas in this study.

Carl Pennypacker is a physicist and educator who has been fortunate to play pivotal roles in some decent projects. He received his B.A. from UC Berkeley in 1972, with the group of Luis Alvarez. Together with Richard Muller, Pennypacker has helped form and develop many of the central ideas that have led to the discovery of Dark Energy. He and his team were winners of the Gruber Prize and the Breakthrough Prize for this work, and the student he co-advised, Saul Perlmutter, went on to accrete the Nobel Prize for this work. Pennypacker helped co-found, with a group of great teachers and educators, the Hands-On Universe project. This project has led to the training of 1000 teachers in the United States, and about 20,000 around the world, and is part of the French National Curriculum and the Bavarian State curriculum.

Jenifer Perazzo is a Hands-On Universe Teacher Lead. She is also a certified Modeling Instruction teacher. During the school year she is a Science Specialist for an elementary school in Pleasanton, CA. She introduces students and teachers to the EU-HOU astronomical image analysis tool, Salsa J, a software program dedicated to image handling and analysis in the classroom. Jenifer created and taught most of the ASAMI activities for the seventh-grade class.

David R. Stronck is a Professor in the Department of Teacher Education, California State University, East Bay. Oregon State University awarded him an M.S. in Biological Sciences and a Ph.D. in Science Education. He is the sole author of 22 articles reporting statistical research in major journals of learned societies.   He has a total of more than 200 publications, including eight books. For ten years, he was the editor of journals for science teachers. Stronck has been the director of projects that have been funded at more than $3 million.   He has directed or co-directed 15 grants for the National Science Foundation. The Genentech Foundation for Biomedical Sciences funded his projects serving high-school students, for more than one million dollars. He has also directed four grants from the U.S. Dept. of Education. He presents at an average of five different conferences annually, e.g., the National Science Teachers Association.

References

Beaton, A.E. 1996. Mathematics Achievement in the Middle School Years: International Association for the Evaluation of Educational Achievement’s Third International Mathematics and Science Study (TIMSS). Chestnut Hill, MA: TIMSS International Study Center, Boston College.     http://eric.ed.gov/?id=ED406419 (accessed June 24, 2015).

Calderon, M. 2007. Teaching Reading to English Language Learners, Grades 6-12: A Framework for Improving Achievement in the Content Areas. Thousand Oaks, CA: Corwin.

Committee on Guidance on Implementing the NGSS. 2015. Guide to Implementing the Next Generation Science Standards. Washington, DC: The National Academies Press.

Czemiak, C.M., W.B. Weber, A. Sandmann, and J. Ahern. 1999. “A Literature Review of Science and Mathematics Integration.” School Science and Mathematics 99 (8): 421–430. http://onlinelibrary.wiley.com/doi/10.1111/j.1949-8594.1999.tb17504.x/epdf.

Fry, R. 2007. How Far behind in Math and Reading Are English Language Learners? Report. Washington, DC: Pew Hispanic Center. http://eric.ed.gov/?id=ED509863 (accessed June 24, 2015).

National Governors Association Center for Best Practices and the Council of Chief State School Offices. 2010. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Offices.

NGSS Lead States. 2013. Next Generation Science Standards.   Washington, DC: The National Academies Press.

Quinn, H., H. Schweingruber, and T. Keller, eds. 2012. A Framework for K-12 Science Education. Washington, DC: The National Academies Press.

Rolstad, K., K. Mahoney, and G.V. Glass. 2005. “The Big Picture: A Meta-Analysis of Program Effectiveness Research on English Language Learners.”   Educational Policy 19 (4): 572–584. http://epx.sagepub.com/content/19/4/572.short (accessed June 24, 2015).

Short, D., and J. Echevarria. 2004. Teacher Skills to Support English Language Learners. Educational Practice Report 3. Santa Cruz, CA: Center for Research on Education, Diversity and Excellence.

Stoddart, T. 2002. “Integrating Inquiry, Science and Language Development for English Language Learners.” Journal of Research in Science Teaching 39 (8): 664–687.

Appendix 1: Alan Friedman and HOU

Alan Friedman established and directed the Lawrence Hall of Science Planetarium (University of California, Berkeley) in 1973. For over a decade his leadership set the legacy of audience participation planetarium shows and hands-on science at Lawrence Hall. He was a pioneer in the field and involved hundreds of planetariums through Participatory Oriented Planetarium (POP) workshops and the publishing of the Planetarium Educator’s Workshop Guide, which evolved into Planetarium Activities for Successful Shows (PASS; now at http://www.planetarium-activities.org/). To this day LHS helps bring that style of show into the digital age and encourages other digital planetariums to include live audience participation in their repertoire of shows, rather than just recorded programs. Among the planetarium shows Alan developed were Stonehenge and Finding Your Star (now Constellations Tonight), in which the presenter hands out star maps to all the audience members and teaches them how to use them. Using star maps was to become a favorite tool of HOU observers in the guise of Uncle Al’s Hands-On Universe Starwheels. Cary Sneider became Planetarium Director after Alan Friedman, and it was under Cary that the first connection with HOU was made in 1991. Cary had been invited to the seminal HOU organizing workshop but was unable to attend and asked Assistant Director Alan Gould to go in his stead. At the workshop, Alan presented an activity from one of the planetarium shows, Moons of the Solar System, in which the audience members kept track of the moons of Jupiter and discovered the relationship between the moons’ orbital periods and their orbital radii. That ultimately evolved into one of the favorite activities in the HOU high school curriculum. Years later, Alan Gould became Co-Director of HOU for a number of projects.

Appendix 2: Typical ASAMI Activities

ASAMI Activity What Students Do Math Common Core Concept
Derive a correct recipe and then make playdoh Students scale from a recipe that requires too much of one ingredient Ratios and proportion
Make a scale map of their school, from Google Maps Use Google Maps and HOU image processing to measure true diameters of objects and measure their school, culminating in a scale map of some buildings, etc. Ratios, proportion, scale, measurement
Make a scale solar system Students take an existing playdoh recipe and scale it for the smaller amount of materials they are given Ratios and proportion
Lunar Craters – find a lunar crater as big as your county from computer images Students find a crater as big as their county, plot a map of the State of California on a moon map, use different map scales and compare maps. Proportion and ratios
Asteroid Impact – drop various size stainless steel balls into birdseed on a tray Students drop various mass spherical objects into bird seed (works better than flour) from various heights, and plot crater size versus height, mass, etc. Energy, proportion, mass, etc.
Water Rockets Build and launch, then measure and graph results from experiments with water rockets Proportion

 

Appendix 3: Model Building in the NGSS

Practices Matrix from the NGSS (http://www.nextgenscience.org/next-generation-science-standards): Model-building in NGSS. The word “model” has been highlighted by the authors of this article.

 

 

From Generation to Generation: Incorporation of Intergenerational Informal Science Education into an Introductory College Science Course

 

Linda Fuselier,
University of Louisville

Abstract

Restoration of forest ecosystems following the loss of biodiversity associated with non-native species invasions is an issue of civic consequence that has the potential to engage audiences of all ages, backgrounds, and abilities. In this project, the strong sense of community connection felt toward a local forest preserve was leveraged to inspire native plant seed collection, propagation, and planting for a community-driven forest restoration project. As part of a larger project, informal science education was integrated into a general education environmental science course to engage college students in this civic project and in intergenerational community building. The introduction of students to informal science education (ISE) through collaboration with an outdoor education center was successful at increasing awareness of ISE as a potential career path, developing environmental science content knowledge, inspiring interest in restoration projects among elder participants, and building community. Intergenerational workshops resulted in bidirectional knowledge exchange among participants related to a strong sense of place shared by both generations.

Background

In 2013, a partnership between a small liberal arts college and an environmental outdoor education center was funded through a SENCER-ISE II grant to infuse civic engagement into informal science learning and integrate informal science education into higher education science teaching. During the first year of the grant work, college students, middle-school students, senior adults, and partnership institutions became an intergenerational community of practice centered around the critical issue of biodiversity loss through species invasions. The overall project included multiple components: young students collecting seeds of native plants, college students cleaning and propagating plants and initiating restoration research, and older community members participating in civic engagement activities related to restoration. The focus of this article is on the incorporation of informal science education methods into a general education, first-year college environmental science course using intergenerational learning and civic engagement. The intention of this portion of the larger project was to enhance student learning and promote community building by involving senior adults and college students in an intergenerational learning experience. The project combines aspects of informal science education with intergenerational learning and civic engagement. The project was designed to strengthen the link between environmental science learning and action (Ballantyne et al. 1998) by engaging participants in a topic relevant to their lives and involving them in interactive learning (Falk 2001).

 

Introduction

Informal Science Education and Civic Engagement

“Experiences in informal environments for science learning are typically characterized as learner-motivated, guided by learner interests, voluntary, personal, ongoing, contextually relevant, collaborative, nonlinear, and open-ended” (National Research Council [NRC] 2009, 11). In formal venues, learning is compulsory, structured, and teacher-centered, with content more central than social aspects of learning (Wellington 1990). Non-formal learning, a process that fits between formal and informal learning, is more structured but is more easily adaptable than formal education (Eshach 2007). The numerous definitions of informal, non-formal, and formal learning were recently reviewed by Stocklmayer et al. (2010). In this study, informal learning is understood as taking place outside of the classroom; it is learner-centered, includes both academic and social aspects of importance and, although it is not entirely unstructured, it relies to some degree on the learner’s intrinsic motivations (Wellington 1990; Malcolm et al. 2003; Martin 2004). Research in teaching and learning in informal settings shows that, among other benefits, informal science education (ISE) is effective in increasing interest and engagement in science and increasing general scientific literacy, (Bouillion and Gomez 2001; NRC 2009; Stocklmayer et al. 2010), and that ISE is pertinent throughout a learner’s lifetime (NRC 2009).

Because informal learning is personal and relevant as well as voluntary (NRC 2009), it is necessarily related to learning through civic engagement. In the spirit of SENCER, civic engagement is both personal and relevant, because society is replete with “wicked problems” that resist simple resolution and require interdisciplinary approaches grounded in civic responsibility (Lawrence 2010). In this sense, learning through civic engagement is similar to community-based service learning in that it is a meaningful connection between students and community, where students use new skills in real-world situations to serve their community. Experiential learning through civic engagement and tackling capacious problems takes this one step further; it exposes the interconnections that make problems “wicked” and promotes deeper learning on the part of both the students and the community. Service learning and civic engagement may be especially important in environmental education where there is a risk of leaving students feeling despondent and powerless as they learn more about environmental issues (Hillcoat et al. 1995). Service and civic engagement have the potential to awaken agency and empower students to make change (Bloom and Holden 2011).

Community-based service learning at its best encourages reflection that promotes civic responsibility, academic success, and personal growth (Arenas et al. 2006). Service learning increases awareness of environmental issues, conservation knowledge, enjoyment of nature, student motivation and engagement in school, and strengthens bonds between community members (Schellner 2008). Importantly, positive environmental attitudes and behaviors ignited through service lasted beyond the service-learning experience (Schellner 2008).

Intergenerational Learning and Community Building

The new generation of older people lead active lifestyles and have interest in future-oriented activities that promote personal fulfillment and social integration characteristic of the “active aging paradigm” (Chadha and Malik, 2010). This project leverages the desire for continued lifelong learning and significant community involvement among elders to facilitate civic engagement through intergenerational learning. Intergenerational learning opportunities are most often defined as occurring with youth under age 21 and adults over age 60 (Kaplan 1997; 2002) and are common in fields of social and health sciences (Roodin et al. 2013). Intergenerational learning programs create intentional exchange of resources and learning among generations (Kaplan 2002). Importantly, intergenerational learning is based on reciprocity of benefit and thus is expected to be mutually beneficial for all generations involved (Ellis and Granville 1999; Tam 2014). Lifelong learning may be intergenerational but typically takes place in informal settings (reviewed in Brostrӧm 2003); thus, the articulation of intergenerational learning in informal settings is a natural combination with potential to enhance education and community connectedness.

Intergenerational learning programs have been successful with a range of age groups in a variety of venues, though most of the documentation of their success comes from students working in gerontology (Roodin et al. 2013). There were both curriculum and relationship-based benefits from a service-learning course in which college students worked with elderly participants (Tam 2014). Community elders working with primary school students (Peterat and Mayer-Smith 2006) showed cross-generational social learning and reciprocity of benefit. On a much larger scale, the Granddad Program in Sweden was successful at bringing senior adult male role models into schools as volunteers (Brostrӧm 2003). Many community-based intergenerational experiences focus on environmental activism, and seniors make especially good environmental steward role models, because they possess the self-motivation for protecting the Earth for future generations (Ballantyne et al. 1998). When seniors were incorporated into a residential outdoor education program, children who worked with senior adults (as compared to the control group) gained more information on a wider variety of topics, and there was a trend toward improved environmental attitudes (Shih-Tsen and Kaplan 2006). In an ISE program, seniors were paired with students on an urban farm, and program participants showed increased environmental awareness associated with the experience (Mayer-Smith et al. 2007).

The benefits of intergenerational service learning programs are well documented (see reviews in MacCallum et al. 2006 and Roodin et al. 2013), and are generally classified as relationship-based and curriculum-based (Tam 2014). Through bidirectional information flow including sharing life experiences and constructive knowledge exchange, participants increase their understanding of each other (Springate et al. 2008). Intergenerational learning programs or courses have the effect of reducing age-related stereotypes (Kaplan, 1997), with students reporting a more positive and appreciative attitude towards the older generation (Zucchero 2009 and 2011; Penick et al. 2014). Benefits to the elderly include benefits attributed to lifelong learning (Brostrӧm 2003): improved self-esteem and life satisfaction (Newman et al. 1997), physical, social and psychological as well as economic benefits (Tam 2011; 2014), maintenance of cognitive functioning (e.g., Ardelt 2000; Boulton-Lewis et al. 2006; reviewed in Tam 2014), and promotion of pro-social values (Brostrӧm 2003).

The benefits to youth from intergenerational learning are better documented than benefits to college students. Intergenerational learning experiences are reported to increase confidence and self-worth and improve practical skills among youth (MacCallum et al. 2006). Youth involved in intergenerational activities showed increased enjoyment in school, were less likely to become involved with drugs, displayed enhanced literacy development (MacCallum et al. 2006) and became more civic-minded and viewed their citizenship in more action-oriented terms (Kaplan 1997). Although many intergenerational service-learning experiences involve young children, working with college students has been shown to enhance the general well-being of older adults also (Hernandez and Gonzalez 2008). Our project adds to this literature by documenting bidirectional information flow and a sense of community belonging among college students and elders.

Project Description

Antioch College and the Glen Helen Outdoor Education Center (OEC) are situated in a Midwestern USA town of approximately 3500 residents, where the median age is 48 and the population is aging; approximately 47.5 percent of the population is aged 50 and older (US Census Bureau 2010).   The College has approximately 200 students and very small class sizes. The OEC is within close walking distance to the college campus.   Over 2700 grade school students and in-service teachers participate in educational programs that meet state teaching standards and are designed and led by a team of paid and trained naturalists at the OEC. The OEC is located within the city limits in a 1000-acre nature preserve (Glen Helen or The Glen) that receives over 10,000 visitors annually and is an important part of the local community.

We used the critical community issue of biodiversity loss to involve students and community members in forest restoration in the local nature preserve. The Glen encompasses a forest ecosystem negatively impacted by invasive species, most notably by bush honeysuckle. Bush honeysuckle has been documented to prevent growth of native understory plants through resource competition, allelopathy, and depleted soil seed banks (Cipollini et al. 2008; Cipollini et al. 2009; McKinney and Goodell 2010; Arthur et al. 2012; Bauer et al. 2012). Forests with invasions of bush honeysuckle also have lower amphibian species diversity and richness, altered patterns of pollinator visitation, song bird assemblages, and soil fungal communities, higher soil compaction, lower soil quality, and lack of certain other qualities that are indicators of a healthy forest understory (Watling et al. 2011). Restoration of forest ecosystems following invasive species removal is dependent on replanting native forest understory species and involves the consideration of numerous intertwined ecological principles that must be in place to sustain and promote the return and establishment of a biodiverse community (Vidra et al. 2007; Swab et al. 2008; Aronson and Handel 2011). Through this project, youth at the OEC, college students, and senior adult community members participated in the propagation of native plants for a forest restoration project in Glen Helen.

As part of our project, college students in the course entitled Introduction to Environmental Science visited the OEC, observed a naturalist-led hike, studied native and invasive species in class and in the Glen, and offered plant propagation workshops to senior adults at a local senior center. Workshops in which students participated were held in the “great room” at the Center, a large, open area. Eight tables with planting supplies were situated in a circle around the room and each table was attended by a student with a different native plant species to propagate. Chairs were arranged so participants could sit or stand at stations and there was ample room for moving from station to station. The workshop began with an introduction to the project, invasive species impacts, and restoration efforts in the Glen. Then participants were encouraged to help clean or plant seeds at any of the stations and to move among stations. The effect was to optimize personal, intergenerational interactions in an experience with direct relevance to people with some connection to the Glen.

The objectives of this curriculum innovation were to

  1. Introduce students to informal science education (ISE) as a potential career path
  2. Teach content knowledge related to invasive species and biodiversity loss
  3. Design and implement an intergenerational learning opportunity that results in bidirectional knowledge sharing

The workshops were designed to engage older adults and college students in meaningful work and ultimately create a sense of community purpose while encouraging environmental responsibility and civic engagement. This type of community connection through active civic engagement promotes the personal fulfillment and social integration sought by elder community members (Chadha and Malik 2010). College students benefit from working with adults of a different generation and forming ties that spill over and enhance community life (Roodin et al. 2013).

Methods

There were two primary activities in the curriculum design; one introduced students to ISE and the second put the students into the position of informal science educators in an intergenerational workshop. We scaffolded the student-led workshops by introducing students to the OEC and having them observe and reflect upon an informal science lesson. The class walked to the OEC at the beginning of the quarter to meet the Director, tour the facility, and discuss OEC programs. During the quarter, students were required to attend one naturalist-led hike, observe the lesson, and submit a reflective assignment within two weeks of completing the hike. The reflection activity included a description of the lesson, suggestions on how to improve or extend the experience, and thoughts on the importance of ISE in education. Two weeks before the workshops, students participated in class work that introduced them to the project, biodiversity, and issues related to invasive species. They chose a native plant (from a list of those available) and completed individual research on the natural history of the plant. Students designed and printed an information sheet on the species and were told to be prepared to describe their species and the project and to answer questions during the workshops. They submitted the species information sheet for feedback and grading before the workshops. Students were divided into two groups to offer two workshops at the local senior center during February 2014. In the workshops, students managed their own “propagation stations,” provided information on their native plants, and cleaned and planted seeds with workshop participants. Students had learned seed cleaning and planting before the workshops in a separate classroom activity.

Students taking the class in fall 2013 participated in the naturalist-led hikes, but workshops were offered only during winter 2014 quarter. Thus, included here are two sets of student reflections on OEC involvement and one set (winter quarter) of workshop assessments. Student responses to an open-ended question on the hike reflection assignment were coded using presence/absence codes based on the assessment prompts (Table 1). Codes included experience (positive or negative), expressed interest in ISE (yes or no), and recognition of ISE as important to the student’s education (yes or no). Two additional codes were added to the analysis of the winter quarter reflections: awareness of ISE before the class (yes or no), and whether or not students noted learning something that they previously did not know about ISE (new learning). To further quantify interest in ISE, students were asked in 2014 if they were interested in a cooperative working experience (co-op) as a naturalist assistant. They could answer yes, no, or maybe and were asked to provide an explanation of their choice. Given the presence/absence format of codes, there was very little room for interpretation. A second coder, unfamiliar with the project, coded the same student responses; the inter-coder reliability, calculated as the proportion of individual excerpts and codes that the individual coders applied similarly, was 95 percent.

To assess knowledge sharing and community building during the workshops, students completed workshop reflection sheets, and older adult participants were asked to complete a post-workshop survey before leaving the Senior Center. Before the start of the workshop, students were asked to keep a tally of the number of participants with whom they interacted and to remember conversation topics. Students completed the reflection sheet immediately at the end of the workshop. The survey for older adult participants included ten statements with 10-point anchored responses that ranged from 1 (not at all) to 10 (very much or a great deal) with the prompts “How much did this workshop…” and “To what degree…” and a space for additional comments.

Four exam questions were used to evaluate student content knowledge about biodiversity and invasive species: (1) What are the five major threats to biodiversity that we discussed in class? (2) What is the number one cause of the loss of biodiversity on the planet?   (3) Outside of bush honeysuckle, what are two additional examples of invasive species that are negatively impacting ecosystems in the USA?  (4) Bush honeysuckle and other invasive plants impact native plants by shading, competition for space and soil nutrients. Describe two additional negative impacts that this invasive has on natural ecosystems (outside of impacts on plants under the honeysuckle). In addition to these questions, students were asked to rate the extent of their knowledge about bush honeysuckle as an invasive species compared to their knowledge before they started the class. Answers were on a five-point Likert scale ranging from none to very high. 

Results

Naturalist-led Hikes

Students who attended their required naturalist hike and submitted a reflection assignment all provided adequate descriptions of the lesson and responded to additional questions appropriately. This indicated that the students attended and engaged in the lesson. Students had an enjoyable experience at the OEC, expressed interest in ISE, recognized the importance of informal learning opportunities and in most cases were interested in additional ISE experiences.   Some students noted that the cold weather was the only aspect of the experience that they did not enjoy, but 100 percent of students in both classes described positive experiences overall.

Some students began with an interest and strengthened or acknowledged that interest, whereas others gained interest in ISE through their participation in the hike at the OEC. Interest ranged from very interested to no interest (Table 1) and, 86 percent (fall) and 87 percent (winter) of students expressed interest in ISE. Students who expressed interest in ISE, recognized ISE as a potential career path and a way to garner teaching experience. One student wrote, “…I am very interested, in fact, that is what I hope to do as a career.” Another wrote, “I am definitely interested in informal science education…. Even if I do not choose being an educator in my profession, I will probably run into a situation where I will be teaching in some way, and informal education can be a great option to handle this opportunity.” One student was interested in education but not specifically informal science education: “…I am somewhat interested in education as a possible career. I’m not entirely sure if informal science education would be the specific career path….” For some students, their experience at the OEC led them to reconsider ISE: “Before this hike I would not have believed I had any interest in informal science education [;] however now I believe I might,” whereas another student, even after this experience, was “still not very interested in informal science education … I have other things that I want to do.” It is not possible to determine whether the lack of interest was because it was specifically science education; none of the students were science majors.

In 2014, when asked about interest in a co-op work position as a naturalist assistant, of the twelve students who replied, only two gave a negative response; the others chose either yes or maybe. The two students who were not interested explained their response by their lack of knowledge in science, lack of interest in working with children, and the need for experience related to their non-science major. Although these two students did not recognize how this experience might benefit them regardless of their major, another student commented, “I would say it’d be a better fit for an environmental science major, or someone who has a bigger interest in being a teacher someday! However, I think it’d be a good experience to have and I would consider it!” Two students who chose “yes” and one who chose “maybe” specifically tied their response to their positive experience on the naturalist-led hike.

Almost all students in both classes (87 percent in fall and 100 percent in winter) provided anecdotes describing the importance of informal learning to their education or, more commonly, in educating youth in environmental science. Many students provided examples of their own positive experiences with informal science education at their grade and secondary schools and through interactions at nature centers. No one described a negative experience with informal science education, and most were very interested in the “outdoors,” and especially in learning more about the specific nature reserve used in this project.

Among the students who described themselves as previously aware of informal science education (86 percent, n = 7, in winter quarter), five of them described how their view changed after the hike. Two admitted that before their experience in the class, they had different concepts of what it meant to work in informal science education (e.g., park ranger). Two students gained appreciation for ISE: “…I never knew how amazing it was” and “Before this hike I knew what informal science education was but I never really considered it as one of the career paths….”   One became aware of the OEC for the first time and another gained awareness of the importance of naturalist jobs: “Looking back however I can understand the importance of her [the naturalist’s] job and of other careers such as hers.”

Increased awareness was often tied to “new learning” about ISE. Although the assessment prompt did not specifically ask about new understanding, half of the students in the 2014 class indicated that they learned something new about ISE through their experience. For example, one student commented, “Visiting the OEC gave me a different perspective on the types of education I might be suited for or interested in” and another, “I had not thought very much about a career in informal science education but now I definitely see how important it is to teach young ones about nature.”

Table 1. Prompts for college student reflection about their experience on a naturalist hike at the outdoor education center
Class Prompt
Fall 2013

(n = 15)

“Write a short paragraph about your experience with the OEC. Include whether or not you might be interested in informal science education and how informal science education has been or may be important to your education.”
Winter 2014

(n = 12)

Same text as above with the following addition: “Were you aware of environmental education/informal science education as a career before this exercise?”

 

Senior Adult Workshops

The workshops received very positive reviews from students and adult participants. The reflections that the participants provided on the surveys indicated that the workshops facilitated bidirectional sharing of knowledge across generations and a sense of community building. One shortcoming of the workshops was that they occurred during a particularly cold and snowy winter, which limited attendance by senior adults.   There were eight students at each workshop and twelve adult participants at the first and only six at the second workshop. Not all participating adults chose to complete a post-workshop survey, and so, our sample sizes for adult reflections are low. The structure of the workshops encouraged adult participants to move from station to station and interact with several students. Thus, although the number of participants was low, all students had the opportunity to engage with multiple participants during the course of the workshop.

Bidirectional Knowledge Sharing

Post-workshop surveys completed by students showed that on average, each student shared their knowledge of native plants with four adult participants and, on average, three older adult participants shared knowledge with the student. Students listed the types of information that they shared with adult participants, which included information on the plant’s habitat, pollination, use of natural insecticides, forest understory, mesic wetlands, similarities to other plants, planting methods, germination requirements, types of plants (herbaceous and woody), and invasive species impacts. The responses indicated that students were synthesizing and sharing what they had previously learned in class as part of this project or other class activities.

The examples that students provided indicated that participants shared their knowledge of plants as well as general knowledge about a wide range of topics.  Students commented that they learned about tree diseases, organic gardening methods, the history of the Glen, how to recognize some native flowers, and how seeds are dispersed. Adult participants were sharing their expertise with students while the students shared information with them. For example, when asked to provide examples of knowledge shared by participants, students wrote

“One woman talked about the dogwoods she had….”

“…the paw paw festival and different kinds of paw paw cultivation…”

“…the trees [she] saw in the Glen…”

“past/current gardening experiences, talking about their lives in general…”

“…The seeds are long because they can be carried easier by the wind….”

“…got a great book recommendation” and

“I feel like I learned a lot from those who visited my station.”

Sense of Community

Student reflections revealed a positive sense of community connectedness. For example, some student responses to the prompt “How did the experience influence your connection to the community (outside of the campus community) and connection to the Glen?” included

“It felt good to chat with community members and to see how they feel about…”

“I loved to meet members of the community … and get to hear their stories.”

“I was able to make connections based on common interests”

“…It made me feel more connected and more open to the community.…”

“I felt more strongly connected to both the Glen and the community, particularly because we took action to improve the Glen with the help of the community.”

 

And several students indicated a desire to become more involved in the community:

“…encourage me to reach out more to the community at large; they are awesome!”

“I would like to … be more involved with the Yellow Springs community.”

Among the eighteen adult participants in the two workshops, only 14 elected to complete a survey. The highest rated survey questions were “To what degree did you enjoy interactions with students?” and “How much did this workshop increase your interest in getting involved further in Glen Helen restoration efforts?” (Figure 1). On average, all responses were over six out of ten possible levels and indicated an overall satisfaction with the workshops. Interestingly, older adults did not feel that they shared their knowledge with students to the same degree that they increased their own knowledge and that students shared with them. This is contrary to the student’s description of knowledge exchange and appreciation for information shared by older adults. Older adult participants liked the degree of interaction possible in the workshop and expressed a stronger personal connection to the community as a result of their participation.

Content Knowledge

Exam questions for students in the environmental science class were graded as “all or none” to assess content knowledge. Fourteen students completed the four assessment questions included on their exam in winter 2014. Among those 14 students only two described their prior knowledge of honeysuckle as an invasive plant as high and both of these students had some experience working with invasive plant removal in the Glen through other opportunities. All students identified the most common cause of biodiversity loss and correctly listed invasive species in addition to bush honeysuckle; 93 percent were able to provide additional negative impacts of honeysuckle on an ecosystem, and 86 percent correctly listed five threats to biodiversity. Despite their perceived initial lack of knowledge about honeysuckle as an invasive species, students gained knowledge about invasive species during the course of the class activities.

Discussion

Students increased their understanding of informal science education, biodiversity, and invasive species impacts and strengthened connections to the local community through participation as informal science educators in intergenerational plant propagation workshops. The naturalist-led hikes provided students with concrete examples of informal science education in action and appropriate scaffolding for stepping into the role of informal science educator. Students and senior adults alike were extremely positive about the workshops, and within the workshops there was successful bidirectional, cross-generational information sharing.

Student participation in naturalist-led hikes as an introduction to ISE was successful at stimulating interest in and increasing awareness of ISE as a potential career path among college students. This project focused on increasing awareness of the OEC as a local environmental education resource and the potential for students to participate in ISE as part of their science career. Other studies have shown that students’ career planning was enhanced and that they changed their beliefs about careers following short summer programs (Barnett et al. 2011). Anecdotally, there is an indication that the interest in ISE persisted among students: one student applied to the OEC for a paid naturalist position.

The combination of ISE, intergenerational learning and civic engagement with college student participants is relatively unique. Informal science education programs at museums or zoos (NRC 2009), for example, are generally designed for unidirectional knowledge flow from an educator to a diverse public audience. Many intergenerational learning programs at the college level are situated in gerontology programs and often these programs neither promote nor are designed for bidirectional knowledge exchange (Roodin et al. 2013; Tam 2014). Such programs are more correctly deemed multigenerational rather than intergenerational (Tam 2014). In the case of this project, workshops were truly intergenerational, and bidirectional knowledge sharing was easily documented. Sharing of knowledge between students and older adult participants suggests that academic knowledge was in no way privileged over community knowledge (Trickett 1997), and this epistemic equality promoted knowledge flow and, most likely, community connectedness.

Community building as an objective of informal science education and intergenerational learning is based in the theoretical framework described as tapping in to “funds of knowledge” (Basu and Barton 2007). These “funds” are the cultural and historical knowledge residing in the community. Communication of this community knowledge may enhance science education by making science more relevant to the lives of students (Basu and Barton 2007). In this project, intergenerational workshops were described by students as strengthening community connectedness, and the appreciation that students expressed for the knowledge shared with them by senior adults appeared to enhance this community connection and support the overall positive evaluation of the experience.

The success of intergenerational experiences in the context of civic engagement is dependent in large measure upon choosing a critical issue whose approach serves both the public and academic communities.   For this project, it was the connection to place, Glen Helen, that was a driving force for a successful program. Place-based experiential learning has been shown to enhance undergraduate student content knowledge in the plant sciences (Bauerle and Park 2012) and influence individual agency related to environmental issues (Rodriguez et al. 2008; reviewed in McIreneny et al. 2011) and public participation in science (Haywood 2014).   Glen Helen is a valued resource in the community, and satisfaction with the workshops was related to the perception that older adult participants were helping the Glen. Workshops also stimulated interest in being involved with Glen Helen restoration projects, and student reflections on the naturalist-led hikes indicated an interest in learning more about Glen Helen.

Students demonstrated an understanding of content related to invasive species, biodiversity, and native plants on an exam, but more impressively, students communicated content knowledge to adult participants in workshops. Communication of their knowledge to community members indicates that students have some confidence in their abilities and understanding of science. When graduate education students assumed the role of informal science educators, they honed communication skills and increased their confidence in using skills and knowledge gained in the classroom (Crone et al. 2011).

The success of the workshops and the project overall can to some degree be attributed to the consideration of recommendations from previous research on intergenerational service learning. In general, students benefit from authentic learning and participatory experience coupled with structured reflection (NRC 2009). This was incorporated into the project in the form of an educator-community partnership rooted in a civic issue relevant to the lives of participants. Intergenerational ISE programs are best when they incorporate opportunities for significant personal interaction (Fenichel and Schweingruber 2010), something that the senior adults prized about their workshop experiences. It is also important that there is a potential for one-on-one interactions and that programs proceed at a leisurely pace (Shih-Tsen and Kaplan 2006) and take into consideration the mobility or limitations of participants. This project offered student-led workshops that had all of these characteristics.

Shortcomings of the project are primarily related to the low participation by older adults and the lack of a control group. Attendance at the workshops was complicated by poor weather, and this is especially pertinent for older adults who may experience decreased mobility. Winter was chosen as the best time for propagation workshops based on the college schedule and conditions needed for germination and establishment of plant stock for the restoration project. Thus, there was a trade-off between appropriate conditions for participants and logistics imposed by the academic and research schedules. Why some senior adults chose not to complete a survey is not clear. Also, it is not possible to know whether student content knowledge was enhanced as a result of the intergenerational interactions, because there was no control group for comparison. Additionally, because some assignments were graded, it is possible that some student responses lack sincerity, but we have no way of knowing whether this is true. Despite low numbers, results indicate a very positive response by both students and adult participants that is sufficient to warrant scaling up the project.

Whether the benefits of the experience are long-lasting or coupled with increased environmental activism is unknown but an interesting question for further research. Civic engagement tends to increase among students who participate in service learning with older adults (Hegeman et al. 2010; Karasik et al. 2004), and these interactions with a larger community may influence personal ecological identities (Morris 2002). Thus, it is possible that programs that combine ISE, civic engagement, and intergenerational learning yield benefits far beyond those documented for this project.

About the Author

Linda Fuselier is faculty in the Biology Department at the University of Louisville where she is responsible for the redesign of a large enrollment, non-majors biology curriculum. Before moving to Louisville, Linda was at Minnesota State University, Moorhead where she was Biology faculty and Director of Women’s and Gender Studies. At MSUM, she worked with biology faculty to infuse research into the undergraduate curriculum and formed an interdisciplinary team of faculty tasked with infusing the SENCER approach into biology, physics and Women’s Studies classes. Her current research involves creating SENCER-based modules using contemporary women’s issues and designing curriculum materials to for science information literacy.

Acknowledgements

Janene Giuseppe, Director of the OEC, coordinated naturalist-led hikes and collaborated on numerous aspects of the project. Corinne Pelzl, Activities Director at the Yellow Springs Senior Center, worked with the College and hosted and helped advertise and organize workshops. Yellow Springs community members and Environmental Science students participated voluntarily in the project and the project received exempt status through the Antioch College IRB. Glen Helen Ecological Institute permitted native plant restoration and George Bieri, Land Manager, provided expertise on local flora and assisted the restoration project. This project was funded by SENCER-ISE II grant from NCSCE.

Literature Cited

Alliance for Service-Learning Education Reform and Close UP Foundation. 1995. Standards of Quality for School-Based And Community-Based Service Learning. Alexandria, VA. http://tncampuscompact.org/files/qsbc.pdf (accessed May 18, 2015).

Ardelt, M. 2000. Intellectual Versus Wisdom-Related Knowledge: The Case for a Different Kind of Learning in Later Years of Life. Educational Gerontology 26: 771–789.

Arenas, A., K. Bosworth, and H.P. Kwandayi. 2006. Civic Service through Schools: An International Perspective. Compare 36 (1): 23–40.

Aronson, M., and S. Handel. 2011. Deer and Invasive Plant Species Suppress Forest Herbaceous Communities and Canopy Tree Regeneration. Natural Areas Journal 31: 400–407.

Arthur, M., S. Bray, C. Kuchle, and R. McEwan. 2012. The Influence of the Invasive Shrub Lonicera maackii, on Leaf Decomposition and Microbial Community Dynamics. Plant Ecology 213: 1571–1582.

Ballantyne, R., S. Connell, and J. Fien. 1998. Students as Catalysts of Environmental Change: A Framework for Researching Intergenerational Influence through Environmental Education.   Environmental Education Research 4: 285–299.

Barnett, M., M.H. Vaughn, E. Strauss, and L. Cotter. 2011. Urban Environmental Education: Leveraging Technology and Ecology to Engage Students in Studying the Environment. International Research in Geographical and Environmental Education 20: 199–214.

Basu, S.J., and A.C. Barton. 2007. Developing a Sustained Interest in Science among Urban Minority Youth. Journal of Research in Science Teaching 44: 466–489.

Bauer, J., S. Shannon, R. Stoops, and H. Reynolds. 2012. Context Dependency of the Allelopathic Effects Of Lonicera maackii on Seed Germination. Plant Ecology 213: 1907–1916.

Bauerle, T. L., and T.D. Park. 2012. Experiential Learning Enhances Student Knowledge Retention in the Plant Sciences. HortTechnology 22: 715–718.

Bouillon, L. M., and L.M. Gomez. 2001. Connecting School and Community with Science Learning: Real World Problems and School-Community Partnerships as Contextual Scaffolds. Journal of Research in Science Teaching 38: 878–898.

Boulton-Lewis, G.M., L. Buys, and J. Lovie-Kitchin. 2006. Learning and Active Ageing. Educational Gerontology 32: 271–282.

Brostrӧm, A. 2003. Lifelong Learning, Intergenerational Learning and Social Capital: From Theory to Practice. Institute of International Education: Stockholm University, Stockholm, Sweden. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.198.7003&rep=rep1&type=pdf (accessed May 17, 2015).

Carrus, G., M. Scopelliti, R. Lafortezza, G. Colangelo, F. Ferrini, F. Salbitano, M. Agrimi, L. Portoghesi, P. Semenzato, and G. Sanesi. 2015. Go Greener, Feel Better? The Positive Effects of Biodiversity on the Well-being of Individuals Visiting Urban and Peri-urban Green Areas. Landscape and Urban Planning 134: 221–228.

Chadha, N., and N. Malik. 2010. Intergenerational Learning Enhances Community Well Being. Indian Journal of Gerontology 24: 403–410.

Chen W., and C. Jim. 2008. Assessment and Valuation of the Ecosystem Services Provided by Urban Forests. In Ecology, Planning, and Management of Urban Forests: International Perspectives, M.M. Carreiro, Y-C. Song, and J. Wu, eds., 53–83. New York: Springer.

Cheng, J., and M.C. Monroe. 2012. Connection to Nature: Children’s Affective Attitude toward Nature. Environment and Behavior 44: 31–49.

Cipollini, K., G. McClain, and D. Cipollini. 2008. Separating Above and Belowground Effects of Alliaria petiolata and Lonicera maackii on the Performance of Impatiens Capensis. The American Midland Naturalist 160: 117–128.

Cipollini, K., E. Ames, and D. Cipollini. 2009. Amur Honeysuckle (Lonicera maackii) Management Method Impacts Restoration of Understory Plants in the Presence of White-tailed Deer (Odocoileus virginiana). Invasive Plant Science and Management 2: 45–54.

Crone, W.C., S.L. Dunwoody, R.K. Rediske, S.A. Ackerman, G.M. Zenner Petersen, and R.A. Yaros. 2011. Informal Science Education: A Practicum for Graduate Students. Innovative Higher Education 36: 291–304.

Ellis, S. W., and G. Granville. 1999. Intergenerational Solidarity: Bridging the Gap through Mentoring Programmes. Mentoring and Tutoring: Partnership in Learning 7 (3): 181.

Eshach, H. 2007. Bridging In-school and Out-of-school Learning: Formal, Non-Formal and Informal Education. Journal of Science Education and Technology 16: 172–190.

Falk, J.H. 2001. Free-choice Science Learning: Framing the Discussion. In Free-choice Science Education: How We Learn Science outside of School, J.H. Falk, ed., 3–20. New York: Teachers College Press.

Fenichel, M., and H.A. Schweingruber. 2010. Surrounded by Science: Learning Science in Informal Environments. Board on Science Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Haywood, B.K. 2014. A “Sense of Place” in Public Participation in Scientific Research. Science Education 98: 64–823.

Hegeman, C., P. Roodin, K. Gilliland, and K.B. O’Flathabháin. 2010. Intergenerational Service Learning: Linking Three Generations: Concepts, History and Outcome Assessment. Gerontology and Geriatrics Education 31: 37–54.

Hernandez, C., and M.Z. Gonzalez. 2008. Effects of Intergenerational Interaction on Aging. Educational Gerontology 34: 292–305.

Hillcoat, J., K. Forge, J. Fien, and E. Baker. 1995. “I Think It’s Really Great That Someone Is Listening to Us…”: Young People and the Environment. Environmental Education Research 1: 159–171.

Kaplan, M. 1997. The Benefits of Intergenerational Community Service Projects. Journal of Gerontological Social Work 28: 211–222.

Kaplan, M.S. 2002. Intergenerational Programs in Schools: Considerations of Form and Function. International Review of Education 48: 305–334.

Karasik, R., M. Maddox, and M. Wallingford. 2004. Intergenerational Service Learning across Levels and Disciplines: One Size (Does Not) Fit All. Gerontology & Geriatrics Education 25: 1–17.

Lawrence, R.J. 2010. Beyond Disciplinary Confinement to Imaginative Transdisciplinarity. In Tackling Wicked Problems through the Transdisciplinary Imagination, V.A. Brown, J.A. Harris, and J.Y. Russell, eds., 16–29. Washington, DC: Earthscan.

MacCallum, J., D. Palmer, P. Wright, W. Cumming-Potvin, J. Northcote, M. Brooker, and C. Tero. 2006. Community Building through Intergenerational Exchange Programs. Report to the National Youth Affairs Research Scheme (NYARS). Australian Government Department of Families, Community Services and Indigenous Affairs (FaCSIA) on behalf of NYARS. http://www.academia.edu/604333/Community_building_through_intergenerational_exchange_programs (accessed May 20, 2015).

Malcolm, J., P. Hodkinson, H. Colley. 2003. The Interrelationships between Informal and Formal Learning. Journal of Workplace Learning 15: 313–318.

Martin, L.M.W. 2004. An Emerging Research Framework for Studying Informal Learning and Schools. Science Education 88: S71–S82.

Mayer-Smith, J., O. Bartosh, and L. Peterat, 2007. Teaming Children and Elders to Grow Food and Environmental Consciousness. Applied Environmental Education and Communication 6: 77-85.

McIreneny, P., , J. Smyth, and B. Down. 2011. “Coming to a Place near You?” The Politics and Possibilities of a Critical Pedagogy of Place-based Education.   Asia-Pacific Journal of Teacher Education 39: 3–16.

McKinney, A., and K. Goodell. 2010. Shading by Invasive Shrub Reduces Seed Production and Pollinator Services In A Native Herb. Biological Invasions 12: 2751–2763.

Morgan, R.E., R.L. Bertera, and L.A. Reid. 2007. An Intergenerational Approach to Informal Science Learning and Relationship Building among Older Adults and Youth. Journal of Intergenerational Relationships 5: 27–43.

Morris, M. 2002. Ecological Consciousness and Curriculum. Journal of Curriculum Studies 34: 571–587.

National Research Council; Committee on Learning Science in Informal Environments, P. Bell, B. Lewenstein, A.W. Shouse, and M.A. Feder, eds. 2009. Learning Science in Informal Environments: People, Places, and Pursuits. Board on Science Education, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Newman, S., C.R. Ward, T.B. Smith, J.O. Wilson, and J.M. McCre. 1997. Intergenerational Programs: Past, Present and Future. Washington, DC: Taylor and Francis.

Penick, J.M., M. Fallshore, and A.M. Spencer. 2014. Using Intergenerational Service Learning to Promote Positive Perceptions about Older Adults and Community Service In College Students. Journal of Intergenerational Relationships 12: 25–29.

Peterat, L., and J. Mayer-Smith. 2006. Farm Friends: Exploring Intergenerational Environmental Learning. Journal of Intergenerational Relationship 4: 107–116.

Rodriguez, A., C. Zozakiewicz, and R. Yerrick. 2008. Students Acting as Change Agents in Culturally Diverse Schools. In The Multiple Faces of Agency: Innovative Strategies for Effecting Change in Urban School Contexts, A. Rodriguez, ed. Rotterdam, The Netherlands: Sense Publishers.

Roodin, P., L.H. Brown, and D. Shedlock. 2013. Intergenerational Service-learning: A Review of Recent Literature and Directions for the Future. Gerontology & Geriatrics Education 34: 3–25.

Schellner, J. 2008. Environmental Service Learning: Outcomes of Innovative Pedagogy in Baja California Sur, Mexico. Environmental Education Research 14: 291–307.

Shih-Tsen, L., and M. Kaplan. 2006. An Intergenerational Approach for Enriching Children’s Environmental Attitudes and Knowledge. Applied Environmental Education and Communication 5: 9–20.

Springate, I., M. Atkinson, and K. Martin. 2008. Intergenerational Practice: A Review of the Literature (LGA Research Report F/SR262). Slough, U.K.: NFER. http://www.nfer.ac.uk/publications/LIG01/LIG01.pdf (accessed May 20, 2015).

Stocklmayer, S.M., L.J. Rennie, and J.K. Gilbert. 2010. The Roles of the Formal and Informal Sectors in the Provision of Effective Science Education. Studies in Science Education 46: 1–44.

Swab, R., L. Zhang, and W. Mitsch. 2008. Effect of Hydrologic Restoration and Lonicera Maackii Removal on Herbaceous Understory Vegetation in a Bottomland Hardwood Forest. Restoration Ecology 16: 453–463.

Tam, M. 2011. Active Ageing, Active Learning: Policy and Provision in Hong Kong. Studies in Continuing Education 33: 289–299.

———. 2014. Intergenerational Service Learning between the Old and Young: What, Why and How. Educational Gerontology 40 (6): 401–413.

Trickett, E.J. 1997. Developing an Ecological Mind-Set on School-Community Collaboration. In Applied Ecological Psychology for Schools within Communities, J.L. Schwartz and W.E. Martin, Jr., eds., 139–163. Mahwah, NJ: Lawrence Erlbaum.

US Census Bureau. 2010. http://www.census.gov/ (accessed May 20, 2015).

Vidra, R., T. Shear, and J. Stucky. 2007. Effects of Vegetation Removal on Native Understory Recovery in an Exotic-rich Urban Forest. Journal of the Torrey Botanical Society 134: 410–419.

Watling, J., C. Hickman, and J. Orrock. 2011. Invasive Shrub Alters Native Forest Amphibian Communities. Biological Conservation 144: 2597–2601.

Wellington, J. 1990. Formal and Informal Learning in Science: The Role of the Interactive Science Centers. Physics Education 25: 247–252.

Zucchero, R. A. 2009. Outcomes of a Comentoring Project: Inspiration and Admiration. Educational Gerontology 35: 63–76.

———. 2011. A Co-mentoring Project: An Intergenerational Service-Learning Experience. Educational Gerontology 37: 687–702.

Meeting the Challenge of Interdisciplinary Assessment

 

Introduction to Interdisciplinary Education

“As the pace of scientific discovery and innovation accelerates, there is an urgent cultural need to reflect thoughtfully about these epic changes and challenges. The challenges of the twenty-first century require new interdisciplinary collaboration, which place questions of meanings and values on the agenda. We need to put questions about the universe and the universal back at the heart of university.” William Grassie (2013)

As the world becomes more complex, given the rapid expansion of technology, the changing nature of warfare, rising energy, and environmental crises, the value of an interdisciplinary education is increasingly obvious. Social, political, economic, and scientific issues are so thoroughly interconnected that they cannot be explored productively, either by experts or students, within clear-cut disciplinary boundaries.

Despite this fact, several problems arise when institutions try to incorporate interdisciplinary education into their programs. Boix Mansilla (2005) noted that the assessment of interdisciplinary work by students is of great concern. She explains that because faculty are often discipline-specific experts, they are unfamiliar with disciplines outside their realm of expertise and have difficulty defining interdisciplinary work. She goes on to explain that, as a consequence, “the issue [of standards] is marred by controversies over the purposes, methods, and most importantly, the content of proposed assessments” (2005, 16).

This paper offers one solution to this dilemma. The following analysis explores the current state of interdisciplinary education, both in academia broadly, and specifically, at West Point through its interdisciplinary Core Program. The sections that follow will highlight the current issues inherent in interdisciplinary education, define interdisciplinary education objectives, and finally, explain the adaptable, multi-functional, interdisciplinary rubric being implemented at the United States Military Academy (USMA), a rubric designed to resolve many of the issues interdisciplinary educators encounter.

The Current State of Interdisciplinary Education in Academia

The demand is clear. Whether we try to take a stance on the stem cell research controversy, to interpret a work of art in a new medium, or to assess the reconstruction of Iraq, a deep understanding of contemporary life requires knowledge and thinking skills that transcend the traditional disciplines. Such understanding demands that we draw on multiple sources of expertise to capture multi-dimensional phenomena, to produce complex explanations, or to solve intricate problems. The educational corollary of this condition is that preparing young adults to be full participants in contemporary society demands that we foster their capacity to draw on multiple sources of knowledge to build deep understanding.” Veronica Boix Mansilla (2005, 14)

There are currently several studies, including evaluation measures, defining the essence of interdisciplinary education. The above quote from Boix Mansilla’s “Assessing Student Work at Disciplinary Crossroads” highlights the challenge educators are experiencing in preparing students to meet today’s most pressing problems. This paper will not attempt to address the structure of interdisciplinary education as an institutional convention, but only to define the essential skills and capacities that a student with interdisciplinary understanding would demonstrate. These definitions are essential to understanding and creating a framework for interdisciplinary learning, which is arguably the first step in adequately integrating it into educational programs. Interdisciplinarity is a difficult construct to quantify, and many educators have been unable to frame a definition of it or to assess it in student work. As a consequence of these and other challenges, only a limited number of colleges or universities have implemented formal interdisciplinary programs at the institutional level.

Several analyses (Boix Mansilla 2005; Boix Mansilla and Dawes Duraising 2007; Rhoten et al. 2008; Stowe and Eder 2002) address the key issues surrounding interdisciplinary learning in higher education and offer proposals on how to address them, starting with the definition of the term “interdisciplinary.” One definition of interdisciplinary understanding is “the capacity to integrate knowledge and modes of thinking drawn from two or more disciplines to produce a cognitive advancement—for example, explaining a phenomenon, solving a problem, creating a product, or raising a new question—in ways that would have been unlikely through single disciplinary means” (Boix Mansilla 2005, 16; Boix Mansilla and Dawes Duraising 2007, 216). A definition is particularly important because “a clear articulation of what counts as quality interdisciplinary work, and how such quality might be measured, is needed if academic institutions are to foster in students deep understanding of complex problems and evaluate the impact of interdisciplinary education initiatives” (Boix Mansilla 2005, 16). An agreed-upon definition is currently lacking in academia, and this has resulted in inconsistent grading, teaching, and learning in interdisciplinary education.

One study of well regarded and established interdisciplinary programs in the U.S., which included Bioethics at the University of Pennsylvania, Interpretation Theory at Swarthmore College, Human Biology at Stanford University, and the NEXA Program at San Francisco State University, involved “69 interviews, 10 classroom observations, 40 samples of student work, and assorted program documentation” (Boix Mansilla and Dawes Duraising 2007, 4). The data were gathered in one-hour to 90-minute semi-structured interviews with faculty and students inquiring about the manner of assessment used in their respective programs. Next, examples of student work were used to give examples of what the institution viewed as meeting the definition of interdisciplinarity. From the interviews and student examples, the authors concluded that there are three core dimensions to student interdisciplinary work: disciplinary grounding, advancement through integration, and critical awareness (Boix Mansilla, 2005, Boix Mansilla and Dawes Duraising 2007). These core elements are represented graphically in Figure 1.

Figure 1. Three Interrelated Criteria for Assessing Students’ Interdisciplinary Work (Boix Mansilla and Dawes Duraising 2007, 223)

The first core element in Figure 1, disciplinary grounding, calls for strong base knowledge in individual disciplines. During the interviews, 75 percent of the interviewed faculty felt that strong subject-area knowledge was necessary for interdisciplinary education that did not sacrifice depth in exchange for breadth. However, the authors noted that the key to successful disciplinary grounding also included the thoughtful selection of which disciplines to use and how to use them. Advancement through integration, the second principle, is universal in all student work in the sense that students are supposed to learn from the work they do; however, what sets it apart in interdisciplinary education is that “students advance their understanding by moving to a new conceptual model, explanation, insight, or solution” (Boix Mansilla and Dawes Duraising 2007, 225). In the study, sixty-eight percent of faculty identified advancement through integration as a necessary element in interdisciplinary understanding and as the quintessential element for the advancement of student understanding. However, various programs and their students interpret this core element differently. For example, some students in in the NEXA Program at San Francisco State University strive for complex explanations, which evaluate the extent to which disciplines are interwoven to create a broad picture of how interconnected different disciplines are on a given topic. Other students in the same program prefer to use aesthetic reinterpretations to connect the literary, historical, and social elements of a given topic. Other students, such as those in the Bioethics program at the University of Pennsylvania, choose to focus on the development of practical solutions based on of the use of multi-disciplined ideas. The final principle from Figure 1, critical awareness, refers to student work being able to withstand examination and criticism and explicitly calls for evidence of student reflection in their work. Student work needs to “exhibit clarity of purpose and offer evidence of reflective self-critique” (Boix-Mansilla and Dawes Duraising 2007, 228).

Rhoten et al. (2008) also conducted a study focused on the similarities and differences between the learning outcomes of liberal arts and interdisciplinary programs. For this particular study, the researchers used student and faculty surveys, interviews, and tests to gather data for their analysis. The authors explain that most liberal arts programs “must develop student capacities to integrate or synthesize disciplinary knowledge and modes of thinking,” which is very similar to the type of synthesis that is expected from an interdisciplinary curriculum (Rhoten et al. 2008, 3–4). The main purpose behind this study was to identify the parallels between interdisciplinary and liberal arts programs, in order to show how a program can be made more interdisciplinary without changing its structure or content. Table 1 shows a summary of several parallels between a liberal arts education and an interdisciplinary education.

Table 1. Comparison of Liberal Arts Education and Interdisciplinary Education Objectives (Rhoten et al. 2008)

Rhoten et al. (2008) also analyzed empirical data to draw out trends on the “222 institutions considered ‘Baccalaureate College-Liberal Arts institutions’ under the 2000 Carnegie Classification system,” whether the interdisciplinary programs offered were majors, minors, optional courses, or required courses (Rhoten et al. 2008, 5). In general, “interdisciplinary programs are still ‘personally driven,’ whereas departments are ‘self-perpetuating'” (Rhoten et al. 2008, 6). “Personally driven” simply means that if students want to broaden their subject-area exposure they must do so on their own. “Self-perpetuating” refers to fact that departments within an institution need to act in their own self-interest in order to survive and thrive; therefore they tend to avoid interdisciplinary efforts. Interdisciplinary education does not support the mission of individual departments, and if students seek it, they must do so on their own initiative. One would therefore conclude that the only way to truly incorporate interdisciplinary education into schools is by making it institutionally mandated, at least for the core curriculum that all students are required to take.

Schools should strive to integrate interdisciplinary efforts into their institutions because “interdisciplinarity breeds innovation” (Rhoten et al. 2008, 12). Although such innovation carries tremendous benefits, the difficulty of measuring student and educator success was again identified as a barrier. Most schools that are already making efforts towards interdisciplinarity believe that they are somewhat successful according to Rhoten et al. (2008). However, in order to mark and measure success, and to continually improve interdisciplinary programs in schools, the authors propose a value-added assessment, which is intended to provide an “assessment regime that measures growth that has occurred as a result of participation in the institution or academic program” (Rhoten et al. 2008, 14). Moreover, some cross-cutting goals that are embedded especially in interdisciplinary studies, such as life-long learning, curiosity, creative thinking, synthesis, and integration, have acquired the reputation of being ineffable and, correspondingly, unassessable” (Rhoten et al. 2008, 83). This common problem was addressed by Stowe and Eder (2002) who identified several assessment measures that are placed on a continuum, as seen in Figure 2. These measures can also be used to better define interdisciplinary standards by providing a multi-tiered adjustable scale that can help to quantify the assessment of student work based on an instructor’s desired outcomes.

Figure 2. Perspectives on Assessment (Stowe and Eder 2002, 84)

Stowe and Eder (2002) state that using a rubric to define and measure interdisciplinary work would improve the “apparently subjective nature” of interdisciplinary assessment. They further recommend the rubric as a “visible standard—a scoring guide—that allows the assessor and the public, for that matter, to recognize expectations and make increasingly fine distinctions about the quantity and quality of student learning” (96). They expand on their recommendation by noting that assessment must be focused on both improving interdisciplinary learning and “improving student learning,” and should be “embedded within larger systems… and create linkages and enhance coherence within and across the curriculum” (80). Without cooperation across different programs, it is impossible to foster an interdisciplinary learning environment.

An example of such cooperation can be seen at USMA, where several academic departments have moved towards a cooperative environment focused on interdisciplinary learning (Elliott et al. 2013). This paper will focus on the education of the USMA Class of 2016 from their freshman year, when the plan to use energy conservation and the NetZero project (an energy initiative by the Dept. of the Army on several Army posts, including West Point, to produce as much energy as it consumes by the year 2020) was adopted to infuse interdisciplinary themes into their core courses. The five Student Learning Outcomes from this effort include four individual discipline-focused outcomes as well as a fifth, which aims to “develop an interdisciplinary perspective that supports knowledge transfer across disciplinary boundaries and supports innovative solutions to complex energy problems/projects” (Elliott et al. 2013, 33). In a larger sense, this objective illustrates that interdisciplinary education addresses the mission of USMA and the Army’s focus on the “development of adaptive leaders who are comfortable operating in ambiguity and complexity will increasingly be our competitive advantage against future threats to our Nation,” as outlined by General Martin Dempsey, Chairman of the U.S. Joint Chiefs of Staff (Elliiott et al. 2013, 30).

Framing the Problem

The Academy produces graduates who can think dynamically in the ever-changing world described in the quotes from Grassie and Boix Mansilla at the beginning of this article. At West Point, this is accomplished by taking not only a multi-disciplinary approach to education, but also an interdisciplinary one. The Academy’s Core Curriculum describes the required classes that all cadets must complete or validate. The Core Curriculum does not include any classes required for a cadet’s major. Other non-academic requirements include three tactics courses and seven physical education courses. The interdisciplinary aspect is a new addition to the curriculum. In recent years, several committees have recommended promoting interdisciplinary approaches to better meet both the Academy’s and the Army’s goals as outlined in Elliott et al. (2013).

To achieve these goals, several academic departments involved in the Core Curriculum developed an interdisciplinary program for the entering plebe class, the Class of 2016. During the first week of classes, freshmen wrote an essay in their Introduction to Mathematical Modeling course, or MA103, about how they would use concepts from different courses to tackle the challenges that NetZero and the alarming problem of energy consumption in the Army pose to West Point. After 30 instruction sessions (approximately thirteen weeks), the freshmen revised these essays in their Composition course EN101. This time they used the knowledge acquired throughout the semester in the English course and in the other courses they were taking. Faculty from the Department of Mathematical Sciences and the Department of English and Philosophy evaluated these revised essays from different perspectives to emphasize the importance and relevance of multiple disciplines. This led to the realization that it was impossible to adequately compare the essays, since the assignments, rubrics, and faculty were not consistent and there was no common rubric to standardize the grading approach. To mitigate this challenge, the essays were compared in our study using the Flesch-Kincaid test (a formula designed to evaluate the difficulty and complexity of technical writing. It consists of two readings: grade level and reading ease) and a comparison of the final grades for the various essays. Scores for a sample of three essays for 25 students, a total of 75 essays, were used to compare improvement in a measureable, quantitative manner. The test consisted of a null hypothesis that there was no significant difference among the ratings, indicating neither improvement nor deterioration of scores from the different assignments throughout the semester, and an alternative hypothesis that there actually was a difference between scores. A two-tailed t-test yielded p-values ranging between 0.3 and 0.6. This indicated that the Flesch-Kincaid results were inconclusive, meaning that neither the null nor the alternative hypothesis could be rejected.

Despite the inconclusive results of the Flesch-Kincaid test, there was a demonstrated improvement in student work, albeit an improvement that was perceived on the basis of a subjective analysis of the essays. Therefore, a new rubric was developed to re-grade all of the essays in a standardized fashion against the desired elements for that particular set of assignments. To facilitate a comparison, this new and straightforward rubric aimed at grading each assignment from the different departments on the same scale. The grades were on a 1–10 scale, and the rubric can be seen in Table 2. The essays were then re-graded according to the same rubric and the results were compared again using a two-tailed t-test.

The challenge in evaluating interdisciplinary work is that the term “interdisciplinary” is not well-defined or broadly understood. This became even clearer after the Chemistry faculty conducted an interdisciplinary group capstone in the General Chemistry course with the Class of 2016 during the second semester of their freshman year. The capstone presented the students a complex and challenging energy problem that was both current and militarily relevant to their future roles as Army officers. This project required groups of students to write a memorandum summarizing their findings on an experimental, portable, and green battery recharger for soldiers in the field, and then to provide a presentation of their results to their commander. Cadets conducted an experiment on the battery recharger to test its efficiency, to compare it to current recharging methods, and to address the social and leadership challenges that would occur when this new equipment was integrated into a unit. In addition, the capstone leveraged the students’ various courses and experiences to scaffold understanding of key concepts and technology necessary to engage the problem. The freshman cadets were expected to utilize what they learned from math modeling, information technology, general psychology, and general chemistry courses in formulating their solution.

The rubric used to grade these capstones was developed by the Chemistry faculty with input from all the participating courses, and then later utilized by the Chemistry faculty in assessing the capstones. The collaborative rubric identified numerous concepts in each course, and as a result, it was several pages long. Perhaps most significantly, it did not define the term “interdisciplinary” for the faculty and the students in the capstone, nor did it make clear the associated expectations. At the conclusion of the rubric, faculty were asked to rate on a 1–10 scale how interdisciplinary their students’ submissions were. The results, displayed in Figure 3, had a standard deviation of .186 and were inconsistent in both the average instructor rating and the range of different ratings faculty assigned. This indicated that the faculty did not share the same understanding of “interdisciplinary” in assessing student work.

Table 2. Rubric used to evaluate the population sample of NetZero essays from fall 2012

Boix Mansilla and Dawes Duraising (2007) state that student interdisciplinary work should “be well-grounded in the disciplines” “show critical awareness,” and “advance student understanding” (223). These criteria both define the basic learning objectives of an interdisciplinary education and address the need for baseline knowledge in the subjects being addressed in student work. While these criteria may not be included in a rubric or other grading mechanism, they provide more of a defined objective regarding interdisciplinary student work.

Although the idea of graduating interdisciplinary-minded students is appealing to many programs, the challenge of measuring the success of interdisciplinary curriculums in producing these “multi-disciplined” graduates has yet to be addressed. The problem of scaling and measuring interdisciplinary education is itself interdisciplinary in nature and, consequently, an abstract idea for many (Boix-Mansilla and Dawes Duraising 2007, 218). Interdisciplinary education evaluation currently lacks a “sound framework” for assessment since the effects of interdisciplinary efforts on student learning are neither well-defined nor proven (Boix Mansilla 2005, 18). As seen in Figure 2 (Stowe and Eder 2002), the assessment of interdisciplinary work is a non-static scale where the balance between the perspectives and entities is never quite the same from project to project, or from class to class. Stowe and Eder (2002) offer a flexible scale for assessment that allows each interdisciplinary quality to be judged according to faculty expectations: how discovery-oriented versus objective-orientated do they want student assignments to be? Rhoten et al. (2008) do correlate several common learning outcomes of a liberal arts education with their interdisciplinary counterparts as seen in Table 1. Although useful for demonstrating extensive possible outcomes and correlations, the linkages are broadly defined and do not specify objectives; this exemplifies the issues of scale, definition, and the non-quantified nature of interdisciplinary education that currently prevail in academia.

Figure 3. Chemistry instructor evaluation of interdisciplinary synergy in capstone projects during Spring 2013. Courtesy of the United States Military Academy Department of Chemistry and Life Sciences.

All of the aforementioned problems can be traced to a lack of clarity on standards (Boix-Mansilla 2005, 16). Stowe (2002) explicitly calls for a standard for grading, collecting data, and creating a shared understanding, which he suggests could be found in a rubric. A standardized rubric, which is adaptable to several mediums and is general enough to be applicable to several disciplines, is desperately needed for evaluating and assessing interdisciplinary work. Such a rubric needs to clearly define the necessary elements of an interdisciplinary product and be sufficiently adaptable to align with project requirements; this would resolve several of the problems we have identified. In addition, Stowe and Eder (2002) call for the inclusion of very specific elements in a rubric, so that it can address current problems and properly evaluate interdisciplinary work. Among these requirements are assessing complex intellectual processes, promoting objectivity, reliability, and validity in assessment, clearly defining learning objectives for students, and being flexible and adjustable for course or curriculum progression (96). Although we conducted a thorough search, we failed to find a rubric that adequately fulfills this need.

Interdisciplinary Rubric Development

The goal of the rubric developed at USMA is to create a grading mechanism that can be used in multiple project mediums across multiple disciplines. Simultaneously this rubric maintains the integrity of the interdisciplinary goals by creating a more defined standard with which to grade interdisciplinary student work. The rubric also contains open areas for point allotment as well as weighting for each category, which allows faculty to allot points and focus where they see fit. Developing such a rubric required several steps: defining the term interdisciplinary, identifying the elements that student work needs to demonstrate in order to illustrate interdisciplinary thinking, creating a model that visually represents the interconnectivity of these elements, and then using the defined elements and model to arrive at the rubric categories.

The first step in the rubric development process was to define the term interdisciplinary:

Interdisciplinary: The seamless integration of multi-dimensional, multi-faceted ideas into a clearly demonstrated understanding of an issue’s breadth and depth, with sound judgment and dynamic thinking.

Boix Mansilla’s definition of interdisciplinary understanding provided the starting point for the development of the rubric. Additionally, material from the research discussed above identified missing elements from Boix Mansilla’s definition. For example, the best students’ interdisciplinary work included ideas from multiple disciplines that were integrated to demonstrate the level of understanding that the student has attained.

The second step in the rubric development process was to expand the definition of interdisciplinary, in order to create a shared understanding between students, faculty, and those evaluating the interdisciplinary work. To this end, the feedback and lessons learned from previous student work were used to identify the elements common to successful interdisciplinary work. These principles include: discipline specific knowledge, multi-perspective understanding, integration, practical integrated solutions, reflection, and clarity of purpose. To illustrate the interconnectivity of these principles, a conceptual model of the characteristics was created. Initially, the intention was to create a linear model to represent the core principles. However, several issues, such as missing connections and limited complexity, led to the immediate conclusion that a linear model could not completely describe complex nonlinear problem solving. The resulting model, which illustrates a cyclical thinking process, is shown in Figure 4.

Figure 4. The Cyclican Model of the Key Interdisciplinary Characteristics. This model demonstrates the interconnectivity of the d defined interdisciplinary elements.

The model begins with the framing and scoping of the problem before the application of discipline-specific knowledge, which as we have seen is an essential starting point for interdisciplinary work. The core principle of the integration of ideas was partitioned into multi-perspective understanding, integration, and practical integrated solutions. Multi-perspective understanding and discipline-specific knowledge are connected by an addition sign, which symbolizes understanding a topic from multiple perspectives. This illustrates that students must be able to use discipline-specific knowledge to make this essential connection. The arrow labeled “integration” in the lower part of the model represents the synthesis of discipline-specific knowledge and multi-perspective understanding into practical integrated solutions. Practical integrated solutions are then connected to reflection via a multiplication sign to show that reflection has a multiplicative effect on interdisciplinary understanding. The arrow labeled “clarity of purpose” represents the cyclical process and shows the compilation of all the previous elements back into discipline-specific knowledge. The knowledge gained from the various parts of the cycle can be used in the further learning of other applicable disciplines. This model’s goal is not to explain the rubric, but to illustrate how interdisciplinary education is cyclical in nature, how the characteristics of interdisciplinary understanding are relevant to interdisciplinary education, and how student learning should continue to build.

Next, the core principles of what makes student work interdisciplinary were established, defined, and examined. The elements in Figure 1 above, taken from Boix Mansilla and Dawes Duraising (2007), were used as a starting point for the development of this rubric’s core principles: be well grounded in the disciplines, show critical awareness, and advance student leaning through understanding (223). For the purpose of this rubric, some elements were modified and expanded to create six core principles. A list of the six core principles that were incorporated into the rubric, along with their definitions, appear in Table 3.

Problem framing and scope are derived from the idea that interdisciplinary work should show critical awareness. The definition used in the rubric is very flexible, so that educators can adapt it for different project mediums and faculty, departments, and/or university requirements. Critical awareness, as defined by Boix Mansilla (2007), includes the definition of purpose as well as the integration of ideas. The definition used for problem framing and scope in the rubric requires that the student’s work have a clearly defined purpose. This was created as a separate category because we had observed a clear trend of misunderstanding among faculty regarding the level of complexity that they expected. This is an important aspect of student interdisciplinary understanding; it allows the faculty to scale assignments according to the expected level of student understanding and allows the student to recognize just how complex and multi-disciplined a product the instructor is seeking. For example, if students were assigned a project on how to effectively stock a warehouse, an instructor would not have the same expectations of a freshman who has taken only introductory courses in mathematical modeling and economics as of a senior who had taken nonlinear optimization, supply chain management, and microeconomics courses. Having this requirement in the rubric makes clear the expectation that students will properly identify what they want to address, and also allows the instructor to have a frame of reference in a project.

The rubric’s second core principle, discipline knowledge is well grounded in the disciplines and is intentionally more open-ended, so that it can be readily adapted to different departments, projects, and situations (Boix Mansilla 2007). Identifying theories, examples, findings, methods, etc. may not be relevant or necessary in a given problem. Therefore, although the evaluator is given an area in the rubric that calls for disciplinary knowledge, the rubric does not explicitly indicate how that knowledge is to be graded. For example, in our warehouse stocking project, a freshman might be expected to mathematically model the effects of changing employee wages on productivity. A university senior, on the other hand, might be expected to produce a business recommendation to stakeholders by addressing the intricacies of supply chain management on warehouse profits as well as its psychological implications for employees. The discipline knowledge area of the rubric enables the evaluator to determine how much knowledge and understanding students are expected to demonstrate, while ensuring that the importance of disciplinary understanding is not lost on an interdisciplinary project.

The integration of ideas principle is really the quintessential element for the interdisciplinarity of this rubric. All six core principles are important interdisciplinary factors, but if this element were removed, the rubric could be used for a project that is not interdisciplinary. Integration of ideas derives its meaning from the critical awareness and advanced student understanding pieces identified above in Figure 1. This rubric defines integration of ideas as multi-dimensional, feasible, practical solutions with multi-faceted and seamlessly connected ideas. It is important to note the difference between being integrated and being seamlessly integrated. The seamless integration of ideas, which can take on different meanings depending on the assignment, is an indicator of true multi-dimensional, multi-faceted understanding. Seamless integration. We define the term seamlessly integrated to mean that ideas are not simply laundry-listed, but instead are connected in an intelligent and logical fashion. The definitional elements of multi-dimensional and multi-faceted identify the need for complexity in student work. It is multi-dimensional when students make use of multiple dimensions of their education or, in other words, use multiple disciplines, in their work. Multi-faceted means that students are able to use evidence and knowledge to back up their multi-dimensional claims. The most important component is that students be able to demonstrate a clear understanding of what they are presenting. This also relates to a student’s ability to demonstrate the span of an issue’s breadth and depth. In other words, students should be able to apply disciplines to an issue or topic with an appropriate understanding of the level of each of the disciplines. The use of extraneous disciplines merely for the sake of incorporating more disciplines does not necessarily make student work interdisciplinary. In fact, it contradicts the idea of advancing the complexity of the student’s thought process.. Students who apply the appropriate level of discipline breadth and depth indicate their ability to use sound judgment or logic, as well as their ability to think dynamically.

The next two core principles, clarity of purpose and reflection, were added to address the students’ failure to internalize what they were learning and understanding; this was revealed during the analysis of the USMA interdisciplinary program. The main challenge was that students did not fully grasp why a given project was interdisciplinary, or why that was important. To alleviate this, the core principle clarity of purpose was added to the rubric to help students understand the “why”; the intent was to motivate them to define the purpose of their investigation and to take an in-depth approach to the problem. This is different from problem framing and scope in a very important way: problem framing and scope focuses on a well-defined thesis statement or purpose statement, whereas clarity of purpose focuses on the content of student work. In other words, problem framing and scope ask whether students have a clearly stated framework for their project, while clarity of purpose asks whether they demonstrate their personal interdisciplinary understanding and then explain it well to their audience. Similarly, the next principle, reflection, calls for a clear and delineated connection of ideas and an indication that students have reflected on the interconnectivity and importance of their areas of study. These two core principles are drivers of internalization and cognitive advancement in interdisciplinary learning. They are particularly important because often students do not reflect on what they have learned. The reflection piece is intended to facilitate a deeper understanding of what they are learning and to encourage students to consider how the material fits into the greater scheme of their education.

The final element of the rubric shown in Table 3 is the presentation principle. This principle calls for information that is presented in a suitable medium with proper tone, word choice, spelling, grammar, etc. In short, did the students address the audience correctly and present their knowledge intelligently while doing so? This section can be adapted to the type of project and course for which the rubric is being used. For example, English faculty would probably expand this section because of its importance to their learning outcomes, whereas chemistry faculty may place more emphasis on the discipline-knowledge portion.

The newly developed rubric was presented to the Math course leaders for use on the freshman’s “mini” capstone exercise in December 2013. The rubric was sent to the faculty with minimal guidance. The feedback from the course director made it clear that the students and faculty did not fully grasp the intention or expectations behind the rubric. A few factors contributed to this: sixty-six percent of the faculty were new to the department; the interdisciplinary expectations were not fully explained to the faculty; although everyone received the rubric, each instructor created his or her own rubric for the mini-capstone; and the students who took the mini-capstone and the faculty who graded their work were under significant time pressure. The mini-capstone in its creation, execution, and grading was not given adequate time due to end of semester requirements at USMA during the November-December time period. An important conclusion from this feedback was that the faculty needed to have a common understanding of what is expected on an interdisciplinary project. To achieve this for the General Chemistry capstone project in the spring of 2014, a grading calibration exercise was conducted. This calibration included good and poor examples of interdisciplinary work from the previous year’s chemistry capstone, and showed faculty how to distinguish between good and poor work and how to use the rubric in assigning a grade.

Implementing the Interdisciplinary Rubric

The first step in implementing the rubric was calibration with the faculty. With such an exercise, the faculty should take away a common understanding of what exactly interdisciplinarity is as well as the knowledge of what constitutes a good final project. The plan for the calibration exercise developed for USMA faculty who would be grading the CH102 General Chemistry capstone in the spring of 2014 was an hour-long presentation and discussion. Prior to the presentation, faculty received a packet of examples of cadet work in each of the major portions of the previous year’s capstone project. The examples included “A” work as well as examples of common integration errors students make: the “laundry list,” the “tacked on at the end,” and the “no real knowledge” integration errors. The “laundry list” is an example of how a student may mention and be knowledgeable in multiple disciplines but does not integrate them, providing instead a “laundry list” of the different disciplines and explaining the relevance of each individually. The “tacked on at the end” error (or whatever we may call it) exemplifies how a student may go in-depth in one discipline, particularly in the discipline for which the assignment was given, then tack on a sentence or two at the end mentioning other disciplines in order to call the project interdisciplinary. The “no real knowledge” example presents a plethora of ideas but does not demonstrate that the student learned or integrated disciplines and/or ideas. With these examples, faculty became more familiar with what correct and incorrect work looked like. The “A” level example was not meant to illustrate the perfect or only solution; it was merely one example. Faculty evaluated each example using the standard A, B, C, D, F grading scale based on how interdisciplinary they felt each project was.

At the start of the presentation portion of the rubric calibration, faculty were introduced to the interdisciplinary characteristics and model from Figure 4. This ensured understanding of interdisciplinary characteristics prior to the introduction to the rubric itself. After the characteristics were covered, the results from the exercise, which the faculty just had completed, were discussed. This clarified any misunderstandings that faculty had about the interdisciplinary characteristics, while the examples of chemistry capstones from the previous year provided a frame of reference. Next the rubric was thoroughly explained, showing how it was scalable, expandable, and concise to meet instructor needs for interdisciplinary student projects.

The General Chemistry capstone rubric for 2014 differs from its 2013 predecessor in two very important ways. First, it is significantly shorter; its two pages (compared to seven pages) emphasize quality over quantity. Instead of listing every detail of the project, the new capstone rubric has five categories that address the math modeling, leadership, information security, oral communication, and the required submission components of the project, all without specific details. This allows the students to be more creative in their answers to the given problem.

The 2013 rubric was not based on any interdisciplinary principles or examples. Instead, it listed specific requirements from the disciplines the students were supposed to integrate. The result was quite the opposite: the 2013 capstone projects tended to be disjointed because of the slew of specific requirements. This year’s capstone rubric incorporates the interdisciplinary principles described in Table 3.Problem framing and scope is addressed in the Project Summary section with the requirement for a bottom line up front (BLUF), or thesis. Discipline knowledge is asked for in the Discrete Dynamic Modeling, Persuasion and Conformity in a Leadership Environment, and Information Security sections. Although the course-specific requirements must be addressed, Integration of ideas is assessed in the Oral Communication and Project Summary sections, which requires that fluid transitions and logically ordered and related ideas be integrated. Appropriate presentation is also adequately addressed in these sections, as the rubric lays out clear expectations of the written and oral presentations for students, including their tone, body language, and level of professionalism. Clarity of purposeand reflectionare asked for in the Project Summary section, which calls for contingency plans and thoroughly explained analysis of the total problem.

Initial instructor feedback on the use of this rubric is that it better defined expectations for the students’ interdisciplinary work, for both the instructor and the students. After using the rubric in the calibration exercise, instructors stated that they felt more confident and prepared than they had in 2013 when there was no such exercise and assessment tool available; this year they understood what was asked of them and of the students. Initial comparisons of the interdisciplinary assessments of the students’ work from 2013 and 2014 are quite positive. On a scale of 0–10, the average interdisciplinary score given by instructors was 5.69 in 2013, with zero being the least interdisciplinary and 10 the most. (See Figure 3 for these data.) In 2014 this improved to 7.79 (actually 15.5/20). There was also less variability between instructors. For example, in 2013 the standard deviation of the mean scores assigned by each of the instructors was 1.86 (Figure 3). In 2014, the standard deviation between the instructors’ mean scores was .98 (1.96/20), a decrease of over 47%.

Future Work and Conclusion

Now that the General Chemistry capstone for USMA Class of 2017 has concluded, several analyses must be completed to evaluate the progress of interdisciplinary education at USMA. At a minimum, an analysis of the grades and feedback from the students and faculty needs to be conducted. The analysis of the grades should include a distribution of grades compared with their expected distribution, as well as a quantitative and a qualitative analysis of the capstones compared to the previous years’ capstones. This could be done using the methods previously employed, including the use of Flesch-Kincaid, paired t-test, the distribution of the faculty’s interdisciplinary rating similar to Figure 3, and/or a cross-course sample of projects re-graded by the course director.

The discussion and research that have taken place at West Point since the first General Chemistry capstone project in 2013 indicate that the results of this year’s changes should be positive. Although there is as yet no statistical evidence to demonstrate improvement, the general understanding of how interdisciplinarity looks, how to produce it, and how to assess it is much more expansive now than in 2013. The reason for this might be that faculty and students at USMA are now experienced with interdisciplinary work and have a clearer understanding of interdisciplinary assessment and its importance over the course of a year.

The world is a complex and rapidly changing place that requires its future scientists, scholars, engineers, teachers, and leaders to think dynamically and across disciplines.. Interdisciplinary assessment is necessary for the future of education, particularly at West Point where we recognize that “adaptive leaders who are comfortable operating in ambiguity and complexity will increasingly be our competitive advantage against future threats to our Nation” (Elliott et al. 2013, 30). Only time will tell whether this interdisciplinary rubric has met its goal of creating a grading mechanism that can be used in multiple project mediums across multiple disciplines. Given the extensive research and analysis done at West Point to create this much- needed and useful tool, the prospects for future interdisciplinary education are promising.

About the Authors

Elizabeth Olcese

Elizabeth Olcese graduated with a Bachelor of Science degree in Operations Research from the United States Military Academy at West Point, NY in 2014. She served as a student researcher for West Point’s Core Interdisciplinary Team focused on enhancing opportunities for interdisciplinary learning in West Point’s core academic curriculum. Upon completion of the Quartermaster Basic Officer Leader Course at the Army Logistics School in Fort Lee, VA, she will serve as a second lieutenant for the 25th Infantry Division at Schofield, Hawaii.

Joseph C. Shannon

Joseph C. Shannon graduated with a doctorate in Curriculum and Instruction with a focus in Science Education from the College of Education at the University of Washington, WA. He is a former member of West Point’s Core Interdisciplinary Team that was focused on enhancing opportunities for interdisciplinary learning in West Point’s core academic curriculum. He is a former Academy Professor at the United States Military Academy and Program Director for the General Chemistry Program in the Department of Chemistry and Life Science. He is currently the Dean of Academic Programs at South Seattle College in West Seattle, Washington.

Gerald Kobylski

Gerald Kobylski graduated with a doctorate in interdisciplinary studies (Systems Engineering and Mathematics) from Stevens Institute of Technology, NJ. He currently is co-leading a thrust to infuse interdisciplinary education into West Point’s core academic curriculum. He is also deeply involved with pedagogy, faculty development, and assessment. Jerry is an Academy Professor at the United States Military Academy, a Professor of Mathematical Sciences, and a Commissioner for Middle States on Higher Education.

Lieutenant Colonel Charles (Chip) Elliott

Lieutenant Colonel Charles (Chip) Elliott graduated with a doctorate in Geography and Environmental Engineering from Johns Hopkins University in Baltimore, MD and is a registered professional engineer in Virginia. He is currently the General Chemistry Program Director and the Plebe (Freshman) Director for the Core Interdisciplinary Team at the United States Military Academy. He has previously taught CH101/102 General Chemistry, EV394 Hydrogeology, EV488 Solid & Hazardous Waste Treatment and Remediation, EV401 Physical & Chemical Treatment, and EV203 Physical Geography. He is currently an Assistant Professor in the Department of Chemistry and Life Science.

References

Boix Mansilla, V. Interdisciplinary Understanding: What Counts as Quality Work?” Interdisciplinary Studies Project, Harvard Graduate School of Education.

Boix Mansilla, V. 2005. “Assessing Student Work at Disciplinary Crossroads.” Change 37 (1): 14–21.

Boix Mansilla, V., and E. Dawes Duraising. 2007. “Targeted Assessment of Students’ Interdisciplinary Work: An Empirically Grounded Framework Proposed. The Journal of Higher Education 78 (2): 216–237.

Elliott, C., G. Kobylski, P. Molin, C.D. Morrow, D.M. Ryan, S.K. Schwartz, J.C. Shannon, and C. Weld. 2013. “Putting the Backbone into Interdisciplinary Learning: An Initial Report.” Manuscript submitted for publication, United States Military Academy, West Point, NY.

Grassie, W. (n.d.). “Interdisciplinary Quotes.” Thinkexist.com. http://thinkexist.com/quotation/as-the-pace-of-scientific-discovery-and/1457907.html (accessed November 16, 2013).

Ivanitskaya, L., D. Clark, G. Montgomery, and R. Primeau. 2002. “Interdisciplinary Learning: Process and Outcomes.” Innovative Higher Education, 27 (2): 95–111.

Newell, W.H. 2006. “Interdisciplinary Integration by Undergraduates.” Issues in Integrative Studies 24: 89–111.

Repko, A.F. 2007. “Interdisciplinary Curriculum Design.” Academic Exchange Quarterly 11 (1): 130–137.

Repko, A.F. 2008. “Assessing Interdisciplinary Learning Outcomes. Academic Exchange Quarterly 12 (3): 171–178.

Rhoten, D., V. Boix Mansilla, M. Chun, and J. Thompson Klein. 2008. “Interdisciplinary Education at Liberal Arts Institutions.” Teagle Foundation White Paper.

Stowe, D.E., and D.J. Eder. 2002. “Interdisciplinary Program Assessment.” Issues in Integrative Studies 20: 77–101.

Appendix 1

Download (PDF, 191KB)

 

Download (PDF, 998KB)