We are pleased to announce the Summer 2018 issue of Science Education and Civic Engagement: An International Journal.
Highlighting the value of international service, Courtney Cox, Sarah Lenahan, Patricia Devine, and Panagiotis Linos (Butler College) describe collaboration among the College of Pharmacy and Health Science, the College of Liberal Arts and Science, and Barnabas Task, a non-profit organization. Students have the opportunity to travel to the Dominican Republic to participate in service activities with medical and dental professionals. They work with community leaders to convey public health information on topics such as nutrition, exercise, smoking cessation, and mosquito-borne illnesses, so that the knowledge can be disseminated throughout the community using local networks. This experience enables students to develop their cultural awareness and illustrates the importance of local knowledge and collaboration in promoting social change.
Susan Huss-Lederman, Prajukti Bhattacharyya, and Brianna Deering (University of Wisconsin-Whitewater) describe their participation in the Do Now U Project, a collaboration between the National Center for Science and Civic Engagement and KQED Public Media. The project paired two courses, Environmental Geology and College Writing in English as a Second Language, and required students to write blog posts on environmental topics. After all the posts had been read and analyzed, one was chosen for publication on the web. This project provides students with valuable opportunities to research open-ended questions with important social impact while learning to collaborate and to communicate effectively.
Ellen Mappen (National Center for Science and Civic Engagement) provides an interesting case history of the beginnings of the SENCER-ISE project, which is a structured collaboration between SENCER and practitioners of informal science education (ISE) based on issues of civic engagement. This account describes the mutually beneficial synergies between formal and informal education and includes evaluation results that demonstrate the effectiveness of project partnerships.
The issue concludes with an insightful review by Katayoun Chamany (Eugene Lang College, New School) of a report from The National Academies entitled “Integration of the Humanities and Arts with Sciences, Engineering, and Medicine: Branches from the Same Tree.” The review situates this new report in a historical context and examines how the integration of disciplinary perspectives from the arts and humanities can enhance science education and motivate students to persist in their scientific studies. We wish to thank all the authors for sharing their accomplishments with the readers of this journal.
– Matt Fisher and Trace Jordan Co-Editors-in-Chief
On May 7, 2018, The National Academies of Science, Engineering, and Medicine (NASEM) released a report, Integration of the Humanities and Arts with Sciences, Engineering, and Medicine: Branches from the Same Tree,which champions the integration of arts and humanities with STEMM (STEM + Medicine).An ad hoc committee, comprising 22 experts spanning education, industry, and policy, met over three years gathering best practices and hosting workshops and open meetings. The committee developed a consensus report and a compendium of more than 200 examples, some of which are SENCER-related projects. Kristin Boudreau, Professor and Department Head of the Humanities and Arts at Worcester Polytechnic Institute, is at the helm of SENCER’s New England Center of Innovation and was a member of the committee charged with developing the consensus report.
The timing of this project and the publication of the report are of import. The project was launched on December 2, 2015, when Obama was in office and a strong focus on STEM education in community colleges was established as a priority. The December workshop, funded by the Andrew Mellon Foundation and hosted by the National Academies of Science Board on Higher Education and Workforce (BHEW), was attended by 110 artists, engineers, educators, policy makers, and industry experts. The ensuing project garnered additional funding from the National Endowment for the Arts (NEA) and the National Endowment for the Humanities (NEH).
Despite cutbacks under the new administration, the project endured and included an investigation of a wealth of resources, models, and institutional examples of organizational and pedagogical change to determine how integrated learning can serve all students. Perhaps, now more than ever, given the growing chasms in our society, integrated learning is essential if we are to provide our students with the tools to address social change, and the findings of this report are useful. During the question and answer period of the meeting that launched thisNASEM report, James Grossman, the Executive Director of American Historical Society, commented that “thinking about teaching in and beyond a discipline has to become as important as thinking about research in and beyond a discipline.” He argues that the challenge of promoting interdisciplinary teaching may require educators and students to reconsider how they identify; that we need to rethink about ourselves (NASEM, 1:12 min time stamp).
The project was spearheaded by the BHEW and other divisions and units within the NASEM, with the specific goal of providing an evidence base for the integration of humanities and arts and STEMM to inform “new projects aimed at improving the understanding and application of STEMM toward the social, economic and cultural well-being of the nation and planet.”The committee analyzed evidence to determine how STEMM experiences enhance the knowledge base of students studying the arts and humanities, so that they make sound decisions across all professional fields and contribute to a vibrant democracy. Likewise, the committee also analyzed evidence regarding the value of including arts and humanities perspectives in STEMM academic programs to produce more effective communicators, problem solvers, and leaders, who recognize the broad social and cultural impacts of STEMM. In both instances, the hypothesis being tested was that student populations could expand their skills of critical thinking, creativity, and innovation using these complementary perspectives and different ways of knowing to develop meaningful lives and careers (see Chapter 6 for examples).
One example in particular stood out because of its effect on retention of the diverse student population served by the City University of New York (CUNY) community colleges.The Guttman Community College’s two-semester City Seminar, fulfills the general education requirements of quantitative reasoning, critical thinking, writing, and reading and has a 27% completion rate as opposed to the 4.1% completion rate of other CUNY community colleges. They credit this success to their interdisciplinary approach, which meets all the general education requirements in one course, rather than distributing them among many.
A closer look at the charge of the NASEM committee suggests that on a national level we are finally beginning to address the criticisms of social science and humanities scholars regarding the 1945 report titled Science: The Endless Frontier. This report championed the unfettered advancement of STEM with no attention given to the valuable insights provided by humanities and social science perspectives. Vannevar Bush, Director of the Office of Scientific Research and Development, authored this six-chapter report as a response to President Franklin D. Roosevelt’s request to expand the goals and benefits of science beyond its wartime focus on the military. Additionally, the report argued that science learning should be more accessible and that scientific research should be more transparent to the American public. The report led to the establishment of the National Science Foundation, with the goal of ensuring national security, economic progress, and cultural growth, akin to the current charge by BHEW.
Some of the criticisms of the Science: The Endless Frontier report are contained in a collection of papers published by scholars in the humanities and social sciences on the 50th anniversary of its publication.Highlights appear in Science the Endless Frontier: Learning from the Past, Designing for the Future, which presents papers from a conference series held between 1994 and 1996 and includes responses and updates to the Bush Report, arguing that a lack of integrated knowledge would mean the demise of a STEM-centric approach to learning. Similarly, in “Is it possible to just teach biology?” (Horton & Freire, 1990), educational philosopher Paulo Freire and founder of the Highlander School Myles Horton also argue that to teach STEM without social context is a mistake. At the NASEM meeting to launch the Branches report, some committee members remarked how these sentiments led to Leadership in Science and Humanities opportunities funded by the Fund for the Improvement of Postsecondary Education (FIPSE) and the NEA in the 1990s, which were not sustained but must now be renewed.
The NASEM report recognizes those early criticisms and acknowledges that change is underfoot.The evolution of their charge is apparent with its emphasis on looking at integration as a two-way phenomenon that will improve the cultural well being of not only the nation, but also the planet. Over the last thirty years, curricular resources for integrated learning have moved beyond the social sciences to include the necessary perspectives that are central to the arts and humanities. The STEAM (STEM +Arts) and STEAMD (STEM+Arts+Design) movements take steps in that direction, with concrete collaborations and multi-institutional efforts underway. Examples include the Vertical Integrated Projects Initiative (VIP), with a strong focus on research, innovation, and design; Creativity Connects, funded by the NEA in 2016, which connects academic institutions with community partners, businesses, and artists; and the Bridging Cultures initiative, launched in 2012 by the NEH.That two of these successful programs—Georgia Tech VIP and Montgomery College Global Humanities Institute —have connections to SENCER is no surprise
Though curricular resources are emerging, a quick review of the archived video footage of the meeting that accompanied the launch of Branches from the Same Tree reveals two things. Committee Chair David J. Skorton, Secretary of the Smithsonian, chuckled multiple times as he revealed that the committee was governed from the ground up, reflecting the horizontal nature that often accompanies interdisciplinary learning.He claimed to have little authority to rein in the committee members, and instead allowed their collective expertise to guide the process. The second interesting reveal is that the committee found little research in the way of“controlled” studies regarding how integrated learning influences student learning outcomes. In response to an attendee’s question regarding challenges (see Chapter 4 and the video link), Chair Skorkin mentioned the number of confounding variables that are part of each student’s life and make controlled studies impossible. In Chapter 4 of the report, the authors also remark that implementation of integrated courses can involve multiple variables that are difficult to tease apart or control, as they are distributed across different institutions and adapted/adopted by different faculty members. Moreover, the integrated course is not always a single treatment or intervention, but instead involves multiple factors, such as content, methodology, pedagogy, and assessment.Despite the limited evidence, the committee members believe that what they have seen is promising for students at two-year and four-year undergraduate institutions, as well as those in graduate programs. Ashley Bear, the NASEM Study Director, feels that evidence gathered from the responses to the “Dear Colleague Letter” provide a rich collection of different methods and approaches to showcasing student learning, as do the comments gathered from employers and alumni, which are encapsulated in Chapter 6 of the report.
In Chapter 3 of the Branches report, “What is Integration?” the authors are careful to point out that disciplinary knowledge without synthesis does little to support the understanding of emergent ideas. Stephen J. Kline’s work on multidisciplinary learning is cited and his attention to emergence reminded me of another important piece of work, by David Edwards, artscientist and author of Artscience: Creativty in a Post-Google Generation (2009). Kline and Edwards advocate thinking more creatively about how arts, social science, and natural sciences can lead to new ways of doing and thinking. Yet many examples of integration remain at the level of service to one or the other discipline, which the report describes as “superficial.” For example, many courses seek to use the arts to communicate scientific knowledge or practice, or they use scientific methods to illuminate art practices as seen in art conservation. As the chapter illustrates, integration is a developmental process. As one moves from multidisciplinary to interdisciplinary to transdisciplinary, the emergent practice, method, or ideas can transform and morph an existing discipline or field, or produce a new one, or use a wholly different integrated approach to addressing a crisis, as seen with Mary Beth Hefferman’s work on the PPE Portrait project, which is designed to address the lack of humanistic interaction in highly contagious infectious disease treatment centers (p. 13 of the report).
Many attendees at the meeting that launched the report’s publication on May 7, 2018 were interested to learn of any potential opposition to the proposed integration model. Committee Member Bonnie Thorton-Hill remarked that many of the best models could be found outside traditional department structures, in institutes and centers. Because investment in infrastructure to support these initiatives may be a significant hurdle for some institutions, many authors of the report and attendees at the meeting saw this as an opportune time for the federal government to take the lead and stimulate implementation and research through funding streams and new initiatives.Further, the committee stressed the need to refrain from draining disciplinary resources but instead to build upon them.Another concern raised by attendees was how this work would be valued in promotion and tenure reviews, federal funding, and national accreditation standards, and some suggestions designed to address these inquiries are provided in Chapter 5 and on pp. 7–8 of the summary report.
Perhaps what was most refreshing about the attendees and the authors of the Branches report was the diversity of disciplinary perspectives, lived experiences, cultural and ethnic backgrounds, and attention to the changing nature of our student populations. Many of the examples presented in the chapters and mentioned at the meeting highlighted the ways in which integrated learning can lead to the development of sound decision-making, empathy, and awareness and tolerance for different ways of knowing and different points of view. These approaches align with the SENCERized approach to teaching and learning.
I would like to end this review with the compendium of more than 200 examples that is provided as a supplement to the Branches from the Same Tree consensus report and the “Gallery of Illuminating and Inspirational Integrative Practices in Higher Education.” The latter includes boxes and images scattered throughout the report, as well as a large collection appearing at the end of the report offering images and descriptions of artistic and humanistic scholarship, education, and practice that have been inspired, influenced, or supported by STEM knowledge, processes, and tools. A few SENCER projects are included in the compendium; some notable exceptions are highlighted below.
In keeping with the proposed next steps presented in the Branches report, Gillian Backus and Rita Kranidis, SENCER Leadership Fellows, have launched a STEM-Humanities Consortium effort.I encourage our SENCER community to take up the charge of contributing to this effort and to think carefully about how best to organize a multi-institutional research effort to assess the effect of integration on student learning, as described in this report.A list of possible research questions to drive such projects appears on p. 92 of the report.
From SENCER Hawaii (https://sencerhawaii.com/about-us/): Traditional Hawaiian values align closely with SENCER’s ideals and objectives for sustainability and stewardship of our community; curricular resources draw on ethics, culture, and history.
Katayoun Chamany is the Mohn Family Professor of Natural Sciences and Mathematics at Eugene Lang College of Liberal Arts at The New School and a Senior SENCER Leadership Fellow. She is the author of Stem Cells Across the Curriculumwhich has been selected as a SENCER model course.She is the recipient of the John A. Moore Award for Science as a Way of Knowing from the Society of Integrative and Comparative Biology and the William E. Bennett Award for Extraordinary Contributions to Citizen Science from the National Center for Science and Civic Engagement.
Edwards, D. (2009). Artscience: Creativity in a Post-Google Generation. Cambridge, MA: Harvard University Press.
Horton, M., & Freire, P. (1990). Is it possible just to teach biology? In Bell, B., Gaventa, J., & Peters, J. M.(Eds.), We make the road by walking: Conversations on education and social change (pp. 102–109). Philadelphia: Temple University Press.
National Academies of Sciences, Engineering, and Medicine. (2018). The Integration of the Humanities and Arts with Sciences, Engineering, and Medicine in Higher Education: Branches from the Same Tree. Washington, DC: The National Academies Press. https://doi.org/10.17226/24988. (Video link to Meeting held on May 7, 2018. (https://livestream.com/NASEM/events/8162141/videos/174486427) Q &A is rich in ideas for implementation and next steps.)
For the last four years, pharmacy, physician assistant, pre-medicine, and nursing students enrolled or associated with Butler University’s College of Pharmacy and Health Sciences (COPHS) and College of Liberal Arts and Science (LAS) have partnered with Barnabas Task to travel to the Dominican Republic (DR) for an annual medical mission trip. Barnabas Task, a nonprofit organization founded in Fort Wayne, Indiana, conducts multiple service trips every year with dental and medical professionals, as well as other volunteers, to the Dominican Republic, Cuba, or Guatemala. Barnabas Task’s mission is “community transformation through leadership development” (Barnabas Task, 2013), and they utilize community health evangelism (CHE) to accomplish this goal. During these mission experiences, students have the opportunity to assist medical providers through patient triage, medical scribing, and medication dispensing.Students also work directly with community leaders to educate them on public health topics including nutrition, exercise, smoking cessation, dental hygiene, and mosquito-borne illnesses. These community leaders can then educate others and spread the knowledge through grass roots. This philosophy of developing a relationship with host communities mirrors the work of Olenick and Edwards(2016). Their article in Nursing for Women’s Health concludes that short-term health missions are more effective when they focus on a “long-term commitment rather than a quick fix.”
Students and volunteers work to form long-term commitments not only by educating community leaders in the DR, but also by working with local students who act as translators within the clinic. Most of the students who made the trip lacked fluency in Spanish, and all volunteers are therefore provided with a translator. Every clinic day, students from Oasis Christian School, which is a part of Santiago’s private school system, help translate for the students and medical volunteers. Students from the local Catholic medical school, Pontificia Universidad Católica Madre y Maestra (PUCMM), also join the clinic daily to translate, triage patients, and fill prescriptions. Some students keep returning to the clinic even after they graduate medical school and volunteer as healthcare providers to help their community. This includes a provider who has made a commitment to visit the clinic quarterly to follow up with patients whose medications for chronic diseases such as diabetes and hypertension may require adjustments. Interactions with the DR students and providers adds another layer of collaboration, where students can learn from one another while caring for underserved populations.
To strengthen these long-term commitments, Barnabas Task turned to Butler University Fairbanks Center for Communications and Technology in 2015 with the goal of developing an electronic means of carrying medical information during the mission trips and accessing these records during future medical trips, thus starting the relationship between Barnabas Task and the Engineering Projects in Community Service (EPICS) course at Butler University. Computer science and software engineering students enrolled in this course meet biweekly to complete a “supervised team software project for a local charity or non-profit organization” (Linos, 2012). This relationship initiated the development of an Electronic Medical Records (EMR) application prototype, which runs as an iOS app. Students in the EPICS course collaborated with Barnabas Task to meet their needs to provide continuity of care and formed a relationship with healthcare students from COPHS to format the iPad application. Currently in the fifth semester of collaboration between EPICS, Barnabas Task, and COPHS, the application continues to be updated and built upon and is now a stable prototype of a bilingual EMR that can preserve patient records, transcribe prescriptions to the clinic’s pharmacy, and maintain medication inventory.
Data on the benefits of EMRs are plentiful. A systematic review published in September 2017 established how EMRs significantly improve documentation of clinical information and enhance quality outcomes in the long-term acute care setting (Kruse et al., 2017). Similar effects can be seen in the inpatient hospital setting. Khalifa and colleagues found that after EMRs had been implemented in their health system, there was “an increase in information access, increased healthcare professionals productivity, improved efficiency and accuracy of coding and billing, improved quality of healthcare, improved clinical management (diagnosis and treatment), reduced expenses associated with paper medical records, reduced medical errors, improved patient safety, improved patient outcomes and improved patient satisfaction” (Khalifa, 2017). A comprehensive review by Keasberry, Scott, Sullivan, Staib, and Ashby (2017) ascertained that EMRs enhance patient safety by including alerts about drug interactions and adverse drug reactions. The utilization of an EMR also improves patient outcomes by increasing to guideline recommendations. EMRs stateside improve hospital processes and patient care, which explains the DR clinic’s need to obtain an EMR to improve clinic processes abroad.
We conducted a thorough search and determined that there are no similar efforts currently described in the literature. However, there are publications that discuss collaborations and active learning as well as the benefits of these types of interactions. A group at the University of Wisconsin created interprofessional groups that served both a local community and a global community in Malawi. They concluded that students had increased their level of understanding in values and ethics, roles and responsibilities, and teamwork as a result of the experience (Dressel et al., 2017). Johnson and Howell (2017) also discuss the benefits of service-learning and interprofessionalism. Healthcare students from different programs including pharmacy, medicine, physical therapy, and nursing traveled to Ecuador for a service-learning opportunity. The authors explain how the students had to work through communication barriers both with their patients and with other healthcare professionals, all of whom spoke a different language. Increasing cross-cultural and interprofessional learning will be crucial in the future due to the diversifying healthcare system. A nursing cultural simulation developed by Carlson et al. (2017) connected nursing students in Hong Kong and Sweden and ultimately ascertained that the intercultural experience developed collaborative skills, including communication, between the two groups of students as they worked to complete a case study. In our literature review we found plenty of interprofessional articles; however, the literature lacks information on students from different colleges collaborating on a project to better the community they plan to serve. Professionals in the healthcare field are being exposed to a wide array of people with different educational backgrounds, and it is important to confront these language and knowledge barriers.
This study was developed in order to (a) assess how information technology affects clinic processes, (b) identify student learning and cultural awareness when collaborating with students from different colleges and globally, and (c) understand how global missions are viewed by the communities being served.
When commencing this project we hypothesized that students would gain knowledge about how to work with other professionals, increase their skills within their various areas of expertise, and develop cross-cultural awareness while helping to improve a community’s health with the creation of an EMR. The institutional review board approved the anonymous survey that was sent to all sixty-five volunteers who worked in the underserved clinic in the DR and the EPICS students who helped develop the EMR but were unable to go to the DR. Using Qualtrics (Qualtrics, Provo, UT, May 2017), an online survey platform, the survey was created to consist of multiple choice and free response questions regarding demographics, role in the project, and experience in the clinic. Utilizing skip logic, participants answered questions written specifically for their role in the clinic (for example, healthcare student; computer science student; translator; etc). The original survey questions are listed in appendix 1. Results from the open-ended questions on the survey were analyzed based upon common themes and similar wording found throughout the participants’ answers. The institutional review board also approved an anonymous quality survey all patients at the clinic eighteen years of age and older had the opportunity to take. Those who participated answered four questions about their time spent in the different stations of the clinic, whether they would recommend the clinic to their friends or family members, and whether they believed the clinic brought hope to the community. If an entire family came to the clinic, one person from the family could complete the survey for their household. In total, 95 patients completed the survey using SurveyMonkey.
Of the 65 clinic volunteers who were sent the survey, 51 elected to complete it for a response rate of 78.5%. The specific roles for each of the responses are illustrated in Figure 1. Starting with student learning, knowledge was gained through this experience through the various collaborations. The EPICS team, healthcare professionals, and Dominican volunteers all had participants who reported their top learning experience was in communication. Three out of five of the EPICS team members stated their top two non-technical learning experiences were in communication and teamwork. Students are also retaining the knowledge from this experience, as five out of five responses by the EPICS team stated they have used the knowledge gained in this course outside school or in another class. One EPICS member conveyed the importance of this class being able to “bridge the gap between those who are very technical, with little healthcare experience, and healthcare clinicians who possess little technical expertise.” Examining the development of technical skills, all of the EPICS students grew in both Xcode (Apple’s software development environment) and Swift (Apple’s programming language) (Apple, 2018). One EPICS student gained experience in setting up an onsite clinic with WiFi to make sure the EMR application could work within the clinic and the iPads could communicate with one another. Not only did EPICS members learn technical skills to be used in their future careers, but students also reported an improvement in their Spanish and an increase in knowledge about the Dominican healthcare system and culture.Similarly, half of the healthcare students reported an increase in knowledge about the Dominican culture, lifestyle, and healthcare system as one of their top three learning experiences. Not only did American students learn from the Dominican students, but four of the six Dominican students who took the survey noted that one of the benefits of the clinic was being able to practice their English, while three of six students stated their main benefit from the clinic was refining their medical skills with the collaboration of American and Dominican providers.
The survey also included questions about the students’ experiences in intercultural and interprofessional relationships. Five out of six EPICS students reported a positive interaction when working with students with a healthcare background. One student, when asked to comment on his or her overall experience with the COPHS and EPICS students, remarked that it “was extremely fulfilling to witness how the efforts of a variety of students can put their knowledge and skills together to make something special happen.”Eleven out of thirteen healthcare students reported a positive experience when collaborating with the EPICS team and one stated specifically that the EPICS team is “important for our clinic running smoothly.”
While healthcare students, the EPICS team, and Dominican students gained great knowledge while working together, so did the healthcare professionals who helped run the clinic. Half of the providers stated there was a benefit to working in a different scope of practice in a different culture and stated that their biggest challenge was language barriers between their patients and sometimes their translators. However, the EMR application may have reduced this language barrier by means of prototype through an English-Spanish toggle. All of the providers who took the survey would be interested in using the application in the future. Three out four healthcare providers stated that the application improved the efficiency of the clinic, and one of the providers stated that the EMR improved patient safety by forgoing legibility issues of doctor’s handwriting and by allowing the provider to see previous visit history and ascertain a past medical history.
Improving clinic operations was important, but so was seeing the hard work come to life.From one of the EPICS students who attended the trip: “There aren’t any words to put in for the experience of the trip. It was incredible and even better on our end to see the work we put in over the semester at work in real time helping people in need. It really gives us a different perspective. It has made me want to go back again next year.”
Both healthcare and EPICS student teams appreciated the each other’s knowledge base and were able to learn from one another. Seven out of seventeen students from EPICS and future healthcare providers suggested there be more meetings between the two student teams to allow more communication and form better relationships and to improve collaboration on the application prior to the trip. One student conveyed his or her suggestion for improved interactions by stating: “I wish the healthcare students could have had a larger impact when it came to some of the formatting in the app.” Another stated, “we could have been helpful when it came to inputting drug names and formatting it the way that most resembles a prescription.”
One example of the collaboration between the two groups was a simulation clinic on Butler University’s campus before heading to the DR. One EPICS student stated: “Witnessing and collaborating with the students who would actually be using the application was vital.
We were able to together identify the most effective and efficient designs for the app, as well as locate bugs throughout the app that we may not have otherwise noticed.” Four of thirteen healthcare students who attended the simulation said the simulation helped students learn how to use it before traveling to DR and six out of thirteen healthcare students noted there was value in the simulation because it worked out issues beforehand and allowed the EPICS team to add more features to application. More collaboration is necessary because while 10 out of 16 users of the EMR said it was a positive experience, five out of the 16 said there was need for improvements. While the EMR needs improvement, all of the 13 healthcare students who took the survey stated that their overall experience was positive.
Finally, knowledge was gained through this experience but so were friendships.
“The trip felt like a once-in-a-lifetime experience. It was incredible to witness both teams’ work and preparation pay off. Our group of students formed a tight-knit group with relationships that will likely last a lifetime. We were also able to form friendships with people there and share our cultures with one another. I greatly enjoyed the activities outside of the clinic—they provided inspiration on how we can continue to make a difference.”
While the application and learning is important for the students, for healthcare professionals the patient is the top priority, and for engineers the customer is the top priority. To ensure our patients were satisfied and to see how an EMR effects clinic processes we interviewed 95 patients to assess where there is room for improvement with our application and clinic in the future. Figures 2–5 represent how patients responded when asked about the amount of time it took to enter the clinic, register at the clinic, see the physician or healthcare provider, and receive their medications. Responses concerning the amount of time it took to enter the clinic were the most evenly distributed of the four figures, ranging from “very fast” to “normal” amount of time. The amount of time to be registered as well as to see a provider were very similarly distributed, with only a small percentage of patients reporting “too long” of a wait. The amount of time to receive medications followed a similar distribution to Figures 3 and 4; however, it was the largest report of “too long” a wait. Patients were also asked if they had attended the clinic previously, which 46 out of the 95 patients who completed the survey had.Of the 60 patients who responded to the question about whether this clinic brings their community hope, all answered “yes” and all 95 patients who answered the survey said they would recommend this clinic to their friends and family.
The professional world becomes more intertwined each day with professionals obtaining multiple degrees, technology advancing at a rapid pace, and the increased need for multiple professionals to be working together to achieve a common goal. Students with healthcare or computer science backgrounds will work together once they enter their careers, because healthcare is constantly in conjunction with, and reliant on, technology. Learning about other disciplines through collaboration towards a mutual goal helps prepare students of both colleges and disciplines to better communicate with people who have different educational backgrounds.
Beyond communication, other lessons learned through this experience included collaboration and teamwork. This project began through collaboration, as Barnabas Task has been collaborating since 2008 with people from varying cultures to facilitate CHE. Butler University began helping staff and supplying clinics in 2014, and the EPICS team was introduced in 2015 to create the EMR application (Barnabas Task, 2013). Similar to the mission trip described by Dressel et al. (2017), students reported an increase in their teamwork skills. The application continually evolves as innovative ideas develop from communication and teamwork between the EPICS and healthcare students. To improve both this learning experience and the application, the EPICS and healthcare teams need more collaborative meetings and communication, which have been set up via live simulated clinic days in the United States. The team views the application working in real time and can modify the application before arriving at the clinic. The need for more simulations was reiterated in the survey results: almost half of students wanted an increase in the number of meetings between the two groups prior to the trip. More meetings will allow for the healthcare students to help update the prescribing and diagnostics parts of the application and to provide recommendations for further clinical functions in the prototype application, including drug interaction reporting and other patient safety features.
It is important that the students gained knowledge from this collaboration, but ultimately the goal is to help the patients in the DR. An EMR application is warranted for helping track past medical records; over half of the patients who took the survey reported being seen in the clinic previously. With patients returning each year, there is clearly a need for the clinic, and the clinic is being utilized as routine care for many people. The application allows past medical records to be viewed, to see progression of disease states and to ensure that the patient is receiving the best care possible. The application improves patient safety by allowing allergies to be documented and viewed through their prior visit history. The support for EMRs improving patient safety has been shown in the work of Khalifa (2017), as there were fewer occurrences of medical error. Providers can also access medication histories to track clinical progression. Not only does the application help prevent medication errors, it also improves the processes of the clinic. Patients are quickly registered and triaged and then sent to see a provider, without the hassle of paper charts. Only two of the 95 patient respondents commented that any step of the clinic took too long. Future development and evolution of the application could help further streamline clinic processes and improve patient satisfaction.
Not only is the application evolving, but so is the EMR EPICS project. There has been a growing number of EPICS students interested in the collaboration with healthcare students. The EMR project continues to attract new and returning Computer Science and Software Engineering (CSSE) students, who find this project intriguing and realize the potential it has for experiential learning. The EMR project has spanned over six consecutive semesters and has currently attracted and engaged 35 CSSE students. The trip teaches students to collaborate with students of different educational backgrounds and helps students discern their future career paths. One of the EPICS students changed his major after exploring his passion for computer programming while working on the EMR project. All participants in the application collaboration group reported some form of educational growth.
Beyond their own education, this experience also exposes students to the education styles of the Dominican Republic. Medical school in the DR takes six years to complete as opposed to the eight years required to achieve a medical degree in the United States. Cultures differ not only in education but also in communication styles and language. Learning to respect the cultures and healthcare systems of other countries will help students become more adaptable and knowledgeable as they embark on their future careers. It is also beneficial to familiarize oneself with other cultures, because many medical professionals are obtaining their degrees abroad, while still wishing to practice in the United States. This trend was voiced by many of the medical students who acted as the group’s translators during the clinic in the DR. As of 2006, approximately 25% of physicians practicing in the United States obtained their medical degree abroad, a number that has been increasing since the 1960s (Boulet, Cooper, Seeling, Norcini, & McKinley, 2009). Not only are physicians with different educational backgrounds practicing medicine in the United States, there has also been an increase in the number of foreign-born United States citizens. With almost 13% of the United States’ population being born in another country, providers will be encountering patients with a variety of backgrounds (Singer, 2013). It is important for healthcare providers to adapt and be knowledgeable of cultures different from their own.Cultural awareness is the main experience gained from clinics where US and DR students volunteering together.
In the future, it would be beneficial to continue to track patient surveys to ensure that the application keeps improving patient satisfaction and clinic efficiency. However, it is reassuring to see that a majority of patients did believe that their wait times were acceptable and that the clinic is currently working at an efficient pace. Looking forward, it would also be appropriate to start examining clinical outcomes of patients, as the EMR is able to track them on a yearly basis to see whether medical interventions are making a long-standing impact on patients’ disease states. As Kruse et al. assert (2017), EMR systems can improve quality outcomes for patients in the acute setting. Data collected from the DR clinic could be examined to determine whether these same improvements can be repeated. Overall, the collaboration between healthcare students and computer science students has led to the production of a functioning, affordable EMR application prototype to improve patient safety and satisfaction. It has also expanded technical and communication skills for students across Butler’s campus and among the DR students that Butler University connects with while in the DR. The goals of this project in the future would be to keep improving the application and eventually provide access to the application to other non-profit organizations to help them serve their patient population.
These data were presented at the National Center for Science & Civic Engagement Conference for Science and Engineering for Social Good in Atlanta, Georgia in February 2018. At the conference many people, including Edward Coyle, co-founder of both the Vertically-Integrated Projects (VIP) program and the Engineering Projects in Community Service (EPICS) gave us advice for proceeding with our project.
About the Authors
Courtney Cox is a current pharmacy student at Butler University and has traveled to the Dominican Republic three times with the team. After graduation in May 2018, she hopes to pursue a career that allows her to continue to work with an underserved population both in the United States and abroad.
Sarah Lenahan, Class of 2019 PharmD candidate at Butler University, has traveled to the Dominican Republic twice working with the Electronic Medical Record application and will be going again during May 2018. She hopes to pursue a career in pharmacy that allows her to integrate her passions of faith, learning, and pharmacy to help underserved patient populations.
PatriciaDevine is an Associate Professor and Campus-Based Experiential Education Director at Butler University College of Pharmacy and Health Sciences. Her passion and research interests are in improving health globally.
PanagiotisLinos has been a professor of Computer Science and Software Engineering at Butler University since 2001. The birth of the EPICS program at Butler is the result of his passion for community service and experiential learning. Before joining Butler, he was the Chairperson of the Computer Science department at Tennessee Technological University.
Apple. (2018). Apple Worldwide Developers Conference. Retrieved fromhttps://developer.apple.com/wwdc.
Barnabas Task: Story-Teller of Many. (2013) Retrieved from http://www.barnabastask.org/.
Boulet, J. R., Cooper, R. A., Seeling, S. S., Norcini, J. J., McKinley, D. W. (2009). U.S. citizens who obtain their medical degrees abroad: an overview, 1992–2006. Health Aff (Millwood), 28(1), 226–233. doi:10.1377/hlthaff.28.1.226
Carlson, E., Stenberg, M., Chan, B., Ho, S., Lai, T., Wong, A., & Chan, E. A. (2017). Nursing as universal and recognisable: Nursing students’ perceptions of learning outcomes from intercultural peer learning webinars: A qualitative study. Nurse Educ Today, 57, 54–59. doi: 10.1016/j.nedt.2017.07.006
Dressel, A., Mikandawire-Valhmu, L., Deitrich, A., Chriwa, E., Mgawadere, F., Kambalametore, S., & Kako, P. (2017). Local to global: Working together to meet the needs of vulnerable communities. J Interprof Care,, 20, 1–3. doi:10.1080/13561820.2017.1329717
Johnson, A. M., & Howell, D. M. (2017). International service learning and interprofessional education in Ecuador: Findings from a phenomenology study with students from four professions. J Interprof Care 31(2), 245–254. doi: 10.1080/13561820.2016.1262337
Keasberry, J., Scott, I.A., Sullivan, C., Staib, A., & Ashby, R. (2017). Going digital: a narrative overview of the clinical and organisational impacts of eHealth technologies in hospital practice. Aust Health Rev 41(6), 646–664. doi: 10.1071/AH16233.
Khalifa M. (2017). Perceived benefits of implementing and using hospital information systems and electronic medical records. Stud Health Technol Inform, 238, 165–168.
Kruse, C. S., Mileski, M., Vijaykuma, A. G., Viswanathan, S. V., Suskandla, U., & Chidambaram, Y. (2017). Impact of electronic health records on long-term care facilities: Systematic review. JMIR Med Inform, 5(3), e35. doi: 10.2196/medinform.7958
Linos, P.K. (2012). Ten Years of EPICS at Butler University: Experiences from Crafting a Service-Learning Program for Computer Science and Software Engineering. In B. A. Nejmeh (Ed.), Service-Learning in Computer and Information Sciences: Practical Applications in Engineering Education (pp. 39–75). Hoboken, NJ: Wiley.
Olenick, P., & Edwards, J. E. (2016 ). Factors to consider when planning short-term global health work. Nurs Womens Health, 20(2), 203–209. doi: 10.1016/j.nwh.2016.01.003
Singer, A. (2013 ). Contemporary immigrant gateways in historical perspective. Daedalus, 142(3), 76–91. doi:10.1162
Is this your first experience with Barnabas Task?
How many times have you worked with Barnabas Task?
a. 1-2 times
b. 3-5 times
c. 6 or more times
What was your role with the EMR app?
a. Healthcare Student
b. Healthcare Provider
c. EPICS Team
d. Translator (PUCMM or OASIS Student)
e. Clinic Organizer
Describe your major.
b. Physician Assistant
Why did you select this project? What was your motivation behind selecting this project?
Name the top three non-technical learning experiences that you took away from the EMR project.
Name the top three technical learning experiences that you took away from the EMR project.
Comment on your overall assessment and grading of your performance throughout this project.
Did you participate in the trip to the DR?
Comment on your overall trip experience.
What did you learn from the PUCMM/OASIS students while working in the clinic?
Comment on the amount of time spent on devotions and reflection.
Did your faith change or grow? Comment on this.
Were you interested in going on the trip to the DR?
What prevented you from going on the trip?
Comment on your experiences of interacting with the healthcare students.
What suggestions do you have to improve the way the two teams interacted?
Did you participate in the EMR simulation in March?
What value did you find in this simulation?
How have you used the knowledge and skills from this course outside of the classroom?
Why did you decide to participate in this trip?
Name the top three learning experiences that you took away from this experience.
Comment on the amount of time spent on devotions and reflections.
Did your faith change or grow? Comment on this.
Comment on your experience with the EPICS team (those that went on the trip and those that did not).
What suggestions do you have to improve the way the two teams interacted?
Comment on your overall experience in the DR.
What did you learn from the PUCMM and OASIS students while working in the clinic?
Comment on your experiences using the EMR app to automate the patient care process in the DR.
What did you like about the EMR app? What would you improve or change?
Did you like the text boxes used for diagnosis?
Did you participate in the EMR simulation?
What value did you find in this simulation?
What is your role and capacity of involvement in the clinic? Comment on your previous involvement with Barnabas Task medical clinics.
Comment on any benefits and challenges you had from your participation in this clinic.
Did you utilize the EMR app?
Describe your overall experience and impression of the EMR app. How did you find it useful? How could it be improved?
How do you think the app affected patient care?
Would you be interested in using it in the future?
What is your role and capacity of involvement in the clinic?
Comment on any benefits and challenges you had from your participation with this clinic.
42. Did you utilize the EMR app?
Describe your overall experience and impression of the EMR app. How did you find it useful? How could it be improved?
44. Would you be interested in using it in the future?
Translators (PUCMM or OASIS students)
What was your role in the clinic? Comment on any previous experiences with Barnabas Task.
46. Comment on any benefits and challenges you had from your participation in the clinic.
47. What did you learn from the American students?
48. Did you use the EMR app?
49. Describe your overall experience and impression of the EMR app. How did you find it useful? How could it be improved?
50. Would you be interested in using it in the future?
This article provides a case history of the beginnings of SENCER-ISE (Science Education for New Civic Engagements and Responsibilities – Informal Science Education), an initiative that encouraged structured partnerships between higher education and informal science educators using civic engagement as a unifying framework for the collaborations. The article provides background on why SENCER-ISE was a natural progression for the National Center for Science and Civic Engagement (NCSCE) to pursue and how SENCER-ISE was implemented. Partnership projects and specific outcomes are provided as examples of the civic engagement cross-sector work and evaluation results are given of the overall efficacy of such partnerships.
Audiences served by informal and formal educators expanded
Civic engagement focus as a strategy for learning
Partners’ areas of expertise respected
These are some of the positive outcomes expressed by educators who participated in SENCER-ISE (Science Education for New Civic Engagements and Responsibilities-Informal Science Education), the National Center for Science and Civic Engagement’s (NCSCE) cross-sector pilot project to bring together individuals from the higher education (HE) and the informal science education (ISE) sectors through civic engagement partnerships (Randi Korn & Associates [RK&A], September 2015). The initiative was a natural outgrowth of NCSCE’s fundamental emphasis on framing teaching and learning around real-world problems and experiences. Civic issues, whether related to water quality, invasive species and habitat loss, or education, formed the underpinnings of the projects developed through SENCER-ISE, an initiative that benefited from the infrastructure provided by NCSCE.
As one informal science education partner noted in an evaluation report from Randi Korn & Associates (RK&A, September 2015),
From just looking at the other projects and learning about the other projects in my cohort, it seems like [our] project was true to what SENCER’s philosophy is, the way SENCER first started. We’re not going to keep science in a bubble or a laboratory, but we’re going to actually apply it. … We went to the workshop before the project really kicked off to learn more about the philosophy,… and how it’s been used to add another dimension to college courses, that was cool, and that’s what made this class so successful, that idea, that philosophy.
This case study will examine the experience of implementing the first stages of SENCER-ISE and will review the initial results. The study will outline the partnership projects to provide the context of how building an initiative around a civic issue can focus implementation efforts, meet actual challenges, and provide benefits to the educators and to the audiences served.
Background: Developing a Concept
In October of 2008, the National Center for Science and Civic Engagement (NCSCE) began a journey that continues as of this writing. Interest in exploring the practicality of civic engagement cross-sector partnerships heightened for NCSCE leadership, a number of informal science educators, and external funders, and they could see potential benefits to justify investing in infrastructure support to strengthen nascent or more casual collaborations. The setting was a MidAtlantic SENCER Center for Innovation regional meeting held at Franklin & Marshall College (NCSCE, MidAtlantic (October 4, 2008) The meeting focused on the critical role of K-8 STEM (science, technology, engineering, and mathematics) education as a “gateway” to STEM achievement.
One of the speakers, the late Alan Friedman, presented on a variety of topics that day, including a breakout session on communicating science to the public. Friedman had been the longtime director of the New York Hall of Science. At the time of the Franklin & Marshall meeting, Friedman was a consultant in museum development and education. He became the founding director of SENCER-ISE.
Through discussions at the meeting about the work of SENCER in engaging students with real-world civic issues, Friedman began to form a kernel of an idea that became the SENCER-ISE initiative. In an email to then NCSCE Executive Director David Burns and others on November 9, 2008, Friedman noted that “informal science education is open to the lessons of SENCER,” in that citizen science and science centers were paying “increasing attention to social issues.” He thought that a “working” conference to investigate the point of view of each sector towards civic engagement and to develop effective strategies to make collaborations work would be a next step. Others at the time wrote about the importance of seeing the formal and informal sectors as a continuum for learning through formal classroom use of “free-choice science learning resources and opportunities … for field trips or … guest speakers” (Liu, 2009). Friedman had something more in mind, in that he saw how SENCER’s model of learning through the lens of civic issues could impact the outcomes of potential partnership projects.
The following October, another MidAtlantic Center meeting at Franklin & Marshall focused on how informal science education experiences could improve college readiness. Friedman was one of the key speakers, along with David A. Ucko. Ucko was then Deputy Division Director, Research on Learning in Formal and Informal Settings, at the National Science Foundation; he, along with Marsha Semmel, are both independent consultants and became senior advisors for informal science education at NCSCE after Friedman’s untimely death. Both Friedman’s and Ucko’s presentations focused on the world of informal science education and its relationship to K-12 and higher education.
Over the next two years, other discussions, presentations, and proposals culminated in SENCER-ISE, an invitational conference held in March of 2011 (funded by the NSF, DRL1001795, and the Noyce Foundation) that brought together 20 SENCER faculty members and other NCSCE staff, with 20 professionals from informal science education institutions, such as science and nature centers, museums, and science media (NCSCE, 2011). As a result of this meeting, the “cross-sector partnership” concept developed into the SENCER-ISE II initiative (aka SENCER-ISE). Six partnerships were funded by the National Science Foundation (DRL1237463) and four by the Noyce Foundation. Eight of these ten partnerships continued with some type of collaboration at least through the end of the funding period.
The purpose of SENCER-ISE, to paraphrase what Ucko noted during a presentation at the 2017 SENCER Summer Institute, was to show that through the framework of civic issues, we could find common ground and “leverage synergies” for cross-sector partnerships that could “foster STEM learning and public engagement” (Concurrent Session on SENCER and Informal Science Education, Summary Slide found here. Ucko had previously written about SENCER synergies with informal science education in the Summer 2015 issue of this journal, which served as a tribute to Alan Friedman and focused on informal science education connections to formal education.
Background:NCSCE’s Path to Cross-Sector Civic Engagement Partnerships
Although there are many differences between formal and informal science education learning environments, there are commonalities between SENCER Ideals, its approach to learning, and the informal science education community’s goals. For NCSCE staff and colleagues, the timely publication of the 2009 NRC report, Learning Science in Informal Environments: People, Places, and Pursuits, fueled the notion that the underlying possibilities of higher education faculty and informal science educators working together collaboratively could evolve into enduring civic engagement partnerships. The NRC report postulated “strands of learning,” which in many ways reflected such SENCER Ideals as starting the learning process with matters of interest to students, beginning with projects that are practical and engaging to students, and locating the responsibilities of discovery in the work of the student (Friedman & Mappen, 2011, p. 32).
The March 2011 invitational conference, with its goals of sharing the strategies higher education and informal science education (HE-ISE) communities used to “implement the civic engagement approach” and “mapping possible collaborations,” found a mutual interest by professionals from both sectors in developing “science-enabled citizens” and in using civic engagement platforms as a bridge across the sectors. Another important focus of discussion at the conference was the importance of “a continuum of engagement to address learner interests and needs from K-12 through higher education and adult learning, including both in-school and out-of-school learning opportunities” (McEver, Executive Summary, 2011). The conference evaluator’s report concluded that “there was a need to build awareness of the value of using civic engagement as a platform to advance science understanding, including what each sector brings to a potential collaboration…” and that “the SENCER-ISE conference successfully sparked ideas and built momentum for collaboration” (RK&A, 2011).The evaluators noted that sustaining the momentum after the conference was a challenge given daily responsibilities, not an uncommon factor in developing and maintaining meaningful partnerships. Two articles by Friedman and Mappen detailed the path to SENCER-ISE through 2012.
The first, published in this journal in 2011, focused both on the idea of differences and commonalities in learning environments and goals between these educational sectors and also on the 2011 conference. The second one, a chapter published in 2012 as part of an edited volume on the expanded use in science education of the SENCER model of learning through the framework of civic issues, looked more deeply into the idea of developing an infrastructure to support partnerships between informal and formal higher educators and the potential benefits and challenges of collaboration “across the HE-ISE divide.”
The 2012 chapter also noted that most interactions between formal and informal education occurred at the K-12 level. The value of this connection between the two sectors can be seen in some earlier works, which also speak to the need to make these relationships more meaningful. An article summarizing two research studies about Informal Science Institutions (ISIs) published in the International Journal of Science Education in 2007 highlighted that these institutions “support K-12 education in the United States in important and varied ways” through field trips and other outreach programs but concluded ISIs had at that time “yet to determine how best to support students and teachers in terms of the actual curriculum and materials used in the classroom,” which could have “rich potential” for school science education (Phillips, Finkelstein, & Wever-Frerichs, 2007). To paraphrase Bevan and Dillon (2010), the “ubiquitous use of field trips” hid the gulf between creating substantial partnerships for learning in formal and informal contexts and one-shot experiences (pp. 176–177). Rivera Maulucci and Brotman (2010) summarized an in-service and preservice teacher training seminar that utilized trips to a museum “as a place to learn science connected to mandated science curricula” in NYC that began to “bridge” the gap between formal and informal science learning by including a local natural history museum, local public schools, and an undergraduate teacher education program as the partners.
From 2008, Friedman’s developing vision for collaboration between higher education and informal science institutions was based on his analysis that the SENCER approach to learning, which engaged “students with real civic and social issues,” could shape students’ understanding of “how important science, technology, engineering and math [was] to their own lives and to their communities.” At the same time, he thought that the informal science education community that he knew so well was “discovering the importance of this strategy” (Friedman, email, November 9, 2008).
That Friedman could imagine the future directionthe informal science education community would take is evidenced by a May 2016 report by the Center for Advancement of Informal Science Education (CAISE, May 23, 2016) that highlighted the expanding landscape of informal science education over the previous ten years. SENCER-ISE was certainly part of this development, with its emphasis on collaborative work across the sectors and the involvement in most of its projects of students at different educational levels communicating science to targeted audiences in schools, science centers, and citizen science organizations. As noted, Friedman saw early on the possibilities of these types of collaborations. One conclusion of the CAISE report for the ISE community is the need to “build greater awareness of the values and goals of universities and academia, e.g., graduate student professional development and undergraduate enrichment experiences” (p. 15). Friedman foresaw this possibility a decade ago, and he also saw how much the higher education community could learn from informal science educators, especially in terms of communicating science to a diverse audience.
Background: From Vision to Implementation
While the major goals of the second phase of SENCER-ISE were to form enduring partnerships around compelling civic issues that could “provide models for others in the wider educational community to follow,” there was an interest in “building the knowledge base” to improve “the fields’ understanding of the nature (challenges and high potential) of HE-ISE partnerships” (email from Wm. David Burns to Alphonse DeSena and Myles G. Boylan, June 6, 2012). NCSCE would provide the infrastructure support to launch new or enhanced partnerships. SENCER Ideals and informal science education’s learning strands offered the intellectual framework for this “experiment.”
From the 2011 conference on, there were certain elements that those involved in creating and implementing the next phase of SENCER-ISE thought necessary for it to succeed. Appendix A lists key themes of discussions that began with the March 2011 conference and continued through a November 2011 follow-up meeting, the December 2012 Leadership Team meeting held after the NSF funding was received (the team included Burns, Friedman, NCSCE staff, representatives from RK&A, Advisory Board members, and others), and into the partnership recruitment and selection process. While not all of the strategies that emerged from these discussions were incorporated into SENCER-ISE, they do provide suggestions for an implementation framework from which to develop and sustain collaborative efforts for those interested in creating or enhancing cross-sector partnerships. The themes include
sharing information, both in person and remotely, including program outcomes;
creating joint experiential opportunities and new learning and work environments around civic engagement that contributes to problem-solving of compelling issues;
securing funding for test beds;
mentoring for project leaders/partners;
demonstrating respect for all partners and their different organizations;
providing institutional leadership support for partnership; and
meeting the challenges of working across sectors.
As a result of outreach to formal and informal science education communities, NCSCE received 30 applications for the initial six partnerships of $50,000 funded by the NSF, payable over a three-year period. Each of the applications was reviewed by at least five members of the Leadership Team and then discussed on a review call in April. When funding from the Noyce Foundation was awarded in July to support four additional partnerships, a decision was made to review again the top-ranked applications that were not selected in the first round.
Table 1 provides an overview of the ten partnerships and the civic issues that were proposed. The reviewers thought that these projects had the potential for longer-term relationships. Appendix B provides project titles and more detailed descriptions about the projects. See also http://sencer-ise.net/partnerships/ for more background information about the original partners, institutions, and activities.
Getting Started – Introducing Partners to NCSCE, SENCER, and SENCER-ISE
SENCER-ISE objectives included building connections and relationships between partners, across partnerships, with the SENCER-ISE staff, and with the larger NCSCE community while applying SENCER’s civic engagement framework. An orientation to SENCER-ISE and participation in a SENCER Summer Institute were two activities planned as part of the implementation process. Given the differences in the award timeframes, the NSF-funded partners attended the institute in the summer of 2013, where they participated in a pre-institute orientation session; the Noyce partners participated in an orientation program in October of 2013 and then attended the institute in 2014, where they also interacted with the NSF-funded partners.
Both orientation sessions provided guidance on the planning process, discussions about known obstacles to cross-sector collaborations, ideas about developing strategies to overcome challenges, and workshops on evaluation planning (clarifying project outcomes, developing indicators, and choosing data collection methods). To continue communications beyond the orientation gatherings, group video conference calls, individual partnership calls with SENCER-ISE staff, and a website for shared information were offered.
Planning and Implementing
Cross-Sector Partnerships: Challenges
Amey, Eddy, and Ozaki’s “Demands for Partnership Collaboration in Higher Education: A Model,” published in 2007 in New Directions for Community Colleges (NDCC), noted that “partnerships in academe are becoming more common” but that “relatively little is known about them.” Thus, these types of collaborations are “often challenging to develop and hard to sustain.” The authors raise questions about each participant’s motivation for engaging in collaborative efforts, differences in the organizational context of the partners, the departure of “critical” personnel, and differences in desired outcomes (pp. 5, 12–13). The focus of the chapter was on K-12 schools and colleges, but the content is highly relevant to the work between informal science education institutions and colleges and universities.
The Executive Summary for the March 2011 conference report, the project proposal, and subsequent experience with implementing SENCER-ISE echo some of the themes and questions raised in the NDCC chapter. Conference participants identified “potential obstacles,” that ranged from mutual misunderstanding about the work of the other sector, conflicting cultures and reward systems, different work patterns and crunch times during the year, and different views of the role of civic engagement. Higher education “participants saw civic engagement with science and technology-based issues as a means towards the end of science learning, while most of the ISE participants saw civic engagement with such issues as a valuable end in itself.”
NCSCE’s grant proposal to the NSF (2012) highlighted some of the key challenges Friedman and others saw in forming non-profit partnerships, especially between higher education and informal science education institutions. These challenges, along with some potential proposed solutions to how they might be overcome, included the following:
Difficulties in establishing and sustaining non-profit partnerships. Initial responsibilities, decision-making prerogatives and commitments from both sides need to be clearly defined from the start, although some flexibility is needed.
Differences in culture. These are rarely accounted for initially and can lead to misunderstandings as the partnership develops. Both sides need to begin to understand the different constraints and values.
Friction caused by time and other resource commitments. These should be defined and agreed to in writing at the beginning.
Institutional vs. individual commitments. These are often not appreciated at the beginning of a partnership.
Ad hoc relationships rarely are sustained. Organic relationships with goals that meet the mission needs of both partners are more likely to succeed.
In designing the plan for SENCER-ISE, the above broad challenges were taken into account. It was thought that they could be mitigated by
setting up a small central office to support the partners;
having partner institutional representatives sign a Memorandum of Understanding about requirements for receiving funds;
providing opportunities for communication between the partnerships through a website that contained information about the partnerships and milestones for activities (timelines) and also through scheduled video conference or telephone calls;
offering evaluation guidelines and training at the beginning of the partnership implementation period;
awarding start-up funds; and
attempting to integrate the partners into the larger NCSCE orbit.
As the partnerships got underway and as they progressed, other challenges cropped up, some more difficult than others to solve, some unique to individual institutions, and some related to reporting requirements and schedules proposed by SENCER-ISE staff.
The partners spoke about some of their challenges in their final reports. For example, faculty sabbaticals and staff changes occurred in over half of the partnerships. In one case, the partners maintained telephone contact, while the faculty partner’s students continued at the ISE facility. There was some scaling back of the project and the ISE educator took on more of a supervisory role. In the other sabbatical case, the program was refocused a bit. In both of these cases, flexibility was important. For the most part, staff changes were overcome, except in two of the partnerships. Both of these involved a faculty member and/or a staff person changing institutions. For one partnership, the changes occurred several times and the final change did the project in. For the other, the missions of each partner were too disparate. Still other challenges, more related to specific institutions, included Institutional Review Board issues, travel for participants, securing additional funds, teacher attrition, attracting sufficient audiences, and for some a concern over the quality of student-collected data. Fortunately, the two partnerships that relied on student data collection reported that the data collected were authentic and of good quality.
To evaluate the SENCER-ISE infrastructure and follow partnership progress, both external and internal evaluation methods were employed. RK&A was engaged to undertake both formative and summative evaluations. Annual reports and quarterly group video or individualized calls with each partnership provided updates about partnership activities. Each partnership also evaluated the impacts of their efforts on populations they served (students, teachers, communities), and these results were reported in final partnership reports.
The formative evaluation examined partner perceptions of the SENCER-ISE infrastructure. RK&A conducted in-depth telephone interviews of 20 participants, representing all ten partnerships, between June and September 2014. About one-half of the interviewees were from higher education and the other half from informal science education. The interviews produced descriptive data that were analyzed qualitatively, “meaning that the evaluator studied the data for meaningful patterns and, as patterns and trends emerged, grouped similar responses” (RK&A, April 2015).
Five trends emerged when the strengths of the SENCER-ISE infrastructure were examined: (a) funds, which helped secure personnel for the project; (b) structure, which for some helped the partners focus on quarterly progress; (c) inspiration, which for some helped to establish a connection with colleagues; (d) encouragement and feedback, which for some provided moral support; and (e) flexibility, which for some meant that the reporting process was adjusted based upon partner feedback. There were no discernible differences in responses by sector.
There were four major challenges: (a) partner relationship, which included for some communication issues and differences in schedules; (b) lack of clear expectations, which for some meant not knowing how much reporting was necessary, even with the Memorandum of Understanding listing reporting dates; (c) limited funds plus workload, which some thought should be adjusted so that some of the administrative work could be lessened; and (d) internal issues, which for some included personnel leaving the institution or a partner being on academic leave. There were few differences by sector.
For the summative evaluation, RK&A employed a “mixed-methods approach to explore the …[evaluation] objectives—in-depth interviews and standardized questionnaires.” Eighteen interviews were conducted with SENCER-ISE partners. As with the formative interviews, these interviews produced descriptive data (RK&A, July 2015).The summative evaluation explored four evaluation objectives. The first three focused on whether the partners: increased their understanding of each other’s field of expertise; appreciated the value of each other’s work and expertise; and increased their understanding of what creates a durable partnership.
The fourth objective explored whether colleagues of the partners realized “the value of the formal/informal education collaboration.”
The evaluators noted that “while these are the evaluation objectives, one can easily see what the project aspired to achieve in how the objectives are expressed. As such, the evaluation objectives can also serve as a list of the project’s outcomes” (RK&A, September 2015).
The responses are summarized in Appendix C, which provides statements made by the interviewees. Overall, the partners did increase their understanding of each other’s work and expertise, did appreciate the value of each other’s work and expertise, and did understand elements of durable partnerships. Some interviewees noted that others at their institutions were drawn to the efforts.
Partnership Results, Impacts, and Sustainability
The work of the partners on their individual initiatives was really the backbone and strength of SENCER-ISE. It is through the lens and words of the partners that we can see the benefits of cross-sector collaborations to learners (students, citizen scientists, community members) and to faculty members and informal science educators. The sections below contain excerpts from the final reporting of eight of the partnerships (October 2016) that were still in existence, starting with some of the reported results.
The partnership reports also provide insight on how cross-sector partnerships can impact science education and educators, including pedagogical methods of the partners and their colleagues and how the involvement of students from different levels of education (graduate, undergraduate, K-12) was a benefit to the work of both sectors.
In terms of the sustainability of cross-sector partnerships the eight were still hoping to keep the partnership relationships going in a variety of ways, even if different from their original projects.
Brooklyn College and the Gateway National Recreation Area of the National Park Service
Awareness of the marine plastic debris issue is growing in the school community. Schools/teachers are engaged in data-driven civic engagement. The marine plastic debris protocols developed through the project are used in undergraduate classes.
Cornell University and the Sciencenter
Sciencenter staff trained students from the Cornell lab on methods in informal science education.Students then came to [the Sciencenter] Head Start family engagement events, and helped facilitate activities with parents and their children. …The students contributed to family engagement events by providing examples of current research about how children learn and how that research can be applied to the activities [the Sciencenter] offered to the parents and their children.
Fordham University and the Wildlife Conservation Society
The content evaluation indicated participation in Project TRUE [Teens Researching Urban Ecology] caused a significant increase in students’ understanding of the scientific process and scientific bias. …After participation in Project TRUE, there was a51.36% increase in students’ understanding of the scientific process, and a 76% increase in students’ ability to recognize types of bias sampling.
New Mexico EPSCoR and the New Mexico Museum of Natural History
Hosted three successful retreats with keynote speakers (John Falk, Jamie Bell, and Rick Bonney). Provided funding for regional gatherings through a mini-grant program.
Paul Smith’s College and The Wild Center
As part of the “Communicating Climate Change” course offered in 2014 and 2015, students were given the opportunity to receive certification as Interpretive Guides through the National Association for Interpretation. … In 2014, eight of the 15 students …participated. In 2015, all 15 of the students received certification.
Raritan Valley Community College and the New Jersey Audubon
Recruited and trained fifty-five … volunteer citizen scientists . … [and] involved … eighty students through participation in course work and volunteer training [over the course of the project]. …Students [for example] led a training session for …citizen scientists in invasive plant identification and gave presentations to local stakeholders.
St. Mary’s College of California and the Lindsay Wildlife Experience
A smartphone app creation was both an instructional experience and it yielded LWE [Lindsay Wildlife Experience] a tool to educate the general public on how to interact with wildlife.
The University of Connecticut and the Connecticut Science Center
During the course of the project two genomics program/exhibit formats targeted at family audiences were designed and tested. One component focused on “Mutations-DNA Matching Pairs” and the other on “STEM Cells.” … Based on a random sample of visitors informally surveyed, …visitor’s post engagement demonstrated a 67% increase in the ability to answer a series of six questions about mutations correctly, and a 75% increase in the ability to select the correct response from a series of four questions about STEM cells.
Brooklyn College/Gateway National Recreation Area of the National Park Service
The project helped to extend notions of place-based environmental education, in particular the ways to connect students who live in urban areas to the environment and related issues through authentic science learning activities. It also provided an example of how schools and teachers could contribute to and use scientific data in the classroom.
Cornell University/Science Center
The ongoing impact will be in the pedagogical methods of the Sciencenter. … Research from the [Cornell] lab … [led to a] new practice of open exploration and sharing research-based content with guests.
Fordham University/Wildlife Conservation Society
One of the major contributions that Project TRUE can have in the field of science education is that a program for students from under-represented populations in STEM fields [using] urban ecology research (i.e., place-based field research) with near peer mentors, as well as mentors from both informal and formal learning environments, can be effective in increasing knowledge [and] increasing student engagement in a sustained topic. …
New Mexico EPSCoR/New Mexico Museum of
One of the major outcomes of this project was uniting the informal scienceeducators within NM ISE Net. … Keynote speakers provided opportunities for learning and … starting points for dialogue. …The educators were connected tolocal NM EPSCoR researchers with the broad goal of improving engagement with the public around energy research.
Paul Smith’s College/The Wild Center
Many of the gatekeeper audiences … were empowered by the student presentations in measurable ways, helping them better engage their broader communities about mitigating the regional impacts of climate change and making more environmentally informed decisions. …The students themselves alsorepresent an important gatekeeper audience. … Environmental science, natural resource, forestry, and outdoor recreation students preparing to enter the workforce are uniquely positioned to be useful interpreters of this information.
Raritan Valley Community College/New Jersey Audubon
The project has demonstrated the success that is possible when sufficient resources (time, energy, money, and expertise, etc.) are devoted towards reaching the goals of conducting research and fostering civic engagement in first- and second-year science students. …These kinds of investments from both parties…are not always available, so it helped [the faculty member] refine and streamline his teaching methods to focus on the essential skills and lessons needed to make student participation in this kind of integrated education-research-engagement project a success. … NJA [New Jersey Audubon] staff have grown to appreciate the value of this type of partnership and working with students and faculty to address conservation issues. …The SENCER model [is] likely to be used in future projects.
St. Mary’s College of California/Lindsay Wildlife Experience
Before SENCER-ISE, LWE did not look beyond its own inside sources for research or sharing. By utilizing student interests in environmental topics, the topics of interpretation to the public have opened up to include an emphasis on the bigger picture of major themes such as conservation, environmental impact, and loss of ecological habitats.
University of Connecticut/Connecticut Science Center
Two areas of the project that are likely to have significant interest among science educators and exhibit developers are the process of engaging high school students in the design and development of science education programs and exhibits, especially in collaborative teams with formal and informal educators and content experts from the research community (typically through universities and colleges). … and the use ofimprovisational training for team building and enhancing the communication skills of program staff and high school students. …The project [also] reframed the methods used by the Co-PI in both classroom and non-classroom settings for genomics discourse.
Brooklyn College/The Gateway National Recreation Area of the National Park Service
[Brooklyn College plans] to continue to collaborate with the NPS [National Park Service] on the marine debris plastic and other science and science education initiatives. The plastics protocol and associated activities will continue to be implemented in the Macaulay Honors Seminar, with plans to integrate it into Introduction to Environmental Science at Brooklyn College.
Cornell University/The Sciencenter
Absolutely! This partnership will continue. The actual research projects will change from year to year.
Fordham University/The Wildlife Conservation Society
Expanded Project TRUE through the funding of an NSF AISL [Advancing Informal STEM Learning] collaborative research grant …, which builds on the SENCER-ISE funded work, [and] will continue until 2019.
New Mexico EPSCoR/New Mexico Museum of
Natural History and Science
NM ISE Net working with NM EPSCoR. … currently discussing ways to build the network. …considering a distributed leadership model.
Paul Smith’s College/The Wild Center
The Co-PIs will look for ways to co-teach again, using the model developed by the project. The Paul Smith’s Co-PI will continue to be an important partner for The Wild Center.
Raritan Valley Community College/New Jersey Audubon
Will likely continue and expand the research, outreach and management efforts in the future. The data set … will provide valuable baseline monitoring data to determine the effectiveness of management efforts (e.g., deer enclosures, hunting programs, invasive removals, etc.).
St. Mary’s College of California/Lindsay Wildlife Experience
The partnership will continue since the College has a Community Engagement requirement as part of the Core Curriculum. Faculty are indeed looking to find various methods to collaborate with community partners. …. The Environmental Science faculty are considering numerous senior capstone projects … in collaboration with LWE. … A Pre-service Teaching Program faculty member has begun planning a collaboration to start in Spring 2017. …A Spanish faculty member has been encouraged to start a collaboration with LWE, and this Spanish translation course will help LWE generate appropriate materials in Spanish starting in 2017.
University of Connecticut/The Connecticut Science Center
The Science Center is still planning on installing and opening a genomics exhibition and program space in 2019-2020. … Retirement of the CSC (Connecticut Science Center) Co-PI … will require transition planning to determine the fesibility of establishing a sustainable collaboration that connects CSC program staff and audiences with the … University.
Building Upon SENCER-ISE
The importance of personal relationships in developing sustainable collaborations is one of the lessons learned from the evaluation of the work of the original ten partnerships. While face-to-face meetings are most preferable, efficiency and costs need to be considered. With funding from the Institute of Museum and Library Services (IMLS), NCSCE implemented “Partnership Champions,” a project that added five additional cross-sector partnerships to SENCER-ISE, this time with a professional development component conducted virtually and with a shorter funding period. (See Appendix D for the listing of partnerships and project titles). Five of the original SENCER-ISE partners took on the role of “eMentors” to a new group of partners and provided guidance, based on their own experiences, on forming and enhancing collaborations. Interim results were reported by Semmel and Ucko (2017) in an overview of SENCER-ISE for the informal learning community. The authors noted the importance of jointly creating an action plan and timeline for completion of project activities. In addition, they cited the need to understand and adapt to the respective organizational cultures and constraints of the HE and ISE partners.
The “Partnership Champions” summative evaluation (RK&A 2018) concluded that the project was a positive experience for the partners, though not without challenges. Factors that supported successful outcomes included ideological alignment, flexible scheduling, openness to each other’s ideas, and alignment with organizational missions. Challenges included prioritizing projects along with other job responsibilities, communication issues, and project administration requirements.
For the new eMentorship component, the RK&A report noted that
…overall, Participants’ experiences with eMentorhsip varied. The eMentorship seems to have been most useful for Partners and most rewarding for eMentors towards the beginning of the project, when Partners needed clarity on SENCER’s vision and help articulating intended outcomes for their projects. …Overall, almost all Partners were grateful for their eMentors help at this stage of the partnerships. …most eMentors said Partners were “open” to hearing their advice, which they appreciated.
For future initiatives that include an eMentoring component, the report suggests that the role of the eMentor needs to be more clearly defined than it was for this short “demonstration” project. Does eMentoring work best for new projects and at the beginning of a project, and how best can eMentors be matched with projects? And, while virtual communication is efficient, some face-to-face interactions are needed.
Broadening the Network
During the 2015 SENCER Summer Institute at Worcester Polytechnic Institute, discussions about the next iteration of SENCER-ISE began. In a follow-up meeting in September, SENCER staff focused on the idea of collaboration with other established networks as a way to scale up the initiative. A Collaborative Planning proposal was submitted to the NSF’s Advancing Informal STEM Learning (AISL) program. to maximize the collective impact of two well-established national STEM learning networks, Nanoscale Informal Science Education Network (NISE Net) and SENCER, by stimulating civic engagement and public understanding of science.
The one-year project was designed in three phases. In Phase I, leaders from SENCER and NISE Net focused on intensive exploration of their own and each other’s networks to map regional hubs and identify pre-existing relationships between individuals and institutions of the two networks, evaluate existing communications strategies, and collect, analyze, and compare evaluation and research findings from both networks. Phase II commenced with a two-day participatory planning workshop attended by leaders from NISE Net and SENCER as well as practitioners, researchers, and administrators with a range of backgrounds and perspectives on network building in both informal and formal education. One of the outcomes of that meeting is an article in this journal by Larry Bell, senior Vice President for Strategic Initiatives at the Museum of Science in Boston and, at the time, principal investigator and director of NISE Net, articulating the role of informal learning institutions in civic engagement (Bell, 2018).
Evaluation by RK&A following the workshop revealed the following insights regarding development of network collaboration, many of which reinforced findings from the evaluation of the SENCER-ISE partnerships. Sufficient time must be allowed for the prospective partners, no matter how willing and well meaning, to learn about each other’s cultures, processes, and future plans. Trust takes time to establish, as does understanding how different organizations and networks function. More time spent working together will encourage stronger relationships between the networks’ leaders and practitioners. In addition, collaboration must mesh with existing plans for each network. Sufficient capacity is also required. Finally, it is critical to clarify terms, goals, and purpose before entering a partnership.
Phase III included a survey of the SENCER and NISE Net networks. The survey proposed a new collaborative project involving SENCER undergraduates who would develop informal learning resources with an ISE partner based on civic engagement. Results from 158 respondents were overwhelmingly positive, indicating strong support from both sectors for future collaboration. Fifty-seven percent of college/university/faculty/staff selected “strongly agree” when asked if participating in the project would enhance student learning; 41% were “very interested” in participating, and 47 respondents asked to be considered as a pilot institution. Among ISE professionals, 57% of respondents indicated they were “interested” in learning more about the project; 46% indicated they were “interested in participating,” and 24% indicated they were “very interested.”
Conclusion – Elements of a Civic Engagement Partnership
In sum, for SENCER-ISE, the following factors influenced partnership development positively:
having the appropriate levels of decision-making authority and organizational support to make the partnership work (including a Memorandum of Understanding);
identifying and sharing common goals and missions;
allocating and devoting adequate time to build the partnership and project;
developing from the start and continuing to update long-term action and evaluation plans;
leveraging the strengths of each partner through clearly articulated roles and responsibilities; and
maintaining regular communication.
Even with challenges, we found important benefits that can accrue to faculty, informal science education professionals, and learners of all ages. These are
For faculty and informal science education professionals:
deepened understanding of the structure and constraints of each other’s professional practices and organizations;
increased respect for the unique skills of professionals from each sector;
expanded access to new audiences;
enhanced pedagogical methods;
increased involvement in civic engagement partnerships and expanded networks; and
heightened view of the role that students, particularly undergraduate students, can play in informal science educational programs.
increased engagement in learning through connections to real-world contexts, authentic research opportunities, community activities, and place-based education;
improved communication skills for students at all levels of education; and
increased involvement in and knowledge of compelling civic issues.
As Amey, Eddy, and Ozaki noted in 2007, “sustainable partnerships are based on being flexible to new inputs and adjusting accordingly. …” Flexibility in responding to changes and challenges, along with adepquate funding and a sufficient time frame to plan and then to work together were certainly relevant to the endeavors of the SENCER-ISE partners and will be for similar collaborations in the future.
About the Author
InJune of 2017, Ellen F. Mappen retired as a senior scholar and the project director for Informal Science Education Programs at NCSCE (SENCER-ISE). She was the founding and long-time director of the Douglass Project for Rutgers Women in Math, Science and Engineering (1986-2003). Under her direction, the project received the 1999 National Science Foundation’s Presidential Award for Excellence in Science, Mathematics, and Engineering Mentoring. In between the women in science program at Douglass College of Rutgers University and NCSCE, she served as the director of the Healthcare Services Program at the New Brunswick Health Science Technology High School. She holds a Ph.D. in History from Rutgers University (1977), with a focus on women’s history. Her dissertation focused on attitudes towards women’s work in late nineteenth and early century London.
Many individuals, only some of whom are noted here, were involved in bringing about SENCER-ISE. The late Alan J. Friedman and the then Executive Director of SENCER Wm. David Burns provided the impetus, theoretical framework, and practical ideas for implementation. The initiative could not have taken shape as it did without the initial involvement of a number of people: SENCER faculty members who came together at the March 2011 conference, along with a group of informal science educators, to examine the feasibility of cross-sector collaborations; Randi Korn of RK&A; Emily Skidmore; Cathy Sigmond; Jonathan Bucki of the Dendros Group, the conference facilitator; and Patrice Legro, who was then at the Marian Koshland Science Museum. The infrastructure support provided by the staff of the National Center for Science and Civic Engagement (NCSCE) over the years was invaluable. Amanda Moodie was there for the 2011 conference. Hailey C. Chenevert, who joined the staff in early 2013 as the program assistant for SENCER-ISE, provided strong outreach to the first ten partners and general support for the initiative. Danielle Kraus Tarka, formerly Deputy Executive Director for NCSCE, provided help and encouragement. Eliza Jane Reilly, the current NCSCE Executive Director, originally served on the Advisory Board, and members of the board gave the benefit of their experience as SENCER-ISE was implemented. Eliza, along with Monica Devanas, the director of the MidAtlantic SENCER Center for Innovation, organized the Franklin & Marshall meetings that introduced Alan Friedman and David Ucko to SENCER. David Ucko and Marsha Semmel stepped in as senior advisors after Friedman’s untimely death. Both offered invaluable comments on a draft of the article (as did Chenevert), and Ucko provided updates on activities that occurred after the author “retired” from NCSCE (that is, for most of the sections on evaluation of Partnership Champions and on “Broadening the Network”). And, finally, the formal and informal science educators who led the partnerships proved willing to take a chance on a venture that was new to most of them. Their involvement and the support of the funding agencies, the National Science Foundation, the Noyce Foundation, and the Institute of Museum and Library Services allowed NCSCE to create and learn from the initiatives.
On more personal levels, in 2006, David Burns offered a “retiree” the opportunity to be part of the SENCER initiative and always provided meaningful advice, support, and, most importantly, longtime friendship. Monica Devanas has continued, ever since we met at Douglass College, to be there as a colleague and friend. Thank you, Marcy Dubroff, for your patience. And, last but not least, Marc Mappen, my husband of almost 50 years, has always supported and inspired me in my efforts and those of our two wonderful children.
The case history is written from the perspective of the author, who served first as the SENCER coordinator for the initiative and then as the director. All errors are entirely hers.
Amey, M. J., Eddy, P. L., & Ozaki, C. C. (2007). Demands for partnership and collaboration in higher education: A model. New Directions for Community Colleges, 139, 5–14.
Bell, L. (2018). Civic engagement and informal science education. Science Education and Civic Engagement: An International Journal, 10(1), 5–13.
Bell, P., Lewenstein, B., Shouse, A. W., & Feder, M. A. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: National Academies Press.
Bevan, B., & Dillon, J. (2010). Broadening views of learning: Developing educators for the 21st century through an international research partnership at the Exploratorium and King’s College London. The New Educator, 6, 167–180.
Center for the Advancement of Informal Science Education (CAISE). (2016). Informal STEM Education: Resources for Outreach, Engagement and Broader Impacts. Retrieved fromhttp://informalscience.org/sites/default/files/CAISE_Broader_Impacts_Report_2016_0.pdf
Friedman, A. J., & Mappen, E. F. (2011). SENCER-ISE: Establishing connections between formal and informal science educators to advance STEM learning through civic engagement. Science Education and Civic Engagement: An International Journal, 3(2), 11–17.
Friedman, A. J., & Mappen, E. F. (2012). Formal/informal science learning through civic engagement: Both sides of the education equation. In R.D. Sheardy & W.D. Burns (Eds.), Science education and civic engagement: The next level (pp. 133–143). Washington, DC: American Chemical Society.
Liu, X. (2009). Beyond science literacy: Science and the public. International Journal of Environmental & Science Education, 4(3), 301–311.
Phillips, M., Finkelstein, D., & Wever-Frerichs, S. (2007). School site to museum floor: How informal science institutions work with schools. International Journal of Science Education, 29(12), 1489–1507.
Rivera Maulucci, M. S., & Brotman, J. S. (2010). Teaching science in the city: Exploring linkages between teacher learning and student learning across formal and informal contexts. The New Educator, 6, 196–211.
Semmel, M., & Ucko, D. (2017). Building communities of transformation: SENCER and SENCER-ISE. Informal Learning Review, 146(Sept./Oct.), 3–7.
Ucko, D. A. (2015). SENCER synergies with informal learning. Science Education and Civic Engagement: An International Journal, 7(2), 21–24.
NCSCE Materials and Evaluation Reports
NCSCE. (2011). Conference Proceedings and Executive Summary. Retrieved through http://sencer-ise.net/background
Randi Korn & Associates (RK&A). (2011). SENCER-ISE Conference: An Evaluation. Retrieved through http://sencer-ise.net/background
Summary of Interview Responses by Objective From RK&A (September 2015)
Higher Education (HE) and Informal Science Education (ISE) professionals increased their understanding of each other’s expertise.
Several interviewees spoke about their partner’s extensive knowledge and skills. HE interviewees spoke about their ISE partner’s science communication skills, and ISE interviewees spoke about their HE partner’s research knowledge.
A few interviewees said they gained a greater understanding of the structure of higher education or informal science organizations, including the barriers or constraints their partners face.
HE and ISE professionals appreciate the values of each other’s work and expertise.
Many interviewees also said they would not have been able to accomplish project goals without their partner’s access to and knowledge of working with a particular audience, such as undergraduates or K-12 teachers and students.
Several interviewees (mostly from ISE) said they gained knowledge about the research their HE partners are conducting and an appreciation for how research can legitimize and support the work that they do.
Several interviewees spoke about their partner’s organizational context and resources as a strength (e.g., ISE praised their HE partners’ access to analytic resources; HE praised their ISE partners’ access to a real-world context).
HE and ISE professionals understand elements of durable partnerships.
Intentional goals that align with each partner’s organizational mission.
Many interviewees said that partners need to share common goals and have a passion for the project. For instance, many partners shared a common passion for environmental protection and advocacy.
Clear articulation of each partner’s roles and responsibilities.
Several interviewees talked about the importance of strategic planning at the outset of a partnership. Interviewees discussed clearly defining roles, responsibilities, and expectations.
Interviewees discussed defining these roles and responsibilities so they leverage the strengths of each partner.
Patience and flexibility to alter roles and responsibilities as conditions change.
Several interviewees talked about being open to change or course correction if a project or partnership is not achieving its original goals. Interviewees tended to speak about flexibility as a personality trait (whether someone is flexible and open-minded). However, interviewees also talked about the importance of reflection in determining whether changes are needed.
Consistent and clear communication.
Many interviewees said that establishing clear and consistent communication is paramount to a successful partnership.
Some spoke about communication as a personality trait (i.e., whether someone is a naturally good communicator); others spoke about the importance of establishing mechanisms for clear communication (phone and in-person conversations instead of email) as well as a consistent timeline (weekly, monthly, etc.).
Other important elements.
Many interviewees underscored the importance of personal relationships when establishing a successful partnership, including a foundation of shared passions and complementary working styles.
Several interviewees mentioned resources but specifically adequate resources to allow each partner to contribute the necessary amount of time to result in a successful project.
A few said partnerships need time to work out kinks and see results. These interviewees also discussed the importance of funders’ recognizing that time (at least a few years) is necessary to create a successful project.
Other HE/ISE professionals value the partnership.
Several interviewees talked about other faculty or students who became interested in collaborating with the ISE partner or in the SENCER model for their course.
A few interviewees said their project collaboration brought them recognition or credibility from other departments or individuals. In one case, this recognition brought additional funding.
This project report details a pilot venture that paired two undergraduate courses at the University of Wisconsin-Whitewater: (a) Environmental Geology, an upper-division general education science course, and (b) College Writing in English as a Second Language (ESL), a first-year composition course for international students whose second language is North American English. Students enrolled in these two courses collaborated in writing blog posts on scientific topics with societal repercussions as part of the Do Now U project, a joint initiative between the National Center for Science and Civic Engagement (NCSCE) and the education division of KQED Public Media. Collaborating in this project enabled students to use the discourse of science in authentic communication with an identified audience while conducting a group project. Evaluation shows that students enjoyed this self-directed learning experience, using digital media to communicate and to create a digital document on a scientific and social issue.
NCSCE sent a call for participation to college educators in fall semester 2016. In early January 2017, interested participants attended a webinar on project participation guidelines. Instructors also selected a date for submitting their posts during spring semester 2017. They then formed student teams, each of which proposed and decided on a topic, formulated a discussion question, and ultimately composed a blog post for the KQED Do Now U website. KQED furnished a template for blog posts, which required background information and explanation of both positive and negative implications of the topic at issue. Posts also included links to relevant videos, images, and other reliable online resources. KQED education staff selected one blog post per participating institution. Once published to the web, the posts were open for public discussion and comments.
Collaborating on a Do Now U Post at the University of Wisconsin-Whitewater
Naturally, Environmental Geology and College Writing in ESL, although both undergraduate courses, differed in several ways. The two sections of Environmental Geology, taught by Bhattacharyya, each enrolled 24 students and met twice a week in 75-minute blocks. The course follows the SENCER approach to inquiry, encouraging students to investigate unsolved problems relevant to today’s society, so that they not only develop content knowledge, but also improve critical thinking skills (Burns, 2002). Environmental Geology is a hands-on, experiential course, required for environmental science majors with an emphasis in the geosciences, but open as an elective to non-majors. Therefore, the students enrolled in the course represented a variety of academic backgrounds and interests. The course is thematically organized to inspire further exploration of topics chosen by students.
College Writing in ESL, team-taught by Huss-Lederman and Deering, enrolled 13 students and met four days a week in 75-minute blocks. The majority of the students who enroll in this course are international students, new to the United States and to university study. They represent a broad range of English proficiency and, like most first-year college students, are novice academic writers.Typically, this writing course has been organized thematically, often with human rights or social responsibility as broad topics, and so developing a semester-long environmental theme for the course was a natural fit. One goal of this composition course is to be an onramp to academic success at the university. Largely, this means providing opportunities for students to improve academic English proficiency, while simultaneously helping students to access programs that position them for success. Participating in this project enabled international students to interact with native English speakers; both groups completed an academic research project, using the SENCER approach to inquiry to enhance college-level, academic literacy in English. By the end of the project, Deering and Huss-Lederman had become advocates for the SENCER approach, continuing to develop project-based learning opportunities for their students throughout the semester even after the collaborative project ended.
In each course, the Do Now U project served a different purpose. In Environmental Geology, the assignment took on a minor role. Participation gave students the opportunity to engage in both writing to learn and writing for an audience beyond their teacher through a novel, small-stakes assignment. It also simulated an increasingly common professional situation—asynchronous collaborative writing in a medium less commonly used in a course assignment, an academic blog post to a website external to the university. Students were placed in groups based on their topic of interest, so students from both sections were required to work together, and in some cases with international students from the writing course. Students developed blog posts outside of class, but incorporated their research into class discussions. Geology students received feedback on topics along with possible questions from Bhattacharyya as comments on homework, and they were free to contact any instructor with questions concerning the posting assignments.
Since the college composition course is devoted to argumentative writing that synthesizes information from external sources, the Do Now U project took on a major role because it required international students to practice these academic skills. Reference librarians offered students a weeklong seminar in identifying and evaluating web-based resources.Students read and wrote short essays, utilizing cause and effect and problem/solution structures. Reading assignments also emphasized summarizing, paraphrasing, and identifying and interpreting quotations—all skills essential to academic writing. Generally, two international students were assigned to Do Now U project groups of two or three geology students, although international students with stronger English proficiency or a more autonomous learning style could decide not to have a composition classmate as a partner. However, for many international students, having a classmate as a partner in this project gave them confidence in the research and collaborative writing process. In fact, the international students continued to develop their English academic writing skills after this project was finished, either by continuing with their original ideas or examining a related environmental topic, which they then presented as posters during the campus Sustainability Day in April.
Although the goals of the geology and English courses were not the same and incorporated the Do Now U project differently, courses had to follow the same timeline for preparing posts. To facilitate the online writing process, instructors also assigned students roles, such as background writer, pro argument or con argument writer, editor, and media finder. Three common collaborative face-to-face sessions were held for students to complete the post together. Ultimately, UW-Whitewater submitted 16 blog posts for consideration. On March 15, 2017 the entry, “Do the Benefits of Aquaculture Outweigh Its Negative Impacts?” was posted.
Evaluating the Project
An online evaluation with questions targeted to each course was sent to all students in March, 2017. There was nearly a 100% response rate by geology students. Thirteen students were enrolled in English 162 when the project started, but only eleven completed the course, and six completed the survey. The findings are summarized below.
In the environmental geology course, collaborating on a blog post for a public media outlet was a novel experience, from determining a topic and refining a discussion question to writing a backgrounder that included links to further information.
95% indicated that they had learned something new about an environmental topic that they had chosen and researched themselves, with some commenting that they had come to understand new perspectives and to identify their own biases.
Many students indicated that working in a group offered them new perspectives on how to work with others; those who worked with international students appreciated the opportunity to do so.
Students enjoyed working with multimedia resources and developing a blog post, as opposed to writing a traditional research paper.
Some students found group work to be frustrating when group members did not contribute to the team effort.
Collaborating to write a blog post for a public media outlet was also a novel experience for the international students. The emphasis in this assignment, as well as in others in the course, was to develop and strengthen collegiate writing proficiency in English. Students were asked to reflect on their development.
On a scale of “not confident” to “very confident,” international students were asked to reflect on their growth as academic writers in English. All students indicated that they felt “somewhat” to “very confident” in their ability to locate appropriate academic resources and to evaluate their reliability.
On a scale of “not confident” to “very confident,” students indicated that they felt confident providing academic summaries of resources and preparing counterarguments.
All students reported that their academic vocabulary had improved.
None of the students indicated disappointment if their team’s work was not chosen for publication. Overall, the experience was positive for students enrolled in both courses.
What the Instructors Learned
This pilot was the first time that these three instructors collaborated on a public writing project, let alone one that paired upper-level students with novice academic writers who communicate through ESL. Observations of students throughout the project, as well as student survey results, led to the following conclusions:
Using the template provided by KQED and reviewing past posts to understand how to complete the assignment from the beginning focused the writing process for all students and made assigning writing roles to students easier. Furthermore, the template’s structural guidelines freed students to focus on refining their questions and finding relevant resources instead of wondering how to organize the information.
Making the theme of the English course environmental sustainability and registering for a blog posting date mid-semester gave the first-year international students time to build background knowledge in order to be strong partners to the geology students. All students ultimately shared common content knowledge, which leveled the playing field for the assignment.
Assigning international students to write the negative position on a topic helped them to conceptualize counterarguments, an important skill in argumentative writing.
Geology students in groups with international students enjoyed the opportunity to meet and work with students from other countries.
All students appreciated the chance to share information with a broader audience outside of their courses.
Although many students liked building a document by communicating online, they also appreciated the face-to-face work. Face-to-face meeting in the university library allowed all students to review work together.
Changes for Future Projects
Overall this pilot worked well; however, certain modifications would improve the structure of future collaborative writing projects. For example, scheduling the English course and the geology courses at the same time of day would allow for more convenient face-to-face collaboration among all students as a learning community. Although most students enjoyed this assignment, some were frustrated when not all group members pulled their weight. Because this also happens in the workplace, students need to know how to manage such situations and how to take responsibility for their specific roles on a team project. Restructuring the course assignments to emphasize individual accountability to the group would help students to develop this skill. Students would benefit from reflecting on the experience of working in groups and learning how individual actions affect the team.
Both collaboration and open-ended research-based projects are high-impact practices (HIPs), noted for promoting strong learning outcomes in higher education that translate to participation in a globalizing society (Kuh, 2008). Indeed, an analysis by Kilgo, Sheets, and Pascarella on the effectiveness of HIPs on the goals of liberal arts education indicates that these two practices are “. . . significant, positive predictors for a variety of liberal arts learning outcomes” (2015, p. 522). Students participating in the Do Now U project worked together to research issues in which society affects the environment. Such learning practices fall within the domains of cognitive and interpersonal competence, integral to 21st-century skills (National Research Council, 2012).Project-based learning is also a natural fit in the SENCER paradigm, as it promotes student-centered, self-directed, deep examination of issues.
Additionally, students participating in groups composed of both U.S. and international students experienced working with individuals from a culture other than their own, an important component of intercultural competence (Kuh, 2008). Although students enrolled in Environmental Geology would have been able to carry out this project on their own, sharing the project with first-year international students enabled all students to improve intercultural competence within an international academic community. The ability to work as a team, not only face-to-face but also online, is an important competency in the global workforce (Moore, 2016).
In the English course, working with unsimplified, authentic texts and communicating with native speakers in English allowed students to conduct research and to write for a specific purpose and audience far beyond their ESL class. Such practice helped them to focus on the intellectual purpose of researched writing rather than on the mechanical aspects of citation and reference, which, although important, should not occupy the forefront of writing to learn (Howard and Jamieson, 2014). Collaborating with students in the geology course on this project required ESL students to become knowledgeable about an environmental concern and to communicate with others using both academically and socially appropriate language in speech and writing. Furthermore, project-based learning naturally promotes the use and development of the four language skills (speaking, reading, writing, and listening) and subskills (vocabulary, grammar, and pronunciation) in an integrated way and fosters learner autonomy (Beckett and Slater, 2005). The sustained opportunity to use academic language beyond the English composition classroom in a scientific theme put these international students on track for academic language development and learning that would serve them in courses beyond this one. Such educational practices may become increasingly important as the number of ESL students enrolled in English-medium institutions of higher education around the world grows (Fenton-Smith, Humphreys, Walkinshaw, Michael, and Lobo, 2017).
For the geology students, the experience of asynchronous, collaborative writing was a gateway into an increasingly common mode of professional communication in both academia and the workplace. Students were also placed in the novel situation of sharing information that they had learned independently with a wider audience. Although the project was a low-stakes assignment in terms of the effect on the course grade, students engaged in several HIPs—collaborative group work, working across cultures, and a writing-intensive assignment, while engaging in self-identified, open-ended questions where science and social responsibility came together.
A SENCER course in the sciences is different from a composition course that uses science topics as a springboard to academic writing, yet the opportunity to communicate about science can reach beyond science courses. Collaborating on Do Now U demonstrated how this type of bridge worked—bringing group writing to a science course and introducing SENCER practices into a composition course for international students. Further, it exemplifies how collaboration between the humanities and natural sciences, using a SENCER approach, benefitted students at different stages of university education.
Special thanks to Andrea Aust, Director of Science Education at KQED Public Media, and her team for guidance and editing support for our students, and to the anonymous reviewers of this manuscript for helpful suggestions.
About the Authors
Prajukti (Juk) Bhattacharyya is a Professor in the Department of Geography, Geology, and Environmental Science at University of Wisconsin-Whitewater. She received her PhD from the University of Minnesota in 2000.Her background is in Hard Rock Geology and Geoscience Education.She teaches courses on volcanoes, structural geology, rocks and minerals, plate tectonics, and environmental geology.Her research interests range from geochemical analyses of igneous and metamorphic rocks to volcanic activities.She is also involved in STEM education research, especially on ways to help students learn and on the assessment of student learning.
Susan Huss-Lederman is Professor of Applied Linguistics and Teaching English as a Second Language (ESL) in the Department of Languages and Literatures at the University of Wisconsin-Whitewater, where she has taught since 1995.Susan has taught ESL for 30 years and has expertise in professional development of pre-service and practicing teachers, as well as in ESL curriculum development. For 13 years, Susan co-directed several federally funded professional development projects for teachers of English language learners in Wisconsin. She has also trained English teachers in Mexico and Ecuador. Currently, under the auspices of the Galápagos Conservancy and the Scalesia Foundation, Susan leads a team of educators offering ongoing professional development in English education for sustainability in the schools of the Galápagos. In 2016, Susan received a Teaching Excellence Award given by the University of Wisconsin System Board of Regents.
Brianna Deering is a Lecturer in the English Language Academy at the University of Wisconsin-Whitewater.Educating students has been her passion for the last 25 years. She began her teaching career in elementary education and transitioned to adult education, with the last five years being at the college level. She has taught a variety of ESL courses, from introductory to advanced English, as well as English for business communication and the healthcare system.She has organized conversation groups, service learning projects, and community outreach programs as ways to expand the cultural knowledge of her international students.
Beckett, G., & Slater, T. (2006). The project framework: A tool for language, content, and skills integration. ELT Journal, 59(2), 108–166. doi: 10.1093/eltj/cci024
Burns, W. (2002). Knowledge to make our democracy. Liberal Education, 88(4), 20–27.
Fenton-Smith, B., Humphreys, P., Walkinshaw, I., Michael, R., & Lobo, A.(2017). Implementing a university-wide credit-bearing English language enhancement programme: Issues emerging from practice. Studies in Higher Education, 42(3), 463–479. doi: 10.1080/03075079.2015.1052736
Howard, R. M., & Jamieson, S. (2014). Researched writing. A Guide to Composition Pedagogies (2nd Edition), 231–247.
Kilgo, C. A., Sheets, J. K. E., & Pascarella, E. T. (2015). The link between high-impact practices and student learning: Some longitudinal evidence. Higher Education, 69(4), 509–525. doi:10.1007/s10734-014-9788-z
Kuh, G. (2008). High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities.
Moore, C. (2016). The future of work: What Google shows us about the present and future of online collaboration. Techtrends: Linking Research and Practice to Improve Learning, 60(3), 233–244. doi:10.1007/s11528-016-0044-5
National Research Council. (2012). Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Washington, DC: The National Academies Press. https://doi.org/10.17226/13398
American Psychological Association (APA). (2018). Civic engagement. Retrieved February 2, 2018 from
Center for Civic Engagement (CCE). (2014). What are civic life, politics, and government? In National standards for Civics and Government, 5–8 content standards. Retrieved February 2, 2018 from http://www.civiced.org/standards?page=58erica.
McCallie, E., Bell, L., Lohwater, T., Falk, J. H., Lehr, J. L., Lewenstein, B. V., Needham, C., & Wiehe, B. (2009). Many experts, many audiences: public engagement with science and informal science education. Washington, DC: Center for Advancement of Informal Science Education (CAISE).
Ostman, R. (2006). STEM community partnerships and organizational change: Testing a scalable model to engage underrepresented children and families. Proposal to National Science Foundation from the Science Museum of Minnesota.
A greenhouse program in a community garden in Brooklyn, New York, is developed for year-round urban farming. The program exercises technical skills to design and build the greenhouse, and also exercises community democracy skills to address interpersonal issues such as land usage in over-crowded spaces and volunteer organization operations. We describe here the planning and construction of the greenhouse and also the process of community group discussion, debate, and voting in a volunteer run community garden.
The urban environment of New York City (NYC) offers an endless supply of sensory and cultural experiences, but it does not offer much by way of open green spaces, and even less access to healthy, locally sourced food. Community gardens are green spaces in which the residents enjoy, steward, and cultivate a small plot of soil in the city. There are more than 900 community gardens across the five boroughs (Design for Public Space 2014), each one with a unique governance and farming mission. Organic farming for food production and education is vital, especially in urban environments where the availability and desire for whole food based diets are rare.
The community garden discussed in this report is located in Northern Brooklyn and occupies the land of three adjoining building lots. The garden has nearly one hundred members, operates a public compost collection system, and has over 1300 square feet of organic vegetable growing space. Until recently, the winter all but stopped our farming activities except for the use of small cold frames to grow greens and seedlings through the colder months. The next step in the garden’s mission to grow food and educate the community was to establish a year-round gardening program in a greenhouse. This project report describes the obvious and non-obvious parts of the project that were important to ensure a successful outcome, including grant writing, technical design and construction, and, most importantly, community democracy.
The greenhouse development was funded by a generous grant from Citizens Committee of New York City. The grant mission statement was to develop a year-round farming space so that seedlings could be grown in the early spring for farm use and public sale, and to offer an educational and public laboratory space for anyone interested in greenhouse growing. The grant was written by three garden members during the winter of 2016 and notice of the $2300 award was given in the spring of 2017.
It is becoming increasingly important, especially in NYC, to justify the use of land space and grant money. There are many groups developing new metrics to understand and measure the impact of their community projects (Design for Public Space 2014). The metrics to measure the outcomes of the greenhouse are
1. Count of seedlings grown that are distributed to the farm
2. Revenue from greenhouse-grown seedlings at public plant sales
3. Record of crop yields from greenhouse-grown plants
4. Record of events and number of garden members working in the greenhouse.
The grant application included a proposed location of the greenhouse with adequate sun in the winter months, since a greenhouse relies on the sun for passive heating. From an aesthetic viewpoint, it is important to place the greenhouse in a position that does not obtrude on the visual experience of the garden. To accommodate these requirements, a south-facing space was chosen on the edge of the farm area, which is visually buffered by surrounding trees to the north. The greenhouse construction must also follow all zoning laws. This type of greenhouse would be considered a noncommercial greenhouse (Rules of the City of New York).In addition, the construction must follow building codes, including the roof loads for snow (Department of Buildings, New York City).
The average price per square foot of Brooklyn real estate is approximately $750 (www.trulia.com). This expense creates a huge pressure on the utilization of open spaces. Allocating eight square feet (worth approximately $48,000) for a greenhouse is thus a difficult decision. Even though the dollar value is not an actual cost, it does reflect the challenges confronted when proposing to use shared open space.
Our community garden is a democratic organization comprised of community volunteers, and the deliberations to build the greenhouse presented a very valuable and in-depth exercise of community democracy. The ages of the participants ranged from children to senior citizens, and the team was comprised of architects, scientists, lawyers, artists, teachers, and corporate workers with varying skill levels specific to greenhouse construction. Some members supported the construction of the greenhouse, whereas other members were opposed to the project. Ideally, a rational and scientific approach can be a valuable strategy for moving forward while acknowledging the input of all members.
The primary question to address was whether or not to add an additional structure in the garden, because the surrounding urban environment is made of human made structures with small amounts of green space. To address this concern, the design of the greenhouse was modified to minimize the total vertical height by making a gable roof instead of a simpler shed roof.A slope is needed for snow and rain runoff, and an angled roof also provides increased light transmission. Additionally, we noted that a Spiraea shrub on the east side and overarching trees on the north of the greenhouse will visually buffer the structure in the summer months. Garden members stressed that a greenhouse structure is visually transparent, and that it is also a natural garden structure with visual vegetation inside.
Aside from the overall visual design of the garden space, we needed to consider sunlight exposure of the greenhouse and the shadows that it casts. A suggestion was made to place the greenhouse in a corner of the garden, but it was not clear how much sunshine the greenhouse would receive during the winter. The greenhouse requires direct sunlight in the winter months, so a suitable location must be far from tall fences or neighboring buildings. The sun’s angle in the winter sky was an important detail to consider when locating the greenhouse. Areas receiving sun in the summer or fall months may not be illuminated in the winter due to neighboring buildings. To address these questions, a sun study was performed to determine the shadows cast by neighboring buildings in the winter months. The results of this study showed that the greenhouse would be in the winter shade if it were located in the back corner of the garden, because of the adjacent buildings and fences. It was also questioned if the greenhouse itself would cast shade on any plants behind the structure. However, this issue is not a serious concern, because the greenhouse is constructed with transparent polycarbonate panels that are 80% transmissive, which means that 64% of incident light can pass through two walls to the plants behind the structure. The final site was chosen as far from southern buildings as possible, and in a position with trees behind so that it would not cast shade on small plants.
Another concern raised was the potential effects of a non-natural structure on pollinating insects. This is a very important issue, because pollinating insects are critical to the natural cycles of a plant ecosystem.We were fortunate that our grant coordinator from Citizens Committee had firsthand knowledge about pollinating insects in urban environments, and she informed us that pollinating insects navigate by sunlight, shade patterns, and color. The transparent panels are expected to have minimal effect on their natural pollinating courses in the warmer months.
Finally, since a greenhouse creates an ideal environment for the growth of plants, it is also conducive to the growth of fungi, pests, and plant pathogens. The interior of the greenhouse remains constantly moist and stays warm. Without electrical fans, the air is stagnant and promotes fungal and bacterial growth.A modern technology solution to this problem is temperature activated vents that mitigate the problem of overheating and can provide air current channels through the structure. These automatic vents do not require electricity and are passively operated by temperature-sensitive wax-filled pistons attached to the windows.It is also necessary to remove any dead plant material as soon as possible to minimize fungal growth. In addition, there are several organic essential oils such as neem, cedar, and citrus that are being tried as fungal deterrents. It is important to address this issue because a disease or pest that grows in the greenhouse might spread into the farm. The community farm is crowded, just like the rest of the city, so plant or airborne diseases and pests can spread quickly. It is critical that the greenhouse be operated with the best scientific practices possible to ensure the well-being of the rest of the communal farm space.
There were three meetings of the general membership, each lasting an hour, to discuss the greenhouse. The garden organization has chosen to operate with a loose interpretation of Robert’s Rules of Order. At the second meeting of discussions, a motion was made to implement the greenhouse.Among the 26 members present, the votes cast were 13 ayes, 10 nays, and 3 abstentions.According to our implementation of Robert’s Rules, any decision is based on the majority of voters present and not on a simple majority of votes. Consequently, the motion did not pass because 14 aye votes were requited for a majority of voters present (abstention votes act as a nay when a majority is defined in this way). The close count of the vote prompted advocates of the greenhouse to propose a revised plan that was scaled down in size as a concession to the opposition concerned with land usage. A new motion was presented the following month and the votes cast were 17 aye and 10 nay with no abstentions. This vote passed the motion so that the greenhouse project could be implemented.
Splitting a community is problematic, both emotionally and politically. Most projects in these types of organizations are of smaller scale with smaller impact, and they move forward with near unanimous support. Overall, the fundamental challenge is to separate the science-based concerns versus emotional concerns and address each appropriately. Emotional resistance can sometimes be overcome by providing a scientific explanation. In other cases, science-based criticisms can lead to very constructive discussions; we can use science to support our ideas but must acknowledge that science can also oppose them. For example, some who were opposed to the project identified specific plant pathogens and microclimate issues that occur in a greenhouse, and this was one of the most important issues to address.Also, the concern to minimize the visual impact while maximizing sunlight exposure led us to a very informative sun study of our garden. This respect for science and rational discussion is critical in our current society, and forward progress can be made by focusing on tangible and rational methods.
All the work described above generated an 8-ft square greenhouse. The future work requires designing the interior space to be most space efficient and to the liking of the members. Initial ideas are to run multiple levels of shelving around the walls to maintain the maximum possible floor space for mobility. However, plants along the south-facing wall will block the sun, and so the density of shelves and plants on the south wall should be carefully considered. An irrigation system is being planned that will take roof runoff into gutters that feed directly into drip irrigation for plants in the greenhouse. The greenhouse will require regular maintenance throughout the year to keep plants watered and to deter infections. Other programs in the garden have been successful in sustaining a group of dedicated workers and a publicly available sign-up schedule, and we hope to replicate the successful model already in place in our garden. Also in progress is a process to plan and coordinate volunteer work. We intend to use the space for projects, instead of allocating space to individual members.as is the case in the rest of the garden.We hope that this will be a more equitable method of sharing the space.
An 8-ft square polycarbonate greenhouse was constructed in a community garden in Brooklyn, NY. This process was completely developed and executed by community volunteers. We have detailed the democratic discussions and scientific arguments needed to move forward through a system of community democracy to achieve success. We found that discussions among a large group of emotionally invested community members can be navigated by applying specific scientific principles in a democratic and objective manner. We hope that this project report can be of use to other community groups looking to undertake complex projects in a diverse community.
The author wishes to thank Citizens Committee for New York City for the generous grant and the entire garden membership of Prospect Heights Community Farm for working through this complex project to a successful completion.
About the Author
Jeff Secor has been a resident of Brooklyn for 10 years and a member of PHCF for nine of those years. He was a freelance gardener around Brooklyn during his graduate studies at the City College of New York. He holds a Ph.D. in physics from CUNY with a specialty in spectroscopy, photosynthesis, and carbon quantum dots. He currently teaches physics at a private school in New York City and teaches workshops on winter gardening structures such as cold frames and greenhouses.
Department of Buildings, New York City. Building Code, Loads, Title 27, Subchapter 9.
Design Trust for Public Space (2014). Five Borough Farm II: Growing the Benefits of Urban Agriculture in New York City.
Rules of the City of New York. Noncommercial Greenhouses Accessory to Residential Uses as a Permitted Obstruction in Required Rear Yards or Rear Yard Equivalents, Chapter 23-0.
Construction Details for the Greenhouse
The materials for constructing the greenhouse are listed in Table 1. The greenhouse framing material was chosen to be cedar wood since it is an excellent exterior wood for greenhouse framing. It lasts through years of weather exposure and acts as its own insect repellent. Cedar wood is also locally available and within the budget of the greenhouse. The transparent covering is made of 6 mm-thick twin wall polycarbonate (PC) greenhouse panels. PC greenhouse panels are a relatively new material. The insulating R value of 1.54 for polycarbonate compares very well to the R value of 1.72 for a ¼-in. spaced double pane window. It is lightweight (a few pounds per 4 ft ×8 ft panel) and has no risk of breaking into sharp pieces as glass could. It should be noted that the PC panels have a slight blurring effect and are not as visually clear as glass. The PC panels are specified to pass 80% of the sun spectrum that is useful for photosynthesis (400–700 nm).
Local building codes were consulted to ensure compliance with applicable laws. The building codes in NYC are available online through the Department of Buildings. In NYC, this type of greenhouse would be considered a noncommercial greenhouse (Rules of the City of New York). This ordinance requires that the greenhouse be more 3 ft from the lot line. The roof was designed to conform to roof load specifications of 30 lb per square foot of horizontal extent (Department of Buildings, New York City). In general, the square foot of horizontal extent is 1 square foot multiplied by the cosine of the roof pitch. Finally, the PC manufacturer’s specifications determined the required roof framing spacing to support the necessary roof load and resulted in roof purlins spaced 24 in. apart.
The greenhouse will be a warm and moist space in the winter, and the surrounding urban environment contains rodents. Galvanized wire mesh should be placed on the subground as a barrier to prevent rodents burrowing into the greenhouse. During the summer the greenhouse can easily rise above 100 °F. The windows for the greenhouse are fitted with automatic wax hinges which actuate according to the interior temperature to prevent excessive heating and promote air circulation in the warmer months. Two vents are placed on the roof panels, and one vent is placed closer to the ground to achieve a chimney effect.
The greenhouse construction was completed in three phases: (a) site preparation, (b) framing construction, and (c) installation of the PC panels. Site preparation is the most physically intensive phase. The existing plants and garden soil were removed in order to level the foundation soil and to make room for the 6 in. x 6 in. foundation timbers. The area was compacted with a 10-in. hand tamper. We chose not to pour a concrete foundation in order to minimize the impact on the natural area and to minimize the eventual work of removing the greenhouse. Once the timbers were leveled in an 8 ft x 8 ft square arrangement, they were bolted together in the corners with 10-in. galvanized lag bolts, and each timber was anchored in place with two rebar “L” shapes inserted 3 ft below ground level. This part of the project took approximately three days over two weekends.
The second phase was constructing the framing. The wall panels were built first using 3-in. coated decking screws. A group of a dozen members, including a 12-year-old boy, assembled the wall panels, thereby gaining first-hand experience with framing squares, drill bits, circular saws, and with creating a level work space in a community garden. Afterwards, another group of members templated the roof boards using a speed square and a circular saw. In order to provide additional support, stainless steel rafter ties connect the wall framing to the roof boards. (Stainless steel does not interact with cedar wood.) The frame was attached to the foundation using 4½-inch stainless steel screws and washers. The entirety of the framing work required five days over three weekends.
Finally, the double walled PC panels were installed. The PC panels can be cut by an electric circular saw.A saw blade with fine teeth must be used when cutting the PC to prevent plastic shrapnel and rough edges. The tops of the PC were sealed with metal foil tape to prevent water from entering the channels. The PC panels were attached directly to the cedar framing using 1 ½- in. dip coated screws with 1-in. neoprene washers. The neoprene washers are common applications where a soft washer is needed in order to prevent cracks and punctures in the panels. It is important not to use galvanized screws as they will cause rust bleeding with the cedar. The framing geometry is made so that all of the panels end on a cedar framing stud. This makes for a more stable structure and also reduces thermal leakage. A door was cut from one of the wall panels and hung on zinc plated hinges. The hinges were installed on the outside of the panel, not in contact with the framing, so there is no danger of galvanic interaction between zinc and cedar.
The targeting of elementary school students early in their education with exposure to the different Science, Technology, Engineering and Mathematics (STEM) fields will provide them future access to college offerings and career possibilities. Faculty and students from New York City College of Technology worked with young students at a local elementary school, creating and implementing programs that will help to strengthen the nation’s STEM workforce and to prepare students to be productive citizens with a strong sense of self.
The New York City College of Technology (informally known as “City Tech”) partnership with P.S. 307 Daniel Hale Williams School began in 2014. The partnership aimed to promote A Better Educated City; an investment in STEM, and our nation’s future.New York City College of Technology is part of the City University of New York (CUNY) system. Daniel Hale Williams is an elementary school serving students in Pre-K through Grade 5, which became a science and technology-themed magnet school for STEM Studies after being a recipient of a grant from the federal Magnet Schools Assistance Program. For the 2017-2018 academic year, 373 students are enrolled at Daniel Hale, where 57% are male students and 43% female students.The race/ethnicity reported by the school includes a 56% Black and 27% Hispanic student population. With a similar male to female ratio of undergraduate students, City Tech reports 30% Black and 33% Hispanic (New York City College of Technology 2017).The large underrepresented population at both schools made the partnership an ideal fit.Initially, college students were hired as interns through the CUNY Service Corps program. The CUNY Service Corps organizes students and faculty across the institution to work on projects that benefit the residents and communities of New York City.These projects aim to advance the “civic, economic and environmental sustainability” of the city (City University of New York [CUNY] 2018). At the core of the Service Corps, launched in 2013 as a response to Hurricane Sandy, is civic-engagement, which aligns with the values of SENCER. Students are paid as interns to work in civic-related jobs in community organizations (CUNY 2018). During the 2014–2015 academic year, two CUNY students worked to develop and implement an Educational Outreach Program that provided students in grades 1–5 with exposure to Science, Technology, Engineering, and Mathematics (STEM) in their elementary school classrooms. To sustain the program beyond the 2014–2015 academic year, the Black Male Initiative, Emerging Scholars, and Perkins Peer Advisement programs at City Tech continued to support the outreach project. Since the program’s inception, a number of City Tech undergraduate students have served as mentors to the elementary school students and have worked with faculty at City Tech and key staff at the local elementary school. The goal of this collaboration, which has spanned a number of years, was to engage college students, elementary school students, college faculty, elementary teachers, and the families of the elementary students in a STEM outreach initiative.
Why is it important to integrate STEM education into the elementary school curriculum?
Many recent studies indicate that the gap in the STEM workforce will continue to widen unless more students decide to enter the STEM fields (Brophy et al. 2008; Brown 2012; Johnson 2013). According to the U.S. Department of Commerce, STEM occupations are growing at 17%, while others are growing at 9.8% (Langdon et al. 2011). To succeed in society today, we should encourage students to solve problems, develop their capabilities in STEM, and become tomorrow’s scientists, inventors, and leaders (Science Pioneers 2017). Exposure to STEM careers at the elementary school level enhances student learning, encourages creativity, and entices curiosity. The National Academy of Engineering and the National Research Council list some benefits of incorporating engineering in K–12 schools: improved achievement in mathematics and science, increased awareness of engineering, understanding and being able to do engineering design, and increased technological literacy (Katehi, Pearson, & Feder 2009). With these studies as a rationale, we developed a multitier approach to integrate STEM into a Pre-K–5 (elementary) school.
The awareness of STEM-related careers was presented to the participating staff, students and families through in-class lesson plans, afterschool programs, and family workshops. Most of the projects centered on science and civil engineering to draw from the strength of the faculty involved.The engineering design process was included in the activities.Students were encouraged to (a) identify the problem, (b) brainstorm solutions, (c) try a design, (d) test, (e) identify strengths and weaknesses, and (f) try again.In order to promote skills associated with a well-rounded scientist and engineer, the activities integrated concepts of cost, schedule, and communication. The majority of the activities (in-class lessons, afterschool program and family workshops) were held at the local elementary school.College students and faculty met and communicated regularly with the staff at the elementary school to plan all activities.We present below the project design of this multitier approach to the community.
The in-class lessons centered on the NYC Scope and Sequence for Science and the Next Generation Science Standards (NGSS).The science focus included the following two topics: The Five Dancing Spheres (biosphere, lithosphere, geosphere, cryosphere, and hydrosphere) and Weathering and Erosion. In each unit, students in grades 3 and 4 explored these science fields and created models to represent and display their learning.The civil engineering focus included the following in-class lesson topics: What is Engineering, Types of Engineering, Structures and Functions, Teams behind Construction, Construction Drawings, and Sustainability.The goal of the in-class lessons was to enhance the existing science curriculum with real-world applications and hands-on projects to help the students better understand the science curriculum. The commitment and participation of teachers from the elementary school were critical to the success of the program.The teachers and undergraduate students met regularly to plan, reflect, and ensure a smooth link between the NGSS curriculum and the in-class lesson topics.The teachers provided insight on teaching techniques for elementary school-age children and diverse learning styles.The undergraduate students worked closely with the teachers and tailored their lessons and activities to the children in the classroom.
The lesson plans for The Five Dancing Spheres curriculum (Figures 1 and 2) at the elementary school is only one example of the approach that we implemented.Each lesson included a visual aspect (examples), vocabulary activity, homework, and a hands-on activity.
The afterschool programs reflected the model used in two local design competitions: West Point Bridge Design and Future City. These competitions are aimed at middle school students to promote interest in civil engineering careers.These projects required students to model the Engineering Design Process. Students used software programs to design their projects, create physical models, and prepare oral presentations.Even though students did not participate in the competitions, they were encouraged to be problem solvers and engineers.Students were encouraged to design, test, and revise their ideas. This provided a great opportunity for students to use their math, science, and technology skills while working with the engineering design process to come up with various solutions.
Engineering concepts such as force and equilibrium were incorporated through the Bridge Design project.Students used the Bridge Design software to design their bridges and simulate the testing of the bridge. Bridge Designer is a zero-cost educational software intended to provide middle school and high school students with a real-world overview of engineering through the design of a steel highway bridge (Ressler 2013).
These elementary students were introduced to concepts of tensile and compressive force.Students created a virtual bridge and a
replica model of their virtual bridge usingbalsa wood (Figure 3). Each material had a cost assigned to it, and students worked to make the strongest and most affordable bridge.
Similarly, concepts such as city planning and sustainable design were taught through the city design project. Future City is a project-based learning program where students in 6th, 7th, and 8th grade imagine, research, design, and build cities of the future (National Engineers Week Future City Competition 2017). Our afterschool partnership brought this project to the elementary students at P.S. 307, and they successfully created their own virtual city using the Sim City software.Students made blueprints of their cities and created a replica model showing a block of their cities using all recyclable materials. In preparing a blueprint, students visualize and sketch their design. Transferring the design from paper to three dimensions helped the students make a connection from 2-D to 3-D, promoting spatial thinking.Spatial thinking has been identified as an important trait for STEM careers (Wai, Lubinski, & Benbow 2009).“Fostering spatial thinking and mathematics learning in elementary school could contribute to a downstream ripple effect, improving students’ interest and success in STEM subjects throughout their education and into their careers” (Burte et al. 2017).
The process of calculating total cost introduced the idea of budgets and the importance of adhering to a budget. Students also had to adhere to a schedule, as they were limited in the amount of time they could work on each portion of the project. Students presented their projects at the end of each program.
Family STEM Workshops
Recognizing the importance of family involvement in a child’s success, the program included interactive STEM workshops and field trips for families that increased their awareness of STEM-related careers. Survey and program assessment data informed planning for the next project year.Topics in the family STEM workshops included, but were not limited to Civil Engineering, Chemistry, Mechanical Engineering, Architectural Engineering, and Computer Systems Technology.One local field trip included the SONY Wonder Technology Lab in New York City.
Some of the activities that were introduced at the workshops were (a) Spooky Materials Testing experiment which included a Mechanical Engineering focus; (b) building a home for turkeys with a Civil Engineering focus; (c) dissolving M&Ms and making slime with Chemistry; (d) learning coding with puzzles with a Computer Engineering focus; and (e) the design and creation of an architectural building model with Architectural Engineering as the focus.
The Spooky Materials Testing experiment (Schooling a Monkey 2018) introduced stress concepts to the elementary students by applying the different types of stresses (tensile, compressive, shear) to different types of candy and comparing the results of the tests on each candy. Students then made connections as to which type of candy, based on the stress concept, would be best for building.
Building a home for a turkey (Preschool STEAM n.d.) introduced the structural concepts and material cost to the students. The goal was to contain the holiday turkeys in a structurally sound and cost-efficient space. There were time limits and cost constraints that the students had to comply with. Students were also given a range of materials, each with a certain cost assigned.
Dissolving M&Ms (American Chemical Society 2018) and making slime (STEAM Powered Family 2018) introduced the concept of chemical experimentation and observation. In both activities, students were able to combine substances and observe the outcomes, which were colorful, fun, and thought provoking. With the help of parents, the students poured rubbing alcohol, water, and oil onto a plate of M&Ms and saw the dissolving effects the different solutions had on the M&Ms.The slime-making activity reinforced the concept of how observations are important in chemical processes.
Learning coding with puzzles introduced the algorithmic concept of coding patterns to the students (Institute of Electrical and Electronics Engineers 2018). This was accomplished through a brief introduction of how to follow steps using “coding language” and a visual puzzle activity that involved critical thinking. The students were then encouraged to “walk out” their coded steps on a large grid that closely followed the worksheet they worked on. As a next step, students and their families applied the skills they had learned to the online software in code.org.
By designing and creating an architectural building model, students were able to see the problem-solving and aesthetic skills it takes to become an architect. Students were given a laser-cut bendable paper set to create 3D models of their structure. Each student received the same pieces, but each individual was able to create entirely different structures by arranging the structure to their liking.
Results and Discussion
The faculty at New York City College of Technology recruited undergraduate students enrolled in the departments of Biological Sciences, Chemistry, and Civil Engineering Technology to serve as mentors, which included a pool of about 750 students. Throughout the years, several programs have provided support to the college students involved in this endeavor.These included the CUNY Service Corps, Emerging Scholars, Perkins Peer Advisement, and the Black Male Initiative programs, all of which have recognized the value of the STEM Outreach program. The success of the partnership and the collaboration of college faculty and students at City Tech has opened the eyes, minds, and future career potential of the elementary students at P.S. 307 Daniel Hale Williams School. It reinforced the need for STEM education in underrepresented learners. The partnership has increased exposure at the elementary school to STEM topics and courses taught at the college level.The outcomes as shown have been favorable and shared with the community at large via showcase presentations, school displays, and conference presentations, and at the college’s annual poster session.
Our success included presenting activities seen as academically challenging (geared only to junior high, high school, or college students) to the elementary school students at P.S. 307, in a way that led to both success and enjoyment for the students. Furthermore, these students were able to figure out what STEM topics they enjoyed by trying many different discipline-oriented workshops. By including the parents in our workshops, we were able to inform them about various fields of engineering, next step school options for their elementary child, and career opportunities.Elementary school students were able to successfully implement the information they were learning through interactive hands-on STEM activities.
Impact on Undergraduate Students
There is a large body of evidence of the positive impact of undergraduate research on college students (Lopatto 2010; Russell, Hancock, & McCullough 2007).George Kuh (2008) also points to high-impact practices such as engagement beyond classroom (internships) and community-based learning that promote student engagement.The STEM outreach that we have described demonstrates that working with community partners such as the elementary school represents a valuable community-based project.The CUNY Service Corps indicate that undergraduates gain “workplace skills and abilities; personal development; civic engagement and social issues awareness” (CUNY 2017).The undergraduate students developed the curriculum under the guidance of the faculty and elementary school teachers.Additionally, the students gained valuable experience for the real world, including organization and communication and presentation skills.
This work brings to the forefront a collaboration that engaged faculty, undergraduates and elementary school students and teachers in a STEM outreach project.The project, which aimed to promote A Better Educated City, has increased awareness of STEM careers among families at the elementary school. Students were engaged in hands-on activities while learning elementary concepts related to STEM. Exposing elementary school students to science and engineering concepts can motivate them to solve various problems more effectively. “Quality STEM education is vital for the future success of students. Integrated STEM education is one way to make learning more connected and relevant for students” (Stohlmann, Moore, & Roehrig 2012, 28). Engineering is traditionally not a subject that is taught in elementary schools. However, it is a powerful method of teaching and motivating students in STEM-related fields. “Research indicates that using an interdisciplinary or integrated curriculum provides opportunities for more relevant, less fragmented, and more stimulating experiences for learners” (Furner & Kumar 2007, 186).Adding science, and more importantly, engineering as a part of the elementary school curriculum can be an effective way for students to strengthen their science, mathematics, and technological skills.
Professors Samaroo and Villatoro thank the following programs for supporting the various undergraduate students involved in this project over the years: Perkins Peer Advisement, Black Male Initiative and Emerging Scholars programs at New York City College of Technology, and the CUNY Service Corps.The authors thank the principals and teachers at Daniel Hale Williams School for opening their classrooms to this project throughout the years. We also acknowledge the faculty from the City Tech who participated in the Family STEM Workshops and the following undergraduates who have contributed to this project: Ramon Romero, Ngima Sherpa, Joyce Tam, Abigail Doris, Dante Francis and Jesam Usani.
About the Authors
Areeba Iqbal earned her Associate in Applied Science in Civil Engineering from New York City College of Technology.She is currently pursuing a Bachelor of Science in Civil Engineering from
Kayla Natal is currently a student at New York City College of Technology, pursuing a Bachelor’s degree in Mechanical Engineering.She also works as a Coordinator for the Peer Advisement Program. Kayla hopes to further her education and pursue a career in Industrial Design.
Servena Narine is a licensed and certified NYC Board of Education teacher. She currently works at Daniel Hale Williams Public School 307 Magnet School for STEM Studies. She has been an educator at P.S. 307 for 22 years. Over the course of her career, she has served as a classroom teacher (Grades Pre-K, 1, 2 and 3), mathematics coach, technology teacher, magnet resource specialist, and mentor. No matter the position, role or duties, she enjoys each, in addition to working with staff, students, parents, and partnerships. She brings to her work a focused and organized structure which has benefited her and the school over the years.
Melanie Villatoro is an assistant professor in the Department of Construction Management and Civil Engineering Technology.She teaches a variety of courses in the civil engineering major including statics, strength of materials, concrete, steel, soil mechanics, and foundations.Prof. Villatoro’s approach to teaching builds on developing rapport with her students.She is highly effective in the classroom and as an advisor and mentor.She is passionate about student retention and performance, as well as STEM Outreach from the elementary to the high school level.
Diana Samaroo is an associate professor and chair of Chemistry Department at New York City College of Technology in Brooklyn, New York. Her pedagogical research is in the area of peer-led team learning in Chemistry and integrating research into the curriculum.With a background in biochemistry, her research interests are in the area of drug discovery, therapeutics, and nanomaterials. She has successfully mentored students through the Louis Stokes Alliance for Minority Participation and the Black Male Initiative and serves on the college’s Undergraduate Research Committee.
American Chemical Society. (2018). Dissolving M&Ms. Retrieved February 5, 2018 from https://www.acs.org/content/acs/en/education/whatischemistry/adventures-in-chemistry/experiments/dissolving-m-ms.html.
Brophy, S., Klein, S., Portsmore, M., & Rogers, C. (2008). Advancing engineering education in P-12 classrooms. Journal of Engineering Education, 97(3), 369–387.
Brown, J. (2012). The current status of STEM education research. Journal of STEM Education: Innovations & Research, 13(5), 7–11. Available from Academic Search Complete, Ipswich, MA. Accessed October 9, 2017.
City University of New York. (2018). CUNY Service Corps. Retrieved February 5, 2018 fromhttp://www1.cuny.edu/sites/servicecorps/.
Furner, M. J., & Kumar, . (2007). The mathematics and science integration argument: A stand for teacher education. Eurasia Journal of Mathematics, Science & Technology Education,3(3), 185–189.
Institute of Electrical and Electronics Engineers. (2018) Try Engineering. Retrieved February 7, 2018 from http://tryengineering.org.
Johnson, C. C. 2013. Conceptualizing integrated STEM education. School Science and Mathematics, 113(8), 367–368.
Katehi, L., Pearson, G., & Feder, M. (2009). The status and nature of K-12 engineering education in the United States. The Bridge on K-12 Engineering Education, 39(3). Retrieved February 5, 2018 from https://www.nae.edu/19582/Bridge/16145/16161.aspx.
Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Washington, DC: Association of American Colleges and Universities.
Langdon, D., McKittrick, G., Beede, D., Khan, B, & Doms, M.(2011). STEM: Good jobs now and for the future. U.S. Department of Commerce, Economics and Statistics Administration. Retrieved February 7, 2018 from http://www.esa.doc.gov/sites/default/files/stemfinalyjuly14_1.pdf.
Ressler, Stephen. (2013). The Bridge Designer Software. Retrieved February 7, 2018 from http://stephenjressler.com/wpbd/.
Russell S. H., Hancock , & McCullough, J. (2007). The pipeline. Benefits of undergraduate research experiences. Science, 316(5824), 548–549.
Schooling a Monkey. (2018). Hands-on teaching ideas. Retrieved February 7, 2018 from http://www.schoolingamonkey.com/engineering-activities-for-kids/.
Stohlmann, M., Moore, T. J., & Roehrig, G. H. (2012). Considerations for teaching integrated stem education. Journal of Pre-College Engineering Education Research (J-PEER), 2(1), 28–34.
STEAM Powered Family. (2018). Slime STEM Activities – Learning with slime, STEM and fun! Retrieved February 7, 2018 from https://www.steampoweredfamily.com/activities/slime-stem-activities-learning-with-slime-stem-and-fun/.
Science Pioneers. (2017). Why STEM education is important for everyone. http://www.sciencepioneers.org/parents/why-stem-is-important-to-everyone.
Ressler, Stephen. (2013). The Bridge Designer Software. Retrieved February 7, 2018 from http://stephenjressler.com/wpbd/.
Wai, J., Lubinski, D. & Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817–835.
Retention efforts in STEM have become a priority of colleges and universities. Two-year college STEM students are particularly affected by factors that contribute to low retention and persistence. To address STEM retention problems, a student support program was developed through National Science Foundation funding to support STEM student success. The program sought to enhance STEM identity, thereby increasing persistence. Participants were required to engage in STEM civic engagement, using their STEM knowledge and skills for community betterment. This study sought to examine the effects of these activities on students’ STEM identity and ultimate persistence. Data were collected over years from participant surveys and interviews. We found that students had cultivated a sense of STEM identity, and that graduation and transfer rates increased as a result of their increased civic engagement. Students who engage in their community develop cultural competency, communication skills, and critical thinking ability and have opportunities to apply their knowledge.
The Role of Two- year Colleges in STEM Education
Two-year colleges are an often overlooked but essential component in the pathway to Science, Technology, Engineering, and Mathematics (STEM) higher education (National Academies of Sciences, Engineering, and Medicine [NASEM] 2016; National Research Council [NRC] 2012).They play a unique role in STEM education, enrolling nearly half of the nation’s undergraduate students (American Association of Community Colleges [AACC] 2014). Community colleges in the United States enroll more than eight million students annually, including 43% of U.S. undergraduates (AACC 2011; Mullin 2012). Approximately 50% of all college students who eventually earn bachelor’s degrees in STEM begin their undergraduate education at two-year colleges (Tsapogas 2004; Starobin & Laanan 2010), and 20% of students who were awarded science and engineering doctoral degrees earned credits at a two-year college at some point in their academic careers (Chen 2013).
Community colleges provide a diverse student body (people of color, women, older students, veterans, international students, first-generation college students, low-income students, and working parents) with access to higher education. According to the American Association of Community Colleges, 52% percent of Hispanic students, 44% of African American students, 55% of Native American students, and 45% of Asian-Pacific Islander students attend two-year colleges (AACC 2011). Additional reports (Provasnik & Planty 2008) show the median age of two-year college students is 24, with 35% of the student population 30 or older. Further data show that 20% of two-year college students are married with children, and an additional 15% are single parents (Provasnik & Planty 2008; Li 2007). Almost half of college-going students attend community colleges at some point in their academic careers; low-income, first generation, and under-represented minority students are more likely to enroll in two-year institutions (NASEM 2016).
Two-year colleges attract many students by providing affordable tuition, flexible scheduling, small class sizes, and access to faculty. These institutional attributes accommodate those two-year college students who take a nonlinear path to degree completion due to family and work obligations (Pérez & Ceja 2009). On account of the rich diversity of their student population, two-year colleges have the potential to increase participation of non-traditional and underrepresented students in STEM.
Retention and Persistence for Community College STEM Students
Retention and persistence of all STEM students continue to be of significant concern as data reveal that more than half of freshman who initially declare STEM majors leave these fields before graduation (President’s Council of Advisors for Science and Technology [PCAST] 2012; Chen 2009; Chen 2013). Among all students who declared their intentions to pursue STEM majors, only 43% were still in a STEM major at the time of their last enrollment, with the others all transitioning to other majors. Even more problematic, only 7.3% of STEM students who began at a two-year college received a STEM bachelor’s degree after six years, compared with 45% of students who started in a four-year program (Chen 2013).
Factors influencing retention and persistence in STEM majors are diverse and often interconnected. Leading reasons for low STEM retention and persistence at both the two-year and four-year colleges are uninspiring introductory courses, lack of math preparation, and an academic culture not welcoming of women, minorities, and non-traditional students (PCAST 2012; Seymour and Hewett 2000; Griffith 2010; Huang, Taddese, & Walter 2000). Additionally, STEM students at the two-year college are affected by external circumstances such as work and family obligations and have fewer economic and social resources and fewer STEM role models than their four-year traditional student counterparts.For the two-year college STEM student, these external circumstances coupled with an unwelcoming STEM culture undermine their sense of identity, belonging, and self-efficacy, which are critical to their STEM retention and persistence.
The Culture of STEM
The explicit and implicit customs, behaviors, and values that are normative within STEM education make up the culture of STEM (NRC 2009). An examination of the culture of STEM education is important because the social, psychological, and structural dimensions of STEM education in two-year and four-year colleges influence student identity, belonging, self-efficacy, and encouragement. The experiences students gather during their interactions with the “STEM culture” of the department or institution drive student awareness and understanding of program standards, academic expectations, STEM identity, and their sense of belonging in the program. More importantly, student experiences within the STEM culture and the encouragement or lack thereof can have a profound impact on the student’s self-efficacy and desire to persist (Cabrera et al. 1999; Eccles, Wigfield, & Schiefele 1998; Reid & Radhakrishnan 2003; Pérez, Cromley, & Kaplan 2014).
Identity/Belongingness, Encouragement, and Self-efficacy
Self-perceptions regarding academic competence are framed by personal and collective identities. Each student has many such identities—racial, ethnic, socioeconomic, professional, sexual/gender, and family. These identities are framed by upbringing, experiences, and society at large and can shift across time either unconsciously or through deliberate effort (Good 2012). Students’ positive identification with their discipline can enhance academic engagement and belongingness and prove to be a great source of encouragement. However, more commonly the obverse is true, especially for non-traditional and underrepresented STEM students. These students often experience challenges such as isolation, invisibility, discrimination, and a sense of not belonging and disconnectedness from external social and cultural networks (Ong 2001; NRC 2012).
Belonging to valued social groups is a fundamental human need; a sense of inclusion is particularly important for underrepresented groups in STEM when stereotypes imply that they might be unsuited to certain settings, such as rigorous academic classes (Baumeister & Leary 1995; Dovidio, Major, & Crocker 2000; Walton & Cohen 2007; Cohen & Steele 2002). Feeling a sense of belonging and acceptance by others in STEM (faculty and peers) is crucial to retention and persistence for these STEM students (Johnson 2012; Palmer, Maramba, & Dancy 2011).
Stereotypical ideas about what constitute appropriate fields of study for two-year college students or comments regarding academic preparedness/achievement in math and science can serve as critical barriers to retention and persistence. According to Starobin & Laanan (2008), even when these students possess a strong math or science background, they often receive little encouragement or support from faculty. Creating a sense of encouragement and a support system for two-year college STEM students is paramount to increasing retention and persistence. Studies show non-traditional and underrepresented minorities need proactive personal encouragement and positive media messages to counteract the status quo “culture of STEM” (Hanover Research, 2014). Programs and activities that facilitate healthy positive relationships and offer encouragement among peers and from faculty promote student engagement and feelings of belonging.
Academic self-efficacy is commonly defined as the belief in one’s capabilities to achieve a goal or an outcome using one’s skills under certain circumstances, and that performance and motivation are determined by how effective people believe they can be. (Snyder & Lopez 2007; Bandura 1982). More specifically, for many two-year STEM students, academic self-efficacy is entangled with STEM identity as it refers to the belief or conviction that they can successfully obtain a STEM degree (Marra et al. 2009).
A major source of academic self-efficacy is simply having the raw knowledge, skills, and experience required to successfully reach a goal or to complete a task; this source of efficacy is commonly referred to as mastery experience (Bandura 1997). In the context of two-year STEM students, this means having a positive experience in completing a STEM task, specific course, and/or obtaining an associate’s degree.
STEM Civic Engagement through Peer Tutoring
STEM civic engagement covers a wide array of activities and learning outcomes in which students participate in the formal and informal STEM processes that address community needs and seek to improve the quality of life for individuals, groups, and entire communities. In this context, STEM civic engagement contributes to student growth by connecting authentic and meaningful service to communities with content and skills acquired in the classroom. Civic engagement activities, such as tutoring others in STEM content, present students with opportunities to reflect upon their own academic goals (also known as metacognition) (NRC, 2000), transform their communities, and identify and address social challenges that are specific to our society, i.e. the lack of STEM subject understanding, the lack of STEM role models, etc.
It is well documented that tutoring has beneficial effects on both the tutor and the tutee.In particular, many studies have shown that tutoring increases the content knowledge as well as the self-concept of the tutor (Britz, Dixon, & McLaughlin1989; Cohen, Kulik, & Kulik 1982; Early 1998).Students who tutor feel more positive towards themselves as students, and they display an improved academic self-concept. Through this enhanced self-concept, students identify themselves more strongly as students of their discipline (Early 1998).Furthermore, students in STEM disciplines who serve as leaders among their peers experience increased self-efficacy and retention, and studies have shown that this trend applies to both majority and underrepresented students.Thus, peer leadership may provide a path for improving retention of underrepresented groups in the field (Hug, Thiry, & Tedford 2011). Additional outcomes for STEM leaders (mentors or tutors) include increased participation in internships and higher GPAs (Monte, Sleeman, & Hein 2007). Other studies indicate that the opportunity to tutor or mentor others allows STEM students to develop a sense of belonging and social relationships that aid in student retention; to some extent, this can be attributed to improved experience with and understanding of STEM culture at the students’ institutions (Kiyama 2014; Kiyama et al. 2014).
Existing research provides a limited understanding of the relationship between identity/belonging, encouragement, self-efficacy, civic engagement, and retention rates for two-year college STEM students. Our study explored the effects of civic engagement volunteer activities on student identity/belonging, encouragement, and self-efficacy.The results show a relationship between these activities and STEM persistence and retention for two-year college STEM students.
Institution and Program
Perimeter College is part of Georgia State University, a diverse, multi-campus urban research university in metropolitan Atlanta. The college is the major provider of associate’s degrees and student transfer opportunities in Georgia and a gateway to higher education, easing students’ entry into college-level study.More than 21,000 students, representing all ages and backgrounds, are enrolled in Perimeter College. Through the college, Georgia State serves the largest number of dual enrollment, international, online, transfer, and first-time freshman students in the University System of Georgia.
Beginning in Spring 2012, through National Science Foundation funding, a Science, Technology, Engineering, and Mathematics Talent Expansion Program (STEP) was developed for two-year, full-time students, with a minimum 2.8 grade point average. To participate, students must have U.S. citizenship or status as permanent resident alien or refugee alien and be majoring in a STEM field of study, declared at any point but usually after the first year of coursework. The objectives of the program are two-fold: (a) to increase the number of students who persist in all STEM fields at the institution (chemistry, biology, math, geology, physics, computer science, and engineering) and (b) to increase the number of students who graduate and/or transfer to four-year colleges/universities to complete their STEM baccalaureate degrees.The demographic breakdown of the STEP participants throughout the lifetime of the program mirrored that of the STEM majors in the institution; the majority of STEP students are underrepresented minorities.
Students participate in the program for an average of three semesters (including a summer semester). Stipends are given to those participants who meet the following criteria each semester: (a) are enrolled as a full-time student (12 credit hours during the fall and spring semester); (b) maintain a cumulative minimum GPA of 2.8 and a minimum semester GPA of 2.5; (c) participate in a minimum of 10 hours of STEM civic engagement activities per semester; (d) participate in a minimum of six STEM–related activities (STEP-sponsored and others). Stipend amounts vary depending on the academic classification of the participant. Additional stipends are given for participation in the Summer Bridge I undergraduate research experience (three weeks), Summer Bridge II undergraduate research experience (eight weeks), and participation in the NSF’s Research Experiences for Undergraduates program. STEP sponsors multiple STEM activities each semester, including STEM industry visits and college visits.
STEM Civic Engagement Activities
Program participants are engaged in the STEM community in a number of ways, some of which are required elements and others that are optional.All program participants are required to attend a number of career workshops and to visit industry sites and four-year institutions.Additionally, throughout their tenure in the program, participants are required to complete a minimum of 10 hours of civic engagement per semester.Many of the students fulfill this requirement by serving as tutors in on-campus student support facilities or off campus in their communities. Additional civic engagement opportunities are available to the students through outreach activities (such as science festivals), environmental clean-ups, and other STEM-related events. Many students (73%) completed more than the required 10 hours per semester of service; the average contribution per semester is 12 hours of service.
In order to determine student outcomes, we tracked students through their program experience and after graduation and transfer to four-year institutions. During their tenure in the program, participants were asked to complete a number of surveys and focus group interviews to determine their reactions to and the perceived outcomes of the various student support activities.Surveys were retrospective in design: students were asked to think back to how they felt at the beginning of the program and compare that to how they felt at the time of taking the survey (usually after one year in the program). This approach maximizes ability to match responses and also eliminates pretest sensitivity and response shift bias, wherein students tend to underestimate or overestimate their attitudes towards the unknown prior to the start of an intervention (Howard 1980; Pratt, McGuigan, & Katzev 2000). In addition to surveys given during students’ tenure in the program, we also administered an alumni survey to those who had completed the program.
In particular, our 23-item student survey drew upon existing instruments designed to assess changes in STEM engagement (Fredricks et al. 2005), STEM identity and belonging, encouragement (Leonowich-Graham & Condley 2010), math and science anxiety (Bai et al. 2009; Glynn and Koballa 2006), commitment to research, and intent to persist (Tocker 2010). Further definition of these psychosocial constructs is presented in Table 1, along with example survey items. Students were asked to respond to survey items using a 5-point Likert scale of agreement (1=Strongly Disagree to 5=Strongly Agree).
To collect qualitative data, students were assembled in groups of 812 to participate in annual focus group interviews.During these interviews, students were asked probing questions regarding their experiences in the program and how they affected their identity, engagement, and intent to persist in STEM. The focus group interview protocol included questions such as the following:
Describe civic engagement activities that you participated in.
Did these activities change the way you think about yourself? About your intended career?
Are you making different decisions because of participating in this program? Explain.
To further explore the link between persistence and gains made by students as a result of the program and civic engagement activities, a multiple regression analysis was conducted whereby the outcome variable was Intention To Persist and the predictor variables were STEM Engagement, STEM Identity and Belongingness, Math and Science Anxiety, Research, and Encouragement. To compute the outcome and predictor values for this analysis, items from the student survey were averaged for each corresponding construct.
Qualitative data gleaned from participants’ open-ended responses to surveys and during focus group interviews suggested that the STEP program positively impacted their motivation to pursue STEM education and careers by enhancing their sense of STEM identity and belonging and by providing social support and encouragement.
[STEP] helped me to be confident and to trust myself that I can do better things if I have the will. It also helped me make the decision that I belong to a STEM family.
STEP enhanced my vision of being a scientist.
I was about to give up on my school.…[A]fter meeting and getting help from different people, I was able to rethink my major and continue my studies.
Additionally, annual surveys completed by program participants demonstrated that they made significant gains in terms of STEM engagement, STEM identity and belongingness, comfort with math and science, encouragement, and intent to persist.Table 2 shows statistically significant gains in attitude measured by these surveys over the course of the program.
Figure 1 summarizes the results of the regression analysis, conducted using data from the alumni surveys administered in 2013 and 2015 (n=39). Students taking the alumni survey had all completed their program and/or transferred to a four-year institution. Alumni survey data were chosen for this regression analysis in order to limit the findings to that of a longer-term student perspective; these students had the benefit of looking back over their entire program experience, and these data represent a more complete picture. The regression model with all five predictors explained 95% of the variance in the outcome variable (R2=.948, F(5,33)= 119.18, p<.001).Controlling for other variables in the model, the results indicate that two variables statistically significantly predict intent to persist:
STEM Identity and Belongingness (ß=.55, p<.001)
Encouragement (ß=.56, p<.001)
This suggests that students’ motivations to pursue additional STEM education and/or careers is contingent on the degree to which the program was able to (a) improve their sense of belonging in STEM and (b) provide encouragement for attaining a STEM degree. This finding corroborates previous research which indicates that STEM persistence increases as students experience a greater sense of belonging and general social support from mentors and colleagues (London et al. 2011).
Quantitative data analysis was limited in that the response rate for the student surveys was not 100%. (Response rate was roughly 85% across all items and multiple administrations of the survey.)Thus, responses might demonstrate a bias towards the positive, as students who felt less compelled to respond to the program survey were often those who had left the program (and usually the institution). Additionally, due to the low sample size, we must use caution when interpreting the results of the regression analysis. Correlations among constructs suggest that multicollinearity may have impacted the results of the regression. To mitigate the effects of multicollinearity, each predictor variable in the regression model was standardized (e.g., converted to a z-score). Furthermore, the results provided in the current report are preliminary and should be replicated using a larger sample size. It is also important to note that disaggregation of data by gender or race/ethnicity did not reveal significant differences among the participating groups of students.
During annual interviews, students were asked about their experiences in program activities, and how they thought these experiences affected them. In particular, we explored which facets of the program led to increased STEM identity and encouragement.Students explained that the volunteer work they did to meet their civic engagement requirements helped them in many ways.Specifically, they were able to solidify their STEM content knowledge and improve their communication and leadership skills:
Being part of [tutoring]… helps you refresh your mind. When you are helping them it helps you refresh your mind. You refresh communication skills.
It improves your leadership skills. One thing that I’ve learned is that you’re more involved in the community and you’re more exposed to the problems of the community. I think that it really improves your communication skills, your leadership, and it helps you learn more about your community.
Participants also felt that civic engagement motivated them to work harder in STEM and gave them a broader perspective on their futures.
It opens your mind up to all that’s out here. It’s opened my mind to what’s out there and made me think that I want to help people. It’s an unselfish thing.
Even being around the other members, outside of class, you get to know them—being around people that are really smart, makes me want to be really smart.
You become more motivated. You want to learn as much as you can. You want to help as much as you can. You want to put things out there so that people can learn from you.
It’s not about improving myself, but improving other people’s lives. I started thinking about non-profits. I started thinking about things that I didn’t think about before.
In short, students explained that participation in civic engagement improved their STEM and soft skills and motivated them to consider a broader range of career options. Their sense of identity as part of a STEM community was solidified through exchanges with their peers as well as with those they were helping.
In order to examine the effect of programmatic activities on actual persistence, we tracked transfer and graduation rates of the scholars, and compared those to non-participant STEM students. Table 3 indicates that program participants were more than twice as likely to complete their program of study and /or transfer to a four-year institution to pursue a STEM degree. Furthermore, STEP students who completed at least 10 hours per semester of civic engagement activities were even more likely to graduate and/or transfer (Table 3).
The culture that students encounter when studying STEM has an effect on their interest, self-concept, sense of connectedness, and persistence in STEM. Students who persist often have to draw upon personal, cultural, and co-curricular resources to counter messages about the nature of ability and stereotypes that they encounter in interactions with faculty and that are embedded in organizational norms and practices.
Interventions aimed at improving participant identity and belonging have been shown to enhance achievement and persistence (Cohen & Garcia 2008). Not surprisingly, students in highly evaluative environments (such as STEM courses) are sensitive to stereotype threat when facing difficult coursework and feedback, suggesting that it is particularly important to focus on improving STEM identity in an effort to increase student success (Cohen & Steele 2002).
Despite limitations of the study discussed in the results section, we found that an increase in STEM identity and belongingness and encouragement predicted an increase in intent to persist, and that actual persistence was improved with civic engagement. We posit that opportunities to guide others through tutoring and other civic engagement activities enhanced STEM identity, as scholars explained to us during interviews.In concurrence with STEM achievement, improved identity and belongingness in STEM led to a substantially higher likelihood of graduation and or transfer, as evidenced by participant graduation and transfer rates in comparison to those of non-participant STEM students at the institution. Participating students still face a number of challenges, as do their non-participating counterparts; though the overall graduation and transfer rate for participants is still alarmingly low, the trend towards success is encouraging and suggests that interventions aimed at increasing STEM identity through civic engagement will increase overall STEM diversity in academe and the workforce
About the Authors
Dr. Pamela M. Leggett-Robinson is the Science Department associate chair and an associate professor of chemistry on the Decatur campus of Georgia State University-Perimeter College. Dr. Leggett-Robinson has served as a program director for several NSF and NIH initiatives and is currently the principal investigator of Georgia State University-Perimeter College’s NSF STEP grant. Her research and scientific presentations focus on natural product chemistry, surface chemistry, and student support programs in STEM education. She holds a BS in Chemistry from Georgia State University, an MS in Bio-Inorganic Chemistry from Tennessee Technological University, and a PhD in Physical Organic Chemistry from Georgia State University. As corresponding author, Dr. Leggett-Robinson can be reached at firstname.lastname@example.org.
Mrs. Naranja Davis is the NSF GSU-PC STEP coordinator. She has worked as a coordinator on several other NSF STEM initiatives over the past 10 years and is experienced in student data systems. Ms. Davis has a BS in Communication with a minor in Public Relations.
Dr. Brandi Villa did her graduate research in areas of applied and environmental microbiology as well as program evaluation of a science education outreach organization. She has been a science educator at middle school, high school, and undergraduate levels for more than a decade and thus brings an educator and researcher’s perspective to the design and implementation of education research and program evaluation. In addition to her passion for all aspects of STEM education, Dr. Villa particularly enjoys challenges related to evaluation design, reporting, and data visualization.
American Association of Community Colleges (AACC). (2011). Fast facts, 2011. Washington, DC: American Association of Community Colleges.
American Association of Community Colleges (AACC). (2014). Fact Sheet. Washington, DC: American Association of Community Colleges.
Bai, H., Wang, L., Pan, W., & Frey, M. (2009). Measuring mathematics anxiety:Psychometric analysis of a bi-dimensional affective scale. Journal of Instructional Psychology, 36(3), 185–193.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman and Company.
Baumeister, R. F., & Leary, M. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497–529.
Britz, M. W., Dixon, J., & McLaughlin, T. F.(1989). The effects of peer tutoring on mathematics performance: A recent review. B.C.J. Special Education, 13,17–33.
Cabrera, A. F., Nora, A., Terenzini, P. T., Pascarella, E. T., & Hagedorn, L. S. (1999). Campus racial climate and the adjustment of students to college: A comparison between white students and African-American students. Journal of Higher Education, 70, 134–160.
Chen, X. (2009). Students who study science, technology, engineering, and mathematics (STEM) in postsecondary education. Washington DC: U.S. Department of Education, National Center for Education Statistics, Institute of Education Sciences. Retrieved January 31, 2018 from https://nces.ed.gov/pubs2009/2009161.pdf.
Chen, X. (2013). STEM attrition: college students’ paths into and out of STEM fields (NCES 2014-001). Washington DC: U.S. Department of Education, National Center for Education Statistics, Institute of Education Sciences. Retrieved January 31, 2018 from https://nces.ed.gov/pubs2014/2014001rev.pdf.
Cohen, G. L., & Garcia, J. (2008). Identity, belonging and achievement: a model, interventions, implications. Current Directions in Psychological Science, 17, 365–369.
Cohen, G. L., & Steele, C. M. (2002). A barrier of mistrust: How negative stereotypes affect cross-race mentoring. In J. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education. San Diego, CA: Academic Press.
Cohen, P. A., Kulik, J. A., & Kulik, C. C. (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19, 237–248.
Dovidio, J. F., Major, B., & Crocker, J. (2000). Stigma: Introduction and overview.In T. F. Heatherton (Ed.), The Social Psychology of Stigma (pp. 1–28). New York: Guilford Press.
Early, J. W. (1998). The impact of peer tutoring on self-esteem and Texas Assessment of Academic Skills mathematics performance of tenth grade students. Unpublished doctoral dissertation, Texas A&M University, College Station, TX.
Eccles, J., Wigfield, A., & Schiefele, U. (1998). Motivation to succeed. In W. Damon (Series Ed.) and N. Eisenberg (Volume Ed.), Handbook of Child Psychology. Vol. 3: Social, Emotional, and Personality Development (5th ed., pp. 1017–1095). New York: Wiley.
Fredricks, J. A., Blumenfeld, P. C., Friedel, J., & Paris, A. (2005). School engagement. In K. A. Moore and L. Lippman (Eds.), Conceptualizing and measuring indicators of positive development: what do children need to flourish? New York: Kluwer Academic/Plenum Press.
Glynn, S. M., & Koballa, T. R., Jr. (2006). Motivation to learn college science. In J. J. Mintzes & W. H. Leonard (Eds.), Handbook of college science teaching. Arlington, VA: National Science Teachers Association Press.
Good, C. (2012). Why do women opt out? Sense of belonging and women’s representation in mathematics. Journal of Personality and Social Psychology, 102(4), 700–717.
Griffith, A. L. (2010). Persistence of women and minorities in STEM field majors: Is it the school that matters? Retrieved January 31, 2018 from http://digitalcommons.ilr.cornell.edu/workingpapers/122/
Hanover Research (2014). Encouraging Students to Embrace STEM programs. Retrieved January 31, 2018 from http://mygaen.org/research/Encouraging%20Students%20to%20Embrace%20STEM%20Programs.pdf.
Howard, G. S. (1980). Response-shift bias: A problem in evaluating interventions with
Huang,G., Taddese, N., & Walter, E. (2000). Entry and persistence of women and minorities in college science and engineering education (NCES 2000–601). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Hug, S., Thiry, H., & Tedford, P. (2011, March). Learning to love computer science: Peer leaders gain teaching skill, communicative ability, and content knowledge in the CS classroom. SIGCSE: Proceedings of the 42nd ACM Technical Symposium on Computer Science Education, Dallas, TX (pp. 201–206).
Johnson, D. R. (2012). Campus racial climate perceptions and overall sense of belonging among racially diverse women in STEM majors. Journal of College Student Development, 53(2), 336–346.
Kiyama, J. M. (2014). Structured opportunities: Exploring the social and academic benefits for peer mentors in retention programs. Journal of College Student Retention: Research, Theory, & Practice, 15, 489–514.
Kiyama, J. M., Luca, S. G., Raucci, M., & Crump-Owens, S. (2014). A cycle of retention: Peer mentors’ accounts of active engagement and agency. College Student Affairs Journal. 32, 81–95.
Leonowich-Graham, P., & Condly, S. (2010). Encouragement may be the missing link in the pursuit of CS / IT Majors. Fall 2010 Mid-Atlantic ASEE Conference,Villanova University. Retrieved January 31, 2018 from https://www.asee.org/documents/sections/middle-atlantic/fall-2010/01-Encouragement-May-Be-the-Missing-Link-in-the-Pursuit-of-CS-I.pdf.
Li, X. (2007). Characteristics of minority-serving institutions and minority undergraduates enrolled in these institutions (NCES 2008-156). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
London, B., Rosenthal, L., Levy, S., & Lobel, M. (2011). The influences of perceived identity compatibility and social support on women in nontraditional fields during the college transition. Basic and Applied Social Psychology, 33, 304–321.
Marra, R. M.,Rodgers, K. A., Shen, D., & Bogue, B. (2009) Women engineering students and self-efficacy: A multi-year, multi-institution study of women engineering student self- efficacy. Journal of Engineering Education ,98(1), 27–38.
Monte, A. E., Sleeman, K. A., & Hein, G. L. (2007, October). Does peer mentoring increase retention of the mentor? 37th ASEE/IEEE Frontiers in Education Conference. Milwaukee, WI.
Mullin, C. M. (2012, February). Why access matters: The community college student body (Policy Brief 2012-01PBL). Washington, DC: American Association of Community Colleges
National Academies of Sciences, Engineering, and Medicine (NASEM). (2016). Barriers and opportunities for 2-year and 4-year STEM degrees: Systemic change to support students’ diverse pathways. Washington, DC: National Academies Press.
National Research Council (NRC). (2000). How people learn: Brain, mind, experience, and school.. Expanded Edition. Washington, DC: National Academies Press.
National Research Council (NRC). (2009). Learning Science in Informal Environments: People, Places, and Pursuits.Washington, DC: The National Academies Press.
National Research Council (NRC). (2012). Community colleges in the evolving stem education landscape: Summary of a summit. Washington, DC: National Academies Press.
Ong, M. (2001). Playing with in/visibility: How minority women gain power from the margins of science culture. Women in Higher Education, 10/11, 4–44.
Palmer, R. T., Maramba, D. C., & Dancy, T. E. (2011). A qualitative investigation of actors promoting the retention and persistence of students of color in STEM. Journal of Negro Education, 80(4), 491–504.
Pérez, P., & Ceja, M. (2009). Best practices and outcomes in transfer to universities. Journal of Hispanic Higher Education. 9(1), 6–21.
Pérez, T., Cromley, J. G., and Kaplan, A. (2014). The role of identity development, values, and costs in college STEM retention. Journal of Educational Psychology, 106(1), 315–329.
Pratt, C., McGuigan, W., & Katzev, A. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 34–349.
President’s Council of Advisors for Science and Technology (PCAST). (2012). Report to the President: Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, DC: Executive Office of the President.
Provasnik, S., & Planty, M. (2008). Community colleges: Special supplement to the Condition of Education 2008 (NCES 2008-033). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Reid, L. D., & Radhakrishnan, P. (2003). How race still matters: The relation between race and general campus climate. Cultural Diversity and Ethnic Minority Psychology, 9, 263–275.
Snyder, C. R., & Lopez, S. J. (Eds.) (2002). Handbook of positive psychology. Oxford: Oxford University Press.
Starobin, S. & Laanan, F. S. (2008). Broadening female participation in science, technology, engineering, and mathematics: Experiences at community colleges. New Directions for Community Colleges, 142, 37–46.
Tocker, Y. (2010).Non-ability correlates of the science/math trait complex: Searching for personality characteristics and revising vocational interests. Doctoral dissertation: Georgia Institute of Technology, Atlanta, GA.
Tsapogas, J. (2004). The role of community colleges in the education of recent science and engineering graduates (InfoBrief 04-315.) Arlington, VA: Division of Science Resources Statistics, National Science Foundation.
Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92(1), 82–96.
Our World in Data is an online publication that will be of interest to many readers of Science Education and Civic Engagement: An International Journal. It brings together in one location data about a number of different topics related to how the world is changing. The site is produced at the University of Oxford by a team led by Max Roser, an economist at the university. Amazingly, the entire project is available free of charge as a public good!
Roser began the project in 2011 and for several years was the sole author until grant funding allowed him to add team members. The long-term goal is to create 275 distinct entries in the site. Entries are gathered into thematic sections; as of January 2018, these include Population, Health, Food, Energy, Environment, Technology, Growth & Inequality, Work & Life, Public Sector, Global Connections, War & Peace, Politics, Violence & Rights, Education, Media, and Culture.
There are several features of the site that make it attractive to educators. The Energy section, for example, is divided into a number of subsections—energy production and changing energy sources, fossil fuels, renewables, carbon dioxide and other greenhouse gases. The section on energy production and changing energy sources is further divided into sections titled “Empirical View,” “Correlates, Determinants, and Consequences,” and “Data Sources.”
There are numerous visualizations for topics such as energy production by source, energy production over time, energy intensities of the economies in various parts of the world, access to electricity, and per capita energy consumption, among many others. Some visualizations present the data over time and allow one to focus on a particular year. Other visualizations provide the option for changing from a graph to a map or changing the axes on a particular graph. Images can easily be downloaded as .png files for use in presentations or other documents. Data used in a particular visualization can be downloaded as a .CSV file that can be opened in Excel. All data are clearly identified regarding point of origin, and the sources appear to be reliable—academic sites, United Nations agencies, the World Bank, the World Health Organization, and others—and one section of the website explains how the team chooses the data that are presented. The site also contains an essay that explains the rationale for Our World in Data: to support better understanding, involvement, and policy making by presenting an accurate picture of global progress in development. Overall, the site conveys a commitment to transparency that is commendable.
I have used some of the visualizations from the site in three different courses this semester: information on energy consumption (per capita and by source) in General Chemistry II and in a course for nonscience majors focused on sustainability, and information on malaria in my biochemistry class. They added a dimension to the classes that would have been very difficult for me to accomplish otherwise.
For educators who want to bring a global dimension to their incorporation of civic engagement into a course, Our World in Data will be an invaluable resource. I highly recommend it.
-Matt Fisher, Science Education and Civic Engagement: An International Journal
Course-based undergraduate research is an effective active, inquiry-based pedagogical tool. In many cases, these research experiences build on established research programs. This project report describes a research course designed to establish a new translational research program in epilepsy and to test the feasibility of engaging students early on in the research process. The outcomes of this class, including research deliverables and student learning gains assessments, indicate that engaging students in research at a very early stage in project development is a meaningful and productive pedagogical framework for student and faculty development. This high-risk model for course and research development is a novel and exciting method for engaging students in mentored research at the undergraduate level.
Mentored research at the undergraduate level is considered a high-impact pedagogical practice (Kuh, O’Donnell, & Reed, 2013), and many STEM courses incorporate students into established research programs and projects. The benefits of course-based research are not limited to students, as faculty research progress can be boosted by the concentrated student collaboration found in these courses. Moreover, students can bring fresh perspectives and make important contributions to research at the point of new project development. Involving students in “early” research (e.g. establishing research aims, refining protocols and procedures, and collecting and analyzing background data) can be a context for simultaneously robust student learning and faculty professional development. However, the risks of failure associated with early research may make faculty reluctant to consider building a research course specifically centered on developing a new and untested project. The course described below provides evidence in favor of building a course around a new research program, using the example of a successful pilot of course-based translational neuroscience research at the undergraduate level. The work of this course, offered at a small liberal arts college, set the stage for a robust, student-centered translational research program that also advanced the instructor’s research agenda.
Translational research: From basic science to disease intervention
The confirmation in humans of the results of basic science research using cell and animal models is a critical step in developing patient-centered interventions to improve human health (US Department of Health and Human Services [USD HHS], 2015). Translational research, which bridges basic science and clinical research, is a major focus of NIH funding and support through the National Center for Advancing Translational Sciences. However, it can be challenging to implement translational research at small colleges and universities, as many of these institutions are not in a position to conduct clinical and patient-centered translational research. These shortcomings may be circumvented through the use of publicly available online databases that provide students and faculty with the opportunity to work directly with human data collected under IRB approval from large research institutions. As funding for basic science research decreases, engaging undergraduate students in the process of translational research is critical to the enhancement of their understanding and appreciation of the fundamental role of basic science in improving the health and well-being of the broader population (Hobin et al., 2012).
Epilepsy and EEG
Approximately two percent (+/- 0.11) of Americans suffer from epilepsy (US DHHS, 2017), a family of disorders in which a person who has previously had a seizure is likely to experience another unprovoked seizure (Fisher et al., 2014). The etiologies of epilepsy are varied and, in many cases, still unknown (Shorvon, 2011). Thus much of the effort in the clinic is aimed at seizure management and prevention.
The monitoring of the epileptic brain via electroencephalography, or the recording and analysis of the electrical signals of the brain, is critical to the management of epilepsy. In particular, many patients with intractable epilepsy, i.e. epilepsy that is resistant to management by medication, undergo long-term intracranial electroencephalography in the inpatient hospital setting to collect electroencephalogram (EEG) signals from up to hundreds of locations across the cortex of the brain over the course of several days. The signals are analyzed to determine whether surgical resection of the epileptic locus, or the portion of the brain implicated in the start of seizure activity, is a possible epilepsy management strategy. Yet EEG analysis is time-consuming and subject to low inter-observer reliability, especially regarding the precise timing and location of seizure onset in the brain (Abend et al., 2011; Benbadis et al., 2009; Tatum, 2013). Therefore, research on the development and use of automated, standardized, and quantitative EEG analysis through computer is an expanding field of inquiry (Acharya et al., 2013; Halford et al., 2011).
Course structure and implementation
Translational research towards understanding how EEG analysis is similar or different among rodent models of epilepsy and human epilepsy in the clinical setting serves as the foundation for the research course described in this report.An advanced topics course (BIOL 373, Advanced Neuroscience Research) was developed and implemented in spring 2017 to model a translational EEG research laboratory environment for eleven undergraduate students. The three goals for this course were to: (1) engage multiple students in a semester-long mentored research experience, (2) determine whether student learning gains through engagement with an early research project are similar to those of students in established research projects, and (3) determine the feasibility of conducting and developing the background work for translational epilepsy research at Beloit College, a small liberal arts college with no clinical research affiliation. In this model, students were full partners with the instructor in the research process to determine the goals and direction of the project. Students gained experience with the research process and its challenges, became familiar with the procedures and outcomes of a basic science investigation of seizure detection in mice (Bergstrom et al., 2013), identified and mined a publicly available human intracranial EEG database, revised and tested a MATLAB-based algorithm—originally developed for seizure identification in mice—on human EEG signal, and established and validated a procedure for quantitative analysis of human intracranial EEG signal.
The course began with a review of research in the analysis of rodent EEG (Bergstrom et al., 2013) and a discussion of the function of translational research. The students and instructor collaboratively identified a strategy for goal-setting and reflection-based assessment that would be completed every two weeks throughout the 15-week semester, with one single-week goal-setting and reflection cycle before the mid-term break. Major assessments for the class were: (1) a public works-in-progress seminar at the Beloit College Student Research Symposium and (2) smaller weekly student-driven lecture/discussion presentations on timely research-related questions of neuroscience and epilepsy in the literature, e.g. neuron and brain anatomy, the action potential, the contribution of interictal spiking brain activity to epileptogenesis, and automated EEG analysis tools. Additional assessments included (1) pre- and post-course Course Undergraduate Research Experience (CURE) survey (Denofrio et al., 2007; Lopatto et al., 2008), (2) Student Assessment of Learning Gains, or SALG survey (Carroll, 2010), (3) and completion of the standard Beloit College end-of-semester course evaluations. Data collection and reporting procedures were approved by the Beloit College Institutional Review Board, and students provided informed consent for their participation in this study.
Students self-identified interests within the project and formed small groups to develop and accomplish sub-goals for the research project. Groups of two to six students were fixed for each two-week goal-setting/reflection period in the first half of the term and worked on goals within the broader research aims, such as identifying data sources, learning basic seizure analysis in EEG, and annotating and implementing MATLAB code. At the midterm, students re-organized into stable groups for the remainder of the semester. These groups were focused on preparing a literature review (four students), establishing a strategy for manual scoring of EEG signals (three students), and revising and analyzing MATLAB algorithm code (three students). One student served as an official liaison between the manual scoring and code revision groups (eleven students total). The two-week reflection cycle was maintained through the second half of the course.Class time (twice a week for 110 minutes per meeting) was used primarily for weekly lab group meetings, student presentations of relevant neuroscience topics, and individual and group work interactions with the instructor.Students were expected to be largely self-directed and to allot additional time outside of class, though logs of work were not required.
Preliminary observations and outcomes
Seven of the eleven course participants completed both the pre- and post-course surveys. Their responses indicate that students in this course made similar learning gains in relevant research skills to those of the CURE survey comparison groups (Denofrio et al., 2007; Lopatto et al., 2008) (n ≤ 9603, Figures 1 and 2, two-sample t test, p > 0.05 for all comparisons). This indicates that engaging students in a course-based project at a very early stage is a meaningful mechanism for research at the undergraduate level and also performs an important role for faculty interested in establishing a new research project or trajectory.
Student responses from the SALG survey and Beloit College course evaluation seem to indicate that students, even while doing translational research, did not make significant connections between the concepts of basic science and translational research. For example, they did not mention translational research in any of their long-form comments. However, students did report in the course evaluations and the SALG that they made clear gains in self-directed learning (Box 1). It is important to note that, while most students had little or no prior experience with neuroscience, epilepsy, EEG, or the MATLAB programming environment, they were junior- or senior-level students who had already had extensive experience with student-driven learning and research design through the broader Beloit College curriculum. Thus it is possible that students at an earlier level of academic development might not have made similar learning gains (Kirschner, Sweller, & Clark, 2006).
Establishing a new research project: Engaging students in faculty development
In many course-based research projects, students are inserted into an already-established research project and are given a single task or experiment to complete by the end of the class. This course was different, in that the students were involved in establishing a new research program from the ground up and therefore were required to consider not only their role in the project but also how the project fit into a much broader context of sustained research. This challenging authentic research experience provided students with many opportunities to develop cognitive skills and resilience around the challenges of research and learning, especially self-directed learning and identifying research and educational resources.Assessment of the learning outcomes of this project indicate that involving students in research at a very early point in the process, even before research aims and procedures are fully developed, can be a powerful learning tool for students.
Involving students early in the development of a new research project can also be an efficient mechanism for increasing faculty research output. The translational research outcomes of this course were significant; the deliverables completed in the class which are relevant to starting a new research project are summarized in Box 2.Further, this preliminary work set the stage for three of the eleven students in the course to continue work with the faculty member on this project after the course, including serving as mentors for two new student researchers. Additional students will be recruited to this project in the future and will eventually see it through to completion and publication.
Together, the research deliverables and learning outcomes analyses suggest that situating early research project activities and goals as the context for a structured undergraduate course is an effective mechanism for faculty to test-drive or establish a new research program that extends beyond the course and, at the same time, engage more students in mentored research.
Challenges and Recommendations
The overt link to the unique niche of translational
research within the biomedical community did not come through in the analysis of student responses, even though students were actively engaged with the process. The concept of translational research is new to most students, and so more careful attention to highlighting the important role of this type of work is needed in models like this. Because this was a laboratory course designed to focus on analysis of EEG signal, the student presentations were primarily focused on the neurological concepts relevant to the project. However, more attention could have been directed to the impact and structure of the bench-to-bedside research model.
A future course is planned around this research project, but it will be situated at a different point in the research process than the course described here. This new course could provide additional opportunities for students to engage with the research process and to gain a broader understanding of the clinical aspects of epilepsy. Three potential additions to the course could include (1) inviting a physician to meet with the class to discuss epilepsy and EEG in the clinical context, (2) including a conference call or in-person meeting with an epilepsy researcher at a large research institution to provide additional input to the project and to model effective research collaboration, and (3) assigning students to prepare patient-centered documents or presentations to explain epilepsy, EEG, and the analysis tools that they are developing.
Finally, it is important to note that this model requires significant buy-in and trust from the students, as it is a high-risk project for both the students and the faculty member, and many students expressed uncertainty regarding their progress at some point in the course. For instance, one student commented on a lack of typical “classroom-like” learning (Box 1) while also noting clear gains in experience. While a neuroscience “crash course” or more regular lectures and activities centered on the concepts of neuroscience might have been useful for content acquisition, it is important to help students recognize that these may be common feelings as they transition from a more typical undergraduate lecture-discussion course format to a student-centered project in which students themselves are responsible for identifying and structuring their learning content. It was useful to have regular check-ins with students to help to normalize feelings of frustration and uncertainty as they encountered research roadblocks and conflicting information from published reports. Still, it is possible that recognizing the emotional investment inherent in research can help students at this stage of their academic career build resilience for future challenges. This hypothesis must be tested as we build new models for engaging students in research at the undergraduate level and in preparation for broader participation within the STEM fields.
Mentored research is a high-impact undergraduate education practice (Kuh, O’Donnell, & Reed, 2013), and STEM educators in particular must therefore be creative and develop more opportunities for students to be involved with and learn from the process. Students can and do make important learning gains through the process of investigating the feasibility of a translational research project and gathering background data and material in support of a larger project. The dual purpose of this course, to engage students in research and to develop a new avenue for a faculty member’s research, situates it as a model through which instructors can recognize and harness the power of students at this stage of the research project. These results should encourage faculty to consider course-based research as a powerful tool that they may wish to use to develop new lines of inquiry, and student contributions to faculty work at all other stages of a research project should be considered an essential component of research at undergraduate institutions.
About the Author
Rachel A. Bergstrom is an assistant professor of biology at Beloit College in Beloit, WI. She is a SENCER Leadership Fellow with two major arms to her research agenda: 1) identification and quantification of ictal and interictal events in EEG, with a focus on seizure diagnosis and prediction, and 2) the intersection of identity and education in STEM, specifically how group work impacts the student experience in the classroom and is related to persistence in STEM.
Abend, N. S., Gutierrez-Colina, A., Zhao, H., Guo, R., Marsh, E., Clancy, R. R., & Dlugos, D. J. (2011). Interobserver reproducibility of electroencephalogram interpretation in critically ill children. Journal of Clinical Neurophysiology: Official Publication of the American Electroencephalographic Society, 28(1), 15–19. https://doi.org/10.1097/WNP.0b013e3182051123
Acharya, U. R., Vinitha Sree, S., Swapna, G., Martis, R. J., & Suri, J. S. (2013). Automated EEG analysis of epilepsy: A review.
Knowledge-Based Systems, 45(Supplement C), 147–165. https://doi.org/10.1016/j.knosys.2013.02.014Benbadis, S. R., LaFrance, W. C., Papandonatos, G. D., Korabathina,
K., Lin, K., Kraemer, H. C., & For the NES Treatment Workshop. (2009). Interrater reliability of EEG-video monitoring. Neurology, 73(11), 843–846. https://doi.org/10.1212/WNL.0b013e3181b78425
Bergstrom, R. A., Choi, J. H., Manduca, A., Shin, H.-S., Worrell, G. A., & Howe, C. L. (2013). Automated identification of multiple seizure-related and interictal epileptiform event types in the EEG of mice. Scientific Reports, 3, 1483. https://doi.org/10.1038/srep01483
Carroll, S. B. (2010). Engaging assessment: Using the SENCER-SALG to improve teaching and learning. In R. D. Sheardy (Ed.), Science education and civic engagement: The SENCER approach (Vol. 1037, pp. 149–198). Washington, DC: American Chemical Society. https://doi.org/10.1021/bk-2010-1037.ch010
Denofrio, L. A., Russell, B., Lopatto, D., & Lu, Y. (2007). Mentoring: Linking student interests to science curricula. Science, 318(5858), 1872–1873. https://doi.org/10.1126/science.1150788
Fisher, R. S., Acevedo, C., Arzimanoglou, A., Bogacz, A., Cross, J. H., Elger, C. E., … Wiebe, S. (2014). ILAE official report: a practical clinical definition of epilepsy. Epilepsia, 55(4), 475–482. https://doi.org/10.1111/epi.12550
Halford, J. J., Pressly, W. B., Benbadis, S. R., Tatum, W. O., Turner, R. P., Arain, A., … Dean, B. C. (2011). Web-based collection of expert opinion on routine scalp EEG: Software development and interrater reliability. Journal of Clinical Neurophysiology, 28(2), 178–184. https://doi.org/10.1097/WNP.0b013e31821215e3
Hobin, J. A., Deschamps, A. M., Bockman, R., Cohen, S., Dechow, P., Eng, C., … Galbraith, R. (2012). Engaging basic scientists in translational research: Identifying opportunities, overcoming obstacles. Journal of Translational Medicine, 10, 72. https://doi.org/10.1186/1479-5876-10-72
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1
Kuh, G. D., O’Donnell, K., & Reed, S. (2013). Ensuring quality & taking high-impact practices to scale.Washington, DC: American Association of Colleges and Universities.
Lopatto, D., Alvarez, C., Barnard, D., Chandrasekaran, C., Chung, H.-M., Du, C., … Elgin, S. C. R. (2008). Undergraduate research. Genomics education partnership. Science, 322(5902), 684–685. https://doi.org/10.1126/science.1165351
Shorvon, S. D. (2011). The etiologic classification of epilepsy. Epilepsia, 52(6), 1052–1057. https://doi.org/10.1111/j.1528-1167.2011.03041.x
Tatum, W. O. (2013). How not to read an EEG: Introductory statements. Neurology, 80(1 Supplement 1), S1–S3. https://doi.org/10.1212/WNL.0b013e318279730e
US Department of Health and Human Services (USDHHS). (2015). Translational science spectrum. Retrieved January 17, 2018, from https://ncats.nih.gov/translation/spectrum
US Department of Health and Human Services (USDHHS). (2017). NIH categorical spending – NIH research portfolio online reporting tools (RePORT). Retrieved January 17, 2018, from https://report.nih.gov/categorical_spending.aspx