Assessment

Institution-Wide Analysis of Academic Outcomes Associated with Participation in UGR: Comparison of Different Research Modalities at a Hispanic-Serving Institution

Samantha Jude Battaglia, Gonzaga University

Lourdes E. Echegoyen, The University of Texas at El Paso

Laura A. Diaz-Martinez, The University of Texas at El Paso, Gonzaga University

Abstract

Most studies on the benefits of participation in undergraduate research (UGR) use data from student participants in undergraduate research programs (URPs), which offer a limited number of positions. In reality, however, the majority of UGR students participate in undergraduate research not in programs (URNPs). The authors conducted an institution-wide study at a Hispanic-serving institution to examine the relationship between academic success and participation in these two UGR modalities. Although there were some differences between URPs and URNPs, participation in research at this institution was largely equitable and inclusive, with UGR demographics that reflected those of the institution, and it was positively associated with increased benefits along multiple academic metrics, regardless of UGR modality. Importantly, these increases were observed for both first time in college and transfer students.

Keywords: undergraduate research, UGR, UGR experiences, research apprenticeship, high-impact educational practices, equity, inclusion

doi: 10.18833/spur/5/3/9

Studies on the benefits of undergraduate research (UGR) participation have uncovered important educational and professional development gains for undergraduate researchers, including career clarification (Craney et al. 2011; Frederick et al. 2021; Thiry, Laursen, and Hunter 2011; Zydney et al. 2002), student motivation toward their courses (Lopatto 2007), increased student retention (Baron et al. 2020; Fakayode et al. 2014; Simmons 2018), increased GPA (Baron et al. 2020; Brown et al. 2020; Collins et al. 2017; Fechheimer, Webber, and Kleiber 2011; Simmons 2018; Whittinghill et al. 2019), protection against GPA decreases (Baron et al. 2020; Brown et al. 2020), increased graduation rates (Baron et al. 2020; Galli and Bahamonde 2018; Hernandez et al. 2018; Whittinghill et al. 2019), increased interest in pursuing graduate degrees (Eagan et al. 2013; Haeger and Fresquez 2016), increased likelihood of enrollment into graduate programs (Bauer and Bennett 2003; Carter, Mandell, and Maton 2009; Eagan et al. 2013; Follmer et al. 2017; Hathaway, Nagda, and Gregerman 2002; Hernandez et al. 2018; Junge et al. 2010; Maton et al. 2009, 2016; Whittinghill et al. 2019; Wilson et al. 2018), and increased participation in the scientific workforce (Hernandez et al. 2018). UGR participation has also been associated with increases in skills such as critical thinking (Thiry et al. 2011; Zydney et al. 2002), problem-solving (Bauer and Bennett 2003; Hathaway et al. 2002), self-confidence/self-efficacy (Adedokun et al. 2013; Estrada, Hernandez, and Schultz 2018; Haeger and Fresquez 2016; Loeser et al. 2021; Thiry et al. 2011), and communication (Monarrez et al. 2020; Zydney et al. 2002).

Importantly, UGR participation has been shown to be particularly impactful for students from underrepresented minority (URM) groups. Participation in UGR for URM students has been associated with persistence (Espinosa 2011; Simmons 2018), higher GPAs (Whittinghill et al. 2019), higher likelihood of reporting plans to pursue graduate degrees in science, technology, engineering, and mathematics (STEM; Eagan et al. 2013), enrollment in further education (Hathaway et al. 2002; Hernandez et al. 2018; Maton et al. 2009, 2016), entrance into science PhD programs (Whittinghill et al. 2019), and continuation into (Hernandez et al. 2018) and success in STEM careers (Martinez et al. 2018). Although many studies have reported gains associated with participation in UGR for students from URMs, only a few studies have focused on Hispanic-serving institutions (HSIs) (Baron et al. 2020; Brown et al. 2020; Collins et al. 2017; Loeser et al. 2021; Simmons 2018) or on student populations with a high percentage of Latinx/Hispanic students (Estrada et al. 2018; Hernandez et al. 2018).

Furthermore, most research on the benefits of UGR has focused only on students who participated in structured UGR programs, a majority of which are for STEM majors (Haeger et al. 2020). Structured UGR programs typically select students on a competitive basis, offer a scholarship or stipend, and immerse undergraduate researchers in a series of professional development activities (e.g., workshops, research presentations) in addition to research opportunities. However, most institutions also offer a diverse range of non-programmatic, less structured, apprentice-style research experiences, such as departmentally supported research assistantships, for-credit research opportunities (e.g., capstone courses, independent studies courses, honors thesis), and volunteer opportunities. Importantly, students in research apprenticeships outside of structured programs typically outnumber those in structured programs because programmatic opportunities are limited by funding and are mostly available in the STEM disciplines. Thus, given the preponderance of UGR opportunities outside of structured programs, studying the benefits provided by these experiences and how these experiences compare with programmatic UGR experiences is imperative.

Efforts to investigate the benefits of UGR participation on a larger scale include institution-wide (Baron et al. 2020; Collins et al. 2017; Fechheimer et al. 2011; Simmons 2018; Whittinghill et al. 2019) and multi-institution (Estrada et al. 2018; Hernandez et al. 2018; Wilson et al. 2018) studies that have examined the relationship between UGR participation and gains in several metrics. These studies showed that UGR participation was associated with student retention (Simmons 2018), increased GPA (Baron et al. 2020; Collins et al. 2017; Fechheimer et al. 2011; Hernandez et al. 2018; Simmons 2018), degree completion (Baron et al. 2020; Hernandez et al. 2018), enrollment in further education, and participation in the scientific workforce (Estrada et al. 2018; Hernandez et al. 2018). However, these studies did not distinguish between the different modalities of UGR experiences (i.e., structured program versus no program).

Research on the effects of these different modes of UGR experiences is limited and has been identified as one of the questions about UGR that need further investigation (Haeger et al. 2020). Pioneering studies addressing these questions surveyed alumni who participated in undergraduate research programs (URPs) and a set of matched controls, some of whom reported participating in undergraduate research not in programs (URNPs; Bauer and Bennett 2003; Hathaway et al. 2002). Both of these studies were conducted at large, primarily white institutions, although the program at one of the institutions was specifically focused on engaging URMs in research (Hathaway et al. 2002). Both studies surveyed thousands of alumni to ask about perceptions of the effects of their undergraduate education, career plans, enrollment in further education, and other variables. These studies reported higher rates of enrollment in further education for students who participated in research either as URPs or URNPs, compared with students who did not participate in either research modality (Bauer and Bennett 2003; Hathaway et al. 2002). Interestingly, there were some differences in outcomes between URPs and URNPs. Although both URPs and URNPs continued into further education at similar rates, URPs had a higher rate of enrollment into professional education than URNPs (Hathaway et al. 2002). Undergraduate researchers, particularly URPs, also reported higher benefits in skills such as communication, locating and analyzing information, and clarification of career goals (Bauer and Bennett 2003).

Given the large spectrum of institutions (e.g., primarily undergraduate, comprehensive, Historically Black, Hispanic-serving), there appears to be a lack of institution-wide studies that investigate the relationship between participation in the different modalities of UGR and academic outcomes. This is particularly true for institutions with student populations that are predominantly URMs and particularly at HSIs. A major hurdle to conducting these institution-wide studies has been the difficulty in tracking student participation in research at the institutional level, which is in part due to the multitude of segregated programs and research modalities at most institutions. In 2014, The University of Texas at El Paso (UTEP) approved a new UGR course that allows effective tracking of all modalities of apprentice-style UGR in a single course. Students conducting UGR or scholarly or creative activities (from here on referred to collectively as UGRs) under the mentorship of a faculty member enroll in this zero-credit course regardless of the modality of their research opportunity (e.g., research as part of a structured program, paid research assistantship, unpaid research experience, for-credit research experience). The existence of this institutional tracking mechanism allowed us to conduct an institution-wide study on the effects of UGR participation that is inclusive of all disciplines and all modalities of research apprenticeships.

Thus, this article examines the relationship between participation in different UGR modalities and academic outcomes at the institutional level using institutional academic metrics. This article sought to address the following four research questions:

1. Is participation in different UGR modalities associated with increased academic performance?

2. Is participation in different UGR modalities associated with increased graduation rates?

3. Is participation in different UGR modalities associated with increased enrollment in further education?

4. Are there differences in these outcomes between first time in college (FTC) and transfer students?

Methodology

Institutional Context

UTEP is a large, public, HSI with an undergraduate student population that is largely Hispanic (>80 percent). It is classified as a doctoral university with very high research activity and high undergraduate enrollment, and it serves a student population that is highly financially disadvantaged and primarily nonresidential. UTEP was selected for this study because of the novel implementation of a zero-credit UGR course as a way for students, faculty, and the institution to track participation in UGR. Since implementation, undergraduate students were able to enroll in the course every semester they conducted research under the mentorship of UTEP faculty. By nature of it being a zero-credit course, enrollment did not interfere with credit requirements for different majors nor impose an added tuition cost for students. Importantly, this course was open to students from all majors on campus. The only requirement for enrollment was approval from the faculty member who served as the research mentor of the student.

UGR Modalities

At UTEP, undergraduate students can engage in research via two general mechanisms: through research apprenticeships under the direct mentorship of a faculty member and through course-based undergraduate research experiences (CUREs). CUREs are research courses embedded within the curricula of some majors (Auchincloss et al. 2014) and they were not included in this study. This study focuses on the effect of apprentice-type research experiences. Students who engaged in research apprenticeships were classified into two groups: URPs and URNPs. URPs typically undergo a competitive selection process, and once they are accepted into a program, they conduct research with a faculty adviser, receive a stipend or scholarship, and participate in a series of structured program activities (e.g., professional development workshops, special seminars, meetings with other program participants, presenting their research in the campus UGR symposium). URNPs typically find research opportunities directly with the faculty advisers. Some URNPs receive a stipend if the faculty member has the independent funding to provide it, but the majority of URNPs do not, as they are either volunteer researchers or participate in research for credit. URNPs at UTEP are not required to participate in structured activities but are invited to participate in some activities organized by some UGR programs and/or the Campus Office of UGR Initiatives (COURI). Importantly, both URPs and URNPs are required to register in the zero-credit UGR course every semester that they conduct research.

Research Design

This was a retrospective quasi-experimental study that compared academic outcomes for four groups of students: URPs, URNPs, and their corresponding matched control groups of students, mURPs and mURNPs, respectively, who did not participate in apprentice-type research opportunities. This research project was reviewed and approved by UTEP’s Institutional Review Board (IRB Reference # 1289740-1).

Subjects and Cohort Description

Institutional data collected from two cohorts were analyzed to define the subjects in each group for this study. Cohort 1 consisted of all incoming undergraduate students for the fall of 2013 (N = 4809). Cohort 2 consisted of all incoming undergraduate students for the fall of 2014 (N = 4840). The URP and URNP groups were defined as follows. First, enrollment in the zero-credit UGR course was used to determine which students in these cohorts conducted research at any point during the 2013–2018 period. Next, records provided by COURI were used to determine which students participated in any of the 13 UGR programs that existed at the institution between 2013–2018. These students constitute the URP group. It is important to note that 11 of the 13 UGR programs available on campus during the time of the study only offered research opportunities for students in STEM majors. For the purpose of this study, STEM majors are defined following the classification used by the National Science Foundation (Fiegener 2015), which includes social, behavioral, and economic sciences.

Students who registered for the research course but did not participate in any of the research programs were included in the URNP group. In addition to URP and URNP students, a third set of students classified as research assistant student employees but not enrolled in the research course was identified. Since these students had not registered for the zero-credit UGR course, the authors could not accurately determine if they had conducted research or not. Therefore, this group was excluded from the study (n = 53 for cohort 1, n = 33 for cohort 2). Students in the two matched control groups, mURP and mURNP, were identified by one-to-one matching to students in the URP and URNP groups as described in the “matching for control groups” section. The process to identify the study groups is summarized in the schematic shown in Figure 1A.

Descriptive Variables Used in the Study

The following variables were used to establish demographic characteristics as well as baseline indicators for matching subjects to their controls. Grade Point Average in the first term (GPA 1st Term) and major were used as baseline academic indicators. Receipt of a federal Pell grant was used as the indicator for socioeconomic status. Federal Pell grants are awarded to students with severe financial need on a noncompetitive basis. Pell grant eligibility requires the applicant to be a citizen of the United States with demonstrable financial need and enrolled in an undergraduate degree at a non-foreign institution (Department of Education 2021). It is important to note that a significant percentage of subjects had “no data” for this variable, and the reason for this could be not submitting a Pell grant application or being ineligible for these types of grants (e.g., international students). Gender was obtained from student registration records and was based on self-identification by the students given a binary choice (female, male). Race and ethnicity were also obtained from student registration records and were based on self-identification by the students. For the purposes of this study, the term URM refers to racial/ethnic groups underrepresented in STEM as defined in the United States Code (U.S. Government Publishing Office 2011). Students were grouped into five categories based on race/ethnicity and were coded as follows: the “non-URM” category included students who identified as White non-Hispanic and Asian Americans; the “Hispanic” category includes students who identified as Hispanic of any race; the “Other URM” category grouped students who identified themselves as Black non-Hispanic, American Indian, Alaskan Native, Native Hawaiian or other Pacific Islander, and two or more races; the “unknown” category included students who provided no information about race/ethnicity and the “international” category grouped all international students except permanent residents and students with visas to domicile, who are included in the race/ethnicity categories above.

Study Variables

The independent variable used in this study is the type of UGR experience: URP, URNP, and their respective matched control groups, mURP and mURNP. For the analysis of subjects by entry status, the students were classified as FTC or transfer student based on whether the participant transferred zero (FTC) or any (transfer) semester credit hours (SCH) from another postsecondary institution when enrolling at the institution. The dependent variables used are commonly used metrics of academic achievement: number of SCH attempted and earned, cumulative GPA, four-year and five-year graduation rates, and enrollment in further education after graduation. SCH attempted is a measure of the course load students enrolled in at the institution. SCH earned represents the courses students completed. Cumulative GPA was the GPA earned by the student from the time the student first enrolled at the institution (fall of 2013 for cohort 1; fall of 2014 for cohort 2) to the end of academic year 2017–2018 (summer of 2018) or graduation if the student graduated before summer of 2018. Four-year graduation was defined for cohort 1 as graduation at or before the end of academic year 2016–2017 and for cohort 2 as graduation at or before the end of academic year 2017–2018. Five-year graduation for cohort 1 was defined as graduation at or before the end of academic year 2017–2018. Five-year graduation for cohort 2 could not be calculated because the data provided for this cohort only spanned four years. Enrollment in further education was based on enrollment verification provided to the institution by the National Student Clearinghouse.

Statistical Analysis

Institutional data were entered and analyzed using SPSS and JMP. Categorical variables were initially coded in SPSS and then imported into JMP for further analysis. For continuous numerical variables (e.g., cumulative GPA, SCH), means were analyzed using one-way ANOVA, followed by all-group comparisons using the Tukey-Kramer multiple comparison test. For categorical variables (e.g., graduation, enrollment into further education), between-groups chi-square analysis was performed using the likelihood ratio test.

Control Group Matching

Because acceptance in research programs is determined through competitive selection, and participation in UGR as a volunteer is subject to volunteer bias, the authors first sought to identify whether the probability of participation in research was associated with any of the descriptive variables. To do this, a logistic analysis was performed using the nominal logistic fit function in JMP with research participation (Yes/No) as the dependent variable to test association with five descriptive variables: GPA 1st Term, gender, socioeconomic status, race/ethnicity, and major. This analysis was conducted using the two full cohorts (Cohort 1 N = 4809; Cohort 2 N = 4840), and it revealed two variables had the largest association with participation in research: GPA in the first term and major (see Table 1).

Importantly, the analysis also showed a slight association between research and race/ethnicity. However, the effect size was <2 for both cohorts, which was not significant for this test (α > .01). The association between research participation and GPA was expected because participation in research, particularly in structured programs, is a highly selective process that typically favors students with higher GPAs. Similarly, a bias in terms of major was expected because 11 out of the 13 UGR programs included in this study were only open to students in STEM majors. These differences underscored the need to identify a control group of subjects that were matched for these two variables.

Case-control matching in SPSS was used to match subjects in the URP and URNP groups to subjects not in these groups. Subjects were matched for GPA in the first term (with a ±0.5 fuzz) and major because these variables showed significant effects in the logistic regression, indicating selection biases for research participation associated with these variables, as indicated previously. Since exact matches by major were not available for all subjects, majors were given numerical codes, with similar majors having more similar numerical codes than more dissimilar majors. Matching was then performed for major with a ±0.5 fuzz. All matching was performed without replacement, giving priority to exact matches and randomizing the case order when drawing matches with a random number seed of 10. Using this protocol, the authors were able to match a great majority of the URP and URNP subjects in both cohorts (cohort 1: URP n = 63, URNP n = 187, exact matches = 175, fuzzy matches = 74, unmatched subjects = 1; Cohort 2: URP n = 87, URNP n = 203, exact matches = 182, fuzzy matches = 106, unmatched subjects = 2). The three unmatched subjects were included in the study.

Results

The overall goal of this study was to perform an institution-wide analysis of the relationship between participation in different modalities of apprentice-type UGR experiences and academic outcomes. As described in the methodology section, the groups compared in this study were URPs, URNPs, and their respectively matched control groups, mURPs and mURNPs. URPs conducted research as part of structured programs, received a scholarship or stipend, and were required to participate in other programmatic activities (e.g., professional development workshops, meetings with program directors and peers, presentations at conferences). URNPs conducted research but were not part of a structured program and were invited, but not required, to participate in any additional programmatic activities. Subjects in the matched control groups were selected by one-to-one matching of URPs and URNPs to non-researchers in the same cohort who had a similar entry status (transfer or FTC), similar GPAs in their first term at the institution, and similar declared majors (see Figure 1A for a schematic summary of the matching process and the matching for control groups section in the methodology for details on the matching procedure).

Verification of Matching Variables

First, the validity of the matching strategy was assessed by comparing the four study groups on the variables used for matching. When compared for entry status, the distribution of transfer and FTC students was similar across URP, URNP, and matched control groups for cohort 1 (see Figure 1B; χ2[DF = 6, N = 499] = 3.969, p = .6808). For cohort 2 there were significant differences between the URP and URNP groups (χ2[DF = 3, N = 578] = 5.1855, p = .0157), with the URNP group having a larger percentage of transfer students (29.06 percent) than the URP group (17.24 percent). This difference was mirrored in their respective matched control groups (mURNP 29.21 percent versus mURP 16.28 percent), indicating the matching process was successful for this variable. It is important to note that the percentage of transfer students participating in research, in either modality, is lower than the population of transfer students at the institution (37.62 percent in cohort 1, 37.10 percent in cohort 2). These results indicate that transfer students are less likely to participate in research and suggest a need for the development of recruitment strategies that specifically target this underserved group.

Next, the matching of the four groups on their baseline GPA (GPA in their first term at the institution; see Figure 1C) was verified. A difference was observed between URP and URNP students in both cohorts with the median GPA 1st term of URPs being 0.25 and 0.22 higher than URNPs in cohorts 1 and 2, respectively. Since UGR programs select students on a competitive basis, this finding is not surprising. Importantly, the baseline GPAs for URPs and URNPs were comparable to their respective matches (mURP and mURNP), indicating that the matching was successful at capturing the baseline GPA differences between URPs and URNPs.

The third variable used for matching was the first selected major. As stated in the methodology, 11 of the 13 UGR programs were only open to students in STEM majors. Thus, the majority of URPs (92.07 percent in cohort 1, 87.36 percent in cohort 2; see Table 2) were students majoring in STEM fields. Interestingly, a majority of URNPs were also students majoring in STEM (83.42 percent in cohort 1, 84.24 percent in cohort 2). The matched groups reflected the major distribution of URPs and URNPs (see Table 2), indicating that the matching by major was successful. Importantly, these data also indicate a severe underrepresentation of non-STEM majors in research since the distribution of STEM majors as a percentage of all undergraduate students in each cohort is much smaller (37.50 percent in cohort 1, 37.42 percent in cohort 2). Different scenarios could explain this. For example, lack of identification of scholarship and creative work with the term “research” can lead to students conducting scholarly work or creative activities to not register for the UGR course. It is also possible that faculty in non-STEM disciplines mentor fewer students than faculty in STEM fields, resulting in fewer opportunities for non-STEM majors to engage in research. Overall, these results indicate a need to increase awareness of and opportunities for research in non-STEM fields.

Participant Demographics

UTEP is an HSI with an undergraduate student population that is majority Hispanic (>80 percent) and a high percentage of financially disadvantaged students. To determine whether any differences existed in demographic variables among the groups included in this study, the demographic characteristics of students who engage in research were analyzed. First, the distribution of students based on socioeconomic status was analyzed using receipt of the need-based federal Pell grant as the indicator. (A description of eligibility for federal Pell grants is provided in the methodology.) This analysis showed that the percentage of financially disadvantaged students was similar across all four groups, with 60 percent or more of the students receiving federal Pell grants (see Figure 2A; cohort 1 χ2[3, N = 488] = 3.078, p = .3798; cohort 2 χ2[3, N = 479] = 4.786, p = .1882). It is important to note that students reporting “no data” for this variable were excluded from this chi-square analysis, but were included in the graphs (Figure 2A) and other analyses. Although the percentages of financially disadvantaged students participating in research are similar to their matched groups, they are slightly different from the institutional levels. The percentage of financially disadvantaged URNPs is 4.9 and 7.3 percent lower than the institutional levels for cohorts 1 and 2, respectively. For URPs the difference depends on the cohort, with cohort 1 having 4.44 percent more financially disadvantaged students than the institution as a whole (74.60 percent of URPs versus 70.16 percent for the institution), whereas the level of financially disadvantaged students in cohort 2 is 6.98 percent lower than the institution (60.91 percent of URPs versus 67.89 percent for the institution). Taken together, these results suggest that financial burden might be a factor deterring participation of undergraduate students in research, particularly outside of programs that offer stipends. Interestingly, the matched groups also show a slightly lower (0.07–3.93 percent) percentage of financially disadvantaged students compared to institutional levels. This similarity is interesting because the groups were matched on baseline GPA, major, and entry status, but not on financial status. These results indicate that although financially disadvantaged students engage in research at UTEP in large numbers, there is a slight underrepresentation of financially disadvantaged students in research compared to the institution as a whole, particularly for students engaging in research outside of programs (URNPs). Importantly, this underrepresentation can be overcome by providing financial support for researchers as suggested by the data in the URP group, cohort 1.

Next, the distribution of students by race and ethnicity across all groups was analyzed. All four groups showed a relatively similar distribution of students from racial and ethnic groups that are traditionally underrepresented in STEM (see Figure 2B). Note that chi-square analysis could not be computed because some of the categories had very low numbers of subjects. The percentage of URMs (Hispanic and other URM categories) in the URP group (84.1 percent for cohort 1; 80.4 percent for cohort 2) and URNP group (81.8 percent for cohort 1; 71.9 percent for cohort 2) is close to but slightly lower than the percentage for the institution as a whole (87.9 percent for cohort 1; 85.3 percent for cohort 2). Interestingly, this underrepresentation of URMs in research is largely due to an overrepresentation of international students, since the percentages of international students are higher in the URP (9.5 percent for cohort 1; 9.19 percent for cohort 2) and URNP groups (9.1 percent for cohort 1; 18.2 percent for cohort 2) than in the institution as a whole (3.6 percent for cohort 1; 6.44 percent for cohort 2). Importantly, the expectation was to see a larger proportion of international students among URNPs compared to URPs because the majority of the UGR programs at the institution (11 out of 13) were funded by grants from US federal agencies, which fund only US citizens and residents. This was the case for cohort 2. Surprisingly, the proportion of international students in the URP and URNP groups was similar for cohort 1. These data indicate that international students engage in all modalities of research at proportionally higher rates than expected, despite opportunities for international students to participate in research programs being limited to nonfederally funded programs financially supported by the on-campus student employment program.

Lastly, the gender distribution across all groups was analyzed (see Figure 2C). Importantly, the available records for student gender identity are binary (i.e., male or female), forcing any student who identifies as nonbinary to choose one of the two options. Analysis of these data showed no difference in gender distribution across the four groups for cohort 2 (χ2[3, N = 578] = 0.969, p = .8088) and a slight difference for cohort 1 (χ2[3, N = 499] = 8.051, p = .0450). This difference is due to a higher representation of females in URNP and mURNP compared with URP and mURP. It is unclear why this gender difference exists in cohort 1 but since this difference is not observed in cohort 2, this is unlikely the result of gender bias in the different research modalities.

Overall, these results indicate that participation in research at UTEP is reasonably equitable in terms of socioeconomic status, gender, race, and ethnicity. However, there is a slight underrepresentation of financially disadvantaged students and an overrepresentation of international students among research participants compared to the institution as a whole. A more detailed analysis is needed to determine the potential causes of these differences.

Participation in Either Research Modality Is Associated with Higher Earned SCH

To begin the analysis of whether participation in UGR is associated with increased academic performance, the authors asked whether there were any differences in the number of SCH earned and the ratio of SCH earned/attempted among the study groups. One-way ANOVA analysis identified significant differences among the groups for these metrics (see Figure 3A–B). Subsequent comparison between the groups via Tukey-Kramer analysis revealed that in both cohorts the average number of SCH earned was significantly greater for research participants (21.33–37.57 percent SCH higher) compared with their non-research matches (URP versus mURP, URNP versus mURNP; Table 3). Similarly, when comparing the average ratio of SCH earned/attempted, URNPs had significantly greater averages than mURNPs for both cohorts. URPs also showed an increased average of SCH earned/attempted when compared to mURPs, albeit this difference was only statistically significant for cohort 2 (see Table 3). These results indicate that participation in research is associated with higher SCH earned and a higher ratio of SCH earned/attempted, particularly for students in the URNP group. Importantly, comparisons between URP and URNPs revealed that these groups earned SCH at similar levels (see Table 3). These results indicate that participation in either modality of research is associated with higher earned SCH.

Participation in Either Research Modality Is Associated with Higher Cumulative GPA and Protection from GPA Decline

Next, cumulative GPA was used as another measure of academic achievement. One-way ANOVA analyses showed that URPs and URNPs earned higher cumulative GPAs than their respectively matched groups (see Figure 3C and Table 3). On average, URPs earned cumulative GPAs that were 0.3794 (cohort 1) and 0.4097 (cohort 2) higher than mURPs. Similarly, URNPs earned cumulative GPAs 0.2118 (cohort 1) and 0.2127 (cohort 2) higher than mURNPs. An important caveat is that URPs undergo stringent selection to be admitted into their research programs that can include evaluation of their GPAs. As shown in Figure 1C, URPs have a significantly higher baseline GPA than URNPs. Thus, the authors reasoned that a more accurate way to measure the effect of participation in research on GPA attainment is to analyze the change in GPA over time (GPA change = cumulative GPA − baseline GPA) rather than cumulative GPA alone. Typically, college students show a decrease in GPA over time (Sell, Naginey, and Stanton 2018). As expected, non-research students (mURP and mURNP) showed a strong decrease in GPA ranging from −0.2494 points to −0.4862 points (see Table 3). Interestingly, the decline in GPA was nearly abolished or even transformed into a slight gain in GPA for students engaged in research, regardless of modality (GPA change ranging from −0.0192 to +0.088).

Comparisons between URP and URNP indicated that although the average cumulative GPA for URPs is higher than that of URNPs, this difference is only statistically significant for cohort 2 but not for cohort 1 (see Table 3). This finding is particularly striking since these two groups showed a statistically significant difference in baseline GPA (see Figure 1C). Similarly, no major difference is observed between the URP and URNP groups in terms of GPA change (see Figure 3D and Table 3), indicating that both types of research opportunities are associated with protection against GPA decline.

Although the results above indicate that participation in UGR experiences, regardless of the type of experience, is associated with higher average GPAs and might protect students from the typical GPA decline observed in college, the possibility that these results are affected by the selective nature of research experiences cannot be ruled out. This could be the case when faculty and program selection committees preferentially select students who consistently maintain a high GPA for their research positions.

Participation in Either Research Modality Is Associated with Increased Graduation Rates

The second question of this study was whether participation in the different types of UGR opportunities is associated with increased graduation rates. To address this question, the four-year and five-year graduation rates for the study groups were compared (see Figure 4A–B). Note that five-year graduation data were only available for cohort 1, as cohort 2 was just beginning its fifth year when the data were collected (see methodology). A positive association between research participation and graduation rate was observed when comparing URPs and URNPs to their respective matched groups. The four-year graduation rates for URPs were 19.04–20.28 percent higher than mURPs (see Figure 4A) and this difference widened to 26.98 percent when looking at the five-year graduation for cohort 1 (see Figure 4B). Similarly, URNPs had 4-year graduation rates that were 26.61 percent and 28.91 percent higher than mURNPs (see Figure 4A) and this difference also widened to 32.92 percent after five years (see Figure 4C). These results indicate that students involved in either modality of research graduated at much higher rates than students who did not participate in research. Interestingly, the difference was more pronounced for URNPs than for URPs.

Participation in Either Research Modality Is Associated with Increased Enrollment into Further Education

The third question of this study was whether participation in the different types of UGR opportunities is associated with increased enrollment in further education after graduation. To address this question, data from the National Student Clearinghouse on enrollment in further education were analyzed for those students who had graduated by the summer of 2018. That is, students who graduated within five years of enrollment for cohort 1 and students who graduated within four years of enrollment for cohort 2. This analysis showed a positive association between participation in either modality of research and enrollment into further education. Both URPs and URNPs had higher rates than their control groups (see Figure 4C), however, the magnitude of the effect was variable across cohorts and comparison groups. Note that for these analyses, research modalities were compared within each cohort separately and comparisons across cohorts were avoided because the numbers of students enrolled in further education are significantly different between cohort 1 and cohort 2, given that students from cohort 1 have had an additional year to continue their studies. Clear increases were observed when comparing URNP versus mURNP within each cohort, with URNPs enrolling at rates 23.34 percent and 17.17 percent higher than mURNPs for cohorts 1 and 2, respectively. URPs also enrolled in further education at higher rates than mURPs but the differences were smaller (12.94 percent in cohort 1, 1.25 percent in cohort 2). These results indicate that although the rate of enrollment in further education is higher for both URPs and URNPs than their respective controls, participation in research for students in the URNP group is associated with a greater effect on their likelihood of pursuing further education than for the URP students.

Participation in Either Research Modality Is Associated with Increased Academic Achievement for Both FTC and Transfer Students

Lastly, the authors asked whether the association between research participation and increased academic achievement was observed in both FTC and transfer students. Similar to the earlier comparisons, graduation rates, cumulative GPA, change in GPA, and enrollment in further education were analyzed for FTC and transfer students. For FTC students, participation in either modality of research was associated with increased four-year graduation rates (see Figure 5A), increased cumulative GPA (see Figure 5B), and increased protection against GPA decline (see Figure 5C). Participation in either research modality was also associated with an increased likelihood of enrollment in further education for graduates in cohort 1, but a similar increase is only seen in URNPs compared with mURNPs for cohort 2 (see Figure 5D).

Interestingly, the results for transfer students are more complex. The four-year graduation rates for transfer students are higher in URNPs compared to mURNPs in both cohorts, and for URPs compared to mURPs in cohort 2, but not in cohort 1 (see Figure 5A). It is important to note that the number of transfer students in the URP and mURP groups (N = 6–13) is smaller than for the URNP and mURNP groups (N = 21–49). Therefore, these data must be interpreted with great caution because of the low statistical power of the groups and because the matched controls for transfer students were not matched for the number of credits transferred. Thus, the relationship between research participation and graduation for transfer students is difficult to ascertain from these data.

Similar to FTC students, transfer student researchers attained higher cumulative GPAs than their controls in both cohorts (see Figure 5B), but the difference is statistically significant only for the comparison between the URNP group and its control group (mURNP). Importantly, the protective effect of research participation against GPA decline is even stronger in transfer students than in FTC students. Transfer students who participated in either research modality actually increased, rather than decreased their GPA (positive GPA change), whereas transfer students who did not engage in research showed GPA decline (see Figure 5D). Lastly, increased enrollment in further education after graduation was observed in URNPs compared to mURNPs for both cohorts of transfer students, but was not observed in URPs compared to mURPs (see Figure 5J). Again, these differences are difficult to interpret due to the small number of transfer students in the study.

Overall, these results suggest that both FTC and transfer students benefit from participation in research, although some differences between the two groups were found. Interestingly, the protective effect of research participation against GPA decline seems to be greater in transfer students compared to FTC (see Figure 5C), to the point that transfer students who engaged in research actually showed gains in GPA over time, rather than declines. In addition, the positive associations observed in transfer students seem to be more marked for URNP than for URP students. However, the numbers of transfer students participating in programs (URPs) are so small that these statistical analyses must be interpreted with great caution. These results, together with the disparities observed in participation of transfer students in research (see methodology), highlight the need to increase participation of transfer students in research as well as to conduct further research on the effects of research participation on transfer students.

Limitations of This Study

This was an institution-wide analysis of the relationship between participation in different research modalities and academic outcomes. However, despite the fact that the zero-credit UGR course used to track participation is open to all majors, the majority of participants were majoring in STEM. Thus, further research is needed to determine whether the benefits identified are applicable to both STEM and non-STEM majors. In addition, the analyses of transfer students must be interpreted with great caution due to the low numbers of transfer students in the study (N = 126 for cohort 1; N = 147 for cohort 2). Additionally, the study is limited to two cohorts of students. The two cohorts were analyzed separately to determine whether the results were consistent in both cohorts. This has been the case in some instances (e.g., increased graduation rates, increased enrollment in further education, increased protection against GPA decline) but in other instances, the results differ between cohorts. An extended analysis of additional cohorts is needed to determine whether these relationships are maintained over multiple cohorts. Similarly, this study analyzed data encompassing a five-year period for cohort 1 and a four-year period for cohort 2. Thus, other important metrics such as six-year graduation could not be calculated. In addition, the rate of enrollment in further education represents enrollment that occurred immediately after graduation, particularly for cohort 2. Since cohort 1 had an additional year at the time the data on enrollment in further education were obtained, the analysis on this metric is limited to within cohort comparisons. Given that many students take a gap year before applying for advanced degrees, it would be interesting to analyze these outcomes over a longer period of time.

Discussion

The goal of this study was to analyze the relationship between participation in different apprenticeship-style research modalities and academic outcomes at a large HSI. Institutional academic metrics (e.g., GPA, graduation rate) were utilized as a measure of academic success. Although these metrics undoubtedly represent a limited view of academic success, using institutional metrics has the advantage of allowing us to perform an institution-wide analysis that is not affected by survey response rates or subjective self-reporting. Overall, the data showed that participation in UGR is associated with higher academic achievement and that this association is largely independent of the type of research opportunity.

The analysis showed that participation in UGR is associated with increased earned SCH (Figure 3A–B), higher cumulative GPAs (Figure 3C), lower GPA declines (Figure 3D), and increased rates of four-year and five-year graduation (Figure 4A–B) for URPs and URNPs compared with matched non-researchers. These results agree with reports made by others showing that UGR participation is associated with increased academic success using these metrics. Because this analysis compared URPs to a matched control group of non-URPs who performed similarly during the first semester, these data suggest that the UGR experience provided an academic advantage for URPs. However, this study cannot directly ascertain whether this is a direct causal relationship between UGR participation and increased academic success, since there are other confounding factors that could influence student participation in UGR or student success that were not controlled for in this study. Qualitative and quantitative studies that include and analyze the effect of additional variables (e.g., intrinsic motivation leading to self-selection, academic self-confidence, research dosage, availability of financial support, quality of mentorship, family expectations) are needed to further define the contribution of UGR to student academic success and, perhaps more importantly, which aspects of the UGR experience are key for this contribution. As this study shows, institutions of higher education already collect a trove of student data that could potentially be leveraged to answer some of these questions at an institutional scale.

Importantly, this study extends the findings of a correlation between UGR and academic success by addressing whether these positive associations are dependent on the type of UGR experience.

Comparisons between the URP and URNP groups for SCH earned, SCH earned/attempted, cumulative GPA, and GPA change showed that these groups reach similar levels of achievement. This observation is striking considering the initial differences observed in baseline GPA between URPs and URNPs (Figure 1C). To address if there were differences between the research modalities, the differences in achievement between URPs and mURPs were compared to the differences between URNPs and mURNPs. Interestingly, URNPs versus mURNPs had a larger difference in all numerical metrics in cohort 1 than URPs versus mURPs (compare URP versus mURP to URNP versus mURNP columns in Table 3), but in cohort 2 the difference is reversed. Thus, in terms of SCH earned, cumulative GPA, and GPA change there is no consistent difference between the two research modalities. However, when graduation rates and enrollment in further education were looked at, the differences for URNPs versus mURNPs were consistently larger than those for URPs versus mURPs in both cohorts. For graduation rate, URNPs graduated at a rate 32.92 percent higher than mURNPs for cohort 1 (five-year graduation) and 28.91 percent higher for cohort 2 (four-year graduation), whereas URPs graduated at a rate 26.98 percent higher than mURPs for cohort 1 and 26.61 percent higher for cohort 2. Similarly, enrollment into further education was 23.34 percent higher for URNPs than mURNPs for cohort 1 and 17.17 percent higher for cohort 2, whereas that of URPs versus mURPs was only 12.94 percent higher for cohort 1 and 1.25 percent for cohort 2. These results suggest that there might be differences in the magnitude of the effect depending on the type of research opportunity. Given that URPs and mURPs have a higher baseline GPA than URNPs and mURNPs, the higher benefits observed in URNPs after accounting for differences in the control groups might indicate that students with slightly lower baseline GPAs benefit more from research experiences than students with higher baseline GPAs. However, more research is needed to elucidate the nature of these differences.

Taking all these results into account, this study suggests that the benefits associated with participation in UGR are largely independent of the type of research modality, although small differences might exist. It is important to note that these results are not evidence of a lack of additional benefits from participation in research programs compared to research outside of structured programs. Others have reported higher benefits in the development of a variety of skills for URPs compared with URNPs (Bauer and Bennett 2003) and higher SCH earned for URPs who participate in additional program activities like workshops (Baron et al. 2020), indicating that participation in structured research programs might offer benefits that are not directly captured by the metrics used in this study. In addition, one major benefit of UGR programs is that they typically offer financial support in the form of scholarships or stipends. The initial analysis of demographic characteristics of undergraduate researchers in this study identified a slight underrepresentation of financially disadvantaged students in research (see Figure 2A), suggesting that financial constraints might be a deterrence factor for research participation. The importance of financial support to overcome this barrier is evidenced by the increased participation of financially disadvantaged students in the URP group in cohort 1, which is the only group with a higher percentage of financially disadvantaged students than the institution. These results highlight the key role that UGR programs can potentially play in ensuring equitable research participation for financially disadvantaged groups and is consistent with prior research indicating that these types of programs are an important tool for diversification of the STEM fields (Wilson et al. 2018).

In addition, this study suggests positive associations between research participation and increased academic success for both FTC and transfer students (see Figure 5). However, the results from the analysis of transfer students should be interpreted with caution because of the comparatively low number of students in this group. These results, together with the finding that transfer students are underrepresented in research (see Figure 1B), highlight the need to enhance efforts to engage the transfer student population in UGR experiences.

Overall, the results of this study at a high Hispanic enrollment HSI show a high degree of equity and inclusion since the demographics of students participating in UGR closely reflect those of the institution. A positive association between participation in research and higher academic achievement was also observed when undergraduate researchers were compared to matched peers who did not participate in research. Although the benefits seem to be largely independent of the modality of the research experience, these results also suggest that slight differences might exist between the different research modalities.

Acknowledgments

The authors wish to thank Dr. Jeffrey Olimpo for helpful discussions and suggestions to improve the manuscript. Thanks also to Dr. Karina Canaba, Joe Ramos, and other staff at UTEP-COURI for kindly providing data on UGR participation. Thanks to Dr. Cathe Lester and staff at the Center for Institutional Evaluation Research and Planning for their help in obtaining participant academic data.

Author Contributions

LADM and LEE designed the study. LADM and SB performed the data analysis and wrote the manuscript.

Conflict of Interest

LADM served as Associate Director for COURI at UTEP from 2015–2019. LEE is the founding Director of COURI. Of the 13 UGR programs included in this study, 7 were implemented and managed by COURI. This research project was reviewed and approved by UTEP’s Institutional Review Board (IRB Reference # 1289740-1).

References

Adedokun, Omolola A., Ann B. Bessenbacher, Loran C. Parker, Lisa L. Kirkham, and Wilella D. Burgess. 2013. “Research Skills and STEM Undergraduate Research Students’ Aspirations for Research Careers: Mediating Effects of Research Self-Efficacy.” Journal of Research in Science Teaching 50(8): 940–951. doi: 10.1002/tea.21102

Auchincloss, Lisa Corwin, Sandra L. Laursen, Janet L. Branchaw, Kevin Eagan, Mark Graham, David I. Hanauer, Gwendolyn Lawrie, et al. 2014. “Assessment of Course-Based Undergraduate Research Experiences: A Meeting Report.” CBE—Life Sciences Education 13(1): 29–40. doi: 10.1187/cbe.14-01-0004

Baron, Sheila I, Pamela Brown, Tammie Cumming, and Michelle Mengeling. 2020. “The Impact of Undergraduate Research and Student Characteristics on Student Success Metrics at an Urban, Minority Serving, Commuter, Public Institution.” Journal of the Scholarship of Teaching and Learning 20(1): 85–104. doi: 10.14434/josotl.v20i1.25423

Bauer, Karen W., and Joan S. Bennett. 2003. “Alumni Perceptions Used to Assess Undergraduate Research Experience.” The Journal of Higher Education 74(2): 210–230. doi: 10.1353/jhe.2003.0011

Brown, Daniel A., Nina B. Wright, Sylvia T. Gonzalez, Nicholas E. Weimer, and Julio G. Soto. 2020. “An Undergraduate Research Approach That Increased Student Success at a Hispanic-Serving Institution (HSI): The SURE Program at Texas State University.” Scholarship and Practice of Undergraduate Research (SPUR) 4(1): 52–62. https://eric.ed.gov/?id=EJ1275694

Carter, Frances D., Marvin Mandell, and Kenneth I. Maton. 2009. “The Influence of On-Campus, Academic Year Undergraduate Research on STEM Ph.D. Outcomes: Evidence From the Meyerhoff Scholarship Program.” Educational Evaluation and Policy Analysis 31(4): 441–462. doi: 10.3102/0162373709348584

Collins, Timothy W., Sara E. Grineski, Jessica Shenberger, Xiaodan Morales, Osvaldo F. Morera, and Lourdes E. Echegoyen. 2017. “Undergraduate Research Participation Is Associated with Improved Student Outcomes at a Hispanic-Serving Institution.” Journal of College Student Development 58(4): 583–600. doi: 10.1353/csd.2017.0044

Craney, Chris, Tara McKay, April Mazzeo, Janet Morris, Cheryl Prigodich, and Robert de Groot. 2011. “Cross-Discipline Perceptions of the Undergraduate Research Experience.” The Journal of Higher Education 82(1): 92–113. doi: 10.1353/jhe.2011.0000

Department of Education. n.d. “Federal Pell Grants.” Accessed January 5, 2021. https://studentaid.gov/understand-aid/types/grants/pell

Eagan, M Kevin, Sylvia Hurtado, Mitchell J. Chang, Gina A. Garcia, Felisha A. Herrera, and Juan C. Garibay. 2013. “Making a Difference in Science Education: The Impact of Undergraduate Research Programs.” American Educational Research Journal 50(4): 683–713. doi: 10.3102/0002831213482038

Espinosa, Lorelle. 2011. “Pipelines and Pathways: Women of Color in Undergraduate STEM Majors and the College Experiences That Contribute to Persistence.” Harvard Educational Review 81(2): 209–241. doi: 10.17763/haer.81.2.92315ww157656k3u

Estrada, Mica, Paul R. Hernandez, and P. Wesley Schultz. 2018. “A Longitudinal Study of How Quality Mentorship and Research Experience Integrate Underrepresented Minorities into STEM Careers.” Ed. Jose Herrera. CBE—Life Sciences Education 17(1): ar9. doi: 10.1187/cbe.17-04-0066

Fakayode, Sayo O., Mamudu Yakubu, Olasumbo M. Adeyeye, David A. Pollard, and Abdul K. Mohammed. 2014. “Promoting Undergraduate STEM Education at a Historically Black College and University through Research Experience.” Journal of Chemical Education 91(5): 662–665. doi: 10.1021/ed400482b

Fechheimer, Marcus, Karen Webber, and Pamela B. Kleiber. 2011. “How Well Do Undergraduate Research Programs Promote Engagement and Success of Students?” Ed. Marshall David Sundberg. CBE—Life Sciences Education 10(2): 156–163. doi: 10.1187/cbe.10-10-0130

Fiegener, Mark K. 2015. Science and Engineering Degrees: 1966–2012. National Center for Science and Engineering Statistics, Detailed Statistical Tables NSF 15-326. Arlington, VA: National Science Foundation. https://www.nsf.gov/statistics/2015/nsf15326/pdf/nsf15326.pdf

Follmer, D. Jake, Sarah Zappe, Esther Gomez, and Manish Kumar. 2017. “Student Outcomes from Undergraduate Research Programs: Comparing Models of Research Experiences for Undergraduates (REUs).” Scholarship and Practice of Undergraduate Research (SPUR) 1(1): 20–27. https://eric.ed.gov/?id=EJ1213755

Frederick, Angela, Sara E. Grineski, Timothy W. Collins, Heather A. Daniels, and Danielle X. Morales. 2021. “The Emerging STEM Paths and Science Identities of Hispanic/Latinx College Students: Examining the Impact of Multiple Undergraduate Research Experiences.” Ed. Stanley M. Lo. CBE—Life Sciences Education 20(2): ar18. doi: 10.1187/CBE.20-08-0191

Galli, Dominique M., and Rafael Bahamonde. 2018. “Assessing IUPUI’s Diversity Scholars Research Program: Lessons Learned.” Scholarship and Practice of Undergraduate Research (SPUR) 1(4): 12–17. doi: 10.18833/spur/1/4/10

Haeger, Heather, John E. Banks, Camille Smith, and Monique Armstrong-Land. 2020. “What We Know and What We Need to Know about Undergraduate Research.” Scholarship and Practice of Undergraduate Research (SPUR) 3(4): 62–69. doi: 10.18833/spur/3/4/4

Haeger, Heather, and Carla Fresquez. 2016. “Mentoring for Inclusion: The Impact of Mentoring on Undergraduate Researchers in the Sciences.” Ed. Pat Marsteller. CBE—Life Sciences Education 15(3): ar36. doi: 10.1187/cbe.16-01-0016

Hathaway, Russel S., Biren (Ratnesh) A. Nagda, and Sandra R. Gregerman. 2002. “The Relationship of Undergraduate Research Participation to Graduate and Professional Education Pursuit: An Empirical Study.” Journal of College Student Development. 43(5): 614-631. https://eric.ed.gov/?id=EJ653327

Hernandez, Paul R., Anna Woodcock, Mica Estrada, and P. Wesley Schultz. 2018. “Undergraduate Research Experiences Broaden Diversity in the Scientific Workforce.” BioScience 68(3): 204–211. doi: 10.1093/biosci/bix163

Junge, Benjamin, Catherine Quiñones, Jakub Kakietek, Daniel Teodorescu, and Pat Marsteller. 2010. “Promoting Undergraduate Interest, Preparedness, and Professional Pursuit in the Sciences: An Outcomes Evaluation of the SURE Program at Emory University.” Ed. Debra Tomanek. CBE—Life Sciences Education 9(2): 119–132. doi: 10.1187/cbe.09-08-0057

Loeser, Matthew R., Makaylah Newkirk, Kara I. Gabriel, and Audrey D. Huerta. 2021. “Development and Assessment of an Undergraduate Research Program at a Two-Year, Rural, Hispanic-Serving Institution: The Essential Role of Partnerships.” Scholarship and Practice of Undergraduate Research (SPUR) 4(3): 22–29. doi: 10.18833/spur/4/3/10

Lopatto, David. 2007. “Undergraduate Research Experiences Support Science Career Decisions and Active Learning.” Ed. Paul Williams. CBE—Life Sciences Education 6: 297–306. doi: 10.1187/cbe.07-06-0039

Martinez, Luis R., Dwayne W. Boucaud, Arturo Casadevall, and Avery August. 2018. “Factors Contributing to the Success of NIH-Designated Underrepresented Minorities in Academic and Nonacademic Research Positions.” CBE—Life Sciences Education 17(2): ar35. doi: 10.1187/cbe.16-09-0287

Maton, Kenneth I., Tiffany S. Beason, Surbhi Godsay, Mariano R. Sto. Domingo, TaShara C. Bailey, Shuyan Sun, and Freeman A. Hrabowski. 2016. “Outcomes and Processes in the Meyerhoff Scholars Program: STEM PhD Completion, Sense of Community, Perceived Program Benefit, Science Identity, and Research Self-Efficacy.” Ed. Pat Marsteller. CBE—Life Sciences Education 15(3): ar48. doi: 10.1187/cbe.16-01-0062

Maton, Kenneth I., Mariano R. Sto Domingo, Kathleen E. Stolle-McAllister, J. Lynn Zimmerman, and Freeman A. Hrabowski, III. 2009. “Enhancing the Number of African-Americans Who Pursue Stem PHDS: Meyerhoff Scholarship Program Outcomes, Processes, and Individual Predictors.” Journal of Women and Minorities in Science and Engineering 15(1): 15–37. doi: 10.1615/JWomenMinorScienEng.v15.i1.20

Monarrez, Angelica, Danielle Morales, Lourdes E. Echegoyen, Diego Seira, and Amy E. Wagler. 2020. “The Moderating Effect of Faculty Mentorship on Undergraduate Students’ Summer Research Outcomes.” Ed. Graham F. Hatfull. CBE—Life Sciences Education 19(4): ar56. doi: 10.1187/CBE.20-04-0081

Sell, Andrea J., Angela Naginey, and Cathy A. Stanton. 2018. “The Impact of Undergraduate Research on Academic Success.” Scholarship and Practice of Undergraduate Research (SPUR) 1(3): 19–29. doi: 10.18833/spur/1/3/8

Simmons, Kiyoko N. “Effect of Undergraduate Research Programs on Retention of Hispanic Students.” PhD Dissertation, The University of New Mexico, 2018.

Thiry, Heather, Sandra L. Laursen, and Anne-Barrie Hunter. 2011. “What Experiences Help Students Become Scientists?: A Comparative Study of Research and Other Sources of Personal and Professional Gains for STEM Undergraduates.” The Journal of Higher Education 82(4): 357–388. doi: 10.1353/jhe.2011.0023

U.S. Government Publishing Office. n.d. “United States Code, Chapter 28.” Accessed January 06, 2021. https://www.govinfo.gov/content/pkg/USCODE-2011-title20/html/USCODE-2011-title20-chap28-subchapIII-partE-subpart3-sec1067k.htm.

Whittinghill, Jonathan C., Simeon P. Slovacek, Laura P. Flenoury, and Vivian Miu. 2019. “A 10-Year Study on the Efficacy of Biomedical Research Support Programs at a Public University.” Scholarship and Practice of Undergraduate Research (SPUR) 3(1): 30–38. doi: 10.18833/spur/3/1/3

Wilson, Alan E., Jenna L. Pollock, Ian Billick, Carmen Domingo, Edna G. Fernandez-Figueroa, Eric S. Nagy, Todd D. Steury, and Adam Summers. 2018. “Assessing Science Training Programs: Structured Undergraduate Research Programs Make a Difference.” BioScience 68(7): 529–534. doi: 10.1093/BIOSCI/BIY052

Zydney, Andrew L., Joan S. Bennett, Abdus Shahid, and Karen W. Bauer. 2002. “Impact of Undergraduate Research Experience in Engineering.” Journal of Engineering Education 91(2): 151–157. doi: 10.1002/j.2168-9830.2002.tb00687.x

Laura A. Diaz-Martinez

The University of Texas at El Paso, Gonzaga University, diazmartinez@gonzaga.edu

Samantha Jude Battaglia is a graduate of Gonzaga University, where she obtained her bachelor’s degree in science in biology with a minor in environmental science. She is currently working through Teach For America as an eighth grade science teacher at Aurora Science and Technology Middle School in Denver, Colorado. After completing her two-year commitment with Teach For America, she plans to attend graduate school in pursuit of a master’s degree in public health with an emphasis in health administration and global health.

Lourdes E. Echegoyen is the founding director of COURI at UTEP, which she has led for the past 11 years. She is also a research associate professor in the department of chemistry and biochemistry. Since her arrival at UTEP in 2010, Echegoyen has received several awards from the National Science Foundation, the National Institutes of Health, the Howard Hughes Medical Institute, and the Department of Energy, all of which are focused on student, faculty, and institutional development. Multiple cross-institutional partnerships and collaborations have ensued from these awards, and her collaborative leadership within UTEP led to the institutionalization of a CURE program for first-year students. Her earlier publications were in the area of supramolecular and fullerene chemistry, whereas her most recent ones emphasize the impact that undergraduate research and related activities have on students, particularly those from minority groups.

Laura A. Diaz-Martinez is an assistant professor of biology at Gonzaga University. She leads two research streams, one on cellular and molecular aspects of cell division and another one on biology education with a focus on integrating education for responsible and ethical research conduct in Course-based Undergraduate Research Experiences (CUREs). Prior to her appointment at Gonzaga University, she served as associate director of the Campus Office of Undergraduate Research Initiatives (COURI) at the University of Texas at El Paso (UTEP).