Role of STEM Identity in Minority Student Success in STEM Research
Youngha Oh, PhD Statistician Investigator UNC – Chapel Hill
Michelle Quirke, MS Project Manager IN LSAMP
Introduction A literature review was conducted to prepare the IN LSAMP Summer Cohort Framework. Identified were the Identity Prominence (IP) and Self-Efficacy (SE) surveys from the 2017 article by Stets, et al.1 The surveys are used in our study with permission from the authors. Both surveys are administered to participants accepted as research scholars in the IN LSAMP program via RedCap at Indiana University. IRB exemption was approved in 2017 for the study. The surveys are administered at two data collection points: pre (orientation) and post (exit session). The program is conducted at 5 different campuses in Indiana during the summer. Programs range from 8 to 10 weeks in length. At the end of the research experience the faculty mentor is asked to evaluate the scholar’s growth in identified areas via the Faculty Exit survey. Each scholar is expected to present their research at the end of the research experience at a campus hosted research seminar. Students may continue research with their lab based upon faculty mentor approval. Students are encouraged to disseminate research at conferences such as ABRCMS, LSMRCE, ERN, and SACNAS.
Research Brief, July 2019 Page 2
IP Survey Methods The IP survey consists of 4 items and they were measured by 5 Likert- Scale, but the scale was converted into the numeric (i.e., 1: Strongly disagree to 5: Strongly agree). The survey was given to participants who were accepted as research scholars in the IN LSAMP program. The IN LSAMP Alliance ran programs across five campuses (Indiana University, Ball State University, Indiana University Northwest, IUPUI, and IU South Bend) in 2019. This IP survey was measured two times, before and after research experience (pre and post); pre time was the beginning of summer research (N = 51) and the post time was the end of summer research (N = 41) in 2019. Paired sample t-tests were conducted to examine whether there was any statistical evidence of the mean difference between pre-test and post-test scores on the 4 items in the IP survey.
Results Descriptive Statistics The descriptive statistics (i.e., Mean, standard deviation [SD]) for both time points are shown in Table 1. The overall mean scores for the 4 items were fairly high for both time points (pre: M = 3.55-3.88, SD = 0.88-1.03 and post: M = 3.93-4.15, SD = 0.75-1.03).
Table 1. Descriptive Statistics for the IP survey
Questions
Pre-N
Pre-Mean
Pre-SD
Post-N
Post-Mean
Post-SD
In general, being a scientist is an important part of my self-image.
51
3.88
1.01
41
4.07
1.03
I have a strong sense of belonging to be the community of the scientist.
51
3.69
1.01
41
4.15
0.88
Being a scientist is an important reflection of who I am.
51
3.84
0.88
41
4.05
0.86
I have come to think of myself as a 'scientist'.
51
3.55
1.03
41
3.93
0.75
Research Brief, July 2019 Page 3
Mean comparisons between pre and post tests The paired sample t-test for pre and post time comparison showed that item 2 (i.e., I have a strong sense of belonging to the community of scientists; ∆M = .47, SD =.62; t(35) = 3.50, p < .001, d = .53) and item 4 (i.e., I have come to think of myself as a 'scientist'; ∆M = .39, SD =.93; t(35) = 3.50, p < .05, d = .42) were statistically significant. These results indicate that item 2 and item 4 had significantly improved at the end of summer semester. In addition, Cohen (1988)2 defined the benchmark of effect sizes d = 2 as "small," d = 5 as "medium," and d = 8 as "large", thus, the ranges of effect size were medium.
FIGURE 1. 2019 IP Survey Outcomes (N = 41)
SE Survey Methods Two different surveys were extracted from well-developed theoretical constructs in a previous literature review of self-efficacy (SE) framework (Stets, et al. 2017) in order to determine the concept of self-efficacy in undergraduate STEM research experiences. This SE survey consists of 10 items and they were measured by 5 Likert-Scale, but the scale was converted into the numeric (i.e., 1: Not at all confident to 5: Absolutely confident).
Research Brief, July 2019 Page 4
The surveys were given to participants who were accepted as research scholars in the IN LSAMP program. Data was collected at program entry in 2019 two times, pre-test and post-test; the beginning of summer research (N = 50) and end of summer research session (N = 41).
Paired sample t-tests were conducted to examine whether there was any statistical evidence of the mean difference between pre-test and post-test scores on the 10 items in the SE survey.
Results Descriptive Statistics The descriptive statistics (i.e., Mean, SD) for both time points are shown in Table 2. The overall mean scores for the 10 items were above average for both time points (pre: M = 2.74-3.74, SD = 1.04-1.45 and post: M = 3.59-4.29, SD = 0.89-1.28).
Table 2. Descriptive Statistics for the SE survey
Questions
Pre-N
Pre-Mean
Pre-SD
Post-N
Post-Mean
Post-SD
Use technical science skills (use of tools, instrument, and/or techniques).
50
3.22
1.25
41
4.07
1.13
Use scientific language and terminology.
50
3.22
1.04
41
3.63
0.89
Generate a research question to answer.
50
2.74
1.34
41
3.78
1.08
Figure out what data/oberservations to collect and how to collect them.
50
3.14
1.20
41
3.93
1.10
Figure out/analyze what data/oberservations mean.
50
3.34
1.19
41
3.88
1.08
Releated results and explanations to the work of others.
50
3.28
1.43
41
3.98
1.11
Report research results in an oral presentation.
50
2.9
1.45
41
3.59
1.28
Uhnderstand the ethics of research.
50
3.74
1.23
41
4.29
0.90
Generally function as a scientist in a research activity.
50
3.28
1.23
41
4.17
0.97
Learn the full range of science skills with appropriate training.
50
3.66
1.29
41
4.29
0.90
Mean comparisons between pre and post tests The paired samples t-test results for mean difference between pre-test and post-test scores on 10 items in the SE survey showed that all 10 items were statistically significant, which means average scores were significantly improved when pre-test was compared with post-tests.
Research Brief, July 2019 Page 5
Item 1 (Use technical science skills (use of tools, instruments, and/or techniques)) was significantly improved compared with pre and post; ΔM = .67, SD =1.20; t(35) = 3.35, p < .002, d = .713. The effect size of Cohen's d indicates that the effect was medium.
Item 2 (Use scientific language and terminology) was significantly improved compared with pre and post; ΔM = .53, SD = 1.18; t(35) = 2.68, p < .011, d = .42. The effect size of Cohen's d indicates that the effect was small.
Item 3 (Generate a research question to answer) was significantly improved compared with pre and post; ΔM = 1.03, SD = 1.61; t(35) = 3.82, p = .001, d = .86. The effect size of Cohen's d indicates that the effect was large.
Item 4 (Figure out what data/observations to collect and how to collect them) was significantly improved compared with pre and post; ΔM = .78, SD = 1.12; t(35) = 4.15, p < .001, d = .69. The effect size of Cohen's d indicates that the effect was medium.
Item 5 (Figure out/analyze what data/observations mean) was significantly improved compared with pre and post; ΔM = .67, SD = 1.04; t(35) = 3.84, p < .001, d = .48. The effect size of Cohen's d indicates that the effect was small.
Item 6 (Relate results and explanations to the work of others) was significantly improved compared with pre and post; ΔM = .92, SD = 1.57; t(35) = 3.49, p < .001, d = .55. The effect size of Cohen's d indicates that the effect was medium.
Item 7 (Report research results in an oral presentation) was significantly improved compared with pre and post; ΔM = .78, SD = 1.15; t(35) = 4.06, p < .001, d = .51. The effect size of Cohen's d indicates that the effect was medium.
Item 8 (Understand the ethics of research) was significantly improved compared with pre and post; ΔM = .53, SD = 1.34; t(35) = 2.36, p = .024, d = .51. The effect size of Cohen's d indicates that the effect was medium.
Item 9 (Generally function as a scientist in a research activity) was significantly improved compared with pre and post; ΔM = 1.08, SD = 1.32; t(35) = 4.93, p < .001, d = .80. The effect size of Cohen's d indicates that the effect was large.
Item 10 (Learn the full range of science skills with appropriate training) was significantly improved compared with pre and post; ΔM = .61, SD = 1.23; t(35) = 2.99, p < .001, d = .57. The effect size of Cohen's d indicates that the effect was medium.
Research Brief, July 2019 Page 6
FIGURE 2. 2019 SE Survey Outcomes (N=41) Faculty
Faculty Mentor Exit Survey Methods The Faculty Mentor Exit survey consists of 10 items and they were measured by 4 Likert-Scale (i.e., 1: Low Level to 4: High Level), but the scale was converted into the numeric. The survey was given to faculty mentors who accepted a scholar into their research lab. The program was conducted at 5 different campuses in Indiana. To date, the Faculty Exit survey was submitted at the end of scholar research experience (N = 47).
Research Brief, July 2019 Page 7
Results Descriptive Statistics The descriptive statistics (i.e., Mean, SD) for both time points are shown in Table 3. The overall mean scores for the 10 items were low for the pre time, but above average for post time points (pre: M = 1.72−2.45, SD = 0.77−0.97 and post: M = 2.62−3.30, SD = 0.69−0.87).
Table 3. Faculty Evaluated Students at Pre and Post
Questions
Pre-N
Pre-Mean
Pre-SD
Post-N
Post-Mean
Post-SD
Use technical science skills (use of tools, instrument, and/or techniques).
47
1.96
0.833
47
3.11
0.699
Use scientific language and terminology.
47
1.98
0.794
47
2.87
0.769
Generate a research question to answer.
47
1.72
0.800
47
2.62
0.874
Figure out what data/oberservations to collect and how to collect them.
47
1.81
0.825
47
3.02
0.707
Figure out/analyze what data/oberservations mean.
47
1.83
0.761
47
2.91
0.775
Releated results and explanations to the work of others.
47
1.74
0.820
47
2.96
0.833
Report research results in an oral presentation.
47
1.72
0.772
47
2.98
0.737
Uhnderstand the ethics of research.
47
2.45
0.996
47
3.30
0.778
Generally function as a scientist in a research activity.
47
1.89
0.866
47
3.00
0.780
Learn the full range of science skills with appropriate training.
47
1.85
0.807
47
2.98
0.794
Mean comparisons between pre and post tests The paired sample t-test for pre and post time comparison showed that all of the 10 items were statistically significant. These results indicate that mentors evaluated scores for all of the items were significantly improved at the pre time when compare with the post time point.
Research Brief, July 2019 Page 8
Item 1 (Use technical science skills (use of tools, instruments, and/or techniques)) was significantly improved compared with pre and post; ΔM = 1.15, SD =0.78; t(46) = 10.10, p < .001, d = 1.51. The effect size of Cohen's d indicates that the effect was large.
Item 2 (Use scientific language and terminology) was significantly improved compared with pre and post; ΔM = 0.89, SD =0.81; t(46) = 7.53, p < .001, d = 1.14. The effect size of Cohen's d indicates that the effect was large.
Item 3 (Generate a research question to answer) was significantly improved compared with pre and post; ΔM = 0.89, SD =0.73; t(46) = 8.40, p < .001, d = 1.07. The effect size of Cohen's d indicates that the effect was large.
Item 4 (Figure out what data/observations to collect and how to collect them) was significantly improved compared with pre and post; ΔM = 1.21, SD =0.78; t(46) = 10.68, p < .001, d = 1.58. The effect size of Cohen's d indicates that the effect was large
Item 5 (Figure out/analyze what data/observations mean) was significantly improved compared with pre and post; ΔM = 1.09, SD =0.88; t(46) = 8.45, p < .001, d = 1.41. The effect size of Cohen's d indicates that the effect was large.
Item 6 (Relate results and explanations to the work of others) was significantly improved compared with pre and post; ΔM = 1.21, SD =0.86; t(46) = 9.69, p < .001, d = 1.48. The effect size of Cohen's d indicates that the effect was large.
Item 7 (Report research results in an oral presentation) was significantly improved compared with pre and post; ΔM = 1.26, SD =0.79; t(46) = 10.85, p < .001, d = 1.67. The effect size of Cohen's d indicates that the effect was large.
Item 8 (Understand the ethics of research) was significantly improved compared with pre and post; ΔM = 0.85, SD =0.98; t(46) = 5.97, p < .001, d = 0.95. The effect size of Cohen's d indicates that the effect was large.
Item 9 (Generally function as a scientist in a research activity) was significantly improved compared with pre and post; ΔM = 1.11, SD =0.84; t(46) = 9.03, p < .001, d = 1.35. The effect size of Cohen's d indicates that the effect was large
Item 10 (Learn the full range of science skills with appropriate training) was significantly improved compared with pre and post; ΔM = 1.13, SD =0.80; t(46) = 9.70, p < .001, d = 1.41. The effect size of Cohen's d indicates that the effect was large.
Program Outcomes We revised our data collection process for the 2020 cohort to collect surveys during an exit session. The goal is to increase efficiency of data collection for coordinators and improve survey participation. We have found that the end of the session varies with research symposiums and lab schedules so scheduling an in-person exit session for the cohorts will improve the administration of the survey.
Conclusion The IN LSAMP program has reported growth among the scholars participating in the summer research and favorable feedback from all faculty mentors supporting scholars in their research labs. The coordinators continue to provide additional resources and work at their campuses to integrate existing services into their professional development activities. Our sincere gratitude for the faculty mentors who allow our scholars to be part of their research and continue to support them beyond their time in the labs. Their involvement allows us to continue growing the program each year.
Research Brief, July 2019 Page 10
11Stets, J. E., Brenner, P. S., Burke, P. J., & Serpe, R. T. (2017). The science identity and entering a science occupation. Social Science Research, 64, 1-14.
2Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates
Research Brief, July 2019 Page 11
Acknowledgements IRB, Principal Investigator, Kim S. Nguyen, EdD, Protocol number 1709058337A001, IUPUI
Study data collected and managed using REDCap electronic data capture tools hosted at Indiana University. 1,2 REDCap (Research Electronic Data Capture) is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources.
1PA Harris, R Taylor, R Thielke, J Payne, N Gonzalez, JG. Conde, Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support, J Biomed Inform. 2009 Apr;42(2):377-81. 2PA Harris, R Taylor, BL Minor, V Elliott, M Fernandez, L O'Neal, L McLeod, G Delacqua, F Delacqua, J Kirby, SN Duda, REDCap Consortium, The REDCap consortium: Building an international community of software partners, J Biomed Inform. 2019 May 9 [doi: 10.1016/j.jbi.2019.103208]
Indiana LSAMP Louis Stokes Alliance for Minority Participation resources and social media channels
