Educational programs at Sam Houston State University (SHSU) regularly identify expected outcomes, assess the extent to which they achieve those outcomes, and provide evidence of seeking improvement based on analysis of the results. This narrative features representative examples of the outcomes assessment process utilized by educational units, including online and hybrid programs, at SHSU. These initial examples are expanded upon in more extensive, college-specific documents highlighting completed unit-level assessment plans from the past three assessment cycles (i.e., 2021-2022, 2022-2023, and 2023-2024). Reviewers may access all archived academic unit assessment plans for these cycles within Assessment portion of SHSU’s 5th Year Interim Report website. This narrative will highlight additional steps taken by SHSU to ensure compliance with the guidelines, recommendations, and requirements of the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC) regarding Standard 8.2.a
Institutional assessment at SHSU is overseen by the Office of Assessment (OA) [1]. Within OA, the Director of Assessment oversees unit entries into the campus’s online assessment management system, Anthology Planning [2]. Furthermore, OA staff members provide regular training, resources, and support to all units across campus conducting annual programmatic assessment. These trainings and resources cover topics including, but not limited to, assessment basics and plan elements, assessment software training, curriculum mapping, assessment best practices, and assessment mini grants [3] [4] [5] [6].
SHSU utilizes a 12-month annual assessment cycle for all units, running from mid-September to mid-September of the following year. This cycle roughly aligns with the University’s academic calendar to give units the flexibility to collect data from all academic semesters (i.e., fall, spring, and summer). The OA works closely with any units relying on end-of-fiscal-year data that may not be available until after the cycle closes to ensure they are able to analyze their results and develop actions for improvement prior to documentation of these items within the system.
The OA communicates with units throughout the assessment cycle to ensure they are proactive and thoughtful about their assessment plans. Iterative deadlines are set throughout the year (e.g., December deadline for entering goals, objectives, indicators/criteria, and KPIs/targets; August deadline for entering findings and results; and September deadline for entering actions and plans for continuous improvements) [7]. OA staff members monitor unit entries throughout the year to ensure participation in the ongoing assessment cycle.
The assessment process for educational programs is overseen by a program coordinator and/or the department chair. These individuals oversee the process and collaborate with their faculty to develop, implement, and continuously reflect upon the assessment plans within their units. The Academic Department Chair Handbook [8] contains a page on assessment [9], in which an Introduction to Assessment for Department Chairs video informs chairs of their specific assessment responsibilities. Additionally, the University Assessment Committee (UAC) [10], appointed by the Provost, meets regularly to discuss issues related, in part, to annual programmatic assessment and use of data for continuous improvement. The UAC is chaired by the Director of the Office of Assessment, and members include the associate deans charged with overseeing assessment in each of the eight academic colleges.
Annual Assessment Plan Elements
Units at SHSU utilize Anthology Planning to document their ongoing assessment plans and reports and are asked to provide the following plan elements.
Examples of Annual Outcomes Assessment
Specific examples from each academic college for the most recently completed assessment cycle (2023-2024) are highlighted below, in detail, to demonstrate how educational units at SHSU define outcomes, assess the extent to which they achieve these outcomes, and utilize the analyzed data to identify actions for improvement. A complete repository for all educational program assessment plans for these cycles is included in the Assessment portion of the SACSCOC 5th Year Interim Report Website.
College of Arts and Media – Animation BFA [11]
For the 2023-2024 assessment cycle, the Animation BFA program identified four learning objectives. These objectives were assessed through faculty evaluation and the application of rubrics to completed assignments.
One of the objectives focused on students enrolled in Animation Concepts and Techniques (ARTS 2343) and their ability to “demonstrate introductory level proficiencies in the post-production process through the creation of short, animated videos that include an editing process, titles, credits, and sound” [12]. Faculty determined that lower scores in this area could be attributed to errors of omission on the part of students rather than students’ proficiency (i.e., students are failing to include the audio element in their project). To address this issue, faculty will require ARTS 2343 students to demonstrate their progress on the audio portion of their project through scheduled mid-production check-ins. Additionally, faculty are allowing students to submit revised work, which will be included as a formal policy in spring 2025 syllabi.
A rubric was used to evaluate students' proficiency in the four areas listed below [13]
The findings for this objective revealed that 97.95% of students demonstrated an adequate, above average, or exceptional level of proficiency in each of the first three areas. However, in the fourth area, the Use of Soundtracks, student proficiency at the adequate, above average, or exceptional level dropped to 87.5% [14]
College of Business Administration - Supply Chain Management BBA [15]
The Supply Chain Management BBA program identified four student learning objectives for the 2023-2024 assessment cycle. To assess student attainment of these objectives, faculty within the program used various assessment methods, including exams, projects, presentations, and research papers. The program used the data generated from these assessments to identify specific actions for improving student learning.
For example, one learning objective identified by the program was: “Students should be able to analyze quantitative and qualitative information when making supply chain decisions” [16]. This objective was separated into four sub-objectives addressing fundamentals, understanding competitive advantage, cultural diversity, and problem-solving. Student attainment of these sub-objectives was assessed as follows:
Findings from these assessments revealed that the percentage of students who met or exceeded expectations was as follows: 43% (Fundamentals), 68% (Understanding Competitive Advantage), 68% (Cultural Diversity), and 100% (Problem-Solving). Although students excelled in problem-solving, some struggled in the other three areas. To improve student learning in these areas, the program will provide more practice problems, introduce more multi-criteria assignments, require daily journals to help students organize and understand their decision-making process, and require additional assignments designed to improve student learning.
College of Criminal Justice - Forensic Science MS [17]
For the 2023-2024 assessment cycle, the Forensic Science MS program identified three learning objectives. To assess these objectives, program faculty tracked the publication of students’ research in peer-reviewed scientific journals and students’ participation in national/international conferences and publications of conference papers, and implemented a faculty-developed scoring methodology applied to a scholarly capstone research report.
For example, one learning objective stated: “Students actively engaged in research projects sponsored by program faculty will develop the technical writing skills and scientific abilities required to publish their findings in journals or present their data at scientific meetings” [18]. Findings for this objective revealed 13 student publications in peer-reviewed journals and eight in conference proceedings. These numbers represented 0.6 publications per student, which exceeded the 0.5 criterion set for the objective's success. The 0.6 publications per student in 2023-2024 represented the highest number of publications in the program’s history [19].
To further increase publications, program faculty will encourage first-year MS students to begin communicating with potential research advisors in the fall semester of their first year. Additionally, program faculty will provide further instruction and guidance on technical writing and in formatting a capstone paper as a manuscript instead of a thesis, potentially leading to increased MS student peer-reviewed journal publications and conference proceedings.
College of Education - Education BS (Middle Level 4-8) [20]
The Education BS (Middle Level 4-8) program identified four learning objectives for the 2023-2024 assessment cycle. Student attainment of these objectives was assessed using content area exams and rubrics for Texas Teacher Evaluation and Support System (T-TESS) observations, the Tk20 online assessment and portfolio system, the Impact on Student Learning Assessment, and the Teacher Candidate Assessment of Readiness (TCAR).
One of the objectives focused on middle-level teacher candidates' (TCs) ability to achieve “a score of ‘proficient’ in 2.2 Content Knowledge on their final T-TESS” [21]. Student attainment of this objective was evaluated using a Tk20 rubric for students’ final T-TESS observation, where they demonstrated their ability to meet the following Texas Education Agency (TEA) requirements for proficiency in 2.2 Content Knowledge:
During the 2023-2024 year, 38 middle-level teacher candidates completed their final T-TESS observation. Of these, 82% (31) scored "proficient" in 2.2 Content Knowledge, a percentage that was 3 percentage points below the 85% criterion selected for successful attainment of the objective. To improve students’ mastery of 2.2 Content Knowledge, program faculty will engage the Educator Preparation Advisory Council (EPAC) to provide additional education to department faculty to strengthen their understanding of the scope and depth of the various content exams. Based upon this enhanced knowledge base, additional support structures will be developed to strengthen preparation support for TCs.
College of Health Sciences - Food Science and Nutrition BS [22]
For the 2023-2024 assessment cycle, the Food Science and Nutrition BS program identified five learning objectives. Student attainment of these objectives was assessed through exams, projects, peer evaluations, assignments, a student self-assessment survey, and an exit survey.
One of the five objectives stated: “Students will demonstrate counseling and education methods to facilitate behavior change and enhance wellness for individuals” [23]. A rubric was used to assess student performance on a near-peer nutrition counseling assignment [24]. The criterion for successfully achieving this objective was that 100% of students would score 80% or higher in their demonstration of wellness counseling.
Findings revealed that 100% (12/12) of the students scored 80% or higher on their demonstration of wellness counseling as part of the Near-Peer consultation project in the Nutrition Assessment course (FSCN 4371). Students’ rubric scores ranged from 88 to 98 out of 100 possible points. Additionally, all students successfully demonstrated their ability to establish rapport with the client, maintain a client-centered approach, use open-ended questions, employ reflective listening techniques and behavior change strategies, and develop a nutritional plan.
Although the criterion for success was met, program faculty noted that all students needed improvement in using a meal plan and writing diet instructions. To improve student learning and skills in these areas, program faculty will engage in more discussion and practice to improve written materials provided in a role play that involves developing a meal plan and diet instruction. An additional action will be to increase the criterion for success to: “100% of students will score 85% or higher on the demonstration of wellness counseling.”
College of Humanities and Social Sciences - Sociology BA/BS [25]
The Sociology BA/BS program identified three learning objectives for the 2023-2024 assessment cycle. Student attainment of these objectives was assessed using evaluation instruments and essays.
One of the objectives stated: “Students enrolled in introductory sociology classes that are included in the SHSU core curriculum will be able to demonstrate their understanding of the basic core concepts of the discipline and the sociological perspective” [26]. Attainment of this objective was assessed using a five-question evaluation instrument for students enrolled in Introduction to Ethnic Studies (SOCI 2319), which is included in Core Component Area IV – Humanities and Visual and Performing Arts and a ten-question evaluation for students enrolled in Introduction to Sociology (SOCI 1301), which is included in Core Component Area V – Social and Behavioral Sciences.
The five-question evaluation for Component Area IV assessed two concepts [27]
The ten-question evaluation for Component Area V evaluated students' abilities in five concepts [28]
For Component Area IV, the criterion was at least 85% of students should correctly answer three of five questions. Additional criteria for success in Component Area IV were at least 85% of students should have at least two correct answers for Concept 1 and at least one correct answer for Concept 2. A total of 159 students completed the five-question instrument. Of these, 94% correctly answered three of five questions. For Concept 1, 96% correctly answered two of three questions, and for Concept 2, 91% correctly answered at least one of two questions.
For Component Area V, the criterion for success was at least 85% of students should correctly answer six of the ten questions. In addition, at least 85% of students should have at least one correct answer for each of the five concepts. A total of 120 students completed the ten-question evaluation. Overall, 84% of the students correctly answered six of the ten questions. The percentage of students correctly answering at least one question in each of the five Concepts was as follows:
Although the program exceeded the criterion for success in Component Area IV, the criterion for success for Component Area V was not met. Based on the results for Component Area V, the program will work with faculty teaching SOCI 1301 to strengthen the relevant dimensions of the curriculum for the course. Specifically, "In Spring 2025, SOCI 1301 instructors will be given access to interactive learning exercises targeting the application of theoretical perspectives and encouraged to use them for each topical, substantive textbook chapter (i.e., all content after Ch. 5)."
College of Osteopathic Medicine - Doctor of Osteopathic Medicine [29]
For the 2023-2024 assessment cycle, the Doctor of Osteopathic Medicine program identified four learning objectives. To assess student attainment of these objectives, faculty within the program used various assessment methods, including performance on the Comprehensive Osteopathic Medical Licensing Examination (COMLEX), evaluation of research and evidenced-based medicine assignments, quizzes, exams, research publications, and self-assessments.
One of the learning objectives focused on level one [30] and level two [31] of the COMLEX exam, which assessed students on the application of osteopathic medical knowledge, osteopathic patient care, osteopathic principles and practice, systems-based practice, practice-based learning and improvement, communication skills, professionalism, and ethics [32]. Findings revealed that first-time pass rates for students at level one was 97.1%, and first-time pass rates for level two students were 99.0% [33]. Both pass rates exceeded the program’s criteria for success, which were 93.0% and 97.1%, respectively. A data comparison also revealed that the first-time pass rate of 97.1% for level one was the highest in the nation [34] [35]. To maintain or exceed these high pass rates, the program will require students to take multiple simulated board exams to help them develop test-taking skills and to provide context regarding the individual preparation required for board exams.
Another objective focused on students’ ability to acquire academic presentation and publication skills in preparation for residency applications during their year-one Research and Evidence-Based Medicine course (REBM 7301). Indicators for success included students’ development of novel ideas into abstracts, manuscripts, grants, and posters, which were tracked using self-reported data contained in the Electronic Residency Application Service (ERAS) system as well as a survey requiring student self-assessment [36]. Findings revealed that the criterion for success was met for both indicators; 62 of 69 (89.9%) student doctors reported 285 abstracts, presentations and publications, averaging 4.6 per student, and students indicated that their presentation proficiency related to research increased from 0.2 to 2.1 points.
Based on the results for this objective, the College of Osteopathic Medicine will provide students with opportunities to participate in research throughout the curriculum, to become involved in faculty-led research, and to develop their own novel research ideas. An additional action that will be taken involves developing a student research tracking system that does not rely on self-reported data from the ERAS application system.
College of Science and Engineering Technology - Geography BS [37]
The Geography BS program identified eight learning objectives for the 2023-2024 assessment cycle. These objectives were assessed through exams containing questions pertaining to introductory concepts of geography, advanced concepts of environmental and physical geography, students’ comprehension of the world in spatial terms, and concepts of human-environment interaction.
One of the eight objectives focused on students’ ability to “demonstrate proficiency in First-Year Foundational Geographic Concepts” [38]. Student attainment of the objective was assessed using embedded exam questions related to introductory concepts in the following four courses:
Students enrolled in GEOG 1300 earned an average score of 64% on questions pertaining to introductory concepts in geography, and students enrolled in GEOG 1401 scored an average of 60%. Students enrolled in GEOG 3301 scored 72%, and students enrolled in GEOG 3350 scored 71%. Compared to the previous assessment of introductory concepts, scores in all four courses were lower this time, although students in GEOG 3301 and 3350 met the 70% benchmark for success. This indicator highlights previously identified weaknesses in student outcomes, particularly in their ability to work with maps, interpret graphs, and analyze basic statistical data. Assessment results suggest that student learning in these areas is more effective in smaller class or lab settings. To address these challenges, program faculty will enhance instruction on these topics within lab sections of GEOG 1401 and incorporate targeted breakout focus groups in courses without dedicated lab components. Additionally, fundamental map skills will be integrated into geotechnical courses, as all majors are now required to complete GEOG 2464 (Introduction to Geographic Information Systems).
Sample Assessment Plans and Reports
As a further demonstration of ongoing quality assessment practices at SHSU, this narrative includes documents containing sample assessment plans for the last three, complete assessment cycles:
Table 1. Sample Assessment Plans
| College of Arts and Media | 2021-2022 [39] | 2022-2023 [40] | 2023-2024 [41] |
| College of Business Administration | 2021-2022 [42] | 2022-2023 [43] | 2023-2024 [44] |
| College of Criminal Justice | 2021-2022 [45] | 2022-2023 [46] | 2023-2024 [47] |
| College of Education | 2021-2022 [48] | 2022-2023 [49] | 2023-2024 [50] |
| College of Health Sciences | 2021-2022 [51] | 2022-2023 [52] | 2023-2024 [53] |
| College of Humanities and Social Sciences | 2021-2022 [54] | 2022-2023 [55] | 2023-2024 [56] |
| College of Osteopathic Medicine* | 2021-2022 [57] | 2022-2023 [58] | 2023-2024 [59] |
| College of Science and Engineering Technology | 2021-2022 [60] | 2022-2023 [61] | 2023-2024 [62] |
*Note: The Doctor of Osteopathic Medicine program is the only academic program within the College of Osteopathic Medicine.
The following procedure was used to select the sample, unit assessment plans: For departments containing two or fewer units, one was selected for inclusion; for departments of three to four units, two were selected; for departments of five to six units, three were selected; and for departments of seven or more units, four were selected. This selection scheme provided roughly 50% of each cycle's academic assessment plans. A complete repository for all educational program assessment plans for these cycles is included in the Assessment portion of the SACSCOC 5th Year Interim Report Website.
Distance Education
For the purposes of programmatic assessment, distance education programs at SHSU are classified as one of two types: (a) fully online programs, in which students can earn a credential through 100% online, asynchronous or synchronous modalities and (b) hybrid programs, in which students can earn more than 50% but less than 100% of a credential through online, asynchronous or synchronous modalities, and the remaining credits through traditional face-to-face modalities [63].
Distance education programs conduct and document their annual assessment efforts in the same manner as their traditional, face-to-face counterparts. As theory and practice regarding distance education assessment have evolved, OA has endeavored to create guidelines for these programs that align with generally recognized best practices. These guidelines are summarized within the Best Practices for Documenting Assessment of Online and Distance Education Programs document [64]. This document, revised in the summer of 2021, provides a summary of the recommendations and guidelines outlined within several SACSCOC documents: (a) Guidelines for the Evaluation of Distance Education (On-line Learning) [65], (b) Distance and Correspondence Education: Policy Statement [66], and (c) Guidelines for Addressing Distance Education and Correspondence Courses: A Guide for Evaluators Charged with Reviewing Distance Education and Correspondence Courses [67].
Programs that are exclusively available online report their assessment results and actions normally within Anthology Planning. Programs employing a hybrid model or programs that offer both a face-to-face and fully online option are expected to disaggregate their assessment results (where appropriate) [68] in Anthology and to use the results from both groups in formulating their actions for improvement [69]. An example of how hybrid programs use disaggregated assessment results to drive improvement is provided below.
During the 2023-2024 cycle, the Marketing BBA program [70] examined assessment results from both online and face-to-face students for the program’s learning objective that states, “students will be able to identify and differentiate fundamental principles and core concepts of marketing” [71]. These core marketing concepts and principles included the following:
This objective was assessed using embedded exam questions from online and face-to-face sections of the Fundamentals of Marketing course (MKTG 3310), with the expectation that 70% of students would score 70% or higher on all assessments. Student performance data for each of the 14 core marketing areas listed above were reported for face-to-face sections and online sections [72].
For example, regarding the concept of characteristics of the global marketplace, 65.52% of students enrolled in face-to-face sections met or surpassed the 70% criterion for the objective. For students enrolled in online course sections, 71.3% met or surpassed the criterion for success. Because students in face-to-face courses did not meet the benchmark for success, program faculty identified three actions designed to improve student learning and performance in this modality:
Graduate Program Review
Graduate program review represents another component of SHSU’s efforts to ensure quality educational programs. In accordance with Texas Administrative Code, Rule 2.181, Review of Existing Degree Programs [73], all graduate programs at SHSU engage in an external review process. This graduate review process is governed by the Texas Higher Education Coordinating Board (THECB) and is overseen at SHSU by the Dean of Graduate Studies [74].
On a rotating, 10-year cycle [75], each graduate program conducts a self-study [76] that addresses the aspects that are common to all graduate programs as well as aspects that are unique attributes of each program. A self-study is but one tool to guide programs in their continuous improvement efforts in meeting the challenge of serving the needs of students, the University, and external stakeholders. The graduate program self-studies provide an overview of the programs as well as a detailed study of the curricula, graduate faculty, program resources, assessment, student success, recruitment, and marketing.
The Self-Study Process
The self-study process incorporates three stages, examples of which are provided here as evidence: (a) the creation of the self-study [77] [78] [79], (b) an external review [80] [81] [82], and (c) the development of an action plan for improvement [83] [84] [85]. The program faculty and the support staff conduct a thorough program review and produce a report with appropriate support documentation. A team of external reviewers reviews the report, visits the campus and consults with program personnel and University administrators, and subsequently provides an evaluation of the program to include program strengths and recommendations for improvement. University leaders, in coordination with faculty, develop an action plan in response to the results of the self-study and external review. Examples of continuous improvement actions resulting from graduate program review include the MS in Digital Forensics' curricular revision to add a thesis option following reviewer feedback [83] and the PhD in Criminal Justice's curricular revisions to include reducing content overlap between the PhD and the MA and re-sequencing courses to resolve bottlenecks [84] The process is as transparent and inclusive as possible. The self-study, the external reviewers’ report, and the response are all submitted to the THECB.
Follow-Up Process
One year following the program review, the program director, chair, academic dean, and graduate dean meet to discuss the progress made on the action plan, which addresses the recommendations for improvement. Any outstanding issues or barriers to improvement are discussed and addressed.
Meta-Assessment
As part of University-wide efforts to promote quality assessment practices, OA also facilitates an annual meta-assessment process, utilizing a locally developed 4-point rubric [86] to evaluate the quality of programmatic assessment plans. This rubric was revised in fall 2022 to provide additional clarity over the prior version [87] and was used to evaluate plans for 2021-2022 and forward. The focus of the meta-assessment review is not on what is being assessed by each unit but rather on the quality of the assessment, with emphasis on the assessment practices and processes themselves.
Each academic college is requested by OA to devise a plan for evaluating the annual assessment plans for all academic programs in the college using the Meta-assessment Rubric [86]. The colleges are given some flexibility in how and when these reviews take place. For example, colleges have implemented a rotational cycle in which all plans are evaluated over a multi-year period, but some evaluate the prior year assessment plans in the following fall semester, after the close of the cycle, and others evaluate them during the summer, so edits can be made before the cycle closes.
Feedback from the annual meta-assessment reviews is used in two ways. First, completed meta-assessment rubrics are returned to the individual units for review and use in the continuous improvement of their assessment practices. Second, data from these reviews are used by University administrators to identify areas where training and resources are needed to improve programmatic assessment efforts. Examples of completed meta-assessment rubrics are provided to highlight this process [88] [89] [90] [91]. In addition, some examples of college-level summary reports are provided here to highlight how meta-assessment is being used at the college and institutional levels [92] [93] [94] [95] [96] [97] [98]. Within these reports, college leaders are asked to reflect upon the strengths and weaknesses they observed in reviewing the completed meta-assessment rubrics for their units and offer strategies for addressing them. These reports, along with the complete rubrics, are then used by OA to enhance assessment-related training and resources.