Skip to main content

Cornell University

Program-Level Assessment Guide

Learning Outcomes

What do we intend our students to learn from the program?

Instructional Methods and Learning Experiences

What methods and activities will we use with our students to help them learn what we intend from the program?

Assessment Methods

How will we know what our students have learned from the program?

“Closing the Loop”

What have we learned from this process that can improve the program?

 

Program-Level Learning Outcomes

What do we intend our students to learn from the program?

If you are working at specifying learning outcomes at the program level, it is important to start by seeing how the program fits into the larger institutional context. Recently, the schools and colleges at Cornell have collaborated in a process whereby the university has identified institutional learning “outcomes”. These are available on the Provost’s Assessment Web site.

The list was composed from reviewing each undergraduate school and college to determine those goals most common to all. At the program level, learning goals should reflect those of the parent school or college, and as suggested above, are more general than course-based outcome statements. The learning outcomes for the undergraduate colleges are available on the same web page listed above, and on the websites of the individual colleges. They are also accessible through the university’s home page.

An example of a general college-level outcome statement listed on the College of Agriculture and Life Sciences web site is, “Integrate quantitative and qualitative information to reach defensible conclusions.”

At the program level, the list of goals will necessarily be more specific to the program or discipline than those of the parent college. Examples of program-specific outcomes statements are, “Students will be able to clearly communicate ideas or research findings in anthropology through both written documents and oral presentations to public and college audiences.” and “Describe and apply basic biological information and concepts.”

Suggestions

At this phase of the assessment process it is helpful to review the most recent academic program review information to determine if learning outcome statements already exist for the program. If so, reviewing them for their relative appropriateness within the department or field may be necessary if they are over 5 years old. The department faculty should be in agreement as to what these most current student learning outcomes are to insure that they are covered within the program curriculum and courses offered. The number of outcomes for the program is best kept to around 5. Below are examples of program-level goal statements from a variety of disciplines:

Anthropology: Students will be able to explain and appropriately apply evoultionary theory to human and nonhuman primate biological phenomena; this should include ability to summarize the basic time-line and processes of general primate and specific hominid biological evolution.

Bioengineering: Students will be able to apply advanced mathematics (including differential equations and statistics), science, and engineering to solve the problems at the interface of engineering and biology.

English: Students will be able to evaluate how a text supports or defies the literary conventions of its genre.

Natural Resources: Students will broaden their social perspectives through exposure to diverse culture and thinking in course works, service projects, and departmental or college seminars.

Program-Level Instructional Methods and Learning Experiences

What methods and activities will we use with our students to help them learn what we intend from the program?

For an academic program, this step can best be accomplished by creating a matrix with the program’s general learning outcomes listed in the first column, and all the required courses (or those that most students in the program take) in the program placed across the top row. If a course supports the achievement of a program learning outcome, this is noted through a mark in the appropriate row and column. This process can help faculty note which courses address which of the program learning outcomes to help ensure that all students in the program graduate having achieved all specified program outcomes.

See an example of a Course Matrix pdf

Program-Level Assessment Methods

How will we know what our students have learned from the program?

Assessment methods should help the faculty in the program answer the question, “How do we know the required learning has taken place on a program-level?” Information gathered in this step is critical in program development and can be in two forms: direct measures and indirect measures of student learning. Direct measures provide observable evidence of the students’ level of learning. Direct measures on a program level include capstone projects, senior theses, exhibits, standardized tests, publications, portfolios, internship ratings of students, among others. Indirect measures evaluate student perceptions of their learning or generally indicate that students are or are not learning, but do not specify what or how much. Indirect measures, such as surveys, focus groups and evaluations, can be used to determine the relative quality of the learning experience.

Example Direct/Indirect Measure Summary Table pdf

Assessments at the program level serve to determine the degree to which the program’s structure, including courses required and assessment methods used provide sufficient evidence that the program has achieved its stated outcomes. In many cases, program assessment may rely on the course-based assessment data to determine its effectiveness. However, there are usually gaps that must be filled. Program-based assessment is looking at the larger picture beyond the courses taught. Examples relevant here include capstone courses, placement tests, exit interviews and alumni surveys. Useful questions at this stage include:

  • Is the program structure attracting sufficient numbers of students to maintain its allocated resources?
  • How well do students who graduate from the program do after graduation?
  • How do students experience the program: as exciting and confidence-building? as stressful? as ambiguous?
  • How well do graduates perform necessary skills both in the job market, and in subsequent education?


At this point in the process it is helpful to generate an assessment matrix like the one below by listing the program outcomes in the first column and inventorying all assessment methods used in the program courses across the top. Next, identify the match between outcome and assessment method. If there is an outcome that it not being assessed, how necessary or central is it? If it is to be retained, how can it be meaningfully assessed?

Sample Outcome Assessment Matrix pdf

Program-Level: "Closing the Loop"

What have we learned from this process that can improve the program?

Closing the loop is the justification of all the investment of thought and time expended in the overall assessment process: how can the program be improved by what has been found through the analysis? As one faculty member has said, “Assessment helps us figure out whether our students are learning what we think they’re learning.” A very important factor in this final phase of assessment is that of sustainability:

  • Have the program faculty developed a process that is useful and beneficial to all involved—faculty members as well as students?
  • Can and will the process be practically replicated as time and circumstance require?
  • Has the process produced the kind of data necessary for critical decisions to be made? How will assessment lead to program improvement?


At the program level it is most effective to view the entire process and the results it produces in terms of the stakeholders: the program faculty and department. It is also suggested to limit the agenda of the response in this step: don’t try to do too much all at once. Look at the data gathered as observations about the program, rather than just abstract measurements of its effectiveness. Thus, the task in this phase of the assessment process is to summarize the student learning information rather than abstractly quantify something that is more holistic. What practical questions and concerns do the faculty have which they hope to resolve? Possible outcomes at this step include more clearly understanding and articulating program learning goals, and knowing how to recognize them in student work. The results can also be used to make decisions about curriculum changes and staffing needs when faculty members retire or new hires join a department.

References

Walvoord, B. E. (2010). Assessment clear and simple:  A practical guide for institutions, departments, and general education (2nd ed.).  San Francisco, CA:  Jossey-Bass.

Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). San Francisco, CA: Jossey-Bass.
  Cornell University