Page 23 - index

SEEKING EVIDENCE

OF IMPACT FOR

INFORMED VERONICA DIAZ, PH.D.

& ROGER YOHE, PH.D.
IMPROVEMENT

According to Kirschner (2012), by their very Approaches and Designs for Collecting Evidence: Te answers
nature, educational institutions almost invariably you fnd depend directly on the questions you ask. For the results
of an evaluation to be truly useful, it is crucial to ask the right
sufer from inertia and defense of the status quo. questions–that is, to design the evaluation in a way that will
While educational leaders look outside of their respective produce information that is useful to you and your stakeholders.
institutions for guidance from peers, foundations, and Using Evaluation Tools Effectively: No doubt we have all
professional organizations, they also must create a culture composed a survey, run a focus group, or conducted a set of
of critical inquiry within the institution and build collective interviews. How confdent are we that we use these common
intelligence in order to solve complex challenges. Te tools efectively? Are we confdent that we are acquainted with
EDUCAUSE Learning Initiative’s Seeking Evidence of Impact all the tools available to us? Do we know when to choose one tool
(SEI) program engages the teaching and learning community over another?
in a collective discussion about ways of gathering evidence of
the impact of innovations and current practices. Tis national Using Evidence to Infuence Teaching Practices: It may
discussion brings together all types of higher education seem that providing evidence alone will guarantee infuencing
institutions and professional associations into a conversation teaching practices, but instances in which evidence infuences
on this theme. Realizing the importance of using evidence to practice are more ofen the exception rather than the rule
improve practice, Mesa Community College (MCC) established (Buskist & Groccia, 2011). Using evidence to actually change how
a team that met regularly over four years to work with the teaching and learning is done is an art unto itself.
college community to create a culture of evidence-based Tese three areas of evidence-based practice are further
decision making dedicated to student success that aligns with explained in detail, and an example from one community college
SEI concepts. Adopting a robust set of data collection and applying these practices is provided.
analysis tools and heeding guidance from EDUCAUSE, MCC
was able to move beyond using perceptions and anecdotes to Approaches and Designs for Collecting Evidence
improve practice. Every evaluation project should have one core question it seeks to
With great change in higher education, it is timely to take a fresh answer; a good research question is two-thirds of the battle. Tis
look at evidence-based practice. Teaching and learning continues underscores the importance of the design and conceptualization
to see a rapid rate of innovation, fueled by constructivist learning of an evaluation project. Mistakes in the design phase will likely
principles and new technologies (McClenney, 2013). But do compromise the project, minimizing the efectiveness of the
we know what works and what does not? Are the gains real or results. Consider the National Research Council’s six criteria that
imagined? Just what is the impact of our projects? Measures of characterize a good research question (Shavelson & Towne, 2002):
impact and efectiveness are essential if we are to make informed • Signifcance: It addresses a question or issue that is seen as
decisions about what we need to support and promote. Perhaps important and relevant to the community;
most importantly, measures of impact are very persuasive for
our staf, including faculty and senior administration. Many • Specifcity: Te question focuses on specifc objectives;
institutions sooner or later will face fnancial difculties; in • Answerability: Te question can be answered by data
such times, when resources of all types are scarce, it is more collection and analysis;
important than ever to renew our thinking about evidence-based • Connectedness: It is linked to relevant research and theory;
practice in teaching and learning.
Te subject of evidence-based practice is a broad one. Brown and • Coherency: It provides coherent explanations that rule out
counter-interpretations; and
Diaz (2011) report it has many dimensions and complexities, so it
is useful to categorize them into these areas: • Objectivity: Te question is free of bias.

LEADERSHIP Vol. 20.3 Winter 2015 23

Client: The Chair Academy Job: CHAIRAcademy_Journal_20.3 winter 2015 Final size: 8.5” x 11” Colors: CMYK Bleeds: Yes
Created by: Goldfsh Creative - Laura Dvir • 602.349.2220 • goldfshcreates@q.com