Page 24 - index

Tis roster presents the important criteria that need to inform considerations include speed, cost, sensitivity, literacy, and
project design. One additional consideration when collecting security, to name a few.
and presenting evidence is the intended audience. Persuading Another method that could result in clearer results is
instructors to change teaching practice based on evidence is a tiered approach. Perhaps you are trying to assess the
the last-mile problem. Constituencies difer with respect to the impact of podcasting on a 2,000-student blended business
kinds of evidence they consider persuasive. It is vital that project communications course. Afer confrming that the entire class
designers take this into consideration at the earliest stages of the has access to the Internet, you might begin your data collection
project. An early decision for almost any evaluation project is the with an electronic survey. Once the survey results are collected
question of what data collection methods are most appropriate. and analyzed, you might conduct focus groups or interviews
Most ofen, this consideration turns to the question of whether to to further probe or clarify information that may not have
use quantitative, qualitative, or mixed research methodology. been evident in the surveys. A small pilot using a couple of
Using Evaluation Tools Effectively designs may be useful to confrm that the desired results can be
obtained using that approach.
Although higher education does maintain well-established According to Brown and Diaz (2011), the way in which results
instructional traditions and practices, a fair amount of are reported is critical. Below are suggestions on how to do so:
innovation and experimentation pervades, and is encouraged
in, the teaching and learning enterprise. Higher education • Keep it simple, to the point, and brief. For example, reports
practitioners are devoting a signifcant amount of time and should include some statements about the innovation and its
resources to said innovations, necessitating designs, methods, impact (such as “podcast access increased by 98% during the
and tools to measure impact and continuously improve. fall semester”), a brief section on data analysis, a discussion
section, and a “bottom line” section where recommendations
A process for evaluating instructional innovations could include or suggestions are made. All other information can be made
questions about the student and faculty member reactions to available in an appendix section for further reading.
the innovation, support for student engagement, and support
for learning and transfer. Brown and Diaz (2011) suggest the • Know who is consuming your data or research report, who
following steps that may guide the process: the decision makers are, and how your data is being used to
• Establish the goals of the evaluation: make which decisions, if any. Match your data-collection
method to the type of data to which your information
What do you want to learn? consumer will respond or is likely to respect.
• Determine your sample: Whom will you ask? • Match your research design to the type of information in
• Choose methodology: How will you ask? which your anticipated consumers are interested or to which
• Create your instrument: What will you ask? they will best respond. Tis can be tricky and requires
advance planning, but the point is to tailor your design
• Pre-test the instrument: Are you getting what you need? to the specifc interests of the consumer and/or decision
Triangulation can be used to combine the advantages of both the makers. For example, a vice president of fnance may be
qualitative and the quantitative approach by utilizing the three more persuaded by quantitative reporting, while a dean
most commonly used evaluation tools: questionnaires (paper or in humanities may fnd qualitative reporting and telling
online), interviews (individual or focus group), and observations of a story more powerful. Tailoring your design in this
(classroom or online). Of course, the type of method you use manner can ensure that you are able to provide the kind of
will largely depend on your goals and audience, but other information that will facilitate the decision process.

24 LEADERSHIP Vol. 20.3 Winter 2015

Client: The Chair Academy Job: CHAIRAcademy_Journal_20.3 winter 2015 Final size: 8.5” x 11” Colors: CMYK Bleeds: Yes
Created by: Goldfsh Creative - Laura Dvir • 602.349.2220 • goldfshcreates@q.com