W. Trey Hill wrote an article for The Chronicle of Higher Education titled What Administrators Don’t Get About Data (12 July 2023).
« Too many campus leaders have trouble discerning useful data from the kind that is vague, irrelevant, or even specious… I’m a dean now, but as a social scientist I was trained extensively in data — how to collect it, how to design surveys, how to ensure validity and reliability, and how to interpret it accurately. »
« Step 1: Know what you want to know. If that seems obvious, let me just note that overreach is one of the most common mistakes made by administrators dipping a toe into the world of data-informed decision making… Whenever you move up the management ranks and acquire new authority, you naturally want to know more — to analyze everything, everywhere, all the time. But you don’t have the bandwidth to do that, nor do your direct reports, who, we all know, will end up doing the legwork to produce a lot of this data that you don’t really need and will never use … With your newly narrow list of things you want to know in hand, you can move to the next step: What data can realistically be collected, and how? »
« Step 2: Lean on experts for design, collection, and analysis. Administrators have a wide range of skill sets and backgrounds, but only a small subset of us learn — at a doctoral level of training — the technical aspects of survey creation, experimental design, and data analysis.
« Step 3: How to decide what to share… Once you begin receiving data, it is time to look back at your list of priorities and what you wanted to know. Think critically about why you wanted to know those things. It’s very likely that the “why” is rooted in solving some problem, for you, for students, for faculty and staff members, or some combination. The “why” should help narrow your focus on the next step: how to share the results. »
« You will probably want to share some results with a small group of people (e.g., your leadership team, your supervisor). Keep in mind that, while many or most of those folks are highly intelligent and have research backgrounds and doctorates, they already have a lot to read. Share too much information, and people may miss the message you want them to focus on. »
« Try to condense the data you’re sharing down to key points. Constantly refer back to your priority list. Recall the core reason why you wanted to know something, and then convey the core back to your target audience. »
« Step 4: Understand the data’s limitations. I see this as the most important step in the process — making sure you and your audience know what the data doesn’t mean.
« In my doctoral program, we were taught to always write a “limitations” section at the end of an experimental paper…. It shows trust in the researcher and demonstrates a transparency about both what the paper claims and what it doesn’t… Sure, acknowledging the weaknesses of your data can detract from the strength of the argument. But doing so will give you more credibility as a leader to the masses of faculty members, who were also trained to critically evaluate arguments. I recall being a young faculty member and feeling embarrassed for one administrator who seemed to have severely misjudged her audience and their ability to see through the facade to the many holes in her argument. By being upfront about data and its value, including its shortcomings, you steal the thunder of counterarguments and show that you are on top of the issue. »
« Data has tremendous potential to improve the quality of higher education, as well as the lives of our students and faculty and staff members. But you, as an administrator, can realize its full potential only if you use data in a thoughtful, ethical, and responsible way. »
See also The Tyranny of Metrics by Jerry Z. Muller.
Articles of Interest 