Tell ACRL to fix its assessment guidelines

(The below is a lightly-edited and augmented Mastodon thread. Please forgive prolixity and redundancy.)

HEY, anti-surveillance librarian-type folks! HEY, anti-surveillance higher-ed-type folks! ACRL is trying to ethicswash surveillance assessment again! You can read how here. You can comment, too! I gave them several earfuls. If you feel so moved, comment yourself.

First, a bit of context for the non-librarian-types. ACRL is the Association for College and Research Libraries. It is a division of the American Library Association focused specifically on academic-library stuff. It has put out guidelines on various matters previously; this is a thing it regularly does. So a set of guidelines on assessment is quite in-scope for them.

Here’s the problem, from where I’m sitting. In 2010, ACRL went all-in on “library learning analytics,” or as I like to call it, “surveillance assessment” (and steal that term all you want, no credit needed, I want it to become common discourse). Here’s where. The so-called “value agenda” has lots of moving parts, but one of them is (my paraphrase, but I have some receipts): “grab all the data on students and use it to the hilt to show how awesome libraries are.” By all means go read the report and see if you get a different impression; I’ll wait.

If any of you are on IRBs or otherwise know some things about research ethics or power, your alarms are already tripping, right? Researchers lab-ratting other people for their own benefit goes real nasty places real fast. And in practice so far, surveillance assessment of students is the only piece of the value agenda that has gotten any traction in libraries. The value-agenda folks have told on themselves time and time again. My favorite example is the table of contents to this 2018 report, which posits privacy as an obstacle to learning analytics. When. They. Tell. You. Who. They. Are. Believe. Them.

There are definitions of learning analytics—most definitions, actually—that posit a priori (and without proof, or indeed much evidence) that it is for the benefit of students. The Value Agenda for Libraries has never claimed this. It has only ever held promoting libraries as a goal. So one thing y’all can tell ACRL is that ACRL needs to make clear that self-interested, self- and/or library-promoting assessment violates the beneficence principle of ethical research and assessment. Beneficence is directly mentioned in the draft.

Okay. Let’s go through the draft now, starting with the Preamble.

“underlying expectation that assessment practitioners represent and promote the voices of all members of their campus communities,”

That is some vocational-awe nonsense right there. How about “inform?” Or “involve?” Or even “protect?”

“Apply ethics to guide all assessment activities.”

And here we have the Empty Genuflection to Ethics, a super-common “feature” of value-agenda discourse. Tell ACRL it’s not good enough. First they need to specify what kinds of ethics are in play. Possibilities include: human-subjects research ethics, library-specific ethics (hold that thought), ethics of care, feminist ethics. Then they need to specify how to apply ethics: specifically, that unethical modes of assessment must not be allowed.

The big library-specific ethics issue—probably obvious to the librarians in the room but let me say it so we’re all on the same page—is privacy. ALA Code of Ethics, Article 3, “We protect each library user’s right to privacy and confidentiality with respect to information sought or received and resources consulted, borrowed, acquired or transmitted.” Please note that it’s confidentiality and privacy. Patrons get both, not one or the other!

The word “privacy” appears zero times in this document. Zero. If you do nothing else, please inform ACRL that the absence of privacy from this document is unacceptable. Cite the ALA Code if you like. This is the big ethicswash in action, y’all! Don’t let ’em get away with it! Tell ACRL that you expect this document to state that any assessment practice that violates patron privacy is unethical. The start of the recently-added Article 9 is good too. “We affirm the inherent dignity and rights of every person.” Surveillance assessment violates both.

Okay, on to proficiencies. The Engaging Ethically section is a pile of ethicswashy nonsense. Pin them down, please. Tell them that wishywashy verbs like “affirm” and “maintain and enhance knowledge and skills” are not gonna cut it. Tell them you need to see verbs associated with actual assessment practices, like “collect, store, analyze, minimize, and secure” for starters. Notice also the absence of anything around data security or data governance anywhere in this document. Unacceptable. Tell them so.

“… advance racial and social justice in our libraries, communities, and profession through awareness, advocacy, education, collaboration, and allocation of assessment resources.”

But not, apparently, by actually doing assessment ethically. Unacceptable. Tell them so.

“Critically engage with questions around current and past assessment practices to ensure ethical practices are being followed and equitable outcomes are being produced.”

No. Ensure ethical assessment practices, full stop. “Critical engagement” is ludicrously insufficient. Do assessment ethically or not at all.

On to the building-knowledge section.

“Paired with the knowledge of social justice, equity, diversity, inclusion, and accessibility issues, practitioners are challenged to consider how assessment activities can be used to dismantle inequitable systems and structures.”

Consider? Just consider? No. We are challenged to dismantle, full stop. Also tell ACRL that practitioners must know about the history and present of inequitable data practices and uses, especially in libraries. None of this techbro-style “who could have known surveillance would be bad for the surveilled?” excuse-making.

On to part 3, on design, collection, and analysis.

There is, again, no mention of privacy here. This is exactly where privacy needs to be spelled out in operationalizable terms. Tell ACRL to do that. There is also no mention of data minimization, data security, or the appropriate boundaries of patron-data sharing here. Unacceptable. Tell ACRL to fix it.

“Balance the need for comprehensive data with being able to critically evaluate what data are needed to make informed decisions.”

Tell ACRL to add that no data gathered or handled in an unethical fashion can be considered “needed” for any decision ever, and that no data that violates the information privacy of library patrons can be considered “needed” for any decision ever.

“Understand and practice appropriate data management and handling techniques.”

This is just so milquetoast. Tell ACRL to be specific! DATA SECURITY. DATA MINIMIZATION. DEIDENTIFICATION. PRIVACY. GOVERNANCE. As is, this is unacceptable. Tell ACRL to fix it.

Section 4, reflecting and making meaning.

“Reflect upon whether the techniques used to analyze and present data are appropriate and effective within a particular context.”

How about reflecting on ethics? And how about adding “collect” to “analyze and present” here? Reflecting on ethics is also not enough. Reflections that lead to the conclusion that a technique is unethical must mean the end of the project, retractions of public communications based on the data if needed, a public incident report, and an apology to affected patrons.

“Determine whether assessment data are sufficient before making decisions or taking actions based on the results.”

This is dangerous. It directly advocates endless data grabs. Tell ACRL to put limits on it. “Determine whether collecting additional data adds risk to patrons, including to patron privacy.”

In the interests of sparing my hands, I’m skipping section 5. I want to get to section 6, the leadership section. There is nothing in it about ensuring ethical assessment. Not a single word. This is absolutely unacceptable. Tell ACRL to fix it. The ethical buck stops with leaders.

“Prioritize social justice, equity, diversity, inclusion, and accessibility goals wherever possible and motivate colleagues and team members to conduct equity-centered assessment.”

Prioritize privacy and ethics!


“Have I considered the values and ethics that guide my work and align to my institution, overall and in specific assessment projects?”

Again, considering is not enough. You enact values and ethics or you don’t. Tell ACRL so, and tell them that not enacting ethics is unacceptable.

Bibliography: contains not one single piece critical of the ethics of surveillance assessment. ACRL done told on itself again. Call them on it! The critical pieces absolutely exist! I dumped half my recent manuscript bibliography in ACRL’s Google form.

So yeah. There’s my take. Do with it as you please!