A Successful Failure: Community Requirements Gathering for DSpace

Before I start this talk, a few disclaimers. First, nobody asked me to gather requirements for DSpace. Second, nobody vetted or co-wrote this presentation. It is entirely my own work, and I own any errors in it and any offense it causes. Third, I am a notorious gadfly and crank. I make trouble. Ask anyone!

That said, then, we all understand that this presentation represents my opinion only and does not represent that of my employer or the DSpace Foundation, right? Because if anyone’s going to land in the soup for this, it should be me, okay? Good. Onward.

DSpace is more than a software package; it’s a social phenomenon. And as such, it’s in trouble. Its developer pool is much too small, which has led to a self-reinforcing problem spiral: patches and hacks languish in the queue, repository managers and end-users get more and more annoyed at how slowly DSpace improves, developers try to placate them at the cost of their coding time, and seeing all this, potential developers decide to donate their effort elsewhere.

At the other end of the community, we have a great many silent end-users. Considering DSpace’s market position as the “out-of-the-box solution,” as well as its history as a research project between close collaborators MIT and Hewlett-Packard, helps explain why. People who pick a supposed out-of-the-box solution in the first place are not coders. In fact, they’re not even expecting to have to talk to developers about the product, and they’re less likely to know how to talk to developers than people who are more accustomed to open-source and its discourse traditions. As for DSpace developers, they currently take the very research-computing, classically open-source-developer perspective that “the price of a voice is code.” So end-users are silent; given the system they’re stuck in, what else can they be?

Technically, DSpace is lagging behind competitors at present. EPrints is much more usable and easier to market to faculty; it is also immensely easier to install. Fedora Commons is much more flexible and scalable, with a better, more generalizable data model. As for software-as-a-service providers, BePress’s Digital Commons is eating DSpace’s lunch on ease-of-use grounds. Librarians I’ve talked to who have looked at both and chosen Digital Commons tell me unanimously that DSpace is too staff-intensive to install and maintain in-house, and too inflexible to run consortially.

So combine DSpace’s social problems with its technical problems, and what we see is a wild profusion of hacks. Is anyone in this room running DSpace unmodified except for HTML and CSS changes? (No hands go up.) Right, I’m not either. We’ve got embargo hacks, electronic thesis and dissertation (ETD) hacks, statistics hacks, persistent-bitstream-URL hacks, authentication hacks, researcher-pages hacks, streaming-multimedia hacks… and worst of all, none of this ever makes its way back into the main DSpace codebase!

What is a hack, fundamentally? A hack represents an end-user need that DSpace is not meeting. And while all these end-user needs are not being met, DSpace somehow has time to add controlled-vocabulary support? Is anyone even using that? (One hand goes up, slowly.) Oh, good, I’d hate to think all that effort was a complete waste. But, seriously, who is setting development priorities here? It’s not me or anybody like me. Who is even being heard when those priorities are set? Again, it sure doesn’t feel like me.

So I said to myself, I said, “IR managers don’t feel they have a voice. Let’s give them one! Developers don’t feel that the community supports them. Let’s show different!” In the best case, engaged repository managers can help convince their library administrations to throw more resources (developer time, Foundation support funding) at DSpace. At the same time, potential developers will see a functioning community that they want to participate in.

The plan I then made included synchronous and synchronous discussion options, because DSpace is global. Moreover, repository managers are accustomed to the mailing lists; not so much to IRC. I designed in an option for private communication, too, because the current atmosphere can feel intimidating. The idea was that there would be a “question of the week,” which would then be summarized to the wiki for future reference, to avoid the disingenuous “gosh, when did you say you wanted that?!” reactions that repository managers often hear when we ask for something.

The venues turned out to be: the dspace-general and dspace-tech mailing lists for public asynchronous comments; the Meebo chat system for public synchronous comments until it got bot-spammed, at which point we moved to DSpace’s IRC channel; and my email for private asynchronous comments.

Discussions included:

  • Users’ most-wanted changes
  • An item-access statistics system
  • What the “ideal” repository system would act like and be capable of
  • Suggestions for improving the item-deposit interfaces
  • Suggestions for DSpace documentation

DSpace developers were extraordinarily good citizens during this experiment, extremely active and respectful. Brad McLean, Mark Diggory, Tim Donohue, Claudia Jürgen, others, the process had their attention, and they were willing to listen. Repository managers… well, that’s where “Houston, we have a problem here.” Despite there being dozens of repository managers for every developer, only a bare handful made any effort to participate at all, though Shane Beers and Christophe Dupriez were heroes. Six questions went out, five-and-a-half discussions were held… and manager participation was so low by the sixth question that it was perfectly clear this effort was a nonviable failure.

Why did it fail? Was I the wrong person to take it on? Very likely! I ran into some time crunches, and I’m also not very popular in the DSpace community (and now you know why). Were email and IRC the wrong venues? Would conference sessions and surveys have done better? Do librarians know how to give good feedback on software? Are the right people on the mailing lists? Do the lists reach local customizers and developers, for example? Or is it something I haven’t thought of? Very likely it is!

Why was it a successful failure? Well, we did surface and document some unmet needs. Those of us repository managers who took the trouble to speak up were heard and heeded. I’m here right now, talking to you frankly and honestly about this. And in the possibly-apocryphal words of Thomas Edison, we’ve learned a way it won’t work.

I want to close by questioning the idea of “a DSpace community” that I often hear people alluding to. Is DSpace one community? Where are the library administrators in this community? And how much faith do institutions have in DSpace’s processes and outcomes? If what DSpace has isn’t a community, what is it? Librarians aren’t used to the open-source “community development” concept to begin with. Might a different model work better? And if there is no community, or if it isn’t powerful enough, will DSpace survive?

I’ve shot my rocket here. The next launch is yours to plan and execute. Thank you.