July 29, 2022
Reviews in the Digital Humanities
By Roopika Risam and Jennifer GuilianoAs a follow on to our Building Data Resilience through Collaborative Networks Symposium, we are publishing a blog series featuring each of our presenters. This is the first in that series.
Founded in 2019 and launched January 2020, Reviews in Digital Humanities is an open-access, peer-reviewed journal facilitating scholarly evaluation and discovery of digital scholarship. It responds to the gap in the peer review process for digital scholarly outputs (e.g., digital archives and collections, multimodal scholarship, exhibits, visualizations, games, tools, applications) by publishing monthly journal issues, through the MIT Knowledge Futures Group platform PubPub, that pair 500-word project overviews submitted by project directors with 500-word reviews by members of the digital humanities community. Each review is documented in the journal’s project registry, tagged with keywords for time period, topic, and method, to aid with project discovery and information sharing.
Peer review is one of the most pressing social justice issues in scholarly communication, complicated by the distinctions in genre between more “traditional” forms of scholarship (i.e. monographs, journal articles, book chapters) and digital scholarship. In many academic fields, research demonstrates that peer review can be racially biased. For scholars of color and Indigenous scholars, whether creating analog or digital scholarship, the storied “Reviewer 2” is not simply offering cruel assessment of their work but often questioning the significance and validity of the research topic and methods deployed. At the same time, digital scholarship is not sufficiently valued because of lack of consensus — among digital humanists and disciplinary colleagues who work in traditional genres alike — on how to evaluate scholarship without peer review, the gold standard of externally vetted scholarship. As a result, digital humanities scholars are at a disadvantage in the tenure and promotion process, while scholars of color and Indigenous digital humanities scholars are doubly penalized.
Reviews intervenes in this problem of peer review, which we see as a primary deterrent to the growth and development of digital humanities in ethnic studies fields (e.g., Black studies, Native and Indigenous studies, Latinx studies, and Asian American studies). Accordingly, the objectives of Reviews are to: 1) foster critical discourse about digital scholarship in a format useful for scholars and evaluators; 2) build the capacity of scholars to review digital scholarship in humanities disciplines; 3) provide peer reviews to substantiate cases for promotion, tenure, and external funding; 4) offer timely peer review in a format advantageous to digital scholarship; and 5) create records for digital projects that persist separately from project themselves to address long-term project discovery. Thus, Reviews promotes justice in digital scholarship in three registers: providing peer review that supports the incorporation of new scholarly perspectives and disciplines into digital humanities through the peer review process; supporting scholars undertaking justice-driven practices that decenter the individuals who have long dominated digital humanities conferences and publications; and offering clear criteria that allow a peer review community to conduct review with a shared system of values balancing technical and humanistic inquiry.
The journal’s model for peer reviewing digital scholarship has five core components. First, Reviews identifies projects for review through multiple avenues. Project directors may self-nominate their projects for review. The journal’s editorial board members nominate projects drawing upon their linguistic, geographic, cultural, and disciplinary knowledge and networks. Additionally, the co-PIs identify projects for review through programs of major digital humanities conferences. Second, recognizing that projects are developed in phases and that “finished” is a moving target for digital humanities projects, the co-PIs allow project directors to seek review at any point in a project’s development that they deem appropriate. Third, responding to the challenge of reviewing digital scholarship, which requires attention to both technical specifications and humanities content, the co-PIs mentor reviewers to address both humanities and technical content. Fourth, to build on the important work done to develop guidelines for evaluating digital scholarship by professional associations, Reviews incorporates these guidelines into the peer review process as appropriate. Fifth, Reviews allow for future re-review of projects as they develop and meet new milestones. With this broad-ranging approach, Reviews addresses the issue of evaluation of digital scholarship for tenure and promotion.
As of July 2022, Reviews has more than 79,900+ page views, with 18,100+ unique users from 125 countries. Among fields of study for the projects we have reviewed, 30% of projects are in African diaspora studies, 21% are in Latinx studies, 9% are in Global Indigenous studies, and 19% are in Queer studies. The remaining 21% come from a variety of fields. We have published 12 special issues focusing on Borderlands, Latinx DH, Jewish DH, Classroom Pedagogy, Sound, Digital Pedagogy, and Black Digital Humanities.
As we plan for the future of Reviews, we have been working with Katherine Skinner and Brandon Locke of Educopia Institute to align our future activities with the objectives of our journal, as well as our commitment to creating avenues to support the growth of digital humanities in ethnic studies fields. Through our work, we have identified several core issues that we need to address to sustain the review culture we are building: keeping apace with demand for peer review, fostering spaces for subsets of our community to grow, and building out the project registry that has become a source of project discovery for digital humanities communities.
To address the first two concerns, we have developed a model for “verticals editors” — small editorial teams that will be responsible for cultivating reviews and review culture in areas of study that are essential to our mission (e.g., African diaspora studies, Global Indigenous studies, Latinx studies, endangered cultural heritage, Queer studies) alongside the open submissions process we run. Collectively, the verticals function like poles of a tent, working alongside us to uphold the mission of the journal. We will train these teams and work with them to support their vertical. We are currently seeking funding to pilot this model for a period of three years. After three years, we will work with the verticals editors to determine whether their vertical should be continued or discontinued (and thus replaced by a new vertical topic). Should a vertical be continued, we will strive to move it into an imprint model with an independent editorial board that operates largely independently under the umbrella of Reviews in Digital Humanities.
We are already experimenting with our imprint model through MOUs that we have signed with other journals and projects, such as the Recovery Hub for American Women Writers. The Recovery Hub runs its own peer review process based on our review model, and we have agreed to publish an issue a year featuring the review content they provide us. In this way, Reviews in Digital Humanities shows potential to be a hub for digital scholarly reviews.
To address the project registry, we intend to bring on a librarian partner with expertise in digital humanities and in issues surrounding discovery, to work with us on making the registry more useful and to explore its role in project sustainability. Through our work in our early years, we’ve seen the value of Reviews in Digital Humanities in multiple registers and look forward to continuing to refine our organizational and governance structure and develop a business model to facilitate a robust digital scholarship review culture in the years to come.