October 11, 2022

A Cohort-Based Pilot of the FOREST Framework for Values-Driven Scholarly Communication

By Sarah Lippincott

The FOREST Framework for Values-Driven Scholarly Communication helps scholarly communication organizations and communities demonstrate, evaluate, and ultimately improve their alignment with key values, including: Financial and Organizational Sustainability; Openness and Interoperability; Representative Governance; Equity, Accessibility, and Anti-Oppression; Sharing of Knowledge; and Transparency.

It does so by defining concrete evaluation mechanisms that communities can use to assess how their policies and practices align with these values. It prompts communities to consider the values they hold and how they demonstrate and communicate their commitment to these values. It provides guidance on actions they can take to manifest their values more effectively, and it encourages communities to see themselves as part of an interconnected system (or commons) in which their actions and decisions directly affect other participants.

This summer, the NGLP project brought together a cohort of scholarly communications communities, including for- and non-profit service and technology providers and library publishers, to pilot the FOREST Framework. 

Pilot participants were asked to evaluate their organization against the Transparency value, using the included worksheets and a meeting facilitation guide provided by the Framework’s co-authors, Sarah Lippincott and Katherine Skinner. The facilitation guide defined the goals and activities for two internal meetings, scripts for the leader of each meeting, instructions for asynchronous work between sessions, and suggestions on tools for group work in virtual or in-person environments.

The first meeting, a reflection session, used the Framework’s reflective practice prompts to bring the team into conversation about Transparency, asking them to note which principles resonated most, identify productive avenues for further discussion, and establish a plan for asynchronous work and documentation gathering prior to the second meeting.

The second meeting, an evaluation session, brought the team together to review documentation and evidence and record its current status on the scoring worksheets included in the Framework. Teams were also encouraged to record goals and milestones for gap areas and growth areas identified in this session and/or in the asynchronous work.

In a debriefing session held with the full cohort, we asked them to reflect on their experiences and surface ideas for improving the FOREST Framework and associated materials.

Our pilot participants overwhelmingly reported that these sessions prompted deeper conversations than they typically make time for within their team and resulted in concrete and specific plans for action, including creating and revising policies and procedures and changing the way they communicate or present themselves to stakeholders.

They found that the scaffolding of the facilitated sessions helped to create a non-judgmental environment in which they could have productive and positive conversations about potentially sensitive topics. Participants were “surprised by the quality of conversation” generated in the meetings and noted that the format encouraged rapid engagement on a deeper level with “no skirting around the issues.” One participant noted that their team “spoke about things in depth that we often don’t and took the time to bond around shared challenges.” Even teams that were “deeply skeptical about structured activities … engaged with it seriously and in a meaningful way.”

The pilot participants valued the Framework’s “practical, step-by-step” approach, which allowed them to come away with a clear action list. Several participants noted that they planned to work through additional values with their teams or incorporate the FOREST Framework into strategic planning exercises. They also said they could see using this tool to evaluate potential collaborators or service providers to ensure they are supporting other values-driven organizations.

In terms of areas for improvement, the pilot participants recommended changes that will make the Framework more applicable to a wide range of organizational models. For example, they encouraged us to modify some terminology to make clear that the Framework doesn’t assume that total transparency is always desirable. They asked for clearer definitions and a statement of assumptions upfront to assist users in situating themselves and their organizations in the Framework’s context. In a few instances, the terminology used or the issue addressed did not sufficiently reflect our strengths-based approach, too heavily focusing on negatives.

They liked the progress bar rating scheme, which encouraged them to see themselves on a spectrum of alignment, but requested some additional options to better reflect their status. For example, they wanted options that acknowledge that they endorse or advocate a principle, but have either not yet taken concrete action towards it or it is not applicable to them directly. They requested an option to represent the idea that, “we’ve never thought about this, but now we’d like to.” Participants also suggested adding a page to the worksheets for them to record a list of follow-up actions or strategies.

Finally, pilot participants helped us identify where we might need to recalibrate parts of the Framework to ensure that it strikes the right balance between holding communities to a high standard while not making it impossible to excel. 

The pilot clearly demonstrated the Framework’s value. It resonated with the scholarly communications community and sparked deep and productive conversations. It effectively engaged a range of organizations in thinking about the core motivation of the Framework, the community’s desire to “work with trustworthy partners” (not just those that claim to be trustworthy). 

We’ll be incorporating the lessons learned from this pilot into the next round of revisions to the Framework and additions to the contextual materials we provide with it. As we do so we welcome any questions or comments to sarah@educopia.org.