Project Management


Methods for understanding large collaborative research projects

Issue: Climate change adaption and mitigation are complex issues that demand insight from a diverse group of scientists and stakeholders. While coordinated efforts across many academic disciplines and stakeholder interest areas may be the only way to address pressing societal challenges, managing these efforts requires valid and reliable measures of collaboration.

Action taken: As part of a multi-method assessment and evaluation plan, we conducted four annual surveys measuring perceptions of trust, collaboration quality, and research productivity in REACCH researchers, graduate students, and other professionals supported by the project. Items with highly variable responses across the project as well as items receiving the highest or lowest scores were highlighted and shared with project participants each year at annual, project-wide meetings. In addition, the use of summed scales and inferential statistics provided a way to make comparisons of trust, collaboration quality, and productivity across disciplines, objectives, and locations. These scores were used in combination with a social network analysis of participant collaboration to further identify areas of success and concern.

Results: The annual surveys gave participants and project leaders a way to quickly identify some areas of concern and more effectively use project-wide meeting time to create solutions. Social network graphs helped identify key collaborators within the project and provided a visualization of how objectives, disciplines, and locations were interacting. This information helped identify people, locations, disciplines, or objective areas that might need additional support.

Results Published In:

  • Mâsse, L.C. et al. (2008). Measuring Collaboration and Transdisciplinary Integration in Team Science. Am J Prev Med;35(2S): S151–S160.

A social network graph illustrating betweenness centrality (a measure of how an individual connects groups within an organization) and self-reported productivity across three major research sites and other geographically distributed people. Two sites have relatively high numbers of well-connected individuals and above-average productivity levels, while Site C and all other sites reported more moderate betweenness centrality and productivity scores.

Inclusive feedback: Making face-to-face meetings more productive

Issue: While some parts of large, interdisciplinary projects can work independently, many research, education, and extension opportunities demand coordination across geographically distributed groups. Good ideas and important information can come from anyone in the project, but electronically supported virtual meetings place limits on the number of people who can meaningfully participate and face-to-face meetings are resource intensive.

Action taken: We used interviews and four annual qualitative surveys of REACCH researchers, graduate students, and other professionals to understand what project management and coordination activities were needed to support the project. The online survey used open-ended questions regarding which project management and coordination activities participants deemed successful and which needed change. In addition, one question asked for other comments and suggestions that respondents wanted to share with the entire project.

Response rates varied between 77% and 88% over the four annual surveys. To help respondents share controversial or otherwise difficult issues, respondent anonymity was protected.The results were synthesized into a report and sent back to the respondents one week before the annual meeting. At the annual meeting, these results helped define the key questions for a set of large-group dialogues and solution-finding exercises.

Results: Including everyone supported by the project and protecting respondent confidentiality helped uncover some challenging project-wide issues that could then be discussed openly, better understood, and resolved. The long-term impacts of our work include graduate students and faculty who are able to address the many the challenges of large, interdisciplinary projects by applying the tools, methods, and models we used during the REACCH project. Participating researchers will also be better able to identify and use project-wide feedback to strengthen future project proposals. We have shared the tools, methods, and models we developed and applied in the REACCH project at two professional conferences focused on team science and evaluation. 

Results Published In:

  • Meyer, D. C. 2014. A mixed-method approach to understanding team science: Working towards useful feedback. Paper presented at the Science of Team Science Conference, August 2014, Austin TX.
  • Meyer, D. C. 2016. Designing for multiple targets: A mixed-method approach to interdisciplinary research assessment. Paper presented at the American Evaluators Association Conference, October, 2016, Atlanta, GA.