What is “Impact” and how do you measure it?

by Anneliese Poetz, KT Manager, NeuroDevNet

For NeuroDevNet, impacts of research and training are achieved when children with neurodevelopmental disorders:
• Are diagnosed sooner
• Receive validated interventions as soon as possible
• And their families are supported through the life span

Related to these, impact is achieved when we make a difference – changes to existing policies or the implementation of new ones, or changes in the way caregivers and/or health practitioners approach their work with children and families. Impact is also helping improve the quality of life for children and families in unexpected ways. In the field of Knowledge Translation (KT), there is still ambiguity about how to measure and report on ‘impacts’ of KT. NeuroDevNet frames its Knowledge & Technology Exchange and Exploitation (KTEE) activities using Phipps’ Co-Produced Pathway to Impact evaluation framework which encompasses the Network’s KT activities as well. If impact is what we are trying to achieve, then KT is one of the means to help us achieve it.

Phipps' Co-Produced Pathway to Impact Evaluation Framework
When thinking about KT in terms of evaluation and reporting on KT activities, several quantitative measures easily come to mind: # of peer-reviewed publications, # of citations of one’s research publications, # of conference presentations, # of KT Products created, etc. However, these measures do not go far enough – notice that these are all indicators in the ‘dissemination’ phase of the CPPI. Typically, this is where KT activities ‘stop’ – it is the point of departure for researchers move onto the next research project. But stopping at the “dissemination” (otherwise referred to as end-of-grant KT) stage doesn’t help you measure the impact of your research.

In order to find out whether your research has been considered useful (in practice, or policy, or otherwise) you have to go and ask the people 1) who you engaged in your research process (integrated Knowledge Translation), and 2) who you imagined would find your research useful once it was completed even if they did not directly participate in informing your research questions or process. Yes, qualitative interviews!

The KT Core conducts qualitative interviews with its researchers, trainees and most importantly its collaborators and partners. There is a lot of good work going on in the Network, and these interviews are for the purpose of discovering stories about how NeuroDevNet’s research and training have made a difference. Some of them might not have otherwise been discovered and/or reported on. The first interview is always with the researcher or trainee. Then, we ask them who their collaborators/partners were, and whether they would be willing to broker an invitation for an interview so we can ask questions about the impact of NeuroDevNet’s work from their perspective. How have we changed things for them in their organization? Their practice? For the children and families they serve?

An example of a story we discovered was through one of our trainees, Angelina Paolozza, in the FASD program of research. Angelina was invited to present at Adopt Ontario after someone from that organization saw her present her research at a local hospital. Angelina was able to adapt her presentation style to be compatible with an audience of prospective parents. After her presentation (the 2 times she has been invited) the audience had the same response – many parents said that now that they understood FASD after hearing her describe her research they would revisit the files they had reviewed on children with FASD. Talk about impact – a child in a stable home has a much improved life trajectory and quality of life. The basic underpinning of any effective KT activity is relationships – and this impact was achieved through the relationship built between NeuroDevNet and Adopt Ontario.

Getting these stories is not just useful for reporting purposes, but it is also valuable for us as a Network to learn what works and what needs more attention/improvement in terms of our collective KT activities. By learning how we can best achieve impact, we can maximize the chances that we can repeat and scale our efforts.

If you are a NeuroDevNet researcher, trainee or collaborator/partner and you have a success story you would like to share, please contact the KT Core and we can help draft it into a formal ‘success story’ to be placed on the NeuroDevNet website as part of a series.

LinkedIn for knowledge translation: using groups for networking

By: Isaac Coplan (KT Coordinator)In-2C-121px-R

Networking is important to knowledge translation (KT), as relationships are a key part of KT processes. This is where social media can be useful in KT. Websites like LinkedIn provide a platform in which to expand your network and meaningfully engage with stakeholders. If used properly, social media can be incorporated into Integrated KT strategies as well as end-of-grant research dissemination.

What is LinkedIn?

In the Social Media for KT resource (What is social media & where to start) I wrote about LinkedIn.

LinkedIn is known to cater to professional audiences. They are also well designed so that search engines (such as Google and Bing) privilege information on their pages. This means that they will frequently be higher up when someone runs a web-search on your name (this process is also called search engine optimization).

LinkedIn was designed with the idea of allowing a place for professionals to connect online. It works as a sort of online resume or CV and online presence that can be populated with links, articles and posts. LinkedIn is not just about seeking employment, it can be an important tool to connect with a wide range of stakeholders. In April 2014, LinkedIn announced that it reached 300 million registered users, up from 200 million in 2013.

LinkedIn Groups

noun_15878_ccOne thing that I should also mention is the power of LinkedIn groups to expand your professional network. In LinkedIn groups, people frequently post questions or scenarios to their group, this allows for a conversation to occur naturally.Research Impact used their LinkedIn group to pose questions to KT practitioners in order to differentiate between knowledge translation and communications.  Analysis from the responses to this question on LinkedIn led to a research paper.

Groups can easily be searched (this Boolean search Tip sheet from LinkedIn is helpful). This provides you with access to over 1.5 Million groups. The search feature easily shows you if any of your existing connections are in groups and the relative popularity. This can allow you to quickly determine the groups that are already relevant to your networks.

How can expanding your LinkedIn network help you with your KT?
There are several benefits of networking that include:

  • Gaining greater visibility in professional circles
  • Being able to contribute to online conversations in your field
  • Providing another place for audiences to discover and contact you

In addition, expanding your research teams’ networks can become a rich source for getting feedback on your work. Two ways that this can be achieved are through:

  1. Gathering feedback from stakeholders to inform your research questions and approach
  2. Evaluating the work you have already completed.

Instead of creating a LinkedIn group that we would have to recruit members for, the KT Core expands our networks (connections to our profile page) by targeting policymakers, practitioners and other researchers that may find NeuroDevNet’s research useful in their work and sending them an invitation to connect.

LinkedIn can also be a part of a strategy to evaluate KT Products. For the evaluation of ResearchSnapshots, the KT Core sent personal messages to selected members of our LinkedIn network.  We asked the same questions of stakeholders in: Cerebral Palsy, Fetal Alcohol Spectrum Disorder, and Autism Spectrum disorder and included a link to the ResearchSnapshots on our website for each of these major projects.  We wanted to answer questions like:

  • Do you find the snapshot a) interesting, b) useful, c) both useful and interesting? D) Neither useful nor interesting?
  • How have/would you use these ResearchSnapshot(s)?
  • If you would not use these ResearchSnapshot(s), why?

This provided the KT Core with valuable insight into the ways that different products are used, or could be used by different knowledge users.

If you are a NeuroDevNet researcher or trainee we can help you learn how to use LinkedIn for KT, or help you advance your existing social media strategy, contact the KT Core!