Sustainability and Knowledge Translation: sessions at Canadian Knowledge Mobilization Forum 2015

by Anneliese Poetz, KT Manager, NeuroDevNet

This past Thursday May 14, 2015 and Friday May 15, 2015 the 2015 Canadian Knowledge Mobilization Forum took place at the Grand Bibliotheque in Montreal, QC. The Canadian Knowledge Mobilization forum is the national conversation on KT/KMb practice and an excellent way not only to build our own skills but to brand NeuroDevNet as a leading KT organization.  In fact, it was during this event that David Phipps (NeuroDevNet KT Lead) received the “2015 President’s Award for innovation” in “recognition of his extraordinary contribution to the field and practice of Knowledge Mobilization in Canada and internationally”.

14/05/2015 ckf15 Photo Pedro Ruiz

Peter Levesque presents Knowledge Translation award to David Phipps at CKF 15
Photo credit: Pedro Ruiz

The overall theme of the conference was “Creativity as Practice: Mobilizing Diverse Ways of Thinking”. I both learned from other presenters, and shared my own knowledge.

In the workshop “Narratives, video and smartphones as KT tools for youth” (by Sean Muir) I learned that the ‘formula’ for maximizing effectiveness of KT with youth is: grab their attention with a shocking image or story, present your content/message, and then end with something positive. Sean used examples of videos and posters to illustrate this point. In the workshop on “Mobilizing your message through documentary video: research findings as cinematic narrative” (Callista Haggis et al.) the takeaways for creating KT videos were “done is better than perfect”, “show don’t tell” and “think about what you want your target audience to think, feel, do”. In this case, the documentary was both to present research findings in an alternative format, as well as to inspire discussion about the issues presented in the video toward possible infrastructure changes to accommodate the needs of an aging population.

NeuroDevNet’s KT Core Lead, David Phipps, participated in leading 2 sessions. One session was with Purnima Sundar (Ontario Centre of Excellence for Child and Youth Mental Health) and Renee Leduc (NCE Secretariat).

ReneePurnimaDavid_CKF15The audience gained insight into the 3 common reasons why research grant applications fail: 1) lack of meaningful end user engagement, 2) unclear pathway to impact, and 3) poor evaluation of KM (Knowledge Mobilization) and of impact. The NCE Secretariat provided tips on how to prepare a successful research funding application, and held an interactive session asking for the audience’s ideas for what the NCE Secretariat could do to help applicants be more successful. Ideas included: successful applicants’ mentoring of new applicants, creation of how-to videos to accompany written grant application instructions, and provision of examples.

David moderated the session on “the paths of sustainability for KMb” in which I was one of the 4 presenters. I presented on the KT Core’s evaluation framework, indicators, and 3 factors relating to sustainability: relevance (how does what we’re doing fit with our priorities), leadership (who is responsible for ensuring outcomes are met), and financial (can cost-effective strategies be used).   The presentations were 10 minutes each. When the presentations were over, each presenter took their discussion question to a corner of the room and invited attendees to join their group (depending on which question most interested them) and discuss it further in terms of their own context.

PicFromDJP_sessionCKF15The questions were:

– How are people attempting to influence sustainability across diverse settings with the use of tools?
– How can we sustain KT implementation through strategic planning?
– How can team capacity and culture be shaped over time to best meet the needs of knowledge users?

And my question was:

– What factors should be considered with respect to sustainability?

I had about 12 people in my breakout discussion group. Although I had a discussion question prepared, I received several questions about what NeuroDevNet’s KT Core does in terms of evaluation and also about database design and development. After the breakout discussions we returned to the large group and each presenter did a ‘report back’ about what their group discussed.

“Anneliese provided a great overview of the process she developed to measure the relevance and impact of knowledge translation products. Her experience was very relevant as our organization is currently exploring different methods of evaluating our work. We look forward to learning more about Anneliese’s indicators and database.”
– Sheena Gereghty, Canadian Centre on Substance Abuse

If you are a NeuroDevNet researcher or trainee and would like help with KT videos, advice on event evaluations and/or evaluation of your other KT activities and products, contact the KT Core to find out how we can help.

What is “Impact” and how do you measure it?

by Anneliese Poetz, KT Manager, NeuroDevNet

For NeuroDevNet, impacts of research and training are achieved when children with neurodevelopmental disorders:
• Are diagnosed sooner
• Receive validated interventions as soon as possible
• And their families are supported through the life span

Related to these, impact is achieved when we make a difference – changes to existing policies or the implementation of new ones, or changes in the way caregivers and/or health practitioners approach their work with children and families. Impact is also helping improve the quality of life for children and families in unexpected ways. In the field of Knowledge Translation (KT), there is still ambiguity about how to measure and report on ‘impacts’ of KT. NeuroDevNet frames its Knowledge & Technology Exchange and Exploitation (KTEE) activities using Phipps’ Co-Produced Pathway to Impact evaluation framework which encompasses the Network’s KT activities as well. If impact is what we are trying to achieve, then KT is one of the means to help us achieve it.

Phipps' Co-Produced Pathway to Impact Evaluation Framework
When thinking about KT in terms of evaluation and reporting on KT activities, several quantitative measures easily come to mind: # of peer-reviewed publications, # of citations of one’s research publications, # of conference presentations, # of KT Products created, etc. However, these measures do not go far enough – notice that these are all indicators in the ‘dissemination’ phase of the CPPI. Typically, this is where KT activities ‘stop’ – it is the point of departure for researchers move onto the next research project. But stopping at the “dissemination” (otherwise referred to as end-of-grant KT) stage doesn’t help you measure the impact of your research.

In order to find out whether your research has been considered useful (in practice, or policy, or otherwise) you have to go and ask the people 1) who you engaged in your research process (integrated Knowledge Translation), and 2) who you imagined would find your research useful once it was completed even if they did not directly participate in informing your research questions or process. Yes, qualitative interviews!

The KT Core conducts qualitative interviews with its researchers, trainees and most importantly its collaborators and partners. There is a lot of good work going on in the Network, and these interviews are for the purpose of discovering stories about how NeuroDevNet’s research and training have made a difference. Some of them might not have otherwise been discovered and/or reported on. The first interview is always with the researcher or trainee. Then, we ask them who their collaborators/partners were, and whether they would be willing to broker an invitation for an interview so we can ask questions about the impact of NeuroDevNet’s work from their perspective. How have we changed things for them in their organization? Their practice? For the children and families they serve?

An example of a story we discovered was through one of our trainees, Angelina Paolozza, in the FASD program of research. Angelina was invited to present at Adopt Ontario after someone from that organization saw her present her research at a local hospital. Angelina was able to adapt her presentation style to be compatible with an audience of prospective parents. After her presentation (the 2 times she has been invited) the audience had the same response – many parents said that now that they understood FASD after hearing her describe her research they would revisit the files they had reviewed on children with FASD. Talk about impact – a child in a stable home has a much improved life trajectory and quality of life. The basic underpinning of any effective KT activity is relationships – and this impact was achieved through the relationship built between NeuroDevNet and Adopt Ontario.

Getting these stories is not just useful for reporting purposes, but it is also valuable for us as a Network to learn what works and what needs more attention/improvement in terms of our collective KT activities. By learning how we can best achieve impact, we can maximize the chances that we can repeat and scale our efforts.

If you are a NeuroDevNet researcher, trainee or collaborator/partner and you have a success story you would like to share, please contact the KT Core and we can help draft it into a formal ‘success story’ to be placed on the NeuroDevNet website as part of a series.

Bringing NCEs together to share KT Best Practices

by Anneliese Poetz, KT Manager, NeuroDevNet

David Phipps, NeuroDevNetKT Core Lead, commenting on one of the presentations

During plenary: David Phipps, NeuroDevNetKT Core Lead, commenting on one of the presentations

During the week of January 26-29, 2015 MEOPAR NCE hosted a symposium in Halifax, Nova Scotia for all NCEs to gather and share what they are doing in terms of “best practices” for KT within their network. There were presentations in the morning, and the afternoons were allocated to 3-hour workshops on various topics.

 

 

Different ways to convey the same message about coastal erosion

From one of the workshops: Different ways to convey the same message about coastal erosion

I learned something important from one of the workshops I attended: that providing the same message in different formats is key for people to understand and remember the message (which is the first step toward being able to apply the message in practice/policy). One format of the message might be a photo that illustrates what might happen in a certain situation, while another way to convey the same message could be an interactive display: either an online tool or a hands-on model that can be physically manipulated to see what happens in different scenarios, yet another option is to hold a community event and encourage broad participation.

There were approximately 100 attendees, which included representatives from NCEs at different stages of maturity. GRAND NCE just finished its first 5 years and provided information about their open source tool they created called the “forum”. It is for project leads to be able to do collaborative reporting with their trainees, upload their presentations and publications, and export citations directly to their common CV. Mike Smit from GRAND said they wished they’d had this at the beginning, however it took them several years to develop – it is open source and an available for any NCE (especially new ones!) to use. TREKK described their quick reference sheets for ER physicians working in a ‘regular’ ER (not specifically for pediatric patients) who need reliable evidence-informed and quick information about how to treat the most common ailments children are brought to the ER for. These evidence-informed tools for practitioners go through a rigorous process before they are finalized. New NCEs such as Glyconet, SERENE-RISC and CellCan commented that this event was a good opportunity to learn from more experienced NCEs about KT practices and management systems.

NeuroDevNet’s KT Core (David Phipps and I) co-presented with the NCE Secretariat (Renee Leduc). Renee presented on progress reporting and KTEE expectations from the perspective of the NCE Secretariat:

and led an exercise with participants that helped them link their Network’s goals with outputs and outcomes:

David and I presented on the Co-Produced Pathway to Impact KTEE evaluation framework, indicators for measuring KT services and impact, and their database system that was created for tracking data on our suite of indicators that were created over the past 16 months:

Anneliese and David provided a hands-on exercise for participants that acted as a “part 2” to Renee’s exercise because following goals, outputs and outcomes is the need to create indicators – so this 2nd handout was a worksheet that helps to fully define indicators:

 

‘your presentation was the most valuable of all the sessions…it was your session alone that made the conference worthwhile attending’CellCan NCE

Booths set up in main area

Booths set up in main area

This event provided a great opportunity to network and get to know other NCEs in the NCE Program. Part of networking included the opportunity to set up a booth at no cost. Across from NeuroDevNet and ResearchImpact booths was the CYCC NCE booth. I tweeted and picked up some copies of checklists they produced for: involvement of children and youth in research, having impact on policy, and others that could be useful to NeuroDevNet’s work as we approach Cycle II. Several attendees found the materials at the NeuroDevNet (and ResearchImpact) booth(s) interesting, particularly the ResearchSnapshots and brochures explaining our services. Many NCEs expressed interest in emulating NeuroDevNet’s KT Core model including the CPPI framework and associated services, as well as our staffing model of a KT Lead, KT Manager and KT Coordinator.

The KT Core live-tweeted from the event from @anneliesepoetz and @neurodevnetKT and several of these were retweeted by @neurodevnet and @ MEOPAR_NCE.

If you are a NeuroDevNet researcher or trainee, or if you represent one of Canada’s NCEs and would like to know more about NeuroDevNet’s KT Core services please visit our website and/or contact the KT Core.

A lot can happen in a year – report on NeuroDevNet’s KT Core

by Anneliese Poetz, KT Manager, NeuroDevNet

It’s hard to believe that on August 6, 2014 it will be one year since I started as Manager of the KT Core for NeuroDevNet. The year has flown by, and I am proud to say we have accomplished a lot in what feels like a very short time. It has truly been a team effort, both within the KT Core and with our Knowledge Translation colleagues in York’s KMb Unit.

The infographic below is a visual representation of the services we have provided to researchers, trainees and partners from August 6, 2013 – present.

Brokering: one of the most memorable relationships we brokered was between NeuroDevNet, the Maternal Infant Child and Youth Research Network (MICYRN) and the new Canadian Clinical Trials Coordinating Centre (CCTCC) prior to submission of NeuroDevNet’s renewal application June 11, 2014. By connecting these organizations and having a conversation about possible ways to work together, several concrete activities were identified and included in the application, relating to the development of NeuroDevNet’s IMPROVE Clinical Trials Network.

Events: Tamara Bodnar and Parker Holman are NeuroDevNet trainees who came up with an innovative curriculum for science teachers so kids can do an experiment and see with their own eyes what the effect of alcohol is on a developing organism. The KT Core helped by providing feedback on their event flyer and the event’s evaluation questions, faxing the event flyer to the list of schools provided by Tammy and Parker, and producing a video about the day.

Products: New things since last year are the production of almost 40 ResearchSnapshots which are clear language summaries of NeuroDevNet-supported scientific research, review and vetting of the most current social media guides based on usefulness to researchers/trainees, our youtube channel where you can find KT videos about NeuroDevNet research, and the KT blog you are reading right now! We only have 6 videos posted on our youtube channel, but we reported 11 because we helped advise on the 5 videos created for the Neuroethics Core’s CENDS video series.

Evaluation: When I started, we had David Phipps’ (Executive Director of Research & Innovation at York University, and NeuroDevNet KT Core Lead) Co-Produced Pathway to Impact Framework and an idea of what services we’d offer,

but since then we have worked together to map the services onto the framework. We have subsequently developed indicators

that will help us evaluate our services so we can make decisions about where it is best to allocate our resources to be the most useful. If our quantitative and qualitative indicators are adopted and/or adapted by other NCEs it could also be possible in the future to compare KT Services across NCEs. Interviews are ongoing, and give us qualitative information about KT successes in the Network about the needs of researchers and trainees that we can use to improve our services.  We are learning about KT successes such as Angelina Paolozza’s presentation to Adopt Ontario. After explaining her eye-tracking research and helping prospective parents understand more about kids with FASD, all of the parents told Angelina they had changed their minds and would now consider adopting a child with FASD. We will be writing and posting some of these “KT Success stories” online.

Planning: The KT Core reviewed 15 grant applications and provided written feedback on the researcher’s KT Planning strategy for their research, which was often followed by a telephone conversation. We were pleased that most of these applications were successful.

Stakeholder engagement: KT depends on relationships. Period. That’s why the KT Core is growing its networks of stakeholders online (see social media on infographic above), and is gearing up for an in-person stakeholder consultation with diverse stakeholders so we can make sure the work we are doing addresses their information needs (thereby increasing the likelihood it will be useful).   We are engaging in conversations using facebook, twitter and LinkedIn and learning a lot about our stakeholders and their information needs. Recently, we distributed the recruitment poster for the FASD Discovery Project’s “Strongest Families” study as well as a resource package for families that don’t qualify to participate. Members of NeuroDevNet can contact the KT Core to ask us to put forth questions to members of our online networks to inform their research, or to disseminate information.

Finally, we’ve refreshed the KT tools section of NeuroDevNet’s website – only the most useful tools and guides for doing KT are there, and are sub-divided into each of the services we provide.   We also provide capacity building/training by request as needed. Now that you have seen examples what we have done, contact the KT Core if you are a NeuroDevNet researcher or trainee and ask how we can help maximize the impact of your research.

NeuroDevNet presents at #CKF14 conference in Saskatoon, SK June 9-10, 2014 – Indicators, evaluation framework, risk perceptions in KT

by Anneliese Poetz, KT Manager, NeuroDevNet

NeuroDevNet’s KT Core recently attended the 3rd annual Canadian Knowledge Mobilization Conference in Saskatoon, Saskatchewan on June 9-10, 2014. There were almost 100 Knowledge Translation/Knowledge Mobilization specialists in attendance, the theme was ‘innovation’.

David Phipps (Lead, KT Core) presented a PechaKucha (20 slides, 20 seconds each) on the Network’s Co-Produced Pathway to Impact (CPPI) framework for evaluating the impact of KTEE, and Anneliese Poetz (Manager, KT Core) presented on indicators that have been mapped onto the CPPI.

NeuroDevNet is in the process of completing a suite of indicator definitions for measuring the impact of their KTEE activities.  Defining indicators for data collection is important for 1) reporting impact to funders, 2) for comparison across KT activities/services 3) in future, comparison across NCEs.  This data will help in terms of decision-making (about how to refine services offered by the KT Core).  It is also useful for gathering ‘institutional memory’ for succession planning.  Indicator definitions need to be more comprehensive than just a title such as “number of…” or “percentage of…”.   The KT Core plans to explore the use of MS Access instead of Excel for ease of data entry.  Anneliese’s presentation can be viewed on slideshare:

An example of a complete definition was handed out to those in attendance at the presentation and can be viewed on slideshare:

 

Anneliese also presented on her previous research about how organizations make decisions using different types of information.  The model that emerged for knowledge translation, introduced the concept of risk perception as an important consideration in any KT strategy.

 

In addition to the CPPI evaluation framework and activities related to measuring impact of the KT Core’s services and the Network’s activities, the KT Core provides consultation services to NeuroDevNet researchers and trainees.

If you would like to know how to evaluate the impact of your KT activities within NeuroDevNet, are planning to host a KT Event and would like assistance developing an evaluation form, or have any questions related to evaluation of your KT Products, contact Anneliese Poetz, KT Core Manager.