What does Program Science have to do with Knowledge Translation?

by Anneliese Poetz, KT Manager, NeuroDevNet

Program science “is the systematic application of scientific knowledge to improve the design, implementation and evaluation of programs”.

The NCE Program in Canada strives to facilitate the achievement of socio-economic impact for Canada through cutting-edge research and innovation.  One social benefit of this Federally-funded research includes implementing innovations into programs that serve Canadians.  NeuroDevNet NCE strives to achieve impact for Canadians affected by neurodevelopmental disorders such as Cerebral Palsy, Fetal Alcohol Spectrum Disorders and Autism Spectrum Disorders.  One avenue is to achieve implementation of evidence-informed innovations into programs.  For example, after refining the exergame prototype, an innovation funded by NeuroDevNet and GRAND NCEs, this technology can expand the physical therapy options for youth with CP.  Integrating the exergame bike into physical therapy programs would be what we refer to as ‘implementation’ (see Figure below).

Phipps’ Co-Produced Pathway to Impact can be used within a Program Science approach toward impact of evidence-based interventions.

NeuroDevNet NCE has adopted Phipps’ Co-Produced Pathway to Impact framework which focuses on stakeholder engagement throughout all stages of the research process, and pushes the boundaries of traditional end-of-grant KT beyond dissemination toward uptake, implementation of new evidence into practice and policy, and evaluation of subsequent impact(s).  Consider that programs are governed by managers and the policies they develop, and the program’s services are delivered by practitioners.  Impact is measured by evaluating both quantitatively and qualitatively, how the program has made a difference for those they serve – in our case, NeuroDevNet is concerned with measuring how research that has been implemented into programs and policies has improved the lives of children and families affected by neurodevelopmental disorders.  So, after the exergame bike has been implemented into a program, we would follow up and evaluate.

Program science has become important for HIV program development because it provides information about what programs work, for which individuals.  The field of program science asks questions that relate to aspects of a program including strategic planning, program implementation (mix of interventions, synergy across interventions) and program management (sustaining effective interventions and modifying programs as new knowledge and interventions emerge, quality improvement processes).

NeuroDevNet supports the Chair in Autism Spectrum Disorders Treatment and Care Research, Jonathan Weiss.  Jonathan specializes in stakeholder consultation with diverse stakeholders including program planners/managers for informing his research, and researching interventions in order to make an evidence-base available for uptake.  This evidence-base can then be used by practitioners and policymakers to inform their decisions with respect to which interventions they choose to provide within programs and organizations (such as schools), and how these interventions are delivered.

NeuroDevNet’s FASD program is embarking on a new project this year to develop and test program materials for frontline workers in Children’s Aid Societies to improve their practice. This project is being done with the full involvement of frontline practitioners and program managers throughout the research process.  When there is an evidence base for these tools and training to show its effectiveness at the study sites, program science can inform the scaling up of the research findings toward improving practice across Canada.

“One aspect of program science is implementation research, which is concerned with the development and implementation of evidence-based interventions…it can also provide information about how interventions can be adapted to new situations or communities…program science typically involves an ongoing process of engagement between researchers, policy makers, program planners, frontline workers and communities through which research is embedded into the design, implementation and continuous improvement of the overall program. Because the focus is on how an entire program impacts a population, program science typically involves consideration of overall health systems” – CATIE

It appears that we (NeuroDevNet, the NCE program, and many Knowledge Translation practitioners) may, in fact, be using a program science approach without knowing it.

How do you think you might already be using a program science approach to your work?

How can the principles of program science that are used for informing HIV programs, be translated into programs for children and families affected by neurodevelpmental disorders?

What can the field of KT learn or adapt from the field of program science? 

Do you think a program science approach can help you (as a researcher, as a KT professional) scale your proven interventions to other health systems, cultures, programs, or geographic regions across Canada and internationally?

What happens when KT Planning and Project Management worlds collide?

by Anneliese Poetz, KT Manager, NeuroDevNet

The answer: you get a hybrid tool for researchers to use for developing a KT plan with activities that are linked with the elements of a project charter.

The dictionary defines “hybrid” as: “a thing made by combining two different elements; a mixture.”

NeuroDevNet’s KT Core recently finalized the creation of a new innovative tool for combining KT Planning with principles from the field of project management. Indeed, it is a ‘mixture’ of elements from both. Someone asked me recently: why did you create the Hybrid KT Planning and Project Management tool (short form: ‘the Hybrid tool’). NeuroDevNet NCE was renewed for another 5 years of funding (Cycle II), and we needed a tool that could be used for any project, that would help us manage KT plans for projects in Cycle II and help keep them on track. We believe that doing this will position us well for applying for Cycle III.

Another person asked me, what was your process for creating the Hybrid tool?

1) Identified a need for a custom tool to facilitate KT Planning for NeuroDevNet’s Cycle II projects after conducting a review of existing KT Planning guides and testing them internally to determine their usefulness

2) Based on the review of KT Planning Guides, we chose 2 key tools to build upon: 1) Melanie Barwick’s (Sick Kids) 13-step Scientist Knowledge Translation Plan template for its comprehensiveness, and 2) Purmina Sundar’s (Ontario Centre of Excellence for Child and Youth Mental Health) form-fillable KMb Toolkit for its ease of use and content, as well as 6 different examples of project charters provided by colleagues

3) Identified areas of overlap (e.g. people involved, identification of activities/deliverables) and linked these with corresponding sections of project charters. I literally pulled these resources apart and put them back together in a different arrangement

4) Composed original text for the purpose of providing additional information (e.g. tips for stakeholder engagement as part of an iKT strategy) and tables that mirror those required for NCE progress reporting

5) Identified places in the document where information would have to be duplicated, and noted where .pdf form-fillable document should auto-populate the information to make it easier for project teams to work with

6) Once the word document was drafted, it was shared with Principal Investigators, NeuroDevNet Headquarters, the KT Core’s KT Steering Committee, and the KT Core’s McMaster and McGill sites for feedback

7) Finalized content in response to feedback and sent to graphic designer who created the working form-fillable version

I recently presented at a KT Conference, and explained some of the requirements of our researchers and how we addressed them before finalizing the Hybrid tool. In the presentation below, you can see the requirements we gathered (from stakeholders listed in step 5 above) and how we addressed them. The presentation ends with a few points as to why we think this tool is important:

http://www.slideshare.net/NeuroDevNet/a-poetz-d-phipps-m-johnny

We wish to thank Melanie Barwick (Sick Kids) and Purnima Sundar (OCE CYMH) for giving us permission to use content from their KT Planning tools.

The Hybrid tool has been peer-reviewed, and tested for functionality. It is not available online yet, but we do plan to make it publicly available.

If you are a NeuroDevNet researcher or trainee and would like to use the Hybrid tool for your KT Planning, contact the KT Core.

Who is minding the “research to impact” shop?

by David Phipps, KT Lead, NeuroDevNet

In a recent knowledge mobilization journal club David Phipps (Executive Director, Research & Innovation Services at York University and KT Lead, NeuroDevNet) questioned, “Whose job is it to ensure research moves from creation to impact? The simple answer is no one. No one is minding the shop. Individuals are acting individually and not in a coordinated fashion.” No one except NCEs like NeuroDevNet.

The role of a Network of Centres of Excellence is to “meet Canada’s needs to focus a critical mass of research resources on social and economic challenges, commercialize and apply more of its homegrown research breakthroughs, increase private-sector R&D, and train highly qualified people”. At NeuroDevNet we do this by focusing our research, training and knowledge translation efforts on three goals:

1.  Earlier diagnosis of neurodevelopmental disorders
2. Application of validated interventions for children with developmental disorders sooner
3. Better supports for children with developmental disorders and their families

Research helps create new knowledge and new understanding in diagnostics, interventions and supports. It is the job of knowledge translation to help connect those researchers and that research to partners and receptors who can turn that research into new products, policies and services that then have an impact on the lives of children living with neurodevelopmental disorders and their children.

There are many (MANY) frameworks and models for knowledge translation. A very popular framework is the knowledge to action (KTA) cycle adopted by CIHR as their framework for KT. This model has a knowledge creation/synthesis component and an implementation into action component. I recently reviewed a paper that asked if and how researchers are using the KTA Cycle.

The answer: not many and not completely.

Many researchers reference KTA but few actually implement it and none report using it in its entirety. In fact, it was never meant to be used from start to finish by a single investigator.

Really? As I asked in that journal club post, “Whose job is it to ensure research moves from creation to impact? The simple answer is no one. No one is minding the shop. Individuals are acting individually and not in a coordinated fashion.”

If no one is minding the shop no wonder it can take a reported average of 17 years for health research to move into clinical practice.

NCEs like NeuroDevNet are accelerating both discovery and application of research by operating in a coordinated fashion. In addition to coordinating research and training NeuroDevNet also provides professional KT services across the network and embeds KT as a partner in projects that have a high potential to create impacts on policies, products and services. In this manner NeuroDevNet KT supports the application of research and facilitates its transition towards impact.

NeuroDevNet is minding the neurodevelopmental shop. And the KT Core is maximizing the impact of research on the lives of children living with neurodevelopmental disorders.

Knowledge Translation (KT) Best Practices for Networks of Centres of Excellence

By: Anneliese Poetz, KT Manager, NeuroDevNet

It all started in Halifax at the NCE KT Best Practices Symposium, hosted by MEOPAR when the NCE Secretariat co-presented with David Phipps and me on indicators and reporting for Knowledge Translation (otherwise known as KM or Knowledge Mobilization).  Afterwards, we were invited to co-present on the first day of a 2-day meeting that took place in Ottawa on March 30 and 31, 2015 on KT Best Practices for NCEs.

David J. Phipps, Lead, Knowledge Translation (KT) Core, NeuroDevNet. Photo by: Hans Posthuma Photography. Manager, Communications – NCE Secretariat

The day that David and I co-facilitated the meeting in Ottawa on behalf of NeuroDevNet, there were 2 other NCEs (Canadian Water Network, PREVNet) and the Ontario Centre of Excellence for Child and Youth Mental Health who also actively participated presenting their tools for KT.  There was a lot of behind the scenes preparation in the months and days leading up to the event.   Each of these 3 organizations provided tools they had created for themselves: CWN and CYMH shared their KT planning tools, while PREVNet’s contribution was an example of an evidence-informed tool for practitioners.

David moderated part 1, which consisted of presentations, panel discussion and Q&A for each of the 3 tools.  Once the audience had a chance to learn what the tools were all about, I facilitated part 2 which was all about applying them.  The KT planning tools were applied to case studies from their respective organizations, and also to the PREVNet anti-bullying guide which was adapted to be a real-life ‘case study’ – in essence, the group would help develop a KT plan for PREVNet to be able to achieve the greatest awareness, dissemination and eventual uptake, implementation and impact of their KT product.

Photo by: Hans Posthuma Photography. Manager, Communications – NCE Secretariat

Representatives from each of these 3 organizations circulated amongst the participants to answer questions and provide guidance if needed.  Overall, the group of over 60 NCE executives took the task seriously and came up with some great ideas!  When the break out groups reported back to the large group, their feedback was typed onto a large screen ‘live’ so everyone could see, and so there would be a record of their ideas – especially for the benefit of PREVNet so they could apply the KT planning ideas suggested by the group.

Sharing tools for KT is important because it helps advance the field of KT, the sense of community among NCEs, and perhaps most importantly maximize the potential for each NCE to achieve the uptake, implementation and impact of their research findings.  Providing attendees the opportunity to learn about and then apply one of the tools in a small group (social learning) was intended to increase the likelihood that they’d use (or adapt) one or more of the tools to their own NCE’s context.

“David and Anneliese facilitated a great hands-on practical session.  Solid KM practices are increasingly recognized as important elements of a network’s strategic plan.  The participants were left with a variety of very useful tools to choose from and apply to their unique needs.”
– Stéphanie Michaud, Deputy Director of the NCE program

The KT Core provides support for KT Events. If you are planning an event that has a KT component, contact the KT Core to find out how we can help.

What is “Impact” and how do you measure it?

by Anneliese Poetz, KT Manager, NeuroDevNet

For NeuroDevNet, impacts of research and training are achieved when children with neurodevelopmental disorders:
• Are diagnosed sooner
• Receive validated interventions as soon as possible
• And their families are supported through the life span

Related to these, impact is achieved when we make a difference – changes to existing policies or the implementation of new ones, or changes in the way caregivers and/or health practitioners approach their work with children and families. Impact is also helping improve the quality of life for children and families in unexpected ways. In the field of Knowledge Translation (KT), there is still ambiguity about how to measure and report on ‘impacts’ of KT. NeuroDevNet frames its Knowledge & Technology Exchange and Exploitation (KTEE) activities using Phipps’ Co-Produced Pathway to Impact evaluation framework which encompasses the Network’s KT activities as well. If impact is what we are trying to achieve, then KT is one of the means to help us achieve it.

When thinking about KT in terms of evaluation and reporting on KT activities, several quantitative measures easily come to mind: # of peer-reviewed publications, # of citations of one’s research publications, # of conference presentations, # of KT Products created, etc. However, these measures do not go far enough – notice that these are all indicators in the ‘dissemination’ phase of the CPPI. Typically, this is where KT activities ‘stop’ – it is the point of departure for researchers move onto the next research project. But stopping at the “dissemination” (otherwise referred to as end-of-grant KT) stage doesn’t help you measure the impact of your research.

In order to find out whether your research has been considered useful (in practice, or policy, or otherwise) you have to go and ask the people 1) who you engaged in your research process (integrated Knowledge Translation), and 2) who you imagined would find your research useful once it was completed even if they did not directly participate in informing your research questions or process. Yes, qualitative interviews!

The KT Core conducts qualitative interviews with its researchers, trainees and most importantly its collaborators and partners. There is a lot of good work going on in the Network, and these interviews are for the purpose of discovering stories about how NeuroDevNet’s research and training have made a difference. Some of them might not have otherwise been discovered and/or reported on. The first interview is always with the researcher or trainee. Then, we ask them who their collaborators/partners were, and whether they would be willing to broker an invitation for an interview so we can ask questions about the impact of NeuroDevNet’s work from their perspective. How have we changed things for them in their organization? Their practice? For the children and families they serve?

An example of a story we discovered was through one of our trainees, Angelina Paolozza, in the FASD program of research. Angelina was invited to present at Adopt Ontario after someone from that organization saw her present her research at a local hospital. Angelina was able to adapt her presentation style to be compatible with an audience of prospective parents. After her presentation (the 2 times she has been invited) the audience had the same response – many parents said that now that they understood FASD after hearing her describe her research they would revisit the files they had reviewed on children with FASD. Talk about impact – a child in a stable home has a much improved life trajectory and quality of life. The basic underpinning of any effective KT activity is relationships – and this impact was achieved through the relationship built between NeuroDevNet and Adopt Ontario.

Getting these stories is not just useful for reporting purposes, but it is also valuable for us as a Network to learn what works and what needs more attention/improvement in terms of our collective KT activities. By learning how we can best achieve impact, we can maximize the chances that we can repeat and scale our efforts.

If you are a NeuroDevNet researcher, trainee or collaborator/partner and you have a success story you would like to share, please contact the KT Core and we can help draft it into a formal ‘success story’ to be placed on the NeuroDevNet website as part of a series.

Bringing NCEs together to share KT Best Practices

by Anneliese Poetz, KT Manager, NeuroDevNet

During plenary: David Phipps, NeuroDevNetKT Core Lead, commenting on one of the presentations

During the week of January 26-29, 2015 MEOPAR NCE hosted a symposium in Halifax, Nova Scotia for all NCEs to gather and share what they are doing in terms of “best practices” for KT within their network. There were presentations in the morning, and the afternoons were allocated to 3-hour workshops on various topics.

 

 

From one of the workshops: Different ways to convey the same message about coastal erosion

I learned something important from one of the workshops I attended: that providing the same message in different formats is key for people to understand and remember the message (which is the first step toward being able to apply the message in practice/policy). One format of the message might be a photo that illustrates what might happen in a certain situation, while another way to convey the same message could be an interactive display: either an online tool or a hands-on model that can be physically manipulated to see what happens in different scenarios, yet another option is to hold a community event and encourage broad participation.

There were approximately 100 attendees, which included representatives from NCEs at different stages of maturity. GRAND NCE just finished its first 5 years and provided information about their open source tool they created called the “forum”. It is for project leads to be able to do collaborative reporting with their trainees, upload their presentations and publications, and export citations directly to their common CV. Mike Smit from GRAND said they wished they’d had this at the beginning, however it took them several years to develop – it is open source and an available for any NCE (especially new ones!) to use. TREKK described their quick reference sheets for ER physicians working in a ‘regular’ ER (not specifically for pediatric patients) who need reliable evidence-informed and quick information about how to treat the most common ailments children are brought to the ER for. These evidence-informed tools for practitioners go through a rigorous process before they are finalized. New NCEs such as Glyconet, SERENE-RISC and CellCan commented that this event was a good opportunity to learn from more experienced NCEs about KT practices and management systems.

NeuroDevNet’s KT Core (David Phipps and I) co-presented with the NCE Secretariat (Renee Leduc). Renee presented on progress reporting and KTEE expectations from the perspective of the NCE Secretariat:

Leduc Phipps Poetz: Development of indicators for measurement at each stage of Knowledge Translation from Research to Impact

and led an exercise with participants that helped them link their Network’s goals with outputs and outcomes:

Leduc handout: Linking Goals, Outputs, Outcomes with your Indicators

David and I presented on the Co-Produced Pathway to Impact KTEE evaluation framework, indicators for measuring KT services and impact, and their database system that was created for tracking data on our suite of indicators that were created over the past 16 months:

Poetz, Phipps, Leduc: Development of Indicators for Measurement at Each Stage of Knowledge Translation from Research to Impact

Anneliese and David provided a hands-on exercise for participants that acted as a “part 2” to Renee’s exercise because following goals, outputs and outcomes is the need to create indicators – so this 2nd handout was a worksheet that helps to fully define indicators:

Poetz Phipps Leduc: Handout_Anatomy of an Indicator Worksheet, MEOPAR KT Best Practices Symposium Halifax, Nova Scotia January 26-29, 2015

 

‘your presentation was the most valuable of all the sessions…it was your session alone that made the conference worthwhile attending’ – CellCan NCE

Booths set up in main area

This event provided a great opportunity to network and get to know other NCEs in the NCE Program. Part of networking included the opportunity to set up a booth at no cost. Across from NeuroDevNet and ResearchImpact booths was the CYCC NCE booth. I tweeted and picked up some copies of checklists they produced for: involvement of children and youth in research, having impact on policy, and others that could be useful to NeuroDevNet’s work as we approach Cycle II. Several attendees found the materials at the NeuroDevNet (and ResearchImpact) booth(s) interesting, particularly the ResearchSnapshots and brochures explaining our services. Many NCEs expressed interest in emulating NeuroDevNet’s KT Core model including the CPPI framework and associated services, as well as our staffing model of a KT Lead, KT Manager and KT Coordinator.

The KT Core live-tweeted from the event from @anneliesepoetz and @neurodevnetKT and several of these were retweeted by @neurodevnet and @ MEOPAR_NCE.

If you are a NeuroDevNet researcher or trainee, or if you represent one of Canada’s NCEs and would like to know more about NeuroDevNet’s KT Core services please visit our website and/or contact the KT Core.