Why use Survey Monkey for surveys when NeuroDevNet has REDCap? Create a REDCap survey in 10 easy steps

by Anneliese Poetz, KT Manager, NeuroDevNet

The KT Core provides services for evaluation mainly for KT events and KT products.  Did you know that REDCap can be used for collecting evaluation/survey data?  As a NeuroDevNet researcher or trainee, you have free access to REDCap using your login and password.  Use REDCap for informing your research and/or for KT evaluation purposes.

What kind of surveys would you want to use REDCap for?

• Data collection for research
• Evaluation of your KT event (either in-person or webinar)
• Evaluation of your KT product (summary, infographic, video, etc.)
• Needs assessment for informing capacity building, research grant applications, etc.
• Any survey you would normally use survey monkey, zoho, or any other free survey maker for

Example: I used REDCap recently to conduct a needs assessment for an upcoming workshop on evaluation of knowledge translation.  I used a mix of multiple choice and open-ended questions, and for one of the questions I used the “slider”.  I wanted to know how attendees rated their ability to do KT evaluation prior to the workshop, so by choosing the “slider” type of question I was able to give survey respondents the ability to slide a horizontal bar between “0” on one end and “100” on another end, sort of like a visual analog scale.  The data from these responses is represented as a scatter plot in the data report. It’s a neat feature I have never seen in any survey software before.  I customized a report of the open-ended text-based answers as well as a separate report that gave me either bar charts or pie charts showing the data from the multiple choice responses.  This was very useful data for informing our approach to the workshop.

What are the benefits of using REDCap for surveys?

• Has the same features as the paid-version of survey monkey
• Data is housed in Canada, by NeuroDevNet Neuroinformatics Core, not in the United States like Survey Monkey
• Can export your data into SPSS, SAS, R, MS Excel for analysis, or as a .csv file
• Can customize reports for data export so you can visually see trends (e.g. a bar or pie chart for multiple choice answers, text entries for open ended questions)
• Can enter email addresses of respondents directly into the survey, so you can track survey responses by user

How can you get started using REDCap?

1. Go to: https://neurodevnet.med.ualberta.ca/
2. Enter your NeuroDevNet username and password
3. Watch tutorial videos if needed, to learn how to use REDCap to develop your survey
4. Enter your questions, and design the survey using multiple choice options or open-ended text-based answers
5. Work with Neuroinformatics (Justin Leong, jleong [at] neurodevnet.ca) to finalize and launch your survey
6. Retrieve your data (using export/report options listed above)

Contact the KT Core if you’d like help drafting evaluation questions.  Here is a step-by-step example of how you can create your own survey in REDCap:

Step 1: Log into REDCap

Step 2: In this window, you will see surveys you have already created. You can click on an existing “Project Title” in the list, or to create a new survey click on the tab “Create New Project”.

Step 3: When you click on “Create New Project” tab, you will see the following screen. Type in a title for your survey, choose an appropriate “purpose” for your survey (I usually choose “Quality Improvement”) and I usually leave the default choice to start the project from scratch.  Then scroll to the bottom of the page and click the button “Create Project”.

Step 4: Set up your survey (project).  Click “enable” for “Use surveys in this project?” and then click on it to open a page that will allow you to upload a logo/photo and edit a message to your survey participants (such as ethics agreement if the survey is for research, an introduction/overview of the survey if you are doing a needs assessment or evaluation, information about how the data will be used etc.). When you are done, click the “I’m done!” button.  To design your survey, click the “Online Designer” button and “I’m done!”.

Step 5:  Design your survey.  You will see an entry under “Instrument Name” that is called “My First Instrument”.  Click on the “edit” button to edit the title of this instrument. If you don’t, and you create a new survey (if you “add new instrument”) you will have problems later that you will have to contact Justin to sort out.  After you are done editing the title to something more meaningful (can be the same title as your project title) click “save”.

Step 6: Click on the “Instrument name” title you just edited.  Click “Add Field” to add your first question to your survey.

Step 7: Design your survey questions.  Choose the type of field you want, for example, text box, multiple choice, true/false, slider (visual analog scale), etc.  The slider is great if you want to ask your survey respondents to rate something based on how they felt about it. It can give you more precise information than a scale from 1-10. For example, you can ask “how would you rate your knowledge about XYZ after this workshop?” and give them a scale from 1-100. Text boxes are good for open-ended questions, and multiple choice can be either “choose one only” or “choose multiple”.

Step 8: Create your questions. The example below shows how you would create a multiple choice question. Type the question you want to ask into “Field Label”. Type the choices for your multiple choice question in the box called “Choices” but don’t type a number, just type one choice per line. The numbers will be added automatically by REDCap.  Type in a meaningful name for the variable, so when you view the report of your data you’ll know which one it is.  I usually type “_text” at the end of answers that are open-ended text based answers so that I can create one report for the text-based answers, and another for the report-based answers.  Choose “yes” or “no” whether you want the question to be mandatory for the user to answer – the default is “no”.  Accept the defaults for everything else and click “save”.

This is what the question looks like:

Step 9: When you are done creating questions for your survey, click the “Project Setup” tab.  Click “I’m done!” for the “Design your data collection instruments” item.  Accept the default values and click “I’m done!” for the next 3 items “Enable optional modules and customizations”, “Set up project bookmarks” and “User Rights and Permissions”.  The next item asks you to test the instrument thoroughly before entering “Production” mode. Once you enter “Production” you will be limited in terms of what you can edit/change.

When you are done testing (and clicked “I’m done!”) click the button that says “Move project to production” to get the survey link that you can send to participants by copying/pasting the link into the body of an email.

Step 10: Get your survey link so you can send it in the body of an email to your participants (for anonymous survey data collection).  You may have to contact Justin for help with this.  If you want to try it on your own, look under the “Data Collection” heading on the left hand side of the project page, and click “Manage Survey Participants”.  Then you should be able to copy/paste the link for your survey into an email or into a tweet or other social media post.  Note: you have to have enabled the survey (step 4) in order to get the link for your survey.

This blog post is not exhaustive in terms of REDCap’s survey functionality.  You can also create custom reports so that as people fill out your survey, you can look at the data in a way that makes it easiest for you. For example, I usually create 2 reports: one for the text-based/open-ended answers and one for the multiple choice answers that are usually bar or pie-charts.  It’s up to you.

REDCap is a great tool for surveys.  You may have used it for research-based surveys, but you may not have thought about using it for conducting needs assessments before conducting a workshop, course or presentation or to assess your end-users’ needs before designing your KT products (summaries, infographics, videos, etc.) for evaluating an event (in-person workshop, stakeholder engagement event, or conference, or a webinar) or your KT products (survey your end-users to find out how they have used your KT Product, and what impact it may have had for informing practice or policy).

If you are a NeuroDevNet researcher or trainee and would like help with evaluation of your events and/or KT activities, contact the KT Core. If you need technical assistance setting up a REDCap survey, contact Justin Leong (jleong [at] neurodevnet.ca) from NeuroDevNet’s Neuroinformatics Core.

 

 

What does Program Science have to do with Knowledge Translation?

by Anneliese Poetz, KT Manager, NeuroDevNet

Program science “is the systematic application of scientific knowledge to improve the design, implementation and evaluation of programs”.

The NCE Program in Canada strives to facilitate the achievement of socio-economic impact for Canada through cutting-edge research and innovation.  One social benefit of this Federally-funded research includes implementing innovations into programs that serve Canadians.  NeuroDevNet NCE strives to achieve impact for Canadians affected by neurodevelopmental disorders such as Cerebral Palsy, Fetal Alcohol Spectrum Disorders and Autism Spectrum Disorders.  One avenue is to achieve implementation of evidence-informed innovations into programs.  For example, after refining the exergame prototype, an innovation funded by NeuroDevNet and GRAND NCEs, this technology can expand the physical therapy options for youth with CP.  Integrating the exergame bike into physical therapy programs would be what we refer to as ‘implementation’ (see Figure below).

Phipps’ Co-Produced Pathway to Impact can be used within a Program Science approach toward impact of evidence-based interventions.

NeuroDevNet NCE has adopted Phipps’ Co-Produced Pathway to Impact framework which focuses on stakeholder engagement throughout all stages of the research process, and pushes the boundaries of traditional end-of-grant KT beyond dissemination toward uptake, implementation of new evidence into practice and policy, and evaluation of subsequent impact(s).  Consider that programs are governed by managers and the policies they develop, and the program’s services are delivered by practitioners.  Impact is measured by evaluating both quantitatively and qualitatively, how the program has made a difference for those they serve – in our case, NeuroDevNet is concerned with measuring how research that has been implemented into programs and policies has improved the lives of children and families affected by neurodevelopmental disorders.  So, after the exergame bike has been implemented into a program, we would follow up and evaluate.

Program science has become important for HIV program development because it provides information about what programs work, for which individuals.  The field of program science asks questions that relate to aspects of a program including strategic planning, program implementation (mix of interventions, synergy across interventions) and program management (sustaining effective interventions and modifying programs as new knowledge and interventions emerge, quality improvement processes).

NeuroDevNet supports the Chair in Autism Spectrum Disorders Treatment and Care Research, Jonathan Weiss.  Jonathan specializes in stakeholder consultation with diverse stakeholders including program planners/managers for informing his research, and researching interventions in order to make an evidence-base available for uptake.  This evidence-base can then be used by practitioners and policymakers to inform their decisions with respect to which interventions they choose to provide within programs and organizations (such as schools), and how these interventions are delivered.

NeuroDevNet’s FASD program is embarking on a new project this year to develop and test program materials for frontline workers in Children’s Aid Societies to improve their practice. This project is being done with the full involvement of frontline practitioners and program managers throughout the research process.  When there is an evidence base for these tools and training to show its effectiveness at the study sites, program science can inform the scaling up of the research findings toward improving practice across Canada.

“One aspect of program science is implementation research, which is concerned with the development and implementation of evidence-based interventions…it can also provide information about how interventions can be adapted to new situations or communities…program science typically involves an ongoing process of engagement between researchers, policy makers, program planners, frontline workers and communities through which research is embedded into the design, implementation and continuous improvement of the overall program. Because the focus is on how an entire program impacts a population, program science typically involves consideration of overall health systems” – CATIE

It appears that we (NeuroDevNet, the NCE program, and many Knowledge Translation practitioners) may, in fact, be using a program science approach without knowing it.

How do you think you might already be using a program science approach to your work?

How can the principles of program science that are used for informing HIV programs, be translated into programs for children and families affected by neurodevelpmental disorders?

What can the field of KT learn or adapt from the field of program science? 

Do you think a program science approach can help you (as a researcher, as a KT professional) scale your proven interventions to other health systems, cultures, programs, or geographic regions across Canada and internationally?

What happens when KT Planning and Project Management worlds collide?

by Anneliese Poetz, KT Manager, NeuroDevNet

The answer: you get a hybrid tool for researchers to use for developing a KT plan with activities that are linked with the elements of a project charter.

The dictionary defines “hybrid” as: “a thing made by combining two different elements; a mixture.”

NeuroDevNet’s KT Core recently finalized the creation of a new innovative tool for combining KT Planning with principles from the field of project management. Indeed, it is a ‘mixture’ of elements from both. Someone asked me recently: why did you create the Hybrid KT Planning and Project Management tool (short form: ‘the Hybrid tool’). NeuroDevNet NCE was renewed for another 5 years of funding (Cycle II), and we needed a tool that could be used for any project, that would help us manage KT plans for projects in Cycle II and help keep them on track. We believe that doing this will position us well for applying for Cycle III.

Another person asked me, what was your process for creating the Hybrid tool?

1) Identified a need for a custom tool to facilitate KT Planning for NeuroDevNet’s Cycle II projects after conducting a review of existing KT Planning guides and testing them internally to determine their usefulness

2) Based on the review of KT Planning Guides, we chose 2 key tools to build upon: 1) Melanie Barwick’s (Sick Kids) 13-step Scientist Knowledge Translation Plan template for its comprehensiveness, and 2) Purmina Sundar’s (Ontario Centre of Excellence for Child and Youth Mental Health) form-fillable KMb Toolkit for its ease of use and content, as well as 6 different examples of project charters provided by colleagues

3) Identified areas of overlap (e.g. people involved, identification of activities/deliverables) and linked these with corresponding sections of project charters. I literally pulled these resources apart and put them back together in a different arrangement

4) Composed original text for the purpose of providing additional information (e.g. tips for stakeholder engagement as part of an iKT strategy) and tables that mirror those required for NCE progress reporting

5) Identified places in the document where information would have to be duplicated, and noted where .pdf form-fillable document should auto-populate the information to make it easier for project teams to work with

6) Once the word document was drafted, it was shared with Principal Investigators, NeuroDevNet Headquarters, the KT Core’s KT Steering Committee, and the KT Core’s McMaster and McGill sites for feedback

7) Finalized content in response to feedback and sent to graphic designer who created the working form-fillable version

I recently presented at a KT Conference, and explained some of the requirements of our researchers and how we addressed them before finalizing the Hybrid tool. In the presentation below, you can see the requirements we gathered (from stakeholders listed in step 5 above) and how we addressed them. The presentation ends with a few points as to why we think this tool is important:

http://www.slideshare.net/NeuroDevNet/a-poetz-d-phipps-m-johnny

We wish to thank Melanie Barwick (Sick Kids) and Purnima Sundar (OCE CYMH) for giving us permission to use content from their KT Planning tools.

The Hybrid tool has been peer-reviewed, and tested for functionality. It is not available online yet, but we do plan to make it publicly available.

If you are a NeuroDevNet researcher or trainee and would like to use the Hybrid tool for your KT Planning, contact the KT Core.