By: Maxine Secskas, CLAME Associate, Headlight Consulting Services, LLC and Chelsie Kuhn, CLAME Specialist, Headlight Consulting Services, LLC
This blog post is the first in a series of 3 posts on components of qualitative methods.
Welcome back to Headlight’s educational blog series on all things collaborating, learning, adapting, monitoring, and evaluation (CLAME). You may have noticed we didn’t post much in 2021, and we are excited to share that was because we were GROWING! We went from a tiny firm with just two partners and one full-time staff, to a headquarters staff of 8 and a full field office in Ethiopia with 6 staff just in the past year, and we’re still expanding our efforts in new countries (check out our Staff pages to see all the amazingly talented individuals who have joined our team!). In 2021, we focused internally to build our own systems grounded in our theory of change, our own internal CLAME, and the procedures and practices that help make this growth sustainable. But, it’s time for us to make sure we’re sharing everything we’ve been learning and helping break down complex practices in the CLAME space into more tangible, action-oriented guidance. We’re restarting our blog with a call back to one of our first blogs and favorite topics – tips and tricks for qualitative analysis. The blog series will continue with a new blog every other Thursday. Please join the conversation, post your questions, comments, and your own learnings. We’re so excited to grow together! – Rebecca Herrington, CEO, Headlight Consulting Services LLC
As Headlight’s team continues to grow, we have identified more useful tips, tricks, and advice for junior analysts as they dive deeper into qualitative analysis projects. Learning Reviews, based on rigorous qualitative analysis of a client’s evidence base, are one of the popular services here at Headlight, and all of our staff have contributed to coding, primary and secondary analysis, and/or the writing of Findings, Conclusions, and Recommendation for a variety of clients. The following Dos and Don’ts are helpful for someone starting a new qualitative project or as a reminder for analysis veterans.
Dos:
- Make sure to code according to the definitions in your codebook. All codes should be well-defined with simple language of both the term(s) used and the codes intended application on excerpts so that any coder can check for proper application. This is especially important when working with a team of coders. It is essential that everyone coding on the same project has a similar understanding of when to use the codes. By making sure to strictly stick to the definitions given in your codebook, you can ensure consistency in coding. Along this line, make sure to continually update your codebook – adding definitions for any emergent codes, and review the codebook regularly.
- Your codebook may be large and include supplementary codes that are not directly related to your learning or evaluation questions. For example, you may have supplementary codes to provide context about the specific activity referenced, timeframe, stakeholders involved, or to identify illustrative bright spots. The codes that are directly based on the learning questions should guide your coding. Once you have coded an excerpt to your learning question codes, you can then consider whether it would add nuance to your evidence to apply any supplementary and emergent codes.
- The codebook should also have space to evolve with new, emergent codes based on sub-trends identified during coding. We recommend adding emergent child codes early and often; this is helpful when you conduct secondary analysis later (check out our Intro to Secondary Analysis blog post coming April 7). Adding nuance and specificity to your codebook, especially early in the coding process (first 33% of the documents), will allow you to identify more specific and meaningful findings during analysis.
- When deciding what to code, consider whether it helps answer the learning questions and is actionable. Your codebook should be grounded in your learning questions, and you should not waste time and space coding excerpts that don’t contribute to your objectives. Focus on coding concerted lessons learned, not just any text that is associated with one of your codes. It is best to code statements that contain evaluative conclusions and evidence rather than just normative statements.
- If you’re using Dedoose, we recommend using descriptors to keep track of which documents have been entirely coded and which are still undergoing coding. By assigning descriptors only after you have finished coding a document and sorting the Media by Descriptor, you can easily see which documents still need to be coded, or are currently being coded, because their descriptor bubble will show a “0” and be in red.
Don’ts:
- DO NOT build your initial codebook to be too large and detailed. Your codebook will grow with more detail through emergent codes and also during secondary analysis. If your codebook is too large at the start of coding, it will be challenging for the coders to keep the whole codebook in mind while reading documents. Also, the more detailed emergent codes that will be added through the coding and analysis process will be more targeted and meaningful than most codes you will put in your initial codebook.
- DO NOT use too many overlapping codes on a single excerpt (for example, multiple sub-child codes under the same parent and child codes). This is helpful when doing secondary analysis later, so excerpts are not repeated too many times under different codes.
- DO NOT wait to add emergent codes. Adding emergent child codes can add nuance to your excerpts and save you time during secondary analysis. It is also helpful to add emergent codes as soon as you suspect they will apply to more than one document so that you do not miss the opportunity to apply them to excerpts in earlier documents. Since you should not be retroactively re-coding early documents, it’s best to build out your emergent codes quickly for any possible trends that relate to your learning or evaluation questions.
- DO NOT forget to add definitions when you add new emergent codes to your codebook. You also need to make sure the people involved in coding are kept up-to-date as the codebook evolves to ensure a common understanding of how the codes are being used. We recommend holding a quick daily meeting with the full coding team to share any emergent codes added to the codebook, troubleshoot challenges encountered, and coordinate progress effectively.
- DO NOT BE AFRAID TO HOLD THE EVIDENCE YOU ARE CODING TO A THRESHOLD STANDARD. Clients depend on us to find the evidence behind what works, and if we try to code something as a success that doesn’t meet the threshold of affirmatively working, then we can’t help them. Also, be wary of adjusting your thresholds of evidence to be subjective for the client’s work. Evidence should meet objective international standards, and you will not be helping your client if you hold them to a different standard.
If you have any questions about qualitative coding or need help implementing a Learning Review, Headlight would love to support you! We have the breadth and depth of experience to tailor-meet your needs. For more information about our services, please email info@headlightconsultingservices.org. Headlight is a certified woman-owned small business and therefore eligible for sole source procurements. We can be found on the Dynamic Small Business Search or on SAM.gov via our name or DUNS number (081332548).
Keep an eye out for upcoming blogs expanding on the topic of qualitative analysis, including primary and secondary analysis!
Comments
no comments found.