🤖 Q&A Research and Discovery Part 3: Qualitative and Quantitative
This is the third part of our Q&A Discovery and Research. This piece will be about qualitative and quantitative data, and it will have a bonus part. You can read part 1 or part 2.
This part will cover:
Any ideas on how we balance qualitative research with quantitative data?
How important do you think quantitative data is compared to qualitative data like interviews, comments and such (bonus part)
Can I trust the result?
The next part will be out next week, it is also the last piece.
⚖️ Any ideas on how we balance Qualitative research with Quantitative data?
⏱️ Short answer: this is a bit of trial and error, my mindset it to always start with the task that needs to be solved and choose method and data form based upon that. The common rule is when you need to understand what is happening, you go for quantitative methods because you need volume in sample and data. If you have to understand why something is happening, you go to qualitative methods.
The question was maybe not around when to use which method and more around how you can work with it, and that requires a longer answer.
🦥 Long answer: This will be quite a personal answer from my history. In my experience, working with quantitative and qualitative methods alongside each other comes in many shapes and forms. Depending on the researcher's own knowledge, experiences, but also expectations from the company you work at. Some companies respond negatively to qualitative data, and you might need to focus a bit more on the quantitative side. Until you have convinced them of the importance of qualitative data because it sure is important. It should lay the entire foundation for “doing the right thing” for your customers, not only “doing it right”.
Finding the right balance is dependent on what you can master as a researcher, as well ass how mature your organisation is
Finding the right balance is dependent on what you can master as a researcher, as well as how mature your organisation is. So, with that said, I can tell you what has worked for me as a researcher. Therefore, to make this a bit less difficult to follow, I will divide this into activities in relation to the different phases in a quite traditional research process. Bear in mind I come from a quite qualitative background, my expertise lies in qualitative studies. But I try to take advantage of the little computer science skills I have to master some quantitative data modelling and analysis as well. So I will not cover in depth data science and data analysis domains because I'm not qualified for that. But I’m a strong advocate for close collaboration with data science departments or colleagues who work closer to those units. Which, in my opinion, might not happen as often as it should. See this as a qualitative researcher exploring quantitative methods, not a quantitative researcher.
Desktop study
So desktop study is all about gathering and making sense of secondary research. Research that relates to the subject you're exploring, it can be internal or external sources of material. And should preferably consist of both qualitative and quantitative studies. But you take what you get. In my experience, there are frequently plenty of great quantitative studies to be sourced but fewer qualitative studies. Your aim here is to detect what has yet not been discovered and what you need to focus upon to complement the secondary research you have analysed.
Planning and executing your research
In this phase, you need to think about what serves your study the best, qualitative or quantitative methods and data. I choose quantitative methods when I need a bigger sample or when it has to go fast and be less expensive. But I constantly try to be aware of what I'm compromising for not doing a qualitative study, and if it’s worth it. And I constantly try to incorporate qualitative questions in surveys. And if the decision lands on a qualitative study, I also try to incorporate quantitative parts as well. So attempt to decide what is the best method based on the question that needs to be answered, resources you got and time and budget. But don't be afraid to mix.
In this phase, I also try to take the analysis method into consideration. Because the way you decide to collect and document your study has a massive impact on how fast you can take your data from a messy dataset into actually being able to make sense out of it. I like adding a more quantitative methodology to my analysis part, which I know not many qualitative researchers do. I always document in Excel or Google sheet, so it is already prepped as a “database”, and can quickly be adapted to fit the software I use for analysis. But it is not the best tool for notes, but personally, I think it’s worth it based upon my way of analysing it. But you choose what works best for you, and serves your purpose the best.
Analysis
Ok, now we’re getting to the interesting part, here you can go rogue with mixing and switching between qualitative and quantitative methods.
Sidenote, I think the big downside of qualitative research is how fixed the finding and insights become after you have done a content analysis or thematic analysis. You have identified themes and codes and have started to make sense of that. However, if you have several stakeholders interested in different target groups, it is quite cumbersome to go back to your data and split the data upon different target groups, demographics, or behaviours. Because your analysis is quite fixed on the themes and codes, you decided upon quite early.
I try to work with the data so it can be worked with more flexibly over a longer period of time, which means I need to use more quantitative methods to do so. I still go through all the data in a more qualitative manner. I still identify themes and codes, but I put it down in an Excel or Google sheet so it exists like parameters in a database. Which means I can use software to easily switch between themes and codes I have detected. Themes and codes which might relate to behaviours and values, those things qualitative data is so good at. But at the same time also open up for demographic segments such as age, income and things some stakeholder might be interested in.
I wrote an example, but it became too long, so if you want to see in depth on how I do it, read more here.
Visualising and communicating
It starts to get a bit repetitive, but it depends. It all depends on who your audience is. I have found it to work well to mix, but communicating qualitative data like quantitative data doesn’t work that well. You might get a feeling it works well, but it really doesn’t say that much, and it is difficult to act upon. Saying “3 respondents out of 8” did this or that often results in questions like “only 8 respondents” rather than focusing on the right thing. So if you’re doing a qualitative study, focus on communicating it qualitatively. Like you’re actually presenting behaviours and values from human beings and not Respondent 1 out of a Sample. But that doesn’t mean you can’t add quantitative parts to it. But try to find ways in which the two compliment each other rather than say the same thing. You don’t have to show a chart on a certain behaviour and then write the same thing in text. Instead, present the insight or the conclusion you have, and then let the quantitative piece support that.
Present the insight or the conclusion you have, and than let the quantitative piece support that.
This might still not have been the answer on how to balance it. However, more about how you can think about mixing methods during a study. I think the balance is so much determined by the situation and researcher. So let that guide you into finding a balance that works for you, your team and your organisation.
🤓 How important do you think quantitative data is compared to qualitative data
⏱️ Short answer: there is only a short answer to this. Neither of them is more important than the other. It is all about choosing the right method for the right problem and purpose.
But how do I know what is right?
Ask someone with more experience than you, start with what you have and what you know, try, fail and learn. Starting an interview study, when you only need to know how old your customer base is, might not be the right way to go about it. As well as doing a quantitative study asking about a person's experience of living a long and healthy life. It is a bit like selecting a bike when you need to cross a river. Analyse the situation and pick from that, and if you don’t have the resources to pick the best method, just adapt and overcome. Maybe you can attach the bike to a piece of wood and still get over that river. It might be a bit more hassle, but you will still cross it.
🎯 Can I trust the result?
⏱️ Short answer: If you don’t work with statistics, and you work with product you will never be able to trust the result 100%, but the good thing is that you don’t need to trust it 100%. That is basically what you need to know.
You need enough information to make he next decision…
You need enough information to make the next decision, and if you work in an agile manner, you will have time to tweak it. You might not want to travel in the opposite direction, but if you do three interviews, I promise that will lead you at least in the right direction. Possibly, not on the best path, but next time you test or gather data the path will become clearer every time. So don’t let perfection stand in the way of progress. You will have time to find the path, my friend.
