A blonde woman, Kasey Jo Orvidas, PH.D., flexing her bicep and showing her back muscles. Text beside image says identifying strong studies

How to Identify Strong Research Studies

evidence based meta analysis randomized controlled trial research research pyramid Feb 11, 2020

My literal life goal is, and always has been, to bring research to the real world.⁣

I realize that most people have zero formal training on how to sift through the available research, so today, I want to share a little about how to do so.⁣

As a coach, it can be challenging to sift through all the available research, especially if you don’t know what kind of information you should look for.

Here’s a breakdown of different types of research you can find. I hope this will help you identify strong research studies and better understand what you’re reading (and the quality of it!). 

 

The Research Pyramid

This pyramid presents the various levels of evidence you can find in published research studies.⁣

 

 

As you move from top to bottom, the AMOUNT of evidence increases, but the QUALITY decreases.⁣ So, whenever possible, focus your efforts on the types of studies near the top of the pyramid. 

That’s not to say that background information and expert opinion aren’t ever valid or useful, but the types of research closer to the top of the pyramid hold more merit and might provide you with better and more accurate information. 

Let’s start from the top!

 

Meta Analysis

 

A meta analysis combines the numerical results from all relevant studies in a new statistical framework to test hypotheses.⁣

A meta analysis combines relevant qualitative and quantitative study data from various studies. The statistical framework has even greater power than any single study because there are more subjects, greater subject diversity, or accumulated effects and results. 

Meta analyses help establish statistical significance when you’re faced with conflicting results or want to get a better “consensus” on a topic. 

 

Systematic Review

A systematic review is a formal, systematic and structured approach to review all the relevant literature on a topic. In other words, a systematic review gets rid of all of the poorly done studies and focuses on gathering data from a group of high-quality studies. 

A systematic review might look at factors such as how many studies have used certain methods and where they were carried out.

It’s important to note that a systematic review is written by a panel of experts who do a comprehensive review to determine which studies should be included. They look at both published and unpublished studies then summarize their findings.

This type of research is helpful because the data can be generalized more broadly than any individual study. Systematic reviews are also considered more reliable and accurate because of the extensive research and review process.

Often you’ll find that a systematic review is combined with a meta analysis.

 

Randomized Controlled Trials (RCTs)

 

Randomized Controlled Trials randomly assign participants into an experimental group to receive what is being tested or to a control group(s) where they receive no treatment or perhaps a placebo. 

In this type of study, researchers compare the experimental group to the control group(s).⁣

An advantage to a well-conducted RCT is that it will remove population bias. Also, the results can be analyzed with well-known tools, meaning there is a clear interpretation of the data. 

However, remember that RCTs do suffer from volunteer biases, so some populations may not be represented in the data. This makes it harder to generalize the results. 

 

Cohort Studies

Cohort studies are where one or more groups of people with a similar characteristic (e.g., birth year) are followed over time to determine what risk factors are associated with a particular outcome (e.g., obesity).⁣

These studies can achieve standardization of criteria and outcomes, but cohorts can be difficult to identify because of confounding variables. 

Confounding variables are those that affect other variables which may distort the data. 

Cohort studies also suffer from a lack of randomization, so there could be imbalances in participant characteristics.

 

Case Control Studies

A case-control study compares one group of people with a certain outcome (e.g., with obesity) to a group without it and looks back in time to compare risk factors (e.g., exercise frequency).⁣

The goal of a case control study is not to change the course of a particular outcome but to determine what caused that outcome. It is designed to estimate the odds of something occurring. 

A case control study can be especially useful for studying rare conditions or diseases while looking at various possible risk factors. It can also answer questions that may be unanswerable through other types of studies.

However, case control studies are lower on the pyramid because they rely on people’s memories. Relying on participants’ memories results in recall bias which means that participants may not accurately recall past experiences and events. 

 

Case Reports

A case report describes an individual case. This type of study is often used with unique or rare conditions that are difficult to study.

They are often at or very near the bottom of the research pyramid, but case reports are where new issues come from and can be the basis of future research if there are enough case reports that show similar data or issues. 

 

Background Information and Expert Opinion

Background info and expert opinion are the anecdotal evidence or information put out there by your favorite fitness influencers (if you can confidently call them experts).⁣

There is often little to no scientific evidence behind these opinions, as they are based on individual experiences. 

 

Apply Your Knowledge

 

Now that you know how to identify strong research studies, it’s time for you to learn how to apply that knowledge in your coaching practice. 

The Health Mindset Coaching Certification was designed to teach coaches like you how to apply mindset psychology and research to their coaching practice to help their clients with increased adherence and success. 

Not only that, but you will also learn how to cultivate a growth mindset in your clients and guide them to sustained behavior change using evidence-based practices in psychology.

If you’re ready to improve your coaching practice, CLICK HERE to join the waitlist for the Health Mindset Coaching Certification and receive 5 FREE lessons in mindset and psychology to get started right away. 



Connect with us!

Email: [email protected]

IG: @coachkaseyjo @kjocoaching

Hi, I'm Kasey!

I coach, mentor, write, and teach with one main focus: Build strong bodies and healthy lifestyles, starting with your mindset.

 

Connect with me on socials: