What It Means To Be Truly Evidence-Based - And How To Sniff Out Poor Research
People are always quick to use phrases such as “evidence-based” or “I did my research” or “I read a study.” And once one of these phrases is used, the listener never really questions what is being said. However, as a listener, it is more important to question what the said evidence is like, rather than questioning if there is any.Andy Bell is a PhD candidate who works in paramedicine and teaches at Griffith University, he spoke with Rebuild Health and Fitness coaches James and Sean about what it means to be truly evidence based. Together, they discuss the understanding of a truly evidence-based approach, its importance, why it should be used, and what should be looked for. Questioning the Expert and BiasesAndy explains people have the tendency to think that if a person is right about one thing, they’re probably right about everything, and that can be a dangerous outlook to have. He says he thinks that the idea of “expert” needs to be avoided. While some people might have detailed information on one topic, no one knows everything. In fact, the more you know about one particular thing in depth, the less you know about everything else. Because there’s only so much a brain can take it. Andy says, “there’s a reason why [everyone in health] are specialties. There’s a reason why you don’t go to your GP to get your dentistry done. Because it would be a nightmare.”And so, it is always important to have a critical approach, even when listening to an expert. Good experts are actually willing to be questioned, they are happy to be proven wrong if the result is actually helping the research of the subject at hand. The best teacher is an eternal student.“If you’ve got someone who’s inflexible of thought, you should probably be weary of them,' Andy says. Additionally, biases tend to rise from people choosing to listen to a particular kind of “expert.” People pick up one particular study, or one idea that fits into their existing mindset. Social media enforces this form of creating biases when someone starts off like that, and then they get a bunch of followers that believe the same one tiny thing which is probably not even correct to begin with.To this idea, James responds with, “that’s just confirmation bias, right? Like, ‘here’s my belief and now I have this other person saying something that I believe in and therefore, that must be true.”Andy goes on to explain that “the hardest part is to recognise that there’s no such thing as objective truth.”Every experiment or research that starts with the aim of finding results or proving something is already “biased” in the beginning because there will always be some parameters against which the results are being judged. And so, it is important to remember that even a fact is not exactly plain old truth. It is just something that hasn’t been disproven yet.James says, “I think it’s really important for people to understand [that] people who actually believe science also have biases. 'And people who believe they don’t have biases, as in they’re purely open to anything, are being untruthful. For most.” Pros of being evidence-basedJames introduces the pros and cons of being evidence-based in the fitness industry. He says that some of the pros are that it augments the results you will get your clients, increases the likelihood of good results, it protects the client, it protects you as a human being, and it separates you from the hobbyists.If one assumes that the appropriate evidence is being worked with, Andy says, “effectively, what this type of evidence can do is it can give you a better chance of producing a better performance, in whatever it may be. Whether or not it’s their training, whether or not it’s nutrition, whether or not it’s other aspects of fitness journey.” Cons of being evidence-basedOn the other hand, some of the cons are how lack of evidence is equated with lack of effectiveness, questionable ‘evidence’ generated by industry, abuse of the term, it’s time consuming, and less profitable. There’s also the consideration of what the source was, and how much the person actually researched. The Hierarchy of quality of evidenceWhen asked about what to look for in evidence, Andy talks about how the quality of evidence can be considered a sort of pyramid, with a hierarchy.'So, the bottom line, the least reliable is in fact your ‘expert.’ Then you start moving up [the pyramid] through different types of studies, like cohort studies. And the bigger the cohort, the more accurate it’s going to become.“Then you’re going to start moving up into things like random control trials. So, now, you’re really starting to like drill into the methodology of the science and things like that. And then at the top you have metanalyses.”Andy then adds, “A metanalysis is a systematic review where they’ve brought together a number of other studies on that one thing, and they’ve analysed all of the data in together and they’ve drawn their conclusion from a bunch of studied data.”Most regular people don’t want to be reading a metanalysis. So, the most common thing people end up doing is either going through cohort studies or listening to the expert. Therefore, we come back to the importance of making sure that the person you are listening to is actually researching as well as possible. Applying an evidence-based approach to fitnessThe two types of analyses usually conducted are quantitative and qualitative. Quantitative analyses work with number and statistical data and those are analysed to draw conclusions. Qualitative analyses, on the other hand, use people’s personal experiences and feelings and emotions to draw conclusions. Quite often, a combination of both is used. It is important to note that subjectiveness is just as important as numbers in context.Taking this into consideration for various research related to fitness, it is significant to consider how everyone has different body types and therefore, the same rules of eating or working out cannot to be applied to everyone without inflexibility. For example, the body of someone who has a lot of muscle will react to the same amount of food much differently than a body that does not have as much muscle.At the end of the day, evidence should inform practise, experience should shape it.When listening to experts, one should critical think about what the experts are talking about. Andy says, “if they are producing content which is all flesh with no real context, you know, the whole thing is focussed on the aesthetic. So, if it’s all about their abs or if it’s all about the way their biceps [look], then that’s probably a no for me straight away.”“Another thing for me was like specifics, absolute specifics,” James adds. When reading up things online, or on social media platform, one should look out for the references. If all references in an article come from websites that end with .org or .com, then chances are that someone is pushing you to check it out because it will eventually profit them.These references are usually paid content so it’s worth being dubious of them, Andy warns.View this post on InstagramA post shared by Rebuild Health and Fitness (@rebuildhealthandfitness) Working with evidence for the general populationOf course, reading metanalyses or going through detailed research papers can be extremely difficult for the general public. There is a lot of information available on the internet, and it’s all an information dump and is very difficult to sort through what’s relevant.It’s definitely worth looking for easier platforms though which information can be consumed. Andy and James talk about how social media can be used as well. It is just extremely important to apply a critical approach towards it and think of all the things that might make them a reliable source.When asked to summarise the main things one should look for, Andy says “you start thinking about ‘is this person someone who is legitimate in that field’? and [that is basically] a combination of qualification and experience. It’s not one or the other, it’s both. Then, you look at the way they deliver that information.”Added to that, it is important to look for where they are getting their information as well.If you want help understanding evidence or research, get in touch team@rebuildhealthandfitness.com
Previous Blogs
