Speaker 1hey maggie so something that you hear quite a lot or see a lot on social media are trainers talking about things being evidence-based and i think there's a an element then of kind of what counts as evidence-based where do you get your evidence from and how do you make sure that the evidence you're relying on is actually accurate and i suppose some of these questions are coming from me who as you well know is not somebody who kind of throws themselves into the scientific research i pretty much go for relying on you i was going to say people who i trust to give me the information but but let's face it i normally just ask you because you do love this stuff so i thought it would be nice to have a little bit of a chat about about this concept of what is evidence based and yet how can we know what we're looking at is right what the things should we look for if we're relying on a study or an abstract that we've seen or even a quote somebody else has shared somewhere and and how do we kind of incorporate that into our practice and what we do and how we talk to clients and all of that
Speaker 2kind of stuff yes it's this is the um evidence-based is is pointing to the fact that somebody is taking what their claim is technically from a reliable source and that reliable source when we talk about being evidence-based should be related to study that is peer-reviewed and potentially maybe a broader group of studies um so that we're not just basing it on one but this is very much that not all research is equal and not all claims about research is equal and knowing the difference is really only way you can tell whether the claim that you're seeing is reliable um and is valid or not and those are the terms we would use scientifically um validity and reliability they are the two really key things there so i guess a really good example that i think every single person listening to this has heard of no matter where they they might even be in the world and that is evidence-based and so i think it's really important to look at that because i think it's really important to look at that and i think it's really important to look at that and i think it's really important to look at that and i think that is a huge um what should we call it i mean i'm not gonna say huge study because it wasn't that was a problem but a huge uh controversy i'm going to use that word around uh the mmr vaccine and autism you'll have heard of that right yeah yeah everyone's heard of that because what happened was in 1998 the lancet which is a really um established prestigious university and it was a stage journal um scientific journal published andrew wakefield's study um him and his colleague published a paper it was based on just 12 children and it claimed that there was a link between the mmr measles mumps rubella vaccine and autism and bowel disease it got huge media coverage like instantly the media everywhere picked it up and it contributed as a result in it to a really sharp drop in the mmr uptake very much in the uk but also far beyond that as well and we're still seeing this as a result of the mmr uptake and we're still seeing this as a result of the mmr uptake and we're still seeing the ramifications of that today now the reality of that story and i don't think a lot of people even know the end of this story so reality of this story is that an investigative journalist uncovered that um first of all andrew wakefield had undisclosed financial conflicts of interest so he'd actually been paid by lawyers building a case against the vaccine manufacturers um and had also filed a patent for a rival single measles vaccine so he had financial interest in proving that there was a problem with the mmr vaccine so uh conflicts of interest that is always something that we look for when we're looking at whether or not a study may be valid who's funded it why are they doing it what financial interest might they have in this the data was then found to be manipulated so children's case studies were misrepresented some had pre-existing developmental issues that were absolutely downplayed so what was presented as the case studies were misrepresented and some had pre-existing developmental issues that were misrepresented and some had pre-existing developmental issues that were misrepresented original data of the children in the first place was also manipulated and false so not only have we got an incredibly small sample size 12 children i mean it's nowhere near enough to actually create a valid sample size to give us reliable data out of it but also he manipulated that data now since then and we're talking what in the last 30 years there have been you can imagine how many studies have been done on this because of this original this original issue and and vaccine um uptake is very important particularly amongst children so we have been you know plugging money into this and numerous large-scale studies much bigger sample sizes much better methodology we'll talk about methodology in a bit but they have found zero link not a single study has found a link between the mmr and autism the lancet retracted the paper in 2010 the lancet retracted the paper in 2010 and the lancet retracted the paper in 2010 but the thing i think a lot of people don't know is the uk general medical council not only found wakefield guilty of serious professional misconduct dishonesty and abuse of developmentally disabled children they struck him off the medical register he is no longer a practicing doctor however he is still around despite his discreditation he and losing his medical license fully he has continued vaccine views internationally. And you are seeing the ramification of that right now in the USA, where they have taken their one in five vaccine and proudly announced they are separating it into five separate vaccines. A lot of that is still the ramifications of Wakefield's 1998 completely debunked, falsified study that he was struck off a medical register for. And yet people are still very uncertain, quite rightly, because what's a claim? What's a concern if it was true, right? So this is the problem with not really knowing the whole picture behind something, but also what the media can play in this. And I think that's even greater these days with social media.
Speaker 1Yeah, I was gonna say that's exactly what I was gonna say. It's kind of like, obviously, we know the media love a soundbite. They like controversy because it brings up more people discussing the point. And then what we get left with is this vague idea that we know something because we've heard it somewhere. And we're not quite sure where it came from or what the basis was. We've just heard this idea, in this case, that, you know, a vaccine might have a negative impact. And that's enough, then, to be able to say, well, we don't know. We don't know. We don't know. We don't know. Give us some sort of vague disquiet about a topic. And from that, particularly when it's something like, are you going to, you know, risk your child or whatever, you're left making a decision based on this vague disquiet, it's going to have an impact. And it's stuff, I mean, we see it all the time when we bring it into the fitness world. There's, you know, people that claim for it. They've got evidence-based stuff that says all sorts of things relating to whether you need to exercise, how much you need to exercise, whether it's better to do it at a certain time of day, whether it's you need to do five minutes a day, or whether you need to do an hour three times a week, or whether you need to do 20 minutes or whatever. And then, obviously, we throw in nutrition, and we've got the same stuff again. We've got those, you know, the arguments that when you eat is more important than what you eat, or that if you eat multiple times a day in small amounts, or you do intermittent fasting, or whichever. And then there can be, you know, completely opposites of each other, but there'll be people claiming that there's evidence to base it to say that their approach is the best approach, and it means more than one of the other approaches. And I mean, we see most of them start with something like this thing is more important than the number of calories you eat. And go from there, not to mention bringing in supplements and other stuff around general health. There's a lot of stuff out there. And it is normally based off of, you know, a nice headline that the media's pulled out, and then other people have used to either sort of support views they already have, or to sell something that they're trying to sell to get some money out of something you kind of like cherry pick the sources they're going to use. And I guess that's part of the like something on my notepad, and then share it, show it to somebody and say, well, there's evidence. There's not actually anything that says that's still a technically accurate statement based on the use of the English words.
Speaker 2And the issue I have is that for most of us, what we're not going to do is start going into Google Scholar, find the original paper and doing the reading of it. And even if we did, would we understand fully what we're reading? Because I had to do an entire year's master's course to understand how to read scientific papers fully and properly to understand whether a reliable and valid source so essentially no is the answer so we should always be curious and I think the best question you can always ask is how reliable and valid is this source and am I going to am I going to instantly agree with it because it meets my world view so for example weight loss is a great example of where this can get quite controversial and actually there are studies that are fully misrepresented a lot of the time because they pull out the key headline from the abstract now when you write an abstract you're selling your study right you are saying what the key parts of this study were what they said and so essentially your abstract is like your introduction at the beginning of your paper and it says these are the key findings that we found out now in there you are not going to put detail you are going to say we had three groups one control and then one that did the diet so the control group did nothing they just carried on their normal lives where this other group that did our diet and then we had this other group that purposefully did something we wanted them to do in another way right and we compared the two so like let's say one did a carb diet one did a fat diet right that was a big thing for a long time right that the Atkins style or that sort of thing over you know carbohydrates and things like that so what we can then see generally speaking usually the presentation in the abstract media will pick up that will go viral is there was greater weight loss in the fat loss group in the in the in fat group there was greater weight loss and then when you read into the detail of it and you look at the methods and all the rest of it what you actually find out is that the eating fat group they lost one pound and the eating carbohydrates group they lost 0.5 of a pound that's your difference like literally that's your difference and so yes okay there was greater weight loss there was greater fat loss but essentially the detail tells you that it was negligible like this is over a three-year period or something like who cares but that is what they're picking up because the person that cares is the person that's now going to sell you all of those diets and all of the supplements that go alongside having that diet and so on and so forth there is a financial vested interest in this the wellness industry is worth over six trillion pounds people talk about big pharma all the time big wellness is bigger right there is so much money in this that there is a reason that we would want to tweak misrepresent maybe is a word for it however you want to say about it there is a reason we would want to get this in present this in the way that that we do without actually going into the full details about it and the nuance and so on and equally as well you know people are studying this stuff all the time so you might have one study that comes out um you know in 2025 and the beginning of 2026 another study comes out that's actually debunks completely what that one said um and no one's going to talk about that no one's talking about the fact that the evidence base shows you that these studies say this happens an equal number of these studies say it doesn't happen well what does that mean for you what does that mean for you that evidence base overall tell us well evidence base tells us that we don't know that we have actually not got any reliable or valid valid evidence at the moment that one or other is the case and a lot of the time that's because it's very difficult to study the things we want to study aspartame was another really big one that it's it's bad for us right and a lot of people advise that we stop to eat and um aspartame and another type of um fake sugars um and then actually the studies that was based off was injecting the product directly into rat's brains at their body weight well one i'm not directing into my brain two i'd love to try i mean i wouldn't it would be bloody horrific um taking my own body weight in aspartame like no thank you that sounds awful like who can drink that many diet cokes i mean it's just impossible right so what we don't always know is is the details of the study and the details of the study are the most important parts of it because there are different types of research design there is an evidence hierarchy even amongst studies and there are always questions that we need to ask when we're being curious or interrogating any source that we're looking at whether that is on the internet and it is something that somebody is telling us on tv or on the internet and it's something that somebody is telling us on tv or on the internet or whether it is actually a scientifically published paper as you mentioned one of the
Speaker 1things i found like when you do look at the detail of a study you tend to come away with more more questions than answers or i do certainly there's always because they have to study things you know it's normally one or a couple of related questions but then you have the fact that in the real world these things don't happen in a vacuum so then you've got all these extra bits that are kind of well what if or what about this thing or does it is there any difference if it's um a different group of people in some way because that's the other thing i think we do working with humans is we we're all individuals with all sorts of differences between us on a lifestyle level on a physiological level and so there's going to be those differences and then as i say that always me with with more questions about stuff but what if and that kind of thing which i guess brings us potentially to um as you mentioned that sort of almost hierarchy of of um evidence that's out there and and what the research is and maybe you could go into that in a little bit more detail
Speaker 2yeah yeah yeah absolutely um so i would i mentioned a word a couple of times and i'll explain that before i sort of explain the studies because i think that is the the ultimate at the top of everything is um when something is peer-reviewed so when things are placed into journals they are peer-reviewed as in they are sent to other scientists within that field and they review the study and then they send questions back and very often the people that conducted that study have to answer those questions and then that is then informed into the study itself before it's published equally the the people themselves have conducted that study absolutely if it is a a high quality study we'll be saying what you just did i have more questions this did not answer everything because it's impossible to answer everything so at the end of this like this has contributed to our knowledge on this subject it has filled this gap this is the gap we were looking to fill but it has also created more questions in x y and z and that will always happen now if a study has not been peer-reviewed which happens a lot of times a lot because people set up their own labs they create their own companies and they they create their own evidence base and they don't put that out to public journals and so they do not get peer-reviewed i would always be questioning why why is this not peer-reviewed why has this not been put into a public journal what are the vested interests what were they looking for and have they found what they were looking for and is there any is there a clear methodology in what they did are there any holes in it because a lot of the time when you have that independent private research which doesn't get peer-reviewed in the public field those things are missing and they are because people are essentially looking to review their own product so they can put it to market and they want to create the evidence base that this product product is good now that happens a lot in the wellness space that can't happen in the pharmaceutical space because there are greater obviously um requirements in a medical capacity for a product than there are in what is essentially considered more like a food product um in the wellness space so something can be promoted as being a detox for example that's a really good example of things that might come into a wellness space less so a medical one and you might see lots of evidence produced for what this detox does but when you dig into it it has been produced in their own labs by that company you can't see the methodology and like what did they do what was the recipe for this study you don't get given any of those details and you don't you don't know sample sizes and a lot of the time it's followed up with anecdote which is the lowest lowest lowest of our evidence base because it is simply just a um it's useful to create a hypothesis out of but we prove nothing with an anecdote you know you'll hear the phrase n equals one like that's all an anecdote is it tells us nothing we can create hypothesis we can test a greater capacity from that but it creates nothing so peer-reviewed is always going to be the top of the chain before we even look at the different types of studies and that might be within it so different research designs are the first one so you have a quality qualitative or a quantitative design but you can also put the two together and you can do mixed methods so you're Quantitative is your numbers. It is basically saying we can pull actual data that we can run through software, usually is used for it, that will then tell us on a large scale, averages, means, so on and so forth. That's versus a qualitative, which is that kind of going into a depth of a lived experience of the people. And that's usually done a bit more by interview. So your quantitative is often, that's when you see your surveys and then you'll get those surveys back and you run the data through and you get your answers to your question. That way, your qualitative, you're going to have a smaller number quite often because you'll have to conduct in-depth interviews with them and you'll talk to people and you'll get a greater understanding of that. So they're both valid. Mixed methods combines the two. So what you might do is say, well, we'll do a, we'll do a survey to get a large scale overview, but then we'll pull some of those people in that have answered our survey and we'll question them in a much deeper interview format. So we can find out much more about the motivations behind it and so on and so forth. So a good example would be, maybe we want to understand how much of the population is now getting 10,000 steps a day. And so you'd send out quantitative survey in order to get the big data, but then you might conduct interviews after that to find out why the people who weren't hitting 10,000 steps a day weren't hitting them because then that would inform what you do next, what more you need to study or what interventions you could put in place because you're always doing it for a reason, you want to find this information out so you can close the gap. So that's your quantitative and qualitative or mixed methods research designs. Then you've got, you could do this cross-sectional or longitudinal. So cross-sectional is a snapshot in time. I'm going to say between, and this is usually like when I did my master's, it was cross-sectional because you can't study longitudinal when you are given 12 weeks to do a study. So that's a really good example. You might say between the 1st of August and the 30th of September, I'm going to research this and you would send out your study or you'd interview those people within that moment of time. So that's a cross-sectional study, it's a snapshot in time. Now longitudinal is much, it's higher on the hierarchy because it is following people over months or even years. Like we really want some amazing data about something. We will follow them for years. Now, obviously that costs money. It takes a lot more to research. And so it's harder to do, but it will show us change. It will show us some potential causation. We can only show correlation because it was a snapshot in time. Because it was a snapshot of a group of people. So there's that as well. Now there are also cohort studies and randomized controls trials. Randomized control trials are your highest possible type of research design because you are controlling the situation. So we can do observational studies where we don't touch anything. We just let people live their lives out in the world and we watch them. We don't touch anything. doing it um so mine was observational when i did my master's because i essentially did not do put people into groups i just said who already does this thing and who already does this thing and then i compared them so they were just doing their real life thing and telling me about it um we can put them into cohorts and we can say uh right i'm going to case study this um but i'm not again i'm not going to intervene these people are already smokers so they're in that cohort these people are non-smokers so they're in that cohort so i'm not going to fiddle with that at all and i'm just going to then watch them and compare them and then we can do randomized control trials where we basically put people into the experimental design that we have created so we say you are going to go into this room and i'm going to present you with um big plates and a small amount of food in it and see how you do it and then we can do it in a different way so i'm not going to um big plates and a small amount of food in it and see how you do it and then we can do it in a i'm going to put this group into another room where i'm going to give them small plates with a big amount of food in it and see how much you eat and i'm going to send you into this room over here and um i'm going to set you up with a different situation entirely create some kind of control where you get an understanding of what's going on maybe they get both and um and that is very much designed within controlled conditions so that you can't do them at a really good example actually of a randomized control trial um that was a big study and you couldn't do today because it would be unethical would be the um minnesota starvation experiment by ansel keys um because he took um he took conscientious objectors and he put them into a laboratory and basically locked them in um and then they were that their calorie intake was completely controlled over time and they were put on a starvation number and they were put on a starvation number and they were put on a starvation number of calories so that they could see what was happening to people who are in concentration camps and so on and so forth to understand it um and all of that was very very tightly closely um monitored and it was set up purposefully that way so that's a really good example of a randomized control trial they had complete control over everything you couldn't do that today because you can't lock people up and force them to eat things in a certain way and deprive them of so ethics always comes into this as well sometimes you can't study like you said humans are humans but a lot of the time you cannot study that because ethically it would be wrong whether that would be physically or psychologically if you are in any way possibly harming that person then you cannot do that study so it's there's a lot of things that we can't study and find out in the way we might want to and that's why we use things like rats and mice and stuff like that but then that doesn't necessarily translate over into humans and that's where our correlation causality problem comes back again was that helpful i mean
Speaker 1there's a lot of different elements isn't there to to kind of be aware of and i guess this is where um it can get really complicated if you don't have the time to sit and pull apart every source that you come across but um before we get to it what are the next things that we i don't want to derail you completely walking through this hierarchy and methodology and stuff is there any more on that before we go off on a tangent
Speaker 2well there's also an evidence hierarchy within that so what we've just been talking about there is individual studies but you also get systematic reviews and meta-analyses so one study is a single data point okay and it might be a really good data point because it might span years but it's still one study and it's a single data point and it's a single data point and it's a single data point a systematic review is going to pull everything that exists on that topic out it's going to analyze it using a defined method so even a systematic review has a method for how they will review it and they will say this is our criteria these are the studies we're going to include these are the studies we aren't we're going to do this because so it's quite rigorous in the way that it is and then it will review those studies together and give you an idea of what's going on and what's not going on and what's not going on and what's not going on of what the different single data points say when we pull them together over time so systematic reviews are really useful ways of seeing what the data tells us when we group it together and a lot of the time what a systematic review when we talked about cycle syncing and fitness training with them with periods that's a really good example where systematic reviews can actually pull together and say there is no single point of evidence here that says one way or the other and the data that we have isn't great because the methodology wasn't great in most of these studies and a lot of them were small sample sizes like it pulls it together and gives you an overview of the good and the bad points of the study the evidence base altogether that we have and then a meta-analysis statistically combines numbers across studies so it's very quantitative so it explains why studies are not good and why they're not good and why they're not good and why they're not good and why they're not good and why they're not good and why they're not good and why they're not good found x is weaker than a meta-analysis of 40 studies found this so essentially what you get is the data from all of those studies pulled together so that you can see it in one data point over time over studies so again that can be really helpful because one study might tell us that there is an you know a really strong um evidence base that suggests that this is occurring this thing is happening and then when you put all of the studies together and put all of that data together and analyze it actually it's suddenly weak and it that that p doesn't exist anymore like it's gone so essentially what we have is a really good um data point for all of the studies together so that is your most superior so if you did want to know about something you could basically google go to google scholar and go um meta analysis and then your topic and then it will show you if there is a meta analysis of those you don't have to go and find every single study that's ever been made on it now sometimes there isn't a lot of evidence in this area um women's fitness women's medical health can be one of those areas um it's gathering data but doesn't have a lot so on and so forth um that is often a point though where you'll see when that comes to light that now people will start doing more studies and then you'll get your meta analysis on it and so on and so forth so people are always looking for the gaps in the data and where there isn't um good evidence on it when a meta analysis is done and it shows that the the evidence isn't what we thought it was in these individual studies um then actually what you'll find is you'll often then get maybe some funding that goes into actually doing a longitudinal study with strong methodology and a larger sample size and so on and so forth so um yeah it's um they're your top of the hierarchy systematic reviews and meta analysis awesome
Speaker 1so with all of this stuff out there what are the sort of things that you would kind of pay particular attention to when you are looking at it to go i'm i'm going to trust this or this this is saying what i think it's saying
Speaker 2so i think what you want to do first of all is look at the abstract to know whether it's a study that is going to tell you the answers you're looking for so like i said the abstract is the overview it's the introduction it's the bit that's going to says this is what we found then skip to the methodology and look at the sample size and what they did was this 10 people was this a randomized control trial and or was this a longitudinal study over many years like what what way were they doing this study uh what was the sample size and the composition of it so for example let's go back to my master's one of my when i was doing my study was that i used a um a convenient sample so basically i put my own study out on my instagram and went who can answer my study that's very convenient but it's only going to hit people who are in my instagram so what are the people in my instagram like well they're like me because they're in my instagram now they're going to snowball that out it's going to gather momentum but is it going to really change the way that you're looking at it so what are the people who aren't like us no probably not so the way in which you're actually getting that sample matters and who that sample is matters so when you look at these studies and they make bold claims you can sometimes go there and you go especially psychology because they have a problem with the type of people answering their questions um and what you tend to do is go into it and find that they had 20 women all college educated all white and it's not really given as much of the the answers that we were looking for in the first place um or it can be the other way around it's like well actually we had a sample size of like 50 men and five women um and so now we know that we can't actually necessarily take this information and say it's it correlates across men and women because we didn't have a large enough female sample size a lot of the time they will make this clear themselves because what a good study should do is pull out the downfalls of it yeah they will actually say these were the problems with this study um and and they recognize that because you can't solve all the problems you know what they are whilst you're doing it you just get on with it um and you know someone else might do a different study that plugs that gap that you couldn't plug at that time you can't do everything at once but essentially it should tell you in the methodology or a little bit further down in the conclusion these were the overall results that we found and the results that we found and the results that we found and the this was the problem with the way that we did it and this might be a better way or future studies might need to and so it'll show you where the gaps are so your key points in your study are always your abstract your methodology and your conclusion and you can largely ignore all the data that they throw in in the middle you don't need to understand how to read that um because essentially if it is published and peer-reviewed then somebody else who knows how to read that has checked it
Speaker 1and it'll learn isn't it let's face it we've all got a lot going on and reading studies for those of us that don't find it fun it's it's gonna be a bit tedious um and i guess that's the other thing then you can um we tend to find people that we kind of trust and then we use them as our shortcut um i've already said here's my source um i like to think that i wouldn't completely take every word you say as there is this like an internal sense check and not because you're not trustworthy but because i think that you should be wary of taking anything that anyone says as completely read i'm sure that however much you like science there is a chance that sometimes you say something because you haven't you can't check every single statement that you make or every single assumption that you make and you can't check every single statement that you make and you can't check every single assumption that you've made on every topic ever so there's gonna be stuff where people that you do trust um you know humans aren't infallible and i guess that's my point is you might find people that you trust but also have that curiosity around the the the information or the bits that you're taking from them because something else i've kind of come across quite often and you do see is that sometimes some of the things that you're taking from them are someone can be an expert on one topic and they are they know all the research on that but that doesn't stop them from being inaccurate on a different topic because they're humans they're fallible they're exposed to the same things as the rest of us when it comes to the media elements for social media whether that's um kind of national international media whatever it is or somebody that they trust that they've taken the same problem but yeah i don't need to check that because i know that person's reliable so that kind of thing i think is really important kind of knowledge almost of what is that person's field of expertise and that's the thing that they probably know inside out and i'm never going to nudge them on that topic but that doesn't stop them from making inaccurate statements elsewhere so there is this kind of if you're going to rely on something questioning it maybe double checking it yourself is that what you'd
Speaker 2recommend yeah i mean definitely i think what's always hardest where you're always going to fall down everybody will is that when you agree with what the person is saying if it fits your value and belief system in the first place you are less likely to question it and so it is always worth trying to question that like when you go oh yeah yeah of course go oh is it of course and then just double checking that and you know it's maybe easier these days simply because we do have access to like ai for example and you can ask it a question and say this person has made this claim can you give me the evidence that exists for and against this and it will give you both of it and it might link and it will hopefully link you to topics don't 100 trust that either because it makes things up all the time but um sometimes it sends you on links you click on it go that's not what you said it was what on earth is that something about oats and wind and i asked you about nutrition but never mind um so do you want to answer that or do you want to answer that i don't know i don't know i have to check everything um but essentially it's it's always worth a quick google on something if it is something that you it's important if you're passing information on to clients or if you're using it to build something out yourself so that you are repeating those claims if even if it matches and especially if it matches your value and belief system already it is worth a double check that actually that is true um and it is so much easier absolutely right when it is somebody that we respect and we follow and we respect and we follow and we respect and we follow and we and we we believe what they tell us most of the time when they are going outside of their field of expertise they are no longer the authority on that subject and it is always worth looking up who is the authority on that subject um so authority is your first place who conducted it what are their credentials in this area and has it been peer reviewed is this information something that exists in a public sphere where other people have access to it and have been able to review it and privately being created by a company that no one else has any access to no one has any idea how they came about it and actually the reason that they're telling you this is because there is monetary financial gain that can be like you know accredited from it so yeah and then your second question is like what that leads into what was the purpose for this what is the agenda behind them telling me this information does it make sense that this information is something that's out there to fill in a gap in knowledge or to support some something moving forward moving forward for the public good. Or does this seem to be something that is only supporting one particular group of people or what financial gain for a company or an individual? Do you know what I mean? So it's essentially that purpose and that funding will always sit alongside the source and the place that that information has come from. That is always worth questioning. That is definitely worth questioning on things where you've got very public figures interviewing people on their podcasts and not going to name any names.
Speaker 1Yeah, I mean, there's a few things on there. One that you mentioned, I think it's worth pointing out is when you're looking for sort of whether you can trust something is to look for what's against it as well as what's for it. Because we always have that almost confirmation bias. And the question is, you want to ask is like, is this right? Rather than kind of and therefore what proves it? This is right, rather than what actually proves this might be wrong and potentially against against that. I think the other thing is that you kind of touched on it there a little bit as well as the idea of authority, because sometimes we think of the authority as almost being the person that's most vocal on something or almost the most the biggest celebrity. And I use the term loosely, but that kind of famous. That's well known, the loudest voice that that's there. And that doesn't necessarily automatically mean they are the person with the most knowledge on that topic. And quite often it's not. So that's it's basically a lot of the people who spend the time doing the studies are as huge assumptions. And this I completely get that big amount of stereotyping. They tend to be quieter people who like to spend time in not necessarily in labs. But with data and that kind of stuff, rather than the people that are front and center shouting loudly about this statement is is is true. And then another thing that you kind of mentioned there that I think is is worth doing, because you hear a lot about, you know, sort of the funding and the fact that, you know, if it's funded and somebody's got somebody who has an invested interest in it, then sometimes that's presented as almost like an automatic. It's not true. Whereas in reality, two things can be true at the same time. And it is possible because getting funding, right, you kind of sometimes the people that are going to pay for something are the ones that have got interested in it. So I guess as long as it has also had things like those peer reviews, it's got it's still got the solid methodology behind it. It's potentially been proven elsewhere and all of that kind of stuff. The funding becomes less important, whereas if it's a sort of one on its own. And there's this mystery around it and it hasn't been peer reviewed and it's very tightly tied to selling this particular thing, then there's probably a little bit more suspicion around it. Would you say that's kind of kind of accurate there? Because it is everything's nuanced, right? There's no kind of absolutes in any of it.
Speaker 2What you're looking for is transparency. Have they declared a conflict of interest? So if they have been funded by a food company to find this out, for example, it should be declared. That is a conflict of interest. It potentially could impact the results. And we need to know that so we can take a little deeper look at it so that it can be scrutinized at a greater level, because when that conflict of interest is there, then it needs maybe a deeper scrutiny to it. But essentially, that should be transparent. It should be really clear and easy to say who funded this or who didn't. Sometimes it is. And at the bottom of a publicly peer reviewed. Study in a journal, it will say declarations of conflict of interest, and it will say whether there are any or whether there are none. Your issue comes when it is you'll go to a Web site and it's a privately owned company and they say our research tells us our data tells us this, this, this and this. Now, that is not transparency that, you know, they're not saying our they never say that that would be more transparent. They say the evidence is the data is that's not transparent. Where did you get that information from? Now, if they then give you the study with the full methodology and the results and so on and so forth. Brilliant. Transparent. There's no conflict of interest there. But often what you find is you cannot read that study. It is inaccessible. And that's where the problem will lie. That's when you say, OK, we need to find other things. So then you can ask the question, has this been replicated anywhere else? Because actually good studies will be replicated. By others, because replication should be able to occur if that study found reliable and valid findings, then it will be possible. And this is why we publish methodology and results, because someone else should be able to come along and go, great, I'll replicate this with this population of people or in this area or in this way. So you should be able to find that you can replicate that study. And that is something people will try to do to prove that it is valid. That this is a source that we can rely on. A really good example of a huge study people might have heard of, because they made a film about it and everything, was the Stanford prison experiment, which he claimed told us something about authority and the way in which when you put a uniform on someone, they become authoritarian and so on and so forth. So many people tried to replicate it. I mean, there were many, many more problems of the study at the time. It was another one. And there's ones where you couldn't do it now because it'd be unethical. But essentially, it couldn't be replicated. It just couldn't be. People have tried and tried and tried and tried, and they have not come out with the same results ever. So that tells us that that study isn't as valid and reliable as we first thought maybe it was at the time. But again, it was one of those ones that became a really big name and all the rest of it because the media went, whoa, look at this, picked it up, ran with it. And people still talk about it as a valid study, whereas really, never been replicated. So it's probably not. So, yeah, other people will scrutinize these things. But that's the question. Is it open for scrutiny? And if it's not open for scrutiny, that's when alarm bells would ring for me and I'd say, well, OK, this is fine. We've got all this information, but you're telling me it and I just have to believe that I can't go in and look for this for myself. Reading quantitative results is really challenging. I had to learn how to do it and it was hard. And a lot of the time, you know, you won't be expected to know what you're looking at, but that is where you can use another source to help you with that. So, you know, your person who loves reading quantitative results and you can go to them and you can say, oh, I found this today, but I'm not really sure about it. And can you have a look at it for me? Or you can give it to AI and say, can you help me understand these results? And it is capable of doing that when you put the data into it. So it's not like you have to take it even at face value if you really want to dig into something and understand it better. I'd say dig into it, understand it better. But a lot of the time you wouldn't need to be able to do that because other people have done it for you. And always, always, always, if you do want to know the evidence base overall on a topic, go, right, let's see if a systematic review or a meta analysis exists on this. Let's see if somebody's already pulled that data pool together and found some statistical results or some qualitative results that are based on that. So we can get a bigger, better picture. Of what the overall evidence base actually looks like. But always question. I just think that's like the basis of life anyway. Like question. Never take anything at face value. Maybe I'm cynical. I don't know. But question. Ask questions. And you'll make sure that you maybe don't get pulled into believing something that doesn't have a full evidence base behind it.
Speaker 1Thank you very much as always. Thank you.
PodcastTranscript AI