Episode 31: Science vs Pseudoscience

In this episode, April realizes that the difference between science and pseudoscience isn't a distinct line, it's more of a continuum. Nevertheless, science remains scientific, while pseudoscience is, well, not so much.

Episode 31 Show Notes:

Here's the Science Council's definition of science:
https://sciencecouncil.org/about-science/our-definition-of-science/

Great discussion of the topic by Lillienfeld, Lynn, and Amaratti:
https://scottlilienfeld.com/wp-content/uploads/2021/01/lilienfeld2015-5.pdf

An extensive discussion of the differences from the Stanford Encyclopedia of Philosophy:
https://plato.stanford.edu/entries/pseudo-science/

Mario Bunge's article on "cognitive fields:"
https://cursosist2010.files.wordpress.com/2010/03/bungepseureduc.pdf

Great article from the Boston Review about Karl Popper and the pros and cons of falsification:
https://www.bostonreview.net/articles/michael-d-gordin-fate-falsification/

Newsweek article about "Behind the Curve:"
https://www.newsweek.com/behind-curve-netflix-ending-light-experiment-mark-sargent-documentary-movie-1343362

Good article that explains "bad" science:
https://www.digitaljournal.com/tech-science/why-some-science-is-actually-bad-science/article/455538

ThoughtCo's article about "hard" and "soft" science:
https://www.thoughtco.com/hard-vs-soft-science-3975989

Well, yeah, bad and fraudulent science is a problem:
https://www.theguardian.com/commentisfree/2023/aug/09/scientific-misconduct-retraction-watch

Fun and interesting (but ultimately serious) webpage with lots of great information about pseudoscience by Dr. Rory Koker:
https://web2.ph.utexas.edu/~coker2/index.files/distinguish.htm

Emily Willingham's Forbes artlcle about commercial pseudoscientific claims:
https://www.forbes.com/sites/emilywillingham/2012/11/08/10-questions-to-distinguish-real-from-fake-science/?sh=156ddfb6146c

Transcript

Episode 31—Distinguishing science from pseudoscience

Hi there, and welcome to episode 31 of Think It Through! In this episode I’ll be looking at something I have always thought was cut-and-dried—the difference between science and pseudoscience. I mean, astrophysics is science, and astrology is pseudoscience, right? Well, yes, that’s right; however, during my research I’ve come to realize that the demarcation between science and pseudoscience is somewhat murkier than I had originally believed. And frankly that could be one of the reasons that some people are pulled into a belief system that seems incredible to good critical thinkers. So here’s what we’re gonna do—first, we will go over the generally accepted definitions of science and pseudoscience to try to suss out the most obvious differences between them. Then we will talk about the murky region in between science and pseudoscience that can make it hard for someone to tell the difference. Finally, we will try to determine if there is a “line” or at least some kind of “boundary” that we can use to tell legitimate science from pseudoscience. 

Ok, let’s get started…

Music

According to the Science Council, a group that sets standards for practicing scientists and science technicians, Science is the pursuit and application of knowledge and understanding of the natural and social world following a systematic methodology based on evidence.

What is standard scientific methodology? It’s of course the scientific method that I’ve talked about in earlier episodes, and includes objective observation, hypothesizing, experimenting, measurement and data collection, using inductive reasoning to come to conclusions based on those measurements and data, repetition, critical analysis, and verification through peer review. So the Science Council’s definition is a pretty good one. Now what about pseudoscience? According to Merriam Webster’s dictionary, it is defined as “a system of theories, assumptions, and methods erroneously regarded as scientific.” That’s a bit vague, can we get a clearer definition? Let’s try this one by Dr. Scott Lillienfeld; “claims that appear to be scientific but that actually violate the criteria of science…(that) exhibit superficial trappings of science but little of its substance.” Ok that’s better. According to the Stanford Encyclopedia of Philosophy, the oldest known use of the English word “pseudoscience” dates from 1796, when the historian James Pettit Andrew referred to alchemy as a “fantastical pseudo-science.” Some of the “science” that alchemists attempted was to turn lead into gold or to find a mysterious elixir that would grant eternal life. This was supposedly achievable through mystical processes like transmutation, that Um, I don’t even pretend to understand but I’ll link to an article about it. Even today, with all our advances in science, we still can’t turn lead into gold or live forever. And back in the 18th century, that historian had the good sense to call alchemy what it was—a pseudoscience.  

How does pseudoscience violate the criteria of science? The late philosopher and physicist Dr. Mario Bunge had some very strong opinions about pseudoscience. Some of his many works specifically addressed that topic; I’ll link to one in the show notes. He divided what he called “cognitive fields” into two categories— “belief fields” and “research fields.” Research fields include those fields of study such as basic and applied science, mathematics, humanities, and technology. He said that these fields have certain components in common. Some of these are: a community of people who have received specialized training and education in their individual fields; a strong tradition of inquiry using strict testing methods; a realistic theory of knowledge; and up-to-date logical or mathematical theories which are testable and subject to correction or change. These kinds of fields of study DO change over time, either slowly or quickly, because of that inquiry.

Belief fields, in his view, would be fields that don’t satisfy those conditions I just mentioned. Examples of belief fields would be religions, political ideologies, and pseudoscience. He puts religion and political ideologies into the same general category as pseudoscience because none of them meet the same criteria as research fields and are therefore considered nonscientific in nature. He specifically breaks down pseudoscience into its component parts, including the following: 

·      unlike science, its conclusions tend not to change over the course of time;

·      any related dogma is staunchly defended;

·      any research practices utilized do not meet the same strict testing criteria as that of true scientific inquiry;

·      much of it refers to things that are not certifiably real, like astral influences, disembodied thoughts or spirits, or any immaterial entities or processes that simply can’t be quantifiably measured;

·      pseudoscience tends to use explanatory models that are not experimentally testable.  

Which leads me to a discussion of the concept of “falsifiability” and the work of the late great philosopher Karl Popper. One of his earliest and greatest desires was to be able to clearly differentiate between science and pseudoscience. He knew that this was a problem because science can and sometimes does make mistakes in its search for the truth, and conversely pseudoscience can occasionally happen to stumble upon some truth. So how might it be possible to determine that something is indeed science? Well, in 1919 two astronomers set up an experiment to test Einstein’s prediction that light is bent by gravitational fields. They did this by measuring the precise degree of the curvature of light from stars behind the solar disc during an eclipse. Their measurements showed that Einstein’s theory was correct. Looking at this, Popper realized that had the experiment showed that Einstein was in error, his theory would have had to be abandoned. Popper said, “One can sum up all this by saying that the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability.” In other words, the point of science is not just to provide evidence that something IS the case, but to try to think of circumstances under which it would NOT be the case. If a theory can be logically contradicted by an empirical test that strictly follows the scientific method, then it is considered false. 

While falsifiability is not a perfect test, it does point out the differences in the ways that science and pseudoscience attempt to prove that their theories are true. Science starts with a question about how things might work, and after some preliminary research, scientists come up with a hypothesis (which is a testable statement that may or may not be supported by evidence). Then they design an experiment that will show whether that hypothesis has any basis in fact. That experiment must conform to strict criteria to make sure it is actually measuring the thing they want to measure. They then conduct the experiment, collect and examine the data, and determine whether or not the hypothesis was shown to be correct. There are several ways this could go—the data could show their hypothesis was correct, in which case they would write up their results and try to get published in a reputable journal, where their report would be subject to peer review so that other scientists could check their work to make sure it’s been done correctly. If it is and their results are published, then other scientists might decide to try and replicate that experiment to see if they come to the same conclusion. And this goes back and forth until finally the vast majority of scientists come to a consensus that it is very likely to be the correct way to look at this particular subject.

Now, if the data does NOT confirm the hypothesis, the scientists have some options; they can reject the hypothesis and move on to another research question; or they can look at the data and see where it leads, formulate a new hypothesis, then conduct another experiment to see if that new hypothesis is supported. Just like any other humans, scientists don’t like to be wrong; however, they understand that an experiment is NOT a failure just because the data doesn’t support their original hypothesis. After all, a hypothesis is s TESTABLE statement that may indeed not be the case. It’s that testability, and the possibility that a hypothesis might be proven false, that helps scientists narrow down their options and zero in on what actually is the case.

Pseudoscience, on the other hand, tends NOT to start with a research question; it generally starts with a statement of belief, as in “The earth is flat; I know it, and I’m pretty sure I can prove that I’m right.” The Netflix 2018 documentary “Behind The Curve” shows some dedicated flat-earthers valiantly attempting to use science to prove that they are correct. One of their experiments, in which they use a very sensitive, very expensive laser gyroscope to prove the earth’s flatness, instead proves exactly the opposite, the gyroscope behaves just as it should on a rotating globe. The person carrying out the experiment later commented, "Now, obviously we were taken aback by that. 'Wow, that's kind of a problem. We obviously were not willing to accept that, and so we started looking for ways to disprove it was actually registering the motion of the Earth." So, when their experiment did not show what they wanted it to show, they simply refused to accept it, which is NOT how science works. Another experimenter in the documentary used sheets of Styrofoam with holes in them set at equidistant points in a line far enough apart to either prove or disprove that there was a bend in the earth by shining a light so it would, if the earth is flat, shine directly through all the holes over that distance. Well, guess what? The experiment showed that indeed the earth is curved. The experimenter’s words when he realized that? He said, “hm, that’s interesting.” Despite these attempts to try and use science to prove the scientific community wrong, they really give themselves away when they say things like, “Deep down inside, I think everyone knows that it’s flat.” That statement of belief informs everything they say and do and exposes flat earth theory as pseudoscience.

Music

While it seems pretty cut-and-dried in terms of what is and is not science, here’s where it gets a little murky. Every legitimate field of study uses research and experimentation to advance knowledge in those fields. But there are the “hard sciences,” or natural sciences, like astrophysics, evolutionary biology, or biochemistry. Then there are more “soft” or social sciences, like sociology, psychology, or my field, communication. It is generally considered more difficult to devise an experiment for social sciences, simply because what researchers are looking at is not a chemical or a mineral or a star from which they can make objective measurements, as in the hard sciences. In the social sciences, researchers are often looking at far more intangible things, like behaviors, thoughts, or feelings. It’s also difficult to set up an experiment that controls for all the possible variables, because humans are all different. And that makes it harder to replicate an experiment; the test subjects are never going to be exactly the same from one experiment to the next. In fact, an article in the Digital Journal by Dr. Tim Sandle (a microbiologist and science reporter) says that researchers from Stanford University stated that they could only replicate the results of about 40% of psychology papers. This doesn’t necessarily mean that the other papers were poorly constructed (although they may have been, and I’ll discuss that shortly); it does however show the challenges of social science research. According to an article in ThoughtCo called “What is the difference between hard and soft science?”, the distinction between them is “a matter of how rigorously a hypothesis can be stated, tested, then accepted or rejected.” Hard sciences, to put it simply, are often more precise than soft sciences, not necessarily better. Even if someone might think that the lack of precision negates the importance of social science research; the knowledge gained from that research is critical to our understanding of ourselves, and that’s as important as our understanding of the world and the universe around us that comes from research in the hard sciences.

Ok, that’s one issue; now it gets even murkier. On the outer boundaries of science, there is what can only be described as BAD science. Bad science can include things like poorly designed experiments, misreading of data collected during experiments, misconduct by researchers, or biased reporting of outcomes due to sponsorship by companies with vested interests in those outcomes. A poorly designed experiment might be one in which the sample size is too small or the test subjects aren’t really representative of the population being studied, or there’s no control group when there should have been one, or really any number of things. According to that Digital Journal article I mentioned before, thousands of published scientific articles have been retracted, or pulled from journals, due to both unintended and deliberate errors. About 20 percent of those retractions were because of unintended errors like miscalculations of statistics; however a far greater number of articles were pulled due to misconduct or fraud. Ivan Oransky and Adam Marcus, co-founders of Retraction Watch, a group that monitors and tracks retracted journal articles, said in a recent article in The Guardian that while only 1 in 1,000 papers ends up getting retracted (which is a tenth of a percent, a very small number comparatively), they believe that number is probably vastly undercounted. The science community, in general, does its best to catch these things during the peer review process, but bad science can and does slip through the cracks and ends up getting published. Scientists are human, and things like peer pressure, competition, and the need for publication in order to maintain one’s position at a university can all factor into those choices. They certainly can’t be justified, though, and these problems play into the overall decline of trust that the general public has in science. 

To sum up this “murky” section of the episode--despite the lack of preciseness in some science research, and even though there is definitely some not-very-good or even fraudulent science going on out there, those things still do not fall into the category of pseudoscience. For better or worse, they attempt to use systematic and critical investigations to acquire an understanding of our world and the people in it. And the continual search for knowledge along with science’s overall ability to self-correct and update what we know based on new information helps us to keep on a slow but continual upward path.

Music

Ok, so now you can see that there’s not a definitive “line” between science and pseudoscience; it’s more of a continuum. So where does that leave us? The Stanford Encyclopedia of Philosophy’s article on science and pseudoscience is a great in-depth discussion of this very problem. I’ll of course link to it in the show notes, but I’ll just briefly touch on some of the ideas in it:

One thing that I found really interesting was that there are two forms of pseudoscience-- pseudo-theory promotion and science denialism. The first one, pseudo-theory promotion, would be things like astrology, homeopathy, ancient astronaut theory, and cryptozoology, in which advocates tend to describe their theories as basically conforming to mainstream science, even though they don’t. Science denialism, on the other hand, produces false controversies to legitimate science in order to prove science wrong and includes things like climate change denial, vaccine denial, creationism (which is basically evolutionary science denial), and even holocaust denial. Both types of pseudoscience conflict with the results and methods that have been generally accepted in the science community, but the former tries to fit in with established science while the latter attempts to overturn established science in favor of their own conclusions.

These two types of pseudoscience may be different, but they do have many things in common. They are both sustained efforts to promote standpoints, ideas, and conclusions that differ from those that have scientific legitimacy. The report says, “Proponents of pseudoscience often attempt to mimic science by arranging conferences, journals, and associations that share many of the superficial characteristics of science, but do not satisfy its quality criteria.” The whole idea is to create the impression that they ARE scientific, yet their ideas are at variance with the most up-to-date and reliable knowledge about a subject.

The article lists criteria that can be used to identify pseudoscience. Here are some of those criteria:

1.     Belief in some special authority: Rather than relying on a consensus of educated, experienced, and knowledgeable researchers about a subject, pseudoscience often sets up some person or persons who supposedly have a special ability to know what is “true,” and the rest of us should accept their judgments without question.

2.     Unrepeatable experiments: Remember, scientists check each other by attempting to repeat experiments. In pseudoscience, reliance is put on experiments that cannot be repeated by others with the same outcome. The inability to replicate some experiments happens in real science too, but the difference is that in pseudoscience, it doesn’t matter that the experiment can’t be repeated; it still carries the same evidentiary weight.

3.     Handpicked examples: A few very compelling examples are often used as evidence, even if they are not truly representative of the things being studied, and are often anecdotal and testimonial in nature. I would also add that one or two examples of something really doesn’t prove anything and falls under the fallacy of hasty generalization, which I’ve talked about in previous episodes.

4.     Unwillingness to test: Much pseudoscience espouses theories that are not tested and may have never been tested, although it is possible to test them. I see this all the time in pseudoscience, someone puts forth an idea and just rolls with it, never bothering to test it or to even check to see if there’s evidence out there that would easily refute it.

5.     Disregarding of refuting information: Observations or experiments that conflict with a pseudoscientific theory are neglected or ignored by pseudoscientists. In fact, unwillingness to accept well-supported factual statements or failure to revise claims in light of new data is one of the hallmarks of pseudoscience.

Dr. Rory Koker, a professor in the physics department at the University of Texas, teaches a course in pseudoscience, and has a great fact sheet with all kinds of information about how to distinguish science from pseudoscience. He lists some differences between them that might be helpful in determining which is which. For instance, the goals of each are different—the primary goal of science is to gain a more complete and unified understanding of our universe, while the primary drivers of pseudoscience are ideological, cultural, or even commercial in nature—it’s really more about belief than knowledge. Another difference--Science continually expands its knowledge through research and is willing to change and evolve based on new information, but because pseudoscience is mostly interested in proving itself correct, it really doesn’t care about information that doesn’t fit in with what it already believes to be the truth, and so its conclusions tend not to change much over time. A third difference--while it’s not unusual for scientists to seek out counterexamples and inconsistencies, questioning their own and others’ theories as an integral part of the search for knowledge, that’s not what happens in pseudoscience at all. Any challenge to accepted dogma is considered hostile and even heretical and can lead to bitter disputes and schisms within a community that believes in that pseudoscience.

A Forbes article by science reporter Emily Willingham titled “10 Questions to Distinguish Real Science from Fake Science” focuses more on the commercial aspect of pseudoscience claims, usually related to homeopathy, naturopathy, so-called “cosmeceuticals,” and magnetic or electromagnetic therapy. She discusses how these claims sound “science-y” even though they are not based on actual science and are usually presented to us through advertisements particularly on social media. Her criteria include things like looking at the source and their agenda (who are these people and what are they trying to get you to do, are they trying to make you feel like you have a problem you didn’t even know existed?), look at the language (are they using a lot of jargon or words that sound science-y and you’re not sure what they mean but it seems like they’re using that language to persuade you that they know what they’re talking about?), does it involve testimonials (which I’ve already mentioned as being generally poor evidence), does it have claims of exclusivity (as in, it’s proprietary or secret or somehow special and they are the only ones that have it), is there a mention of a conspiracy of any kind (like, doctors don’t want you to know about this, or the government has been hiding this for years), does this thing claim to cure multiple disorders at once like cancer and adhd, and are you unable to find any scientific research associated with this product, just claims that there has been research done but no links to that research? All of those things are red flags for pseudoscientific claims, so unless you want to end up with a bunch of useless items like magnetic bracelets that don’t help your arthritis or magic creams that don’t make your wrinkles disappear, it would be wise to be scroll past those ads.

Music

In the end, Dr. Coker says that the best way to spot pseudoscience is to know as much as possible about the real thing, science itself. It’s absolutely true that the more we understand science, the easier it is to recognize its opposite. I’d like to conclude with a quote from him that points out why it’s so important for us to understand the difference; I’ll bet some of his reasons are ones you hadn’t even considered:

It is, unfortunately, vital for each citizen to learn to distinguish carefully between science and pseudoscience. In a democracy, every voter must be capable of seeking and recognizing authentic sources of information. Pseudoscience often strikes educated, rational people as too nonsensical and preposterous to be dangerous, a source of amusement rather than fear. Unfortunately, this is not a wise attitude. Pseudoscience can be extremely dangerous. Penetrating political systems, it has justified atrocities in the name of racial or religious purity, purging of university faculty in math and science, and interference with and discouragement of basic scientific research; penetrating the educational system, it drives out science and sensibility; penetrating the health professions it dooms thousands to unnecessary death or suffering; penetrating religion, it generates fanaticism, intolerance, and holy war; penetrating the communications media, it makes it nearly impossible for voters to obtain factual information on public issues of extreme importance— a situation which at present has reached crisis proportions in the U. S. 

That’s all for now, and I hope you use the information in this episode to help you Think It Through.