A statistical analysis resulting in an inductive inference following from a flow of deductive steps under a set of assumptions can only be supported by a valid model if random samples are taken. There are no magical mathematical or scientific tricks around this foundation, but scientists often take shortcuts; because doing it the right way is hard, time consuming and costly. However, the tendency to ignore the basic foundations of Statistics has lead science into what some consider a crisis, in which far too many published studies do not meeting the replication criteria of science. Articles are being published for their eye catching titles, rather than their scientific value, because real value is hard to obtain, but slap some “stats” on a news worthy title and you have a quick sell.
I would take this study with a giant grain of salt. Their parameters are poorly defined, they failed to clearly detail their "recruitment" methods, there is selection bias and they didn't seem to fully address the limitations of their approach (I am always cautious of a study that does not address its own limitations).
I am a graduate student in the field of Statistics, and this study looks like something one of my professors would use an example of what not to do.
From the study,
We recruited 85 adults (50 women, mean age = 23.35 years, SD = 7.82) in the Paris region. All had attended university (mean length of university curriculum = 2.85 years, SD = 1.18), but none majored in mathematics. Considering the low complexity of the math problems involved, participants’ curriculum was a clear indicator that they possessed the mathematical expertise required to solve the problems. Sample size was determined using uncertainty and publication bias correction on results from a previous study (Gros et al., 2016), following Anderson, Kelley, and Maxwell’s recommendations (2017).
We recruited 25 experts (two women, mean age = 23.59 years, SD = 2.81) who had successfully passed the entrance exam of the Science section at the École Normale Supérieure (ENS Ulm) in Paris. This exam is considered as the most demanding one in France, with an entrance rate of 2.02% among university-educated participants (“SCEI Statistics”, 2017). The ENS ranked second in Times Higher Education’s World University Rankings 2016–2017 for Best Small University (Bhardwa, 2017). Although the population sample was smaller than in the first study due to the number of graduates from École Normale Supérieure being limited, sample size was deemed sufficient using uncertainty and publication bias correction on results from a previous study (Gros et al., 2016), following Anderson et al.’s recommendations (2017).
Edited by Jeremiah, 27 July 2019 - 10:02 AM.