Now these are my random thoughts on the specific part of that «Vernaculis vs The Alternative Hypothesis» thing, which obviously is yet another nature/nurture debate.
If you look at the comments below the second video and the third link, you may find the tendency to use the «correlation does not imply causation» argument to dismiss the studies — the argument that (at least in my opinion) is on par with «strawman!» or «Absence of evidence is not evidence of absence» ones and constitute the set of the most abused arguments out there. I’m going to address the whole argument first and then deal with the «debate» itself.
I would say that stating «Correlation does not imply causation» is always a bad idea. So why is it so?
Let’s start with looking at correlation. Correlation in a wide sense is merely any kind of dependence between two sets of data (there is such thing as autocorrelation, where the two sets of data are produced from the single source, but that doesn’t actually change anything). The word itself is actually an umbrella term for the wide range of relationships which are usually describe by parameters ranging from Pearson coefficient, which is the most common one, to something like coefficient of determination or the explained sum of squares.
Taken from Wiki article: Pearson coefficient for (x,y) distribution.
But getting the parameters usually isn’t enough. One of the ways to make inferences from a data set is Null Hypothesis Significance Testing (the reason I’m picking this one as an example of hypothesis testing is because it’s frequently presented in TAH’s video and I’m planning to take some shots at it) that formulates the null and alternative hypothesis (which are mutually exclusive) and proceeds to use statistical estimates to find out whether one can actually reject the null hypothesis, and therefore accept the alternative one.
Causation, on the other hand, is a mere notion of connection/dependence between two events.
The argument itself states that the mere connection between two sets of data (however strong it may be) isn’t enough to deduce a cause and effect relation between them, which is obviously true. One can make the examples of spurious relationships or Simpson’s paradox as some good examples of this rule.
Simpson’s paradox: the trend line is reversed when 2 data sets are combined
There are some research methods that aim at reducing the initial sample size biases, thus reducing the probability of running into spurious relationships (e.g. RCT).But then, results of any randomized test don’t imply causation either (and some fields don’t allow these kinds of tests — in this case, you can’t randomly replace genes while preserving the rest of the conditions).
And there is actually no way out of this. Causation cannot be deduced from any statistical data. There is no property in any statistical distribution that would tell us how would that distribution behave if the external conditions happen to change. Actually, the correct answer to «How would this distribution change if we changed X and X only» is any random distribution, and the answer would be compatible with all of the laws of probability theory we know.
That brings us to the first point. While statistics and correlation deal with our beliefs about the world in the case of partial knowledge, the causation is the property of the world itself and deals with dynamic changes in conditions. Therefore, saying «Correlation does not imply causation» when given some statistical data is close to asking «Prove me that X is OBJECTIVELY true. What I currently see is that you have some evidence, but you can’t be 100% sure, can you?» and is conflating our model of the world with the world itself while having a smell of trying to make your point unfalsifiable.
And while there is such thing as causal calculus (a short introduction, this one is not so brief), and it allows you to make assumptions about causal relations (e.g. the short example), it deals with a completely different approach to causation while still not being able to actually prove it (what is can is calculate the probability of them — the thing that some people like to state as «While correlation does not imply causation, it is highly correlated with causation»). And even then the usual network’s size is a bit…large.
The typical causal network. Taken from here.
But how often do we need causation? The one thing that we value in our models in their ability to predict things. And statistical data actually has predictive capabilities. As a part of regression analysis, some attempts have been made to resolve the «correlation isn’t causation» problem, which resulted in things like Granger causality or convergent cross mapping. And they can’t determine causality — rather than that, the thing they find is so-called «predictive causality», which is the ability to predict the pattern based on another pattern. Thus, every time you actually ask for causation, it’s better to determine whether you need causation in the first place — and more often than not it turns out that we don’t. What we usually seek for is a good predictive model of the events, and statistics has all the right tools for the given task.
Polynomial regression and Granger causality.
And the final one — while correlation does not imply causation, correlation is the evidence towards causation. We call event X an evidence towards hypothesis Y if X is actually more probable given Y opposing to non-Y (mutual information for binary (X, Y) pair is positive). And it’s quite easy to see that causation is more likely to generate correlated samples than the absence of causation. The strength of the evidence is subject to discussion, but it doesn’t prevent it from being evidence. Some of the common «fallacies» (this one included) simply go away when you switch from classical logic to probabilistic one.
What is a general trend, though, is the tendency to use this argument to dismiss any statistical data that runs counter to one’s beliefs (rarely you will find someone resisting the temptation to throw any data that confirms one’s viewpoints, however weak is the evidence it provides). It is a simple cop-out we tend to use when it suits us, usually without bothering to get enough background in statistics or probabilistic rules of inference (and I’m pretty sure it’s a great idea to try the latter before going on the «This is a fallacy!» spree all around the internet). Argument win buttons are easy to use , but they are not always right. And you’re always running the risk of looking like R.A. Fisher that used the correlation card to dismiss the fact that smoking caused cancer.
The other thing I’m going to tackle is the overuse of Null Hypothesis Significance Testing (NHST) in TheAlternativeHypothesis’s videos, and why it’s generally bad to draw large conclusions from NHST tests.
First of all, I’d like to point out that NHST is actually used in almost every study that TAH used there: heritability of political views, another one, this one has Chi-squared test, heritability with age are good examples. NHST is highly criticised nowadays, and it’s hard to find anyone who hasn’t yet taken a punch at it. Therefore, there is a point in stating that current NHST widespreadness is more attributable to custom than its actual benefits.
- NHST doesn’t answer the question you actually wanted to. Instead, it answers exactly the opposite one. While we would like to get the probability of the null hypothesis given the data we have, NHST tells us the probability of getting the data if the null hypothesis is true.
- If one understands the p-value correctly, it’s easy to realise that small p-values actually don’t mean that the probability of the mistake is low. Instead, NHST tends to focus on type 1 errors while tolerating type 2 errors by stating that the given data is unlikely under the null hypothesis. But it doesn’t actually say that the data is more likely to get under the alternative one.
- NHST in incredibly biased towards large sample sizes. While it may be useful to provide multiple researches with medium sample sizes, the results may be statistically insignificant. Surprisingly, grouping all the studies into one large sample may fix the problem if one uses NHST. Large sample sizes allow even the smallest differences in the results become significant.
- It insists on the binary result. We don’t get the probability of the alternative hypothesis, we don’t get to make any useful inferences. Instead, we are forced to retain or reject the null hypothesis. NHST simply doesn’t give enough useful information about the sample.
Also, p=0.05 is fucking cancerous.
The more extensive version of criticism can be found here. NHST is one of the weakest tests out there and basing all your conclusions on NHST studies is an extremely unreliable approach.
At this point I’ll try throwing some points around.
First of all, I don’t agree with all the points TAH makes, so don’t expect me to defend them. IMO, some of his inferences are profoundly unfounded, which doesn’t make the things I will be responding to any better.
When you try to clain that X are genetically predisposed towards Y, it is not merely enough to show that X tend to be more Y. (TAH’s video comment)
When the «X tend to be more Y» part is shown, we are left with the question about the reason. Without any other data, the correct a priori assumption is to attribute the results to a certain combination of nature and nurture.
But TAH has never claimed that genetics is the only or even the deciding factor that contributes to any of the IQ/political views/insert another thing here. Therefore, he is not wrong here.
If a white person is raised in Saudi Arabia, chances are extremely high they will be a Muslim. Islam is a part of culture. (TAH’s video comment)
You also have not scientifically even quantified “culture” which is another problem for you. You seem to be under the misapprehension that culture is not the very definition of nurture. (Blog post)
So what you’re actually stating is that being a muslim is totally a matter of nurture here, and this is a positive claim that has to be proven (and to explain why it seems to contradict the studies presented by TAH).
Your general strategy across this video seems to be to let the opponent make all the positive claims while trying to dismiss them as being unfounded (which isn’t bad in any way). But your position is such that it’s almost impossible to restrain from making positive claims.
And yes, this one will be hard to prove, because you will have to prove that no matter what the nature is, getting raised in Saudi Arabia results in becoming a muslim (yes, it’s a mirror). I would say that given the links, it is extremely hard to assume nature away when talking about religion, but i may be wrong here.
Being as close to environmental determinism as you are should bear at least some of the burden of proof.
You are very keen on leaning on studies that simply cite things that correlate. For example, you cite a study which makes the claim that IQ correlates with political view. However, clearly this study does not control for all or even most outside factors so there’s no way to determine that IQ is a deciding factor on what political views one holds. (Blog post)
and a lot of your commenters, seem to have an issue with my saying “Correlation doesn’t imply causation” in regards to a lot of your points. Even though they are actually valid points. (Blog post)
Also, regression analysis is a statistical estimate, not a biological experiment or study. So when you ask me to stop remaining skeptical towards a shaky estimate, I’m going to have to politely refuse that request. (Blog post)
I have seen zero separated at birth studies trying to see whether people of a certain race still tend to prefer the culture that corresponds with their race, I guess, or any twin studies. That being, twins growing up in completely different cultures and seeing which they are more predisposed to when they are adults. (Blog post)
Almost all of this is just a variation of the «correlation — causation» idea.
You can’t simply cite something that states “Asians tend to prefer collectivism.” There is absolutely zero genetic aspect involved within that.
TAH seemed to actually deal with this point. And these things seem to involve genetics, tbh.
Taken from population differences in individualism.
No, I will not drag race here. Dragging race is actually establishing another genetic connection I am not willing to make. But simply saying that genetics has nothing to do with culture while running around with that «OMG, correlation!» table doesn’t seem like the great approach to me.