Chapter 1: The Role of Correlation in Science

Correlation serves as a foundational concept in various scientific fields, from medicine to social sciences. For instance, if you're aiming to develop a new pharmaceutical treatment, gathering biomarkers that show a positive correlation with the disease can be a starting point. You can then experiment with different chemical combinations that support this correlation, relying on trial and error to potentially discover a new cure.

Imagine now that you're examining whether the pricing of your new drug influences your company's stock market performance, or vice versa. Conducting a linear regression analysis on historical data for both variables is a common practice in the industry. If your results indicate no correlation between drug pricing and stock value, you might feel confident in setting the drug's price independently of market fluctuations. However, this assumption could lead you astray, and the misunderstanding surrounding correlation may be the culprit.

This text aims to clarify the concept of correlation, beginning with its historical development. We will examine the relationship between correlation and regression, while incorporating elements of analytic geometry. Familiarity with regression concepts will enhance your understanding as we progress. By the end of this discussion, you should be better equipped to navigate the common misconceptions surrounding these statistical concepts.

Section 1.1: The Genesis of Regression

Our narrative begins with Francis Galton, a distinguished polymath whose explorations led him to investigate heredity. He observed that children of taller parents typically grew taller than average but remained shorter than their parents, while children of shorter parents followed a similar pattern. Galton labeled this observation "regression," essentially identifying the tendency of traits to revert toward the average.

However, Galton sought to quantify the influence of regression alongside heredity on height. He aimed to determine the extent to which each factor contributed to a child's final height.

Subsection 1.1.1: Galton's Correlation Analysis

Galton meticulously collected data on the heights of parents and their children, creating a table that illustrated the relationship between these two groups. He noticed a distinctive elliptical pattern emerging, indicative of a non-random relationship. This observation was documented in his 1886 paper, “Regression Towards Mediocrity in Hereditary Stature.”

By plotting the height of parents on the horizontal axis and that of their children on the vertical axis, Galton pioneered the concept of scatter plots, a key tool in modern data visualization.

Section 1.2: Correlation Illustrated through Scatter Plots

To illustrate the role of chance and heredity in height determination, let's consider three scenarios:

1. If heredity were the sole determinant, the scatter plot would yield a perfectly diagonal line, indicating that every child would match their parent's height.
2. Conversely, if height were entirely dictated by chance, the plot would resemble a circle, showing no discernible relationship between parent and child heights.
3. In reality, Galton’s data presented a mix of both influences, resulting in an elliptical scatter plot.

This led to the following insights:

1. A fully deterministic outcome results in a straight line.
2. An outcome driven solely by chance manifests as a circle.
3. A mixed influence yields an ellipse with varying eccentricity.

This eccentricity measure is what Galton termed correlation, a concept further refined by scholars such as Karl Pearson.

Chapter 2: Misunderstandings Surrounding Correlation

The first video, "Normality is not an assumption of correlations - Most MISUNDERSTOOD concept in Statistics," explores the intricacies of correlation and its common misinterpretations.

Understanding the phrase "correlation does not imply causation" is crucial. It serves as a reminder that just because two variables show a correlation does not mean one causes the other. For instance, while specific biomarkers may correlate positively with a disease's cure, this does not guarantee a causal relationship.

Many researchers now grasp the importance of interpreting correlation without imposing causal assumptions. However, the non-transitive nature of correlation continues to confuse many.

Consider the relationship (a > b > c); if ‘a’ is greater than ‘b’ and ‘b’ is greater than ‘c’, then it naturally follows that ‘a’ is greater than ‘c’. This transitive property, however, does not apply to correlation.

Suppose your new drug elevates certain biomarkers, which are correlated with a disease's cure. The intuitive leap to conclude that your drug cures the disease is flawed.

The second video, "The Most Misunderstood Concepts in Physics," highlights similar misconceptions in scientific reasoning, reinforcing the need for careful analysis.

It's also critical to recognize that uncorrelated variables aren't necessarily unrelated. The common practice of searching for linear relationships in regression analyses can mislead researchers. Just because two variables are deemed uncorrelated doesn't imply independence; they may have a complex, non-linear relationship that could evolve over time.

In summary, correlation is a nuanced concept that demands careful interpretation to avoid missteps in scientific inquiry.

References and Credits:

Francis Galton, Jordon Ellenberg.