What is multicollinearity and why is it important to recognise when applying AI?

Multicollinearity is a statistical concept that frequently occurs in AI and machine learning models and can lead to biased results and interpretations. In this blog, I explain what multicollinearity is, why it can be a problem in predictive models, and how to address it.

What is multicollinearity?

Multicollinearity occurs when two or more independent variables in a statistical model are highly correlated with each other. This means that these variables contain similar information, making it more difficult to determine the individual effect of each variable on the dependent variable you are trying to predict. As a result, the estimates of the model parameters can become unreliable, which negatively affects the model's predictions.

A simple example

Suppose you want to predict an employee's salary and you use the following characteristics as input variables:

• Age
• Years of work experience
• The sector in which someone works

These variables are called independent variables, as they can all influence the dependent variable, in this case, salary. However, in this example, age and years of work experience can be strongly correlated. After all, the older someone is, the more years of work experience that person likely has. This creates a high correlation between these two variables, which is a typical form of multicollinearity.

Why is multicollinearity a problem?

If variables are strongly correlated, they can cause problems in your model. This is because it becomes difficult to determine which of the variables truly influences the outcome. The model can, therefore, provide very biased predictions. In our example, the AI model might unfairly overestimate the influence of age and underestimate the influence of work experience, or vice versa. This leads to reduced accuracy and reliability of the model.

How do you recognise multicollinearity?

You can detect multicollinearity by using the Variance Inflation Factor (VIF). This measure indicates how much the variance of a model parameter increases due to the presence of correlation between the independent variables. If the VIF value of a variable is greater than 5, you are likely dealing with multicollinearity.

How do you solve multicollinearity?

1. Removing one of the highly correlated variablesIf two variables contain almost the same information, you might consider removing one of them. In our example, you could choose to remove either age or years of work experience from the model.
2. Using PCA (Principal Component Analysis)PCA is a technique that transforms highly correlated variables into new, unrelated variables. This preserves the information while minimising the effect of multicollinearity.
3. Combining variablesIn some cases, you can combine the variables. For example, instead of using age and years of work experience separately, you could create a new variable that represents the ratio between the two.

Conclusion

Multicollinearity can significantly impact the performance of your AI models. By being aware of this issue and addressing it with techniques such as removing redundant variables, PCA, or combining variables, you can ensure that your models become more robust and reliable.

Want to learn more about how to optimise your AI models? Watch the video.

• Best AI tools
• GEO: discoverable in AI search engines
• Writing AI prompts