{"id":8257,"date":"2020-05-06T00:57:45","date_gmt":"2020-05-06T00:57:45","guid":{"rendered":"https:\/\/www.thecoderschool.com\/?p=8257"},"modified":"2022-10-13T20:49:56","modified_gmt":"2022-10-13T20:49:56","slug":"advanced-machine-learning-techniques-principal-component-analysis","status":"publish","type":"post","link":"https:\/\/www.thecoderschool.com\/blog\/advanced-machine-learning-techniques-principal-component-analysis\/","title":{"rendered":"Advanced Machine Learning Techniques: Principal Component Analysis"},"content":{"rendered":"

By Camille D., Age 17<\/strong><\/p>\n

This article will focus on a method data scientists and programmers use to make data easier to explore, visualize, and interpret data, called <\/span>principal component analysis (PCA)<\/b>. The explanations in this article assume some background in linear algebra and statistics.<\/span><\/p>\n

PCA is based on <\/span>dimensionality reduction<\/b>: \u201cthe process of reducing the number of random variables under consideration by obtaining a set of principal variables,\u201d in other words, transforming a large dataset into a smaller one without extracting too much key information. This process is considered expensive for machine learning algorithms; a little accuracy must be traded for simplicity. Minimizing this cost is part of the job for PCA. <\/span><\/p>\n

The first step of PCA is <\/span>standardization<\/b>, the process that is the least mathematically involved. Standardization takes care of the variances within the initial variables, specifically with regards to their ranges. For example, the value of one variable may lie within the range of 0 to 10, and the value of another within the range of 0 to 1. The variable whose possible value lies between 0 and 10 will carry a greater weight over the second variable, leading to biased results. Mathematically, this can be addressed by subtracting the dataset\u2019s <\/span>mean<\/span><\/a> from the value of the variable and dividing this result by the set\u2019s <\/span>standard deviation<\/span><\/a>.<\/span><\/p>\n

\"\"<\/p>\n

After standardization is performed, the values of each variable will all be within the same range.<\/span><\/p>\n

Note that standardization is different from <\/span>normalization<\/b> in descriptive statistics. Normalization rescales the values into a range from 0 to 1, while standardization rescales the dataset to have a mean of 0 and a standard deviation of 1.<\/span><\/p>\n

In almost any case, of course, this will yield a value smaller than 1.<\/span><\/p>\n

The second step, <\/span>covariance matrix computation<\/b>, is where things unfortunately begin to get more complicated. We first must understand the definition of <\/span>covariance<\/b>: \u201ca measure of how much two random variables vary together.\u201d<\/span><\/p>\n

\"\"<\/p>\n

 <\/p>\n

**Covariance differs from <\/span>correlation<\/b> in that correlation describes how strongly two variables are related, while covariance indicates the extent to which two random variables change with one another. The values of covariance lie between -\u221e and \u221e, while the values of correlation lie between -1 and 1. Correlations can be obtained only when the data is standardized. <\/span><\/p>\n

Covariance matrix computation aims to investigate how the variables in the input dataset are related to one another. This is important because it helps detect redundant information that may come from a high correlation between two elements. We compute a covariance matrix to determine these correlations. The covariance matrix is an <\/span>n<\/span>n<\/span> matrix, where <\/span>n<\/span>is the number of dimensions, that has entries of all possible covariances within the dataset.<\/span><\/p>\n

A couple notes:<\/span><\/p>\n