LDA

Steps

• Compute the dd-dimensional mean vectors for the different classes from the dataset.
• Compute the scatter matrices (in-between-class and within-class scatter matrix).
• Compute the eigenvectors $(e_1,e_2,...,e_d{e}_1,e_2,...,e_d)$ and corresponding eigenvalues $({\lambda }_1,{\lambda }_2,...,{\lambda }_d{\lambda }_1,{\lambda }_2,...,{\lambda }_d)$ for the scatter matrices.
• Sort the eigenvectors by decreasing eigenvalues and choose $k$ eigenvectors with the largest eigenvalues to form a $d\times k$ dimensional matrix $W$ (where every column represents an Eigenvector).
• Use this $d\times k$ Eigenvector matrix to transform the samples onto the new subspace. This can be summarized by the matrix multiplication: $Y=X\times WY=X\times W$ (where $X$ is a $n\times d$-dimensional matrix representing the $n$ samples, and $y$ are the transformed $n\times k$-dimensional samples in the new subspace).