Categories

Revision to LDA numerical example

There was typing mistake in calculating the pool within group covariance matrix.

It was writen as 3/7*0.1349+4/7*2.142=1.689 while the correct one should be 4/7*1.349+3/7*2.142=1.689

Thanks for Gal and  Dafna Hirschfeld for pointing this out.

See the updated version in : http://people.revoledu.com/kardi/tutorial/LDA/Numerical%20Example.html

Spanish Translation of Numerical Example for K means

Spanish Translation of Numerical Example for K means tutorial is available.  Thanks to Jaime Orjuela (Escuela Colombiana de Ingeniería) from Columbia.

See: http://people.revoledu.com/kardi/tutorial/kMean/EjemploNumerico.htm

Data Table in MS Excel

Sometimes, you would like to create a table from a series of complicated formulas. In that case, you need Data Table.

A new tutorial on how to use Data table (What if Analysis) in Excel using simple example how to create data table with one input variable and two input variables is available in http://people.revoledu.com/kardi/tutorial/What-If-Analysis/index.html

Footrule distance for non-unique rank

I have updated footrule distance tutorial with additional example of non-unique rank.  Often we encounter rank that repeating such as 1, 2, 3, 3, 4 instead of 1,2,3,4,5. How to compute footrule distance for such non-unique rank?

Please visit 

http://people.revoledu.com/kardi/tutorial/Similarity/FootruleDistance.html for the answer.

Update Distance Matrix of K-Means Clustering Tutorial

 

Code for distance matrix computation is updated. Now it can handle multidimensional variables. 
The previous code that work only for 2 dimensions is given here for historical purposes.
function d=dist(A,B)
% DIST return distance matrix between point A=[x1 y1] and B=[x2 y2]
% Number of points in A and B are not necessarily the same.
% It can be use for distance-in-a-slice, distance-between-slice,
% and distance between guest-points-and-the-points
% A and B must contain two columns,
% first column is the X coordinates
% second column is the Y coordinates
% The distance matrix are distance between points in A as row
% and points in B as column.
% example: distance-in-a-slice= dist(A,A)
% distance between guest-point-and-the-point = dist(A,B) , hA=hB
% distance between slices = dist(A,B), with hA ~= hB or hA=hB
% A=[1 2; 3 4; 5 6]; B=[4 5; 6 2; 1 5; 5 8]
% dist(A,B)= [ 4.24 5.00 3.00 7.21;
% 1.41 3.61 2.24 4.47;
% 1.41 4.12 4.12 2.00 ]
[hA,wA]=size(A)
[hB,wB]=size(B)
if hA==1& hB==1 
d=sqrt(dot((A-B),(A-B)));
else
C=[ones(1,hB);zeros(1,hB)]
D=flipud(C)
E=[ones(1,hA);zeros(1,hA)]
F=flipud(E)
G=A*C % = repmat(A(:,1),1,hB)
H=A*D % = repmat(A(:,2),1,hB)
I=B*E % = repmat(B(:,1),1,hA)
J=B*F % = repmat(B(:,2),1,hA)
d=sqrt((G-I’).^2+(H-J’).^2)
end

I updated the Matlab code for distance matrix computation. Now it can handle multidimensional variables.  Here is the new code:

function d=DistMatrix(A,B)

             %%%%%%%%%%%%%%%%%%%%%%%%%

             % DISTMATRIX return distance matrix between points in A=[x1 y1 ... w1] and in B=[x2 y2 ... w2]

             % Copyright (c) 2005-2009 by Kardi Teknomo,  http://people.revoledu.com/kardi/

             %

             % Numbers of rows (represent points) in A and B are not necessarily the same.

             % It can be use for distance-in-a-slice (Spacing) or distance-between-slice (Headway),

             %

             % A and B must contain the same number of columns (represent variables of n dimensions),

             % first column is the X coordinates, second column is the Y coordinates, and so on.

             % The distance matrix is distance between points in A as rows

             % and points in B as columns.

             % example: Spacing= dist(A,A)

             % Headway = dist(A,B), with hA ~= hB or hA=hB

             %          A=[1 2 3; 4 5 6; 2 4 6; 1 2 3]; B=[4 5 1; 6 2 0]

             %          dist(A,B)= [ 4.69   5.83;

             %                       5.00   7.00;

             %                       5.48   7.48;

             %                       4.69   5.83]

             %

             %          dist(B,A)= [ 4.69   5.00     5.48    4.69;

             %                       5.83   7.00     7.48    5.83]

             %%%%%%%%%%%%%%%%%%%%%%%%%%%

             [hA,wA]=size(A);

             [hB,wB]=size(B);

             if wA ~= wB,  error(’ second dimension of A and B must be the same’); end

             for k=1:wA

                  C{k}= repmat(A(:,k),1,hB);

                  D{k}= repmat(B(:,k),1,hA);

             end

             S=zeros(hA,hB);

             for k=1:wA

                  S=S+(C{k}-D{k}’).^2;

             end

             d=sqrt(S);

 

 

For historical purposes, I posted here the previous code that work only for 2 dimensions:

 

function d=dist(A,B)

% DIST return distance matrix between point A=[x1 y1] and B=[x2 y2]

% Number of points in A and B are not necessarily the same.

% It can be use for distance-in-a-slice, distance-between-slice,

% and distance between guest-points-and-the-points

% A and B must contain two columns,

% first column is the X coordinates

% second column is the Y coordinates

% The distance matrix are distance between points in A as row

% and points in B as column.

% example: distance-in-a-slice= dist(A,A)

% distance between guest-point-and-the-point = dist(A,B) , hA=hB

% distance between slices = dist(A,B), with hA ~= hB or hA=hB

% A=[1 2; 3 4; 5 6]; B=[4 5; 6 2; 1 5; 5 8]

% dist(A,B)= [ 4.24 5.00 3.00 7.21;

% 1.41 3.61 2.24 4.47;

% 1.41 4.12 4.12 2.00 ]

 

[hA,wA]=size(A)

[hB,wB]=size(B)

if hA==1& hB==1 

d=sqrt(dot((A-B),(A-B)));

else

C=[ones(1,hB);zeros(1,hB)]

D=flipud(C)

E=[ones(1,hA);zeros(1,hA)]

F=flipud(E)

G=A*C % = repmat(A(:,1),1,hB)

H=A*D % = repmat(A(:,2),1,hB)

I=B*E % = repmat(B(:,1),1,hA)

J=B*F % = repmat(B(:,2),1,hA)

d=sqrt((G-I’).^2+(H-J’).^2)

end

Finding Eigen Value using Microsoft Excel Tutorials

 

Here is frequently asked question from web visitors:
Thanks first of all for putting up this great site! 
I have a question regarding calculation of eigen values for a 3×3 matrix. While I could easily calculate eigen values and vectors for a 2×2 matrix (because after you get one value, ou can simply subtract it from the total variance to get the other), I am having some trouble calculating the same for a 3×3 where we would have three eigen values. I am not able to follow how we reiterate to get the complete set of eigen values.
For the first time, we set value of determinant=0, by changing value of lambda.
Thus we get the first eigen value. How do we get the next two? I tried your eigen value example, but I only found one of the three eigen values, namely 0.9233.The other two, 1 and 27.0767 were never found. Any idea why this happened?
Thank you for your questions. I can
understand your problem of getting other eigen values. Your problem
happen because of the initial value you set is far from the eigen
value. This is normal problem of difference equation. Please realize that
the approximation method I put in this tutorial is very sensitive to initial value. You
may get other eigen values if you can guess the nearest initial value.
Setting different initial value may give the other eigen value if the
eigen value is a basin of attraction.
Secondly, eigen value of non-symmetric matrix may contain complex
eigen value which cannot be solved using this kind of iteration. Other
method for symmetric matrix is to to hold the eigen value that you
have gotten and use MS excel Solver to find the other eigen values.
If you realy want to use Eigen value and Eigen vector for real problem
using MS excel, I suggest you to install free Add Ins library
matrix.xla from
http://digilander.libero.it/foxes/SoftwareDownload.htm. Its great
program that you may use many matrix functions.

Here is a typical frequently asked question from web visitors:

“I have a question regarding calculation of eigen values for a 3×3 matrix. While I could easily calculate eigen values and vectors for a 2×2 matrix (because after you get one value, ou can simply subtract it from the total variance to get the other), I am having some trouble calculating the same for a 3×3 where we would have three eigen values. I am not able to follow how we reiterate to get the complete set of eigen values.For the first time, we set value of determinant=0, by changing value of lambda. Thus we get the first eigen value. How do we get the next two? I tried your eigen value example, but I only found one of the three eigen values, namely 0.9233.The other two, 1 and 27.0767 were never found. Any idea why this happened?”

 I can understand your problem of getting other eigen values. Your problem happen because of the initial value you set is far from the eigen value. This is normal problem of difference equation. Please realize that the approximation method I put in this tutorial is very sensitive to initial value. You may get other eigen values if you can guess the nearest initial value. Setting different initial value may give the other eigen value if the eigen value is a basin of attraction.

Secondly, eigen value of non-symmetric matrix may contain complex eigen value which cannot be solved using this kind of iteration. Other method for symmetric matrix is to to hold the eigen value that you have gotten and use MS excel Solver to find the other eigen values.

If you realy want to use Eigen value and Eigen vector for real problem using MS excel, I suggest you to install free Add Ins library matrix.xla from http://digilander.libero.it/foxes/SoftwareDownload.htm. It is a great program that you may use many matrix functions.