Best way to compute (X'X)^-1

Jun 20, 2011 at 5:50 PM

Hi,

I need to calculate (X'X)^-1 in order to calculate the standard errors in a regression analysis.

What's the most efficient way to calculate this using Math.Net?

Thanks,

Andrew

 

Jun 20, 2011 at 6:42 PM

Hi,

You could do this: x.TransposeThisAndMultiply().Inverse().

TransposeThisAndMultiply is more efficient than x.Transpose().Multiply(x).

Regards,
Marcus 

Jun 21, 2011 at 10:53 AM
Edited Jun 21, 2011 at 10:57 AM

Great, thanks.

Am I right that if I change X to the R of a QR decomposition first then that will make it more efficient too?

Something like:

(X'X)^-1

=X^-1.X'^-1

=((QR)^-1).((QR)'^-1)

=(R^-1.Q^-1).((R'Q')^-1)

=(R^-1).(Q^-1).(Q'^-1).(R'^-1)

=(R^-1)(R'^-1) because (Q^-1).(Q'^-1)=(Q'Q)^-1 = I^-1 = I

=(R'R)^-1

This assumes that it is more efficient to invert R than X - is that right? (Assuming X is a dense matrix)

 

Thanks for your help,

Andrew

 

PS Just tried x.transposethisandmultiply() in VB.Net and it produces an error saying that it doesn't accept that number of arguments (i.e., 0). It works with x.transposethisandmultiply(x) though - I don't know if this is a bug or not.

Jun 21, 2011 at 12:35 PM

Hi Andrew,

>Am I right that if I change X to the R of a QR decomposition first then that will make it more efficient too?

I'm not sure, maybe someone else can comment. In the past, I've used QR Solve to compute the betas, then calculated yhat, and then finally errors using y-yhat.

>S Just tried x.transposethisandmultiply() in

oops, that should have been S Just tried x.TransposeThisandMultiply(x).

Jun 21, 2011 at 2:33 PM

Hi Marcus,

Thanks for that. I am using QR.Solve to calculate the Betas but I need the (X'X)^-1 in order to calculate the standard errors of the regression and hence the t-stats, p-values, etc.

Cheers,

Andrew

 

Jun 21, 2011 at 3:32 PM
Edited Jun 21, 2011 at 3:33 PM

>calculate the standard errors of the regression and hence the t-stats, p-values, etc.

doh, for some reason I thought you only needed the errors. From Numerical Methods in Statistics:

(X'X)^-1 = R^-1 * R^-T  = inverse(R) * transpose(inverse(R))

So using your already computed QR (used for calculating beta)

rinverse = qr.R.Inverse();
xxt = rinverse.TransposeAndMultiply(rinverse).

Jun 21, 2011 at 3:44 PM

So if my code is:

 

Betas = Xmatrix.QR.Solve(YVector)

rinverse = qr.R.Inverse();
xxt = rinverse.TransposeAndMultiply(rinverse)

 

Does the program "know" that the R has already been computed or do i have to do something else?

 

Jun 21, 2011 at 6:57 PM

>Does the program "know" that the R has already been computed or do i have to do something else?

Xmatrix.QR() will do the decomposition, so R will exist at that point.

Jun 22, 2011 at 9:41 AM
Edited Jun 22, 2011 at 10:05 AM

Just tried it - doesn't work - I think because R isn't a square matrix.

So it needs to be

Betas = Xmatrix.QR.Solve(YVector)

RMatrix=XMatrix.QR.R
XXT=RMatrix.TransposeThisAndMultiply(RMatrix).Inverse()

which does work!

Thanks for all your help.