Math.NET Numericshttp://mathnetnumerics.codeplex.com/project/feeds/rssMath.NET Numerics is an open source numerical library for the .NET Framework and Mono.New Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">@cuda, Great, I am checking that out, thanks<br />
</div>amiranonWed, 22 Apr 2015 06:11:43 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150422061143ANew Post: UserEvd(Complex32 3x3) gives wrong eigenvectors?http://mathnetnumerics.codeplex.com/discussions/263117<div style="line-height: normal;">Hi guys,
<br />
<br />
I have stumbled into similar problem described in this discussion about eigenvectors mismatch between mathnet and matlab.
<br />
<br />
Has there been any update on this?
<br />
<br />
regards.<br />
</div>deepakdnathTue, 21 Apr 2015 09:26:12 GMTNew Post: UserEvd(Complex32 3x3) gives wrong eigenvectors? 20150421092612ANew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">doh, there was a problem with my test. The accuracy isn't that bad. <br />
first take is at: <a href="https://github.com/cuda/mathnet-numerics/tree/moving_stats" rel="nofollow">https://github.com/cuda/mathnet-numerics/tree/moving_stats</a> . see the MovingStatistics class<br />
supports: mean, variance, min, and max. need to add skewness and kurtosis<br />
<br />
Lets move the discussion to github.<br />
</div>cudaMon, 20 Apr 2015 10:13:56 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150420101356ANew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">@amiranon, what I was suggesting was not the "Naïve algorithm," but "Computing shifted data" for each window. I'll push my poping/pushing branch to github tomorrow. It started to lose accuracy after the third update. But again, I don't think this is the way to go. Skewness and kurtosis are going to be even a bigger problem.<br />
</div>cudaMon, 20 Apr 2015 08:34:47 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150420083447ANew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">@cuda, thanks for your reply. according to this page - <a href="http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Na.C3.AFve_algorithm" rel="nofollow">http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Na.C3.AFve_algorithm</a> re-calculating is called the "Naïve algorithm".
<br />
<br />
and this method - <a href="http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data" rel="nofollow">http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data</a> should be pretty accurate.
<br />
<br />
the difference between this approach (Computing shifted data) and regular calculation of std. dev (Naïve algorithm) results should be around 0.000000000001.
<br />
<br />
the only thing need to be added to the library is the Pop() method (or Remove()) so we could use it like a queue for calculating the desired window<br />
</div>amiranonMon, 20 Apr 2015 08:13:42 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150420081342ANew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">There is a similar request up on GitHub - <a href="https://github.com/mathnet/mathnet-numerics/issues/264" rel="nofollow">https://github.com/mathnet/mathnet-numerics/issues/264</a> .
<br />
I tinkered around with this a bit and found that the std dev got inaccurate pretty quickly doing something similar to the links above (of course I could have screwed things up). My opinion is that we re-compute the statistics over each window. It will be slower, but more accurate. There is overlap with MathNet.Filtering and the question is how do we handle that. Perhaps we should move the discussion over to GitHub, and we can decided on the best approach. <br />
</div>cudaMon, 20 Apr 2015 05:46:20 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150420054620ANew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">@bdodson thanks for your reply, it is actually expensive if running this for each user, and window of 5 was just an example, it may be a window of 100.
<br />
<br />
moving standard deviation / rolling standard deviation / sliding window is well known term. I wish Math.Net will include this method in the library, also with tests so I won't need to implement on my own and leave it to the pros.
<br />
<br />
ref:
<br />
<a href="http://matlabtricks.com/post-20/calculate-standard-deviation-case-of-sliding-window" rel="nofollow">http://matlabtricks.com/post-20/calculate-standard-deviation-case-of-sliding-window</a>
<br />
<br />
<a href="http://jonisalonen.com/2014/efficient-and-accurate-rolling-standard-deviation/" rel="nofollow">http://jonisalonen.com/2014/efficient-and-accurate-rolling-standard-deviation/</a><br />
</div>amiranonSun, 19 Apr 2015 11:04:03 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150419110403ANew Post: Numerics FFT result interpretationhttp://mathnetnumerics.codeplex.com/discussions/631552<div style="line-height: normal;">You actually have two peaks of the same magnitude, one at index 100 and one at index 900, hence 200 Hz (100 * 2 Hz).
<br />
<br />
This is expected since for real-valued signals the FT will be real-even and imagniary-odd, so there is a symmetry between positive and negative frequencies. Indeed, you'll be only interested in the positive frequencies at indices 0-500, which represent the frequencies 0-1 kHz in steps of 2 Hz (fs/N) - which is all you can represent with a 2 kHz sampling rate.
<br />
<br />
Be aware that you need to look at the magnitude of the frequency space values, not only their real value. In fact your input sample signal is odd, so the frequency space is imaginary only (the real values are all zero).<br />
</div>cdrnetSun, 19 Apr 2015 06:20:43 GMTNew Post: Numerics FFT result interpretation 20150419062043ANew Post: Numerics FFT result interpretationhttp://mathnetnumerics.codeplex.com/discussions/631552<div style="line-height: normal;">I am new to Math .net numerics and still trying to understand the FFT library. Based on the some examples, I've written following code:<br />
<pre><code> Complex[] samples = new Complex[1000];
double[] s = Generate.Sinusoidal(1000, 2000.0, 200.0, 10.0);
for (int i = 0; i < 1000; i++)
samples[i] = new Complex (s[i],0);
for (int i = 0; i < samples.Length; i++)
{
chart1.Series["Series1"].Points.AddXY
(i,samples[i].Real);
}
Fourier.Forward(samples,FourierOptions.Matlab);
for (int i = 0; i < samples.Length; i++)
{
chart2.Series["Series1"].Points.AddXY
(i, samples[i].Real);
}</code></pre>
<img src="http://i.imgur.com/YjSCa1a.png" alt="Image" />
<br />
<br />
based on my understanding of FFT, i should get impulse at 200Hz and frequency resolution should be fs/N.
<br />
But the peak is coming at 900th sample, which i'm not sure pointing to 200Hz. If my interpretation is correct?
<br />
Any help would be appreciated. Thanks in advance.<br />
</div>ragdarbariSat, 18 Apr 2015 21:22:47 GMTNew Post: Numerics FFT result interpretation 20150418092247PNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">I (as an outside observer) can't find anything like this in the code. In principle you could write a Pop() function as long as you have the x value for the item you're removing by undoing what Push does. This doesn't work for the Minimum and Maximum properties of course (I just noticed that the RunningStatistics.Combine method doesn't handle those either even though it could).
<br />
<br />
Either way, someone would have to keep a rolling queue. If you're just tracking a window of 5 samples, it's actually not expensive at all to just recompute the statistics again with each new value (it's like 20 floating point ops I'd guess compared to maybe 5 to keep a running tally).<br />
</div>bdodsonThu, 16 Apr 2015 00:56:37 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150416125637ANew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples?http://mathnetnumerics.codeplex.com/discussions/622048<div style="line-height: normal;">RunningStatistics: Is there a way to track only a window of the last 5 samples?
<br />
<br />
MathNet.Numerics.Statistics has only Push() method, there is no Pop() method.
<br />
<br />
Let's say my window size is set to 5, and I've already sampled 5 values using Push(), I would want at that point that when I sample the next (sixth) value, it would remove the first value.
<br />
<br />
Because there is no method for removing a sample from the sampled values in a running/accumulating standard deviation, I am not able to track only the last window of the whole sampled values.
<br />
<br />
that way I could use my own logic for a fixed-sized queue and would be able to keep tracking the mean, variance and std. deviation without recalculating the whole last window of samples every time I sample a new value.
<br />
<br />
If there is no "Pop" method (to remove the first value), then is there already a feature like WindowedStandardDeviation when using RunningStatistics that I'm missing?<br />
</div>amiranonSun, 12 Apr 2015 07:18:39 GMTNew Post: RunningStatistics: Is there a way to track only a window of the last 5 samples? 20150412071839ANew Post: Integrating SIMD support + Spatial codehttp://mathnetnumerics.codeplex.com/discussions/622039<div style="line-height: normal;">I was thinking of starting to integrate some SIMD support (using System.Numerics Vector<T> code) into Math.Net Numerics.
<br />
<br />
Probably the first thing I would try to do is vectorize things like DenseVector.DoAdd, DoNegate, DoSubtract, scalar multiply, dot product, etc.
<br />
<br />
I was thinking of just unrolling the loops by Vector<T>.Count and doing the actual operations using Vector<T>.
<br />
<br />
I could probably do the simple Matrix versions too (like add and subtract and other element-wise operations).
<br />
<br />
Would that be helpful? Is someone also working on that already?
<br />
<br />
<br />
I also have some spatial code that might be interesting to integrate into Math.Net - I have code for Euclidean and Projective Transforms (it uses System.Numerics). I also have an implementation of kd-tree k nearest neighbors translated from <a href="https://github.com/ethz-asl/libnabo/" rel="nofollow">https://github.com/ethz-asl/libnabo/</a> if there's interest in that (I used DenseColumnMajorMatrixStorage to represent the point clouds and query points).<br />
</div>bdodsonSun, 12 Apr 2015 02:14:25 GMTNew Post: Integrating SIMD support + Spatial code 20150412021425ANew Post: SubMatrix copyhttp://mathnetnumerics.codeplex.com/discussions/588460<div style="line-height: normal;">No concrete plans. But FFT is high on the list to work on, mainly to bring in native provider support.<br />
</div>cdrnetMon, 23 Mar 2015 19:54:24 GMTNew Post: SubMatrix copy 20150323075424PNew Post: SubMatrix copyhttp://mathnetnumerics.codeplex.com/discussions/588460<div style="line-height: normal;">I would prefer to keep it in a 2D array but maybe I should rethink this. I was not aware that DenseMatrix.OfArray(a) creates a new copy of the data and stores it in a different format.
<br />
<br />
BTW, are ther any plans to add 2D FFT support?
<br />
<br />
Thanks<br />
</div>jfraschillaMon, 23 Mar 2015 14:27:35 GMTNew Post: SubMatrix copy 20150323022735PNew Post: SubMatrix copyhttp://mathnetnumerics.codeplex.com/discussions/588460<div style="line-height: normal;">The matrix storage can also copy directly from one sub-matrix to another sub-matrix (<a href="http://numerics.mathdotnet.com/api/MathNet.Numerics.LinearAlgebra.Storage/MatrixStorage%601.htm#CopySubMatrixTo" rel="nofollow">Storage.CopySubMatrixTo</a>), so you could copy directly from A to B without extracting the 4 matrices.
<br />
<br />
It actually does use BlockCopy internally if applicable, although this does not help that much in case of sub-matrices. However, in this case you'll loose a lot already by the conversion from 2D arrays to matrices and back to 2D arrays, since dense matrices use a single (1D) column-major array internally. I assume the 2D arrays are a requirement in your scenario?<br />
</div>cdrnetSun, 22 Mar 2015 18:00:39 GMTNew Post: SubMatrix copy 20150322060039PSource code checked in, #9771f1bdd124d2688296bf08b849766e6a5b8826http://mathnetnumerics.codeplex.com/SourceControl/changeset/9771f1bdd124d2688296bf08b849766e6a5b8826Release v3.6.0 Christoph RueggSun, 22 Mar 2015 13:31:58 GMTSource code checked in, #9771f1bdd124d2688296bf08b849766e6a5b8826 20150322013158PSource code checked in, #a6878cc7d1bfa9befe0a5c4ca39325436ad215a1http://mathnetnumerics.codeplex.com/SourceControl/changeset/a6878cc7d1bfa9befe0a5c4ca39325436ad215a1Distributions: ChiSquared.CDF test fix for more robust implementation #288 Christoph RueggSun, 22 Mar 2015 12:45:00 GMTSource code checked in, #a6878cc7d1bfa9befe0a5c4ca39325436ad215a1 20150322124500PSource code checked in, #4814200541623dea5c45745971a8812e15845ae4http://mathnetnumerics.codeplex.com/SourceControl/changeset/4814200541623dea5c45745971a8812e15845ae4Merge remote-tracking branch 'bbieniek/master' Christoph RueggSun, 22 Mar 2015 12:10:26 GMTSource code checked in, #4814200541623dea5c45745971a8812e15845ae4 20150322121026PSource code checked in, #9dbf23a9283dcb6049271b38447ca77d259068dbhttp://mathnetnumerics.codeplex.com/SourceControl/changeset/9dbf23a9283dcb6049271b38447ca77d259068dbTests: port vector storage combinator tests to matrices Christoph RueggSun, 22 Mar 2015 12:07:23 GMTSource code checked in, #9dbf23a9283dcb6049271b38447ca77d259068db 20150322120723PSource code checked in, #36b568c5f12371a587829c01cb9b1446728d13cfhttp://mathnetnumerics.codeplex.com/SourceControl/changeset/36b568c5f12371a587829c01cb9b1446728d13cfTests: cleanup, simplifications Christoph RueggSun, 22 Mar 2015 11:08:26 GMTSource code checked in, #36b568c5f12371a587829c01cb9b1446728d13cf 20150322110826A