Percentile obsolete, how to use quantile?

Aug 27, 2013 at 8:12 PM
I see that Percentile is obsolete, to be removed in 3.0. I have been attempting to use Quantile in 2.6.1.30 to get behavior similar to that of percentile for an array of tau values, adapting examples from documentation. I am interested in obtaining 21 points of the cumulative distribution function (0.0, .05, .10,... 0.95,1.0), for data generated from a Monte Carlo simulation.

1) What, if any, is the equivalent use of quantile similar to percentile.Compute(list of values)?
2) in the following code, the value calculated by testQuantile is nowhere near the value of percentile 25

Much appreciate any suggestions, expect that I am missing something obvious.
var rmmSamples=new Normal(10.0,20.0).Samples().Take(10000);  // generate test data
var percentile = new Percentile(rmmSamples);
var percentile25 = percentile.Compute(0.25);
var percentiles = percentile.Compute(new[] {0.0, .25, .5, .75,1.0 });
var testQuantile = SortedArrayStatistics.Quantile(rmmSamples.ToArray(), .25);
        
Thanks in advance. Wonderful library, very useful.

Dick
Cincinnati, Ohio, USA
Coordinator
Aug 27, 2013 at 8:58 PM
Edited Aug 27, 2013 at 9:00 PM
Hi,

2) The result is wrong because the array was not sorted (which is expected by SortedArrayStatistics, other than ArrayStatistics). Once sorted, the result should be close, although not exactly the same because by default the new Quantile uses the R8 quantile definition while Percentile by default uses R6/Nist definition.
Array.Sort(rmmSamples); // use .ToArray on the first line in your sample to get consistent data to compare
SortedArrayStatistics.Quantile(rmmSamples, .25) // R8-quantile
SortedArrayStatistics.QuantileCustom(rmmSamples, .25, QuantileDefinition.Nist) // R6/Nist-quantile
1) Instead of sorting yourself, you can also use Statistics.QuantileFunc (or Statistics.QuantileCustomFunc if you want a quantile definition other than R8) directly and get something closer to the original code:
var quantile = Statistics.QuantileFunc(rmmSamples); // can be an unsorted IEnumerable
quantile(0.25) // evaluate for a single tau
new[] {0.0, .25, .5, .75,1.0 }.Select(quantile) // evaluate for a tau sequence
Thanks,
Christoph
Aug 28, 2013 at 8:11 PM
Thank you very much, it worked exactly as desired. I will post sample code so that others can view it, as soon as I clean some things up.

Dick
Sep 2, 2013 at 4:36 PM
Below is the sample code and results using both percentile and quantile. One question: is there a preferred collection, other than List<double>, in which I should be accumulating values from the Monte Carlo simulation to pass to the statistics?

Dick
Code

           // grab 10000 normal variables
          var normalDist = new Normal(10.0, 20.0);
          List<double> sampleValues = new List<double>();
          // simulate Monte Carlo by adding one variable at a time
          for (int i = 0; i < 10000; i++)
               sampleValues.Add(normalDist.Sample());

          sampleValues.Sort();

          
          var statistics = new DescriptiveStatistics(sampleValues);
          Console.WriteLine(@"Descriptive Statistics Mean {0}", statistics.Mean);
          var percentile = new Percentile(sampleValues);
          var percentile25 = percentile.Compute(0.25);
          Console.WriteLine(@"25th Percentile via Percentile {0}", percentile25);
          IEnumerable<double> tester = new[] { 0.0, .25, .5, .75, 1.0 };
          var percentiles = percentile.Compute(tester); //new[] { 0.0, .25, .5, .75, 1.0 });
          Console.WriteLine("Using percentile:");
          foreach (var item in tester.Zip(percentiles, (a1, b1) => new { a1, b1 }))
            {
                Console.WriteLine("Tau {0:0.00} \t\t Result {1,15:#0.00000000000}", item.a1, item.b1);

            }
          Console.WriteLine("Using quantiles:");
          double test1 = SortedArrayStatistics.Quantile(sampleValues.ToArray(), .25); // R8-quantile
          Console.WriteLine(@"SortedArrayStatistics.Quantile(sampleValues.ToArray(), .25) {0}", test1);

          double test2 = SortedArrayStatistics.QuantileCustom(sampleValues.ToArray(), .25, QuantileDefinition.Nist); // R6/Nist-quantile
          Console.WriteLine(@"sampleValues.ToArray(), .25, QuantileDefinition.Nist  {0}", test2);
         
          var quantile = Statistics.QuantileFunc(sampleValues); // can be an unsorted IEnumerable
          double test4=quantile(0.25); // evaluate for a single tau
          Console.WriteLine(@"quantile=Statistics.QuantileFunc(sampleValues)  quantile(0.25) {0}", test4);
          IEnumerable<double> results; 
           
          results=tester.Select(quantile);
          foreach (var item in tester.Zip(results, (a, b) => new { a, b }))
            {
                 Console.WriteLine("Tau {0:0.00} \t\t Result {1,15:#0.00000000000}",item.a,item.b);
         
            }
     


Results
Descriptive Statistics Mean 9.73486414629633
25th Percentile via Percentile -3.78866441056619
Using percentile:
Tau 0.00         Result -70.39610167743
Tau 0.25         Result  -3.78866441057
Tau 0.50         Result   9.80521165941
Tau 0.75         Result  23.49979823217
Tau 1.00         Result  75.69263872325
Using quantiles:
SortedArrayStatistics.Quantile(sampleValues.ToArray(), .25) -3.78843245134223
sampleValues.ToArray(), .25, QuantileDefinition.Nist  -3.78866441056619
quantile=Statistics.QuantileFunc(sampleValues)  quantile(0.25) -3.78843245134223
Tau 0.00         Result -70.39610167743
Tau 0.25         Result  -3.78843245134
Tau 0.50         Result   9.80521165941
Tau 0.75         Result  23.49827683832
Tau 1.00         Result  75.69263872325