This project has moved. For the latest updates, please go here.

Denoising Autoencoders...

May 26, 2014 at 9:03 PM
I am working on building a denoising autoencoder using Numerics.
This is basically a single hidden layer neural net where the target vector is also the input vector.
Prior to feeding the inputs to the net however a percentage of the inputs are set to 0.0 (usually 50%). This can be accomplished by pointwise multiplication by a vector which is 50% 1.0s and 50% 0.0s.
I am planning to use mini-batches so my input vectors will actually be matrices (one input vector per column). To perform this "masking" operation I thought to use pointwise multiplcation by a same dimensional random matrix of 0.0s and 1.0s.
The masking matrix I will generate with the generate random matrix function using the discrete uniform distribution (with min 0.0 and max 1.0).

Any thoughts on whether there is a more efficient way of accomplishing this?
May 26, 2014 at 9:36 PM
Actually it seems that the random matrix function does not like discrete uniform distributions as an argument.
Here is what I worked out:
I basically cycle through the storage of the matrix and use the result of a random number to assign some entries to 0.0
Here is a function in
  Public Function Mask(ByRef Matrix As DenseMatrix, ByVal proportion_zero As Double) As DenseMatrix
        Dim Masked As DenseMatrix = Matrix.Clone
           For r = 0 To Matrix.RowCount - 1
              For c = 0 To Matrix.ColumnCount - 1
               If Rnd() < proportion_zero Then
                    Masked.Storage(r, c) = 0
               End If
               Next c
         Next r
    Return Masked
    End Function
The arguments are Matrix(the matrix to mask), and proportion_zero (the proportion of entries which should be zero).
The function returns the result of the masking.