|
| GNeuralNet () |
|
| GNeuralNet (GDomNode *pNode, GLearnerLoader &ll) |
| Load from a text-format. More...
|
|
virtual | ~GNeuralNet () |
|
void | addLayer (GNeuralNetLayer *pLayer, size_t position=INVALID_INDEX) |
| Adds pLayer to the network at the specified position. (The default position is at the end in feed-forward order.) Takes ownership of pLayer. If the number of inputs and/or outputs do not align with the previous and/or next layers, then any layers with FLEXIBLE_SIZE inputs or FLEXIBLE_SIZE outputs will be resized to accomodate. If both layers have fixed sizes that do not align, then an exception will be thrown. More...
|
|
void | align (const GNeuralNet &that) |
| Swaps nodes in hidden layers of this neural network to align with those in that neural network, as determined using bipartite matching. (This might be done, for example, before averaging weights together.) More...
|
|
void | autoTune (GMatrix &features, GMatrix &labels) |
| Uses cross-validation to find a set of parameters that works well with the provided data. That is, this method will add a good number of hidden layers, pick a good momentum value, etc. More...
|
|
void | backpropagate (const double *pTarget, size_t startLayer=INVALID_INDEX) |
| This method assumes that the error term is already set at every unit in the output layer. It uses back-propagation to compute the error term at every hidden unit. (It does not update any weights.) More...
|
|
void | backpropagateSingleOutput (size_t outputNode, double target, size_t startLayer=INVALID_INDEX) |
| Backpropagates error from a single output node over all of the hidden layers. (Assumes the error term is already set on the specified output node.) More...
|
|
void | bleedWeightsL1 (double beta) |
| Adjust the magnitudes of the incoming and outgoing connections by amount beta, such that sum of the absolute values of the weights remains constant. If beta is 0, no bleeding will occur. If beta is 1, total bleeding will occur, such that the sum of the absolute values of the input and output weights are the same. Typically, only a small amount of bleeding is desirable, so values close to 0 (like 0.00001) are used. More...
|
|
void | bleedWeightsL2 (double beta) |
| Adjust the magnitudes of the incoming and outgoing connections by amount beta, such that sum-squared-weights remains constant. If beta is 0, no bleeding will occur. If beta is 1, total bleeding will occur, such that the sum of the squares of the input and output weights are the same. Typically, only a small amount of bleeding is desirable, so values close to 0 (like 0.00001) are used. More...
|
|
virtual bool | canImplicitlyHandleMissingFeatures () |
| See the comment for GTransducer::canImplicitlyHandleMissingFeatures. More...
|
|
virtual bool | canImplicitlyHandleNominalFeatures () |
| See the comment for GTransducer::canImplicitlyHandleNominalFeatures. More...
|
|
virtual bool | canImplicitlyHandleNominalLabels () |
| See the comment for GTransducer::canImplicitlyHandleNominalLabels. More...
|
|
virtual void | clear () |
| See the comment for GSupervisedLearner::clear. More...
|
|
GMatrix * | compressFeatures (GMatrix &features) |
| Performs principal component analysis (without reducing dimensionality) on the features to shift the variance of the data to the first few columns. Adjusts the weights on the input layer accordingly, such that the network output remains the same. Returns the transformed feature matrix. More...
|
|
void | containIntrinsics (GMatrix &intrinsics) |
| Finds the column in the intrinsic matrix with the largest deviation, then centers the matrix at the origin and renormalizes so the largest deviation is 1. Also renormalizes the input layer so these changes will have no effect. More...
|
|
void | contractWeights (double factor, bool contractBiases) |
| Contract all the weights in this network by the specified factor. More...
|
|
void | copyPrediction (double *pOut) |
| This method assumes forwardProp has been called. It copies the predicted vector into pOut. More...
|
|
void | copyStructure (GNeuralNet *pOther) |
| Makes this neural network into a deep copy of pOther, including layers, nodes, settings and weights. More...
|
|
void | copyWeights (GNeuralNet *pOther) |
| Copy the weights from pOther. It is assumed (but not checked) that pOther already has the same network structure as this neural network. This method is faster than copyStructure. More...
|
|
size_t | countWeights () const |
| Counts the number of weights in the network. (This value is not cached, so you should cache it rather than frequently call this method.) More...
|
|
size_t | countWeights (size_t layer) const |
|
void | descendGradient (const double *pFeatures, double learningRate, double momentum) |
| This method assumes that the error term is already set for every network unit (by a call to backpropagate). It adjusts weights to descend the gradient of the error surface with respect to the weights. More...
|
|
void | descendGradientSingleOutput (size_t outputNeuron, const double *pFeatures, double learningRate, double momentum) |
| This method assumes that the error term has been set for a single output network unit, and all units that feed into it transitively (by a call to backpropagateSingleOutput). It adjusts weights to descend the gradient of the error surface with respect to the weights. More...
|
|
void | diminishWeights (double amount, bool regularizeBiases=true, size_t startLayer=0, size_t layerCount=INVALID_INDEX) |
| Diminishes all weights in the network by the specified amount. This can be used to implemnet L1 regularization, which promotes sparse representations. That is, it makes many of the weights approach zero. More...
|
|
void | forwardProp (const double *pInputs, size_t maxLayers=INVALID_INDEX) |
| Evaluates a feature vector. (The results will be in the nodes of the output layer.) The maxLayers parameter can limit how far into the network values are propagated. More...
|
|
double | forwardPropSingleOutput (const double *pInputs, size_t output) |
| This is the same as forwardProp, except it only propagates to a single output node. It returns the value that this node outputs. If bypassInputWeights is true, then pInputs is assumed to have the same size as the first layer, and it is fed into the net of this layer, instead of the inputs. More...
|
|
void | gradientOfInputs (double *pOutGradient) |
| This method assumes that the error term is already set for every network unit. It calculates the gradient with respect to the inputs. That is, it points in the direction of changing inputs that makes the error bigger. (Note that this calculation depends on the weights, so be sure to call this method before you call descendGradient. Also, note that descendGradient depends on the input features, so be sure not to update them until after you call descendGradient.) More...
|
|
void | gradientOfInputsSingleOutput (size_t outputNeuron, double *pOutGradient) |
| This method assumes that the error term is already set for every network unit. It calculates the gradient with respect to the inputs. That is, it points in the direction of changing inputs that makes the error bigger. This method assumes that error is computed for only one output neuron, which is specified. (Note that this calculation depends on the weights, so be sure to call this method before you call descendGradientSingleOutput.) Also, note that descendGradientSingleOutput depends on the input features, so be sure not to update them until after you call descendGradientSingleOutput.) More...
|
|
double | improvementThresh () |
| Returns the threshold ratio for improvement. More...
|
|
GMatrix * | internalTraininGMatrix () |
| Gets the internal training data set. More...
|
|
GMatrix * | internalValidationData () |
| Gets the internal validation data set. More...
|
|
void | invertNode (size_t layer, size_t node) |
| Inverts the weights of the specified node, and adjusts the weights in the next layer (if there is one) such that this will have no effect on the output of the network. (Assumes this model is already trained.) More...
|
|
GNeuralNetLayer & | layer (size_t n) |
| Returns a reference to the specified layer. More...
|
|
size_t | layerCount () const |
| Returns the number of layers in this neural network. These include the hidden layers and the output layer. (The input vector does not count as a layer.) More...
|
|
double | learningRate () const |
| Returns the current learning rate. More...
|
|
virtual void | maxNorm (double max) |
| Scales weights if necessary such that the manitude of the weights (not including the bias) feeding into each unit are <= max. More...
|
|
double | momentum () const |
| Returns the current momentum value. More...
|
|
GNeuralNetLayer & | outputLayer () |
| Returns a reference to the last layer. More...
|
|
void | perturbAllWeights (double deviation) |
| Perturbs all weights in the network by a random normal offset with the specified deviation. More...
|
|
virtual void | predict (const double *pIn, double *pOut) |
| See the comment for GSupervisedLearner::predict. More...
|
|
virtual void | predictDistribution (const double *pIn, GPrediction *pOut) |
| See the comment for GSupervisedLearner::predictDistribution. More...
|
|
void | pretrainWithAutoencoders (const GMatrix &features, size_t maxLayers=INVALID_INDEX) |
| Pretrains the network using the method of stacked autoencoders. This method performs the following steps: 1- Start with the first layer. 2- Create an autoencoder using the current layer as the encoder and a temporary layer as the decoder. 3- Train the autoencoder with the features. 4- Discard the decoder. 5- Map the features through the encoder to obtain a set of features for training the next layer, and go to step 2 until all (or maxLayers) layers have been pretrained in this manner. More...
|
|
void | printWeights (std::ostream &stream) |
| Prints weights in a human-readable format. More...
|
|
GNeuralNetLayer * | releaseLayer (size_t index) |
| Drops the layer at the specified index. Returns a pointer to the layer. You are then responsible to delete it. (This doesn't resize the remaining layers to fit with each other, so the caller is responsible to repair any such issues before using the neural network again.) More...
|
|
void | scaleWeights (double factor, bool scaleBiases=true, size_t startLayer=0, size_t layerCount=INVALID_INDEX) |
| Multiplies all weights in the network by the specified factor. This can be used to implement L2 regularization, which prevents weight saturation. More...
|
|
void | scaleWeightsSingleOutput (size_t output, double lambda) |
| Just like scaleWeights, except it only scales the weights in one of the output units. More...
|
|
virtual GDomNode * | serialize (GDom *pDoc) const |
| Saves the model to a text file. More...
|
|
void | setImprovementThresh (double d) |
| Specifies the threshold ratio for improvement that must be made since the last validation check for training to continue. (For example, if the mean squared error at the previous validation check was 50, and the mean squared error at the current validation check is 49, then training will stop if d is > 0.02.) More...
|
|
void | setLearningRate (double d) |
| Set the learning rate. More...
|
|
void | setMomentum (double d) |
| Momentum has the effect of speeding convergence and helping the gradient descent algorithm move past some local minimums. More...
|
|
void | setUseInputBias (bool b) |
| Specify whether to use an input bias. (The default is false.) This feature is used with generative-backpropagation, which adjusts inputs to create latent features. More...
|
|
void | setValidationPortion (double d) |
| Set the portion of the data that will be used for validation. If the value is 0, then all of the data is used for both training and validation. More...
|
|
void | setWeights (const double *pWeights) |
| Sets all the weights from an array of doubles. The number of doubles in the array can be determined by calling countWeights(). More...
|
|
void | setWeights (const double *pWeights, size_t layer) |
|
void | setWindowSize (size_t n) |
| Sets the number of epochs that will be performed before each time the network is tested again with the validation set to determine if we have a better best-set of weights, and whether or not it's achieved the termination condition yet. (An epochs is defined as a single pass through all rows in the training set.) More...
|
|
double | sumSquaredPredictionError (const double *pTarget) |
| This method assumes forwardProp has been called. It computes the sum squared prediction error with the specified target vector. More...
|
|
virtual bool | supportedFeatureRange (double *pOutMin, double *pOutMax) |
| See the comment for GTransducer::supportedFeatureRange. More...
|
|
virtual bool | supportedLabelRange (double *pOutMin, double *pOutMax) |
| See the comment for GTransducer::supportedFeatureRange. More...
|
|
void | swapNodes (size_t layer, size_t a, size_t b) |
| Swaps two nodes in the specified layer. If layer specifies one of the hidden layers, then this will have no net effect on the output of the network. (Assumes this model is already trained.) More...
|
|
virtual void | trainIncremental (const double *pIn, const double *pOut) |
| See the comment for GIncrementalLearner::trainIncremental. More...
|
|
void | trainIncrementalWithDropConnect (const double *pIn, const double *pOut, double probOfDrop) |
| Presents a pattern for training. Applies dropConnect to the weights. Note that when training with dropConnect is complete, you should call scaleWeights(1.0 - probOfDrop, false) to compensate for the scaling effect dropConnect has on the weights. More...
|
|
void | trainIncrementalWithDropout (const double *pIn, const double *pOut, double probOfDrop) |
| Presents a pattern for training. Applies dropout to the activations of hidden layers. Note that when training with dropout is complete, you should call scaleWeights(1.0 - probOfDrop, false, 1) to compensate for the scaling effect dropout has on the weights. More...
|
|
virtual void | trainSparse (GSparseMatrix &features, GMatrix &labels) |
| See the comment for GIncrementalLearner::trainSparse Assumes all attributes are continuous. More...
|
|
size_t | trainWithValidation (const GMatrix &trainFeatures, const GMatrix &trainLabels, const GMatrix &validateFeatures, const GMatrix &validateLabels) |
| Train the network until the termination condition is met. Returns the number of epochs required to train it. More...
|
|
bool | useInputBias () const |
| Returns whether this neural network utilizes an input bias. More...
|
|
void | weights (double *pOutWeights) const |
| Serializes the network weights into an array of doubles. The number of doubles in the array can be determined by calling countWeights(). More...
|
|
void | weights (double *pOutWeights, size_t layer) const |
|
size_t | windowSize () |
| Returns the number of epochs to perform before the validation data is evaluated to see if training should stop. More...
|
|
| GIncrementalLearner () |
| General-purpose constructor. More...
|
|
| GIncrementalLearner (GDomNode *pNode, GLearnerLoader &ll) |
| Deserialization constructor. More...
|
|
virtual | ~GIncrementalLearner () |
| Destructor. More...
|
|
void | beginIncrementalLearning (const GRelation &featureRel, const GRelation &labelRel) |
| You must call this method before you call trainIncremental. More...
|
|
virtual bool | canTrainIncrementally () |
| Returns true. More...
|
|
virtual bool | isFilter () |
| Only the GFilter class should return true to this method. More...
|
|
| GSupervisedLearner () |
| General-purpose constructor. More...
|
|
| GSupervisedLearner (GDomNode *pNode, GLearnerLoader &ll) |
| Deserialization constructor. More...
|
|
virtual | ~GSupervisedLearner () |
| Destructor. More...
|
|
void | basicTest (double minAccuracy1, double minAccuracy2, double deviation=1e-6, bool printAccuracy=false, double warnRange=0.035) |
| This is a helper method used by the unit tests of several model learners. More...
|
|
virtual bool | canGeneralize () |
| Returns true because fully supervised learners have an internal model that allows them to generalize previously unseen rows. More...
|
|
void | confusion (GMatrix &features, GMatrix &labels, std::vector< GMatrix * > &stats) |
| Generates a confusion matrix containing the total counts of the number of times each value was expected and predicted. (Rows represent target values, and columns represent predicted values.) stats should be an empty vector. This method will resize stats to the number of dimensions in the label vector. The caller is responsible to delete all of the matrices that it puts in this vector. For continuous labels, the value will be NULL. More...
|
|
void | precisionRecall (double *pOutPrecision, size_t nPrecisionSize, GMatrix &features, GMatrix &labels, size_t label, size_t nReps) |
| label specifies which output to measure. (It should be 0 if there is only one label dimension.) The measurement will be performed "nReps" times and results averaged together nPrecisionSize specifies the number of points at which the function is sampled pOutPrecision should be an array big enough to hold nPrecisionSize elements for every possible label value. (If the attribute is continuous, it should just be big enough to hold nPrecisionSize elements.) If bLocal is true, it computes the local precision instead of the global precision. More...
|
|
const GRelation & | relFeatures () |
| Returns a reference to the feature relation (meta-data about the input attributes). More...
|
|
const GRelation & | relLabels () |
| Returns a reference to the label relation (meta-data about the output attributes). More...
|
|
double | sumSquaredError (const GMatrix &features, const GMatrix &labels) |
| Computes the sum-squared-error for predicting the labels from the features. For categorical labels, Hamming distance is used. More...
|
|
void | train (const GMatrix &features, const GMatrix &labels) |
| Call this method to train the model. More...
|
|
virtual double | trainAndTest (const GMatrix &trainFeatures, const GMatrix &trainLabels, const GMatrix &testFeatures, const GMatrix &testLabels) |
| Trains and tests this learner. Returns sum-squared-error. More...
|
|
| GTransducer () |
| General-purpose constructor. More...
|
|
| GTransducer (const GTransducer &that) |
| Copy-constructor. Throws an exception to prevent models from being copied by value. More...
|
|
virtual | ~GTransducer () |
|
virtual bool | canImplicitlyHandleContinuousFeatures () |
| Returns true iff this algorithm can implicitly handle continuous features. If it cannot, then the GDiscretize transform will be used to convert continuous features to nominal values before passing them to it. More...
|
|
virtual bool | canImplicitlyHandleContinuousLabels () |
| Returns true iff this algorithm can implicitly handle continuous labels (a.k.a. regression). If it cannot, then the GDiscretize transform will be used during training to convert nominal labels to continuous values, and to convert nominal predictions back to continuous labels. More...
|
|
double | crossValidate (const GMatrix &features, const GMatrix &labels, size_t nFolds, RepValidateCallback pCB=NULL, size_t nRep=0, void *pThis=NULL) |
| Perform n-fold cross validation on pData. Returns sum-squared error. Uses trainAndTest for each fold. pCB is an optional callback method for reporting intermediate stats. It can be NULL if you don't want intermediate reporting. nRep is just the rep number that will be passed to the callback. pThis is just a pointer that will be passed to the callback for you to use however you want. It doesn't affect this method. More...
|
|
GTransducer & | operator= (const GTransducer &other) |
| Throws an exception to prevent models from being copied by value. More...
|
|
GRand & | rand () |
| Returns a reference to the random number generator associated with this object. For example, you could use it to change the random seed, to make this algorithm behave differently. This might be important, for example, in an ensemble of learners. More...
|
|
double | repValidate (const GMatrix &features, const GMatrix &labels, size_t reps, size_t nFolds, RepValidateCallback pCB=NULL, void *pThis=NULL) |
| Perform cross validation "nReps" times and return the average score. pCB is an optional callback method for reporting intermediate stats It can be NULL if you don't want intermediate reporting. pThis is just a pointer that will be passed to the callback for you to use however you want. It doesn't affect this method. More...
|
|
GMatrix * | transduce (const GMatrix &features1, const GMatrix &labels1, const GMatrix &features2) |
| Predicts a set of labels to correspond with features2, such that these labels will be consistent with the patterns exhibited by features1 and labels1. More...
|
|
void | transductiveConfusionMatrix (const GMatrix &trainFeatures, const GMatrix &trainLabels, const GMatrix &testFeatures, const GMatrix &testLabels, std::vector< GMatrix * > &stats) |
| Makes a confusion matrix for a transduction algorithm. More...
|
|