Facilitates mixing multiple types of layers side-by-side into a single layer. More...

#include <GNeuralNet.h>

Inheritance diagram for GClasses::GLayerMixed:

Public Member Functions
	GLayerMixed ()
	General-purpose constructor. (You should call addComponent at least twice to mix some layers, after constructing this object.) More...

	GLayerMixed (GDomNode *pNode)
	Deserializing constructor. More...

	~GLayerMixed ()

virtual void	activate ()
	Applies the activation function to the net vector to compute the activation vector in each component, then aggregates all the activation vectors into a single activation for this layer. More...

virtual double *	activation ()
	Returns the activation values from the most recent call to feedForward(). More...

void	addComponent (GNeuralNetLayer *pComponent)
	Adds another component of this layer. In other words, make this layer bigger by adding pComponent to it, as a peer beside the other components in this layer. More...

virtual void	backPropError (GNeuralNetLayer *pUpStreamLayer, size_t inputStart=0)
	Calls backPropError for each component, and adds them up into the upstreams error buffer. (Note that the current implementation of this method may not be compatible with GPU-optimized layers. This method still needs to be audited for compatibility with such layers.) More...

GNeuralNetLayer &	component (size_t i)
	Returns the specified component. More...

virtual void	computeError (const double *pTarget)
	Computes the error terms associated with the output of this layer, given a target vector. (Note that this is the error of the output, not the error of the weights. To obtain the error term for the weights, deactivateError must be called.) More...

virtual void	copyBiasToNet ()
	Calls copyBiasToNet for each component. More...

virtual void	copyWeights (GNeuralNetLayer *pSource)
	Copy the weights from pSource to this layer. (Assumes pSource is the same type of layer.) More...

virtual size_t	countWeights ()
	Returns the number of double-precision elements necessary to serialize the weights of this layer into a vector. More...

virtual void	deactivateError ()
	Copies the error vector into the corresponding buffer for each component, then calls deactivateError for each component. More...

virtual void	diminishWeights (double amount, bool regularizeBiases)
	Calls diminishWeights for each component. More...

virtual void	dropConnect (GRand &rand, double probOfDrop)
	Randomly sets some of the weights to 0. (The dropped weights are restored when you call updateWeightsAndRestoreDroppedOnes.) More...

virtual void	dropOut (GRand &rand, double probOfDrop)
	Calls dropOut for each component. More...

virtual double *	error ()
	Returns a buffer used to store error terms for each unit in this layer. More...

virtual void	feedIn (const double *pIn, size_t inputStart, size_t inputCount)
	Feeds a portion of the inputs through the weights and updates the net for each component. More...

virtual size_t	inputs ()
	Returns the number of values expected to be fed as input into this layer. More...

virtual void	maxNorm (double max)
	Calls maxNorm for each component. More...

virtual size_t	outputs ()
	Returns the number of nodes or units in this layer. More...

virtual void	perturbWeights (GRand &rand, double deviation, size_t start=0, size_t count=INVALID_INDEX)
	Calls perturbWeights for each component. More...

virtual void	renormalizeInput (size_t input, double oldMin, double oldMax, double newMin=0.0, double newMax=1.0)
	Adjusts weights such that values in the new range will result in the same behavior that previously resulted from values in the old range. More...

virtual void	resetWeights (GRand &rand)
	Calls resetWeights for each component. More...

virtual void	resize (size_t inputs, size_t outputs, GRand *pRand=NULL, double deviation=0.03)
	Throws an exception if the specified dimensions would change anything. Also throws an exception if pRand is not NULL. More...

virtual void	scaleUnitIncomingWeights (size_t unit, double scalar)
	Scale weights that feed into the specified unit. More...

virtual void	scaleUnitOutgoingWeights (size_t input, double scalar)
	Scale weights that feed into this layer from the specified input. More...

virtual void	scaleWeights (double factor, bool scaleBiases)
	Calls scaleWeights for each component. More...

virtual GDomNode *	serialize (GDom *pDoc)
	Marshall this layer into a DOM. More...

virtual const char *	type ()
	Returns the type of this layer. More...

virtual double	unitIncomingWeightsL1Norm (size_t unit)
	Compute the L1 norm (sum of absolute values) of weights feeding into the specified unit. More...

virtual double	unitIncomingWeightsL2Norm (size_t unit)
	Compute the L2 norm (sum of squares) of weights feeding into the specified unit. More...

virtual double	unitOutgoingWeightsL1Norm (size_t input)
	Compute the L1 norm (sum of absolute values) of weights feeding into this layer from the specified input. More...

virtual double	unitOutgoingWeightsL2Norm (size_t input)
	Compute the L2 norm (sum of squares) of weights feeding into this layer from the specified input. More...

virtual void	updateBias (double learningRate, double momentum)
	Calls updateBias for each component. More...

virtual void	updateWeights (const double *pUpStreamActivation, size_t inputStart, size_t inputCount, double learningRate, double momentum)
	Calls updateWeights for each component. More...

virtual void	updateWeightsAndRestoreDroppedOnes (const double *pUpStreamActivation, size_t inputStart, size_t inputCount, double learningRate, double momentum)
	This is a special weight update method for use with drop-connect. It updates the weights, and restores the weights that were previously dropped by a call to dropConnect. More...

virtual size_t	vectorToWeights (const double *pVector)
	Deserialize from a vector to the weights in this layer. Return the number of elements consumed. More...

virtual size_t	weightsToVector (double *pOutVector)
	Serialize the weights in this layer into a vector. Return the number of elements written. More...

Public Member Functions inherited from GClasses::GNeuralNetLayer
	GNeuralNetLayer ()

virtual	~GNeuralNetLayer ()

virtual void	copySingleNeuronWeights (size_t source, size_t dest)

void	feedForward (const double *pIn)
	Feeds in the bias and pIn, then computes the activation of this layer. More...

virtual void	feedIn (GNeuralNetLayer *pUpStreamLayer, size_t inputStart)
	Feeds the previous layer's activation into this layer. (Implementations for specialized hardware may override this method to avoid shuttling the previous layer's activation back to host memory.) More...

GMatrix *	feedThrough (const GMatrix &data)
	Feeds a matrix through this layer, one row at-a-time, and returns the resulting transformed matrix. More...

virtual void	getWeightsSingleNeuron (size_t outputNode, double *&weights)
	Gets the weights and bias of a single neuron. More...

virtual void	setWeightsSingleNeuron (size_t outputNode, const double *weights)
	Gets the weights and bias of a single neuron. More...

virtual void	updateWeights (GNeuralNetLayer *pUpStreamLayer, size_t inputStart, double learningRate, double momentum)
	Refines the weights by gradient descent. More...

virtual void	updateWeightsAndRestoreDroppedOnes (GNeuralNetLayer *pUpStreamLayer, size_t inputStart, double learningRate, double momentum)
	Refines the weights by gradient descent. More...

virtual bool	usesGPU ()
	Returns true iff this layer does its computations in parallel on a GPU. More...

Protected Attributes
GMatrix	m_activation

std::vector< GNeuralNetLayer * >	m_components

GMatrix	m_inputError

Additional Inherited Members
Static Public Member Functions inherited from GClasses::GNeuralNetLayer
static GNeuralNetLayer *	deserialize (GDomNode *pNode)
	Unmarshalls the specified DOM node into a layer object. More...

Protected Member Functions inherited from GClasses::GNeuralNetLayer
GDomNode *	baseDomNode (GDom *pDoc)

Detailed Description

Facilitates mixing multiple types of layers side-by-side into a single layer.

Constructor & Destructor Documentation

GClasses::GLayerMixed::GLayerMixed ( )

General-purpose constructor. (You should call addComponent at least twice to mix some layers, after constructing this object.)

GClasses::GLayerMixed::GLayerMixed ( GDomNode * pNode )

Deserializing constructor.

GClasses::GLayerMixed::~GLayerMixed ( )

Member Function Documentation

virtual void GClasses::GLayerMixed::activate ( )

virtual

Applies the activation function to the net vector to compute the activation vector in each component, then aggregates all the activation vectors into a single activation for this layer.

Implements GClasses::GNeuralNetLayer.

virtual double* GClasses::GLayerMixed::activation ( )

inlinevirtual

Returns the activation values from the most recent call to feedForward().

Implements GClasses::GNeuralNetLayer.

void GClasses::GLayerMixed::addComponent ( GNeuralNetLayer * pComponent )

Adds another component of this layer. In other words, make this layer bigger by adding pComponent to it, as a peer beside the other components in this layer.

virtual void GClasses::GLayerMixed::backPropError	(	GNeuralNetLayer *	pUpStreamLayer,
		size_t	inputStart = `0`
	)

virtual

Calls backPropError for each component, and adds them up into the upstreams error buffer. (Note that the current implementation of this method may not be compatible with GPU-optimized layers. This method still needs to be audited for compatibility with such layers.)

Implements GClasses::GNeuralNetLayer.

GNeuralNetLayer& GClasses::GLayerMixed::component ( size_t i )

inline

Returns the specified component.

virtual void GClasses::GLayerMixed::computeError ( const double * pTarget )

virtual

Computes the error terms associated with the output of this layer, given a target vector. (Note that this is the error of the output, not the error of the weights. To obtain the error term for the weights, deactivateError must be called.)

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::copyBiasToNet ( )

virtual

Calls copyBiasToNet for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::copyWeights ( GNeuralNetLayer * pSource )

virtual

Copy the weights from pSource to this layer. (Assumes pSource is the same type of layer.)

Implements GClasses::GNeuralNetLayer.

virtual size_t GClasses::GLayerMixed::countWeights ( )

virtual

Returns the number of double-precision elements necessary to serialize the weights of this layer into a vector.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::deactivateError ( )

virtual

Copies the error vector into the corresponding buffer for each component, then calls deactivateError for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::diminishWeights	(	double	amount,
		bool	regularizeBiases
	)

virtual

Calls diminishWeights for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::dropConnect	(	GRand &	rand,
		double	probOfDrop
	)

virtual

Randomly sets some of the weights to 0. (The dropped weights are restored when you call updateWeightsAndRestoreDroppedOnes.)

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::dropOut	(	GRand &	rand,
		double	probOfDrop
	)

virtual

Calls dropOut for each component.

Implements GClasses::GNeuralNetLayer.

virtual double* GClasses::GLayerMixed::error ( )

inlinevirtual

Returns a buffer used to store error terms for each unit in this layer.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::feedIn	(	const double *	pIn,
		size_t	inputStart,
		size_t	inputCount
	)

virtual

Feeds a portion of the inputs through the weights and updates the net for each component.

Implements GClasses::GNeuralNetLayer.

virtual size_t GClasses::GLayerMixed::inputs ( )

virtual

Returns the number of values expected to be fed as input into this layer.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::maxNorm ( double max )

virtual

Calls maxNorm for each component.

Implements GClasses::GNeuralNetLayer.

virtual size_t GClasses::GLayerMixed::outputs ( )

virtual

Returns the number of nodes or units in this layer.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::perturbWeights	(	GRand &	rand,
		double	deviation,
		size_t	start = `0`,
		size_t	count = `INVALID_INDEX`
	)

virtual

Calls perturbWeights for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::renormalizeInput	(	size_t	input,
		double	oldMin,
		double	oldMax,
		double	newMin = `0.0`,
		double	newMax = `1.0`
	)

virtual

Adjusts weights such that values in the new range will result in the same behavior that previously resulted from values in the old range.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::resetWeights ( GRand & rand )

virtual

Calls resetWeights for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::resize	(	size_t	inputs,
		size_t	outputs,
		GRand *	pRand = `NULL`,
		double	deviation = `0.03`
	)

virtual

Throws an exception if the specified dimensions would change anything. Also throws an exception if pRand is not NULL.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::scaleUnitIncomingWeights	(	size_t	unit,
		double	scalar
	)

virtual

Scale weights that feed into the specified unit.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::scaleUnitOutgoingWeights	(	size_t	input,
		double	scalar
	)

virtual

Scale weights that feed into this layer from the specified input.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::scaleWeights	(	double	factor,
		bool	scaleBiases
	)

virtual

Calls scaleWeights for each component.

Implements GClasses::GNeuralNetLayer.

virtual GDomNode* GClasses::GLayerMixed::serialize ( GDom * pDoc )

virtual

Marshall this layer into a DOM.

Implements GClasses::GNeuralNetLayer.

virtual const char* GClasses::GLayerMixed::type ( )

inlinevirtual

Returns the type of this layer.

Implements GClasses::GNeuralNetLayer.

virtual double GClasses::GLayerMixed::unitIncomingWeightsL1Norm ( size_t unit )

virtual

Compute the L1 norm (sum of absolute values) of weights feeding into the specified unit.

Implements GClasses::GNeuralNetLayer.

virtual double GClasses::GLayerMixed::unitIncomingWeightsL2Norm ( size_t unit )

virtual

Compute the L2 norm (sum of squares) of weights feeding into the specified unit.

Implements GClasses::GNeuralNetLayer.

virtual double GClasses::GLayerMixed::unitOutgoingWeightsL1Norm ( size_t input )

virtual

Compute the L1 norm (sum of absolute values) of weights feeding into this layer from the specified input.

Implements GClasses::GNeuralNetLayer.

virtual double GClasses::GLayerMixed::unitOutgoingWeightsL2Norm ( size_t input )

virtual

Compute the L2 norm (sum of squares) of weights feeding into this layer from the specified input.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::updateBias	(	double	learningRate,
		double	momentum
	)

virtual

Calls updateBias for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::updateWeights	(	const double *	pUpStreamActivation,
		size_t	inputStart,
		size_t	inputCount,
		double	learningRate,
		double	momentum
	)

virtual

Calls updateWeights for each component.

Implements GClasses::GNeuralNetLayer.

virtual void GClasses::GLayerMixed::updateWeightsAndRestoreDroppedOnes	(	const double *	pUpStreamActivation,
		size_t	inputStart,
		size_t	inputCount,
		double	learningRate,
		double	momentum
	)

virtual

This is a special weight update method for use with drop-connect. It updates the weights, and restores the weights that were previously dropped by a call to dropConnect.

Implements GClasses::GNeuralNetLayer.

virtual size_t GClasses::GLayerMixed::vectorToWeights ( const double * pVector )

virtual

Deserialize from a vector to the weights in this layer. Return the number of elements consumed.

Implements GClasses::GNeuralNetLayer.

virtual size_t GClasses::GLayerMixed::weightsToVector ( double * pOutVector )

virtual

Serialize the weights in this layer into a vector. Return the number of elements written.

Implements GClasses::GNeuralNetLayer.

Member Data Documentation

GMatrix GClasses::GLayerMixed::m_activation

protected

std::vector<GNeuralNetLayer*> GClasses::GLayerMixed::m_components

protected

GMatrix GClasses::GLayerMixed::m_inputError

protected

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation