NumModel2LayerNetC - Two layer feed forward neural network.

User Documentation

Centre for Vision, Speech & Signal Processing

Comments:
Simple neural network class. The network constructed is fully connected, feed-forward and 2-layer (i.e. inputs, hidden-layer, output-layer). At present only the back-propagation training algorithm has been implemented. More algorithms will follow.

Parent Classes:

NumModelC

Methods:

NumModel2LayerNetC()

Constructor takes no parameters

NumModel2LayerNetC(const WeightMatrixC & flw,const NetVectorC & flb,const WeightMatrixC & slw,const NetVectorC & slb)

NumModel2LayerNetC(istream & in)

Construct from the input stream

void Train(const NumVVDataSetC & data)

Routine that trains the neural network on the dataset This assumes the dataset has been normalised

void InitialiseRandom()

Initialise the neural network structure with random values

void PresentInput(const VectorC & input,const VectorC & target)

Present a constant input/output pair across the network

void PresentInput(const VectorC & input)

Present an input and corresponding target output to the network

RealT InputError(const VectorC & in,const VectorC & target)

Compute the square sum error of a sample

RealT MeanSquareError(const NumVVDataSetC & in)

Compute the total square sum error of a sampleset

NetVectorC DeltaOutputs()

Compute the deltas for the output layer

NetVectorC DeltaHidden(NetVectorC & deltaOutU)

Compute the deltas for the hidden units

IndexT MaxOutput()

Return the index of the maximum valued current output

RealT PercentageCorrect(const NumVVDataSetC & in)

Calculate the percentage correct on a labelled dataset

void SetHiddenUnits(IntT num)

Set the number of hidden units to use

void SetInitFlag(BooleanT flag)

Set the number of hidden units to use

IntT InputSize() const

Return the input dimensionality

IntT HiddenUnitSize() const

Return the number of hidden units

IntT OutputSize() const

Return the output dimensionality

NetVectorC InputV() const

Look at the current input to the network

WeightMatrixC FirstLayerW() const

Return the first layer weights matrix

NetVectorC FirstLayerB() const

Return the first layer bias

NetVectorC HiddenOutputs() const

Return the outputs of the hidden layer

WeightMatrixC SecondLayerW() const

Return the second layer weight matrix

NetVectorC SecondLayerB() const

Return the second layer bias values

NetVectorC OutputV() const

The output of the network

NetVectorC TargetV() const

The target pattern

void RemoveInputNode(IntT index,BooleanT delhu = TRUE)

IntSArray1dC UnitSelectionAlg(const NumVVDataSetC & dset)

Perform average weight magnitude selection on a dataset. It assumes that the datasets have not been normalised

trainset	dataset of training vector pairs
verifyset	dataset used for cross-validation
AcceptableError	user sets what error they are prepared to accept
StopAt	if error level never reached then stop at this number of inputs

IntSArray1dC AWMS(NumVVDataSetC & trainset,NumVVDataSetC & verifyset,IntT StopAt,RealT AcceptableError)

Perform average weight magnitude selection on a dataset. It assumes that the datasets have not been normalised

trainset	dataset of training vector pairs
verifyset	dataset used for cross-validation
AcceptableError	user sets what error they are prepared to accept
StopAt	if error level never reached then stop at this number of inputs

IntSArray1dC Mao(const NumVVDataSetC & dset)

Inherited members from TrainOptionsC

void SetCycles(IntT cyc)

Set the number of epochs

void SetLearningRate(RealT lr)

Set the learning rate

void SetMomentum(RealT mom)

Set momentum

void SetBounds(RealT low,RealT high)

Set the bounds for random initialisation

void SetPenaltyTerm(BooleanT flag)

Penalise high weights and force low weights to zero

void SetVerbose(BooleanT flag)

Print out training progress to screen

void SetTrainingAlg(TrainMethod trainAlg)

Set the training algorithm ONLINEBP | ONLINEBPM | OFFLINEBP | OFFLINEBPM

IntT Cycles() const

Get the current number of cycles

RealT LearningRate() const

Get the current value of the learning rate

RealT Momentum() const

Get the current value of the momentum

RealT LowBounds() const

Get the low bounds of initialisation

RealT HighBounds() const

Get the higher bounds of initialisation

BooleanT PenaltyTerm() const

Get the status of the penalty term flag

BooleanT Verbose() const

Get the status of the verbose flag

TrainMethod TrainingAlg() const

Get the current training algorithm

#include "amma/Num/NumModel.hh"

NumModelC Copy() const

Makes a deep copy

void Initialise(const NumVVDataSetC & train)

Initialises model to an untrained state

train	data set provided to give model example of data

Must be overloaded in derived class that train incrementally and provides a mechanism for restarting training from scratch. Default behaviour is to do nothing.

void Fit(const NumVVDataSetC & train)

Used to train the model using set of input/output vectors
It should be assumed that the same sort of data will be provided for fitting as was presented to the Initialise function previously. ie same preprocessing should be applied as was appropriate for the initialisation sample set.

#include "amma/Num/NumFunc.hh"

BooleanT IsA(const DPEntityC & pb): Check if an object is a NumFuncC.
NumFuncC Copy() const: Makes a deep copy
NumFuncC & operator=(const NumFuncC & oth): Assignment operator
VectorC Evaluate(const VectorC & X) const: Evaluates function Y=f(X)
DListC<VectorC> Evaluate(const DListC<VectorC> & listX) const: Evaluates function Y=f(X) for a list of X
VectorC operator()(const VectorC & X) const: Evaluates function Y=f(X)
MatrixC Jacobian(const VectorC & X) const: Evaluates Jacobian df(X)/dX
Note that this is not const=0 since the base class will calculate a numerical estimate of the Jacobian using differences if the member function is not overloaded in derived classes to provide an analytical solution.
const StringC GetInfo() const: Gets string describing object
const StringC GetName() const: Gets type name of the object
UIntT SizeX() const: Input vector dimension
UIntT SizeY() const: Output vector dimension
BooleanT Save(ostream & out) const: Writes object to stream, cna be loaded using constructor

#include "amma/DP/Process.hh"

VectorC Apply(const VectorC & dat): Apply operation.
NB. This may become constant in the future, but the situation isn't clear at the moment.
IntT ApplyArray(const SArray1dC<VectorC> & in,SArray1dC<VectorC> & out): Streaming version.
const DPProcessC<VectorC,VectorC> operator=(const DPProcessC<VectorC,VectorC> & in): Assignment operator
const DPProcessC<VectorC,VectorC> Copy() const: Make a copy of this process. Some type defs.
const type_info & InputType() const: Get input type.
const type_info & OutputType() const: Get input type.
BooleanT IsStateless() const: Is operation stateless ?
DPProcessBaseBodyC::ProcTypeT OpType() const: Operation type lossy/lossless.
const DPProcessBaseC & operator=(const DPProcessBaseC & oth): Assignment operator.

#include "amma/DP/Entity.hh"

const DPEntityC & operator=(const DPEntityC & dat): Assignment.
DPEntityBodyC & Body(): Access body.
const DPEntityBodyC & Body() const: Access body.
BooleanT Save(ostream & out) const

#include "amma/RCHandleA.hh"

RCAbstractC Abstract(): Create an abstract handle to this object.
BooleanT IsHandleType(const DT &) const: Is handle of given type ?
void CheckHandleType(const DT & dummy) const: Check handle type. Throw an expception if not.
BooleanT Save(ostream & out) const: Save to ostream.
RCHandleAC<DPEntityBodyC> Copy() const: Creat a deep copy of this object.
protected:

#include "amma/RCHandle.hh"

BooleanT IsValid() const: Check its a valid handle.
BooleanT IsValidObject() const: Check its a valid handle, and that the object is pointed to is also valid.
RCHandleC<DPEntityBodyC> & operator=(const RCHandleC<DPEntityBodyC> & oth): Assignment.
BooleanT operator==(const RCHandleC<DPEntityBodyC> & oth) const: Comparison operator.
BooleanT operator!=(const RCHandleC<DPEntityBodyC> & oth) const: Comparison operator.
UIntT Hash() const: Hash function.x
BooleanT IsConst() const: Is object constant ?
BooleanT IsNotConst() const: Is object not constant ?
void SetConst(void) const: Lock the object.
This is const as a convience though it actual modified the object, often object you wish to ensure are constant already have const set. protected:
void Invalidate(): Turn this into an invalid handle.
DPEntityBodyC & Body(): Direct access to body.
const DPEntityBodyC & Body() const: Constant access to body.

Programmer:Kieron Messer, Documentation by CxxDoc: Tue Mar 20 10:48:08 2001

NumModel2LayerNetC

Two layer feed forward neural network.

include	"amma/Num/NumModel2LayerNet.hh"
User Level:	Default
Library:	NumModelNeuralNet
Example:	NetTest.cc
Section:	Numerical Methods.Multidimensional Models.Implementation Pattern Recognition.Neural Networks
In Scope:	std