DYPOF2: Neural Network Design Based on Symmetric Potential Radial Basis Functions 

Abstract

The Radial Basis Function Neural Networks (RBFNs) correspond to a particular class of function approximators, which can be trained using a set of training patterns. The learning strategy used in RBFNs consists of approximating an unknown function with a linear combination of non-linear functions called basis functions. The latter have radial symmetry with respect to a center. The universal approximation capabilities of RBFNs guarantee the existence of an approximator that can achieve an approximation with every degree of precision by having a high, but finite number of units. The result, however, does not suggest any direct method for constructing it, meaning that it is not always possible to find the best approximation within a specified class of approximators even when the analytical expression of the function is given.

We propose to design: a) A new strategy of shape-adaptive radial basis functions based on symmetric potential functions; b) The topology of DYPOF2 (DYnamic POtential Functions) neural network (NN), which is based on radial basis functions NN and symmetric potential functions with a two-stage training procedure. The learning part of training includes static (fixed number n of radial basis functions) and dynamic stages (the parameters governing the basis functions whish correspond to hidden units are automatically determined by using unsupervised methods). The dynamic stage leads to neural network structural changes and thus allows us to control the model complexity through the selection of RBFs and number of hidden units.

A fundamental component in building DYPOF2 is a potential function entity (PFE), designed to generate a respective decision potential function. The desirable shape of the potential field characterizing the distribution of training set is synthesized by adjusting the weights as well as the parameter vectors of cumulative potential functions generated by the PFEs. The automatic adjustment of the minimum necessary number of hidden units – learning adjustment units (LAU) - for a given set of teaching patterns provides the network with a capability of performing dynamic adaptation and self-organization.

We will investigate the dependence of our method on these parameters and apply it to several artificial and benchmark data sets in order to study the power of the PFEs in generating classification solutions for various shapes of teaching patterns that are robust with respect to noise in the data.

Project Outcomes

• Development of new strategy of shape-adaptive radial basis functions based on potential functions and optimization procedure for positioning of the centers during the learning process.

• Development of learning algorithm, which includes static (fixed number n of radial basis functions) and dynamic (during the computation is able to add or delete one or more radial basis functions) phases.

• Development of a three-layer RBF neural network architecture with symmetric potential functions and modified competitive Hebbian learning in order to maintain the neighborhood topology.

• Analysis of how the shape of the classes changes with dimension of the input data, choice of symmetric potential function and the number of centers.

• Comparison of results with some standard RBF network models. Several benchmark data sets will be considered to show the classification performance on the training and test sets achieved by the proposed approach and some other neural network models.