As noted above clustering the factor space allows

As noted above, clustering the factor space allows to create a representative sample containing the training examples with the most unique sets of attributes for training an MLP. A similar approach involving the use of self-organizing Kohonen maps for clustering is mentioned, for example, in Ref. [8]. The stage of factor space formation is crucial when training the neural network of the MLP type by the EBP algorithm. The following requirements are imposed on the factor space:
The third requirement is imposed in order to obtain the given accuracy of the neural network training in a finite number of steps. Ref. [9] presents the relationship between the training error and the number of free parameters W (of the architecture of the neural network) and the number of training examples N: where ε is the tolerable accuracy of the training error and O(…) is the order of the quantity in brackets. To increase the likelihood of adequate training of the MLP, the factor space was divided into three sets: the training, the testing and the validating ones [10]. The first set was used for adjusting the free parameters of the neural network, the second for independently testing the already trained neural network, and the third for avoiding the overtraining effect, which consisted in memorizing instead of generalizing the training set. The NNtool Box from the MatLab package uses 80% of randomly selected vectors from the factor space for training. This partition does not seem optimal because there is a very low probability of choosing the vectors with unique combinations of attributes, i.e., making such a partition for which the S63845 of the training set would be maximal and equal to log2 ( is the size of the training set). In order to increase the entropy of the training set, it is proposed to perform a cluster analysis [11] of the factor space, dividing it into the training, the testing and the validation sets for forming a representative sample. To apply the cluster analysis algorithms effectively, it is very important to determine the number of prototypes correctly. Kohonen\'s self-organizing maps should be considered one of the most reliable clustering methods [6]. The number of prototypes should be specified to perform clustering using these maps, but the network is capable of independently determining cluster centers, as it is self-organized and no teacher is needed for the training. Additionally, the implementation of Kohonen\'s self-organizing maps is simple, and receiving a response after the data has passed through the map\'s layers is guaranteed. Input data was generated for the experiment, forming a factor space described by four input (x1, x2, x3, x4) and one output (y) parameters. The relationship between these parameters is given by the function
Problem setting Let be a factor space,where , , M is the number of vectors in the factor space. Using Kohonen\'s self-organizing maps, we need to find such a partition of the factor space into three sets (the teaching T, the validation V and the testing E), for which the condition is fulfilled. Here H(T) and H0(T) are the entropy values of the training set using clustering and for a random partition of the factor space into the representative sample, respectively; is the maximal entropy of this set ( is the size of the training set comprising 80% of the factor space).
Kohonen\'s neural network description Ref. [9] notes that it is advisable to arrange the CEs in the form of a two-dimensional lattice, because this topology ensures that each neuron has a lot of neighbors. This arrangement determines which elements will be adjusted within a radius of the winning CE. The set of adjustable CEs is specified by the norm selected in the weight space; this norm corresponds to the geometry of the neighborhood of the selected radius. In the simplest case a CE is equal to unity (only the weights of the winning CE are adjusted). The distribution layer (DL) in Fig. 1 corresponds to the input one, and the Kohonen layer (KL) contains CEs forming a rectangle.