Hopfield Networks as Associative Memory

(Documentation of the programs xhfn and whfn)

Contents Introduction Pattern Storage and Retrieval Parameters Copying Download Contact

Introduction

Hopfield networks are a special kind of recurrent neural networks that can be used as associative memory. Associative memory is memory that is addressed through its contents. That is, if a pattern is presented to an associative memory, it returns whether this pattern coincides with a stored pattern. The coincidence need not be perfect, though. An associative memory may also return a stored pattern that is similar to the presented one, so that noisy input can also be recognized.

Hopfield networks are used as associative memory by exploiting the property that they possess stable states, one of which is reached by carrying out the normal computations of a Hopfield network. If the connection weights of the network are determined in such a way that the patterns to be stored become the stable states of the network, a Hopfield network produces for any input pattern a similar stored pattern. Thus noisy patterns can be corrected or distorted patterns can still be recognized.

Details about the structure of a Hopfield network and about how to construct a Hopfield network that stores given patterns can be found in most textbooks on artificial neural networks.

back to the top

Pattern Storage and Retrieval

With the programs xhfn and whfn the storage and retrieval of simple two-dimensional bit patterns in a Hopfield networks can be demonstrated. The main window of the program displays a grid of neurons. The fields of this grid are colored according to the activation of the corresponding neuron: grey corresponds to an activation of 1, white to an activation of -1. The activation of a neuron may be changed by clicking on the corresponding field. In this way a pattern can be entered. The pattern may then be stored by selecting the menu entry "Actions > Store Pattern" (or, in the Unix version, by pressing "t"). What patterns have already been stored can be reviewed by repeatedly selecting "Actions > Next Pattern" (or, in the Unix version, by pressing "n"). Already stored may also be deleted again by selecting "Actions > Delete Pattern" (or, in the Unix version, by pressing "d"), when the pattern to be deleted is shown in the main window.

Once all desired patterns have been stored, the retrieval capacities of the network may be tested. To do so set the neuron activations to a slightly distorted version of the pattern and start the update process by selecting "Actions > Start Update" (or, in the Unix version, by pressing "x"). The activations of the neurons may also be initialized randomly by selecting "Actions > Initialize" (or, in the Unix version, by pressing "i"). The update process then brings the network into a stable state, which (hopefully) is the stored pattern that is similar to the one entered. It may, however, also converge to a spurious stable state that is not similar to any stored pattern.

As an example, let us consider the set of eight characters shown below. These patterns can be found in the file hfnd/ex/numbers.hfn, which may be loaded through "File > Load Network".

In order to test the Hopfield network that stores these patterns, select "Actions > Initialize", which sets the neuron activations randomly. Then start the computations of the network with "Actions > Start Update" and follow how a stored pattern is reconstructed. However, the reconstruction is not always perfect. Due to the method used to store the patterns, the complement of the stored patterns are also stable states, so that sometimes such a complement is reached. In addition there are some spurious stable states that do not correspond (exactly) to stored patterns.

back to the top

Parameters

Dialog to set the neuron grid parameters

If the menu entry Settings > Grid... is selected (or, in the Unix version, if `g' for `grid' is pressed), a dialog box is opened in which the grid parametersmay be entered (this dialog box is shown below).

In the first two input fields the width and the height of the Hopfield network may be entered. The third field allows a specification of the size of the square that is used to visualize the activation of a neuron.

Dialog to set the update parameters

If the menu entry Settings > Parameters... is selected (or, in the Unix version, if `p' for `parameters' is pressed), a dialog box is opened in which the update parameters of the Hopfield network can be specified (this dialog box is shown below).

The update mode is either "single neuron", which means that the Hopfield network is redisplayed after each update of a neuron, or "all neurons", which means that the Hopfield network is redisplayed only after all neurons have been updated. The delay value controls the amount of time that has to pass between to redisplays of the Hopfield network, so that the computations can be followed conveniently.

Loading and saving a Hopfield network

A Hopfield network (or rather the set of stored patterns) can be saved to a file and reloaded later. To do so one has to select the menu entries File > Save Network... or File > Load Network..., respectively. (Alternatively, in the Unix version, one may press `s' for `save' and `l' for `load'.) The file selector box that is opened, if one of these menu entries are selected, is shown below. (In the Windows version the standard Windows file selector box is used.)

The edit field below 'Directory/Filter' shows the current path, the list box below 'Files' the files in the current directory matching the current file name pattern. (A file name pattern, which may contain the usual wildcards '*' and '?', may be added to the path. When the file name pattern has been changed, pressing the 'Update' button updates the file list.) The line below 'Selection' shows the currently selected file. It may be chosen from the file list or typed in.

back to the top

Copying

Both programs, xhfn and whfn, are free software; you can redistribute them and/or modify them under the terms of the GNU General Public License as published by the Free Software Foundation.

Both programs are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

back to the top

Download

Download pagewith most recent version.

back to the top

Contact

back to the top