\documentclass[12pt,a4paper]{article}
\usepackage{graphicx}
\pagestyle{empty}
\oddsidemargin 2.1mm
\textwidth 155mm
\topmargin -15mm
\textheight 240mm
\def\act{\mathop{\rm act}\nolimits}
\begin{document}
%-----------------------------------------------------------------------
\noindent
{\bf Artificial Neural Networks and Deep Learning}
\hfill Summer 2017 \\
Christian Borgelt \hfill from 2017.06.27
%-----------------------------------------------------------------------
\vskip3ex
\centerline{\bf Exercise Sheet 5}
%-----------------------------------------------------------------------
\subsubsection*{Exercise 28 \quad\rm Competitive Learning /
Learning Vector Quantization}
\hangindent-52mm\hangafter-5\noindent
\hbox to0pt{\vbox to1.5ex{\noindent\hbox to\textwidth{\hss
\includegraphics{nnex-10}}\vss}\hss}%
Consider the eight training data points that are shown on the right,
which belong to two classes~$A$ (empty circles) and $B$ (filled
circles). This pattern set is to be quantized with the help of two
reference vectors. Which final posiiton will be reached by these two
reference vectors in an ideal case, if
\begin{enumerate}\itemsep0pt\parskip2pt
\item[a)] only the ``attraction rule''
(patterns of the same class attract reference vectors),
\item[b)] both the ``attraction rule''
and the ``repulsion rule''
(patterns of different class repell reference vectors)
\end{enumerate}\hangindent0mm
are used to change the positions of the reference vectors?
(Hint: You need not calculate the steps of the procedure in detail.
The solution can be read directly from the structure of the training
patterns.)
%-----------------------------------------------------------------------
\subsubsection*{Exercise 29 \quad\rm {\tt xlvq/wlvq}:
Competitive Learning / Learning Vector Quantization}
The programs that are available at
{\tt http://www.borgelt.net/lvqd.html} visualize learning vector
quantization for two-dimensional data. (Higher-dimensional data may
be loaded. In this case the program offers the option to select the
two of the input variables that are to be used --- see menu entry
{\tt Settings > Attributes}.) Apply the program to the Iris data
with the input quantities petal length and petal width and let the
program determine three reference vectors! (An input file in the
appropriate format is contained in the source package.) Compare
using the class information (one reference vector per class) with
not using it (three reference vectors)!
%-----------------------------------------------------------------------
\subsubsection*{Exercise 30 \quad\rm
{\tt xsom/wsom}: Self-organizing Maps}
The programs that are available at
{\tt http://www.borgelt.net/somd.html} visualize the development of
a self-organizing map, which is trained with points that are chosen
randomly from a two-dimensional shape. Execute several training runs
with this program! Vary the parameters, especially the shape of the
area, from which the data points are chosen, the learning rate and
the size of the neighborhood. What happens if the learning rate or
the neighborhood radius is reduced too quickly ? (The learning rate
is computed according to the formula $\eta(t) = \eta_0 t^{-\kappa}$.
The values for $\eta_0$ (initial learning rate) and $\kappa$
(learning rate decay exponent) can be entered in the parameter
dialog box. The neighborhood radius is computed with an analogous
formula.)
%-----------------------------------------------------------------------
\newpage
\subsubsection*{Exercise 31 \quad\rm Hopfield Networks: State Graph}
\hangindent-36mm\hangafter-6\noindent
\hbox to0pt{\vbox to1.5ex{\noindent\hbox to\textwidth{\hss
\includegraphics{nnex-11}}\vss}\hss}%
The diagram on the right shows a simple Hopfield network. For this
network, determine the final state(s) that are reached starting from
the initial state $(\act_{u_1}, \act_{u_2}, \act_{u_3}) = (-1,-1,-1)$! \\
(Hint: Use a state transition graph. In this graph, mark the initial
state and the final state(s). Notice that there is no explicit start
neuron and that therefore you have to consider all possible subsequent
states.)
%-----------------------------------------------------------------------
\subsubsection*{Exercise 32 \quad\rm Hopfield-Netze: Energy Function}
Determine the energy function of the Hopfield network of Exercise~31!
With the help of this energy function, compute the energies of the
individual states of the network. Then arrange the states according
to their energy, that is, draw a new state transition graph, in which
the location of the states indicates their energy!
%-----------------------------------------------------------------------
\subsubsection*{Exercise 33 \quad\rm Hopfield Networks:
Pattern Recognition}
We want to store the two patterns $(-1, +1, -1, +1)$ and
$(+1, -1, -1, +1)$ in a Hopfield network with four neurons, that is,
these two patterns are supposed to be the stable states of the network,
which cannot be left, regardless of which neuron is updated.
\begin{enumerate}\itemsep0pt
\item[a)] Compute the connection weights and the threshold values
of the neurons of a Hopfield network that stores the abovementioned
patterns!
\item[b)] How many other patterns can be stored in this network?
\item[c)] Find two more patterns that could be stored in the network
constructed in part~a) without ``forgetting'' the old patterns!
(Of course, in order to actually store these patterns, the
connection weights may have to be modified.)
\end{enumerate}
%-----------------------------------------------------------------------
\subsubsection*{Exercise 34 \quad\rm {\tt xhfn/whfn}:
Pattern Recognition}
The programs that are available at
{\tt http://www.borgelt.net/hfnd.html} visualize pattern recognition
by a Hopfield network. Use this program to store a few patterns in the
represented Hopfield network (for example, the seven digits and one
block symbol used in the lecture, which are also avaliable in the file
{\tt hfnd/ex/numbers.hfn} in the source package). Afterward initialize
the network randomly and observe which of the stored patterns is
recognized. Are only the stored patterns recognized? Which other
patterns are recognized as well/instead? How many patterns can be
stored before reliable recognition breaks down?
%-----------------------------------------------------------------------
\end{document}