artificial Neural Networks
M01_HAYK1399_SE_03_C01.QXD
10/1/08
7:03 PM
Page 47
C H A P T E R
1
Rosenblatt’s Perceptron
ORGANIZATION OF THE CHAPTER
The perceptron occupies a special place in the historical development of neural networks: It was the first algorithmically described neural network. Its invention by
Rosenblatt, a psychologist, inspired engineers, physicists, and mathematicians alike to devote their research effort to different aspects of neural networks in the 1960s and the 1970s. Moreover, it is truly remarkable to find that the perceptron (in its basic form as described in this chapter) is as valid today as it was in 1958 when Rosenblatt’s paper on the perceptron was first published.
The chapter is organized as follows:
1. Section 1.1 expands on the formative years of neural networks, going back to the pioneering work of McCulloch and Pitts in 1943.
2. Section 1.2 describes Rosenblatt’s perceptron in its most basic form. It is followed by
Section 1.3 on the perceptron convergence theorem. This theorem proves convergence of the perceptron as a linearly separable pattern classifier in a finite number time-steps. 3. Section 1.4 establishes the relationship between the perceptron and the Bayes classifier for a Gaussian environment.
4. The experiment presented in Section 1.5 demonstrates the pattern-classification capability of the perceptron.
5. Section 1.6 generalizes the discussion by introducing the perceptron cost function, paving the way for deriving the batch version of the perceptron convergence algorithm. Section 1.7 provides a summary and discussion that conclude the chapter.
1.1
INTRODUCTION
In the formative years of neural networks (1943–1958), several researchers stand out for their pioneering contributions:
• McCulloch and Pitts (1943) for introducing the idea of neural networks as computing machines.
47
M01_HAYK1399_SE_03_C01.QXD
48
9/10/08
Chapter 1
9:24 PM
Page 48
Rosenblatt’s Perceptron
• Hebb
10/1/08
7:03 PM
Page 47
C H A P T E R
1
Rosenblatt’s Perceptron
ORGANIZATION OF THE CHAPTER
The perceptron occupies a special place in the historical development of neural networks: It was the first algorithmically described neural network. Its invention by
Rosenblatt, a psychologist, inspired engineers, physicists, and mathematicians alike to devote their research effort to different aspects of neural networks in the 1960s and the 1970s. Moreover, it is truly remarkable to find that the perceptron (in its basic form as described in this chapter) is as valid today as it was in 1958 when Rosenblatt’s paper on the perceptron was first published.
The chapter is organized as follows:
1. Section 1.1 expands on the formative years of neural networks, going back to the pioneering work of McCulloch and Pitts in 1943.
2. Section 1.2 describes Rosenblatt’s perceptron in its most basic form. It is followed by
Section 1.3 on the perceptron convergence theorem. This theorem proves convergence of the perceptron as a linearly separable pattern classifier in a finite number time-steps. 3. Section 1.4 establishes the relationship between the perceptron and the Bayes classifier for a Gaussian environment.
4. The experiment presented in Section 1.5 demonstrates the pattern-classification capability of the perceptron.
5. Section 1.6 generalizes the discussion by introducing the perceptron cost function, paving the way for deriving the batch version of the perceptron convergence algorithm. Section 1.7 provides a summary and discussion that conclude the chapter.
1.1
INTRODUCTION
In the formative years of neural networks (1943–1958), several researchers stand out for their pioneering contributions:
• McCulloch and Pitts (1943) for introducing the idea of neural networks as computing machines.
47
M01_HAYK1399_SE_03_C01.QXD
48
9/10/08
Chapter 1
9:24 PM
Page 48
Rosenblatt’s Perceptron
• Hebb
References: 1. The network organization of the original version of the perceptron as envisioned by Rosenblatt (1962) has three types of units: sensory units, association units, and response 2. Proof of the perceptron convergence algorithm presented in Section 1.3 follows the classic look of Nilsson (1965).