Le Cun and Backpropagation

A couple of posts ago I wrote about an interview with Yann Le Cun. I subsequently found this interesting note in Smith’s Neural Networks for Statistical Modeling:

“Backpropagation is an example of multiple invention. David Parker (1982,1985) and Yann LeCun (1986) working independently of each other and of the Rumelhard group, published similar discoveries. But none of these workers made the first discovery of backpropagation. that honor goes, belatedly, to Paul Werbos, whose 1974 Harvard Ph.D. thesis, _Beyond Regression_ contains the earliest exposition of the techniques involved (Werbos 1974).
“Werbos’ 1974 discovery had gone unappreciated, but Rumelhard, Hinton, and Williams’ 1986 discovery did not. It kindled a firestorm of interest in Neural Networks.”
From Smith, 1993

Backpropagation is an example of multiple invention. David Parker (1982,1985) and Yann LeCun (1986) working independently of each other and of the Rumelhart group, published similar discoveries. But none of these workers made the first discovery of backpropagation. That honor goes, belatedly, to Paul Werbos, whose 1974 Harvard Ph.D. thesis, Beyond Regression, contains the earliest exposition of the techniques involved (Werbos 1974).

Werbos’ 1974 discovery had gone unappreciated, but Rumelhart, Hinton, and Williams’ 1986 discovery did not. It kindled a firestorm of interest in Neural Networks.

I don’t know any details of Le Cun’s discovery in 1986, but I’m curious to look it up.  Note that it was published in the same year as the Rumelhart paper. Here’s the full reference:

Le Cun, Yann. 1986. Learning Processes in a Asymetric Threshold Network. In Disordered Systems and Biological Organization, ed. E. Bienenstock, F. Fogelman Soulie, and G. Weisbuch. Berlin: Springer.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s