About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
Neural Networks
Paper
Improved local learning rule for information maximization and related applications
Abstract
For a neural network comprising feedforward and lateral connections, a local learning rule is proposed that causes the lateral connections to learn directly the inverse of a covariance matrix. In contrast to earlier work, the rule involves just one processing pass through the lateral connections for each input presentation, and consists of a simple anti-Hebbian term. This provides an effective and simple method for online network learning algorithms that implement optimization principles, drawn from statistics or from information or control theory, for which a running estimate of the covariance matrix inverse is useful. An application to infomax learning (mutual information maximization) in the presence of input and output noise is used to illustrate the method. © 2005 Elsevier Ltd. All rights reserved.