Unsupervised Connectionist Learning

• Purposes of unsupervised learning: dimensionality reduction, data compression, clustering, regularity detection
• Auto-association
  • Patterns are associated with themselves
  • Purposes: dimensionality reduction, data compression, (pattern completion)
  • Implementation: Hopfield nets (pattern completion only), other constraint satisfaction (settling) networks with hidden layers, feedforward nets (can be trained with back-propagation)
• Competitive learning
  • Hidden-layer units compete to classify input patterns
  • Purposes: regularity detection, clustering
  • Simple version
    • One or more hidden layers, no output layer
    • Each hidden layer organized in clusters; each cluster is meant to represent one form of exhaustive classification of the input patterns
    • Weights into each hidden unit always sum to 1.0
    • Binary units
    • For each input pattern, the unit with the highest activation in each hidden cluster is the winner
    • Weights into the winner from units that are on are incremented; those from other units are decremented

      

      j the winning unit in a cluster, a_i the activation of the ith input unit, n_active the number of active input units, eta the learning rate.

    • Weights into losing units (in the simple version of the algorithm) are not changed
    • Hidden units tend to represent categories: cluster size affects number (and type) of categories found