Alexnet Architecture (2012)

 

 

1. Deep Architecture 

5 convolutional +3 fully connected with 60 million parameters

 

2. ReLU Activation

First major CNN to use ReLU instead tanh/sigmoid. 

 

3. Dropout Regularization

50% dropout in FC layers to prevent overfitting

 

4. GPU Acceleration

Trained on two GTX 580 GPUs

 

5. Data Augmentation

Image translations, horizontal reflections, and PCA color augmentation.

 

6. Local Response Normalization

Normalization technique applied after ReLU

(later replaced by Batch Normalization)