StarMirror/data-science-ipython-notebooks
1
0
Fork 0
mirror of https://github.com/donnemartin/data-science-ipython-notebooks.git synced 2024-03-22 13:30:56 +08:00
Code Issues Projects Releases Wiki Activity
804c35657d
data-science-ipython-notebooks/deep-learning/keras-tutorial/2.2.2 Supervised Learning - ConvNet HandsOn Part II.ipynb

1006 lines
37 KiB
Python
Raw Normal View History

Add Keras tutorials resources #49 (#51)
2017-08-26 20:25:13 +08:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
Fix #49: Keras tutorial links (#52)
2017-08-26 20:47:27 +08:00
"Credits: Forked from [deep-learning-keras-tensorflow](https://github.com/leriomaggio/deep-learning-keras-tensorflow) by Valerio Maggio"
Add Keras tutorials resources #49 (#51)
2017-08-26 20:25:13 +08:00
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Convolution Nets for MNIST"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Deep Learning models can take quite a bit of time to run, particularly if GPU isn't used. \n",
"\n",
"In the interest of time, you could sample a subset of observations (e.g. $1000$) that are a particular number of your choice (e.g. $6$) and $1000$ observations that aren't that particular number (i.e. $\\neq 6$). \n",
"\n",
"We will build a model using that and see how it performs on the test dataset"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using Theano backend.\n",
"Using gpu device 0: GeForce GTX 760 (CNMeM is enabled with initial size: 90.0% of memory, cuDNN 4007)\n"
]
}
],
"source": [
"#Import the required libraries\n",
"import numpy as np\n",
"np.random.seed(1338)\n",
"\n",
"from keras.datasets import mnist"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"from keras.models import Sequential"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"from keras.layers.core import Dense, Dropout, Activation, Flatten"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"from keras.layers.convolutional import Convolution2D\n",
"from keras.layers.pooling import MaxPooling2D"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"from keras.utils import np_utils\n",
"from keras.optimizers import SGD"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Loading Data"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [],
"source": [
"path_to_dataset = \"euroscipy_2016_dl-keras/data/mnist.pkl.gz\"\n",
"\n",
"#Load the training and testing data\n",
"(X_train, y_train), (X_test, y_test) = mnist.load_data(path_to_dataset)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"X_test_orig = X_test"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Data Preparation"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [],
"source": [
"img_rows, img_cols = 28, 28\n",
"\n",
"X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)\n",
"X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)\n",
"\n",
"X_train = X_train.astype('float32')\n",
"X_test = X_test.astype('float32')\n",
"\n",
"X_train /= 255\n",
"X_test /= 255"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# Seed for reproducibilty\n",
"np.random.seed(1338)\n",
"\n",
"# Test data\n",
"X_test = X_test.copy()\n",
"Y = y_test.copy()\n",
"\n",
"# Converting the output to binary classification(Six=1,Not Six=0)\n",
"Y_test = Y == 6\n",
"Y_test = Y_test.astype(int)\n",
"\n",
"# Selecting the 5918 examples where the output is 6\n",
"X_six = X_train[y_train == 6].copy()\n",
"Y_six = y_train[y_train == 6].copy()\n",
"\n",
"# Selecting the examples where the output is not 6\n",
"X_not_six = X_train[y_train != 6].copy()\n",
"Y_not_six = y_train[y_train != 6].copy()\n",
"\n",
"# Selecting 6000 random examples from the data that \n",
"# only contains the data where the output is not 6\n",
"random_rows = np.random.randint(0,X_six.shape[0],6000)\n",
"X_not_six = X_not_six[random_rows]\n",
"Y_not_six = Y_not_six[random_rows]"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# Appending the data with output as 6 and data with output as <> 6\n",
"X_train = np.append(X_six,X_not_six)\n",
"\n",
"# Reshaping the appended data to appropraite form\n",
"X_train = X_train.reshape(X_six.shape[0] + X_not_six.shape[0], \n",
" 1, img_rows, img_cols)\n",
"\n",
"# Appending the labels and converting the labels to \n",
"# binary classification(Six=1,Not Six=0)\n",
"Y_labels = np.append(Y_six,Y_not_six)\n",
"Y_train = Y_labels == 6 \n",
"Y_train = Y_train.astype(int)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(11918, 1, 28, 28) (11918,) (10000, 1, 28, 28) (10000, 2)\n"
]
}
],
"source": [
"print(X_train.shape, Y_labels.shape, X_test.shape, Y_test.shape)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"# Converting the classes to its binary categorical form\n",
"nb_classes = 2\n",
"Y_train = np_utils.to_categorical(Y_train, nb_classes)\n",
"Y_test = np_utils.to_categorical(Y_test, nb_classes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# A simple CNN"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"#Initializing the values for the convolution neural network\n",
"nb_epoch = 2\n",
"batch_size = 128\n",
"# number of convolutional filters to use\n",
"nb_filters = 32\n",
"# size of pooling area for max pooling\n",
"nb_pool = 2\n",
"# convolution kernel size\n",
"nb_conv = 3\n",
"\n",
"sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Step 1: Model Definition"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"model = Sequential()\n",
"\n",
"model.add(Convolution2D(nb_filters, nb_conv, nb_conv,\n",
" border_mode='valid',\n",
" input_shape=(1, img_rows, img_cols)))\n",
"model.add(Activation('relu'))\n",
"\n",
"model.add(Flatten())\n",
"model.add(Dense(nb_classes))\n",
"model.add(Activation('softmax'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Step 2: Compile"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"model.compile(loss='categorical_crossentropy',\n",
" optimizer='sgd',\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Step 3: Fit"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 11918 samples, validate on 10000 samples\n",
"Epoch 1/2\n",
"11918/11918 [==============================] - 0s - loss: 0.2890 - acc: 0.9326 - val_loss: 0.1251 - val_acc: 0.9722\n",
"Epoch 2/2\n",
"11918/11918 [==============================] - 0s - loss: 0.1341 - acc: 0.9612 - val_loss: 0.1298 - val_acc: 0.9599\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x7f6ccb68f630>"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.fit(X_train, Y_train, batch_size=batch_size, \n",
" nb_epoch=nb_epoch,verbose=1,\n",
" validation_data=(X_test, Y_test))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Step 4: Evaluate"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test score: 0.129807630396\n",
"Test accuracy: 0.9599\n"
]
}
],
"source": [
"# Evaluating the model on the test data \n",
"score, accuracy = model.evaluate(X_test, Y_test, verbose=0)\n",
"print('Test score:', score)\n",
"print('Test accuracy:', accuracy)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Let's plot our model Predictions!"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA50AAABVCAYAAADKf4AOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm4VmW5/z+PgMIJA4UABYckPZoYaJSWeEmWph3MMicQ\nTpAoTimCx711b3NhbGWTECAKKhYdUBQnNAtSK0xIJcejRqeccjghDj9NTAXx+f1x32tP7I17eOf3\n+7mufT3rfdaz1nvfe71reu4pxBgRQgghhBBCCCGywTb5FkAIIYQQQgghROmil04hhBBCCCGEEFkj\nKy+d3bp1WxdCiG3969at27psyJNtpK/0lc7St5iQvtJXOkvfYkL6St9S0rnc9E0J2YjpDCHE9uw3\nhECMMWRcoCwjfVu9XVnp69uWlc7StziQvq3erqz09W3LSmfpWxxI31ZvV1b6+rZFp3O56Zsi91oh\nhBBCCCGEEFlDL51CCCGEEEIIIbJGzl46V6xYwd57781ee+1FbW1trr42b5xyyin07duXL3zhC/kW\nJWeU2zGWvqVNuekL5aez9C1tpG9po+es0j/G0rfEiDFm/M92W8/mzZvjwIED44svvhg3btwYBw8e\nHNeuXRub4ttlRaZs/jXVN8YYH3jggfj444/H/fbbb4t1pahva45xuelbSjpL39LWt7U6S9/i+NM1\nS/qWur4x6jlL16zi1bnc9E3/cmLpXLNmDXvuuSe77bYbXbp04aSTTuLOO+/MxVfnjWHDhrHDDjvk\nW4ycUW7HWPpK31Kj3HSWvtK3lCg3fUHPWaV+jKVv6embk5fOV199lV122aXu84ABA3j11Vdz8dUi\nR5TbMZa+0rfUKDedpa/0LSXKTd9ypNyOsfQtPX1z8tLppuRGhFCU2X5FC5TbMZa+0rfUKDedpa/0\nLSXKTd9ypNyOsfQtPX1z8tI5YMAAXnrppbrPr7zyCjvvvHMuvlrkiHI7xtJX+pYa5aaz9JW+pUS5\n6VuOlNsxlr4lqG8uAmQ/+uijuuDYDz/8MA4ePDj++c9/Lpng2Kb6przwwgtx0KBBza4rNX1bc4zL\nTd9S0ln6lra+rdVZ+hbHn65Z0rfU9U3Rc5auWcWoc7npm/51zsqbbBM6derE3LlzOeKII/j44485\n5ZRT2GeffZodG0KypX25QIgxabWde9SoUaxcuZI333yTXXfdlSlTpjBu3LgtxpWKvq09xuWmL5SG\nztK3tPUFncPSt55S0Fn6lra+oOcsXbPqKQWdy0Hf4G/cGSWEENuzX/NdviTj8mSKlv6R0rd1lJu+\nvi3lpLP0LSykryF9W0+56Sx9Cwvpa0jf1lOMOpebvik5iekUQgghhBBCCFGe6KVTCCGEEEIIIUTW\nyElMp2g7J8TPArALLwNwxcsXA5DsWj/mG/EAAI56bwUAG7pflUMJhRBCCCGKld0BiBMsDrT2Guut\neBzC/oXrvihEsSJLpxBCCCGEEEKIrCFLZ4ERX5sCQG2TMNxab7s16FsdHgPg3aF9AAikQclJ1uQr\nCPolAExdZ/+ke+NyAO4PD+VLogxhlusH4pS6Y1txrq0Js8vk2AohSoyjYdgXm1+16mEAkni7tc/4\nne50YFWSdclEC5yfWHvFTGv3nsTotdcBsGj2aQCMOffaRpss/q9TfZskBwJ2lN0BeCBeCcBMf97q\n4mvPH/Jj4KOcSyVEtqiOG+kULgcgmW994fTcW/OzYuns2rXrayEE2vq33Xa9siFO1pG+0lc6S99i\nQvpKX+ksfYsJ6St9S0nnctM3JSslUxp9QQHXkmkrram10xF942tTqO3b/LqKI6y95TcjOP6JuwGo\n3b/xmL/HGQDMC/9srwiN5cmyvu3mWZudmfE5mzOZ/JCLcFDSod22tpZS1nTunQAw+416Md739s74\nWwAeDH/I6Ffm9Bgfmdh3PmNfGV5uxyzb/MSsIEB7rL4F+5u+OwHgkhEm3j11x/sJoP3nc270HW/f\nNXcXAGafZZaQicEDpNptnf+GNXcPs3bEPO9/rcUtCvb4ZomC1PfWBIBHv/d5AA44aC21Dzc/tOJQ\na2++39oXG6yrbKEkQEHqnEVyo+9w+65VXwPgNj/l0gqBrwJrPmEPe3v7BWCvJ1ycIUmbJcmFviOj\nJccYGk4BYNJS3++vfbcLk47svk3o97wl2dP3aGummefF9RWjAHgpLAHqPQkrXoOf9RkJwCnhp96b\n3n/aTiEc4zh/CsnpjfumpF+ZJJn9LpVMEUIIIYQQQgiRDxTTWQh8JwFgXt8pdV0V37U23GEzEZX3\nLPeOh4FvAxDHmkWhdqGt+ip/tP0wKKvi5psrB1rsSJ2WHbRw5p/vAxB/aJNDc0o0ad5py2cD8GYH\nrjq/nfBVvnb8gwBs06sU/lEnALDxZDv2Nd57RPg6AA9yEx2xdGafo/krZuGcd7b1nHu8xXpNZKd2\n7tMsnHHQIQCsHGG9X+vts7JvJO3cb7YYDsDKWAHAobPXECZe7etatsoWPd0TAJa+a9aD7YJ54Dx7\nnN3H7vVh9zbdrgG192dJNtEmHowTAahpYp/4i7dVR8Dh3/YPQ5ts/IE1NcPrt/n9ENvR1w7yc/ah\nJHPCZoCjuQuAf/jnMX7N4oQkL/KIbNKXU+JmAOb1tB/vlZW25m1vUwtnGtM7sy+AWT+fi78HYGCY\nkH1Rs0hTKydA9SUXATA12TZncsjSKYQQQgghhBAiaxSWpfOJBID4vM2S/f27n+Ef7AzAV857wsYs\n9LFvJzkVLatYSBSbljW0cL7jK5Mthp8R3wVgYZNZyTE1t7a4TUkwNAFg12CKd/Y6pbQqErPwOCza\nrNtvF+wBwJxTWx77x/Vm+RqeZuq98EhbMS3JmnyZY0cArpluqXjf7MCeTuIm1o/fzT/58eexDuwx\nz6yyuLeaYY27KzdYO6X7szkWqLWcDED8+V7UWok7DoimRHArbXtZ6Bkl0wze18anbSEkHdpvpuny\nxiQANu7fA6iX13Joe7AiS3MtVu4w4xjPu4WzLaQ5Cvh5y2Mq+7ddpJzRM4Hv2OK1Px8DwKnnL7aO\n7azpdekrALzVr38BWucbMCihV5jSqKvKEuITzjRLZXUS4Z4VvrZpgK7ZiaqjPbPsEC5npa+J/+3x\n+3s94j2/zJzcHeDz/BmAzf558VHpzTfJhzj54xsJALE6UDvcutI8Eg3jGwHCv7vV+u27YJmbvb9z\ns49am1UxO8Kw+Dk+H+yCc2ULYyosMTMzm3kGWxbWZUewAmBqyJ2FM0WWTiGEEEIIIYQQWaOgLJ3x\neI9r+lva87r/wVQ3Zx3sa4b/qJ1f4m7Z4SyftVmWtHNHGWREAsC5RM6943+8c2aLw68+YTJQX7uz\nbFhgzVNDrK2++FFfkeRDmg5zRLBYqDmtGDvHsxofy1EArPSA1jAi1mU+LVjmngPAzLPN0tkpWixy\neyzUX2U1M+9IP+3nbTFaOncHIC6yf0LSZG23ZQVel3X+ngB1Vk6AI8J/+9Iv2r/fQQmvuCdD1V+t\nqzL8X/v3lxUslviZ3jZnm8bhNryZvr9hXwC6dX/Se27PjWgZxyzatdGuUhW/uBLGJrbKDQCeY5gD\nzKGBx97yz+7SMGnHy5h11YX24Wy7uVfe45aR/kVy7rqXzZo/2TWnd5jCjQtt1Xpva5psMumyAYD5\neZzhSTMfumswAF8JZ/moV7MhbdsYCkvdmSD9DX/1Ncue3TrvArONpVaTWfE0+gaLkazZy4fc6nVa\njysES+cFrHDLboUftDFVSf7EySn+IDHrDAA2XmLX2pnD6+MZUxrHN8Ir0cp09F/4FrVu5U/etpP8\ng54FaOkclACwJIQW/U0ejgsBCOFfAFwZ7ZlyY7g+y8KVLwX10hkm2IPWMZMtgPfOX4zkmO/b8s/4\nAQA79LKo9ZpLbZuqnaGmhWcSvwfy/U9Z+9P3AN9uUTwOgDGhkJLuJJ84YmTclcVNHtYr/AW8svq2\nzItUQMSfu+KftaZ66l3
"text/plain": [
"<matplotlib.figure.Figure at 0x7f6cab246898>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"slice = 15\n",
"predicted = model.predict(X_test[:slice]).argmax(-1)\n",
"\n",
"plt.figure(figsize=(16,8))\n",
"for i in range(slice):\n",
" plt.subplot(1, slice, i+1)\n",
" plt.imshow(X_test_orig[i], interpolation='nearest')\n",
" plt.text(0, 0, predicted[i], color='black', \n",
" bbox=dict(facecolor='white', alpha=1))\n",
" plt.axis('off')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Adding more Dense Layers"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"model = Sequential()\n",
"model.add(Convolution2D(nb_filters, nb_conv, nb_conv,\n",
" border_mode='valid',\n",
" input_shape=(1, img_rows, img_cols)))\n",
"model.add(Activation('relu'))\n",
"\n",
"model.add(Flatten())\n",
"model.add(Dense(128))\n",
"model.add(Activation('relu'))\n",
"\n",
"model.add(Dense(nb_classes))\n",
"model.add(Activation('softmax'))"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 11918 samples, validate on 10000 samples\n",
"Epoch 1/2\n",
"11918/11918 [==============================] - 0s - loss: 0.3044 - acc: 0.9379 - val_loss: 0.1469 - val_acc: 0.9625\n",
"Epoch 2/2\n",
"11918/11918 [==============================] - 0s - loss: 0.1189 - acc: 0.9640 - val_loss: 0.1058 - val_acc: 0.9655\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x7f6cf59f7358>"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.compile(loss='categorical_crossentropy',\n",
" optimizer='sgd',\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(X_train, Y_train, batch_size=batch_size, \n",
" nb_epoch=nb_epoch,verbose=1,\n",
" validation_data=(X_test, Y_test))"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test score: 0.105762729073\n",
"Test accuracy: 0.9655\n"
]
}
],
"source": [
"#Evaluating the model on the test data \n",
"score, accuracy = model.evaluate(X_test, Y_test, verbose=0)\n",
"print('Test score:', score)\n",
"print('Test accuracy:', accuracy)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Adding Dropout"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"model = Sequential()\n",
"\n",
"model.add(Convolution2D(nb_filters, nb_conv, nb_conv,\n",
" border_mode='valid',\n",
" input_shape=(1, img_rows, img_cols)))\n",
"model.add(Activation('relu'))\n",
"\n",
"model.add(Flatten())\n",
"model.add(Dense(128))\n",
"model.add(Activation('relu'))\n",
"model.add(Dropout(0.5))\n",
"model.add(Dense(nb_classes))\n",
"model.add(Activation('softmax'))"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 11918 samples, validate on 10000 samples\n",
"Epoch 1/2\n",
"11918/11918 [==============================] - 0s - loss: 0.3128 - acc: 0.9097 - val_loss: 0.1438 - val_acc: 0.9624\n",
"Epoch 2/2\n",
"11918/11918 [==============================] - 0s - loss: 0.1362 - acc: 0.9580 - val_loss: 0.1145 - val_acc: 0.9628\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x7f6ccb180208>"
]
},
"execution_count": 48,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.compile(loss='categorical_crossentropy',\n",
" optimizer='sgd',\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(X_train, Y_train, batch_size=batch_size, \n",
" nb_epoch=nb_epoch,verbose=1,\n",
" validation_data=(X_test, Y_test))"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test score: 0.11448907243\n",
"Test accuracy: 0.9628\n"
]
}
],
"source": [
"#Evaluating the model on the test data \n",
"score, accuracy = model.evaluate(X_test, Y_test, verbose=0)\n",
"print('Test score:', score)\n",
"print('Test accuracy:', accuracy)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Adding more Convolution Layers"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"model = Sequential()\n",
"model.add(Convolution2D(nb_filters, nb_conv, nb_conv,\n",
" border_mode='valid',\n",
" input_shape=(1, img_rows, img_cols)))\n",
"model.add(Activation('relu'))\n",
"model.add(Convolution2D(nb_filters, nb_conv, nb_conv))\n",
"model.add(Activation('relu'))\n",
"model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))\n",
"model.add(Dropout(0.25))\n",
" \n",
"model.add(Flatten())\n",
"model.add(Dense(128))\n",
"model.add(Activation('relu'))\n",
"model.add(Dropout(0.5))\n",
"model.add(Dense(nb_classes))\n",
"model.add(Activation('softmax'))"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"collapsed": false,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 11918 samples, validate on 10000 samples\n",
"Epoch 1/2\n",
"11918/11918 [==============================] - 1s - loss: 0.4707 - acc: 0.8288 - val_loss: 0.2307 - val_acc: 0.9399\n",
"Epoch 2/2\n",
"11918/11918 [==============================] - 1s - loss: 0.1882 - acc: 0.9383 - val_loss: 0.1195 - val_acc: 0.9621\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x7f6cc97b8748>"
]
},
"execution_count": 51,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.compile(loss='categorical_crossentropy',\n",
" optimizer='sgd',\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(X_train, Y_train, batch_size=batch_size, \n",
" nb_epoch=nb_epoch,verbose=1,\n",
" validation_data=(X_test, Y_test))"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test score: 0.11954063682\n",
"Test accuracy: 0.9621\n"
]
}
],
"source": [
"#Evaluating the model on the test data \n",
"score, accuracy = model.evaluate(X_test, Y_test, verbose=0)\n",
"print('Test score:', score)\n",
"print('Test accuracy:', accuracy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise\n",
"\n",
"The above code has been written as a function. \n",
"\n",
"Change some of the **hyperparameters** and see what happens. "
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"# Function for constructing the convolution neural network\n",
"# Feel free to add parameters, if you want\n",
"\n",
"def build_model():\n",
" \"\"\"\"\"\"\n",
" model = Sequential()\n",
" model.add(Convolution2D(nb_filters, nb_conv, nb_conv,\n",
" border_mode='valid',\n",
" input_shape=(1, img_rows, img_cols)))\n",
" model.add(Activation('relu'))\n",
" model.add(Convolution2D(nb_filters, nb_conv, nb_conv))\n",
" model.add(Activation('relu'))\n",
" model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))\n",
" model.add(Dropout(0.25))\n",
" \n",
" model.add(Flatten())\n",
" model.add(Dense(128))\n",
" model.add(Activation('relu'))\n",
" model.add(Dropout(0.5))\n",
" model.add(Dense(nb_classes))\n",
" model.add(Activation('softmax'))\n",
" \n",
" model.compile(loss='categorical_crossentropy',\n",
" optimizer='sgd',\n",
" metrics=['accuracy'])\n",
"\n",
" model.fit(X_train, Y_train, batch_size=batch_size, \n",
" nb_epoch=nb_epoch,verbose=1,\n",
" validation_data=(X_test, Y_test))\n",
" \n",
"\n",
" #Evaluating the model on the test data \n",
" score, accuracy = model.evaluate(X_test, Y_test, verbose=0)\n",
" print('Test score:', score)\n",
" print('Test accuracy:', accuracy)"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 11918 samples, validate on 10000 samples\n",
"Epoch 1/2\n",
"11918/11918 [==============================] - 1s - loss: 0.5634 - acc: 0.7860 - val_loss: 0.3574 - val_acc: 0.9363\n",
"Epoch 2/2\n",
"11918/11918 [==============================] - 1s - loss: 0.2372 - acc: 0.9292 - val_loss: 0.2253 - val_acc: 0.9190\n",
"Test score: 0.225333989978\n",
"Test accuracy: 0.919\n",
"1 loop, best of 1: 5.45 s per loop\n"
]
}
],
"source": [
"#Timing how long it takes to build the model and test it.\n",
"%timeit -n1 -r1 build_model()"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Batch Normalisation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Normalize the activations of the previous layer at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## How to BatchNorm in Keras\n",
"\n",
"```python\n",
"from keras.layers.normalization import BatchNormalization\n",
"\n",
"BatchNormalization(epsilon=1e-06, mode=0, \n",
" axis=-1, momentum=0.99, \n",
" weights=None, beta_init='zero', \n",
" gamma_init='one')\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [],
"source": [
"# Try to add a new BatchNormalization layer to the Model \n",
"# (after the Dropout layer)"
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Reference in New Issue Copy Permalink