mirror of
https://github.com/donnemartin/data-science-ipython-notebooks.git
synced 2024-03-22 13:30:56 +08:00
1722 lines
516 KiB
Python
1722 lines
516 KiB
Python
|
{
|
||
|
|
"cells": [
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"Credits: Forked from [deep-learning-keras-tensorflow](https://github.com/leriomaggio/deep-learning-keras-tensorflow) by Valerio Maggio"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "24ca50a0-b9ad-425d-a28f-6bd3ca3ac378"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "slide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"# Introduction to Deep Learning"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "3896ecb2-47fa-4dbc-a2ff-9c76e9dfe93e"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"Deep learning allows computational models that are composed of multiple processing **layers** to learn representations of data with multiple levels of abstraction."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "8c3060aa-fee9-438c-bc60-4685c0eb4750"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. "
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "6287766f-972f-4b4d-bee7-5418f0af74de"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"**Deep learning** is one of the leading tools in data analysis these days and one of the most common frameworks for deep learning is **Keras**. "
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "2f1c6299-954a-461a-b5c1-a86a1d12ad15"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"The Tutorial will provide an introduction to deep learning using `keras` with practical code examples."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "5e13607b-3ec5-4a95-a2d8-f898f20748da"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "slide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"# Artificial Neural Networks (ANN)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "4fa2e86a-be32-4e78-96d9-f511a07e3908"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"In machine learning and cognitive science, an artificial neural network (ANN) is a network inspired by biological neural networks which are used to estimate or approximate functions that can depend on a large number of inputs that are generally unknown"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "df0121bc-10f1-4ace-840e-6fc89c6fdc7f"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"An ANN is built from nodes (neurons) stacked in layers between the feature vector and the target vector. "
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "c25d7194-10bd-4196-9d4c-592bf6e188f9"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"A node in a neural network is built from Weights and Activation function"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "15260f90-13d1-4fcc-afc6-379c507cb950"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"An early version of ANN built from one node was called the **Perceptron**"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "92d4603e-7e39-4156-818c-785df6189fe8"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "-"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"<img src =\"imgs/Perceptron.png\" width=\"85%\">"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "356d5ec7-3392-4daa-9671-4cc7111c5c91"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"The Perceptron is an algorithm for supervised learning of binary classifiers. functions that can decide whether an input (represented by a vector of numbers) belongs to one class or another.\n",
|
||
|
|
"\n",
|
||
|
|
"Much like logistic regression, the weights in a neural net are being multiplied by the input vertor summed up and feeded into the activation function's input."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"A Perceptron Network can be designed to have *multiple layers*, leading to the **Multi-Layer Perceptron** (aka `MLP`)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"<img src =\"imgs/MLP.png\" width=\"85%\">"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"nbpresent": {
|
||
|
|
"id": "5678486b-caf4-440b-be62-2f1286982c71"
|
||
|
|
},
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"The weights of each neuron are learned by **gradient descent**, where each neuron's error is derived with respect to it's weight."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"Optimization is done for each layer with respect to the previous layer in a technique known as **BackPropagation**."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"<img src =\"imgs/backprop.png\" width=\"80%\">"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "slide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"# Building Neural Nets from scratch \n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Idea:\n",
|
||
|
|
"\n",
|
||
|
|
"We will build the neural networks from first principles. \n",
|
||
|
|
"We will create a very simple model and understand how it works. We will also be implementing backpropagation algorithm. \n",
|
||
|
|
"\n",
|
||
|
|
"**Please note that this code is not optimized and not to be used in production**. \n",
|
||
|
|
"\n",
|
||
|
|
"This is for instructive purpose - for us to understand how ANN works. \n",
|
||
|
|
"\n",
|
||
|
|
"Libraries like `theano` have highly optimized code."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "skip"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"(*The following code is inspired from [these](https://github.com/dennybritz/nn-from-scratch) terrific notebooks*)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 1,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "skip"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Import the required packages\n",
|
||
|
|
"import numpy as np\n",
|
||
|
|
"import pandas as pd\n",
|
||
|
|
"import matplotlib\n",
|
||
|
|
"import matplotlib.pyplot as plt\n",
|
||
|
|
"import scipy"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 2,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "skip"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Display plots inline \n",
|
||
|
|
"%matplotlib inline\n",
|
||
|
|
"# Define plot's default figure size\n",
|
||
|
|
"matplotlib.rcParams['figure.figsize'] = (10.0, 8.0)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 3,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "skip"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"import random\n",
|
||
|
|
"random.seed(123)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 4,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"#read the datasets\n",
|
||
|
|
"train = pd.read_csv(\"data/intro_to_ann.csv\")"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 5,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"X, y = np.array(train.ix[:,0:2]), np.array(train.ix[:,2])"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 6,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"(500, 2)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 6,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"X.shape"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 7,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"(500,)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 7,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"y.shape"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 8,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.collections.PathCollection at 0x110b4b0f0>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 8,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAHfCAYAAABnDB0iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlczdkbB/DPl0i29r20WCqUXWlkQiiyhmhs2bINhrEN\nYxnC2LexZB0zslOEknXQZI/QogVZi7Rfbff5/WHc39y56Ea3Wzzv1+u+Xpx7lufeZnicc77nCEQE\nxhhjjDGmGBWUHQBjjDHG2JeMky3GGGOMMQXiZIsxxhhjTIE42WKMMcYYUyBOthhjjDHGFIiTLcYY\nY4wxBfrsZEsQBBNBEM4IgnBPEIRIQRDGf6DeGkEQ7guCECEIQuPPHZcxxhhjrDxQKYE+CgBMIqII\nQRCqA7guCMJJIop+V0EQBDcAtYmoriAI9gA2AnAogbEZY4wxxsq0z57ZIqLnRBTxz6+zAEQBMP5P\nte4Adv5T5zIAdUEQ9D93bMYYY4yxsq5E92wJgmAOoDGAy/95yxhA0r9+/wSyCRljjDHG2BenJJYR\nAQD/LCEeADDhnxkuqbff00TmniBBEPjuIMYYY4yVG0T0vhxHSokkW4IgqOBtovUHEQW+p8pjAKb/\n+r0JgKfv64vvaiy/5s6di7lz5yo7DPaJ+OdXfvHPrnzjn1/5JQhF5lkASm4ZcRuAe0S0+gPvHwEw\nCAAEQXAAkEZEL0pobMYYY4yxMuuzZ7YEQfgGwHcAIgVBuIm3y4M/ATADQETkR0THBUHoLAhCHIBs\nAN6fOy5jjDHGWHnw2ckWEV0CUFGOeuM+dyxWtjk7Oys7BPYZ+OdXfvHPrnzjn9+XTyhLe6QEQaCy\nFA9jjDHG2IcIgiDXBnm+rocxxhhjTIE42WKMMcYYUyBOthhjjDHGFIiTLcYYY4wxBeJkizHGGGNM\ngTjZYowxxhhTIE62GGOMMcYUiJMtxhhjjDEF4mSLMcYYY0yBONlijDHGGFMgTrYYY4wxxhSIky3G\nGGOMMQXiZIsxxhhjTIE42WKMMcYYUyBOthhjjDHGFIiTLcYYY4wxBeJkizHGGGNMgTjZYowxxhhT\nIE62GGOMMcYUiJMtxhhjjDEF4mSLMcYYY0yBONlijDHGGFMgTrYYY4wxxhSIky3GGGOMMQXiZIsx\nxhhjTIE42WKMMcYYUyBOthhjjDHGFIiTLcYYY4wxBeJkizHGGGNMgTjZYowxxhhTIE62GGOMMcYU\niJMtxhhjjDEF4mSLMcYYY0yBONlijDHGGFMgTrYYY4wxxhSIky3G2Ec9f/4cJ0+exL1795QdCmOM\nlUucbDHG3quwsBDjxo2DjY0NFi3+FZ06dYKzszOeP3+u7NAYY6xc4WSLMfZeixcvRuSdu7h3Pw4n\nQkMRFRcPB8dv0LdvX2WHxhhj5YpARMqOQUIQBCpL8TD2NTM2NsaR4yfQoGFDSVlhYSHq162DoKAg\n2NraKjE6xhhTPkEQQERCUfV4ZosxJkMsFuPZs2ewqV9fqrxixYqwsrZGUlKSkiJjjLHyh5MtxpiM\nChUqwM7ODmdOnZIqz8zMxLWrV2FnZ6ekyBhjrPzhZIsx9l6zZs3CGJ+ROHPqFIgI92Nj4dW3Dzw8\nPGBiYqLs8BhjrNxQUXYAjLGyqXfv3hCLxZj242RER0dDU1MTo0aNws8//6zs0BhjrFzhDfKMsSLl\n5eWhUqVKEIQi94EyxthXQ94N8pxsMcYYY4x9An4akTHGGGOsDOBkizHGGGNMgTjZYowxxhhTIE62\nGGOMMcYUiJMtxhhjjDEF4mSLMcYYY0yBONlijDHGGFMgTrYYY4wxxhSIky3GGGOMMQXiZIsxxhhj\nTIE42WKMMcYYUyBOthhjjDHGFIiTLcYYY4wxBeJkizHGGGNMgTjZYowxxhhTIE62GGOMMcYUiJMt\nxhhjjDEF4mSLMcYYY0yBONlijDHGGFMgTrYYY4wxxhSoRJItQRC2CoLwQhCE2x94/1tBENIEQbjx\nz2tWSYzLGGOMMVbWqZRQP9sBrAWw8yN1/iKibiU0HmOMMcZYuVAiM1tEdBHA6yKqCSUxFmOMMcZY\neVKae7YcBEG4KQjCMUEQ6pfiuIwxxhhjSlNSy4hFuQ7AjIhyBEFwAxAAoN77Ks6dO1fya2dnZzg7\nO5dGfIwxxhhjH3Xu3DmcO3eu2O0EIiqRAARBMANwlIjs5KibCKAZEaX+p5xKKh7GGGOMMUUSBAFE\nVOQ2qZJcRhTwgX1ZgiDo/+vXLfE2yUt9X13GGGOMsS9JiSwjCoLgD8AZgLYgCI8AzAFQGQARkR+A\n3oIgjAaQD0AEwLMkxmWMMcYYK+tKbBmxJPAyImOMMcbKC2UsIzLGGGOMsf/gZIsxxhhjTIE42WKM\nMcYYUyBOthhjjDHGFIiTLcYYY4wxBeJkizHGGGNMgTjZYowxxhhTIE62GGOMMcYUiJMtxhhjjDEF\n4mSLMcYYY0yBONlijDHGGFMgTrYYY4wxxhSIky3GGGOMMQXiZIsxxhhjTIE42WKMMSXKzs5GVFQU\n0tLSlB0KY0xBONlijDElEIvFmDVrFkxNTdG9Rw+Ym5vDx8cHIpFI2aExxkoYJ1uMlWFZWVlYt24d\n+np6YtSoUbh8+bKyQ2IlxNfXF2fOnsOVmxG4fS8Kd2PvI+XVK4wZM0bZoTHGSphARMqOQUIQBCpL\n8TCmTK9evUKbNm1Qp25d9PDwwJPHT7Dxt3WYMmUKJkyYoOzw2GcoKCiAkZERzvx1AXXq1pWUp6en\nw6ZObURHR0NPT0+JETLG5CEIAohIKLJeWUpuONli7/PixQskJSWhdu3a0NTUVHY4pWby5MnIyMrC\n2vUbJGWPHj2CfdMmiIqKgoGBgRKjY58jJSUF1tbWeJKcIvNeawd7/LZuHezt7ZUQGWOsOORNtngZ\nkZVZWVlZGDBgAGxsbDBi5EhYWlpi0qRJKCgoUHZopSIwMBCjxoyVKqtVqxY6dOqE48ePKykqVhI0\nNTWhoqKC2JgYqfK0tDTEx8XB0tJSSZExxhSBky1WZo0cORIkCIhNfIC/r17D7aho3Iy4hV9++aVU\n4xCJRNi/fz/8/PwQHR1dauMKggCxWCxTTmIxKlTg/3XLMxUVFUycOBEjvL3xIDERAJCcnIwRQ73R\nu3dv6OrqKjlCxlhJ4mVEViY9e/YMDRo0wP0HD1GtWjVJeWJCAto4tsLTp09RqVIlhcdx8eJFeHh4\nwK5RIxgZGyPkxAl06dIFfn5+qFixokLHnjZtGp4nJ2Pj5i0QhLez1IkJCXBs2QKxsbH8F3I5JxaL\nsWDBAqxZswbVa9RAeloaBgwYgGXLlkFVVVXZ4THG5MB7tli5Fh4eju+/H48L4eEy7xnp6iA2NhY6\nOjoKjSEnJwcWFhbYuuN3uHTsCODtmUjdOndG/36eGDdunELHT0tLQ9u2baGrp4ee/2yQ3+K3CfPm\nzYOPj49Cx2al582bN3jy5An09fVRvXp1ZYfDGCsG3rPFyrU6deogLu4+UlNTpcrv3rkDNTW1Utko\nf/ToUTRu0kSSaAFAtWrVMHvuXGzfvl3h42toaCAsLAz9+/XD5bAw5GRl4uTJk5xolSFEhM/9B2KV\nKlVQu3ZtTrQY+4JxssXKJB0dHXh5eWHIgO/w6NEjAEDUvXsYOngQfvzxxxJZwiMiBAUFwbNfP7h3\n7Yo1a9YgOztb8v6rV69gbGIi087E1BSvXr367PHloaamBm9vb+zYsQMrVqyAnZ1dqYz7Tl5eHt68\neVOqY5YHDx48QL9+/aCmpoaqVavCy8sLSUlJyg6LMVZGcbLFyqxVq1ahSePGaNW8GWoZGqBLp44Y\nPGgQJk6cWCL9//DDD5g6bRqc27XDd4MG4dTp03ByckJGRgYAoE2bNgg5cULmRO9DBw+iTZs2JRJD\nWfXkyRP09fSEuro61NXV4dKhAyIiIpQdVpmQmpqKNm3awMqmPh48eYr4R0mwqF0Hbdq0QXp6urLD\nY4yVQbxni5V5ubm5eP3
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.figure.Figure at 0x10eada2b0>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"#Let's plot the dataset and see how it is\n",
|
||
|
|
"plt.scatter(X[:,0], X[:,1], s=40, c=y, cmap=plt.cm.BuGn)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"## Start Building our ANN building blocks\n",
|
||
|
|
"\n",
|
||
|
|
"Note: This process will eventually result in our own Neural Networks class"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### A look at the details"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"<img src=\"imgs/mlp_details.png\" width=\"85%\" />"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Function to generate a random number, given two numbers\n",
|
||
|
|
"\n",
|
||
|
|
"**Where will it be used?**: When we initialize the neural networks, the weights have to be randomly assigned."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 9,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# calculate a random number where: a <= rand < b\n",
|
||
|
|
"def rand(a, b):\n",
|
||
|
|
" return (b-a)*random.random() + a"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 10,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Make a matrix \n",
|
||
|
|
"def makeMatrix(I, J, fill=0.0):\n",
|
||
|
|
" return np.zeros([I,J])"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Define our activation function. Let's use sigmoid function"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 11,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# our sigmoid function\n",
|
||
|
|
"def sigmoid(x):\n",
|
||
|
|
" #return math.tanh(x)\n",
|
||
|
|
" return 1/(1+np.exp(-x))"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Derivative of our activation function. \n",
|
||
|
|
"\n",
|
||
|
|
"Note: We need this when we run the backpropagation algorithm\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 12,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# derivative of our sigmoid function, in terms of the output (i.e. y)\n",
|
||
|
|
"def dsigmoid(y):\n",
|
||
|
|
" return y - y**2"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Our neural networks class\n",
|
||
|
|
"\n",
|
||
|
|
"When we first create a neural networks architecture, we need to know the number of inputs, number of hidden layers and number of outputs.\n",
|
||
|
|
"\n",
|
||
|
|
"The weights have to be randomly initialized."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"```python\n",
|
||
|
|
"class ANN:\n",
|
||
|
|
" def __init__(self, ni, nh, no):\n",
|
||
|
|
" # number of input, hidden, and output nodes\n",
|
||
|
|
" self.ni = ni + 1 # +1 for bias node\n",
|
||
|
|
" self.nh = nh\n",
|
||
|
|
" self.no = no\n",
|
||
|
|
"\n",
|
||
|
|
" # activations for nodes\n",
|
||
|
|
" self.ai = [1.0]*self.ni\n",
|
||
|
|
" self.ah = [1.0]*self.nh\n",
|
||
|
|
" self.ao = [1.0]*self.no\n",
|
||
|
|
" \n",
|
||
|
|
" # create weights\n",
|
||
|
|
" self.wi = makeMatrix(self.ni, self.nh)\n",
|
||
|
|
" self.wo = makeMatrix(self.nh, self.no)\n",
|
||
|
|
" \n",
|
||
|
|
" # set them to random vaules\n",
|
||
|
|
" self.wi = rand(-0.2, 0.2, size=self.wi.shape)\n",
|
||
|
|
" self.wo = rand(-2.0, 2.0, size=self.wo.shape)\n",
|
||
|
|
"\n",
|
||
|
|
" # last change in weights for momentum \n",
|
||
|
|
" self.ci = makeMatrix(self.ni, self.nh)\n",
|
||
|
|
" self.co = makeMatrix(self.nh, self.no)\n",
|
||
|
|
"```"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Activation Function"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"```python\n",
|
||
|
|
"def activate(self, inputs):\n",
|
||
|
|
" \n",
|
||
|
|
" if len(inputs) != self.ni-1:\n",
|
||
|
|
" print(inputs)\n",
|
||
|
|
" raise ValueError('wrong number of inputs')\n",
|
||
|
|
"\n",
|
||
|
|
" # input activations\n",
|
||
|
|
" for i in range(self.ni-1):\n",
|
||
|
|
" self.ai[i] = inputs[i]\n",
|
||
|
|
"\n",
|
||
|
|
" # hidden activations\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" sum_h = 0.0\n",
|
||
|
|
" for i in range(self.ni):\n",
|
||
|
|
" sum_h += self.ai[i] * self.wi[i][j]\n",
|
||
|
|
" self.ah[j] = sigmoid(sum_h)\n",
|
||
|
|
"\n",
|
||
|
|
" # output activations\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" sum_o = 0.0\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" sum_o += self.ah[j] * self.wo[j][k]\n",
|
||
|
|
" self.ao[k] = sigmoid(sum_o)\n",
|
||
|
|
"\n",
|
||
|
|
" return self.ao[:]\n",
|
||
|
|
"```"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### BackPropagation"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"```python\n",
|
||
|
|
"def backPropagate(self, targets, N, M):\n",
|
||
|
|
" \n",
|
||
|
|
" if len(targets) != self.no:\n",
|
||
|
|
" print(targets)\n",
|
||
|
|
" raise ValueError('wrong number of target values')\n",
|
||
|
|
"\n",
|
||
|
|
" # calculate error terms for output\n",
|
||
|
|
" output_deltas = np.zeros(self.no)\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" error = targets[k]-self.ao[k]\n",
|
||
|
|
" output_deltas[k] = dsigmoid(self.ao[k]) * error\n",
|
||
|
|
"\n",
|
||
|
|
" # calculate error terms for hidden\n",
|
||
|
|
" hidden_deltas = np.zeros(self.nh)\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" error = 0.0\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" error += output_deltas[k]*self.wo[j][k]\n",
|
||
|
|
" hidden_deltas[j] = dsigmoid(self.ah[j]) * error\n",
|
||
|
|
"\n",
|
||
|
|
" # update output weights\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" change = output_deltas[k] * self.ah[j]\n",
|
||
|
|
" self.wo[j][k] += N*change + \n",
|
||
|
|
" M*self.co[j][k]\n",
|
||
|
|
" self.co[j][k] = change\n",
|
||
|
|
"\n",
|
||
|
|
" # update input weights\n",
|
||
|
|
" for i in range(self.ni):\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" change = hidden_deltas[j]*self.ai[i]\n",
|
||
|
|
" self.wi[i][j] += N*change + \n",
|
||
|
|
" M*self.ci[i][j]\n",
|
||
|
|
" self.ci[i][j] = change\n",
|
||
|
|
"\n",
|
||
|
|
" # calculate error\n",
|
||
|
|
" error = 0.0\n",
|
||
|
|
" for k in range(len(targets)):\n",
|
||
|
|
" error += 0.5*(targets[k]-self.ao[k])**2\n",
|
||
|
|
" return error\n",
|
||
|
|
"```"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 13,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Putting all together\n",
|
||
|
|
"\n",
|
||
|
|
"class ANN:\n",
|
||
|
|
" def __init__(self, ni, nh, no):\n",
|
||
|
|
" # number of input, hidden, and output nodes\n",
|
||
|
|
" self.ni = ni + 1 # +1 for bias node\n",
|
||
|
|
" self.nh = nh\n",
|
||
|
|
" self.no = no\n",
|
||
|
|
"\n",
|
||
|
|
" # activations for nodes\n",
|
||
|
|
" self.ai = [1.0]*self.ni\n",
|
||
|
|
" self.ah = [1.0]*self.nh\n",
|
||
|
|
" self.ao = [1.0]*self.no\n",
|
||
|
|
" \n",
|
||
|
|
" # create weights\n",
|
||
|
|
" self.wi = makeMatrix(self.ni, self.nh)\n",
|
||
|
|
" self.wo = makeMatrix(self.nh, self.no)\n",
|
||
|
|
" \n",
|
||
|
|
" # set them to random vaules\n",
|
||
|
|
" for i in range(self.ni):\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" self.wi[i][j] = rand(-0.2, 0.2)\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" self.wo[j][k] = rand(-2.0, 2.0)\n",
|
||
|
|
"\n",
|
||
|
|
" # last change in weights for momentum \n",
|
||
|
|
" self.ci = makeMatrix(self.ni, self.nh)\n",
|
||
|
|
" self.co = makeMatrix(self.nh, self.no)\n",
|
||
|
|
" \n",
|
||
|
|
"\n",
|
||
|
|
" def backPropagate(self, targets, N, M):\n",
|
||
|
|
" \n",
|
||
|
|
" if len(targets) != self.no:\n",
|
||
|
|
" print(targets)\n",
|
||
|
|
" raise ValueError('wrong number of target values')\n",
|
||
|
|
"\n",
|
||
|
|
" # calculate error terms for output\n",
|
||
|
|
" output_deltas = np.zeros(self.no)\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" error = targets[k]-self.ao[k]\n",
|
||
|
|
" output_deltas[k] = dsigmoid(self.ao[k]) * error\n",
|
||
|
|
"\n",
|
||
|
|
" # calculate error terms for hidden\n",
|
||
|
|
" hidden_deltas = np.zeros(self.nh)\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" error = 0.0\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" error += output_deltas[k]*self.wo[j][k]\n",
|
||
|
|
" hidden_deltas[j] = dsigmoid(self.ah[j]) * error\n",
|
||
|
|
"\n",
|
||
|
|
" # update output weights\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" change = output_deltas[k] * self.ah[j]\n",
|
||
|
|
" self.wo[j][k] += N*change + M*self.co[j][k]\n",
|
||
|
|
" self.co[j][k] = change\n",
|
||
|
|
"\n",
|
||
|
|
" # update input weights\n",
|
||
|
|
" for i in range(self.ni):\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" change = hidden_deltas[j]*self.ai[i]\n",
|
||
|
|
" self.wi[i][j] += N*change + M*self.ci[i][j]\n",
|
||
|
|
" self.ci[i][j] = change\n",
|
||
|
|
"\n",
|
||
|
|
" # calculate error\n",
|
||
|
|
" error = 0.0\n",
|
||
|
|
" for k in range(len(targets)):\n",
|
||
|
|
" error += 0.5*(targets[k]-self.ao[k])**2\n",
|
||
|
|
" return error\n",
|
||
|
|
"\n",
|
||
|
|
"\n",
|
||
|
|
" def test(self, patterns):\n",
|
||
|
|
" self.predict = np.empty([len(patterns), self.no])\n",
|
||
|
|
" for i, p in enumerate(patterns):\n",
|
||
|
|
" self.predict[i] = self.activate(p)\n",
|
||
|
|
" #self.predict[i] = self.activate(p[0])\n",
|
||
|
|
" \n",
|
||
|
|
" def activate(self, inputs):\n",
|
||
|
|
" \n",
|
||
|
|
" if len(inputs) != self.ni-1:\n",
|
||
|
|
" print(inputs)\n",
|
||
|
|
" raise ValueError('wrong number of inputs')\n",
|
||
|
|
"\n",
|
||
|
|
" # input activations\n",
|
||
|
|
" for i in range(self.ni-1):\n",
|
||
|
|
" self.ai[i] = inputs[i]\n",
|
||
|
|
"\n",
|
||
|
|
" # hidden activations\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" sum_h = 0.0\n",
|
||
|
|
" for i in range(self.ni):\n",
|
||
|
|
" sum_h += self.ai[i] * self.wi[i][j]\n",
|
||
|
|
" self.ah[j] = sigmoid(sum_h)\n",
|
||
|
|
"\n",
|
||
|
|
" # output activations\n",
|
||
|
|
" for k in range(self.no):\n",
|
||
|
|
" sum_o = 0.0\n",
|
||
|
|
" for j in range(self.nh):\n",
|
||
|
|
" sum_o += self.ah[j] * self.wo[j][k]\n",
|
||
|
|
" self.ao[k] = sigmoid(sum_o)\n",
|
||
|
|
"\n",
|
||
|
|
" return self.ao[:]\n",
|
||
|
|
" \n",
|
||
|
|
"\n",
|
||
|
|
" def train(self, patterns, iterations=1000, N=0.5, M=0.1):\n",
|
||
|
|
" # N: learning rate\n",
|
||
|
|
" # M: momentum factor\n",
|
||
|
|
" patterns = list(patterns)\n",
|
||
|
|
" for i in range(iterations):\n",
|
||
|
|
" error = 0.0\n",
|
||
|
|
" for p in patterns:\n",
|
||
|
|
" inputs = p[0]\n",
|
||
|
|
" targets = p[1]\n",
|
||
|
|
" self.activate(inputs)\n",
|
||
|
|
" error += self.backPropagate([targets], N, M)\n",
|
||
|
|
" if i % 5 == 0:\n",
|
||
|
|
" print('error in interation %d : %-.5f' % (i,error))\n",
|
||
|
|
" print('Final training error: %-.5f' % error)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Running the model on our dataset"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 14,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"name": "stdout",
|
||
|
|
"output_type": "stream",
|
||
|
|
"text": [
|
||
|
|
"error in interation 0 : 53.62995\n",
|
||
|
|
"Final training error: 53.62995\n",
|
||
|
|
"Final training error: 47.35136\n",
|
||
|
|
"1 loop, best of 1: 97.6 ms per loop\n"
|
||
|
|
]
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"# create a network with two inputs, one hidden, and one output nodes\n",
|
||
|
|
"ann = ANN(2, 1, 1)\n",
|
||
|
|
"\n",
|
||
|
|
"%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=2)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Predicting on training dataset and measuring in-sample accuracy"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 15,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"name": "stdout",
|
||
|
|
"output_type": "stream",
|
||
|
|
"text": [
|
||
|
|
"1 loop, best of 1: 22.6 ms per loop\n"
|
||
|
|
]
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"%timeit -n 1 -r 1 ann.test(X)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 16,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/html": [
|
||
|
|
"<div>\n",
|
||
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
||
|
|
" <thead>\n",
|
||
|
|
" <tr style=\"text-align: right;\">\n",
|
||
|
|
" <th></th>\n",
|
||
|
|
" <th>actual</th>\n",
|
||
|
|
" <th>prediction</th>\n",
|
||
|
|
" </tr>\n",
|
||
|
|
" </thead>\n",
|
||
|
|
" <tbody>\n",
|
||
|
|
" <tr>\n",
|
||
|
|
" <th>0</th>\n",
|
||
|
|
" <td>1.0</td>\n",
|
||
|
|
" <td>0.491100</td>\n",
|
||
|
|
" </tr>\n",
|
||
|
|
" <tr>\n",
|
||
|
|
" <th>1</th>\n",
|
||
|
|
" <td>1.0</td>\n",
|
||
|
|
" <td>0.495469</td>\n",
|
||
|
|
" </tr>\n",
|
||
|
|
" <tr>\n",
|
||
|
|
" <th>2</th>\n",
|
||
|
|
" <td>0.0</td>\n",
|
||
|
|
" <td>0.097362</td>\n",
|
||
|
|
" </tr>\n",
|
||
|
|
" <tr>\n",
|
||
|
|
" <th>3</th>\n",
|
||
|
|
" <td>0.0</td>\n",
|
||
|
|
" <td>0.400006</td>\n",
|
||
|
|
" </tr>\n",
|
||
|
|
" <tr>\n",
|
||
|
|
" <th>4</th>\n",
|
||
|
|
" <td>1.0</td>\n",
|
||
|
|
" <td>0.489664</td>\n",
|
||
|
|
" </tr>\n",
|
||
|
|
" </tbody>\n",
|
||
|
|
"</table>\n",
|
||
|
|
"</div>"
|
||
|
|
],
|
||
|
|
"text/plain": [
|
||
|
|
" actual prediction\n",
|
||
|
|
"0 1.0 0.491100\n",
|
||
|
|
"1 1.0 0.495469\n",
|
||
|
|
"2 0.0 0.097362\n",
|
||
|
|
"3 0.0 0.400006\n",
|
||
|
|
"4 1.0 0.489664"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 16,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"prediction = pd.DataFrame(data=np.array([y, np.ravel(ann.predict)]).T, \n",
|
||
|
|
" columns=[\"actual\", \"prediction\"])\n",
|
||
|
|
"prediction.head()"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 17,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "fragment"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"0.076553078113180129"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 17,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"np.min(prediction.prediction)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"### Let's visualize and observe the results"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 18,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "skip"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Helper function to plot a decision boundary.\n",
|
||
|
|
"# This generates the contour plot to show the decision boundary visually\n",
|
||
|
|
"def plot_decision_boundary(nn_model):\n",
|
||
|
|
" # Set min and max values and give it some padding\n",
|
||
|
|
" x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
|
||
|
|
" y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
|
||
|
|
" h = 0.01\n",
|
||
|
|
" # Generate a grid of points with distance h between them\n",
|
||
|
|
" xx, yy = np.meshgrid(np.arange(x_min, x_max, h), \n",
|
||
|
|
" np.arange(y_min, y_max, h))\n",
|
||
|
|
" # Predict the function value for the whole gid\n",
|
||
|
|
" nn_model.test(np.c_[xx.ravel(), yy.ravel()])\n",
|
||
|
|
" Z = nn_model.predict\n",
|
||
|
|
" Z[Z>=0.5] = 1\n",
|
||
|
|
" Z[Z<0.5] = 0\n",
|
||
|
|
" Z = Z.reshape(xx.shape)\n",
|
||
|
|
" # Plot the contour and training examples\n",
|
||
|
|
" plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)\n",
|
||
|
|
" plt.scatter(X[:, 0], X[:, 1], s=40, c=y, cmap=plt.cm.BuGn)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 19,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.text.Text at 0x110bdb940>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 19,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAHpCAYAAABeGEKRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XVUVVkbx/HvFhBEMDFRxG7BbkHFwO4Yu7tbx24dW2fU\nGXvs7lbUsRW7SVsBQUSkz/sHDOMVUBwvA/o+n7VYA/ueffbDZQm/2WeffZSmaQghhBBCCP1JltgF\nCCGEEEL8aCRgCSGEEELomQQsIYQQQgg9k4AlhBBCCKFnErCEEEIIIfRMApYQQgghhJ5JwBJCJAlK\nqXdKKWt9HKuUuq2UqhLPc0UopXLF51h9UkqNV0qti+exJ5VSnRO6JiGE/kjAEkJ8kVKqo1LqplLq\nvVLquVLqV6VUan2OoWmauaZpHl97rFJqlVJq0ievF9E07XR8h/6qQvVLNiIU4gclAUsI8VlKqSHA\ndGAIkAooB+QAjiqlDP/F+Qz0W+E3U4ldgBDixyMBSwgRJ6WUOTAB6Ktp2lFN08I1TXsMtCAyZLWN\nOk5nFkkpZaeUevLR1+5KqeFKqRtAgFIqxu+ejy/VRZ1vsVJqn1LKXyl1XimV89NjlVLdgDbA8Kjj\ndn80XrWoz0srpc4ppXyVUs+UUoviGwyjLs1NVkqdjbosuVsplU4p9adS6q1S6qJSyuqj4ysopS5F\njXVRKVX+o9eslVJOUf0OAxafjFUuahxfpdQ1pZRdfGoUQiRNErCEEJ9TATAGdn7cqGnae+AgUOMz\nfT+9/NUKcATSaJoWEc/jxwNpAFdg6qfHapr2O7AemKVpWipN0xrGct5wYCCQDigPVAN6f6buT7Uk\nMsRlBfIA54AVQFrgflSNKKXSAvuA+UB6YB6wP6odYANwmchgNQXo8PcASinLqL6TNE1LCwwFtiul\n0n9FnUKIJEQClhDicywA7zgC0Qs+mYX5ggWapj3XNC04jtc/vVS3Q9O0q1FjrwdsP3NsnDRNc9Y0\n7ZIW6TGwHPia2aFVmqZ5aJr2jshQ6app2smourYCxaOOqws81DRtg6ZpEZqmbSIygNVXSmUHSgHj\nNE0L1TTtDLD3ozHaAPs1TTscVfNx4ApQ5yvqFEIkIRKwhBCf4w1YxHZJD8gS9Xp8Pf3KsV9+9Hkg\nYPaV/QFQSuVVSu1VSr1QSvkRORP2NcHw1Ueff4jl67/rygp4ftLXE7CMes1X07QPn7z2txxAC6XU\nm6gPX6AikPkr6hRCJCESsIQQn3MeCAaafNyolEpJ5OW+Y1FN7wHTjw7JEsu5EuqOuS+d9zfgHpBb\n07Q0wBgSZmH7c8D6kzYr4BmRs31plVIpPnntb0+AtZqmpYv6SBt1p+TsBKhTCPEfkIAlhIiTpmn+\nwCRgkVKqllLKMGr/qS3AY+DPqEOvA3WUUmmVUpmBAf9hma+Az+1jZQ74a5oWqJQqAPRKoDoOAHmV\nUq2UUgZKqZZAQWBv1KXJK8BEpZSRUqoSUP+jvn8SeSmxplIqmVLKJOpGgawJVKsQIoFJwBJCfFbU\nLMpo4BfgLZGzWp6Ag6ZpoVGHrQNuAh7AIWDTp6eJz1BfU9ZHn68ACkddWtsRy+tDgTZKKX9g2VfW\nFu+aNE17A9SLGs876r91NU3zjTrkJyK3uPABxgJrPur7FGhI5PvsReT7O5R/fkfLfllCfGeUpsm/\nWyGEEEIIfZIZLCGEEEIIPZOAJYQQQgihZxKwhBBCCCH07KufI5bQlFKyKEwIIYQQ3w1N02Js/ZIk\nZ7A0TdPrx/jx4/V+TvmQn82P/iE/m6T7IT+bpPshP5uk+5FQP5u4JMmAJYQQQgjxPZOAJYQQQgih\nZ/8XAcve3j6xSxBxkJ9N0iU/m6RLfjZJl/xskq7/+meT5DYaVUppSa0mIYQQQojYKKXQvpdF7kII\nIYQQ3zMJWEIIIYQQeiYBSwghhBBCzyRgCSGEEELomQQsIYQQQgg9k4AlhBBCCKFnErCEEEIIIfRM\nApYQQgghhJ5JwBJCCCGE0DMJWEIIIYQQeiYBSwghhBBCzyRgCSGEEELomQQsIYQQQgg9k4AlhBBC\nCKFnholdwH9N81md2CUIIYQQ4j+g0ndMtLFlBksIIYQQQs8kYAkhhBBC6JkELCGEEEIIPZOAJYQQ\nQgihZxKwhBBCCCH0TAKWEEIIIYSeScASQgghhNAzCVhCCCGEEHomAUsIIYQQQs8kYAkhhBBC6JkE\nLCGEEEIIPZOAJYQQQgihZxKwhBBCCCH0TAKWEEIIIYSeScASQgghhNAzCVhCCCGEEHomAUsIIYQQ\nQs8kYAkhhBBC6Nk3ByylVDal1Aml1F2l1C2lVP84jluolHqklLqulLL91nGFEEIIIZIqQz2cIwwY\nrGnadaWUGXBVKXVE07T7fx+glHIEcmuallcpVRZYCpTTw9hCCCGEEEnON89gaZr2UtO061GfBwD3\nAMtPDmsIrI065iKQWimV6VvHFkIIIYRIivS6BkspZQ3YAhc/eckSePLR18+IGcKEEEIIIX4I+rhE\nCEDU5cFtwIComSydl2PposV1rgkTJkR/bm9vj729vR4qFEIIIYT4Nk5OTjg5OX3xOKVpceaceFNK\nGQL7gIOapi2I5fWlwElN0zZHfX0fsNM07VUsx2r6qCkums/qBDu3EEIIIZIOlb5jwo+hFJqmxZhI\n0tclwpXA3djCVZQ9QPuoQsoBfrGFKyGEEEKIH8E3XyJUSlUE2gC3lFLXiLz0NxrIAWiapi3XNO2A\nUqqOUsoFeA90+tZxhRBCCCGSqm8OWJqmnQUM4nFc328dSwghhBDieyA7uQshhBBC6JkELCGEEEII\nPZOAJYQQQgihZxKwhBBCCCH0TAKWEEIIIYSeScASQgghhNAzCVhCCCGEEHomAUsIIYQQQs8kYAkh\nhBBC6JkELCGEEEIIPZOAJYQQQgihZxKwhBBCCCH0TAKWEEIIIYSeScASQgghhNAzCVhCCCGEEHom\nAUsIIYQQQs8kYAkhhBBC6JkELCGEEEIIPZOAJYQQQgihZxKwhBBCCCH0TAKWEEIIIYSeScASQggh\nhNAzCVhCCCGEEHomAUsIIYQQQs8kYAkhhBBC6JkELCHEVwkNDSMgICixyxBCiCRNApYQIl783gbS\nZcAa0uXpT8YCAylTYxrHTt1J7LKEECJJkoAlhPgiTdOo23oRBqa5uefijvfbdwz7eQZteqzg4lXX\nxC5PCCGSHMPELkAIkfSdPHMP//caC39dhlIKgAYNG/Hi2XNmLfqT7atzJ3KFQgiRtMgMlhDii27c\neYJd1erR4epvVatX59bdp4lUlRBCJF0SsIQQX2RtZcGtG9ditN+6dROr7BaJUJEQQiRtErCEEF9U\nt4YNTx+7s/y3X4mIiADAzdWVcaNH0K9LlUSuTgghkh6laVpi16BDKaUlZE2az+oEO7cQP7KHLi9p\n03MF3r5BZMqYARcXV8YNq0//7g6JXZoQQsRKpe+Y8GMohaZpKka7BCwhRHxpmsad+8/wextI8aI5\nSJnSOLFLEkKIOCVmwJK7CIUQ8aaUokjBbIldhhBCJHmyBksIIYQQQs8kYAkhhBBC6JkELCGEEEII\nPZOAJYQQQgihZxKwhBBCCCH0TAKWEEIIIYSeScASQgghhNAzCVhCCCGEEHomAUsIIYQQQs8kYAkh\nhBBC6JkELCGEEEIIPZOAJYQQQgihZxKwhBBCCCH0TC8BSym1Qin1Sil1M47X7ZRSfkop56iPn/Ux\nrhBCCCFEUmSop/OsAhYBaz9zzGlN0xroaTwhhBBCiCRLLzNYmqb9Bfh+4TClj7GEEEIIIZK6/3IN\nVjml1DWl1H6lVKH/cFwhhBBCiP+Uvi4RfslVIIemaYFKKUdgF5AvroMnTJgQ/bm9vT329vYJXZ8Q\nQgghxBc5OTnh5OT0xeOUpml6GVAplQPYq2lasXgc6w6U1DTtTSyvafqqKTaaz+oEO7cQQgghkg6V\nvmPCj6EUmqbFWAalz0u
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.figure.Figure at 0x10eb09208>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"plot_decision_boundary(ann)\n",
|
||
|
|
"plt.title(\"Our initial model\")"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"**Exercise**: \n",
|
||
|
|
"\n",
|
||
|
|
"Create Neural networks with 10 hidden nodes on the above code. \n",
|
||
|
|
"\n",
|
||
|
|
"What's the impact on accuracy?"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 20,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Put your code here \n",
|
||
|
|
"#(or load the solution if you wanna cheat :-)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 21,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"name": "stdout",
|
||
|
|
"output_type": "stream",
|
||
|
|
"text": [
|
||
|
|
"error in interation 0 : 34.91394\n",
|
||
|
|
"Final training error: 34.91394\n",
|
||
|
|
"Final training error: 25.36183\n",
|
||
|
|
"1 loop, best of 1: 288 ms per loop\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.text.Text at 0x11151f630>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 21,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAHpCAYAAABeGEKRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd0VEUfxvHvpPeeQBJCQu8B6UUh9CICoqgggqAIKKCC\nDVSaioiivoAFewUREVTALmChSY/0EkhI771tdt4/dgkJSSDIhgT8fc7JYXdumdndkDyZmTtXaa0R\nQgghhBCWY1XdDRBCCCGEuN5IwBJCCCGEsDAJWEIIIYQQFiYBSwghhBDCwiRgCSGEEEJYmAQsIYQQ\nQggLk4AlhLiqlFJGpVT9SuzXQykVdYV1BSmlMpRS6krbY2lKqTlKqU8vsv0fpVT3CrZd9L1RSn2o\nlJpviXZaSmU+CyGuJxKwhPgXlFL3KqUOKKWylVIxSqk3lVLu1d2uylBKRSilelVjEy5n8b0rWqhP\nax2ltXbT5gX/lFKblFLj/20dSqkWSqkflFKJSqmicrZ7KqXWKqWyzO/zyEs18SJtb6m1/v3fHFsT\nVfKzEOK6IQFLiMuklJoBvAjMANyAzkAw8LNSyuZfnM/asi2s8WpaD8bltKcQWAVUFAzeBPIAX2A0\n8JZSqtmVNU8IcS2SgCXEZVBKuQJzgSla65+11kVa60jgDkwha7R5v1JDNBcO6Zh7N55QSu0HspRS\nZf4vmoeuJiqljimlkpVSyy7YPl4pdci87XulVF1zeRdzD0ug+XlrpVSqUqqxUuoToC7wnXm45rFy\n6u2hlIpSSj2ulIpXSkUrpYYqpQYqpY4qpZKUUjNL7G+nlHrdvN9ZpdRrSinbEtsfN/fynVVKjaNE\nz4v52FeUUmeUUrHmnkD7SnwOc5VSS8yPbcw9RgvNzx2UUrlKKXelVLD5fbRSSj0P3AQsM7/2JSVO\n2bei97kkrfUxrfWHwKFy2uQEDAee0Vrnaq3/Ar4F7rnIS7FXSn1sbk+4UqptifMV9zSaX9NHSqkU\npdQ/QIcL6r5BKbVbKZWulPoCcLhg+2Cl1F7z98GfSqlWF9QzQym137x9pVLKrrzGqguGNUu+v+bn\nm5RS8811ZJh7+7wu3Leiz8L8vROvlEpTSu1TSjW/yHsnRI0mAUuIy9MVsAfWlizUWmcD3wN9L3Ls\nhUM6dwEDAQ+ttbGCY24G2gFtgDuUUv0AlFLDgKeAYZh6S/4AVprbsg14G/hYKeUAfALMMoeDMUAk\nMNg8XPNKBfXWBuyAAGAO8C5wN3AD0B2YrZQKMe/7DNARCAVamx8/Y27nAGA60BtoBPS5oJ5FQEPz\nsQ2BQGB2BW0qaQvQw/y4AxBX4nlX4IjWOt38XJvfl2cwvU9TzK99Wonzlfs+X6bGgEFrfbJE2X6g\nxUWOuQVYAbgD3wFvVLDfXKCe+as/MPbcBnOYXQt8DHgBq4HbSmxvC7wPTDBvXw58WzIEAyOAfubz\ntwbuvUibL/w+vvD5SHP7fDH9X3nswn3L+yzM7/mNQEOttQdwJ5B8kXYIUaNJwBLi8vgASRUEoljz\n9sr6n9Y6Rmudf5F9XtRaZ2qto4BNmAIAwAPmbcfMbVkItFFKBZm3zwM8gJ3AWa31Wxec91LDYgXA\nAq11EfAFptf1utY6R2t9CDiIKRQBjALmaa2TtdbJ5rrP9dqMAD7UWh/WWudiCgol674feFRrnW4O\nqQsx/YK+lG1AI6WUJ6bA9z4QaO5F6o4pgF2Oit7ny+ECpF9Qlg64XuSYP7XWP5rnJX3K+ff0QiOA\n583vUzRQsvetC2CjtV5i7lFdA/xdYvv9wNta613a5FMgH9PQ9jn/01rHa63TMAW9f/P6z/lQa33S\n/H395WWcqxDTe9VcKaW01ke11vFX0A4hqpUELCEuTxLgo8oZ0gP8zdsr62wl9in5CyYH0y9xMA1H\n/s88ZJSC6S99jakHCK21AfgIU+/Jq5fRpnOSz01GBnLN/yaU2J5boi0BmHrFzjljLju3LeqCbQAo\npXwBJ2B3idfxPeB9qcZprfOAXUAYpkC1GdiKqQekB5cfsCp6ny9HFqY5eSW5AZkXOSbugnodKvje\nCqD098uZEo/9gegL9i+5PRiYce49VkqlAnU4/xmBZV7/ORe+pkqdS2u9CViGqRcvTin1tlLqStoh\nRLWSgCXE5dmG6a//4SULlVLOmIb7fjEXZWMKD+f4l3OuK7kKLAqYqLX2Mn95aq1dtNbbze0JxDS0\n9yHw6gXDQZa++iwG0y/xc4LNZWDq1Qu6YNu5+pMw/QJuUeJ1eGitK3s15u9AL0w9JH+bn/fHNGRY\n0dV3VXnl3THARinVoERZa0y9fVeqvPex5LbAC/avW+JxFPBCOd8rq/5FOyrzfV1ZZT4LrfUyrXV7\nTH8YNAEev4LzC1GtJGAJcRm01hnAfGCpUqq/eYJ1CKahkEjgM/Ou+4BBynTZfm3gYQs35W1g1rlJ\nwOYJ3beX2P4h8K7W+n5MYef5EtviAEuu+7QSeEYp5aOU8gGexTTcBab35V6lVDPz8F3x/CpzD9m7\nwOvm3iyUUoGXMf9pCzAGOGTusduMaTgswjxUeU7JIcl4rvC1myfh25seKvtzE8K11jnA18B8pZST\nUqobMITz70WlTl9B+ZfATKWUh1KqDjClxLZtgEEpNVUpZa2UGo5pHtw57wKTlFIdze13VkoNMv9R\ncLn2Ad2VaU0rd0zzAC9HhZ+FUqq9UqqjMl2Jm4vpaswyS2EIca2QgCXEZdJavwzMAl7BNMdmG6Yh\nmT5a60Lzbp8CB4DTwA+Y5jGVOk1lqqroudZ6Hab5Sl8opdLMdQ0AUEpNA/w4H2bGYwo53czPFwLP\nmoeLpleiHRdtC6bwtsvchv3mxy+Y2/kD8DrwG6Yenl8vOM+TwAlgu/l1/IRpsnhlbMV0tdwWc12H\nMP1ivnB4sGRb/weMMF8t+HolXlspSqlgcx3h5v1ygSMldnkIUw9PAvA5MElrfbiSr+fCuks+nocp\nwEdg+n76pHgn0/fccGAckIJpvtaaEtt3Y5rgvsw8DHuMEpPkuYxePa31L5iWqTiAqdfwu4u0v9xT\nlHh84WfhhikMpmB6nUmY/o8JcU1S56dZCCGEEEIIS5AeLCGEEEIIC5OAJYQQQghhYRKwhBBCCCEs\n7LLvm1bVlFIyKUwIIYQQ1wytdZkrgGtkD5bW2qJfc+bMsfg55Us+m+v9Sz6bmvsln03N/ZLPpuZ+\nVdVnU5EaGbCEEEIIIa5lErCEEEIIISzsPxGwwsLCqrsJogLy2dRc8tnUXPLZ1Fzy2dRcV/uzqXEL\njZpuol6z2iSEEEIIUR6lFPpameQuhBBCCHEtk4AlhBBCCGFhErCEEEIIISxMApYQQgghhIVJwBJC\nCCGEsDAJWEIIIYQQFiYBSwghhBDCwiRgCSGEEEJYmAQsIYQQQggLk4AlhBBCCGFhErCEEEIIISxM\nApYQQgghhIVJwBJCCCGEsDAJWEIIIYQQFmZT3Q242ra2GlzdTRBCCCHEVdA1fH211S09WEIIIYQQ\nFiYBSwghhBDCwiRgCSGEEEJYmAQsIYQQQggLk4AlhBBCCGFhErCEEEIIISxMApYQQgghhIVJwBJC\nCCGEsDAJWEIIIYQQFiYBSwghhBDCwiRgCSGEEEJYmAQsIYQQQggLk4AlhBBCCGFhErCEEEIIISxM\nApYQQgghhIVJwBJCCCGEsDAJWEIIIYQQFiYBSwghhBDCwq44YCml6iilflNKHVJKhSulplWw3xKl\n1HGl1D6lVJsrrVcIIYQQoqayscA5DMB0rfU+pZQLsFsp9ZPW+si5HZRSA4EGWutGSqlOwNtAZwvU\nLYQQQghR41xxD5bWOk5rvc/8OAs4DAResNtQ4BPzPjsAd6VUrSutWwghhBCiJrLoHCylVAjQBthx\nwaZAIKrE82jKhjAhhBBCiOuCJYYIATAPD34FPGzuySq1uZxDdEXnmjt3bvHjsLAwwsLCLNBCIYQQ\nQogrs3nzZjZv3nzJ/ZT
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.figure.Figure at 0x10eb09908>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"# %load solutions/sol_111.py\n",
|
||
|
|
"ann = ANN(2, 10, 1)\n",
|
||
|
|
"%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=2)\n",
|
||
|
|
"plot_decision_boundary(ann)\n",
|
||
|
|
"plt.title(\"Our next model with 10 hidden units\")\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"**Exercise:**\n",
|
||
|
|
"\n",
|
||
|
|
"Train the neural networks by increasing the epochs. \n",
|
||
|
|
"\n",
|
||
|
|
"What's the impact on accuracy?"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 22,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": true
|
||
|
|
},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"#Put your code here"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 23,
|
||
|
|
"metadata": {
|
||
|
|
"collapsed": false,
|
||
|
|
"scrolled": false,
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"name": "stdout",
|
||
|
|
"output_type": "stream",
|
||
|
|
"text": [
|
||
|
|
"error in interation 0 : 31.63185\n",
|
||
|
|
"Final training error: 31.63185\n",
|
||
|
|
"Final training error: 25.12319\n",
|
||
|
|
"Final training error: 24.92547\n",
|
||
|
|
"Final training error: 24.89692\n",
|
||
|
|
"Final training error: 24.88124\n",
|
||
|
|
"error in interation 5 : 24.86083\n",
|
||
|
|
"Final training error: 24.86083\n",
|
||
|
|
"Final training error: 24.83512\n",
|
||
|
|
"Final training error: 24.80603\n",
|
||
|
|
"Final training error: 24.77544\n",
|
||
|
|
"Final training error: 24.74477\n",
|
||
|
|
"error in interation 10 : 24.71498\n",
|
||
|
|
"Final training error: 24.71498\n",
|
||
|
|
"Final training error: 24.68669\n",
|
||
|
|
"Final training error: 24.66021\n",
|
||
|
|
"Final training error: 24.63569\n",
|
||
|
|
"Final training error: 24.61313\n",
|
||
|
|
"error in interation 15 : 24.59246\n",
|
||
|
|
"Final training error: 24.59246\n",
|
||
|
|
"Final training error: 24.57359\n",
|
||
|
|
"Final training error: 24.55639\n",
|
||
|
|
"Final training error: 24.54072\n",
|
||
|
|
"Final training error: 24.52646\n",
|
||
|
|
"error in interation 20 : 24.51350\n",
|
||
|
|
"Final training error: 24.51350\n",
|
||
|
|
"Final training error: 24.50171\n",
|
||
|
|
"Final training error: 24.49100\n",
|
||
|
|
"Final training error: 24.48127\n",
|
||
|
|
"Final training error: 24.47242\n",
|
||
|
|
"error in interation 25 : 24.46436\n",
|
||
|
|
"Final training error: 24.46436\n",
|
||
|
|
"Final training error: 24.45702\n",
|
||
|
|
"Final training error: 24.45030\n",
|
||
|
|
"Final training error: 24.44414\n",
|
||
|
|
"Final training error: 24.43846\n",
|
||
|
|
"error in interation 30 : 24.43319\n",
|
||
|
|
"Final training error: 24.43319\n",
|
||
|
|
"Final training error: 24.42828\n",
|
||
|
|
"Final training error: 24.42366\n",
|
||
|
|
"Final training error: 24.41929\n",
|
||
|
|
"Final training error: 24.41510\n",
|
||
|
|
"error in interation 35 : 24.41107\n",
|
||
|
|
"Final training error: 24.41107\n",
|
||
|
|
"Final training error: 24.40715\n",
|
||
|
|
"Final training error: 24.40331\n",
|
||
|
|
"Final training error: 24.39952\n",
|
||
|
|
"Final training error: 24.39576\n",
|
||
|
|
"error in interation 40 : 24.39200\n",
|
||
|
|
"Final training error: 24.39200\n",
|
||
|
|
"Final training error: 24.38821\n",
|
||
|
|
"Final training error: 24.38438\n",
|
||
|
|
"Final training error: 24.38048\n",
|
||
|
|
"Final training error: 24.37649\n",
|
||
|
|
"error in interation 45 : 24.37237\n",
|
||
|
|
"Final training error: 24.37237\n",
|
||
|
|
"Final training error: 24.36806\n",
|
||
|
|
"Final training error: 24.36353\n",
|
||
|
|
"Final training error: 24.35868\n",
|
||
|
|
"Final training error: 24.35340\n",
|
||
|
|
"error in interation 50 : 24.34754\n",
|
||
|
|
"Final training error: 24.34754\n",
|
||
|
|
"Final training error: 24.34086\n",
|
||
|
|
"Final training error: 24.33302\n",
|
||
|
|
"Final training error: 24.32348\n",
|
||
|
|
"Final training error: 24.31138\n",
|
||
|
|
"error in interation 55 : 24.29529\n",
|
||
|
|
"Final training error: 24.29529\n",
|
||
|
|
"Final training error: 24.27275\n",
|
||
|
|
"Final training error: 24.23928\n",
|
||
|
|
"Final training error: 24.18646\n",
|
||
|
|
"Final training error: 24.09789\n",
|
||
|
|
"error in interation 60 : 23.94185\n",
|
||
|
|
"Final training error: 23.94185\n",
|
||
|
|
"Final training error: 23.66093\n",
|
||
|
|
"Final training error: 23.16905\n",
|
||
|
|
"Final training error: 22.38000\n",
|
||
|
|
"Final training error: 21.27360\n",
|
||
|
|
"error in interation 65 : 19.93871\n",
|
||
|
|
"Final training error: 19.93871\n",
|
||
|
|
"Final training error: 18.52756\n",
|
||
|
|
"Final training error: 17.16893\n",
|
||
|
|
"Final training error: 15.93090\n",
|
||
|
|
"Final training error: 14.83582\n",
|
||
|
|
"error in interation 70 : 13.88300\n",
|
||
|
|
"Final training error: 13.88300\n",
|
||
|
|
"Final training error: 13.06081\n",
|
||
|
|
"Final training error: 12.35255\n",
|
||
|
|
"Final training error: 11.74050\n",
|
||
|
|
"Final training error: 11.20877\n",
|
||
|
|
"error in interation 75 : 10.74440\n",
|
||
|
|
"Final training error: 10.74440\n",
|
||
|
|
"Final training error: 10.33728\n",
|
||
|
|
"Final training error: 9.97939\n",
|
||
|
|
"Final training error: 9.66423\n",
|
||
|
|
"Final training error: 9.38631\n",
|
||
|
|
"error in interation 80 : 9.14093\n",
|
||
|
|
"Final training error: 9.14093\n",
|
||
|
|
"Final training error: 8.92402\n",
|
||
|
|
"Final training error: 8.73205\n",
|
||
|
|
"Final training error: 8.56193\n",
|
||
|
|
"Final training error: 8.41096\n",
|
||
|
|
"error in interation 85 : 8.27675\n",
|
||
|
|
"Final training error: 8.27675\n",
|
||
|
|
"Final training error: 8.15722\n",
|
||
|
|
"Final training error: 8.05052\n",
|
||
|
|
"Final training error: 7.95506\n",
|
||
|
|
"Final training error: 7.86944\n",
|
||
|
|
"error in interation 90 : 7.79246\n",
|
||
|
|
"Final training error: 7.79246\n",
|
||
|
|
"Final training error: 7.72306\n",
|
||
|
|
"Final training error: 7.66035\n",
|
||
|
|
"Final training error: 7.60354\n",
|
||
|
|
"Final training error: 7.55195\n",
|
||
|
|
"error in interation 95 : 7.50499\n",
|
||
|
|
"Final training error: 7.50499\n",
|
||
|
|
"Final training error: 7.46215\n",
|
||
|
|
"Final training error: 7.42298\n",
|
||
|
|
"Final training error: 7.38707\n",
|
||
|
|
"Final training error: 7.35410\n",
|
||
|
|
"1 loop, best of 1: 14.5 s per loop\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.text.Text at 0x1115951d0>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"execution_count": 23,
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "execute_result"
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAHpCAYAAABeGEKRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd0VMXbwPHvpJDeE1pIQu+EgIACCkEQCCAgYkNFQUFA\nQQQLoCLqT8VesGFBxVcUEEEUxUaXJh3pJRAgCem9bnbeP3aJ2XRgwybh+Zyz52Rvm2fv3c0+OzN3\nRmmtEUIIIYQQ1mNn6wCEEEIIIWobSbCEEEIIIaxMEiwhhBBCCCuTBEsIIYQQwsokwRJCCCGEsDJJ\nsIQQQgghrEwSLCGsSCm1Vik1tpLbGpVSTS+zvH+VUr2sEY81KaV6K6XOlLP+I6XU0+WsL/PcKKXu\nU0pttEac1lTRtbAFpVSI+Vza9H+9UmqUUmq1jWMo9z0nhLVJgiWqFaXU/UqpfUqpTKVUtFLqQ6WU\nl63jqiKXPQid1rq91noDgFLqOaXUwks9llLKUSm1VCkVaf5SLpEsKKVeVUolKKXilVKvVhReOXFP\n1Fq/dCn7VnL9FWfNa2FlZZ4rpdTDSql/lFI5SqkFpazvq5Q6pJTKUEr9pZQKLrKujlJqgVIq1fxZ\nfazMALRepLUeWGTfy/5xUZ7SkvBKvOeEsCpJsES1oZSaDrwCTAc8geuAEOAPpZTDJRzP3roRWp2y\ndQCl2AjcDcQUX6GUeggYCnQAQoEhSqnxVRRHdTw3tdE54EXg8+IrlFJ+wDLgacAX2AksLrLJ80Az\nIAi4EXhSKdW/kuVecoJcyc+1upwyhLAKrbU85GHzB+ABpAO3FlvuBpwH7jc//wJ4ocj63sCZIs8j\ngSeBvUA2YFdKWUZgInAUSAVeAJoCm4EU4DvAocj244BjQAKwAmhQZN1NwCEgGZgHrAPGFlk/FjgI\nJAK/AsHF4mhaSnzhwL4iz/8EthV5vhEYWuT13ggMAHLNj3Rgt3n9WvPr2wSkAasB30pcjzNAr2LL\n/gYeLPbaNpexf2/zMaaZr9+5C9ewjOv4BBANnAXGAAUXzg2mL/eV5mu11fx6NhTZtzXwu/kcHwJu\nK1bO+8DP5te/BWhSXszFlkUCN5r/fg5TgvGV+Vj7gc7Fty3nWtwPnDDvewK4q4w4uprfi8nm8zYP\ny/ejEXgI0/s3EXi/yDo74A0gHjgOTDKfyxKfg2JlvggsKLZsHLCpyHNXIAtoaX5+FuhbZP0LwKIy\njn8fsNH893rza8gwn4vbzMuHALvNr3sT0KG8zzXwlPk1pgH/AsOLvB+ygXzz+U8q4z1X3ue6vHPc\nDNPnPAWIA7611v9BedSuh9RgieqiB+AELC+6UGudiSkxuamcfYv/Ur0TiAC8tdbGMvYZAHTCVEv2\nJDAfuAvTr/EO5r9RSt0IvAyMBBoAUZgSMJRS/sD3wCzAH9OXZs8LBSilhgMzgOFAAKbE6NtyXscF\nW4BmSilf86/1dkCgUspNKeUMXANssDgBWv9mjnOx1tpDa92pyOq7MH3BBWA6x49XIobStMP0BXfB\nXvOystTHlDg3BB4EPiituVcpNRBTItYXaAH0K7bJh5i+2OsBD2BK7C7s64opufo/TNfgLuBDpVSb\nIvvfiSk58sZ0jS6nafJmYBHgBfwEfFDiAKVcC3Oc7wIDtNaemN7ve8ooowCYiimx7I4paZtUbJvB\nmN4HYcDtRWqOxgODgI5AF0zv20tlcb211lmYzl87pZQ3puu6r8j2Fb0ftPk4vc3PO2itPbXWS5VS\nnTHVoo3D9LrnAyuVUo5F9i/+uT4O9DSfz+eB/1NK1dNaHwYmAFvM59+3eCDlfa6LKOscvwj8prX2\nBhphSoCFKEESLFFd+AMJZSREMeb1lfWu1jpaa51bzjZztdaZWutDmH79/q61Pq21TseU0F1IUEYB\nn2ut92qt84GZwHXmvigRwAGt9XKtdYHW+h0gtkgZ44FXtNZHza9rLhCmlAoqL3hz3DuAXpi+JPdh\n+kXfE1NCeFRrnXIR5+MLrfUJ83GXYPrCuBTumGqRLkg1LytLHvCi+dz8iqnGolUp291mjvGQ1job\nmIO5idDcOXsE8KzWOkdrfQBTDdIFQ4BIrfVCbbIHU7NW0cTiB631TvM1+IZLf/1gqtH5TWutga8x\nNZVWVgHQQSnlrLU+b37vlaC13qW13m5+PVHAJ5hq14p6RWudrrU+g6mW8sJrug14x/z+T8HU5H6p\nil9vzM89zOs0Jd8PHhdx/KLNwA8CH2utd5hf99eYagCvK7KNxedaa71Ma33e/PdSTLVR3SpZdmmf\n6+5F+5hR9jnOB0KUUoFa6zyt9eaLeM3iKiIJlqguEgD/Mu52amBeX1lnK7FNXJG/szE1YxV9fiFx\naAicvrDCXKOWBASa1xW/U67o8xDgXaVUklIqCVNTgzbvW5ENQB9MSdY68yMc0xft+krsX1TRpC+L\n8pOi8mRg6ht3gad5WVkSiyXMZZVd/DyeLvJ3AGCP5TUtuj4EU8KbZH4kY/ryrFdkG2u9/tKO5VyZ\nO/TMtT93YGqajlFK/aSUKi3ZRCnVwrw+RimVgqnGrfgPjKLv16KvqbxzebGKX2/Mz9PN6xQl3w/p\nl1hWCDC92HVshOn1XGDxuVZKjVZK7VZKJZu3b0flf4iV9rlOxPKzWdY5fgLTd+d2pdR+pdSYSpYp\nrjKSYInqYgumX6wjii5USrlhqin607woE1NfkAsalHIsa3Zujcb0z79oPH6Y+sbEAMHFti9aO3UG\neEhr7Wt++Git3bXWWytR7npMCdUN5r83YEquelF2glXVnXoPYGp6uiDMvOxyxWB53kL477XEA4Zi\n64ue8zPAumLn2FNr/cglxGHx3jI3zwZcwnGglGuhtf5Da90fU9PpEeDTMvb9CFNfsmbmZqinqXyn\n/9LO5aU6QJHaPvN7vxnwr7l2LAbL90NHLv39cAZ4qZTPStFO9YXn1FzT9Akwybytj7lsVXzbMpT1\nua7wx5nWOk5rPV5rHYipKfLDqrwjUtRckmCJakFrnYapk+w8pdQApZSDUqoxpiatKEx9bMDUb2WQ\nUspHKVUfeLSKQ1sEjFFKhSqlnDD129hqbrpZBbRVSg1XStkrpR7F9OV5wcfALKVUWwCllJdSqrJ9\nYjZjak7rBmzXWh/E9IVwLcX6XxVxHmislLrkO/DMt947m586mV/zBQuBaUqphkqphpj6TX1xqWUV\nsQS4XynVxtxXafaFFeYasB+AOUopF/O5vK/Ivj8DLZVS95jfM45KqS5l1Q5V4CimGqkI812rzwB1\nKtinrHNtcS2UUnWVUjebX18+phogQxn7egBpWusspVRrTLVelbUEmKKUClRK+WDqCF528Kb3rTOm\nWkIHpZRTkbv0lmPqb3WL+X0wG9irtT5mXr8QeEYp5W2OcxyVfz/EYrqx5IJPgQlKqW7muNyUUoPM\niU9p3DB1RE9QStmZa5HaF1l/HmhUrA9XUWV9rsscu+0CpdRIpdSFmq4UcxwFFe0nrj6SYIlqQ2v9\nOqYO429g6s+xBVM1fj9zPwkw9XvZB5zCdEdc8Y6planFKb5NeeM1rQGexfQlfw5ogqmzLVrrREx9\nXl7F1ITZDFNfqQv7rsDU7+o7c1PPPmBg0cOXU24Wptvi/9VaX/gi3gKc0loXbS4teoylmL7wE5VS\nOyoqowxHMNXkNMR0frMu9EvRWs/H1LF7v/m1/KS1LqsWpjSlxqK1Xg28A6zBlOT8VWyTyZiSjhhg\ngflxYd8MoD+maxJtfszF1Jn/opiT/EmYOlufxdTcVVGNhi7j7+LXQmEafuQcpvdKL0p2XL/gceBu\npVQaps7eFb3Hiz7/FPgNU4fzHZj6o5XnGUzNX09hGp4jC1ONGeb32a2Yko8kTHc33llk3+eAk5g+\no2uBV7XWf1RQ3gVzgIXm5sCRWuudmBK0983N6UexTKQtXrO5/9qbmO4qjcXUPLipyCZrMNVoxSql\ninYHuLB/mZ/r0sorpiu
|
||
|
|
"text/plain": [
|
||
|
|
"<matplotlib.figure.Figure at 0x11156e160>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"# %load solutions/sol_112.py\n",
|
||
|
|
"ann = ANN(2, 10, 1)\n",
|
||
|
|
"%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=100)\n",
|
||
|
|
"plot_decision_boundary(ann)\n",
|
||
|
|
"plt.title(\"Our model with 10 hidden units and 100 iterations\")\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {
|
||
|
|
"slideshow": {
|
||
|
|
"slide_type": "subslide"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"source": [
|
||
|
|
"# Addendum\n",
|
||
|
|
"\n",
|
||
|
|
"There is an additional notebook in the repo, i.e. [A simple implementation of ANN for MNIST](1.4 (Extra) A Simple Implementation of ANN for MNIST.ipynb) for a *naive* implementation of **SGD** and **MLP** applied on **MNIST** dataset.\n",
|
||
|
|
"\n",
|
||
|
|
"This accompanies the online text http://neuralnetworksanddeeplearning.com/ . The book is highly recommended. "
|
||
|
|
]
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"metadata": {
|
||
|
|
"anaconda-cloud": {},
|
||
|
|
"kernelspec": {
|
||
|
|
"display_name": "Python 3",
|
||
|
|
"language": "python",
|
||
|
|
"name": "python3"
|
||
|
|
},
|
||
|
|
"language_info": {
|
||
|
|
"codemirror_mode": {
|
||
|
|
"name": "ipython",
|
||
|
|
"version": 3
|
||
|
|
},
|
||
|
|
"file_extension": ".py",
|
||
|
|
"mimetype": "text/x-python",
|
||
|
|
"name": "python",
|
||
|
|
"nbconvert_exporter": "python",
|
||
|
|
"pygments_lexer": "ipython3",
|
||
|
|
"version": "3.4.3"
|
||
|
|
},
|
||
|
|
"nbpresent": {
|
||
|
|
"slides": {
|
||
|
|
"5445cab1-b2b9-4492-a3be-c4063bd67610": {
|
||
|
|
"id": "5445cab1-b2b9-4492-a3be-c4063bd67610",
|
||
|
|
"prev": "ef2af7e1-6294-42cf-a434-b1e17cf679a7",
|
||
|
|
"regions": {
|
||
|
|
"4baa3e75-a346-47d1-a015-c9040545adbd": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.4,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.5
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "8c3060aa-fee9-438c-bc60-4685c0eb4750",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "4baa3e75-a346-47d1-a015-c9040545adbd"
|
||
|
|
},
|
||
|
|
"bde8ff6d-7eb8-4d83-b6ba-f57b94dc889d": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "3896ecb2-47fa-4dbc-a2ff-9c76e9dfe93e",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "bde8ff6d-7eb8-4d83-b6ba-f57b94dc889d"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"d263b7c7-85dc-422a-aec7-fa2c09bc1d23": {
|
||
|
|
"id": "d263b7c7-85dc-422a-aec7-fa2c09bc1d23",
|
||
|
|
"prev": "ff705989-f5c1-46a1-8f7a-0fe14a949357",
|
||
|
|
"regions": {
|
||
|
|
"053c599b-2e88-4270-bc7b-1299a2a2bd20": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "4fa2e86a-be32-4e78-96d9-f511a07e3908",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "053c599b-2e88-4270-bc7b-1299a2a2bd20"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"e3153a08-42da-4785-a596-7a9ebb6a6f19": {
|
||
|
|
"id": "e3153a08-42da-4785-a596-7a9ebb6a6f19",
|
||
|
|
"prev": "5445cab1-b2b9-4492-a3be-c4063bd67610",
|
||
|
|
"regions": {
|
||
|
|
"215ebdb3-236e-4a0d-aa65-6b4fb839e5b3": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.4,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.5
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "2f1c6299-954a-461a-b5c1-a86a1d12ad15",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "215ebdb3-236e-4a0d-aa65-6b4fb839e5b3"
|
||
|
|
},
|
||
|
|
"6b33306d-283d-47ef-afec-1f8b5e7fa9a5": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "6287766f-972f-4b4d-bee7-5418f0af74de",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "6b33306d-283d-47ef-afec-1f8b5e7fa9a5"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"edcbed78-ec6f-4c1d-84f0-fb1601a07f1d": {
|
||
|
|
"id": "edcbed78-ec6f-4c1d-84f0-fb1601a07f1d",
|
||
|
|
"prev": "f912a9b4-1188-440a-b13e-86224d1dada5",
|
||
|
|
"regions": {
|
||
|
|
"b619970a-3b72-4add-88af-158f75e81f2f": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "356d5ec7-3392-4daa-9671-4cc7111c5c91",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "b619970a-3b72-4add-88af-158f75e81f2f"
|
||
|
|
},
|
||
|
|
"d86b2236-e08b-44cf-9298-0c818a1c9c88": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.4,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.5
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "5678486b-caf4-440b-be62-2f1286982c71",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "d86b2236-e08b-44cf-9298-0c818a1c9c88"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"ef2af7e1-6294-42cf-a434-b1e17cf679a7": {
|
||
|
|
"id": "ef2af7e1-6294-42cf-a434-b1e17cf679a7",
|
||
|
|
"prev": "f59d310a-ea9f-4360-8151-6ea1c66a66ac",
|
||
|
|
"regions": {
|
||
|
|
"e33b4032-a118-4ab7-920b-5cd4c1bb0523": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "24ca50a0-b9ad-425d-a28f-6bd3ca3ac378",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "e33b4032-a118-4ab7-920b-5cd4c1bb0523"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"f59d310a-ea9f-4360-8151-6ea1c66a66ac": {
|
||
|
|
"id": "f59d310a-ea9f-4360-8151-6ea1c66a66ac",
|
||
|
|
"prev": null,
|
||
|
|
"regions": {}
|
||
|
|
},
|
||
|
|
"f607e60a-8fc2-4061-bbbe-fb75f498cec7": {
|
||
|
|
"id": "f607e60a-8fc2-4061-bbbe-fb75f498cec7",
|
||
|
|
"prev": "d263b7c7-85dc-422a-aec7-fa2c09bc1d23",
|
||
|
|
"regions": {
|
||
|
|
"cea17e10-48c1-4b1a-a39e-5462b17adc89": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "df0121bc-10f1-4ace-840e-6fc89c6fdc7f",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "cea17e10-48c1-4b1a-a39e-5462b17adc89"
|
||
|
|
},
|
||
|
|
"fe327edc-53cc-4a66-b242-ffbc1148099f": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.4,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.5
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "c25d7194-10bd-4196-9d4c-592bf6e188f9",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "fe327edc-53cc-4a66-b242-ffbc1148099f"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"f912a9b4-1188-440a-b13e-86224d1dada5": {
|
||
|
|
"id": "f912a9b4-1188-440a-b13e-86224d1dada5",
|
||
|
|
"prev": "f607e60a-8fc2-4061-bbbe-fb75f498cec7",
|
||
|
|
"regions": {
|
||
|
|
"213f5824-f494-4cf6-8458-5dd5d629794c": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.4,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.5
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "92d4603e-7e39-4156-818c-785df6189fe8",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "213f5824-f494-4cf6-8458-5dd5d629794c"
|
||
|
|
},
|
||
|
|
"b141fdfe-4270-4293-a445-2b5642daf56e": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "15260f90-13d1-4fcc-afc6-379c507cb950",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "b141fdfe-4270-4293-a445-2b5642daf56e"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"ff705989-f5c1-46a1-8f7a-0fe14a949357": {
|
||
|
|
"id": "ff705989-f5c1-46a1-8f7a-0fe14a949357",
|
||
|
|
"prev": "e3153a08-42da-4785-a596-7a9ebb6a6f19",
|
||
|
|
"regions": {
|
||
|
|
"fe64f274-127a-4ab1-9a01-d011d67aa8fb": {
|
||
|
|
"attrs": {
|
||
|
|
"height": 0.8,
|
||
|
|
"width": 0.8,
|
||
|
|
"x": 0.1,
|
||
|
|
"y": 0.1
|
||
|
|
},
|
||
|
|
"content": {
|
||
|
|
"cell": "5e13607b-3ec5-4a95-a2d8-f898f20748da",
|
||
|
|
"part": "whole"
|
||
|
|
},
|
||
|
|
"id": "fe64f274-127a-4ab1-9a01-d011d67aa8fb"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"themes": {
|
||
|
|
"default": "cdfd905a-2df0-447c-8f49-becf28e8e1b1",
|
||
|
|
"theme": {
|
||
|
|
"cdfd905a-2df0-447c-8f49-becf28e8e1b1": {
|
||
|
|
"id": "cdfd905a-2df0-447c-8f49-becf28e8e1b1",
|
||
|
|
"palette": {
|
||
|
|
"19cc588f-0593-49c9-9f4b-e4d7cc113b1c": {
|
||
|
|
"id": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c",
|
||
|
|
"rgb": [
|
||
|
|
252,
|
||
|
|
252,
|
||
|
|
252
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"31af15d2-7e15-44c5-ab5e-e04b16a89eff": {
|
||
|
|
"id": "31af15d2-7e15-44c5-ab5e-e04b16a89eff",
|
||
|
|
"rgb": [
|
||
|
|
68,
|
||
|
|
68,
|
||
|
|
68
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"50f92c45-a630-455b-aec3-788680ec7410": {
|
||
|
|
"id": "50f92c45-a630-455b-aec3-788680ec7410",
|
||
|
|
"rgb": [
|
||
|
|
155,
|
||
|
|
177,
|
||
|
|
192
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"c5cc3653-2ee1-402a-aba2-7caae1da4f6c": {
|
||
|
|
"id": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
|
||
|
|
"rgb": [
|
||
|
|
43,
|
||
|
|
126,
|
||
|
|
184
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"efa7f048-9acb-414c-8b04-a26811511a21": {
|
||
|
|
"id": "efa7f048-9acb-414c-8b04-a26811511a21",
|
||
|
|
"rgb": [
|
||
|
|
25.118061674008803,
|
||
|
|
73.60176211453744,
|
||
|
|
107.4819383259912
|
||
|
|
]
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"rules": {
|
||
|
|
"blockquote": {
|
||
|
|
"color": "50f92c45-a630-455b-aec3-788680ec7410"
|
||
|
|
},
|
||
|
|
"code": {
|
||
|
|
"font-family": "Anonymous Pro"
|
||
|
|
},
|
||
|
|
"h1": {
|
||
|
|
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
|
||
|
|
"font-family": "Lato",
|
||
|
|
"font-size": 8
|
||
|
|
},
|
||
|
|
"h2": {
|
||
|
|
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
|
||
|
|
"font-family": "Lato",
|
||
|
|
"font-size": 6
|
||
|
|
},
|
||
|
|
"h3": {
|
||
|
|
"color": "50f92c45-a630-455b-aec3-788680ec7410",
|
||
|
|
"font-family": "Lato",
|
||
|
|
"font-size": 5.5
|
||
|
|
},
|
||
|
|
"h4": {
|
||
|
|
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
|
||
|
|
"font-family": "Lato",
|
||
|
|
"font-size": 5
|
||
|
|
},
|
||
|
|
"h5": {
|
||
|
|
"font-family": "Lato"
|
||
|
|
},
|
||
|
|
"h6": {
|
||
|
|
"font-family": "Lato"
|
||
|
|
},
|
||
|
|
"h7": {
|
||
|
|
"font-family": "Lato"
|
||
|
|
},
|
||
|
|
"pre": {
|
||
|
|
"font-family": "Anonymous Pro",
|
||
|
|
"font-size": 4
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"text-base": {
|
||
|
|
"font-family": "Merriweather",
|
||
|
|
"font-size": 4
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"nbformat": 4,
|
||
|
|
"nbformat_minor": 0
|
||
|
|
}
|