2015-04-12 02:01:49 +08:00
{
2015-04-15 02:24:23 +08:00
"cells": [
2015-04-12 02:01:49 +08:00
{
2015-04-15 02:24:23 +08:00
"cell_type": "markdown",
"metadata": {},
"source": [
"# scikit-learn"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* Machine Learning Models Cheat Sheet\n",
"* Estimators\n",
"* Introduction: Iris Dataset\n",
"* Overview: K-Nearest Neighbors Classifier\n",
"* Support Vector Machine Classifier\n",
"* Support Vector Machine with Kernels Classifier"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import seaborn; \n",
"from sklearn.linear_model import LinearRegression\n",
"from scipy import stats\n",
"import pylab as pl\n",
"\n",
"seaborn.set()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Machine Learning Models Cheat Sheet"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:33:28 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAACEoAAAUrCAYAAAA3g9BsAAAAAXNSR0IArs4c6QAAAAlwSFlzAAAG\nbgAABm4BRLh3LwAAAAd0SU1FB90EEw8XG1Hi8acAAAAGYktHRAD/AP8A/6C9p5MAC5xvSURBVHja\n7N0HfBR1+sfxv3eo2M7ee+9nOT0PxcKJZzlUVFA5sYsKdk9PTz3RS68EAklIQhopEAgJkEKAAAkh\nBFJJQJoo0gWVTrb//vuMbm53szUkkIQPr9f79dJkZnZ2ZrKZ7PPd5/k/pVQLAAAAAAAAAAAAAADA\nkeD/OAgAAAAAAAAAAAAAAICgBAAAAAAAAAAAAAAAAEEJAAAAAAAAAAAAAAAAghIAAAAAAAAAAAAA\nAAAEJQAAAAAAAAAAAAAAAAhKAAAAAAAAAAAAAAAAEJQAAAAAAAAAAAAAAAAgKAEAAAAAAAAAAAAA\nAEBQAgAAAAAAAAAAAAAAgKAEAAAAAAAAAAAAAAAAQQkAAAAAAAAAAAAAAEBQAgAAAAAAAAAAAAAA\ngKAEAAAAAAAAAAAAAAAAQQkAAAAAAAAAAAAAAACCEgAAAAAAAAAAAAAAAAQlAAAAAAAAAAAAAAAA\nCEoAwKFnsVhaDAZDy/79+3W7du3S/fTTT7qtW7fqN2zYoP/+++9brV+/3qUffvjBJVnflY0bN7ba\nvHmz/scff9QeUx573759uhbrP6PRqO0X5wcAAAAAAAAAAAAgKAHgCCehhp07d+o2bdqkX716tb6h\nocFQXV1tqKysNM6fP984e/ZsY1FRkWn69OmmqVOnmnJycswTJ040p6SkmMePH28eO3asJSYmxhIR\nEWEJDg5W//3vf7uswMBAFRoaqmRfo6OjLWPGjLGMGzfOkpCQYElOTjanpqZqzy07O9s8efJkU15e\nnva8CwsLTSUlJaY5c+YYFyxYYKyqqjLW1dUZmpubDWvWrNECHlu3btX98ssvWjhDghlcWwAAAAAA\nAAAAACAoAQCHqKODFOuls8J3332nl2K+BB/KysqMM2bM0IIOEyZMMEtIQEIDXTnY0J1JKEMCGaNH\nj7bEx8db5JhnZmaac3NztfCFBC/knEggpaamxrBs2TLDypUr9XLOpDPGjh07dHv27NHp9Xq6YQAA\nAAAAAAAAAICgBIAjlxTPZXSFdDSQbg/S5SE9Pd0sxfjIyEhLQEAAQYUeSEIt0gVDOmAkJSWZMzIy\nzJMmTTJPmzbNJB0/pNtFRUWFUUIx0hVkxYoVhm+//VYbTyKhGRlFcuDAAZ3ZbObnCAAAAAAAAAAA\nAAQlAHQt0kVgy5Yt+qamJoOMepAREImJiWY6QKAjyIgUCdXExsZaZHyKjByRcSNynUnoYu7cuVqn\ni9raWm28yNq1a/UbN25s7XIh41r4OQUAAAAAAAAAAABBCQB+kU/2//TTT7rVq1frq6qqjIWFhVpn\nCOkaQDEf3WG0iC1sIR0uJk6cqI0VkfEupaWlxvLycq27RWNjo2HVqlV66YKydetW3c6dO3UtLS06\nxokAAAAAAAAAAAAQlADQg+3bt0+3cuVKvYxHyMnJMcv4BCk0d9cieZB138NCQlR0eLiKjYpSCTEx\nKjk2VqXHx6us8eNV7oQJKj8tTRVOnKhmZWeruZMnq/KpU1XltGmdqsL6GPNyc9WcSZNUSVaW9vgF\n6ekqLyVFTU5OVtmJiSrDuo+p48apJOv+JowercZFR6vRkZHacwm3PqfgoCDF+JJDIywsTMXExFhk\nbIytq4WMEikuLjaVlZUZFy1aZJQRM8uXL9fGiGzatEkv4aK9e/fqjEYjry0AAAAAAAAAAAAEJQB0\nBfJJ+R9//FEnBd6CggLT2LFju1yHiNDgYC3gkDJunBYgKMzMVPOnTFE1M2aoFaWlat38+WpDebna\nsmiR2lFdrXbV1Kh99fVKv2yZsixfrtSKFT2e2fo8Ddbne6ChQe2tq1M7rcdAjsWWykr1/YIFavXc\nuaqppETVzpypFuXnq/m5uVooZEZGhpoyYYIWGJHjK2EMCWJEhIaq7hyO6ZKBnaAgFRUVZZGfseTk\nZHNmZqZ5ypQpppkzZ5pmz55trKioMC5ZssR6GpcZpHPL+vXr9du2bdPt2rVLZ/3XQlcLAAAAAAAA\nAAAAghIA2sFgMLR89913einKZmVlmcPDww95MEI6IESFhWmdHSYmJKhpqamqNDtb67LQUFSk1syd\nqzZXVmqBB2NT0xERdOiqTM3Nan99vfpl6VK1rapKC6SsLStTy0tLVX1hoaouKNC6b8zOydE6Ysi5\nzElKUmlxcSpxzBg1NjpaO9fS+YKwxMH/3ERERLQGLeTnVzpalJSUmBYsWKCFLJqamgxr167Vuln8\n8ssvjA0BAAAAAAAAAAAgKAEceXbv3q2TUQBSTE1KSjIfqi4BMu4iPiZG6wAhoywkACFFdul2cKR0\neYAjOe8tjY1qd22t2r54sdq0cKHWDWTl7NmqsbhYLZ0xQy3My1Nl1uulOCtLFaSladePhGlkVErc\nqFFqVESENkaFcSP+BSwiIyO1gMWECRO0kSH5+fmmWbNmGcvLy41Lly41yKidzZs36/fs2aMzm828\ndgIAAAAAAAAAAIISALoPGaMhnyyfOnWqKSYmptO7RcRERGiFbClsL5k+XX07b57WfYAwBDqbrrFR\n6zoinS5+KC/XxossKynRAhcVeXlal4uZEydqo0Uyfwtb2DpcBDFWxGOwIjo62iLBqpycHHNhYaFJ\nAhX19fVatwoZB7J//34dr7cAAAAAAAAAAICgBIDDQtrqS5v9OXPmGGNjYzslGBEeEqKSx47VPuEv\nBegVpaVacdrAWAx0YzLWRTqc7Kiu1rpbSMhHru26wkJVlZ+v5uXmagGgfOt1n5OYqFLj4rRxMRIO\nCg0OPuIDFUFBQWr06NGWlJQUc25urjYCpLKy0rhs2TKDjPfZsWOHzvqP12kAAAAAAAAAAEBQAkDH\nhCPWr1+vl8JkR3eNCA4KUmlxcdr4gzVlZWp/fT1FdcDdKJGGBrVz6VK1ddEitX7BArVqzhxtjIh0\nVymfOlWV5uSoGRkZKnfCBJURH6+SYmNVbFSUiggNVYFHyPiQUOtzTUhIsEyePNk0e/ZsY01NjdaZ\n4ueff9aZTCZe0wEAAAAAAAAAAEEJAK5JQVGKizNnzjRFRkZ2WDgiOjxcG01QPX262rxwoTI3N1ME\nBw4Rw7Jlak9dndq+eLHaWFGh1paVqeZZs1TtzJmqcto0NXfyZFWUmanyUlNV1vjx2vgQCVqEh4b2\nmDEf0pUiIyPDPGPGDNPChQuNy5cvN2zevFl/4MABxnsAAAAAAAAAAACCEsCRxmAwtKxcuVKfn59v\nCg8Pt3REUXL86NFa4bWppETtrKmhWA10444WBxoa1E9LlmijQ6QDzDLrz7WEnuZPmaKNDZGARWZC\ngtbJYkxkpAoLCelWQQp53UtMTDRPmTLFJOOF6urqDOvWrdPv3LlTZzab+T0BAAAAAAAAAAAISgA9\ngU6na2lubjZIYTDkIIuaocHBamJCglowZYpaN2+e0jU2UmAGjnDm5cvVvvp6taO6Wm2oqFCr585V\ndYWF2uvEzIkTVXZiohaoigwL69IhisDAQBUXF2eR18r58+drnSi2b9/OOA8AAAAAAAAAAEBQAugO\npLDX1NRkyM7ONgcFBR1U8TBxzBi1MC9PbV20SPvEOYVhAO1lam5WO5cuVRvKy9WK0lJVXVCgZufk\nqGmpqSotLk4bAxJ8kK9ZnRWgyM3NbQ1Q/PjjjwQoAAAAAAAAAAAAQQmgK9i/f7+uoqLCGB0dfVBj\nNSaMHauq8vO1gibFXQCHWktDg/px8WKtc01DUZGqyMvTRvzkJidrYz/Cu8DIDwIUAAAAAAAAAACA\noARwGG3btk03Y8YMU3BwcLsKfgEBASo9Pl4tnTFD7amtpVALoMs70NCgtlRWquWlpapy2jQ1MyND\nex0bHRmpvaYRoAAAAAA
"text/plain": [
"<IPython.core.display.Image object>"
]
},
"execution_count": 2,
"metadata": {
"image/png": {
"width": 800
2015-04-12 04:33:28 +08:00
}
2015-04-15 02:24:23 +08:00
},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import Image\n",
"Image(\"http://scikit-learn.org/dev/_static/ml_map.png\", width=800)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Estimators"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Given a scikit-learn *estimator* object named `model`, the following methods are available:\n",
"\n",
"- Available in **all Estimators**\n",
" + `model.fit()` : fit training data. For supervised learning applications,\n",
" this accepts two arguments: the data `X` and the labels `y` (e.g. `model.fit(X, y)`).\n",
" For unsupervised learning applications, this accepts only a single argument,\n",
" the data `X` (e.g. `model.fit(X)`).\n",
"- Available in **supervised estimators**\n",
" + `model.predict()` : given a trained model, predict the label of a new set of data.\n",
" This method accepts one argument, the new data `X_new` (e.g. `model.predict(X_new)`),\n",
" and returns the learned label for each object in the array.\n",
" + `model.predict_proba()` : For classification problems, some estimators also provide\n",
" this method, which returns the probability that a new observation has each categorical label.\n",
" In this case, the label with the highest probability is returned by `model.predict()`.\n",
" + `model.score()` : for classification or regression problems, most (all?) estimators implement\n",
" a score method. Scores are between 0 and 1, with a larger score indicating a better fit.\n",
"- Available in **unsupervised estimators**\n",
" + `model.predict()` : predict labels in clustering algorithms.\n",
" + `model.transform()` : given an unsupervised model, transform new data into the new basis.\n",
" This also accepts one argument `X_new`, and returns the new representation of the data based\n",
" on the unsupervised model.\n",
" + `model.fit_transform()` : some estimators implement this method,\n",
" which more efficiently performs a fit and a transform on the same input data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction: Iris Dataset"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:33:28 +08:00
{
2015-04-15 02:24:23 +08:00
"name": "stdout",
"output_type": "stream",
"text": [
"['target_names', 'data', 'target', 'DESCR', 'feature_names']\n",
"(150, 4)\n",
"(150, 4)\n",
"(150,)\n",
"['setosa' 'versicolor' 'virginica']\n",
"['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\n"
2015-04-12 04:33:28 +08:00
]
2015-04-15 02:24:23 +08:00
}
],
"source": [
"from sklearn.datasets import load_iris\n",
"iris = load_iris()\n",
"\n",
"n_samples, n_features = iris.data.shape\n",
"print(iris.keys())\n",
"print((n_samples, n_features))\n",
"print(iris.data.shape)\n",
"print(iris.target.shape)\n",
"print(iris.target_names)\n",
"print(iris.feature_names)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:33:28 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFkCAYAAAD165gcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8VFXawPHfnZn0TgihBEKTQw9NmggoNoqoWFcFG4pd\ndxXL7rvuru66urq2VXTXBoJYEUWKoii9KL2FAwSQDiEhvc7Mff+YEMpMkiHJJJPh+frhY+7k3HOe\nk/bMPefecwzTNBFCCCGEf7DUdwBCCCGEOEkSsxBCCOFHJDELIYQQfkQSsxBCCOFHJDELIYQQfkQS\nsxBCCOFHbL6qWCllAd4DOgBO4G6ttfZVe0IIIUQg8OUV82VAhNZ6EPAs8A8ftiWEEEIEBF8m5kIg\nRillADFAiQ/bEkIIIQKCz4aygWVAKLANiAeu9GFbQgghREAwfLUkp1Lqj7iGsv+klEoCfgK6aq09\nXjmbpmkahuGTWIQQQpw1n/1B7tVvUrUTz9pV9wd8ovDlFXMEkFP28XEgCLBWVNgwDNLTc30YTt1J\nSIiSvvgh6Yt/kr74p4SEqPoO4Zzly8T8EvChUmoJrqT8tNa60IftCSGEEA2ezxKz1joLuMZX9Qsh\nhBCBSBYYEUIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyI\nJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkII\nIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hi\nFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDCj0hiFkIIIfyIJGYhhBDC\nj0hiFkIIIfyIzZeVK6VuA24vOwwDUoBErXWOL9sVQgghGiqfJmat9RRgCoBS6k3gPUnKQgghRMXq\nZChbKdUH6KK1fq8u2hNCCCEaKp9eMZ/ij8BfqyqUkBDl+0jqiPTFP0lf/JP0RYiTfJ6YlVKxQAet\n9aKqyqan5/o6nDqRkBB1TvVl27btrF23mT69u9OhQ/s6iuzsnWvfl4ZC+uKf5A1G/amLK+bBwII6\naEfUg29mzed/762j1NGU6Z9+yX0T+jJyxMX1HZYQQjRYdTHH3AFIq4N2RD34etY67M7mGIaFUkcL\nvv5mdX2HJIQQDZrPr5i11i/7ug1Rf5xO84zjegpECCEChCwwImpk2NDzMMgEwEIGwy5W9RyREEI0\nbHV1V7YIUOPGXUty8nK2pu6mW9cBDBrUr75DEkKIBk0Ss6ixIUMGMmTIwPoOQwghAoIMZQshhBB+\nRK6Yhd9ZuuwXvvt+DTYr3H77SFont6rvkASwctUaZs/5BasFxo29gnbt2tR3SEJUSil1OdBKa/2u\nF2UTgWe01g9U8PkUYLTW+rlaDtONJGbhV9au28QL//qZUkdzALbpybz3v98TGSmLHdSnLVu38fwL\nP1Bc2gKArdum8t9JD9KoUaN6jkyIimmtvz+LskcAj0m57PMbgA21EVdVJDELv7J4ydrypAyQnpnI\nihW/cumlsmhJfVq4cHV5UgbIykli8eKVXH31iHqMSojTKaVmAK9rrReX7dGwAJgEvAPMBo4Bc4FF\nwJtALnAUKMK1bPSnWusBSqmNwEKgO2ACVwG9gAla698ppe4C7gWswCyt9V+VUg8C1wARZe1co7Uu\nrU4/ZI5Z+JVGcZE4HEXlxxYjh5YtW1RyhqgLCY2jMZ0nvy8G2SQny/dF+J13gdvKPr4D1z4NJyQC\nl2qtX8KVqG/TWg/DtQDW6QsyQBQwXWs9FDgADD9RRimVADwJDNJa9wKClVJRQCPgEq11f1wXvedX\ntxOSmIVfueXmq+mdkg3OPVgtuxhzVQs6dqzZs9G7du3hD4+9wg03Pcsrr36AU1ZBOWvXXTeKvr3z\nwbkHi5HGqOHx9OyZUt9hCXGm+UBfpVQcMAgoPOVzu7XW9rKPm2mtU8s+XgIYHupaV/b/fUDoKa+3\nBTZrrYsBtNZ/1FrnAqXAJ0qp94AkajAiLUPZwq9YrVZefOFxcnKyCQoKJiwsrEb1mabJ3577iMPp\nrhuVtu8sJCLiMybc87vaCPecYbFY+Ptzj5KTk43NFkR4eHh9hySEG621Uyn1Ba4r4pmA45RPn/qO\nfJ9SqlNZch5QQXVnXkWfkAZ0VEoFa61LlFKfAW8BV2mt+yulwoHV1ODCVxKz8EvR0TG1Uk9ubg5H\njlrK3w9brGHs2p1ZK3Wfi2rr+yKED30I7AQmAhdxMsGemmjvBz5QSuUBJcB+D2VOVV6H1vqYUupF\nYJFSygRmAb8C+Uqpxbjml9cCzarbAUnMIqBFRUUTG2PneI7r2OksJbFJza7ChRD+S2u9DwgpO5xy\nyqdOXQWpL3BlWZJ9DijWWv92oozWuvxZQK3106ect6jstSln1A0wrHZ6IHPMIsAZhsHEx8aQ1Gw/\n8XH76dsrlwcfuLW+wxJC1K8jwPyyK9wUXEPRfkOumEXA6927O++/2z2gNrEXQlSf1noGMKO+46iI\nXDELIYQQfkQSsxDCK6ZpcuTIYTIyMuo7FCECmgxlCyGq5HA4eOqPr7JufSkWq4NLL27GxMfvqu+w\nhAhIcsUshKjSZ599w/pNMViDkjAsycxfkMuvv66p77CECEiSmIUQVTqeXYDFcsriR0YUBw4eqb+A\nhAhgkphFg2WaJj8uWMisWfMoKiqq+gRRbZcO609oyL7y4/jY/Vw0dGAlZwghqkvmmEWDZJomE598\nmfUbI8AI4quvX+Ct/0wkIiKivkMLSB06tOfZZ67im2+XYbGY3Db2LmJiYus7LCECkiRm0SAtW7aS\ndRtCsQW59mk+eKQNU6d9zb0TbqnnyAJXSkoXUlK61HcYQgQ8GcoWDVJxcTGnv680sNtl1yghRMMn\niVk0SIMHX0Db5HRMpwPTNImN2sX1111W32EJIUSNyVC2aJCCgoL4zxtP8PH0rykpcTDmmvtITEys\n77CEEKLGJDGLBis0NJS77rypvsMQQohaJUPZQgghhB+RK2bhd1asWM1389dgtcAdt4+kZcukGtWX\nn5/PW5M+obTUoGvXllw1OvDnok3T5P33P2fP3uMktYjmnrtvwmKR9+FCNASSmIVf2bBhC8+/+CMl\n9hYAbN32Pu/99w9ERkZVu86JT77Gzt1JGIaFRctSsZfaufbaEbUVsl96+d/vM/8nJxZLBKtW53Ms\n47/83x/vq++whBBekLfQwq8sXLS6PCkDpGc0ZcXK6q/JnJeXy67dTgzD9aNuGI34dc2emobp97am\nZmKxhAFgsYSybVtWPUckhPCWT6+YlVJPA1cCQcCbWuspvmxPNHyxMeE4HNlYrSEAWI0cklo0rXZ9\nYWHhhIU6KCh2HZumSXhYbUTq38LDjUqPhRD+y2dXzEqpocAArfVAYCjQ1ldticBx661j6NEtE9O5\nF6uxm6tGNaNTp47Vrs9qtXLH7YMIC07DdB4gqdkeHnygZndym6bJG29M4b4HXuHxJ15jz569FZad\nMWMuDzz0Kg898grLl/9ao3bPxn0TRtEoZhcO+35io3dx7z0j66xtIUTNGKZp+qRipdTzgAl0AaKB\niVrrysYkzfT0XJ/EUtcSEqKQvlSfaZocP55JcHAIkZGRtVJnUVERwcFODCMMw6jZ1eO7733K51/l\nlA8VN03YzeQP/uxW7+IlK3n+xeWYJAAQFryXdybdRdOm1R8BOMGb74vdbicjI4P4+HhsNv+9nUR+\nX/xTQkKUz4ZZevWbVO3Es3bV/QE//OPLOeYEoDdwHXAv8LEP2xIBxDAMGjWKr7WkDK5nnhMTE2uc\nlAHSdmWUJ2WAw0cs5ObmuJVbv357eVIGyCtM5Ndf19e4fW/ZbDYSExP9OikLIdz58jf2GJCqtbYD\n25VSRUqpxlrrYxWdkJB
"text/plain": [
"<matplotlib.figure.Figure at 0x10d318b90>"
]
},
2015-04-12 04:33:28 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# 'sepal width (cm)'\n",
"x_index = 1\n",
"# 'petal length (cm)'\n",
"y_index = 2\n",
"\n",
"# this formatter will label the colorbar with the correct target names\n",
"formatter = plt.FuncFormatter(lambda i, *args: iris.target_names[int(i)])\n",
"\n",
"plt.scatter(iris.data[:, x_index], iris.data[:, y_index],\n",
" c=iris.target, cmap=plt.cm.get_cmap('RdYlBu', 3))\n",
"plt.colorbar(ticks=[0, 1, 2], format=formatter)\n",
"plt.clim(-0.5, 2.5)\n",
"plt.xlabel(iris.feature_names[x_index])\n",
"plt.ylabel(iris.feature_names[y_index]);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## K-Nearest Neighbors Classifier\n",
"\n",
"The K-Nearest Neighbors (KNN) algorithm is a method used for algorithm used for **classification** or for **regression**. In both cases, the input consists of the k closest training examples in the feature space. Given a new, unknown observation, look up which points have the closest features and assign the predominant class."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:33:28 +08:00
{
2015-04-15 02:24:23 +08:00
"name": "stdout",
"output_type": "stream",
"text": [
"['versicolor']\n",
"['setosa' 'versicolor' 'virginica']\n",
"[[ 0. 0.8 0.2]]\n"
2015-04-12 04:33:28 +08:00
]
},
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAFgCAYAAABEyiulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VFX6wPHvJJPeSELovV2qCCKgSAcLIChVVNTVXXVd\nXXV3rdtcd127a10Lrj8UKQJKV5AuRemhcwIJISQEEkhvM5ny+2NCCGSSDExP3s/z8Dw5M2fufe8w\nyTvn3HPfq7NarQghhBDCvwR4OwAhhBBCXDlJ4EIIIYQfkgQuhBBC+CFJ4EIIIYQfkgQuhBBC+CFJ\n4EIIIYQf0ns7gCuRnV0o17wJIYRoMBISonQ1PScjcCGEEMIPSQIXQggh/JAkcCGEEMIPSQIXQggh\n/JAkcCGEEMIPSQIXQggh/JAkcCGEEMIPSQIXQggh/JAkcCGEEMIPub0Sm6ZpTYDdwEilVFKVx58G\nHgKyKx56pOrzQgghhKiZWxO4pmlBwKdAsZ2n+wIzlFJ73RmDEEIIUR+5ewr9TeBjINPOc9cBL2qa\ntlnTtOfdHIcQQghRr7htBK5p2gNAtlLqR03TXgAuL8g+D/gIKAQWa5o2Vim10l3xXC5h4ypP7UoI\n4UcWMsXbIQg/NmxYocf25c4R+K+A0ZqmbQCuBb6sOB9+wXtKqRylVDmwEujjxliEEEKIesVtI3Cl\n1NALP1ck8UeUUlkV7Rhgv6Zp3YESYATwP3fFIoQQQtQ3nrwfuE7TtOlApFJqZsV57w2AAVirlJI5\nbSGEEMJBHkngSqnhF36s8tg8bOfBhRBCCHGFpJCLEEII4YckgQshhBB+yJPnwIUQwufIZWPCX8kI\nXAghhPBDMgIXwsMS9+8neelS9OXlBF1/PWNuv93bIQkh/JAkcCE86HxhIWc++YRJ584BkJqSwtb4\neAbdeKOXIxNC+BuZQhfCg5KSk+lbkbwB2hmNnE9O9mJEQgh/JQlcCA/q2K4dB2JjK9unAwOJbtPG\nixEJIfyVTKEL4UFNGjUi8qGH+HbZMoLKyzH37cudQ4fW/ULhMrLqXNQXksCF8LAB/fszoH9/b4ch\nhPBzMoUuhBBC+CFJ4EIIIYQfkgQuhBBC+CE5By6EqLdkwZqoz2QELoQQQvghSeBCCCGEH5IELoQQ\nQvghSeBCCCGEH5IELoQQQvghSeBCCCGEH5IELoQQQvghSeBCCCGEH5IELoQQQvghSeBCCCGEH5IE\nLoQQQvghqYUuRC3OnDvHjl9+ITI2luE33ohOp/N2SEIIAUgCF6JGqenpHHjzTW7PzCRHp2PugQPc\n8+ij3g5LCCEAmUIXokaJq1dze2YmOiDeaqXj1q1k5uZ6OywhhAAkgQvhMKtMnwshfIgkcCFqcO0t\nt7C0RQusQLZOR8qgQTSPjfV2WEIIAcg5cCFq1K5VK8L++ldWbN9OZKNG3H3DDd4OSQghKkkCF6IW\nTePjuX3MGG+HIYQQ1cgUuhBCCOGHZAQuhKg3FjLF2yEI4TEyAhdCCCH8kCRwIYQQwg/JFLrwaZt/\n+onc/fsxhodz87RpREdEeDsk4UNkylw0ZJLAhc/asmULTWbOZLDBgAX4Kj2d+//6V6lHLoQQyBS6\n8GG5+/ejGQyA7YPa9fhxsgsKvBuUEEL4CBmBC59lDA/HDARWtLOiorg2LMybIQkvkylzIS6SBC58\n1i3TpvFVejqdjx3jXGQk8VOmEBoc7O2whBDCJ0gCFz4rMiyMB/78Z3KLi4kMDSVYLx/XhkZG3ELU\nTP4iCp+m0+mIi4z0dhhCCOFzZBGbEEII4YckgQshhBB+yO1T6JqmNQF2AyOVUklVHr8d+CtgAr5Q\nSn3u7liEEEKI+sKtI3BN04KAT4FiO4+/A4wGhgIPVyR6IYQQQjjA3SPwN4GPgRcue7wbcFwplQ+g\nadoWYAiwyM3xCOE3rFYr3379NcH792MMCaH9xIlc17evt8MSQvgIt43ANU17AMhWSv1Y8VDV+pfR\nQH6VdiEQ465YhPBHP65ezYgVKxh/8iSTk5I487//kV9S4u2whBA+wp1T6L8CRmuatgG4FviyyjR5\nPhBVpW8UkOvGWITwO4bTp4mzWivbXbOzOZWV5cWIhBC+xG1T6EqpoRd+rkjijyilLvz1OQp01jQt\nFtv58SHYptuFEBWi2rcnIzCQlmYzAAdatGBUs2ZejkoI4Ss8WchFp2nadCBSKTVT07Q/AKuxzQL8\nTymV6cFYhPB5w4cPZ0VeHnsSEzGGhtJt4kQiQ0O9HZYQwkforFWm6Hxddnahy4JN2LjKVZsSQriJ\nlFIV/mbYsEKXbi8hIarG+ydLKVUhhM+QhC2E46QSmxBCCOGHZAQuhPAaGXELcfVkBC6EEEL4IUng\nokHarhRr9uzBZDJ5OxQhhLgqMoUuGpxXnn2WkamptAT+GRPDcx98QLhcniWE8DMyAhcNyrdr1zIp\nNZWBQHfghfx8PnjvPW+HJYQQV0wSuGhQzpw5Q/Mq7VAgoLTUW+EIIcRVkyl00aBMGz+e/1u9micN\nBnTA4oAABo4Z4+2wGgxZdS6E60gCFw1K4+hobn/tNd744AOCLRZ6TZjA4P79vR2WEEJcMUngosHp\n2LIlz732mrfDEEIIp8g5cCGEEMIPSQIXQggh/JAkcCGEEMIPyTlwIYTbyKpzIdxHErjwuBNZWbz3\nr38RYLEw9be/ZWCPHt4OyaMM5eX8uGoVVoOB6wYPpmXTpt4OSQi/l5KSxO7dikaNghk5cjQBAc5P\nMBsMJfzww1rMZhg+fCBxcU1cEKnrSAIXHnUmN5evn3iCt61WAoAP//EPAl56if7du3s7NI8wmc3M\nefNN7k1MJBhYumUL1ueeo1Xz5nW+Vghh36FD+3n//Xxyc28H8jl2bA6PPTbDqW2Wlxv497/nc+TI\nfUAgP/+8iBdfHOpTSVzOgQuPeuPDD/mj1UogoAMeB+Z/8omXo/KcfUlJjK5I3gATTp9m94YNXo1J\nCH+3ceMJcnMHV7Ri2LmzFSUl+U5tc8+enzlyZCK2ca6OtLTJbNjws7OhupQkcOFRwaGhlFRpmwBz\nYKC3wvG40JAQSqpM7VkAawM6fiHcITDQfElbrzcQGBjk1DZDQkLgkr9WZgIDdU5t09UkgQuPevnJ\nJ3ktKIgzQB7wj4AAXvjb37wdlsd0b9+eXcOGcSoggEJgdpcujBg3ztthucxCplzyTwhPGD/+Olq1\n+g4oITj4MKNHGwkJCXdqm717D2TQoDVAJpBP9+6zGTNmtCvCdRmd1Wr1dgwOy84udFmwCRtXuWpT\n4goZjUb+9cknGMrKeO6xx4iLjPR2SB5ltVrZdfAghYWF3NCvH2HBwXW/yE9I0hbeUlSUR2LiLpo1\na06nTq5ZGGu1Wtm37xcMBgN9+95AUFBIna8ZNqzQJfu+ICEhqsZhvyRwIYTLSAIXDZ0nE7isQhdC\nXDVJ2EJ4j5wDF0IIIfyQJHAhhBDCD0kCF0IIIfyQnAMXHndIKdT33xNosdB86FD69+vn9DZLDAZW\nzJ5NaE4O5tatGT9tGoE1lFJUyckcWr6cQLOZJoMGccPAgU7vXwghPE0SuPCoM+fPc+qDD5iYlQXA\n9iNHOPr883Tt1Mmp7X730UdM/+UXAoHCXbtYYjYz6d57q/U7n59P0vvvMzEzE4A9hw6xPyqKaxpY\nPXYhhP+TBC48ak9iIjdXJG+AAQUFLNu3z+kEHn3yJBfqmUUBwSdO2O2379AhhlUkb4C+RUUsO3hQ\nErgDZMW5EL5FzoELj2rTpg0qNLSyfSYggJhmzZzebmlMTOXPVqA0Otr+/lu3Jin8YoWm8zod4QkJ\nTu9fCCE8TUbgwqN6du7MD5MmkbR2LYFmM4aBA5kyaJDT2+1z333M++ILos+fJ7dlS262M30O0Kl1\na9ZMncp3q1YRZDJR3K8f04YPd3r/QgjhaVKJTXiF1WrFarW65J69VZktlhoXr12+f4vV6lDfhkqm\nzIW4clKJTdR7Op0Onc7
"text/plain": [
"<matplotlib.figure.Figure at 0x10d700bd0>"
]
},
2015-04-12 04:33:28 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"from sklearn import neighbors, datasets\n",
"\n",
"iris = datasets.load_iris()\n",
"X, y = iris.data, iris.target\n",
"\n",
"# create the model\n",
"knn = neighbors.KNeighborsClassifier(n_neighbors=5, weights='uniform')\n",
"\n",
"# fit the model\n",
"knn.fit(X, y)\n",
"\n",
"# What kind of iris has 3cm x 5cm sepal and 4cm x 2cm petal?\n",
"X_pred = [3, 5, 4, 2]\n",
"result = knn.predict([X_pred, ])\n",
"\n",
"print(iris.target_names[result])\n",
"print(iris.target_names)\n",
"print(knn.predict_proba([X_pred, ]))\n",
"\n",
"from fig_code import plot_iris_knn\n",
"plot_iris_knn()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Support Vector Machine Classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Support Vector Machines (SVMs) are a powerful supervised learning algorithm used for **classification** or for **regression**. SVMs draw a boundary between clusters of data. SVMs attempt to maximize the margin between sets of points. Many lines can be drawn to separate the points above:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAFVCAYAAADc5IdQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXd8G3d65/9BB9g7RVGFokSBRSySrGarN1uy1UnZluS6\n2b1k95JNsr/NZu+VzS+X7OVyv73b3OWSbdmsd23JtkiqWpKtZku2ZEsukkiKBaQoUZ29omMw8/tj\ngAFAgkQRMADJ5/168UXgO8DMdwhwPvOU7/NIOI4DQRAEQRCRQxrpCRAEQRDEZIfEmCAIgiAiDIkx\nQRAEQUQYEmOCIAiCiDAkxgRBEAQRYUiMCYIgCCLCyIN5k1ar/TGALQAUAP5Vp9P9IaSzIgiCIIhJ\nRMCWsVarXQ1gmU6nexrAagC5IZ4TQRAEQUwqgrGMNwKo02q1RwEkAPhhaKdEEARBEJOLYMQ4HcB0\nAC+At4qPA8gP5aQIgiAIYjIRjBh3A2jU6XQMgGatVmvWarVpOp2ue/gLOY7jJBLJE0+SIAiCIMYR\nAQtfMGJ8CcD3Afxcq9VOBRALoMfrbCQSdHUNBXGI6CI9PZ7OI0qYCOcATIzzmAjnANB5RBMT4RwA\n/jwCJeAELp1OdxLAda1W+yV4F/V3dToddZsgCIIgiCAJammTTqf7UagnQhAEQRCTFSr6QRAEQRAR\nhsSYIAiCICIMiTFBEARBRBgSY4IgCIKIMCTGBEEQBBFhSIwJgiAIIsKQGBMEQRBEhCExJgiCIIgI\nQ2JMEARBEBGGxJggCIIgIgyJMUEQBEFEGBJjgiAIgogwJMYEQRAEEWFIjAmCIAgiwpAYEwRBEESE\nCaqfMRGdDA4M4ItfXoKqXgG7hoV6QyyeLl8OiUQS6akRBEEQY0BiPEHo6ejB1b2fYl/tXsggAwC0\nf9COE9ePYss/7ojw7AiCIIixIDf1BOHLf/4cr9a+IggxAEyxT8FT785HS01zBGdGEARB+ILEeIIQ\nU6OCBCPd0SXGYrSdaI3AjAiCIAh/ITGeIHCjfJIcOHBSTtzJEARBEAFBYjxBMC+2wg77iPGriVeR\nX1EUgRkRBEEQ/kJiPEFY/oPV+O3Tv4MBBmGsQdOI29+5hxlzZkZwZgRBEIQvKJt6ghAXF4cXKnfi\n5P4z4K7bwWpYTNmRjQ3LNkV6agRBEIQPSIwnEEqlEmveXB/paRAEQRABQm5qgiAIgogwJMYEQRAE\nEWFIjAmCIAgiwpAYEwQxoeA4DkNDg2AYJtJTIQi/oQQugogwDMPg0oGLsF+xgZMC6jUxeGbXCmrw\nEQQXfnke7CEb0u+loT91APoNRqz/yXNQKBSRnhpBjAmJMUFEEJvNhmNvVGHfmT2IRzwAoLO6E0cu\nVmHHv1SQIAfAhV+cxzP/bTGm2qbyA/2AudWMAwPvY+v/2RXZyRGED8hNTRAR5OJvz+P1M68KQgwA\nGVwGtlZtxlenrkRwZuMLlmXBVdtcQuxADTXyPpqN9gePIzQzgvAPEmOCiCDSq4AGmhHjWWwWBs/3\nR2BG45PBwQFk3s/wum1R31PQfdEo8owIIjBIjAkikrCjb5Jw5KL2l9jYOPSm9Hnddlt9G1O100Se\nEUEEBokxQUQQdglghnnEeLu0HbGr4728g/CGQqHAwFo9rLCO2HZl2VfIK5kbgVkRhP+QGBNEBFn5\n7bV4a90fYIRRGOtBD47uPIElW5+O4MzEh+M4mM1mcFxwLT83/NdNeHv3AXyR+AVssKFR3Yj/WPN7\nLPvfK0M8U4IIPZRNTRARRKlUYtvbFTjxh9PgrtoBGaBYpcKOFydXJvVnv70AW5UJifcTMZQ6BPNG\nG9b/+DnI5f5fopRKJbb9azketj3A4c8/wNSCadg6n7KoifFBUGKs1WqvARhwPL2t0+m+FbopEcTk\nQqFQYM0frQf+KNIziQwXf/0xlv7DQkyzOuK63YBRZ8T7/VXY8r92Bry/7JxpyM6hGDExvghYjLVa\nrRoAdDrdmtBPhyCIyQTLsrBXWV1C7CAGMcg9lYPOH3YiY4r3LGmCmEgEEzMuBRCj1WpPa7Xa81qt\ndkmoJ0UQxORgYKAfU9umeN22uGcRdJcbRJ4RQUSGYMTYAOBnOp3uWQB/DOCAVqulRDCCIAImNjYO\nPSm9Xre1qduQmZcl8owIIjIEEzNuBnALAHQ6XYtWq+0BkAXgobcXp6dPjOUZdB7Rw0Q4B2BinEco\nzuHyZgtsv7BBAc/60ddXXsfedXufeP/+MBE+C2BinMdEOIdgCEaM3wBQAuB7Wq12KoAEAKPWmuvq\nGgpyatFDeno8nUeUMBHOAZgY5xGqc1j212vwh8f7UXauBAuHFqBV2YrPll7Gon96RpS/0UT4LICJ\ncR4T4RyA4G4oghHj/wDwllar/dTx/A2dTjdGHSGCIIjRUavV2PbrctxtbsN7n1chKz8bLywNPIua\nIMYzAYuxTqdjALwShrkQBDGJmTk3BzPn5kR6GgQRESjxiiAIgiAiDIkxQRAEQUQYKodJBMU93V00\n/KIWMU0a2DV2MCtZrPmzDQGVLyQIgiB46MpJBMw93V08eq0Nr9zeI4yZPjfh7aYD2PmbFyM4M4Ig\niPEJuamJgGn6RR223H7BY0wDDdZ9uBp1l2siNCuCIIjxC1nGADoetuP6f3wNZY8C9pkslvzR00hI\nSIz0tKIWdYPa6/gcyxx8deE6ip8pFXlGBEEQ45tJL8Y3Tl8D+1cW7Hv8EiSQgAGDQ0cOY9avtJhV\nlBvp6UUljIbxOs6CBacJrhctQRDEZGZSu6ntdju6/792PPt4IyTge8fKIceLut1o/icqUD8azAoW\nZphHjJ/NOIcFryyOwIwIgiDGN5NajGsuXcOqmyu8bsv8Oh1DQ4Miz2h8sObPN+D3W97BHeUdALxF\nfCbjLKz/hUVqemqEZ0cQBDH+mNRuaqvZBhWn8rpNySjAMN7dsZMdhUKBnb99ETUXr+PKZ1+Di+Ew\n/5VFSMtIi/TUCIIgxiWTWoznr16IT/M+w46W7SO2PSrrQGHyggjManwgkUhQtnoBsDrSMyEIghj/\nTGo3tUqlguQ7ctQl3PQYv5B1EVn/eVqEZkWMZ9pu3UHNF9dhNo+MqRMEQYzGpLaMAeCZ11ahblYt\nDlS+D2WXApbpVsx9sxCzCimTOhLUXrqBzhOPILFLEbcqAYufXwaJRBLpafnknu4u6v/mBkquzkOB\neTa+mv05rC/bsebPNvi9D921Rtz/8i6SZqdg4fpF4+K8CYIIDZNejAGgeGUJileWRHoak56Tf3sM\nz7y1BOssfFLd43ce49CW97HjV7shk8kiPLvRYRgG9X96Ha/ecDUzy27Nxr2f3cPn6Z/h6Ze9Jwk6\nMRgMOPvdk3jmwtNYbqrAQ9lDnHzqCBb8yxJMnZUd0FxsNhtaGpqRmJKI7Onk3SGI8cKkdlMT0UPN\nxetY/rulyLPkCWNZbBZeObYHn771SQRn5psrhy5j242tI8ZnWGbAeFjv8/2f/M0ZvPnh6yg0FQAA\nsu3ZeOPqa7j+wy8Dmsenv/kEX6y5iOkbMmBbbsCHLx3H/eZ7Ae2DIIjIQGJMRAVdJ9sxxzpnxHgc\n4sB9Ft1Z7eY2ExLhvWKbslMx5nstFgtSLyZBhpGW/9Kri6G70eTXHL489Dnm/3QedjXvQC5yscj0\nFF79eC/qvvcNbDabX/sgCCJykBgTUYGEGT0+Ota2aCAmLw7dkm6v2yzZ1jHfq9frkTKQ4nXbdMt0\ndLV1+DWHweoB5JpH5jlsr9mGL6ou+bUPgiAiB4kxERWon4lFl6RrxDgDBrYyewRm5D9Ltz+N44s+\nGDHeHNOCpArvQuskOTkZD2c/8rrtSsZVFDwzz685KNu9W+CJSIS1zeLXPgiCiBwkxkRUsGzHM6je\ndBhGGIUxO+z4/ZI/4JnvrhJtHgzD4LP3L+D833yEsz//EP19fT7fI5VKsfhXK/DW5rfxSdIF1Mnr\nUDmvCg1/p8OiHUt9vlf
"text/plain": [
"<matplotlib.figure.Figure at 0x1037a5490>"
]
},
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"from sklearn.datasets.samples_generator import make_blobs\n",
"X, y = make_blobs(n_samples=50, centers=2,\n",
" random_state=0, cluster_std=0.60)\n",
"\n",
"xfit = np.linspace(-1, 3.5)\n",
"plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='spring')\n",
"\n",
"# Draw three lines that couple separate the data\n",
"for m, b, d in [(1, 0.65, 0.33), (0.5, 1.6, 0.55), (-0.2, 2.9, 0.2)]:\n",
" yfit = m * xfit + b\n",
" plt.plot(xfit, yfit, '-k')\n",
" plt.fill_between(xfit, yfit - d, yfit + d, edgecolor='none', color='#AAAAAA', alpha=0.4)\n",
"\n",
"plt.xlim(-1, 3.5);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Fit the model:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"text/plain": [
"SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0,\n",
" kernel='linear', max_iter=-1, probability=False, random_state=None,\n",
" shrinking=True, tol=0.001, verbose=False)"
]
},
"execution_count": 7,
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "execute_result"
}
],
"source": [
"from sklearn.svm import SVC\n",
"clf = SVC(kernel='linear')\n",
"clf.fit(X, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the boundary:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def plot_svc_decision_function(clf, ax=None):\n",
" \"\"\"Plot the decision function for a 2D SVC\"\"\"\n",
" if ax is None:\n",
" ax = plt.gca()\n",
" x = np.linspace(plt.xlim()[0], plt.xlim()[1], 30)\n",
" y = np.linspace(plt.ylim()[0], plt.ylim()[1], 30)\n",
" Y, X = np.meshgrid(y, x)\n",
" P = np.zeros_like(X)\n",
" for i, xi in enumerate(x):\n",
" for j, yj in enumerate(y):\n",
" P[i, j] = clf.decision_function([xi, yj])\n",
" # plot the margins\n",
" ax.contour(X, Y, P, colors='k',\n",
" levels=[-1, 0, 1], alpha=0.5,\n",
" linestyles=['--', '-', '--'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the following plot the dashed lines touch a couple of the points known as *support vectors*, which are stored in the ``support_vectors_`` attribute of the classifier:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAdwAAAFRCAYAAADejRzzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8VNeZ+P/PjDSjmVFDnQ4SghG9I0TvvRhsbAO2Yzt2\n4jhtk/1usrspu9lNNvltstndZNNjxxUMLmADNsVgTO+IImAAiV6FEEJlRtPu74+xBg33CtSmSHre\nr5dfWOfemfvMRejROfc85+gURUEIIYQQwaUPdwBCCCFEWyAJVwghhAgBSbhCCCFECEjCFUIIIUJA\nEq4QQggRApJwhRBCiBCIDuabFxeXN3vNUVKShdLSquZ+W3Efuc+hIfc5NOQ+h4bcZ5+0tHidVnuL\n6+FGR0eFO4Q2Qe5zaMh9Dg25z6Eh9/nBWlzCFUIIIVoiSbhCCCFECEjCFUIIIUJAEq4QQggRApJw\nhRBCiBCQhCuEEEKEgCRcIYQQIgQk4QohhBAh0KiVpqxW6z8BcwED8H82m+31Zo1KCCGEaGUa3MO1\nWq0TgDybzTYKmABkNXNMQgghRKvTmB7uNOCY1WpdDSQA/9C8IQkhhBCtT2MSbhrQBZiDr3f7EZDT\nnEEJIYQQrY1OURq2oY/Vav05UGyz2X79xdf5wBSbzXbr/nPdbo8ii1kLIYRoYzR3C2pMD3cH8G3g\n11artSMQC5RonRiMbZrS0uIpLi5v9vcVgeQ+h4bc59CQ+xwacp990tLiNdsbPGnKZrOtAw5brdZ9\n+IaTX7bZbM2+760QQgjRmjSqLMhms32/uQMRQgghWjNZ+EIIIYQIAUm4QgghRAhIwhVCCCFCQBKu\nEEIIEQKScIUQQogQkIQrhBBChIAkXCGEECIEJOEKIYQQISAJVwghhAgBSbhCCCFECEjCFUIIIUJA\nEq4QQggRApJwhRBCiBBo1G5BonU4uGYft1fcwnQ9huoMJ4mLkhj+yMhwhyWEEK2SJNw2atcb27D+\nSzYzKif7287sOMv24q2MfXFC+AITQohWSoaU2yCPx4Pzb3ZyKq0B7T3t2ShvunG5XGGKTAghWi9J\nuG3QuaIi+p3oq3lsyKlBnCk4HeKIhBCi9ZOE2wbFJ8RzO7ZU81iJ6TYJyQkhjkgIIVo/SbhtUEZG\ne86OLNI8dnLkKTp37RLiiIQQovWThNtGDfi3Ibw+6A3ucheAcsp5o/9b9P63AWGOTAghWieZpdxG\ndc7uQsa69mxasRXXeSfRXQ1MfnImRqMx3KEJIUSrJAm3DTMYDIx7amK4wxBCiDZBhpSFEEKIEJCE\nK4QQQoSADCkLEWEURUGn04U7jIh39fwVCtYfxZxqIXf+KAwGQ7hDEuKBJOEKESH2rthF+fI7mM/F\n4Ep2Uz3NzZTvzSAqKircoUUURVFY+/3V9F7dk8V3FlFOOR//5hM6/LQrfcb1C3d4QtRJEq4QEWDP\nsp30+ucsrFW9fA3XoKqgiuU3VzLvvx8Nb3ARZsvvNrLwtXm0ox0ACSTw5KknePf779N9cxYWiyXM\nEQqhTZ7hChFmiqJQ8XbZvWT7BQsWen2czbWLV8MUWYTa5PUn29rmFs5mz7KdYQhIiPqRhCtEmDkc\nDpKK1AkEYHTpKAq2Hg1xRJHNUKb9rNaECaXEG+JohKg/SbhChFlMTAyV7ao0j102XCa1e1qII4ps\n9iyHZvsFwwWShqWEOBoh6k8SrhBhptfrqRhfhQv1tohbhm5l4NghYYgqcnV5PpPdaXsC2ty4+WT8\nBgZPGhqmqIR4OJk0JUQEmPSv0/jbrTcY+ekIBlT157r+OuuHbqT/L4e0uhIhRVHY/L8b0X/ixXjL\niKNrNfFPtiP3iVH1en2fMX05/tujLPvrCswnY3DHeagcbWfmj+e3unslWhedoigNfpHVaj0ElH3x\nZZHNZvuy1nnFxeUNf/OHSEuLp7i4vLnfVtxH7nNo3H+fzx49w7ldZ2mXmcywaSNaZQL5+EcfMe/P\ns0hSkvxtZ01nKfipjVHPjGvQe9W3Zlm+n0ND7rNPWlq85jdlg3u4VqvVBGCz2WQRXiGaWfaAnmQP\n6BnuMILmdkkJHVdlBCRbgGxHNgffOoTydMMW/WiNv5CI1qsxz3AHAhar1brBarVutlqtuc0dlBCi\ndTqx/Th5N0dqHutU2JHbt2+HOCIhQqcxCbcS+KXNZpsOvAS8bbVaZfKVEOKhUrqmccV4RfNYaWIp\nsbGxIY5IiNBpzKSp08BZAJvNdsZqtZYAHQDVv6KkJAvR0c2/LF1aWnyzv6dQk/scGm3pPqdNz2XZ\nmGVYt1gD2r14qZxSSZcuwSuBakv3OZzkPtetMQn3OWAA8HWr1doRSACuaZ1YWqpdW9gU8lA+NOQ+\nh0ZbvM+9/30Qr5W/wcyD08nwZnA65jSfj9/OlB/PCtq9aIv3ORzkPvvU9UtHYxLuK8DfrFbrti++\nfs5ms8nyLkKIeuncswud1nZm/yd7KCu6Q8dhnXlk5OPhDkuIoGtwwrXZbG7g6SDEIoRoI3Q6HSNm\n5YU7DCFCSiY7CSGEECEgK02JoHE4HHz+m80Y9kah9+qxD6xm9HfGkZCYGO7QhBAi5CThiqBwu92s\n+9JqvvzZsxjw7e7i3enltX2vM3HlDOLi4sIcoRBChJYMKYug2LV8O0s/e9KfbAH06HnmwNPs+uO2\nB7xSCCFaJ+nhNoPCw2co/KsNU2EMrgQPUdMMjP/ypDa97Jx7v5N41FPjo4nGeES+7YQQbY/85Gsi\n2+5TuF6qZOm1xf62km0lfFT0IXP+45EwRhZeXmPdlWKeGKkiE0K0PTKk3EQX/1jIxGsTAtpSvCn0\ner8HVy9oL2HXFqTP68DZmLOq9lu6WxgmxYQhIiGECC9JuE1kKjBpto8qzaNg3dEQRxM5BowbxK4X\n93HUfMzfVmgs5IPFHzF28YTwBSaEEGEiQ8pN5DG7Ndvt2DEkGjSPtRUzfzyX03NPsfyjleCG9Okd\nWDBmUbjDEkKIsJCE20SVeQ68Ni/6+wYL1mV/wshHG7aZdmvUa3AOvQbnhDuMsPB6vbhcLmJiZAhd\nCCEJt8nG/2gSfz73Cgu2zyPDm4EHDx93+Zh2P0zBZNIebhaNY7fb2fP2Djw3PMT2jSd3Xh56feQ9\nFamoqODzH39K/M44LBVmSrPvkPx8GkPnj3joaxVFoWD/McpulDFgwkDi4xNCELEQIhQk4TZRXHw8\nC1Y8zr41u6k6UoEnUWHEl0aS2K5duENrVU7vtXH574uYd3ouZszc1N3kw9feY/xfp5CUmhzu8PwU\nRWHji2t5YfPzRPHF1pTFcOTEEfKNBxk0c2idry06ehbbDwoYc3AUae4B7Oi8g9tP3GXa92fV+/pu\nt5tTR09iibeQ1bNHUz+OEKIZ6RRFCdqbFxeXN/uby/ZPoRFJ99nr9bJp1lqeOrQ0oF1B4fXH3mT2\n7xeEKTK1/K2H6PlUN7o7u6uOLZu6gqlvBybPmvvsdDr5fPpGlhQsDjh+Lfoau/7jAOOenfDQa+96\nYxvOVxwMPzmUuzF3OTLsGD1/3Jseg3s25SO1CpH0/dyayX32SUuL11yEIfLG44S4z+HPDzAxf4Kq\nXYeO5D3tcDqdoQ+qDsUHb2omWwDTubqf5e5ZsZN5BXNV7R3cHXCutT/0uvkbDtLzX7NYdPJRutOd\nAdUDeHrnUi588yyVlZX1jl8IETyScEXEK79ZTqo3VfOYpTIWp7M6xBHVLaZjDOVo/4bvTK77FwPn\nlWri0F5f2njr4bPdb628SZ+K3qr2R07PZ89rOx76eiFE8EnCFRFv8PShbO3wueaxm72LiYtTLyEZ\nLnmPjWFN37Wq9lJdKUyPqvN18X0TuKG/oXmsusvDe/DGG9pJOYYYdFcf+nIhRAhIwhURL7FdO249\nWcr16OsB7YeSDpHy5fSgXvvGpeus/9ZHbBu9iW2jPmX9Nz7i2vm6M5jBYCDrv6y8OeQtrumv4cXL\nZ6lb+fCFdUz6xtQ6Xzd
"text/plain": [
"<matplotlib.figure.Figure at 0x10d6a4250>"
]
},
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='spring')\n",
"plot_svc_decision_function(clf)\n",
"plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1],\n",
" s=200, facecolors='none');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use IPython's ``interact`` functionality to explore how the distribution of points affects the support vectors and the discriminative fit:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAd8AAAFVCAYAAACuK+XmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8W+d1+P8PuDcJAlziFDWgRWpL1N57WcNaHvFIUsdJ\nHDdJv2nSNm2SNkl/SRNnNnFi19sa1rRlbcnae09oU5siAU5wgADu7w+KEKELWiIFgCB53n7pZeHi\nAvchBN5zx3nO0SiKghBCCCF8J6ClByCEEEK0NxJ8hRBCCB+T4CuEEEL4mARfIYQQwsck+AohhBA+\nJsFXCCGE8LGg5rzIYDD8EJgBBAN/NBqN73p0VEIIIUQb1uQzX4PBMBoYYjQahwKjgWwPj0kIIYRo\n05pz5jsROGUwGFYDMcA/eXZIQgghRNvWnOCbAKQD06k7610LdPPkoIQQQoi2rDnBtwg4ZzQabcAF\ng8FQbTAY9EajsejhFRVFUTQazRMPUgghhGhFHhn4mhN8dwPfAX5jMBg6AJGAye3WNRoKC8ubsQnR\nFAkJ0fI5e5l8xt4nn7H3yWfsGwkJ0Y9cp8kJV0ajcR1wzGAwHKTukvOrRqNRujMIIYQQj6lZU42M\nRuMPPD0QIYQQor2QIhtCCCGEj0nwFUIIIXxMgq8QQgjhYxJ8hRBCCB+T4CuEEEL4mARfIYQQwsck\n+AohhBA+JsFXCCGE8DEJvkIIIYSPSfAVQgghfEyCrxBCCOFjEnyFEEIIH5PgK4QQQviYBF8hhBDC\nxyT4CiGEED7WrH6+wvdO7jxBwfJbBBcHUZ1pJfdrfemQleq17SmKwokdxyjaUoAjWKHj7M50ye3q\nte0JIUR7IsG3Fdj55na6/aIz4yzDAVBQWL9pPZV/sdC5v+cDosPhYPVrnzBx9VgmWEcBcOyd42z8\n+jom/XDaY79PscnMwTf3EXItmFptLZ2e6UqnnC4eH68QQrQ2ctnZz1ksFkLfDKCnpYdzmQYNU/On\ncuWNi17Z5s73tjN/2Rw6Wjs6l/W19KH///bm7IHTj/Ue185e5eTMwyz+7Xzmr5rLM28vhHm17P94\nj1fGLIQQrYkEXz93cO0+xl8f7/a52GPRWK1Wj2/T8UUtWrSq5d2ru3N71fXHeo/zvz7NvItzCWjw\nFcsrHozt99XU1NR4bKxCCNEaSfD1c0EhQdiwuX3OEWRHo9F4fJsB1sa/FpqaR39lrFYrcYdj3D43\n8fIEDny2t9lj8xWbzcbuFTvZ/If1GI+ea+nhCCHaGAm+fm7Q9CFs6rzF7XOlAyoIDg72+Dare1lx\n4FAtN2MmdEjEI1+vKAoo7p/ToEGxN/Kkn7h05AJbJ61n9DeGsvhn89HOjuLdp9+VM3YhhMdI8PVz\noaGhhH03kp26Xc5lNmws6baUnj/o7ZVtDv3mCN7t9z5KgwhqxcrS8csZMnfYY425tG+Z2+c2d9zC\noBlDPDZWTym4dZfNv1rPxv9cx5nXjvPsqcXOS+89q3qw6JNFbP3ZxhYepRCirZBs51Zg4Lw88nOv\n8eH7SwgqDsaebSfva8OJjnZ/afdJxcTGMvyjMbz/uyWEnwjBEeSgNs/OjG/PJTAw8LHeo8v3urHG\nuJaZV2egoe7S+LGYYyivBhAeHu6VcTfXF/+7lYTfa1lkeppd7KIHPVTrhBBC+M6QFhidEKItkuDb\nSmR2zSLzZ1k+215cvJYpP5ne7Ndn9+5C5KoY3v/rEsLzQ7Bqa0lflMWwQaM8OMond/X8FTJ+k8Lg\n0sEAlFCCDp3bdUNKg1EUxSv32YUQ7YsEX+E1SR2SniiA+8KlJedZXLrA+bg3vTnCEQYwQLVuZZdq\nCbxCCI+Qe76iXQuscr2MnkkmF7hAJZUuy0/FnUL3fKIvhyaEaMPkzFe0ayEDwjH9n8nlUvMCFrCW\ntRTpTeij9FRn15D9rSz6DR/YgiMVQrQlEnxFuzZkzjCWfbKMr21/iaD7vw6BBFLb3cawJWNJSEkA\nICEhmsLC8pYcaqtlsVgICgoiNDTUI+93O/8WZzacJFwfQd6sYQQFyW5MtD7yrRXtWmBgINPeeYqP\n/mcZofuDCawNpDq3ht7f6u8MvA+7eeUGp986Qei9EGqSreR+tS8dMr3X5KK1Or39JHf/dJP4M1pq\ng2sxDy6h778NJDkjpVnv53A4WPfPa+i+uguLSp6mnHI+/8N6kn+aQc+RvTw8eiG8S6MoXi14oMjZ\ngvfJWZn31X/GJzYfQ/M9G2PvjqkrGILCptRNhLwRTa9ROS09TL9x6egFbF+pYkTBcJfl7/R+l/Gf\nTXN7Fvyo7/HWP2xi8s/GEUecy/JlnT4hb9tIv5vC5o9kX+EbCQnRj8zMlIQrIR6Toijc+80txt0d\n65y7rEHDpFuTuP0/+Xj5QLZVufx/RlXgBZh/4mn2vr/LzSseTbPZoQq8ADMvT2ffh7ub9Z5CtBS5\n7CzEY7p47gL9jvd1+1z3YwZu3rxBenqGj0fln8Kuh7ldHkEEyiV7s94zqNT97iqMMBSzuhxqverq\navZ9sAvF6KBWa6PvSwNJTJbMddGyJPgK0QQaxf3VpABHAHY583Wy6mrdLrdjx65vPFB+mepONeCm\nx0V+cD7a/u4LoxTcKuDQi7tZcHx+XeBHYdOSzdz8r3z6zZDsddFy5LKzEI+pS/euHO19zO1zZ/ud\nl7PeBmKf0nIl7Ipq+efpnzPgpcHNes/UFzLZrz/gssyOnfWjNtJ3bH+3rzn6swO8cPwrRFDXEESD\nhkl3J1L63yZqa90fIAjhCxJ8hXhMGo0G/evJ7Ejc6VymoLA1ZRvJ/5gm1a8aGDRzCIf/6TjrMtZR\nRRX3uMdHvZYQ+Sst2vj4Zr1nz5G9qPxDLR+OW8Kq1NUs67qc91/8mCl/n+X2s7fb7UQfjHTen29o\nyoXJ7F+7p1njEMIT5LKzEE3QZ0p/rmVf5YN3Pia0IISaFCs9XswlvbOc9T5szLcnUPFiBes2bCIs\nJpyx4yY/dmOOxuSM603OuN6PVWPb4XAQVOt+FxdOOFaLtIgULadZwddgMBwFSu8/vGI0Gl/23JCE\n8G9Zho5k/aJjSw+jVYiKimLUvLEef9/HucoQHBxMWW4FbFY/t6XDFgY+lefxcQnxuJocfA0GQxiA\n0Wgc4/nhCCGE52R8K5utZ7cy7tY457KrYVcp+YqFmJjYFhyZaO+ac+bbG4gwGAwb77/+R0aj8cAj\nXiPakPLyMgoL79GhQxphYe6nlAjhD7oN6c7VD67wwVsfE5YfSm2cjdhZWsbNnNjSQxPtXHOCrwX4\nldFofMtgMHQB1hsMhq5Go7F58wdEq1FZWcm2H26kw7Zk0u+lcTRrH2UzLEz6l2mSbCT8Vsee2XT8\nTXZLD0MIF00uL2kwGEKAAKPRWH3/8QFgjtFovOVmdZn42IZ8sOgDFi5Z6GxAAHXN53f8eAezfjKr\nBUfW8oqKiggKCiIuTl2BSQjR7jzybKQ5Z74vArnANw0GQwcgBrjT2MpSR9T7nrRe67k9Z7nx1lXC\nLoVii7FhHwfjvjORgIAHM9FuXbtJ5/WdXQIvQBxx1C61c+eV4jbdXaaxz/jUthPc/cNN0k+mYg2q\n5c7AArr+sAcde8qZVlP5U93h/EvXuHX+Jp36dyEpJamlh+Mx/vQZt2UJCdGPXKc5e8u3gP8zGAz1\nkx1flEvOrdeZnacJflXhmXsLncsqDlawJH85s96Y51x25eglJpeOc/cWJN9OorS0FJ3OfZWhturK\nqUsEfMfO4oIHnx2bYOm1ZejX64mOjmm5wbUxlZWVnNpzghh9LN36dPfabY4SczE7X99K7925jKkY\nxrH44+yfuIspv55JSEiIV7Yp2qcmF9kwGo02o9H4nNFoHHn/z35vDEz4xu03rzPs3lCXZVFE0efT\nXK4ZrzqXZeRkcTbKTW0/4F5SITEx7S/QXHrHyOiCUarlcy7MZv/fpICDp2z73WZOjD7IkGf6kzI9\nng0z13L52EWvbGvXP27
"text/plain": [
"<matplotlib.figure.Figure at 0x10fc0a390>"
]
},
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"from IPython.html.widgets import interact\n",
"\n",
"def plot_svm(N=100):\n",
" X, y = make_blobs(n_samples=200, centers=2,\n",
" random_state=0, cluster_std=0.60)\n",
" X = X[:N]\n",
" y = y[:N]\n",
" clf = SVC(kernel='linear')\n",
" clf.fit(X, y)\n",
" plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='spring')\n",
" plt.xlim(-1, 4)\n",
" plt.ylim(-1, 6)\n",
" plot_svc_decision_function(clf, plt.gca())\n",
" plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1],\n",
" s=200, facecolors='none')\n",
" \n",
"interact(plot_svm, N=[10, 200], kernel='linear');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Support Vector Machine with Kernels Classifier\n",
"\n",
"Kernels are useful when the decision boundary is not linear. A Kernel is some functional transformation of the input data. SVMs have clever tricks to ensure kernel calculations are efficient. In the example below, a linear boundary is not useful in separating the groups of points:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFRCAYAAAChXA4CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdcVGe++PHPzNCrdLGL5QhWxAKKIgqIvcSaxJisSTbZ\nzfa7eWVrdvduufduyS937252U3ZN1BSjxlgpIoKCIhYEBEdF7A0V6X3m98cYBRmM0g4zfN+vl6+X\nPOec53x5HPlynvMUjdFoRAghhBBdg1btAIQQQgjxgCRmIYQQoguRxCyEEEJ0IZKYhRBCiC5EErMQ\nQgjRhUhiFkIIIboQG7UDACgqKmvXOVseHk4UF1e2Z5UWTdqjKWmPB6QtmpL2aEra44H2bgsfH1dN\nS8es8onZxkandghdirRHU9IeD0hbNCXt0ZS0xwOd2RZWmZiFEEIISyWJWQghhOhCJDELIYQQXYgk\nZiGEEKILkcQshBBCdCGSmIUQQoguRBKzEEII0YVIYhZCCCG6kDat/KUoykTgv/R6feRD5T8A1gBF\n94q+qdfrT7flXkIIIUR30OrErCjK68CzQLmZw2OBVXq9/nhr6xdCCCG6o7Z0ZZ8FFgPm1vsMAX6q\nKMp+RVHeaMM9hBBCiG6l1YlZr9dvAepbOPwJ8E1gOhCuKMqc1t5HiO6itraWnJwTaochhFBZR+0u\n9bZery8FUBRlJxAM7GzpZA8Pp3ZfINzHx7Vd67N00h5NdcX22LRpE7m5ufj7exEYGNhp9+2KbaEm\naY+mpD0e6Ky2aPfErCiKO5CtKEoQUInpqfmDR13T3tuK+fi4UlRU1q51WjJpj6a6anuMGDGOI0ey\n2LjxC77xDW8cHBw6/J5dtS3UIu3RlLTHA+3dFo9K8u0xXcoIoCjKSkVRXtLr9SXAG0AykArk6vX6\nuHa4jxBWzcvLi7CwyZSXl5Gamqx2OEIIlbTpiVmv158HJt37+yeNyj/B9J5ZCPEEJk4M49SpfLKy\njhMYOJy+ffupHZIQopPJAiNCdCE6nY6ZM2eh0WhISNhNfX1L4yuFENZKErMQXUzv3n0IDh7L7du3\nOXQoXe1whBCdTBKzEF3QlCnTcHV1IyPjIEVFRV9/gRDCakhiFqILsre3JyZmJg0NDcTH78JgMKgd\nkhCik0hiFqKLGjRoCIGBQVy9eoWsrGNqhyOE6CSSmIXowiIjo3BwcCQ1dR+lpSVqhyOE6ASSmIXo\nwlxcXIiMnE5tbS2JifEYjUa1QxJCdDBJzEJ0cSNGjKJfv/4UFJxFrz+ldjhCiA4miVmILk6j0TBz\n5ixsbGzYsyeBqqoqtUMSQnQgScxCWAAPD08mTZpCZWUF+/btVTscIUQHksQshIUYP34Cvr5+5OSc\n4MKF82qHI4ToIJKYhbAQOp2O2NjZ95frrKurUzskIUQHkMQshAXp2dOfceMmUFxcTHr6AbXDEUJ0\nAEnMQliYyZOn4O7uTmZmBjduXFc7HCFEO5PELISFsbOzIyZmFgaDgfj43bJcpxBWRhKzEBZo4MAA\nhg8fyfXr1zhyJFPtcIQQ7UgSsxAWKjJyBo6OTqSlpXL3brHa4Qgh2okkZiEslJOTE9OnR1FXV0dC\nQpws1ymElZDELIQFCwoazsCBAZw/X8jJk7lqhyOEaAeSmIWwYBqNhpiYWGxtbUlOTqKiokLtkIQQ\nbSSJWQgL5+7egylTIqiqqiQ5OUntcIQQbSSJWQgrMHbsOPz9e5GXl8u5c2fVDkcI0QaSmIWwAlqt\nlpiYWWi1WhIT46mtrVU7JCFEK0liFsJK+Pn5MWFCKCUlJRw4kKJ2OEKIVpLELIQVCQubjKenJ0eP\nHuHatatqhyOEaAVJzEJYEVtbW2JiZmE0GomL20VDQ4PaIQkhnpAkZiGsTL9+/Rk1agxFRTfJzMxQ\nOxwhxBOSxCyEFYqIiMTZ2YX09APcuXNb7XCEEE9AErMQVsjR0ZGoqBjq6+tluU4hLIwkZiGs1NCh\nCoMHD+HixQvk5JxQOxwhxGOSxCyEldJoNERHz8Te3p59+/ZSXl6udkhCiMcgiVkIK+bq6sbUqdOo\nrq5m795EtcMRQjwGScxCWLkxY8bSu3cfTp3K5+zZM2qHI4T4GpKYhbByGo2GmTNno9PpSEyMp6am\nRu2QhBCPIIlZiG7A29ub0NBJlJWVkpqarHY4QohHkMQsRDcxcWIYXl7eZGUd5/LlS2qHI4RogSRm\n0e0UZBcQ//0dpMxNJOnpOFI/2tct5vna2NgQGzsbgPj43dTX16sckRDCHBu1AxCiM505oqf61VKe\nvbDyftnN5JvsPrud2b+Zr2JknaN37z4EB4/l2LGjZGQcZOHC2WqHJIR4SJuemBVFmagoSrMXVoqi\nzFMU5bCiKOmKorzYlnsI0Z4K/36W6RemNynzbfAl4LP+XL98TaWoOteUKdNwdXXj0KF0ioqK1A5H\nCPGQVidmRVFeB94D7B8qtwX+AkQDEcDLiqL4tiVIIdqLY46D2fLw4snkbMvq5GjUYW9vT3T0TBoa\nGti2bVu36MYXwpK05Yn5LLAY0DxUHgic1ev1JXq9vg44AExtw32EaDcNDua3QayhBhsX206ORj2D\nBw9h2LBALl26RFbWMbXDaeJc3lkS344jee0eqqqq1A5HiE7X6sSs1+u3AOZGj7gBJY2+LgPcW3sf\nIdpTVWg1BgzNyncFxBG6dLIKEaln+vRoHBwcSE3dR1lZqdrhYDAY2Pq9TdjN1fD075Yy7/VYDk8/\nQFZc1/rFQYiO1hGDv0oA10ZfuwLFj7rAw8MJGxtduwbh4+P69Sd1I9IeJgvfnsdHFz5i/r75eOKJ\nAQNJvZLw/4Mf/fp1rzcuPj6uxMTEsG3bNjIyUlmxYgUazcMdYK2nz9ST+3+52J2xo96jHteFrsx4\ncUaL99j+h+2s/GQpLrgA4IgjSwoWs/1X23FYCK6unfMZlv8rTUl7PNBZbdERifkUMERRFA+gAlM3\n9h8fdUFxcWW7BuDj40pRUVm71mnJpD0a07Bqzyq2/SOOmuxKDG5Ggp8fj7ev92O3UW1tLQc/PUDd\nxVrsBtkzaekUbGwsc4JDcHAwaWmHOXYsm759B6Mow9ql3vy0PAzfquGpa0/dL7uRcIN1Jz5l1ptz\nzV5Tua36flJuLPZcLJvf+pIZr85sl9geRf6vNCXt8UB7t8Wjknx7/DQxAiiKshJw0ev17ymK8kMg\nHlNX+Qd6vb57DHcVFkGn0xG+bCose/JrL+SfJ/+1bBblLMAFF0oo4Yu1Wwn+x0R6Dezd/sF2MI1G\nQ0xMLGvXfsCePQn069cfR0fHNtd76e+FPHNtRZMyv3o/+n7mT9ErRfj4+TS7xqbM/I8jW2wx3pUB\naqL7aNN0Kb1ef16v10+69/dP9Hr9e/f+vkOv10/Q6/Xj9Hr9O+0RqBBdQd4vTrAq55n7T3buuPP8\n8dWc+OURlSNrPU9PLyZNCqeiopyUlLYv12k0GnHKMz/6PeLWVLK2HTV7rHqI+TW8z9udxzu0e71m\nEN2brPwlxGO6cO48ow6PMHts4MH+Fj0nePz4ifj4+JKdncXFixfaXF+9vfnR75VUYu9ub/bYwJcH\ns69nSpOyOuqIi0xg9LTgNsckhKWwzBdjQqig9HYJg6v7mT3mUe5BeXkZPj7Nu2izU7K4sfkqtqU2\nVA+qZfw3w/Dy9erocJ+ITqcjNnY269d/SELCblavXoOtbeumj2k0GipCKzGeM6J5aDbl7qFxTFoY\nafa6IeMU9P88xYb3PsUx34F6p3qqw2uZ85NF7TooTYiuThKzEI9JGR1IxpBU+p1pnpzzh59iSr+o\nZuXJ/5fIyD8FMaNyCgAGDGyO38KgfwXSd6j5JK8Wf/9ehISM58iRwxw8mMbUqdNaXVf4m9N47/wH\nPJW+CC+8aKCBXX130ePnPtjZ2bV4nRI2DCWsfQagCWGppCtbiMdkZ2eHYZWWcw7nmpTnO5/CbrUT\nOl3TKX93i4vp8a4rgZU
"text/plain": [
"<matplotlib.figure.Figure at 0x10f8a67d0>"
]
},
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"from sklearn.datasets.samples_generator import make_circles\n",
"X, y = make_circles(100, factor=.1, noise=.1)\n",
"\n",
"clf = SVC(kernel='linear').fit(X, y)\n",
"\n",
"plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='spring')\n",
"plot_svc_decision_function(clf);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A simple model that could be useful is a **radial basis function**:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAFBCAYAAAD69Z+AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXeUG9d5//29MwNgF7sLbCe3ktxlbyJVKFEsYpNpRZTk\nolhSHNtJnNclid+c/Ow4VhInOYnzyuWnxEVOlNiW7NiOZVuWbHWJKpRoWRIlsVPsdcnlNm7fxQKY\n8v4xGMwMMOgzwAVwP+f4WBwsZu4MZu53nuc+hSiKAgaDwWAwGMnhCj0ABoPBYDCKASaYDAaDwWCk\nARNMBoPBYDDSgAkmg8FgMBhpwASTwWAwGIw0YILJYDAYDEYaCMk+HBycYDknDAaDwSgrmppqiNV2\nZmEyGAwGg5EGTDAZDAaDwUgDJpgMBoPBYKQBE0wGg8FgMNKACSaDwWAwGGnABJPBYDAYjDRggslg\nMBgMRhowwWQwGAwGIw2YYDIYDAaDkQZMMBkMBoPBSAMmmAwGg8FgpAETTAaDwWAw0oAJJoPBYDAY\nacAEk8FgMBiMNGCCyWAwGAxGGjDBZDAYDAYjDZhgMhgMBoORBkwwGQwGg8FIAyaYjLJFlmXIslzo\nYTAYjCKBCSaj7FCFUoSihMHzAM8DHAcAEgARiqIUeIQMBoNGhEIPgMHIF7IsQ1HCUBQRAAEhPAgh\nIAQgBBDFEGRZgsdTBQAw6qaimP/NYDDKDyaYjJJHdbvKUBQJiiIBUACQpN/RRNSIJphMSBmM8oQJ\nJqNkMQulAkIIkgul9pm1oGoCyoSUwShPmGAySg4roSSxKmcjTEgZjPKACSajZMi3UKYiXSFlIspg\nFAdMMBlFj11CqX1FUeJFzk6SCaksK5DlMAACjnMxIWUwKIIJJqNooc2izBVC1PSWcDgEjuPB8y7m\n1mUwKIIJJqPoKDWhTAZbH2Uw6IEJJqNocF4ojVGydJOJkLJiRgyGPTDBZFCPs0KZOiezmLASUo6L\ntz6ZNcpgZA4TTAa1qJV5QpFiAxwI4WwQytIRx0xghRgYjNxhgsmgjmwq8zAyJ5VbNxyeASEceN7N\nhJTBABNMBkUoihIVSXNlHjZT5xNC1N9ClkUQwsHlcgNgFimDwQSTUXCshLJUo16LDePvwAoxMMod\nJpiMgpGZUNrtlk08o7P2XpmRTEiZNcooJZhgMvIOLRalosiQpDAUhQfHsdawdsMCjRilBhNMRt6g\nRSgjowEgQZIkSFLY9Ikqogo4zq7IXIYGK8TAKGaYYDIchxahVF2tkmELAc+7QIgamSvLIgBAliXI\nsvnvNPHURZQJqZ0kr68rQpJEcJwLhPBMRBkFgwkmwzHoE8rYmZYHxwngedUdK4ohiGIIguAGISSS\nBypHxFSCapHq39aE0yimAAtYshM1Yld9meF5ATyvbmfro4xCwASTYTv0CqWWpiIjmbCpuYf65Kzt\nS5YlKIouovp/x3+f4/gYi5SJqJ2w9VFGIWCCybANVUREKIqqIPYIZebft7YoecO+Mi+uSggBz5sf\nF/XFQDEIpy6okhR7DObWdRq2PspwGiaYjJzRLEq1j6MEQLWuCjEOVQyNYqUKpSZMsSkj5n9nVnxd\nfyEwn6smomYrND23LhPQeHJN82GF6hl2wQSTkTWxrlfDJ04cLek4UgllPlGFlAfH8abtmbh1ZVlC\nOBwsoJga3dh0YPf5s0L1jExhgsnImERrlPlO+HdKKJ06jXTdulqEbmy6i3qdjRYpz6oiOUCy9VFJ\nkiDLIjhOYBG7ZQgTTEba0BXMEyuUHNSOJsnHowp7os9sGmAGxLp1FUVBMDgVqeHqiXPrqsXoWbRu\nvtEvpQxZDoPjOPC8LpjMIi0PmGAyUlIKQllsEELAcVZu3fi10URuXS1St/jXSOlTn1h3Lgs0Kg+Y\nYDISQotQRkYDQDT8O1ehLM6ZS015sQoy0gKNjGukqjVq/r7RrctHRTT2OmoTO136SsNg0lvbZYXq\nSxMmmIw4shdKeyc03aIE9InKSYsysyhZWjC6dY1rpNbRulqPUXNJwNh0FzrESaWUBIUVqi9umGAy\notBiUVq7XgFAKFKXYmGwitZVr61iqmKUOHdUDXIBChmtq0PDT68FtjkRscvcuvTDBJPhgFBm90Rr\nk3lsvVd1W7Guv9GF1pQ7sVtXiuSMagFGSopo3cRuXfsoT4VIx62r1T8mRGBCmgeYYJYxdFmUsULJ\nQS8IIMZ9h2EvZreumgcaCgXAcTwEwW2wSLU10tRuXVVM7byf2AsTYBbScDgERZFRUVHNCjHkASaY\nZYiaQB+KTHqkYG62ZEIZX5mHvT4XAk34snfrkhgRLeZoXYC+gg56Y3VWiMF5mGCWEWaLUoT6sOX/\nFkhHKPM7FkYmpOvWTSda15j6UgxFGGiLHlaU1GNh66P2wQSzDLByveprg/kdh/Uapd2uu+xIVG+W\nkR6xbl2N2Ghdo1vX3HfU2q1L1+9Bo4WZed1mlj+aHUwwS5h8r1EmqqKjC6UM84RDh1AynMWOaF1A\n7VfKOr3oOPEiwQrVJ4cJZgmSmVDqayA2jsAwFi09hAklQye5W1eOs0gBtbauuSSgJsRmES23eysf\n55vu+qj6mwGlWpqRCWYJQUvUqzoWmoUy/+5oRnpoIgjwUbduMDgNRZHhdldG0110i1RM0MA7Pu3F\nDpzKwyxWYtdHRTEIWZbgdleZ/q5UrFEmmCUATUKpo5kC6ppWIfpjMkoFVaT0aF2XujXLaN3ScevS\ntp5qfKHQhbRUxBJgglnU0CSU6oMSG8zDhJJhF/H3dWZu3fQaeBdTpxfaInaNFMP1ywYmmEWIPUJp\np4tKrQhj2jux59ZK1o4rV+KDJoqzlqwT0BSZmulQrNy66n7MBeqTdXox9hw15446seafLfRZmHRd\nH/thgllEOGNR5lLGziiUWk9HKeF3coOeCTwf0CJYtFgKdgwjUbSulUWqu3WNVab0tW9JChfcrUvL\nPWJEnZdK16vEBLMIoNP1anxYeWhuLDXYh74HmVHMOGe1xDbwjh4xgVtXIxwOGvZRWLcuLS81NAq4\n3TDBpBi9hJ0IbT2QVqG0+ntaHuRMKeRzX6zXzHnye10SuXVnZiYBEAiCKwe3rl3Q6JIt7XuYCSaF\nmC3K+PXBfI8lvtVWYqEsNtTJTrPaCz0aBs0YU0oEwW3aHm+RSgndunZF69IX9EOngNsJE0yKSFzC\nzgmS77fUhVJFgiyLCAZDpknLKKIMRiyx90Wmbl37onXpEij6BNx+mGBSQHprlE5Zmeb9lrpQ6mX6\nAG1tTFt71ct/yQgGpxBvDfCU5LiWD8W8LpYsWlevp5uOW5ePsUhpvf+sBbyIf8I4mGAWEPqCeWKF\nsjAdRJwg0flxnAC32x2ZxGSEwwFoQpnMGnCqkgzDGjqurz0WHSEEPG+eeuPdurqgxhZh0Dq9GFvf\n0eARKYcqSEwwCwBNQqk+bBLsE0onys5lv09rodTQz5EQEnGDARzHwe2ujH7fKm8veSUZXURLefIo\nN5y0lFK5dc1WqNbAW0WN2A0WPFq3HGCCmUdoE0odbfIvNYvSuuemVaGFRCS2BpKtTYUN33c6UpKR\nb/L581nljgLqskE4PANZlqMt0BK5dY09R528BxNZmMwly8gIXSjVGzozobT3xs6f69X+3Ll03E7p\nNKdWFN1izeZhTrw2ZRbQRAnwxdE8mZaAEppmW1quCSLFATgAMlwuT9RFG1vNKFUDb3s9IvRcH6dg\ngukguQll3N5yHksi16Q6+Rc/he6QQgiXdruqxM2TecP3Cr8uRQ/sOsRjFihzA299ard262opa/Ee\nETvduqVkXQJMMB3BKJTaTV3YYJ5ErknR8jvFhnWZvuyF0s7ITCtr1LrLhjFvDwAUBINTcdZoua1L\n0TThFmtQS6YNvLN16xbr9ckEJpg2YiWUhRxLatdkAQZmI4nq2aZfyzK2YHw+S5kl7rIRCgUif8dF\nJrDE1mhxpBvkTgmfWtb
"text/plain": [
"<matplotlib.figure.Figure at 0x10ff08750>"
]
},
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
}
],
"source": [
"r = np.exp(-(X[:, 0] ** 2 + X[:, 1] ** 2))\n",
"\n",
"from mpl_toolkits import mplot3d\n",
"\n",
"def plot_3D(elev=30, azim=30):\n",
" ax = plt.subplot(projection='3d')\n",
" ax.scatter3D(X[:, 0], X[:, 1], r, c=y, s=50, cmap='spring')\n",
" ax.view_init(elev=elev, azim=azim)\n",
" ax.set_xlabel('x')\n",
" ax.set_ylabel('y')\n",
" ax.set_zlabel('r')\n",
"\n",
"interact(plot_3D, elev=[-90, 90], azip=(-180, 180));"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In three dimensions, there is a clear separation between the data. Run the SVM with the rbf kernel:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
2015-04-12 04:35:40 +08:00
{
2015-04-15 02:24:23 +08:00
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFRCAYAAAChXA4CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdcVFme8P9PFTnnnEEtkgFBRUyIOWCbW7vtNN09aWf3\nmdlnd367szM7aWdmJ2zPzvPqiR2mg922bWizqK22oiJiQHIBkgUFlFSEivf3R2khDbYKCFVw3v3q\nl9SN5x6K+tY995zvkUmShCAIgiAI5kE+2gUQBEEQBKGXCMyCIAiCYEZEYBYEQRAEMyICsyAIgiCY\nERGYBUEQBMGMiMAsCIIgCGbEerQLANDU1DGsY7Y8PBxpaekazkNaNFEffYn66CXqoi9RH32J+ug1\n3HXh4+Mie9i6MXnHbG1tNdpFMCuiPvoS9dFL1EVfoj76EvXRayTrYkwGZkEQBEGwVCIwC4IgCIIZ\nEYFZEARBEMyICMyCIAiCYEZEYBYEQRAEMyICsyAIgiCYERGYBUEQBMGMiMAsCIIgCGZkSJm/FArF\nLOC/lUrlwi8t/x7wKtB0b9E3lEpl6VDOJQiCIAjjwaADs0Kh+D6wDVANsHo68IJSqbw22OMLgiAI\nwng0lKbscmA9MFC+z0TgBwqFIlOhUPzbEM4hCIIgCOPKoAOzUqncC+gesnoH8A0gDZirUChWDfY8\ngiAIgjCePK3Zpf6gVCrbARQKxWEgATj8sI09PByHPUG4j4/LsB7P0on66EvURy9RF32J+uhL1Eev\nkaqLYQ/MCoXCDchTKBSxQBfGu+Z3vmqf4Z5WzMfHhaamjmE9piUT9dGXqI9eoi76EvXRl6iPXsNd\nF18V5IcjMEsACoViK+CsVCrfuvdc+TSgBj5XKpUZw3AeQRAEQRjzhhSYlUplFZBy7+cdDyzfgfE5\nsyAIgiAIT0AkGBEEQRAEMyICsyAIgiCYERGYBUEQBMGMiMAsCIIgCGZEBGZBEARBMCMiMAuCIAiC\nGRGBWRAEQRDMiAjMgiAIgmBGRGAWBEEQBDMiArMgCIIgmBERmAVBEATBjIjALAiCIAhmRARmQRAE\nQTAjIjALgiAIghkRgVkQBEEQzIgIzIIgCIJgRkRgFgRBEAQzIgKzIAiCIJgREZgFQRAEwYyIwCwI\ngiAIZkQEZkEQBEEwIyIwC4IgCIIZEYFZEARBEMyICMyCIAiCYEZEYBYEQRAEMyICsyAIgiCYERGY\nBUEQBMGMiMAsCIIgCGZEBGZBEARBMCMiMAuCIAiCGRGBWRAEQRDMiAjMgiAIgmBGRGAWBEEQBDMi\nArMgCIIgmBERmAVBEATBjIjALAiCIAhmxHq0CyAII+1G3g2yfn0J+3I7dK56bJbbM++FBchkstEu\nmiAIggjMwvhSdllJz7fa2Va91bSs8XQjR8sPsvJna0axZIIgCEZDaspWKBSzFArF6QGWpysUiksK\nheKCQqF4bSjnEIThVPmnctKq0/os89X7ErkzjFt1DaNUKkEQhF6DDswKheL7wFuA3ZeW2wBvAEuA\nBcDXFQqF71AKKQjDxSHffsDlc1vmkH8gd4RLIwiC0N9Q7pjLgfXAlx/MxQDlSqWyTalUaoFzwPwh\nnEcQho3eXj/gcjVqrJ1tRrg0wkAqiso58YcMTr/3Od3d3aNdHEEYcYMOzEqlci+gG2CVK9D2wOsO\nwG2w5xGE4dSd3IMBQ7/lRyIzSN40ZxRKJNxnMBjY9392Y7taxnO/2ET695dzKe0cuRlXR7togjCi\nnkbnrzbA5YHXLkDLV+3g4eGItbXVsBbCx8fl0RuNI6I+jNb+IZ0Pqj9gzRdr8MQTAwZOBp4k4Fd+\nhIaOzycuT+u9ocxRUvBmAbZltug8dLisdWHRa4se2vv94K8OsnXHJpxxBsABBzbeWM/BnxzEfi24\nuIzMe1j8rfQl6qPXSNXF0wjMJcBEhULhAXRibMb+7Vft0NLSNawF8PFxoampY1iPaclEfTxIxguf\nv8CBv2SgzuvC4CqR8PIMvH29H7uONBoNWZ+cQ1ujwTbKjpRN87C2tswBDk/rvVF8vgjDt9VsaNhg\nWnb7+G0+vP4JK368esB9ug70mILyg5ZXLGfP7/ez6FvLhr2cXyb+VvoS9dFruOviq4L8cHyaSAAK\nhWIr4KxUKt9SKBT/DBzD2FT+jlKpFN1dBbNhZWXF3M3zYfOT71tdXEXxd/JYl/8MzjjTRhufvbeP\nhL/MIjAiaPgLa6Fq/1TJ8w1b+izz0/kRsjOApm824ePn028f646BP45ssEFqlZ5KOQXBHA0pMCuV\nyiog5d7POx5Yfgg4NKSSCYIZKvrRdV7M32Z67YYbL197iQ/+czuBH4rADCBJEo5FA/d+X9A8n08O\n7GHJ68v7reuZqIbC/vtU2VbhnTw+HzMI45NIySkIj6m6ooopl+IHXBeRFUZTU9MIl8h86ewG7v3e\nRRd2bnYDrov4+gS+8D/TZ5kWLRkLjzM1NWHYyygI5soyH4wJwihov9PGhJ7QAdd5qDxQqTrw8enf\nRJt3Jpfbe+qxabemJ0rDjG/MxsvX62kXd9TIZDI6k7uQKiRkXxpNeXRSBilrFw6438QkBcq/lvDR\nW5/gUGyPzlFHz1wNq/59nUiXKowrIjALwmNSTI0he+JZQsv6B+fiuBLmhS7ut/z0myeY/LtYFnXN\nA8CAgT3H9hL1bgwhkwYO8mPB3B+n8lbVO2y4sA4vvNCj50jIEdx/6IOtre1D91PMjkYxO3oESyoI\n5kcEZkF4TLa2thhekFPxywoieyJNy4udSrB9yRErq75D/lpbWnD/mwsxXb2BRo6cTaUb+ej3nxDy\n57EbmN083FmzZyNn955Hnd+FwV1ixivJuHt4jHbRBMHsicAsCE9gwTfTuOR7gew9l7G9ZYM6UIPX\nFh9SVs3rt23Op9k8e2v9gMexvzrwc9axxMrKirmb5sOm0S6JIFgWEZgF4QnNXJ9iTEb7CDIrGRIP\nGeYjHpkKgvAQIjALY4IkSVw5mUPrjbuEzopg0jTFaBeJmZuTOf7mcVbWr+y3rjvx6eeALioqpKqq\nErlcjkwmQ6/Xk5w8G0/P4e14Vl5eRk1NNSARHBzKpEmjX/eCYMlEYBYs3s2KOq59N5ulOUsI0geR\n71DAvoWfsvRPq3F0dBy1crm6utHzHT3XfpVLQsc0AHTo2Bn3KVO/n/TUzpuXl0t1dRUxMXGsXGnM\nslVcXIRK1cFnn+2mpaWFhIREnJ1dWLgwZcBjXLmSQ1dXF7a2dtjb22FnZ4+dnR2BgUHY2dmh0+k4\nc+YUXV3dTJw4iTlzjE351dVVHDy4H3t7O1JTF2FjIyYGEYQnJQKzYPFy/zWHVy6+bHo9uTue2CMx\nfPjDj1n1xtrRKxgw77VUlAnFfLxzJzYdNmgn6Jj99QW4uLgO6bg5Odk0NzfT3d1Fd3c3XV2ddHd3\nM2lSNAEBAaSn973uixcv0NTUaHr90UcfEBISSlLSZKysnPodPz8/j8bG2/2Wv/TSq7i4uLB//17S\n09fi7GxMoblv3x5UKhV2dnY4ODgik8n53//9Ha+88hre3v2HkAmC8HAiMAsWrfhqEbMvzeq33Aor\nPM66otFovnJ4zkhQJMagSIx55HYajYa7d+9w584d7txp5u7dO8yfnzpg03NpqZKbN+sAkMvlODg4\n0tXVhZWVFVOn9ibjkCSJ5uZmZs6chbW1DTKZzPT/0aOHcHV1pbOzfzKQFStWo9Go0WjU9PSoUat7\nUKvVODk5sX//XjZv3tqnF3p7eztNTY3o9fo+5963bw8vv/xav1zihYUF2Nra4ubmjru7+6j/jgZD\nkiTyzubSdL0R53BnZq6ejVwucjYJQycCs2DRmipvM0M9ZcB1Hm0edHaqsLX1HOFSPbkDBz6jpKS4\n3/KYmLgBA/Py5auQycDBwRF7e3tkMhkHD+5jyZLeiR6u7L/E3bebCCsMRe2gomrWbZJ+moxfiD8A\nW7e+QE5ODrGx0/sd38/
"text/plain": [
"<matplotlib.figure.Figure at 0x10ff5b610>"
]
},
2015-04-12 04:35:40 +08:00
"metadata": {},
2015-04-15 02:24:23 +08:00
"output_type": "display_data"
2015-04-12 02:01:49 +08:00
}
],
2015-04-15 02:24:23 +08:00
"source": [
"clf = SVC(kernel='rbf')\n",
"clf.fit(X, y)\n",
"\n",
"plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='spring')\n",
"plot_svc_decision_function(clf)\n",
"plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1],\n",
" s=200, facecolors='none');"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"SVM additional notes:\n",
"* When using an SVM you need to choose the right values for parameters such as c and gamma. Model validation can help to determine these optimal values by trial and error.\n",
"* SVMs run in O(n^3) performance. LinearSVC is scalable, SVC does not seem to be scalable. For large data sets try transforming the data to a smaller space and use LinearSVC with rbf."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
2015-04-12 02:01:49 +08:00
}
2015-04-15 02:24:23 +08:00
},
"nbformat": 4,
"nbformat_minor": 0
}
